Five renames to trip on.
It tripped on none.

@tanstack/react-query 5.101.4 · claude-opus-5

AI coding assistants write library code from memory. When a library ships a big release that renames or removes things, the assistant keeps writing the old version. It reads fine & it doesn't build.

SDKProof measures how often that happens. It gives a model real coding jobs for one library, then compiles every answer against the real installed package with tsc, the TypeScript compiler. A task passes only if it compiles — no AI judges another AI.

This page is TanStack Query, the data-fetching library for React. Thirteen ordinary jobs — fetch a record, cache it, page through a list — one shot each, no docs, no retries.

All thirteen compiled. v5 renamed a lot of things a model could still be writing from its v4 memory. It wrote none of them.

This measures the model, not TanStack Query.

100~~/100~~ 13 of 13 compiled

13 compiled 0 did not compile 0 refused

Model: claude-opus-5 Tasks: 13 Package: @tanstack/react-query 5.101.4 Run: 26 July 2026 Pass: it compiles

What it could have got wrong

Five things v5 changed, side by side

Every line on the left was correct v4 and is an error against the installed v5. A model still writing v4 from memory fails all five.

The model wrote the right-hand column on its own. The prompts name the job — "fetch a to-do", "keep the last page on screen" — never the option names.

What a v4 memory writes fails against 5.101.4

// two positional arguments
useQuery(['todo', id], fetchTodo);

// how long to keep unused cache
{ cacheTime: 5 * 60_000 }

// hold the last page while loading
{ keepPreviousData: true }

// the first status
q.status === 'loading'

// infinite list
useInfiniteQuery({ queryKey, queryFn,
  getNextPageParam });

Every line here was correct in v4 and is an error against 5.101.4.

What the model actually wrote @tanstack/react-query 5.101.4

// one object
useQuery({ queryKey: ['todo', id],
  queryFn: fetchTodo });

// renamed
{ gcTime: 5 * 60_000 }

// now a helper you import
{ placeholderData: keepPreviousData }

// renamed
q.status === 'pending'

// a starting page is required now
useInfiniteQuery({ queryKey, queryFn,
  initialPageParam: 0, getNextPageParam });

tsc --noEmit — 0 errors

placeholderData is the interesting one. v4's keepPreviousData: true became a function you import and pass in. That's not a rename you can guess — the model had to have read v5 code. It imported the helper without being asked to.

What a 100 means here

Not luck. Age.

v5 shipped in 2023. The model has seen years of v5 code, years of migration guides, and years of people answering questions about it. By now the v4 shapes are the unusual ones.

Compare the other end of the board. TanStack Table v9 is fresh, and the same model wrote the v8 API on all twelve tasks — 0 of 12 compiled. Same maintainers, same model, same day. The only thing that differs is how long the current version has been out.

So a 100 is a statement about time, not quality. It also has a shelf life: the day this library ships a release that renames something, this number drops and stays down until models retrain.

All thirteen

What the model wrote

Marked ones are the version-specific probes — tasks written to catch a v4 answer.

useQuery — object form useMutation select enabled multi-part queryKey staleTime invalidateQueries dependent queries error handling gcTime placeholderData: keepPreviousData status: 'pending' initialPageParam

How this was measured

The compiler has the last word

Thirteen realistic TanStack Query tasks, written by claude-opus-5, each dropped into a small project with @tanstack/react-query 5.101.4, react and @types/react actually installed, then run through tsc --noEmit. A task passes only if it compiles.

One model: Everything here is claude-opus-5. Another model will score differently.
One shot, no docs: The model gets the task and nothing else. No release notes, no editor rules, no retries.
Small numbers: Thirteen tasks. A clean sweep rules out a common failure, not a rare one.
Compiles is not correct: tsc, the TypeScript compiler, checks that the API exists and the types line up. It never runs the code, so nothing here says the fetching logic behaves properly.
Refusals: A refusal is neither a pass nor a fail — the model writes no code, so there is nothing to compile, and refused tasks drop out of the denominator. This run had none. Stripe had five, which is why its 100 is out of 10 rather than 15.
It measures the model: 100 says claude-opus-5 knows current TanStack Query. It says nothing about whether the library is well designed.