When a library ships a breaking major, AI assistants keep writing the old API — it looks right, but it doesn't compile against the new version. SDKProof measures exactly how much: it has a model solve real tasks, then type-checks the code against your real installed package.
Each score is claude-opus-4-8 across ~10–15 realistic tasks, type-checked against the current package.
The score isn't about how smart the model is — it's about how recently your SDK changed. Zod 4 and the Vercel AI SDK shipped their breaking changes in 2025, so today's models have largely absorbed them (90 each). Prisma 7 is newer, and the same model still writes its old v6 setup (80).
Right after a major the gap is wide; as models retrain it narrows — then re-opens on your next major and shifts on every model release. It's a moving number, which is why it's worth monitoring, not auditing once.
A model solves ~10–15 realistic tasks for the library. Prompts name the functions, never the option names.
Each solution is dropped into a project with the real installed package and run through tsc --noEmit.
Pass = compiles clean. Failures are the compiler's own diagnostics — hallucinated members, wrong args, removed options.
@remix-run/* imports removed
Clerk v6 — auth() now async
TanStack Query v5 — cacheTime → gcTime
Point SDKProof at any TypeScript package and see what agents get wrong. It's open source — run it in a few minutes.
Get it on GitHub →