modelharness

Make every model cheaper or better. Measured on four Claude models — none got worse.

What it actually is: a zero-config Claude Code plugin. On every session start it injects a ≈910-token behavioral core — six working practices distilled from how Fable 5 was trained to operate — plus three on-demand skills and a fresh-context verifier agent. No commands to learn; the model just starts working differently:

✅ Grounded progress — only claims backed by a tool result; "tests fail" said plainly	⚡ Act, don't overplan — enough information means act, not narrate options
🎯 Autonomy calibration — decides minor things itself, asks only on scope or destructive actions	🔍 Self-verification loops — a checkable definition of done, real checks on a cadence, a fresh-context verifier before "done"
🔀 Delegation triggers — explicit rules for when to fan work out to subagents	📝 Cross-session memory — writes lessons and plans to files, so the next session can pick up the work

What the 17 tasks test

🐛 Bug hunts 4 tasks · 96 runsFind and fix planted defects: TTL cache, CSV quoting, rate limiter, date rollover	✨ Features from spec 4 tasks · 96 runsBuild to a written spec: retry backoff, config merging, cursor pagination, slugify	♻️ Refactors 2 tasks · 48 runsRestructure code with zero behavior change, verified structurally
🧠 Long-horizon builds 2 tasks · 48 runsMulti-stage pipelines where later steps depend on earlier decisions	🧩 Spec-dense traps 3 tasks · 72 runs18+ interacting rules (discount engine, mini-interpreter) that punish shallow reading	🔁 Session handoffs 2 tasks · 48 runsA fresh session must finish another session's work — memory is the only bridge

17 tasks × 3 attempts × 8 configurations = 408 runs. Grading is hidden and binary: test suites the agent never sees decide pass/fail. No LLM judge. Every task ships with a reference solution proving it solvable.

The results

The numbers, exactly

Same 17 tasks, 3 runs per configuration. Higher pass rate and lower cost/time are better. 🟢 = better with modelharness, 🔴 = worse (explained in the last row).