Cap Evolve

cap-evolve

watch capability evolve

cap-evolve improves an AI agent's prompts, tools, and skills by learning from failed evaluation traces.

You bring the agent and the eval you already have. cap-evolve runs the loop — evaluate → diagnose the failures → propose an edit → keep it only if it beats a held-out split by a significant margin → commit — and reports one honest number. It optimizes what your agent reads, not its weights.

▶︎ Watch the 85-second demo · plays inline on the site

Quickstart · Documentation · Examples · Results · Contributing

Why cap-evolve

• Optimize more than prompts. System prompts, executable tool code, MCP tool surfaces, and whole skill packages — pick one or several and optimize them jointly.
• Learn from real agent failures. Every iteration reads full trajectories and per-task causal feedback (which task ids a prior edit broke and fixed), so edits are large and don't regress the wins.
• Keep evaluation honest. Acceptance is a val-only significance gate (Δ > k·SE); the test split is sealed and scored exactly once. Both live in the core, not in editable docs.
• Inspect every change. Each candidate is a git commit; the dashboard shows costs, timing, diffs, lineage, and a tasks × iterations pass/fail heatmap.

Try it in two minutes — no API key required

toy_calc is a deterministic stand-in agent that only answers correctly when its system prompt contains a [CALC] marker. The mock optimizer adds it, so the score provably rises — no model is called.

git clone https://github.com/skillberry-ai/cap-evolve.git
cd cap-evolve

python3 -m venv .venv && source .venv/bin/activate
pip install ./core                 # package: cap-evolve-core · CLI: cap-evolve · zero runtime deps

bash examples/toy_calc/run.sh

Expected — the seed prompt scores 0.0 on val; the optimized prompt is gate-accepted and scores 1.0 on the sealed test split:

baseline_val 0.0  ->  test_reward 1.0   (gate-accepted, test sealed) + dashboard.html

Open the printed dashboard.html in any browser. Full walkthrough: Getting started.

The CLI

Start with no arguments — cap-evolve prints a branded home screen with the golden path and every command grouped by what it's for.

cap-evolve                         # home: the 3-step path + all commands
cap-evolve init                    # scaffold a project and write capevolve.yaml
cap-evolve doctor                  # readiness check: what's missing + the command that fixes it
cap-evolve algorithms              # the five algorithms and the exact spec lines to pick one
cap-evolve help <command>          # full help with copy-paste examples

doctor is the one to run before spending anything. Every failing row names the fix:

See what actually changed

Every candidate is a snapshot, so you can read the edit that moved the number — unified below 120 columns, side-by-side above:

cap-evolve diff --best             # seed → the winning candidate
cap-evolve diff cand_0003          # against its parent
cap-evolve diff cand_0003 --stat   # just the per-file +/- counts

Watch a run, live