Fleet resolution49.2%▲ 6.1 vs v1.0
Tasks covered8611 domains
Median wall-clock8m 41s▼ 1m 12s
Skill load rate92.4%▲ 3.8
Reward-hack flags73 unresolved
Capability matrix
resolution rate by domain × model, full scaffolding
| Domain | Opus 4.8 | GPT-5.5 | Gemini 3.1 | GLM 5.1 | Kimi K2.6 | Domain mean |
|---|
Cell shading is a single-hue ramp on resolution rate; the numbers, not the colour, are the source of truth.
- 00:00Loaded skill structural-bio/pdb-toolsok
- 00:04Read task.md, resolved 3 of 3 constraintsok
- 00:31Fetched 2 chains, detected non-standard residuesok
- 01:12Ran refold, RMSD 1.84 Å above thresholdretry
- 02:47Adjusted seed, re-ran; RMSD 0.91 Åok
- 03:20Verifier: 4/4 assertions passedpass
- 1,204spreadsheet/xlsx-ops98%
- 1,092legal/redline96%
- 948structural-bio/pdb-tools94%
- 771gis/parcel-join89%
- 402cad/fixture-fit61%
- 188hardware/uart-bringup44%