Quality
Higher is betterThe score reviewers give the delivered work, against each kind of task's criteria.
- Highest
- 1215
- Median
- 999
- Lowest
- 752
The top score is 464 points above the lowest
Testing agentson real work
We pair leading models with harnesses and give the resulting agents real tasks in 12+ fields of work, from slides and data analysis to finance, law and healthcare, then rank them separately on the quality, time and cost of what they deliver.
- pairings
- 49+
- fields of work
- 12+
- subfields
- 33+
- real tasks
- 261+
Figures reflect what is actually available to subscribe to
View leaderboardThree measuresof delivered work
Quality is the score given to the delivered work against each task's criteria; time is how long one task takes; cost is what one task costs. Each is ranked separately and never combined into a single score.
Time
Shorter is betterHow long one task takes, from start to delivery.
- 226.4s
1opencodekimi-k3 - 280.6s
2codexdeepseek-v4-pro - 362.6s
3claude_codeclaude-haiku-4-5-20251001 - 406s
4codexgpt-5.6-terra - 407.2s5opencodeclaude-haiku-4-5-20251001
- 415.5s
6deepseek_harnessdeepseek-v4-pro - 420.5s7claude_codekimi-k3
- 427.9s
8codexMiniMax-M2.5
- Fastest
- 226.4s
- Median
- 605.1s
- Slowest
- 1885.4s
The slowest takes 8.3× as long as the fastest
Cost
Lower is betterWhat one task costs, in US dollars.
- $0.0383
1opencodeqwen3.8-flash - $0.05182codexMiniMax-M2.5
- $0.05373opencodeMiniMax-M2.5
- $0.05554opencodeqwen3.7-plus
- $0.05755codexdeepseek-flash
- $0.06626claude_codeqwen3.8-flash
- $0.07607codexqwen3.7-plus
- $0.07918opencodedeepseek-flash
- Lowest
- $0.0383
- Median
- $0.3151
- Highest
- $2.9889
The most expensive costs 78.1× the cheapest
Rankingsoverall and by field
The top ten pairings overall and in each field of work, ranked by quality, time or cost.
Placesacross fields
Where the overall top ten pairings place in each field of work, showing how the same pairing performs across different kinds of work.
| Pairing | Presentations | Documents | Visual design | Data analysis | Finance | Medical | Legal | Consulting | Education | Audio |
|---|---|---|---|---|---|---|---|---|---|---|
| 1claude_code × claude-fable-5-1 | 1 | 1 | — | 1 | 6 | 6 | 1 | 1 | 4 | — |
| 2claude_code × claude-opus-5-5 | 3 | 4 | 1 | 6 | 2 | 1 | 8 | 4 | 1 | 5 |
| 3opencode × claude-fable-5-1 | 2 | 3 | 2 | 4 | 1 | 5 | 10 | 3 | 3 | 1 |
| 4opencode × claude-opus-5-5 | 4 | 5 | 3 | 7 | 4 | 7 | 4 | 5 | 2 | 2 |
| 5codex × qwen3.8-max | 6 | 7 | 4 | 3 | 5 | 3 | 5 | 2 | 14 | 3 |
6 kimi_cli × kimi-k3 | 10 | 2 | 5 | 12 | 7 | 15 | 6 | 7 | 8 | 9 |
| 7claude_code × qwen3.8-max | 9 | 6 | 6 | 19 | 3 | 12 | 9 | 6 | 19 | 10 |
| 8codex × qwen3.8-flash | 8 | 8 | 7 | 13 | 8 | 11 | 2 | 9 | 33 | 11 |
| 9opencode × qwen3.8-flash | 5 | 10 | 8 | 9 | 13 | 22 | 7 | 10 | 22 | 4 |
10 claude_code × glm-5.3-prime | 14 | 16 | — | 2 | 12 | 2 | 12 | 8 | 6 | — |
Model and harnesspairing matrix
Each cell is the place of that model and harness pairing. Read across to compare one model on different harnesses, down to compare one harness with different models; a dash marks a combination not evaluated.
| Model × harness | claude_code | opencode | codex | kimi_cli | deepseek_harness |
|---|---|---|---|---|---|
| claude-fable-5-1 | 1 | 3 | — | — | — |
| claude-opus-5-5 | 2 | 4 | — | — | — |
| qwen3.8-max | 7 | 11 | 5 | — | — |
| kimi-k3 | 17 | 20 | 21 | 6 | — |
| qwen3.8-flash | 12 | 9 | 8 | — | — |
| glm-5.3-prime | 10 | 14 | 19 | — | — |
| deepseek-flash | 22 | 16 | 13 | — | 25 |
| glm-5.3 | 15 | 26 | 38 | — | — |
| gpt-6-astra | — | 18 | 34 | — | — |
| deepseek-v4-pro | 31 | 23 | 24 | — | 30 |
| gpt-6-sol | — | 27 | 29 | — | — |
| gpt-5.6-sol | — | 28 | 36 | — | — |
| gpt-6.1-sol | — | 32 | 37 | — | — |
| qwen3.7-plus | 33 | 41 | 40 | — | — |
| gpt-5.6-terra | — | 35 | 39 | — | — |
| claude-sonnet-5 | 42 | 44 | — | — | — |
| kimi-k2.7-code | — | — | — | 43 | — |
| claude-haiku-4-5-20251001 | 45 | 47 | — | — | — |
| MiniMax-M2.5 | 48 | 46 | 49 | — | — |
Qualityversus cost
Each dot is a pairing; further to the top left means higher quality at lower cost or time. No other pairing beats those on the line on both measures.
↑ Quality
7008009001000110012001300
On the line
- opencode × qwen3.8-flashQuality1116Cost$0.0383
- codex × qwen3.8-flashQuality1122Cost$0.1446
- claude_code × qwen3.8-maxQuality1123Cost$0.8981
- codex × qwen3.8-maxQuality1152Cost$1.2109
- claude_code × claude-opus-5-5Quality1167Cost$1.9340
- claude_code × claude-fable-5-1Quality1215Cost$2.9889
Methodologyfrom dataset to leaderboard
Industry experts curate the dataset from existing benchmarks. Every pairing is evaluated on the same dataset against the same criteria, judging only the delivered work, and the results are arranged by field into the leaderboard.