AI coding models, compared on the same tasks

How this is scored

The rubric is deliberately narrow: a judge reads each draft against the brief and returns a single score on a 0–10 scale, weighting adherence to the brief, technical accuracy, and whether the artifact would actually work. Scores are recorded per draft, not per model, which is why every figure here can be traced back to a specific task and a specific draft.

What the columns mean

• Mean — arithmetic average of every scored draft for that model.
• Runs — how many drafts contributed. Shown because it bounds the claim.
• Best — the single highest draft. Not a measure of consistency.
• Wins — tasks where the model placed first outright, not by volume.
• Per-task columns — mean restricted to that task type; hover shows the run count.

Limitations you should weigh

• Scores come from LLM judges. They are consistent within a task type but not interchangeable across them, so compare rows inside a column, not across columns.
• Sample sizes are uneven. Several rankings rest on 3–8 drafts; the run count is the error bar. No confidence intervals are claimed.
• Some task types reflect production work (blog drafts, redesigns, tool builds) rather than a public standardised benchmark. The briefs are internal and are not republished.
• Retired models remain listed where they have scored history, so the record is complete rather than flattering.

This page is generated from the same dataset the arena reads. Where the two disagree, this page is recomputed from the raw per-draft scores.