# // code-intelligence benchmarks## // methodology## // per-model breakdown## // per-problem comparison
## // key takeaways
Correctness
Run against 4-5 test cases per problem. Pass/fail scoring with edge cases.
Complexity
AST analysis of cyclomatic complexity, nesting depth, loop count, and line count.
Readability
Heuristic scoring of naming, comments, blank lines, and line length distribution.
Style
Pythonic idiom detection: type hints, list comprehensions, enumerate/zip, docstrings.
How scores work: Each model is given the same 5 coding problems at temperature 0.2. Solutions are evaluated programmatically — no human judgment involved. Composite = correctness × 0.4 + complexity × 0.2 + readability × 0.2 + style × 0.2. New models added weekly.
## // overall standings#1
Groq GPT-OSS 20B
86.9
#2
Groq GPT-OSS 120B
86.4
#3
Xiaomi MiMo 2.5
86.2
#4
GPT-OSS 20B
85.8
#5
DeepSeek V4
85.6
#6
Qwen 3.5 Omni Plus (Nexum)
85.6
#7
DeepSeek V4 Flash (direct)
85.5
#8
OpenRouter GPT-OSS 20B
85.5
#9
Qwen 3.5 Plus Thinking (Nexum)
85.3
#10
Qwen 3.6 Max Preview (Nexum)
85.3
#11
Qwen 3.6 Plus Thinking (Nexum)
85.2
#12
Xiaomi MiMo 2.5 Pro
85.2
#13
Nemotron 3 Ultra 550B
85.0
#14
Qwen 3.6 Plus (Nexum)
85.0
#15
Qwen 3.7 Plus Thinking (Nexum)
84.8
#16
Qwen 3.7 Max (Nexum)
84.7
#17
z.ai GLM-5.2
84.7
#18
Hy3 (Nexum)
84.6
#19
Devstral Latest (Mistral)
81.9
#20
Codestral Latest (Mistral)
80.8
#21
Groq Llama 3.3 70B
80.4
#22
Groq Llama 4 Scout 17B
79.4
#23
Gemma 4 12B Coder (local)
77.6
#24
Groq Llama 3.1 8B
77.4
#25
Mistral Large
77.3
#26
Cerebras GPT-OSS 120B
75.9
#27
GPT 5.5 Instant 14K
40.3
Groq GPT-OSS 20B
96.0
Correctness
71.8
Complexity
83.4
Readability
87.2
Style
Two Sum
84.2
FizzBuzz
93.6
Merge Intervals
90.7
LRU Cache
93.1
JSON Parser
72.8
Groq GPT-OSS 120B
96.0
Correctness
65.0
Complexity
84.3
Readability
90.6
Style
Two Sum
84.9
FizzBuzz
88.6
Merge Intervals
91.6
LRU Cache
93.1
JSON Parser
73.6
Xiaomi MiMo 2.5
96.0
Correctness
67.8
Complexity
85.7
Readability
85.6
Style
Two Sum
84.3
FizzBuzz
94.0
Merge Intervals
91.2
LRU Cache
88.1
JSON Parser
73.4
GPT-OSS 20B
96.0
Correctness
63.7
Complexity
85.5
Readability
87.6
Style
Two Sum
84.9
FizzBuzz
87.7
Merge Intervals
91.4
LRU Cache
91.8
JSON Parser
73.0
DeepSeek V4
96.0
Correctness
64.3
Complexity
85.0
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
91.9
JSON Parser
73.4
Qwen 3.5 Omni Plus (Nexum)
96.0
Correctness
63.7
Complexity
85.8
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
91.2
LRU Cache
91.3
JSON Parser
73.7
DeepSeek V4 Flash (direct)
96.0
Correctness
63.9
Complexity
83.8
Readability
87.6
Style
Two Sum
84.3
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
92.2
JSON Parser
72.4
OpenRouter GPT-OSS 20B
96.0
Correctness
63.8
Complexity
84.1
Readability
87.6
Style
Two Sum
84.9
FizzBuzz
87.7
Merge Intervals
91.1
LRU Cache
91.8
JSON Parser
72.0
Qwen 3.5 Plus Thinking (Nexum)
96.0
Correctness
61.7
Complexity
86.3
Readability
86.6
Style
Two Sum
84.3
FizzBuzz
87.7
Merge Intervals
91.4
LRU Cache
89.2
JSON Parser
74.0
Qwen 3.6 Max Preview (Nexum)
96.0
Correctness
61.5
Complexity
86.3
Readability
86.6
Style
Two Sum
84.3
FizzBuzz
87.7
Merge Intervals
91.2
LRU Cache
89.1
JSON Parser
74.0
Qwen 3.6 Plus Thinking (Nexum)
96.0
Correctness
61.5
Complexity
85.8
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
91.9
LRU Cache
89.3
JSON Parser
72.8
Xiaomi MiMo 2.5 Pro
96.0
Correctness
61.1
Complexity
85.1
Readability
87.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
90.8
LRU Cache
89.5
JSON Parser
73.6
Nemotron 3 Ultra 550B
96.0
Correctness
61.7
Complexity
84.8
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
89.1
JSON Parser
73.4
Qwen 3.6 Plus (Nexum)
96.0
Correctness
62.0
Complexity
84.2
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
89.4
JSON Parser
72.8
Qwen 3.7 Plus Thinking (Nexum)
96.0
Correctness
61.9
Complexity
83.6
Readability
86.6
Style
Two Sum
84.3
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
89.2
JSON Parser
72.2
Qwen 3.7 Max (Nexum)
96.0
Correctness
61.8
Complexity
83.1
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
89.2
JSON Parser
71.7
z.ai GLM-5.2
96.0
Correctness
65.5
Complexity
80.0
Readability
86.0
Style
Two Sum
84.3
FizzBuzz
87.7
Merge Intervals
90.7
LRU Cache
92.6
JSON Parser
68.2
Hy3 (Nexum)
96.0
Correctness
64.6
Complexity
79.7
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
90.8
LRU Cache
90.8
JSON Parser
69.5
Devstral Latest (Mistral)
92.0
Correctness
64.4
Complexity
74.8
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
91.2
LRU Cache
91.5
JSON Parser
55.1
Codestral Latest (Mistral)
84.0
Correctness
62.9
Complexity
86.5
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
91.2
LRU Cache
90.7
JSON Parser
50.2
Groq Llama 3.3 70B
80.0
Correctness
72.5
Complexity
81.5
Readability
87.8
Style
Two Sum
84.2
FizzBuzz
93.6
Merge Intervals
91.2
LRU Cache
91.3
JSON Parser
41.5
Groq Llama 4 Scout 17B
76.0
Correctness
72.2
Complexity
85.9
Readability
87.2
Style
Two Sum
84.2
FizzBuzz
93.6
Merge Intervals
91.2
LRU Cache
91.9
JSON Parser
36.3
Gemma 4 12B Coder (local)
76.0
Correctness
65.2
Complexity
85.0
Readability
85.6
Style
Two Sum
84.3
FizzBuzz
87.7
Merge Intervals
91.4
LRU Cache
90.8
JSON Parser
33.6
Groq Llama 3.1 8B
76.0
Correctness
70.9
Complexity
77.1
Readability
87.2
Style
Two Sum
84.3
FizzBuzz
93.6
Merge Intervals
91.4
LRU Cache
91.8
JSON Parser
26.1
Mistral Large
76.0
Correctness
61.5
Complexity
86.2
Readability
86.6
Style
Two Sum
84.2
FizzBuzz
87.7
Merge Intervals
91.2
LRU Cache
89.2
JSON Parser
34.0
Cerebras GPT-OSS 120B
76.0
Correctness
57.7
Complexity
82.0
Readability
87.6
Style
Two Sum
84.9
FizzBuzz
87.7
Merge Intervals
91.1
LRU Cache
85.4
JSON Parser
30.2
GPT 5.5 Instant 14K
0.0
Correctness
30.0
Complexity
86.0
Readability
85.6
Style
Two Sum
40.6
FizzBuzz
40.0
Merge Intervals
40.8
LRU Cache
40.6
JSON Parser
39.6
Two Sum
| Model | ✓ Correctness | ◉ Complexity | ◎ Readability | ✦ Style | Composite |
|---|---|---|---|---|---|
| Groq GPT-OSS 20B | 80.0 | 88.0 | 85.0 | 88.0 | |
| Groq GPT-OSS 120B | 80.0 | 86.5 | 85.0 | 93.0 | |
| Xiaomi MiMo 2.5 | 80.0 | 88.5 | 85.0 | 88.0 | |
| GPT-OSS 20B | 80.0 | 86.5 | 85.0 | 93.0 | |
| DeepSeek V4 | 80.0 | 88.0 | 85.0 | 88.0 | |
| Qwen 3.5 Omni Plus (Nexum) | 80.0 | 88.0 | 85.0 | 88.0 | |
| DeepSeek V4 Flash (direct) | 80.0 | 88.5 | 85.0 | 88.0 | |
| OpenRouter GPT-OSS 20B | 80.0 | 86.5 | 85.0 | 93.0 | |
| Qwen 3.5 Plus Thinking (Nexum) | 80.0 | 88.5 | 85.0 | 88.0 | |
| Qwen 3.6 Max Preview (Nexum) | 80.0 | 88.5 | 85.0 | 88.0 | |
| Qwen 3.6 Plus Thinking (Nexum) | 80.0 | 88.0 | 85.0 | 88.0 | |
| Xiaomi MiMo 2.5 Pro | 80.0 | 88.0 | 85.0 | 88.0 | |
| Nemotron 3 Ultra 550B | 80.0 | 88.0 | 85.0 | 88.0 | |
| Qwen 3.6 Plus (Nexum) | 80.0 | 88.0 | 85.0 | 88.0 | |
| Qwen 3.7 Plus Thinking (Nexum) | 80.0 | 88.5 | 85.0 | 88.0 | |
| Qwen 3.7 Max (Nexum) | 80.0 | 88.0 | 85.0 | 88.0 | |
| z.ai GLM-5.2 | 80.0 | 88.5 | 85.0 | 88.0 | |
| Hy3 (Nexum) | 80.0 | 88.0 | 85.0 | 88.0 | |
| Devstral Latest (Mistral) | 80.0 | 88.0 | 85.0 | 88.0 | |
| Codestral Latest (Mistral) | 80.0 | 88.0 | 85.0 | 88.0 | |
| Groq Llama 3.3 70B | 80.0 | 88.0 | 85.0 | 88.0 | |
| Groq Llama 4 Scout 17B | 80.0 | 88.0 | 85.0 | 88.0 | |
| Gemma 4 12B Coder (local) | 80.0 | 88.5 | 85.0 | 88.0 | |
| Groq Llama 3.1 8B | 80.0 | 88.5 | 85.0 | 88.0 | |
| Mistral Large | 80.0 | 88.0 | 85.0 | 88.0 | |
| Cerebras GPT-OSS 120B | 80.0 | 86.5 | 85.0 | 93.0 | |
| GPT 5.5 Instant 14K | 0 | 30 | 85.0 | 88.0 |
FizzBuzz
| Model | ✓ Correctness | ◉ Complexity | ◎ Readability | ✦ Style | Composite |
|---|---|---|---|---|---|
| Groq GPT-OSS 20B | 100.0 | 100 | 80.0 | 88.0 | |
| Groq GPT-OSS 120B | 100.0 | 68.0 | 85.0 | 90.0 | |
| Xiaomi MiMo 2.5 | 100.0 | 100 | 85.0 | 85.0 | |
| GPT-OSS 20B | 100.0 | 68.5 | 85.0 | 85.0 | |
| DeepSeek V4 | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.5 Omni Plus (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| DeepSeek V4 Flash (direct) | 100.0 | 68.5 | 85.0 | 85.0 | |
| OpenRouter GPT-OSS 20B | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.5 Plus Thinking (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.6 Max Preview (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.6 Plus Thinking (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Xiaomi MiMo 2.5 Pro | 100.0 | 68.5 | 85.0 | 85.0 | |
| Nemotron 3 Ultra 550B | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.6 Plus (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.7 Plus Thinking (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Qwen 3.7 Max (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| z.ai GLM-5.2 | 100.0 | 68.5 | 85.0 | 85.0 | |
| Hy3 (Nexum) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Devstral Latest (Mistral) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Codestral Latest (Mistral) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Groq Llama 3.3 70B | 100.0 | 100 | 80.0 | 88.0 | |
| Groq Llama 4 Scout 17B | 100.0 | 100 | 80.0 | 88.0 | |
| Gemma 4 12B Coder (local) | 100.0 | 68.5 | 85.0 | 85.0 | |
| Groq Llama 3.1 8B | 100.0 | 100 | 80.0 | 88.0 | |
| Mistral Large | 100.0 | 68.5 | 85.0 | 85.0 | |
| Cerebras GPT-OSS 120B | 100.0 | 68.5 | 85.0 | 85.0 | |
| GPT 5.5 Instant 14K | 0 | 30 | 85.0 | 85.0 |
Merge Intervals
| Model | ✓ Correctness | ◉ Complexity | ◎ Readability | ✦ Style | Composite |
|---|---|---|---|---|---|
| Groq GPT-OSS 20B | 100.0 | 83.5 | 85.0 | 85.0 | |
| Groq GPT-OSS 120B | 100.0 | 83.0 | 85.0 | 90.0 | |
| Xiaomi MiMo 2.5 | 100.0 | 82.2 | 89.0 | 85.0 | |
| GPT-OSS 20B | 100.0 | 82.7 | 89.3 | 85.0 | |
| DeepSeek V4 | 100.0 | 83.5 | 85.0 | 85.0 | |
| Qwen 3.5 Omni Plus (Nexum) | 100.0 | 82.2 | 89.0 | 85.0 | |
| DeepSeek V4 Flash (direct) | 100.0 | 83.5 | 85.0 | 85.0 | |
| OpenRouter GPT-OSS 20B | 100.0 | 83.0 | 87.3 | 85.0 | |
| Qwen 3.5 Plus Thinking (Nexum) | 100.0 | 82.7 | 89.3 | 85.0 | |
| Qwen 3.6 Max Preview (Nexum) | 100.0 | 82.2 | 89.0 | 85.0 | |
| Qwen 3.6 Plus Thinking (Nexum) | 100.0 | 81.8 | 92.5 | 85.0 | |
| Xiaomi MiMo 2.5 Pro | 100.0 | 83.8 | 85.0 | 85.0 | |
| Nemotron 3 Ultra 550B | 100.0 | 83.5 | 85.0 | 85.0 | |
| Qwen 3.6 Plus (Nexum) | 100.0 | 83.5 | 85.0 | 85.0 | |
| Qwen 3.7 Plus Thinking (Nexum) | 100.0 | 83.5 | 85.0 | 85.0 | |
| Qwen 3.7 Max (Nexum) | 100.0 | 83.5 | 85.0 | 85.0 | |
| z.ai GLM-5.2 | 100.0 | 83.5 | 85.0 | 85.0 | |
| Hy3 (Nexum) | 100.0 | 83.8 | 85.0 | 85.0 | |
| Devstral Latest (Mistral) | 100.0 | 82.2 | 89.0 | 85.0 | |
| Codestral Latest (Mistral) | 100.0 | 82.2 | 89.0 | 85.0 | |
| Groq Llama 3.3 70B | 100.0 | 82.2 | 89.0 | 85.0 | |
| Groq Llama 4 Scout 17B | 100.0 | 82.2 | 89.0 | 85.0 | |
| Gemma 4 12B Coder (local) | 100.0 | 81.8 | 90.0 | 85.0 | |
| Groq Llama 3.1 8B | 100.0 | 82.7 | 89.3 | 85.0 | |
| Mistral Large | 100.0 | 82.2 | 89.0 | 85.0 | |
| Cerebras GPT-OSS 120B | 100.0 | 83.0 | 87.3 | 85.0 | |
| GPT 5.5 Instant 14K | 0 | 30 | 89.0 | 85.0 |
LRU Cache
| Model | ✓ Correctness | ◉ Complexity | ◎ Readability | ✦ Style | Composite |
|---|---|---|---|---|---|
| Groq GPT-OSS 20B | 100.0 | 87.4 | 88.2 | 90.0 | |
| Groq GPT-OSS 120B | 100.0 | 87.4 | 88.2 | 90.0 | |
| Xiaomi MiMo 2.5 | 100.0 | 68.2 | 87.3 | 85.0 | |
| GPT-OSS 20B | 100.0 | 81.0 | 88.0 | 90.0 | |
| DeepSeek V4 | 100.0 | 81.4 | 88.2 | 90.0 | |
| Qwen 3.5 Omni Plus (Nexum) | 100.0 | 79.8 | 86.7 | 90.0 | |
| DeepSeek V4 Flash (direct) | 100.0 | 79.0 | 92.2 | 90.0 | |
| OpenRouter GPT-OSS 20B | 100.0 | 81.0 | 88.0 | 90.0 | |
| Qwen 3.5 Plus Thinking (Nexum) | 100.0 | 69.0 | 87.0 | 90.0 | |
| Qwen 3.6 Max Preview (Nexum) | 100.0 | 68.2 | 87.3 | 90.0 | |
| Qwen 3.6 Plus Thinking (Nexum) | 100.0 | 69.0 | 87.4 | 90.0 | |
| Xiaomi MiMo 2.5 Pro | 100.0 | 65.0 | 87.3 | 95.0 | |
| Nemotron 3 Ultra 550B | 100.0 | 68.6 | 87.0 | 90.0 | |
| Qwen 3.6 Plus (Nexum) | 100.0 | 69.8 | 87.1 | 90.0 | |
| Qwen 3.7 Plus Thinking (Nexum) | 100.0 | 69.0 | 87.0 | 90.0 | |
| Qwen 3.7 Max (Nexum) | 100.0 | 69.0 | 87.0 | 90.0 | |
| z.ai GLM-5.2 | 100.0 | 87.0 | 89.0 | 87.0 | |
| Hy3 (Nexum) | 100.0 | 76.6 | 87.2 | 90.0 | |
| Devstral Latest (Mistral) | 100.0 | 80.6 | 86.9 | 90.0 | |
| Codestral Latest (Mistral) | 100.0 | 75.8 | 87.5 | 90.0 | |
| Groq Llama 3.3 70B | 100.0 | 79.8 | 86.7 | 90.0 | |
| Groq Llama 4 Scout 17B | 100.0 | 80.6 | 88.8 | 90.0 | |
| Gemma 4 12B Coder (local) | 100.0 | 81.0 | 88.0 | 85.0 | |
| Groq Llama 3.1 8B | 100.0 | 81.0 | 88.0 | 90.0 | |
| Mistral Large | 100.0 | 69.0 | 87.0 | 90.0 | |
| Cerebras GPT-OSS 120B | 100.0 | 50.4 | 86.7 | 90.0 | |
| GPT 5.5 Instant 14K | 0 | 30 | 87.8 | 85.0 |
JSON Parser
| Model | ✓ Correctness | ◉ Complexity | ◎ Readability | ✦ Style | Composite |
|---|---|---|---|---|---|
| Groq GPT-OSS 20B | 100.0 | 0 | 79.0 | 85.0 | |
| Groq GPT-OSS 120B | 100.0 | 0 | 78.1 | 90.0 | |
| Xiaomi MiMo 2.5 | 100.0 | 0 | 82.0 | 85.0 | |
| GPT-OSS 20B | 100.0 | 0 | 80.0 | 85.0 | |
| DeepSeek V4 | 100.0 | 0 | 82.0 | 85.0 | |
| Qwen 3.5 Omni Plus (Nexum) | 100.0 | 0 | 83.3 | 85.0 | |
| DeepSeek V4 Flash (direct) | 100.0 | 0 | 72.0 | 90.0 | |
| OpenRouter GPT-OSS 20B | 100.0 | 0 | 75.1 | 85.0 | |
| Qwen 3.5 Plus Thinking (Nexum) | 100.0 | 0 | 85.0 | 85.0 | |
| Qwen 3.6 Max Preview (Nexum) | 100.0 | 0 | 85.0 | 85.0 | |
| Qwen 3.6 Plus Thinking (Nexum) | 100.0 | 0 | 79.0 | 85.0 | |
| Xiaomi MiMo 2.5 Pro | 100.0 | 0 | 83.0 | 85.0 | |
| Nemotron 3 Ultra 550B | 100.0 | 0 | 82.0 | 85.0 | |
| Qwen 3.6 Plus (Nexum) | 100.0 | 0 | 79.0 | 85.0 | |
| Qwen 3.7 Plus Thinking (Nexum) | 100.0 | 0 | 76.0 | 85.0 | |
| Qwen 3.7 Max (Nexum) | 100.0 | 0 | 73.5 | 85.0 | |
| z.ai GLM-5.2 | 100.0 | 0 | 56.1 | 85.0 | |
| Hy3 (Nexum) | 100.0 | 6.2 | 56.2 | 85.0 | |
| Devstral Latest (Mistral) | 80.0 | 2.5 | 28.0 | 85.0 | |
| Codestral Latest (Mistral) | 40.0 | 0 | 86.2 | 85.0 | |
| Groq Llama 3.3 70B | 20.0 | 12.4 | 67.0 | 88.0 | |
| Groq Llama 4 Scout 17B | 0.0 | 10.0 | 86.6 | 85.0 | |
| Gemma 4 12B Coder (local) | 0.0 | 6.0 | 77.2 | 85.0 | |
| Groq Llama 3.1 8B | 0.0 | 2.5 | 43.0 | 85.0 | |
| Mistral Large | 0 | 0 | 85.0 | 85.0 | |
| Cerebras GPT-OSS 120B | 0 | 0 | 66.0 | 85.0 | |
| GPT 5.5 Instant 14K | 0 | 30 | 83.2 | 85.0 |
All models handle easy problems
Two Sum and FizzBuzz are essentially solved — all models score 80+ on correctness. The differentiation happens on harder tasks.
JSON parser is the big differentiator
Writing a JSON parser from scratch pushes models to demonstrate real coding ability — this is where the gap widens significantly.
DeepSeek V4 shines on LRU Cache
Uses OrderedDict.move_to_end() for elegant O(1) implementation — scores highest on this problem.
Complexity scores tank on long solutions
JSON parsers are inherently long (50-200 lines), which drags complexity scores. This is expected — we're measuring conciseness, not quality.