English Performance
English Scores by Model
Average of 2000 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
English Competency & Task Scores
Average of 2000 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Model | EN | Code Generation | Inst. Following | Knowledge | Math | Reasoning | Long Context |
|---|---|---|---|---|---|---|---|
 Gemma 4 31B | 65.33 [-1.72, +1.81] | 73.77 [-2.61, +2.40] | 91.69 [-2.16, +1.95] | 68.47 [-0.70, +0.70] | 37.76 [-3.59, +3.67] | 79.53 [-4.08, +3.75] | 40.74 [-8.37, +8.39] |
 Qwen 3.5 122B MoE Alibaba | 59.64 [-1.64, +1.55] | 60.18 [-2.71, +2.75] | 87.74 [-2.38, +2.49] | 73.88 [-0.71, +0.68] | 13.57 [-2.49, +2.73] | 65.35 [-4.45, +4.29] | 57.12 [-7.75, +7.50] |
 SEA-LION v4.5 (Qwen) 27B AISG | 58.47 [-1.68, +1.67] | 59.89 [-2.67, +2.67] | 85.73 [-2.46, +2.51] | 70.14 [-0.71, +0.73] | 13.89 [-2.60, +2.84] | 65.99 [-4.34, +4.21] | 55.19 [-8.19, +7.82] |
Qwen 3.6 27B Alibaba | 57.44 [-1.56, +1.53] | 59.09 [-2.67, +2.70] | 84.93 [-2.60, +2.48] | 71.30 [-0.69, +0.67] | 11.60 [-2.34, +2.65] | 61.85 [-4.26, +4.31] | 55.87 [-7.50, +7.00] |
 Mistral Medium 3.5 128B Mistral AI | 53.28 [-1.49, +1.54] | 51.03 [-2.71, +2.65] | 81.21 [-2.81, +2.64] | 62.90 [-0.74, +0.66] | 8.26 [-1.89, +2.08] | 75.71 [-3.87, +3.69] | 40.59 [-7.34, +7.16] |
 Llama 4 Scout 109B MoE Meta | 46.63 [-1.55, +1.50] | 31.07 [-2.51, +2.62] | 84.70 [-2.72, +2.69] | 60.67 [-0.76, +0.75] | 11.33 [-2.46, +2.50] | 64.09 [-4.65, +4.29] | 27.92 [-6.80, +6.95] |
 Nemotron 3 Super 120B MoE NVIDIA | 44.83 [-1.42, +1.47] | 51.58 [-2.70, +2.61] | 76.75 [-2.86, +2.84] | 55.52 [-0.65, +0.64] | 11.25 [-2.27, +2.41] | 50.02 [-4.14, +4.12] | 23.84 [-5.84, +5.91] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 41.68 [-1.24, +1.23] | 43.26 [-2.66, +2.57] | 80.59 [-2.91, +2.80] | 43.59 [-0.78, +0.78] | 9.31 [-1.92, +2.04] | 61.85 [-3.63, +3.73] | 11.48 [-4.23, +4.52] |
Gemma 4 (E2B) 5B | 41.10 [-1.26, +1.19] | 45.11 [-2.73, +2.63] | 78.19 [-3.12, +2.95] | 44.57 [-0.78, +0.79] | 6.58 [-1.71, +1.77] | 58.08 [-4.19, +4.30] | 14.07 [-3.95, +3.84] |
 Olmo 3.1 32B AI2 | 41.09 [-1.01, +0.97] | 40.20 [-2.63, +2.71] | 86.75 [-2.67, +2.39] | 51.81 [-0.73, +0.71] | 8.87 [-2.04, +2.21] | 57.95 [-3.97, +4.03] | 0.98 [-0.98, +2.02] |
 GLM 4.7 Flash 30B MoE Z.ai | 34.94 [-1.03, +1.04] | 32.40 [-2.24, +2.34] | 76.56 [-2.65, +2.53] | 46.76 [-0.60, +0.61] | 7.06 [-1.71, +1.76] | 33.33 [-3.12, +3.34] | 13.54 [-3.42, +3.71] |
 MERaLiON 2 10B A*STAR | 23.14 [-0.91, +0.87] | 6.32 [-1.24, +1.33] | 62.37 [-3.20, +3.41] | 39.16 [-0.79, +0.76] | 1.58 [-0.62, +0.72] | 28.54 [-3.78, +3.54] | 0.85 [-0.85, +1.77] |
 Apertus 8B Swiss AI | 18.49 [-0.81, +0.81] | 4.95 [-1.01, +1.05] | 64.93 [-3.13, +3.14] | 25.92 [-0.65, +0.63] | 1.13 [-0.49, +0.54] | 13.05 [-3.06, +2.97] | 0.98 [-0.98, +2.02] |
 Tiny Aya Water 3B CohereLabs | 16.50 [-0.72, +0.74] | 3.46 [-0.77, +0.89] | 63.77 [-2.82, +3.02] | 22.57 [-0.67, +0.68] | 1.00 [-0.39, +0.44] | 7.21 [-2.27, +2.44] | 0.98 [-0.98, +2.02] |