SEA Performance
Jul 10, 2026
Overall SEA Average
Average of 30 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Language Performance by Model
Average of 30 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Model | SEA | MY | TL | ID | MS | TA | TH | VI |
|---|---|---|---|---|---|---|---|---|
 Gemma 4 31B | 75.16 ± 0.03 | 69.04 ± 0.09 | 77.18 ± 0.07 | 78.90 ± 0.07 | 76.65 ± 0.05 | 76.27 ± 0.08 | 71.04 ± 0.07 | 77.08 ± 0.08 |
 SEA-LION v4.5 (Qwen) 27B AISG | 72.69 ± 0.05 | 63.38 ± 0.21 | 73.38 ± 0.12 | 75.57 ± 0.13 | 73.89 ± 0.13 | 75.62 ± 0.11 | 72.34 ± 0.11 | 74.62 ± 0.16 |
 Qwen 3.5 122B MoE Alibaba | 70.92 ± 0.06 | 61.32 ± 0.21 | 72.44 ± 0.13 | 75.54 ± 0.13 | 74.62 ± 0.13 | 70.69 ± 0.20 | 67.54 ± 0.13 | 74.27 ± 0.21 |
Qwen 3.6 27B Alibaba | 67.25 ± 0.07 | 48.01 ± 0.23 | 67.35 ± 0.22 | 74.15 ± 0.18 | 72.68 ± 0.19 | 67.61 ± 0.27 | 68.82 ± 0.23 | 72.09 ± 0.22 |
 Llama 4 Scout 109B MoE Meta | 63.45 ± 0.04 | 52.67 ± 0.12 | 65.30 ± 0.10 | 67.87 ± 0.09 | 68.74 ± 0.12 | 63.74 ± 0.14 | 61.79 ± 0.08 | 64.02 ± 0.10 |
 Mistral Medium 3.5 128B Mistral AI | 60.43 ± 0.09 | 33.06 ± 0.39 | 66.85 ± 0.16 | 67.79 ± 0.16 | 66.69 ± 0.16 | 61.31 ± 0.37 | 61.03 ± 0.15 | 66.24 ± 0.18 |
SEA-LION v4.5 (Gemma E2B) 5B AISG | 49.31 ± 0.05 | 33.73 ± 0.18 | 53.19 ± 0.17 | 57.11 ± 0.11 | 55.36 ± 0.19 | 47.59 ± 0.13 | 44.91 ± 0.14 | 53.31 ± 0.19 |
 Nemotron 3 Super 120B MoE NVIDIA | 47.94 ± 0.09 | 6.11 ± 0.17 | 55.43 ± 0.19 | 63.86 ± 0.17 | 62.21 ± 0.19 | 27.05 ± 0.25 | 59.20 ± 0.16 | 61.70 ± 0.21 |
Gemma 4 (E2B) 5B | 47.21 ± 0.06 | 32.45 ± 0.14 | 51.55 ± 0.13 | 54.13 ± 0.14 | 52.92 ± 0.18 | 46.80 ± 0.16 | 42.63 ± 0.11 | 49.95 ± 0.20 |
 MERaLiON 2 10B A*STAR | 44.17 ± 0.07 | 10.74 ± 0.20 | 54.25 ± 0.20 | 58.00 ± 0.14 | 55.29 ± 0.13 | 34.17 ± 0.18 | 44.84 ± 0.21 | 51.89 ± 0.20 |
 Olmo 3.1 32B AI2 | 40.87 ± 0.07 | 6.14 ± 0.11 | 48.90 ± 0.18 | 55.80 ± 0.16 | 57.95 ± 0.20 | 24.24 ± 0.19 | 41.83 ± 0.18 | 51.25 ± 0.18 |
 GLM 4.7 Flash 30B MoE Z.ai | 34.83 ± 0.11 | 10.22 ± 0.21 | 40.08 ± 0.31 | 46.99 ± 0.25 | 48.83 ± 0.29 | 20.45 ± 0.36 | 33.02 ± 0.27 | 44.19 ± 0.35 |
 Apertus 8B Swiss AI | 27.62 ± 0.09 | 8.46 ± 0.28 | 24.63 ± 0.22 | 35.66 ± 0.20 | 40.45 ± 0.29 | 16.72 ± 0.25 | 32.96 ± 0.36 | 34.44 ± 0.23 |
 Tiny Aya Water 3B CohereLabs | 19.79 ± 0.08 | 4.96 ± 0.16 | 24.05 ± 0.30 | 33.29 ± 0.19 | 32.73 ± 0.27 | 3.21 ± 0.10 | 8.10 ± 0.21 | 32.17 ± 0.26 |