SEA Performance
Overall SEA Average
Average of 2000 bootstraps. 95% CI are shown.
Legend:
Reasoning model
No open weights
·
Model Size: All
·
Open/closed models
Language Performance by Model
Average of 2000 bootstraps. 95% CI are shown.
Model | SEA | MY | TL | ID | MS | TA | TH | VI |
|---|---|---|---|---|---|---|---|---|
GPT 6 Astra OpenAI | 79.94 [-0.54, +0.53] | 76.97 [-1.85, +1.79] | 81.32 [-1.19, +1.21] | 83.47 [-1.09, +1.07] | 80.21 [-1.31, +1.28] | 80.18 [-1.92, +1.97] | 76.77 [-1.12, +1.06] | 80.68 [-1.19, +1.16] |
Gemini 4 Hydrogen | 79.94 [-0.54, +0.52] | 78.31 [-1.81, +1.66] | 80.22 [-1.17, +1.16] | 83.19 [-1.06, +1.05] | 81.18 [-1.31, +1.30] | 79.05 [-1.79, +1.84] | 76.89 [-1.02, +1.01] | 80.72 [-1.14, +1.12] |
SEA-LION v5 (Qwen) 397B MoE AISG | 77.10 [-0.53, +0.53] | 70.58 [-1.88, +1.69] | 78.43 [-1.28, +1.27] | 80.29 [-1.17, +1.21] | 79.53 [-1.61, +1.52] | 76.55 [-1.79, +1.60] | 75.43 [-1.12, +1.09] | 78.87 [-1.34, +1.17] |
Opus 4.8 Anthropic | 76.67 [-0.60, +0.57] | 68.67 [-2.13, +2.06] | 78.27 [-1.37, +1.28] | 82.04 [-1.21, +1.14] | 78.97 [-1.48, +1.52] | 75.47 [-2.01, +1.98] | 74.44 [-1.29, +1.25] | 78.83 [-1.39, +1.40] |
Qwen 3.5 397B MoE Alibaba | 76.46 [-0.58, +0.54] | 69.20 [-1.89, +1.86] | 78.24 [-1.23, +1.23] | 79.68 [-1.27, +1.22] | 79.40 [-1.51, +1.42] | 76.24 [-1.84, +1.61] | 74.01 [-1.22, +1.21] | 78.43 [-1.26, +1.22] |
Inkling 975B MoE NVFP4 Thinking Machines | 76.26 [-0.48, +0.48] | 65.92 [-1.57, +1.63] | 77.70 [-1.14, +1.15] | 79.75 [-1.11, +1.03] | 80.56 [-1.42, +1.30] | 78.62 [-1.30, +1.25] | 73.19 [-1.16, +1.14] | 78.09 [-1.11, +1.09] |
DeepSeek V4 Pro 1600B MoE DeepSeek | 75.88 [-0.54, +0.54] | 69.72 [-1.65, +1.69] | 77.74 [-1.26, +1.28] | 79.03 [-1.19, +1.11] | 79.35 [-1.41, +1.36] | 75.71 [-1.77, +1.60] | 72.36 [-1.27, +1.24] | 77.24 [-1.30, +1.26] |
Gemma 4 31B | 75.71 [-0.59, +0.56] | 68.91 [-1.79, +1.80] | 78.91 [-1.36, +1.29] | 79.07 [-1.31, +1.32] | 78.32 [-1.64, +1.62] | 74.07 [-1.79, +1.65] | 73.20 [-1.29, +1.31] | 77.48 [-1.30, +1.28] |
GLM 5.3 743B MoE Z.ai | 75.00 [-0.54, +0.53] | 65.91 [-2.06, +1.84] | 76.84 [-1.34, +1.27] | 79.64 [-1.12, +1.04] | 79.15 [-1.45, +1.39] | 75.83 [-1.75, +1.68] | 70.61 [-1.40, +1.34] | 77.01 [-1.31, +1.24] |
SEA-LION v5 (Qwen) 27B AISG | 72.82 [-0.57, +0.57] | 63.36 [-1.63, +1.59] | 73.82 [-1.27, +1.26] | 78.41 [-1.28, +1.25] | 74.11 [-1.74, +1.75] | 71.84 [-1.95, +1.80] | 71.93 [-1.23, +1.20] | 76.24 [-1.34, +1.23] |
Muse Glimmer 30B Meta | 72.70 [-0.54, +0.55] | 64.71 [-1.68, +1.71] | 75.47 [-1.42, +1.32] | 75.36 [-1.22, +1.21] | 76.59 [-1.57, +1.61] | 71.72 [-1.67, +1.61] | 69.40 [-1.22, +1.22] | 75.66 [-1.32, +1.23] |
MiniMax M3 428B MoE Minimax | 71.76 [-0.60, +0.56] | 61.86 [-1.84, +1.83] | 74.02 [-1.54, +1.41] | 75.53 [-1.25, +1.22] | 75.86 [-1.53, +1.47] | 72.89 [-1.63, +1.60] | 68.60 [-1.36, +1.36] | 73.54 [-1.41, +1.43] |
Qwen 3.8 27B Alibaba | 68.48 [-0.56, +0.55] | 58.13 [-1.59, +1.55] | 69.99 [-1.38, +1.37] | 74.13 [-1.38, +1.27] | 71.13 [-1.70, +1.71] | 67.11 [-1.75, +1.76] | 66.42 [-1.25, +1.27] | 72.46 [-1.41, +1.39] |
SEA-LION v5 (Nemotron) 4B AISG | 68.39 [-0.60, +0.60] | 58.91 [-1.88, +1.72] | 73.53 [-1.40, +1.41] | 72.09 [-1.50, +1.43] | 70.26 [-1.84, +1.93] | 69.13 [-1.92, +1.76] | 65.33 [-1.30, +1.29] | 69.46 [-1.61, +1.64] |
Mistral Medium 3.5 128B Mistral AI | 64.90 [-0.57, +0.57] | 40.14 [-1.51, +1.61] | 70.30 [-1.45, +1.42] | 71.20 [-1.34, +1.29] | 73.97 [-1.67, +1.65] | 66.20 [-1.58, +1.53] | 61.76 [-1.45, +1.49] | 70.69 [-1.55, +1.43] |
SEA-LION v5 (Qwen) 9B AISG | 64.69 [-0.58, +0.56] | 55.52 [-1.56, +1.46] | 62.51 [-1.55, +1.44] | 68.82 [-1.44, +1.34] | 68.28 [-1.69, +1.74] | 65.43 [-1.78, +1.69] | 64.11 [-1.30, +1.23] | 68.13 [-1.42, +1.46] |
Nemotron 3 Ultra 550B MoE NVIDIA | 63.94 [-0.60, +0.59] | 35.03 [-1.78, +1.69] | 70.76 [-1.29, +1.27] | 73.39 [-1.40, +1.39] | 73.71 [-1.55, +1.44] | 60.92 [-1.78, +1.79] | 64.94 [-1.45, +1.38] | 68.83 [-1.46, +1.57] |
GPT OSS 120B MoE mxfp4 OpenAI | 63.50 [-0.64, +0.65] | 45.84 [-1.87, +1.93] | 70.99 [-1.47, +1.51] | 68.17 [-1.50, +1.41] | 68.87 [-1.81, +1.77] | 62.24 [-2.02, +1.89] | 62.43 [-1.35, +1.37] | 65.95 [-1.51, +1.54] |
SEA-LION v5 (Gemma E2B) 2B AISG | 60.77 [-0.60, +0.60] | 50.67 [-1.81, +1.77] | 66.14 [-1.57, +1.51] | 63.50 [-1.46, +1.37] | 62.14 [-1.80, +1.80] | 62.39 [-1.88, +1.81] | 59.02 [-1.24, +1.20] | 61.52 [-1.44, +1.50] |
Nemotron 3.5 Bolt (E4B) 4B NVIDIA | 60.56 [-0.55, +0.56] | 52.65 [-1.85, +1.71] | 65.69 [-1.47, +1.44] | 62.56 [-1.29, +1.15] | 62.27 [-1.55, +1.52] | 60.66 [-1.70, +1.68] | 59.16 [-1.32, +1.25] | 60.91 [-1.44, +1.37] |
Command A+ 218B MoE CohereLabs | 53.87 [-0.54, +0.57] | 15.73 [-1.25, +1.21] | 59.37 [-1.33, +1.33] | 65.12 [-1.40, +1.34] | 62.27 [-1.67, +1.77] | 55.99 [-1.62, +1.56] | 56.11 [-1.37, +1.30] | 62.49 [-1.39, +1.45] |
Gemma 4 (E2B) 2B | 53.62 [-0.60, +0.62] | 40.86 [-1.85, +1.79] | 58.98 [-1.57, +1.57] | 56.99 [-1.27, +1.25] | 55.78 [-1.62, +1.57] | 54.64 [-1.71, +1.70] | 52.60 [-1.44, +1.33] | 55.45 [-1.44, +1.40] |
Apertus 1.5 70B Swiss AI | 51.82 [-0.58, +0.59] | 21.01 [-1.31, +1.32] | 57.94 [-1.55, +1.55] | 58.67 [-1.38, +1.35] | 61.88 [-1.68, +1.63] | 51.53 [-1.99, +1.94] | 52.46 [-1.55, +1.55] | 59.22 [-1.45, +1.44] |
Olmo 3.1 32B AI2 | 37.92 [-0.59, +0.60] | 6.56 [-1.11, +1.15] | 48.83 [-1.89, +1.87] | 49.42 [-1.51, +1.52] | 52.41 [-1.91, +1.92] | 24.86 [-1.57, +1.53] | 35.98 [-1.67, +1.68] | 47.37 [-1.56, +1.54] |
Legend:
Reasoning model
No open weights
·
Model Size: All
·
Open/closed models