Vietnamese Performance
Vietnamese Scores by Model
Average of 2000 bootstraps. 95% CI are shown.
Gemma 4
Qwen 3.5
Qwen 3.6
SEA-LION v4.5 (Qwen)
Muse Glimmer
SEA-LION v4.8 (Nemotron)
GPT OSS
Mistral Medium 3.5
SEA-LION v4.8 (Nemotron)
Apertus 1.5
GLM 4.7 Flash
SEA-LION v4.5 (Gemma E2B)
Gemma 4 (E2B)
Nemotron 3.5 Lightning
Nemotron 3 Super
MERaLiON 2
Olmo 3.1
Tiny Aya WaterLegend:
Reasoning model
·
Model Size: ≤200B
·
Open models only
Vietnamese Competency & Task Scores
Average of 2000 bootstraps. 95% CI are shown.
Model | VI | Cultural | Inst. Following | Knowledge | Multi-Turn Chat | NLG | NLR | NLU | Safety |
|---|---|---|---|---|---|---|---|---|---|
Gemma 4 31B | 77.09 [-1.73, +1.64] | 75.32 [-9.47, +9.97] | 96.21 [-3.35, +2.72] | 84.11 [-4.20, +3.85] | 87.12 [-3.31, +3.40] | 56.21 [-0.91, +0.92] | 77.20 [-2.26, +2.25] | 68.11 [-4.62, +4.41] | 72.42 [-3.89, +3.68] |
Qwen 3.5 122B MoE Alibaba | 76.39 [-1.59, +1.56] | 75.21 [-9.07, +9.24] | 94.92 [-3.14, +2.82] | 84.32 [-4.07, +3.68] | 87.06 [-3.23, +3.26] | 55.18 [-0.86, +0.83] | 76.17 [-2.20, +2.27] | 70.61 [-4.34, +4.26] | 67.66 [-4.59, +4.41] |
Qwen 3.6 27B Alibaba | 76.12 [-1.46, +1.39] | 75.69 [-7.74, +6.89] | 94.40 [-3.92, +3.34] | 83.69 [-3.82, +3.48] | 84.71 [-3.30, +3.19] | 54.67 [-0.81, +0.78] | 79.22 [-2.18, +1.96] | 71.71 [-4.22, +3.84] | 64.91 [-4.28, +3.85] |
SEA-LION v4.5 (Qwen) 27B AISG | 73.85 [-1.40, +1.40] | 74.95 [-6.98, +6.52] | 94.92 [-2.89, +2.35] | 81.53 [-3.95, +3.72] | 79.32 [-4.05, +4.01] | 51.78 [-0.78, +0.76] | 77.49 [-2.13, +2.11] | 68.00 [-4.15, +4.07] | 62.77 [-4.01, +3.77] |
Muse Glimmer 30B Meta | 73.52 [-1.47, +1.38] | 61.37 [-7.98, +7.60] | 95.88 [-3.26, +2.34] | 81.49 [-4.24, +4.01] | 87.09 [-2.85, +2.82] | 53.87 [-0.76, +0.73] | 71.21 [-2.27, +2.33] | 69.61 [-4.30, +4.22] | 67.62 [-4.02, +3.78] |
SEA-LION v4.8 (Nemotron) 120B MoE AISG | 70.36 [-1.65, +1.67] | 67.63 [-8.61, +8.08] | 82.03 [-6.32, +5.71] | 79.64 [-4.14, +4.07] | 84.88 [-3.15, +3.10] | 53.05 [-0.77, +0.80] | 73.88 [-2.32, +2.20] | 67.34 [-4.32, +4.08] | 54.40 [-4.21, +3.87] |
GPT OSS 120B MoE mxfp4 OpenAI | 69.81 [-1.69, +1.59] | 66.08 [-7.69, +7.34] | 88.25 [-5.16, +4.49] | 74.79 [-4.46, +4.42] | 83.50 [-3.98, +3.64] | 53.40 [-0.71, +0.69] | 71.74 [-2.38, +2.49] | 65.06 [-4.17, +3.94] | 55.67 [-4.45, +4.75] |
Mistral Medium 3.5 128B Mistral AI | 68.82 [-1.47, +1.43] | 46.55 [-6.47, +6.56] | 86.44 [-5.25, +4.64] | 77.99 [-3.95, +4.10] | 82.88 [-3.98, +3.94] | 55.73 [-0.90, +0.90] | 69.46 [-2.18, +2.08] | 71.23 [-3.86, +3.85] | 60.30 [-4.30, +3.97] |
SEA-LION v4.8 (Nemotron) 30B MoE AISG | 64.86 [-1.67, +1.63] | 50.84 [-7.97, +8.65] | 78.37 [-6.11, +5.79] | 74.65 [-4.61, +4.18] | 83.06 [-3.60, +3.29] | 52.08 [-0.78, +0.84] | 66.52 [-2.17, +2.19] | 65.85 [-4.20, +3.98] | 47.50 [-4.18, +4.07] |
Apertus 1.5 70B Swiss AI | 64.66 [-1.62, +1.63] | 62.81 [-7.24, +7.51] | 83.05 [-5.08, +4.45] | 61.50 [-4.70, +4.38] | 68.25 [-4.93, +4.72] | 53.48 [-0.72, +0.74] | 66.15 [-2.40, +2.38] | 68.00 [-4.02, +3.97] | 54.04 [-4.29, +4.30] |
GLM 4.7 Flash 30B MoE Z.ai | 64.36 [-1.53, +1.53] | 56.45 [-5.44, +5.95] | 79.89 [-5.48, +5.00] | 67.93 [-4.43, +4.32] | 73.84 [-4.61, +4.92] | 53.10 [-0.71, +0.73] | 69.12 [-2.24, +2.15] | 65.56 [-4.35, +4.00] | 49.03 [-3.99, +4.07] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 61.95 [-1.75, +1.78] | 50.31 [-7.80, +8.56] | 85.92 [-5.56, +5.04] | 55.50 [-5.09, +5.33] | 76.67 [-4.80, +4.41] | 53.25 [-0.70, +0.65] | 61.11 [-2.78, +2.60] | 64.01 [-4.31, +4.02] | 48.80 [-4.95, +4.88] |
Gemma 4 (E2B) 5B | 61.19 [-1.66, +1.69] | 45.89 [-6.21, +6.56] | 86.79 [-5.60, +5.00] | 55.54 [-5.21, +5.50] | 77.49 [-4.92, +4.49] | 52.37 [-0.64, +0.60] | 61.42 [-2.76, +2.56] | 62.12 [-4.52, +4.22] | 47.86 [-5.05, +5.01] |
Nemotron 3.5 Lightning 30B MoE NVIDIA | 60.97 [-1.55, +1.52] | 35.49 [-5.04, +5.37] | 75.68 [-6.76, +6.35] | 72.44 [-4.40, +4.47] | 83.06 [-3.51, +3.59] | 48.94 [-0.69, +0.67] | 61.21 [-2.51, +2.39] | 63.49 [-4.11, +4.00] | 47.42 [-4.70, +4.67] |
Nemotron 3 Super 120B MoE NVIDIA | 55.31 [-1.48, +1.55] | 62.76 [-7.88, +7.88] | 80.78 [-5.18, +4.94] | 50.82 [-3.65, +3.68] | 83.20 [-3.11, +3.01] | 51.89 [-0.70, +0.70] | 40.11 [-2.28, +2.32] | 36.31 [-3.73, +3.54] | 36.61 [-4.22, +3.79] |
MERaLiON 2 10B A*STAR | 51.89 [-1.81, +1.85] | 50.65 [-7.78, +8.30] | 63.93 [-6.90, +6.43] | 27.41 [-5.24, +5.43] | 50.51 [-4.86, +4.97] | 52.58 [-0.73, +0.71] | 56.05 [-3.12, +2.91] | 55.45 [-4.04, +3.85] | 58.56 [-5.29, +4.88] |
Olmo 3.1 32B AI2 | 51.07 [-1.89, +2.01] | 33.95 [-9.34, +9.17] | 81.14 [-6.73, +5.89] | 31.23 [-5.94, +5.81] | 69.95 [-4.87, +4.84] | 51.36 [-0.77, +0.76] | 41.53 [-3.72, +3.58] | 64.92 [-4.02, +4.23] | 34.47 [-4.80, +5.26] |
Tiny Aya Water 3B CohereLabs | 32.15 [-1.36, +1.29] | 0.00 [-0.00, +0.00] | 65.89 [-6.49, +6.37] | 30.80 [-5.14, +5.07] | 35.17 [-4.74, +4.75] | 39.23 [-0.63, +0.58] | 20.62 [-2.33, +2.24] | 36.76 [-3.11, +3.03] | 28.76 [-3.51, +3.71] |
Legend:
Reasoning model
·
Model Size: ≤200B
·
Open models only