Thai Performance
Thai Scores by Model
Average of 2000 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Thai Competency & Task Scores
Average of 2000 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Model | TH | Cultural | Inst. Following | Knowledge | Multi-Turn Chat | NLG | NLR | NLU | Safety |
|---|---|---|---|---|---|---|---|---|---|
 SEA-LION v4.5 (Qwen) 27B AISG | 72.32 [-1.26, +1.28] | 86.81 [-4.72, +4.30] | 91.69 [-4.57, +4.06] | 57.49 [-4.44, +4.26] | 82.41 [-3.64, +3.49] | 58.30 [-0.78, +0.81] | 74.46 [-2.26, +2.20] | 63.93 [-3.34, +2.97] | 63.48 [-4.22, +3.91] |
 Gemma 4 31B | 71.09 [-1.40, +1.30] | 85.06 [-4.79, +4.42] | 93.25 [-4.50, +4.12] | 58.71 [-5.25, +5.23] | 82.24 [-4.16, +3.91] | 59.45 [-0.93, +0.95] | 79.71 [-2.21, +2.24] | 63.48 [-3.43, +3.24] | 46.81 [-4.38, +3.89] |
 Qwen 3.6 27B Alibaba | 68.85 [-1.38, +1.34] | 79.22 [-4.84, +4.35] | 85.45 [-5.95, +5.18] | 52.73 [-4.66, +4.63] | 83.30 [-3.28, +3.27] | 57.84 [-0.77, +0.80] | 71.33 [-2.12, +2.08] | 62.79 [-2.98, +2.73] | 58.14 [-4.20, +4.10] |
Qwen 3.5 122B MoE Alibaba | 67.47 [-1.50, +1.54] | 72.69 [-6.40, +6.11] | 85.42 [-6.67, +5.58] | 56.98 [-4.38, +4.31] | 83.82 [-3.47, +3.36] | 58.33 [-0.74, +0.77] | 64.63 [-2.30, +2.25] | 62.99 [-3.34, +2.99] | 54.91 [-4.35, +4.30] |
 Llama 4 Scout 109B MoE Meta | 61.81 [-1.66, +1.69] | 76.89 [-6.49, +6.48] | 87.18 [-6.31, +5.69] | 42.05 [-5.45, +5.43] | 63.66 [-5.32, +4.94] | 58.52 [-0.88, +0.90] | 62.87 [-2.86, +2.82] | 61.76 [-3.45, +3.25] | 41.56 [-4.76, +4.63] |
 Mistral Medium 3.5 128B Mistral AI | 61.13 [-1.60, +1.58] | 61.81 [-6.23, +5.89] | 71.11 [-7.48, +6.52] | 49.43 [-4.65, +4.27] | 76.25 [-4.42, +4.07] | 54.09 [-0.77, +0.79] | 61.21 [-2.68, +2.72] | 59.98 [-3.13, +3.17] | 55.13 [-3.70, +3.44] |
 Nemotron 3 Super 120B MoE NVIDIA | 59.28 [-1.44, +1.46] | 65.87 [-5.09, +4.46] | 69.16 [-8.28, +7.47] | 48.17 [-4.39, +4.35] | 82.82 [-3.41, +3.32] | 55.23 [-0.66, +0.72] | 63.98 [-2.36, +2.32] | 56.74 [-3.12, +2.79] | 32.29 [-3.89, +3.98] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 44.94 [-1.64, +1.63] | 12.60 [-4.51, +4.66] | 76.11 [-7.11, +6.76] | 25.40 [-4.63, +4.47] | 69.40 [-4.92, +4.63] | 51.54 [-0.72, +0.71] | 38.12 [-3.54, +3.70] | 53.70 [-4.14, +3.86] | 32.69 [-4.35, +4.68] |
 MERaLiON 2 10B A*STAR | 44.83 [-1.72, +1.68] | 37.97 [-6.46, +6.87] | 53.47 [-7.84, +7.41] | 38.90 [-4.86, +4.66] | 41.50 [-4.08, +4.35] | 50.72 [-0.56, +0.61] | 55.58 [-3.26, +3.04] | 31.87 [-3.36, +3.28] | 48.60 [-4.19, +4.03] |
Gemma 4 (E2B) 5B | 42.68 [-1.58, +1.55] | 0.00 [-0.00, +0.00] | 74.24 [-7.49, +7.26] | 25.16 [-4.88, +4.91] | 69.61 [-4.81, +4.44] | 54.19 [-0.63, +0.64] | 31.73 [-3.25, +3.15] | 52.64 [-4.13, +4.05] | 33.86 [-4.69, +4.77] |
 Olmo 3.1 32B AI2 | 41.86 [-1.92, +1.80] | 49.91 [-6.76, +6.60] | 60.98 [-8.48, +8.65] | 16.79 [-4.75, +4.50] | 58.06 [-6.22, +6.18] | 46.98 [-0.65, +0.66] | 29.96 [-4.13, +3.97] | 49.98 [-3.62, +3.47] | 22.21 [-4.23, +4.24] |
 GLM 4.7 Flash 30B MoE Z.ai | 33.06 [-1.33, +1.36] | 9.99 [-2.68, +2.83] | 58.13 [-7.75, +7.75] | 12.57 [-2.31, +2.35] | 50.16 [-5.03, +5.16] | 47.80 [-0.67, +0.67] | 13.30 [-1.74, +1.71] | 47.54 [-3.08, +3.03] | 24.98 [-3.17, +3.26] |
 Apertus 8B Swiss AI | 33.05 [-1.32, +1.31] | 16.29 [-1.89, +1.73] | 52.82 [-7.45, +7.80] | 13.55 [-2.66, +2.75] | 37.54 [-5.37, +5.39] | 50.72 [-0.67, +0.69] | 25.18 [-1.92, +1.82] | 37.47 [-2.94, +2.71] | 30.83 [-2.76, +2.82] |
 Tiny Aya Water 3B CohereLabs | 8.04 [-0.74, +0.75] | 0.00 [-0.00, +0.00] | 22.85 [-4.61, +4.90] | 0.05 [-0.05, +0.49] | 7.47 [-1.96, +2.17] | 20.04 [-0.49, +0.47] | 0.00 [-0.00, +0.00] | 13.92 [-2.45, +2.55] | 0.00 [-0.00, +0.00] |