Indonesian Performance
Indonesian Scores by Model
Average of 2000 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Indonesian Competency & Task Scores
Average of 2000 bootstraps. 95% CI are shown.
Model Size: ≤200B
Open instruct models only
Model | ID | Cultural | Inst. Following | Knowledge | Linguistic Diagnostics | Multi-Turn Chat | NLG | NLR | NLU | Safety |
|---|---|---|---|---|---|---|---|---|---|---|
 Gemma 4 31B | 78.92 [-1.34, +1.26] | 82.76 [-5.83, +5.46] | 96.62 [-3.05, +2.43] | 77.72 [-5.01, +4.78] | 74.36 [-4.07, +3.91] | 85.12 [-3.84, +3.61] | 56.10 [-0.71, +0.70] | 89.01 [-1.96, +1.94] | 81.98 [-3.25, +3.08] | 66.57 [-6.43, +5.67] |
 SEA-LION v4.5 (Qwen) 27B AISG | 75.60 [-1.38, +1.46] | 79.26 [-5.39, +5.33] | 91.89 [-4.27, +3.94] | 78.74 [-4.29, +4.46] | 57.21 [-4.38, +4.28] | 85.04 [-3.69, +3.43] | 54.85 [-0.60, +0.63] | 88.23 [-2.02, +1.88] | 79.81 [-3.42, +3.27] | 65.37 [-6.50, +6.18] |
 Qwen 3.5 122B MoE Alibaba | 75.57 [-1.25, +1.31] | 79.63 [-5.60, +5.15] | 90.93 [-4.74, +4.31] | 77.38 [-4.54, +4.21] | 62.25 [-4.02, +3.92] | 85.61 [-3.58, +3.44] | 54.74 [-0.54, +0.56] | 87.88 [-2.02, +1.94] | 78.00 [-3.51, +3.26] | 63.70 [-4.20, +3.91] |
Qwen 3.6 27B Alibaba | 74.05 [-1.25, +1.26] | 75.50 [-5.10, +4.85] | 90.72 [-4.17, +4.04] | 76.02 [-4.14, +4.36] | 55.22 [-3.66, +3.68] | 85.92 [-3.51, +3.47] | 54.43 [-0.51, +0.53] | 87.26 [-1.92, +1.76] | 79.90 [-3.25, +3.07] | 61.50 [-5.16, +4.86] |
 Llama 4 Scout 109B MoE Meta | 67.86 [-1.58, +1.56] | 63.27 [-6.46, +6.12] | 92.65 [-4.56, +4.02] | 66.14 [-5.73, +5.57] | 54.77 [-4.51, +4.48] | 69.48 [-4.82, +4.72] | 55.36 [-0.68, +0.71] | 75.12 [-2.59, +2.74] | 76.12 [-3.74, +3.56] | 57.81 [-6.39, +6.67] |
 Mistral Medium 3.5 128B Mistral AI | 67.83 [-1.37, +1.29] | 47.52 [-4.92, +4.75] | 86.20 [-5.84, +5.23] | 72.72 [-4.68, +4.70] | 50.06 [-3.48, +3.57] | 82.55 [-3.56, +3.55] | 54.76 [-0.61, +0.62] | 78.88 [-2.23, +2.09] | 75.10 [-3.93, +3.60] | 62.63 [-5.31, +4.36] |
 Nemotron 3 Super 120B MoE NVIDIA | 63.84 [-1.41, +1.39] | 54.22 [-4.98, +4.69] | 79.78 [-6.56, +6.29] | 67.77 [-4.82, +4.48] | 41.26 [-3.10, +3.16] | 84.74 [-3.76, +3.51] | 51.93 [-0.44, +0.43] | 70.87 [-2.38, +2.34] | 70.88 [-3.54, +3.40] | 53.10 [-5.41, +5.35] |
 MERaLiON 2 10B A*STAR | 58.01 [-1.66, +1.65] | 44.10 [-5.45, +5.64] | 72.44 [-7.33, +6.97] | 50.25 [-5.80, +5.70] | 44.88 [-4.58, +4.72] | 54.76 [-5.15, +5.14] | 55.33 [-0.69, +0.67] | 72.98 [-2.86, +2.74] | 71.10 [-3.81, +3.60] | 56.21 [-5.72, +5.10] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 57.13 [-1.59, +1.66] | 49.87 [-6.25, +6.03] | 89.16 [-5.12, +4.53] | 43.53 [-6.24, +5.89] | 34.62 [-4.72, +5.25] | 73.02 [-5.39, +5.28] | 53.80 [-0.60, +0.62] | 60.66 [-3.87, +3.68] | 64.10 [-4.14, +4.07] | 45.43 [-5.26, +5.50] |
 Olmo 3.1 32B AI2 | 55.89 [-1.56, +1.62] | 54.90 [-4.61, +4.64] | 82.76 [-6.10, +5.57] | 45.45 [-5.70, +5.76] | 27.66 [-3.78, +4.05] | 71.90 [-4.92, +4.61] | 51.62 [-0.55, +0.54] | 66.97 [-3.53, +3.59] | 62.84 [-4.24, +3.97] | 38.91 [-6.21, +7.03] |
Gemma 4 (E2B) 5B | 54.22 [-1.51, +1.58] | 30.94 [-5.20, +5.43] | 85.58 [-5.70, +5.37] | 42.73 [-5.97, +6.19] | 34.80 [-5.04, +5.01] | 74.18 [-5.12, +5.04] | 53.41 [-0.57, +0.59] | 59.24 [-3.93, +3.71] | 63.77 [-4.16, +4.26] | 43.37 [-4.67, +5.16] |
 GLM 4.7 Flash 30B MoE Z.ai | 46.89 [-1.29, +1.21] | 14.73 [-1.62, +1.62] | 76.11 [-5.16, +4.84] | 42.76 [-4.72, +4.61] | 25.85 [-3.07, +3.02] | 57.86 [-4.89, +5.00] | 51.04 [-0.62, +0.60] | 37.34 [-1.94, +1.99] | 64.07 [-3.34, +3.10] | 52.23 [-5.15, +5.62] |
 Apertus 8B Swiss AI | 35.67 [-1.20, +1.23] | 2.61 [-1.29, +1.34] | 73.72 [-6.10, +5.93] | 24.58 [-4.16, +4.13] | 21.26 [-1.94, +1.93] | 44.60 [-5.68, +5.66] | 53.26 [-0.62, +0.65] | 23.20 [-2.01, +1.91] | 46.71 [-3.01, +2.86] | 31.06 [-3.53, +4.36] |
 Tiny Aya Water 3B CohereLabs | 33.23 [-1.13, +1.13] | 0.00 [-0.00, +0.00] | 70.29 [-6.00, +5.19] | 34.32 [-4.99, +5.06] | 13.43 [-2.02, +2.41] | 30.62 [-4.69, +4.81] | 40.71 [-0.53, +0.50] | 30.60 [-2.48, +2.35] | 46.21 [-3.03, +3.09] | 32.90 [-2.50, +2.51] |