Filipino Performance
Filipino Scores by Model
Average of 2000 bootstraps. 95% CI are shown.
Gemma 4
Qwen 3.5
Qwen 3.6
SEA-LION v4.5 (Qwen)
Muse Glimmer
GPT OSS
SEA-LION v4.8 (Nemotron)
Mistral Medium 3.5
Nemotron 3 Super
GLM 4.7 Flash
SEA-LION v4.8 (Nemotron)
SEA-LION v4.5 (Gemma E2B)
Gemma 4 (E2B)
Apertus 1.5
Nemotron 3.5 Lightning
MERaLiON 2
Olmo 3.1
Tiny Aya WaterLegend:
Reasoning model
·
Model Size: ≤200B
·
Open models only
Filipino Competency & Task Scores
Average of 2000 bootstraps. 95% CI are shown.
Model | TL | Cultural | Inst. Following | Knowledge | Multi-Turn Chat | NLG | NLR | NLU | Safety |
|---|---|---|---|---|---|---|---|---|---|
Gemma 4 31B | 78.44 [-1.30, +1.27] | 78.44 [-3.54, +3.31] | 94.02 [-4.49, +3.84] | 85.11 [-3.91, +3.89] | 84.18 [-4.21, +3.83] | 58.92 [-0.85, +0.92] | 82.85 [-2.51, +2.59] | 76.85 [-4.13, +3.82] | 67.19 [-5.43, +5.20] |
Qwen 3.5 122B MoE Alibaba | 75.37 [-1.31, +1.33] | 74.38 [-3.78, +3.50] | 91.27 [-4.72, +3.98] | 83.71 [-4.00, +3.66] | 82.52 [-4.04, +3.86] | 56.21 [-0.69, +0.71] | 79.45 [-2.74, +2.63] | 74.66 [-4.15, +3.71] | 60.74 [-5.51, +5.68] |
Qwen 3.6 27B Alibaba | 75.23 [-1.25, +1.28] | 74.53 [-4.02, +3.69] | 91.30 [-4.64, +4.17] | 81.84 [-3.76, +3.70] | 82.78 [-3.55, +3.39] | 54.55 [-0.62, +0.65] | 80.02 [-2.80, +2.72] | 76.06 [-4.02, +3.88] | 60.76 [-4.85, +4.76] |
SEA-LION v4.5 (Qwen) 27B AISG | 75.16 [-1.36, +1.36] | 74.95 [-4.04, +3.79] | 92.05 [-4.78, +3.91] | 82.14 [-3.94, +3.69] | 78.89 [-4.76, +4.48] | 55.54 [-0.66, +0.69] | 80.91 [-2.71, +2.59] | 74.07 [-4.49, +4.11] | 62.76 [-4.87, +4.75] |
Muse Glimmer 30B Meta | 74.28 [-1.32, +1.30] | 67.22 [-4.33, +4.01] | 92.75 [-4.90, +4.16] | 83.65 [-4.07, +3.68] | 84.39 [-3.98, +3.78] | 54.55 [-0.55, +0.57] | 74.56 [-2.72, +2.54] | 73.66 [-4.29, +4.01] | 63.49 [-4.71, +4.65] |
GPT OSS 120B MoE mxfp4 OpenAI | 71.38 [-1.43, +1.46] | 69.04 [-4.17, +3.93] | 83.45 [-5.83, +5.48] | 75.24 [-4.44, +4.26] | 82.34 [-4.20, +4.07] | 54.29 [-0.54, +0.58] | 74.95 [-2.84, +2.86] | 71.62 [-4.78, +4.43] | 60.09 [-5.11, +5.00] |
SEA-LION v4.8 (Nemotron) 120B MoE AISG | 71.01 [-1.47, +1.39] | 74.12 [-3.32, +3.08] | 78.17 [-6.03, +6.00] | 80.25 [-4.12, +3.92] | 79.99 [-4.15, +3.96] | 50.82 [-0.63, +0.67] | 74.34 [-2.88, +2.64] | 71.22 [-4.59, +4.17] | 59.21 [-5.20, +4.88] |
Mistral Medium 3.5 128B Mistral AI | 70.12 [-1.41, +1.48] | 66.40 [-3.85, +3.77] | 77.02 [-6.31, +5.95] | 78.60 [-4.35, +4.11] | 79.50 [-4.71, +4.21] | 56.98 [-0.78, +0.82] | 71.21 [-2.74, +2.71] | 68.46 [-4.72, +4.21] | 62.81 [-4.22, +4.26] |
Nemotron 3 Super 120B MoE NVIDIA | 66.92 [-1.39, +1.35] | 71.72 [-3.54, +3.35] | 77.04 [-6.21, +5.81] | 79.20 [-3.82, +3.72] | 79.22 [-3.56, +3.53] | 49.95 [-0.51, +0.56] | 70.65 [-2.87, +2.82] | 65.67 [-4.42, +4.16] | 41.89 [-4.76, +5.00] |
GLM 4.7 Flash 30B MoE Z.ai | 62.97 [-1.47, +1.44] | 56.67 [-3.77, +3.57] | 76.53 [-5.70, +5.26] | 67.49 [-4.45, +4.17] | 63.66 [-5.53, +5.50] | 51.69 [-0.65, +0.73] | 67.41 [-2.91, +2.85] | 69.72 [-4.07, +3.74] | 50.57 [-4.43, +4.59] |
SEA-LION v4.8 (Nemotron) 30B MoE AISG | 61.82 [-1.52, +1.35] | 65.38 [-3.81, +3.68] | 74.42 [-5.84, +5.70] | 68.25 [-4.67, +4.67] | 76.75 [-4.38, +3.96] | 46.49 [-0.60, +0.62] | 62.60 [-3.30, +3.38] | 62.81 [-4.57, +4.30] | 37.87 [-3.83, +3.67] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 60.73 [-1.60, +1.60] | 53.07 [-4.46, +4.30] | 80.23 [-6.66, +5.96] | 49.90 [-4.86, +4.81] | 74.47 [-4.87, +4.65] | 54.69 [-0.65, +0.72] | 59.90 [-3.35, +3.45] | 68.43 [-4.78, +4.47] | 45.18 [-4.55, +4.55] |
Gemma 4 (E2B) 5B | 60.68 [-1.60, +1.60] | 53.49 [-4.34, +4.21] | 78.84 [-6.46, +6.04] | 51.62 [-5.21, +5.17] | 74.64 [-4.87, +4.60] | 54.07 [-0.63, +0.67] | 59.36 [-3.52, +3.42] | 69.72 [-4.59, +4.28] | 43.65 [-4.58, +4.50] |
Apertus 1.5 70B Swiss AI | 59.00 [-1.52, +1.52] | 58.43 [-3.92, +3.77] | 70.01 [-5.61, +5.11] | 55.98 [-4.82, +4.77] | 61.32 [-5.33, +5.01] | 51.27 [-0.49, +0.51] | 62.76 [-3.45, +3.27] | 63.11 [-4.85, +4.31] | 49.14 [-4.80, +4.98] |
Nemotron 3.5 Lightning 30B MoE NVIDIA | 58.69 [-1.57, +1.58] | 64.29 [-3.78, +3.44] | 50.87 [-7.18, +7.23] | 66.63 [-4.72, +5.03] | 77.62 [-4.50, +4.23] | 44.92 [-0.56, +0.61] | 60.36 [-3.60, +3.52] | 68.64 [-4.48, +4.09] | 36.16 [-3.83, +3.82] |
MERaLiON 2 10B A*STAR | 54.24 [-1.69, +1.66] | 52.41 [-3.72, +3.99] | 56.31 [-7.50, +7.51] | 49.59 [-6.22, +5.95] | 44.16 [-4.49, +4.54] | 51.11 [-0.59, +0.59] | 64.32 [-3.70, +3.39] | 66.76 [-5.69, +5.27] | 49.24 [-4.34, +4.52] |
Olmo 3.1 32B AI2 | 48.91 [-1.92, +1.87] | 40.94 [-4.39, +4.11] | 63.96 [-8.01, +8.19] | 40.76 [-6.26, +5.78] | 66.22 [-6.29, +6.02] | 49.62 [-0.65, +0.67] | 40.64 [-4.43, +4.44] | 60.34 [-6.16, +5.76] | 28.77 [-4.69, +4.59] |
Tiny Aya Water 3B CohereLabs | 24.05 [-1.30, +1.19] | 11.98 [-2.35, +2.52] | 67.03 [-6.31, +5.71] | 22.71 [-4.25, +4.67] | 32.37 [-4.43, +4.46] | 36.14 [-0.55, +0.56] | 2.26 [-2.19, +2.08] | 6.52 [-3.10, +3.09] | 13.37 [-2.29, +2.37] |
Legend:
Reasoning model
·
Model Size: ≤200B
·
Open models only