Elo Performance
Overall Elo Ratings
Average of 2000 bootstraps. 95% CI are shown.
Gemma 4
Qwen 3.5
Qwen 3.6
Muse Glimmer
SEA-LION v4.5 (Qwen)
GPT OSS
SEA-LION v4.8 (Nemotron)
Mistral Medium 3.5
Gemma 4 (E2B)
SEA-LION v4.5 (Gemma E2B)
SEA-LION v4.8 (Nemotron)
GLM 4.7 Flash
Nemotron 3 Super
Nemotron 3.5 Lightning
MERaLiON 2
Olmo 3.1
Tiny Aya WaterLegend:
Reasoning model
·
Model Size: ≤200B
·
Open models only
Language Elo Ratings by Model
Average of 2000 bootstraps. 95% CI are shown.
Model | SEA | MY | TL | ID | MS | TA | TH | VI |
|---|---|---|---|---|---|---|---|---|
Gemma 4 31B | 1206.0 [-8.11, +7.77] | 1288.6 [-23.26, +23.46] | 1226.5 [-20.56, +22.07] | 1187.9 [-19.22, +18.33] | 1192.4 [-20.39, +20.20] | 1223.3 [-20.55, +20.92] | 1177.8 [-16.92, +18.48] | 1188.5 [-21.53, +20.89] |
Qwen 3.5 122B MoE Alibaba | 1149.8 [-6.67, +7.03] | 1227.0 [-18.88, +18.55] | 1134.8 [-16.18, +17.01] | 1122.6 [-15.21, +16.08] | 1141.5 [-18.51, +19.50] | 1187.6 [-18.29, +19.33] | 1117.6 [-15.84, +15.65] | 1156.8 [-18.47, +19.52] |
Qwen 3.6 27B Alibaba | 1138.8 [-6.65, +6.67] | 1208.6 [-16.51, +17.11] | 1113.7 [-15.71, +15.66] | 1139.3 [-17.16, +16.78] | 1153.0 [-18.97, +19.90] | 1156.8 [-15.86, +16.41] | 1117.7 [-14.38, +14.57] | 1118.0 [-17.03, +18.23] |
Muse Glimmer 30B Meta | 1102.2 [-7.10, +7.36] | 1184.9 [-18.63, +19.06] | 1115.4 [-17.90, +18.37] | 1069.6 [-15.72, +16.38] | 1099.2 [-19.20, +20.64] | 1137.2 [-18.47, +18.37] | 1077.5 [-15.15, +15.01] | 1069.0 [-16.57, +16.49] |
SEA-LION v4.5 (Qwen) 27B AISG | 1098.9 [-6.48, +6.30] | 1182.8 [-17.15, +17.30] | 1108.9 [-16.15, +16.36] | 1104.7 [-15.58, +16.59] | 1116.5 [-18.89, +20.19] | 1061.6 [-15.00, +15.57] | 1106.9 [-13.72, +14.76] | 1043.7 [-16.19, +15.37] |
GPT OSS 120B MoE mxfp4 OpenAI | 1019.6 [-5.84, +6.21] | 1014.7 [-13.13, +12.70] | 1043.9 [-15.69, +15.45] | 1020.4 [-13.83, +14.41] | 1024.5 [-16.75, +16.92] | 1022.2 [-13.80, +13.97] | 1014.2 [-13.51, +13.52] | 1009.2 [-15.08, +15.35] |
SEA-LION v4.8 (Nemotron) 120B MoE AISG | 1011.3 [-5.56, +5.53] | 909.4 [-13.49, +13.44] | 1039.8 [-13.95, +13.80] | 1034.5 [-13.30, +13.79] | 1039.1 [-15.23, +16.02] | 970.7 [-13.77, +13.57] | 1063.3 [-13.58, +12.79] | 1014.3 [-13.82, +14.68] |
Mistral Medium 3.5 128B Mistral AI | 1006.1 [-5.71, +5.80] | 954.7 [-11.92, +11.37] | 1035.0 [-14.46, +14.61] | 1014.5 [-14.53, +15.30] | 1015.1 [-14.87, +16.07] | 1025.8 [-13.91, +13.64] | 980.1 [-13.47, +13.33] | 1022.4 [-13.52, +14.35] |
Gemma 4 (E2B) 5B | 949.9 [-5.98, +6.11] | 1005.1 [-13.99, +13.50] | 942.9 [-14.29, +14.09] | 928.8 [-15.29, +15.01] | 934.4 [-16.50, +16.32] | 984.1 [-14.88, +15.59] | 936.4 [-13.75, +13.36] | 930.6 [-14.30, +15.48] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 949.8 [-5.78, +5.98] | 1009.1 [-13.99, +13.52] | 944.8 [-14.42, +14.85] | 918.6 [-13.18, +13.72] | 933.7 [-16.41, +15.97] | 979.0 [-13.92, +14.17] | 938.6 [-12.83, +13.22] | 937.7 [-15.64, +15.29] |
SEA-LION v4.8 (Nemotron) 30B MoE AISG | 897.4 [-5.29, +5.23] | 766.4 [-13.87, +14.02] | 911.7 [-13.41, +12.78] | 921.6 [-13.08, +11.68] | 904.1 [-13.94, +13.27] | 821.5 [-14.07, +13.63] | 976.4 [-12.63, +12.44] | 933.3 [-13.19, +13.18] |
GLM 4.7 Flash 30B MoE Z.ai | 895.6 [-4.94, +5.23] | 842.1 [-12.29, +11.97] | 923.1 [-12.27, +12.19] | 922.5 [-11.74, +11.62] | 908.9 [-13.80, +12.10] | 844.6 [-11.47, +11.31] | 899.3 [-11.42, +11.56] | 909.3 [-12.36, +12.83] |
Nemotron 3 Super 120B MoE NVIDIA | 879.0 [-5.53, +5.80] | 688.0 [-14.49, +15.51] | 956.2 [-14.69, +14.58] | 965.6 [-13.02, +13.64] | 950.1 [-14.43, +15.23] | 695.3 [-15.25, +16.00] | 938.2 [-13.17, +13.56] | 849.3 [-13.53, +14.14] |
Nemotron 3.5 Lightning 30B MoE NVIDIA | 845.8 [-5.83, +5.92] | 572.9 [-16.82, +16.00] | 899.8 [-13.59, +14.23] | 916.1 [-12.34, +13.15] | 854.4 [-14.47, +14.93] | 665.1 [-17.94, +17.58] | 938.0 [-12.85, +13.39] | 913.8 [-14.94, +14.69] |
MERaLiON 2 10B A*STAR | 842.0 [-5.67, +5.57] | 734.5 [-16.39, +16.44] | 892.0 [-13.08, +13.52] | 883.3 [-12.41, +12.97] | 843.4 [-15.74, +16.04] | 787.7 [-14.11, +13.99] | 837.4 [-13.35, +13.23] | 873.3 [-13.42, +13.77] |
Olmo 3.1 32B AI2 | 839.4 [-6.35, +5.94] | 713.8 [-19.13, +17.67] | 872.2 [-15.25, +15.58] | 875.3 [-14.33, +14.64] | 886.5 [-15.19, +16.04] | 761.6 [-17.02, +16.48] | 828.0 [-15.77, +16.00] | 883.5 [-16.40, +16.60] |
Tiny Aya Water 3B CohereLabs | 610.6 [-7.57, +7.59] | 487.2 [-27.88, +26.99] | 665.8 [-15.32, +15.43] | 662.7 [-14.64, +15.30] | 685.6 [-16.31, +16.21] | 395.0 [-24.53, +22.82] | 560.3 [-18.60, +18.10] | 649.8 [-15.44, +15.23] |
Legend:
Reasoning model
·
Model Size: ≤200B
·
Open models only