Elo Performance
Overall Elo Ratings
Average of 2000 bootstraps. 95% CI are shown.
Gemini 3.1 Pro Preview
Gemma 4
Opus 4.8
Qwen 3.5
Qwen 3.6
DeepSeek V4 Pro
GLM 5.2
Muse Glimmer
SEA-LION v4.5 (Qwen)
Inkling Small
MiniMax M3
Nemotron 3 Ultra
GPT OSS
SEA-LION v4.8 (Nemotron)
Mistral Medium 3.5
Gemma 4 (E2B)
SEA-LION v4.5 (Gemma E2B)
Command A+
SEA-LION v4.8 (Nemotron)
Nemotron 3 Super
Nemotron 3.5 Lightning
MERaLiON 2
Olmo 3.1Legend:
Reasoning model
No open weights
·
Model Size: All
·
Open/closed models
Language Elo Ratings by Model
Average of 2000 bootstraps. 95% CI are shown.
Model | SEA | MY | TL | ID | MS | TA | TH | VI |
|---|---|---|---|---|---|---|---|---|
Gemini 3.1 Pro Preview | 1214.7 [-8.90, +8.69] | 1337.4 [-24.52, +26.24] | 1247.0 [-21.54, +21.97] | 1189.3 [-20.90, +22.04] | 1178.6 [-23.33, +23.42] | 1235.1 [-22.52, +22.66] | 1175.4 [-18.44, +19.19] | 1196.4 [-21.31, +22.03] |
Gemma 4 31B | 1206.0 [-8.11, +7.77] | 1288.6 [-23.26, +23.46] | 1226.5 [-20.56, +22.07] | 1187.9 [-19.22, +18.33] | 1192.4 [-20.39, +20.20] | 1223.3 [-20.55, +20.92] | 1177.8 [-16.92, +18.48] | 1188.5 [-21.53, +20.89] |
Opus 4.8 Anthropic | 1188.6 [-7.97, +8.51] | 1242.3 [-21.07, +22.00] | 1177.1 [-20.04, +21.16] | 1197.0 [-19.02, +20.50] | 1152.0 [-21.83, +22.00] | 1200.6 [-21.24, +21.67] | 1176.2 [-17.77, +18.29] | 1212.4 [-22.14, +23.46] |
Qwen 3.5 397B MoE Alibaba | 1183.6 [-7.45, +7.44] | 1271.0 [-19.66, +19.94] | 1177.3 [-18.22, +19.26] | 1166.6 [-17.73, +17.78] | 1163.1 [-20.65, +21.43] | 1219.6 [-21.67, +21.59] | 1147.5 [-16.30, +17.38] | 1183.3 [-19.37, +20.32] |
Qwen 3.6 27B Alibaba | 1138.8 [-6.65, +6.67] | 1208.6 [-16.51, +17.11] | 1113.7 [-15.71, +15.66] | 1139.3 [-17.16, +16.78] | 1153.0 [-18.97, +19.90] | 1156.8 [-15.86, +16.41] | 1117.7 [-14.38, +14.57] | 1118.0 [-17.03, +18.23] |
DeepSeek V4 Pro 1600B MoE DeepSeek | 1120.1 [-6.26, +5.98] | 1199.0 [-16.09, +16.05] | 1165.5 [-17.48, +17.77] | 1096.6 [-13.82, +14.98] | 1114.2 [-17.96, +18.82] | 1134.2 [-14.60, +15.50] | 1076.4 [-12.64, +13.65] | 1095.1 [-15.47, +16.10] |
GLM 5.2 754B MoE FP8 Z.ai | 1105.3 [-6.36, +6.94] | 1195.2 [-16.57, +16.19] | 1133.8 [-16.55, +17.97] | 1071.0 [-15.65, +15.23] | 1073.7 [-18.32, +19.66] | 1114.6 [-15.72, +16.07] | 1087.5 [-15.15, +15.43] | 1099.8 [-16.58, +16.43] |
Muse Glimmer 30B Meta | 1102.2 [-7.10, +7.36] | 1184.9 [-18.63, +19.06] | 1115.4 [-17.90, +18.37] | 1069.6 [-15.72, +16.38] | 1099.2 [-19.20, +20.64] | 1137.2 [-18.47, +18.37] | 1077.5 [-15.15, +15.01] | 1069.0 [-16.57, +16.49] |
SEA-LION v4.5 (Qwen) 27B AISG | 1098.9 [-6.48, +6.30] | 1182.8 [-17.15, +17.30] | 1108.9 [-16.15, +16.36] | 1104.7 [-15.58, +16.59] | 1116.5 [-18.89, +20.19] | 1061.6 [-15.00, +15.57] | 1106.9 [-13.72, +14.76] | 1043.7 [-16.19, +15.37] |
Inkling Small 276B MoE Thinking Machines | 1097.0 [-6.11, +6.26] | 1151.5 [-15.34, +16.43] | 1065.1 [-14.01, +14.17] | 1077.4 [-14.72, +14.57] | 1106.9 [-17.98, +17.93] | 1137.2 [-16.86, +17.74] | 1093.8 [-14.13, +14.40] | 1078.1 [-15.38, +16.18] |
MiniMax M3 428B MoE Minimax | 1083.2 [-5.84, +6.26] | 1148.1 [-14.91, +15.82] | 1083.4 [-14.83, +14.83] | 1068.9 [-13.71, +14.32] | 1057.1 [-14.89, +14.82] | 1120.5 [-15.58, +15.95] | 1077.3 [-14.34, +13.78] | 1057.7 [-14.49, +13.99] |
Nemotron 3 Ultra 550B MoE NVIDIA | 1030.3 [-5.72, +5.59] | 941.1 [-13.27, +13.26] | 1077.6 [-15.52, +14.72] | 1072.3 [-14.48, +14.98] | 1065.4 [-15.72, +16.62] | 1005.2 [-14.31, +14.91] | 1038.2 [-13.66, +13.72] | 1010.8 [-14.09, +14.85] |
GPT OSS 120B MoE mxfp4 OpenAI | 1019.6 [-5.84, +6.21] | 1014.7 [-13.13, +12.70] | 1043.9 [-15.69, +15.45] | 1020.4 [-13.83, +14.41] | 1024.5 [-16.75, +16.92] | 1022.2 [-13.80, +13.97] | 1014.2 [-13.51, +13.52] | 1009.2 [-15.08, +15.35] |
SEA-LION v4.8 (Nemotron) 120B MoE AISG | 1011.3 [-5.56, +5.53] | 909.4 [-13.49, +13.44] | 1039.8 [-13.95, +13.80] | 1034.5 [-13.30, +13.79] | 1039.1 [-15.23, +16.02] | 970.7 [-13.77, +13.57] | 1063.3 [-13.58, +12.79] | 1014.3 [-13.82, +14.68] |
Mistral Medium 3.5 128B Mistral AI | 1006.1 [-5.71, +5.80] | 954.7 [-11.92, +11.37] | 1035.0 [-14.46, +14.61] | 1014.5 [-14.53, +15.30] | 1015.1 [-14.87, +16.07] | 1025.8 [-13.91, +13.64] | 980.1 [-13.47, +13.33] | 1022.4 [-13.52, +14.35] |
Gemma 4 (E2B) 5B | 949.9 [-5.98, +6.11] | 1005.1 [-13.99, +13.50] | 942.9 [-14.29, +14.09] | 928.8 [-15.29, +15.01] | 934.4 [-16.50, +16.32] | 984.1 [-14.88, +15.59] | 936.4 [-13.75, +13.36] | 930.6 [-14.30, +15.48] |
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG | 949.8 [-5.78, +5.98] | 1009.1 [-13.99, +13.52] | 944.8 [-14.42, +14.85] | 918.6 [-13.18, +13.72] | 933.7 [-16.41, +15.97] | 979.0 [-13.92, +14.17] | 938.6 [-12.83, +13.22] | 937.7 [-15.64, +15.29] |
Command A+ 218B MoE CohereLabs | 927.3 [-5.30, +5.56] | 848.2 [-12.31, +12.62] | 899.5 [-13.69, +13.87] | 932.1 [-12.39, +12.85] | 937.2 [-15.49, +15.86] | 972.1 [-13.40, +13.32] | 940.4 [-12.26, +12.48] | 949.0 [-14.04, +13.90] |
SEA-LION v4.8 (Nemotron) 30B MoE AISG | 897.4 [-5.29, +5.23] | 766.4 [-13.87, +14.02] | 911.7 [-13.41, +12.78] | 921.6 [-13.08, +11.68] | 904.1 [-13.94, +13.27] | 821.5 [-14.07, +13.63] | 976.4 [-12.63, +12.44] | 933.3 [-13.19, +13.18] |
Nemotron 3 Super 120B MoE NVIDIA | 879.0 [-5.53, +5.80] | 688.0 [-14.49, +15.51] | 956.2 [-14.69, +14.58] | 965.6 [-13.02, +13.64] | 950.1 [-14.43, +15.23] | 695.3 [-15.25, +16.00] | 938.2 [-13.17, +13.56] | 849.3 [-13.53, +14.14] |
Nemotron 3.5 Lightning 30B MoE NVIDIA | 845.8 [-5.83, +5.92] | 572.9 [-16.82, +16.00] | 899.8 [-13.59, +14.23] | 916.1 [-12.34, +13.15] | 854.4 [-14.47, +14.93] | 665.1 [-17.94, +17.58] | 938.0 [-12.85, +13.39] | 913.8 [-14.94, +14.69] |
MERaLiON 2 10B A*STAR | 842.0 [-5.67, +5.57] | 734.5 [-16.39, +16.44] | 892.0 [-13.08, +13.52] | 883.3 [-12.41, +12.97] | 843.4 [-15.74, +16.04] | 787.7 [-14.11, +13.99] | 837.4 [-13.35, +13.23] | 873.3 [-13.42, +13.77] |
Olmo 3.1 32B AI2 | 839.4 [-6.35, +5.94] | 713.8 [-19.13, +17.67] | 872.2 [-15.25, +15.58] | 875.3 [-14.33, +14.64] | 886.5 [-15.19, +16.04] | 761.6 [-17.02, +16.48] | 828.0 [-15.77, +16.00] | 883.5 [-16.40, +16.60] |
Legend:
Reasoning model
No open weights
·
Model Size: All
·
Open/closed models