Elo Performance
Overall Elo Ratings
Average of 2000 bootstraps. 95% CI are shown.
Legend:
Reasoning model
No open weights
·
Model Size: All
·
Open/closed models
Language Elo Ratings by Model
Average of 2000 bootstraps. 95% CI are shown.
Model | SEA | MY | TL | ID | MS | TA | TH | VI |
|---|---|---|---|---|---|---|---|---|
GPT 6 Astra OpenAI | 1238.0 [-9.11, +9.12] | 1361.5 [-24.05, +23.40] | 1230.7 [-23.06, +23.28] | 1212.8 [-22.30, +22.82] | 1224.6 [-24.13, +24.19] | 1260.8 [-24.40, +26.18] | 1210.8 [-17.99, +19.10] | 1218.9 [-22.51, +23.66] |
Gemini 4 Hydrogen | 1234.1 [-8.88, +8.96] | 1382.1 [-22.58, +23.02] | 1220.1 [-22.76, +22.49] | 1208.6 [-21.09, +23.04] | 1222.4 [-25.61, +25.22] | 1246.4 [-23.92, +25.37] | 1209.0 [-17.56, +17.34] | 1214.0 [-20.58, +21.40] |
Gemma 4 31B | 1178.2 [-7.92, +7.90] | 1252.4 [-20.93, +22.90] | 1189.9 [-22.27, +22.22] | 1162.0 [-18.89, +18.83] | 1175.3 [-19.40, +20.01] | 1189.0 [-19.16, +20.35] | 1158.6 [-17.73, +17.81] | 1159.9 [-20.42, +20.55] |
SEA-LION v5 (Qwen) 397B MoE AISG | 1175.5 [-8.11, +7.83] | 1260.0 [-19.26, +18.12] | 1155.6 [-19.91, +20.99] | 1161.9 [-18.49, +20.23] | 1152.3 [-20.47, +20.67] | 1205.4 [-20.08, +21.68] | 1167.9 [-18.43, +18.14] | 1170.6 [-18.49, +19.76] |
Opus 4.8 Anthropic | 1171.5 [-7.83, +8.22] | 1222.0 [-19.11, +18.67] | 1139.9 [-19.03, +20.24] | 1178.8 [-19.64, +21.52] | 1148.7 [-21.46, +23.05] | 1187.5 [-20.27, +23.37] | 1165.6 [-17.80, +18.46] | 1196.0 [-21.13, +21.95] |
Qwen 3.5 397B MoE Alibaba | 1161.5 [-7.82, +7.73] | 1238.5 [-19.55, +19.55] | 1145.1 [-18.42, +19.22] | 1147.8 [-17.42, +18.51] | 1148.8 [-20.58, +21.58] | 1195.9 [-19.51, +20.24] | 1133.4 [-15.61, +16.26] | 1163.7 [-20.88, +21.54] |
Inkling 975B MoE NVFP4 Thinking Machines | 1128.0 [-6.80, +6.87] | 1149.6 [-14.37, +14.48] | 1123.0 [-17.08, +17.07] | 1112.6 [-16.55, +16.10] | 1133.4 [-18.31, +18.96] | 1178.0 [-17.89, +18.86] | 1108.7 [-16.15, +16.78] | 1123.0 [-16.86, +18.01] |
DeepSeek V4 Pro 1600B MoE DeepSeek | 1122.0 [-6.37, +6.72] | 1208.4 [-16.27, +17.39] | 1137.6 [-16.58, +18.60] | 1098.8 [-15.04, +15.85] | 1121.1 [-18.31, +18.10] | 1136.8 [-15.04, +15.65] | 1087.7 [-13.51, +14.17] | 1104.7 [-15.58, +16.63] |
GLM 5.3 743B MoE Z.ai | 1105.7 [-6.76, +6.70] | 1151.8 [-15.23, +15.46] | 1096.7 [-16.41, +17.23] | 1094.6 [-16.79, +17.14] | 1137.3 [-19.56, +18.87] | 1123.7 [-16.14, +16.13] | 1070.2 [-15.19, +16.35] | 1098.5 [-16.83, +16.65] |
SEA-LION v5 (Qwen) 27B AISG | 1099.6 [-6.76, +6.66] | 1152.2 [-15.74, +16.43] | 1046.9 [-16.81, +16.58] | 1103.7 [-18.19, +19.46] | 1062.3 [-18.75, +20.62] | 1131.1 [-17.92, +17.90] | 1134.1 [-15.98, +16.27] | 1100.6 [-16.65, +18.85] |
Muse Glimmer 30B Meta | 1084.8 [-6.89, +6.78] | 1153.1 [-17.36, +18.66] | 1085.5 [-18.44, +19.98] | 1053.9 [-16.34, +16.24] | 1081.7 [-20.17, +20.94] | 1121.7 [-18.20, +19.13] | 1067.2 [-15.69, +15.67] | 1066.1 [-17.10, +18.48] |
SEA-LION v5 (Nemotron) 4B AISG | 1070.7 [-6.54, +6.85] | 1118.4 [-17.49, +17.47] | 1096.8 [-17.34, +16.73] | 1050.0 [-16.75, +16.90] | 1043.9 [-17.67, +17.30] | 1100.6 [-17.38, +18.07] | 1070.1 [-16.25, +16.06] | 1043.4 [-15.79, +15.95] |
MiniMax M3 428B MoE Minimax | 1054.2 [-5.85, +5.92] | 1102.0 [-14.08, +14.62] | 1043.9 [-14.48, +14.62] | 1038.5 [-13.41, +14.01] | 1037.9 [-14.47, +15.55] | 1098.8 [-16.21, +16.79] | 1050.8 [-13.74, +14.60] | 1035.4 [-14.92, +14.65] |
Qwen 3.8 27B Alibaba | 1053.0 [-6.47, +6.56] | 1075.5 [-15.65, +15.27] | 1021.2 [-15.37, +15.96] | 1053.2 [-15.62, +15.66] | 1040.4 [-16.84, +18.30] | 1057.1 [-16.12, +15.90] | 1060.9 [-15.36, +15.48] | 1083.6 [-16.46, +16.33] |
Nemotron 3 Ultra 550B MoE NVIDIA | 998.3 [-5.86, +6.16] | 875.7 [-16.83, +14.58] | 1047.4 [-15.04, +15.02] | 1042.9 [-14.57, +14.67] | 1036.9 [-15.81, +15.81] | 970.6 [-14.82, +15.02] | 1007.7 [-14.30, +14.16] | 989.6 [-14.05, +14.99] |
Mistral Medium 3.5 128B Mistral AI | 984.0 [-5.57, +5.82] | 902.4 [-11.10, +11.03] | 994.9 [-14.60, +14.98] | 1004.3 [-14.92, +15.55] | 1022.4 [-15.55, +16.41] | 983.8 [-13.69, +13.88] | 956.3 [-13.97, +13.37] | 1016.3 [-14.18, +14.20] |
SEA-LION v5 (Qwen) 9B AISG | 976.6 [-5.65, +5.73] | 1040.5 [-15.29, +15.93] | 881.7 [-12.83, +13.36] | 967.9 [-14.01, +14.29] | 952.6 [-14.49, +14.85] | 1020.8 [-15.23, +15.66] | 997.1 [-12.76, +14.42] | 992.6 [-14.39, +13.95] |
Nemotron 3.5 Bolt (E4B) 4B NVIDIA | 973.4 [-6.29, +6.58] | 1032.7 [-17.78, +16.52] | 984.2 [-15.81, +16.44] | 945.3 [-15.64, +16.76] | 949.3 [-17.73, +18.87] | 999.0 [-16.86, +17.57] | 981.5 [-15.12, +15.06] | 939.0 [-16.24, +16.42] |
GPT OSS 120B MoE mxfp4 OpenAI | 971.4 [-6.28, +6.03] | 951.7 [-15.62, +14.92] | 1001.1 [-15.28, +15.09] | 972.6 [-14.03, +14.98] | 987.9 [-15.65, +16.44] | 965.2 [-15.66, +14.52] | 962.5 [-13.80, +13.55] | 961.8 [-15.24, +15.48] |
SEA-LION v5 (Gemma E2B) 2B AISG | 963.7 [-6.23, +6.11] | 1008.3 [-16.97, +16.59] | 967.6 [-14.29, +14.90] | 946.4 [-14.48, +14.83] | 929.2 [-17.10, +17.68] | 1005.5 [-16.64, +16.77] | 960.6 [-14.83, +15.12] | 944.6 [-14.48, +15.84] |
Command A+ 218B MoE CohereLabs | 883.0 [-6.03, +6.11] | 750.4 [-18.88, +18.92] | 859.0 [-14.00, +14.56] | 897.1 [-13.54, +13.69] | 904.5 [-15.81, +17.10] | 929.9 [-14.23, +13.94] | 896.7 [-13.91, +12.77] | 907.1 [-15.10, +14.90] |
Gemma 4 (E2B) 2B | 871.4 [-6.13, +6.54] | 893.4 [-16.62, +17.05] | 887.5 [-15.07, +15.38] | 850.1 [-15.85, +16.60] | 853.1 [-18.66, +18.62] | 902.9 [-17.73, +17.40] | 865.8 [-15.09, +15.91] | 850.5 [-15.44, +15.78] |
Apertus 1.5 70B Swiss AI | 850.4 [-6.06, +6.19] | 776.4 [-20.38, +18.49] | 828.6 [-13.95, +14.08] | 858.1 [-14.45, +14.57] | 880.9 [-14.90, +14.78] | 857.9 [-16.42, +16.34] | 855.0 [-12.91, +13.38] | 869.7 [-13.83, +13.86] |
Olmo 3.1 32B AI2 | 776.8 [-7.40, +6.94] | 601.3 [-20.26, +20.99] | 830.3 [-16.90, +16.79] | 809.3 [-15.57, +16.44] | 815.2 [-17.52, +17.97] | 720.8 [-21.92, +21.64] | 749.0 [-16.36, +16.34] | 828.9 [-19.05, +18.00] |
Legend:
Reasoning model
No open weights
·
Model Size: All
·
Open/closed models