SEA-HELM Leaderboard

English Performance

English Scores by Model

Average of 2000 bootstraps. 95% CI are shown.

Legend:

Reasoning model

Model Size: ≤200B

Open models only

English Competency & Task Scores

Average of 2000 bootstraps. 95% CI are shown.

Model	EN	Code Generation	Inst. Following	Knowledge	Math	Reasoning	Long Context
Muse Glimmer 30B Meta	70.73 [-1.48, +1.51]	58.48 [-2.75, +2.89]	94.49 [-1.56, +1.44]	61.78 [-0.64, +0.62]	51.92 [-3.46, +3.58]	76.31 [-4.00, +3.89]	81.42 [-6.67, +6.21]
Qwen 3.5 122B MoE Alibaba	69.77 [-1.67, +1.63]	63.90 [-2.63, +2.59]	87.86 [-2.14, +2.07]	77.79 [-0.68, +0.67]	43.50 [-3.41, +3.54]	76.61 [-3.74, +3.90]	68.97 [-7.97, +7.78]
SEA-LION v4.8 (Nemotron) 120B MoE AISG	66.19 [-1.61, +1.64]	64.77 [-2.60, +2.50]	85.63 [-2.16, +2.06]	69.26 [-0.69, +0.64]	39.00 [-3.72, +3.59]	76.42 [-3.75, +3.69]	62.05 [-7.77, +7.09]
SEA-LION v4.5 (Qwen) 27B AISG	66.04 [-1.55, +1.55]	57.98 [-2.65, +2.54]	88.75 [-2.34, +2.21]	72.98 [-0.66, +0.70]	26.34 [-3.09, +3.08]	80.56 [-3.46, +3.32]	69.65 [-7.27, +7.10]
Nemotron 3 Super 120B MoE NVIDIA	65.15 [-1.70, +1.66]	66.89 [-2.60, +2.56]	88.79 [-2.08, +1.94]	68.23 [-0.71, +0.68]	37.63 [-3.59, +3.41]	78.36 [-3.71, +3.55]	50.99 [-7.87, +8.13]
Qwen 3.6 27B Alibaba	64.48 [-1.48, +1.55]	53.70 [-2.72, +2.67]	90.27 [-2.29, +2.13]	74.90 [-0.64, +0.65]	17.91 [-2.77, +2.83]	81.06 [-3.34, +3.45]	69.05 [-7.30, +7.08]
Mistral Medium 3.5 128B Mistral AI	63.38 [-1.55, +1.60]	53.24 [-2.46, +2.55]	91.25 [-1.99, +1.89]	62.69 [-0.70, +0.73]	30.41 [-3.36, +3.57]	77.26 [-3.78, +3.73]	65.42 [-6.92, +7.33]
Gemma 4 31B Google	63.06 [-1.07, +1.06]	80.09 [-2.38, +2.27]	94.92 [-1.62, +1.52]	67.66 [-0.67, +0.65]	56.21 [-3.65, +3.63]	78.48 [-4.12, +4.00]	0.98 [-0.98, +2.02]
GPT OSS 120B MoE mxfp4 OpenAI	62.19 [-1.72, +1.70]	68.18 [-2.48, +2.35]	85.52 [-2.25, +2.32]	51.62 [-0.71, +0.71]	45.48 [-3.45, +3.49]	77.64 [-4.11, +3.71]	44.66 [-8.16, +8.09]
SEA-LION v4.8 (Nemotron) 30B MoE AISG	59.62 [-1.56, +1.55]	60.99 [-2.71, +2.70]	88.03 [-1.94, +1.82]	51.56 [-0.54, +0.55]	34.50 [-3.61, +3.47]	68.25 [-3.81, +3.73]	54.40 [-7.03, +6.97]
Nemotron 3.5 Lightning 30B MoE NVIDIA	57.73 [-1.60, +1.59]	57.45 [-2.78, +2.70]	91.10 [-1.87, +1.71]	44.28 [-0.44, +0.42]	30.69 [-3.26, +3.52]	65.61 [-3.72, +3.72]	57.27 [-7.52, +7.23]
GLM 4.7 Flash 30B MoE Z.ai	57.36 [-1.53, +1.56]	54.31 [-2.68, +2.74]	84.69 [-2.23, +2.11]	61.71 [-0.70, +0.70]	32.38 [-3.37, +3.24]	67.80 [-3.64, +3.58]	43.26 [-6.76, +6.74]
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG	46.49 [-1.32, +1.33]	50.81 [-2.72, +2.62]	84.45 [-2.59, +2.50]	42.75 [-0.73, +0.74]	19.40 [-2.68, +2.69]	64.00 [-3.27, +3.42]	17.51 [-5.26, +5.99]
Gemma 4 (E2B) 5B Google	45.26 [-1.30, +1.29]	53.13 [-2.77, +2.68]	83.58 [-2.78, +2.49]	43.65 [-0.77, +0.75]	21.39 [-2.95, +3.09]	60.51 [-3.57, +3.73]	9.30 [-4.43, +5.08]
Olmo 3.1 32B AI2	41.09 [-1.01, +0.97]	40.20 [-2.63, +2.71]	86.75 [-2.67, +2.39]	51.81 [-0.73, +0.71]	8.87 [-2.04, +2.21]	57.95 [-3.97, +4.03]	0.98 [-0.98, +2.02]
Apertus 1.5 70B Swiss AI	40.55 [-1.21, +1.23]	19.56 [-2.13, +2.11]	85.00 [-2.36, +2.36]	46.00 [-0.73, +0.73]	16.97 [-2.74, +2.74]	58.52 [-3.86, +3.61]	17.27 [-4.40, +4.73]
MERaLiON 2 10B A*STAR	23.14 [-0.91, +0.87]	6.32 [-1.24, +1.33]	62.37 [-3.20, +3.41]	39.16 [-0.79, +0.76]	1.58 [-0.62, +0.72]	28.54 [-3.78, +3.54]	0.85 [-0.85, +1.77]
Tiny Aya Water 3B CohereLabs	16.50 [-0.72, +0.74]	3.46 [-0.77, +0.89]	63.77 [-2.82, +3.02]	22.57 [-0.67, +0.68]	1.00 [-0.39, +0.44]	7.21 [-2.27, +2.44]	0.98 [-0.98, +2.02]

No records

Legend:

Reasoning model

Model Size: ≤200B

Open models only