SEA-HELM Leaderboard

SEA Performance

Overall SEA Average

Average of 2000 bootstraps. 95% CI are shown.

Legend:

Reasoning model

Model Size: ≤200B

Open models only

Language Performance by Model

Average of 2000 bootstraps. 95% CI are shown.

Model	SEA	MY	TL	ID	MS	TA	TH	VI
Gemma 4 31B Google	76.17 [-0.60, +0.59]	70.97 [-2.06, +1.97]	78.44 [-1.30, +1.27]	79.75 [-1.25, +1.20]	76.27 [-1.70, +1.77]	76.40 [-1.78, +1.63]	74.30 [-1.23, +1.22]	77.09 [-1.73, +1.64]
Qwen 3.5 122B MoE Alibaba	74.66 [-0.64, +0.61]	69.16 [-1.88, +1.89]	75.37 [-1.31, +1.33]	77.93 [-1.29, +1.25]	76.31 [-1.72, +1.80]	74.71 [-1.94, +1.89]	72.75 [-1.39, +1.32]	76.39 [-1.59, +1.56]
Qwen 3.6 27B Alibaba	74.64 [-0.65, +0.60]	68.37 [-2.18, +2.12]	75.23 [-1.25, +1.28]	77.96 [-1.22, +1.20]	76.26 [-1.76, +1.72]	75.32 [-1.90, +1.80]	73.22 [-1.30, +1.30]	76.12 [-1.46, +1.39]
SEA-LION v4.5 (Qwen) 27B AISG	73.24 [-0.56, +0.57]	67.57 [-1.90, +1.88]	75.16 [-1.36, +1.36]	76.51 [-1.23, +1.23]	75.23 [-1.73, +1.67]	71.38 [-1.78, +1.58]	73.00 [-1.37, +1.28]	73.85 [-1.40, +1.40]
Muse Glimmer 30B Meta	72.13 [-0.59, +0.56]	65.95 [-2.02, +2.01]	74.28 [-1.32, +1.30]	75.61 [-1.13, +1.10]	74.14 [-1.83, +1.79]	72.03 [-1.72, +1.63]	69.38 [-1.31, +1.33]	73.52 [-1.47, +1.38]
GPT OSS 120B MoE mxfp4 OpenAI	66.14 [-0.64, +0.66]	46.90 [-1.92, +2.03]	71.38 [-1.43, +1.46]	72.28 [-1.35, +1.33]	69.68 [-1.82, +1.98]	66.05 [-1.98, +1.97]	66.88 [-1.46, +1.39]	69.81 [-1.69, +1.59]
Mistral Medium 3.5 128B Mistral AI	64.71 [-0.59, +0.59]	41.83 [-1.83, +1.81]	70.12 [-1.41, +1.48]	70.09 [-1.18, +1.13]	68.93 [-1.70, +1.66]	68.80 [-1.66, +1.61]	64.35 [-1.46, +1.42]	68.82 [-1.47, +1.43]
SEA-LION v4.8 (Nemotron) 120B MoE AISG	63.44 [-0.63, +0.61]	31.35 [-1.80, +1.80]	71.01 [-1.47, +1.39]	73.10 [-1.16, +1.21]	73.25 [-1.71, +1.74]	56.35 [-1.88, +1.87]	68.66 [-1.37, +1.39]	70.36 [-1.65, +1.67]
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG	59.23 [-0.66, +0.67]	47.68 [-2.22, +2.09]	60.73 [-1.60, +1.60]	63.96 [-1.38, +1.37]	62.28 [-1.83, +1.81]	59.06 [-1.72, +1.73]	58.94 [-1.62, +1.58]	61.95 [-1.75, +1.78]
Gemma 4 (E2B) 5B Google	58.82 [-0.64, +0.66]	46.95 [-2.07, +2.06]	60.68 [-1.60, +1.60]	64.13 [-1.39, +1.41]	61.46 [-1.88, +1.80]	59.03 [-1.83, +1.83]	58.32 [-1.55, +1.58]	61.19 [-1.66, +1.69]
Apertus 1.5 70B Swiss AI	55.20 [-0.66, +0.61]	19.21 [-1.22, +1.25]	59.00 [-1.52, +1.52]	64.69 [-1.37, +1.36]	65.54 [-1.90, +1.72]	54.49 [-1.96, +2.09]	58.81 [-1.59, +1.57]	64.66 [-1.62, +1.63]
GLM 4.7 Flash 30B MoE Z.ai	53.76 [-0.57, +0.57]	17.90 [-1.25, +1.21]	62.97 [-1.47, +1.44]	67.89 [-1.32, +1.27]	65.50 [-1.76, +1.79]	38.42 [-1.46, +1.52]	59.29 [-1.52, +1.54]	64.36 [-1.53, +1.53]
SEA-LION v4.8 (Nemotron) 30B MoE AISG	51.57 [-0.52, +0.52]	10.61 [-1.02, +0.98]	61.82 [-1.52, +1.35]	65.87 [-1.29, +1.33]	62.09 [-1.70, +1.63]	33.14 [-1.16, +1.17]	62.60 [-1.40, +1.37]	64.86 [-1.67, +1.63]
Nemotron 3 Super 120B MoE NVIDIA	49.30 [-0.51, +0.53]	4.98 [-0.67, +0.78]	66.92 [-1.39, +1.35]	70.78 [-1.22, +1.20]	68.66 [-1.71, +1.80]	21.83 [-1.25, +1.21]	56.61 [-1.29, +1.44]	55.31 [-1.48, +1.55]
Nemotron 3.5 Lightning 30B MoE NVIDIA	46.06 [-0.54, +0.54]	3.79 [-0.52, +0.57]	58.69 [-1.57, +1.58]	64.34 [-1.36, +1.37]	58.76 [-1.68, +1.75]	17.23 [-1.13, +1.21]	58.66 [-1.51, +1.60]	60.97 [-1.55, +1.52]
MERaLiON 2 10B A*STAR	44.14 [-0.63, +0.68]	10.64 [-1.37, +1.50]	54.24 [-1.69, +1.66]	58.01 [-1.66, +1.65]	55.29 [-1.90, +1.95]	34.10 [-1.54, +1.46]	44.83 [-1.72, +1.68]	51.89 [-1.81, +1.85]
Olmo 3.1 32B AI2	40.88 [-0.69, +0.64]	6.06 [-1.06, +1.13]	48.91 [-1.92, +1.87]	55.89 [-1.56, +1.62]	58.14 [-2.02, +2.01]	24.19 [-1.62, +1.61]	41.86 [-1.92, +1.80]	51.07 [-1.89, +2.01]
Tiny Aya Water 3B CohereLabs	19.75 [-0.40, +0.43]	4.84 [-0.84, +0.88]	24.05 [-1.30, +1.19]	33.23 [-1.13, +1.13]	32.71 [-1.47, +1.54]	3.24 [-0.60, +0.65]	8.04 [-0.74, +0.75]	32.15 [-1.36, +1.29]

No records

Legend:

Reasoning model

Model Size: ≤200B

Open models only