SEA-HELM Leaderboard

Burmese Performance

Burmese Scores by Model

Average of 2000 bootstraps. 95% CI are shown.

Legend:

Reasoning model

Model Size: ≤200B

Open models only

Burmese Competency & Task Scores

Average of 2000 bootstraps. 95% CI are shown.

Model	MY	Cultural	Inst. Following	Knowledge	NLG	NLR	NLU	Safety
Gemma 4 31B Google	70.97 [-2.06, +1.97]	60.17 [-9.33, +8.78]	85.32 [-6.82, +6.18]	79.22 [-4.39, +4.03]	57.95 [-0.71, +0.71]	70.40 [-3.62, +3.40]	65.41 [-4.32, +3.98]	78.31 [-5.19, +4.69]
Qwen 3.5 122B MoE Alibaba	69.16 [-1.88, +1.89]	60.19 [-8.30, +8.57]	84.67 [-6.79, +6.08]	77.96 [-4.26, +4.41]	55.15 [-0.64, +0.64]	68.11 [-3.50, +3.39]	62.40 [-4.25, +3.72]	75.67 [-3.90, +3.69]
Qwen 3.6 27B Alibaba	68.37 [-2.18, +2.12]	68.10 [-9.26, +8.94]	79.01 [-6.89, +6.36]	74.96 [-4.25, +4.08]	53.69 [-0.62, +0.62]	68.32 [-3.48, +3.52]	64.02 [-4.00, +3.50]	70.50 [-8.53, +6.49]
SEA-LION v4.5 (Qwen) 27B AISG	67.57 [-1.90, +1.88]	60.83 [-8.23, +8.44]	80.89 [-6.89, +6.36]	74.50 [-4.46, +4.46]	54.43 [-0.63, +0.64]	68.16 [-3.21, +3.13]	63.13 [-4.11, +3.99]	71.01 [-4.52, +4.18]
Muse Glimmer 30B Meta	65.95 [-2.02, +2.01]	47.13 [-9.54, +9.44]	82.53 [-7.16, +6.72]	77.25 [-4.58, +4.38]	53.47 [-0.53, +0.54]	62.46 [-3.52, +3.28]	67.54 [-4.15, +3.99]	71.25 [-3.86, +3.90]
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG	47.68 [-2.22, +2.09]	43.38 [-6.10, +6.46]	66.81 [-7.94, +7.69]	47.33 [-5.41, +5.42]	29.15 [-0.55, +0.54]	48.04 [-3.86, +3.73]	54.35 [-4.71, +4.68]	44.72 [-6.48, +7.87]
Gemma 4 (E2B) 5B Google	46.95 [-2.07, +2.06]	43.64 [-6.47, +6.84]	66.21 [-8.09, +7.54]	47.38 [-5.67, +5.49]	21.25 [-0.56, +0.52]	47.32 [-3.75, +3.75]	55.12 [-4.74, +4.56]	47.72 [-5.85, +7.04]
GPT OSS 120B MoE mxfp4 OpenAI	46.90 [-1.92, +2.03]	25.95 [-5.11, +5.23]	67.32 [-7.82, +7.43]	58.88 [-4.97, +4.95]	47.76 [-0.49, +0.50]	44.00 [-3.40, +3.40]	54.14 [-4.28, +4.15]	30.24 [-6.11, +6.88]
Mistral Medium 3.5 128B Mistral AI	41.83 [-1.83, +1.81]	37.80 [-6.28, +6.64]	46.67 [-6.55, +6.33]	43.46 [-4.50, +4.38]	42.93 [-0.75, +0.75]	34.60 [-2.88, +2.93]	49.51 [-4.27, +4.01]	37.84 [-3.75, +3.80]
SEA-LION v4.8 (Nemotron) 120B MoE AISG	31.35 [-1.80, +1.80]	37.68 [-8.02, +8.08]	61.65 [-7.27, +6.98]	34.60 [-4.10, +4.19]	15.66 [-0.45, +0.46]	16.18 [-1.91, +1.91]	43.46 [-3.86, +3.78]	10.20 [-2.89, +2.80]
Apertus 1.5 70B Swiss AI	19.21 [-1.22, +1.25]	0.00 [-0.00, +0.00]	57.38 [-7.26, +6.99]	0.00 [-0.00, +0.00]	48.05 [-0.56, +0.56]	9.12 [-1.51, +1.55]	15.84 [-3.28, +3.18]	4.08 [-2.86, +2.89]
GLM 4.7 Flash 30B MoE Z.ai	17.90 [-1.25, +1.21]	1.78 [-1.78, +2.57]	55.30 [-7.05, +7.20]	8.17 [-3.17, +3.04]	37.57 [-0.55, +0.59]	11.82 [-1.72, +1.71]	10.63 [-2.35, +2.66]	0.00 [-0.00, +0.00]
MERaLiON 2 10B A*STAR	10.64 [-1.37, +1.50]	15.36 [-6.71, +7.59]	30.34 [-7.09, +7.66]	0.00 [-0.00, +0.00]	22.97 [-0.53, +0.57]	5.85 [-1.65, +1.71]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]
SEA-LION v4.8 (Nemotron) 30B MoE AISG	10.61 [-1.02, +0.98]	0.00 [-0.00, +0.00]	53.13 [-6.88, +6.62]	0.68 [-0.68, +2.45]	18.97 [-0.53, +0.54]	1.47 [-1.33, +1.36]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]
Olmo 3.1 32B AI2	6.06 [-1.06, +1.13]	0.00 [-0.00, +0.00]	29.66 [-7.29, +7.96]	0.00 [-0.00, +0.00]	12.77 [-0.48, +0.49]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]
Nemotron 3 Super 120B MoE NVIDIA	4.98 [-0.67, +0.78]	0.00 [-0.00, +0.00]	17.99 [-4.36, +5.14]	0.00 [-0.00, +0.00]	14.63 [-0.49, +0.49]	0.00 [-0.00, +0.00]	2.25 [-1.50, +1.47]	0.00 [-0.00, +0.00]
Tiny Aya Water 3B CohereLabs	4.84 [-0.84, +0.88]	0.00 [-0.00, +0.00]	24.08 [-5.95, +6.17]	0.00 [-0.00, +0.00]	9.77 [-0.29, +0.29]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]
Nemotron 3.5 Lightning 30B MoE NVIDIA	3.79 [-0.52, +0.57]	0.00 [-0.00, +0.00]	16.40 [-3.53, +3.98]	0.00 [-0.00, +0.00]	10.14 [-0.39, +0.39]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]	0.00 [-0.00, +0.00]

No records

Legend:

Reasoning model

Model Size: ≤200B

Open models only