SEA-HELM Leaderboard

Burmese Performance

Jul 10, 2026

Burmese Scores by Model

Average of 30 bootstraps. 95% CI are shown.

Model Size: ≤200B

Open instruct models only

Burmese Competency & Task Scores

Average of 30 bootstraps. 95% CI are shown.

Model Size: ≤200B

Open instruct models only

Model	MY	Cultural	Inst. Following	Knowledge	NLG	NLR	NLU	Safety
Gemma 4 31B Google	69.04 ± 0.09	69.13 ± 0.44	79.53 ± 0.47	65.64 ± 0.15	57.91 ± 0.05	67.96 ± 0.11	62.69 ± 0.06	80.42 ± 0.10
SEA-LION v4.5 (Qwen) 27B AISG	63.38 ± 0.21	62.81 ± 0.82	73.67 ± 0.67	56.43 ± 0.35	54.73 ± 0.05	59.26 ± 0.30	62.22 ± 0.20	74.54 ± 0.48
Qwen 3.5 122B MoE Alibaba	61.32 ± 0.21	60.27 ± 0.89	72.93 ± 0.77	59.58 ± 0.45	54.06 ± 0.07	55.99 ± 0.37	58.37 ± 0.19	68.07 ± 0.24
Llama 4 Scout 109B MoE Meta	52.67 ± 0.12	46.06 ± 0.51	71.77 ± 0.75	45.52 ± 0.16	53.91 ± 0.10	51.14 ± 0.12	56.92 ± 0.10	43.34 ± 0.14
Qwen 3.6 27B Alibaba	48.01 ± 0.23	0.32 ± 0.28	71.87 ± 0.87	53.79 ± 0.54	50.08 ± 0.09	36.78 ± 0.54	58.83 ± 0.46	64.44 ± 1.08
SEA-LION v4.5 (Gemma E2B) 5B AISG	33.73 ± 0.18	21.89 ± 0.56	55.70 ± 0.91	22.02 ± 0.41	6.82 ± 0.00	34.32 ± 0.41	43.97 ± 0.22	51.41 ± 0.15
Mistral Medium 3.5 128B Mistral AI	33.06 ± 0.39	22.81 ± 1.82	49.03 ± 1.23	24.22 ± 0.93	31.07 ± 0.19	27.92 ± 0.73	48.99 ± 0.45	27.39 ± 1.11
Gemma 4 (E2B) 5B Google	32.45 ± 0.14	19.53 ± 0.59	54.47 ± 0.90	20.60 ± 0.39	6.82 ± 0.00	32.85 ± 0.34	44.59 ± 0.24	48.29 ± 0.17
MERaLiON 2 10B A*STAR	10.74 ± 0.20	16.27 ± 1.16	30.10 ± 0.89	0.00 ± 0.00	22.98 ± 0.09	5.83 ± 0.18	0.00 ± 0.00	0.00 ± 0.00
GLM 4.7 Flash 30B MoE Z.ai	10.22 ± 0.21	0.11 ± 0.12	36.87 ± 1.17	0.00 ± 0.00	23.37 ± 0.21	4.52 ± 0.32	6.70 ± 0.71	0.00 ± 0.00
Apertus 8B Swiss AI	8.46 ± 0.28	0.59 ± 0.39	29.53 ± 1.52	0.00 ± 0.00	26.48 ± 0.20	0.58 ± 0.30	2.03 ± 0.67	0.00 ± 0.00
Olmo 3.1 32B AI2	6.14 ± 0.11	0.13 ± 0.26	30.07 ± 0.70	0.00 ± 0.00	12.75 ± 0.08	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
Nemotron 3 Super 120B MoE NVIDIA	6.11 ± 0.17	0.16 ± 0.18	30.67 ± 1.13	0.00 ± 0.00	11.91 ± 0.09	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00
Tiny Aya Water 3B CohereLabs	4.96 ± 0.16	0.00 ± 0.00	24.87 ± 1.13	0.00 ± 0.00	9.82 ± 0.09	0.00 ± 0.00	0.00 ± 0.00	0.00 ± 0.00

No records