SEA-HELM Leaderboard

Malay Performance

Malay Scores by Model

Average of 2000 bootstraps. 95% CI are shown.

Legend:

Reasoning model

Model Size: ≤200B

Open models only

Malay Competency & Task Scores

Average of 2000 bootstraps. 95% CI are shown.

Model	MS	Cultural	Inst. Following	Knowledge	Multi-Turn Chat	NLG	NLU	Safety
Qwen 3.5 122B MoE Alibaba	76.31 [-1.72, +1.80]	64.19 [-7.98, +7.61]	88.70 [-5.61, +4.99]	85.99 [-3.99, +3.47]	86.35 [-3.37, +3.17]	91.94 [-0.24, +0.24]	73.43 [-1.94, +1.96]	43.58 [-5.24, +4.90]
Gemma 4 31B Google	76.27 [-1.70, +1.77]	65.45 [-7.42, +7.46]	90.72 [-5.48, +4.75]	83.96 [-4.08, +3.96]	85.29 [-3.74, +3.49]	93.09 [-0.20, +0.20]	74.15 [-2.07, +1.92]	41.21 [-5.47, +5.34]
Qwen 3.6 27B Alibaba	76.26 [-1.76, +1.72]	71.14 [-7.99, +7.27]	89.19 [-5.85, +5.10]	84.00 [-3.91, +3.54]	86.08 [-3.15, +3.20]	92.32 [-0.23, +0.22]	71.78 [-1.99, +1.99]	39.31 [-5.30, +5.17]
SEA-LION v4.5 (Qwen) 27B AISG	75.23 [-1.73, +1.67]	67.57 [-7.07, +7.23]	88.32 [-5.58, +5.25]	83.31 [-3.97, +3.94]	82.12 [-4.01, +3.67]	92.44 [-0.23, +0.22]	72.29 [-2.05, +2.00]	40.57 [-5.29, +4.99]
Muse Glimmer 30B Meta	74.14 [-1.83, +1.79]	53.14 [-8.78, +8.52]	88.57 [-5.59, +5.36]	82.38 [-4.50, +4.00]	85.45 [-3.87, +3.46]	91.68 [-0.25, +0.24]	72.50 [-1.99, +1.94]	45.25 [-4.60, +4.21]
SEA-LION v4.8 (Nemotron) 120B MoE AISG	73.25 [-1.71, +1.74]	59.32 [-7.43, +7.32]	81.75 [-6.27, +5.75]	81.83 [-3.95, +3.59]	84.33 [-3.91, +3.46]	88.67 [-0.32, +0.32]	70.74 [-1.96, +2.00]	46.14 [-5.37, +4.95]
GPT OSS 120B MoE mxfp4 OpenAI	69.68 [-1.82, +1.98]	54.19 [-7.55, +7.31]	80.29 [-6.72, +6.61]	74.95 [-4.62, +4.22]	83.22 [-4.56, +4.18]	90.96 [-0.27, +0.26]	69.77 [-2.05, +1.94]	34.39 [-5.21, +5.20]
Mistral Medium 3.5 128B Mistral AI	68.93 [-1.70, +1.66]	33.04 [-6.04, +6.63]	80.25 [-6.44, +6.06]	79.98 [-4.39, +4.19]	82.84 [-4.09, +4.12]	91.94 [-0.22, +0.22]	71.48 [-1.91, +1.83]	42.96 [-4.99, +4.39]
Nemotron 3 Super 120B MoE NVIDIA	68.66 [-1.71, +1.80]	54.88 [-7.63, +7.01]	74.55 [-6.58, +6.16]	80.14 [-3.85, +3.61]	81.37 [-3.75, +3.75]	86.21 [-0.31, +0.31]	68.72 [-1.93, +1.89]	34.73 [-4.59, +4.44]
Apertus 1.5 70B Swiss AI	65.54 [-1.90, +1.72]	50.35 [-6.61, +6.66]	81.21 [-6.09, +5.57]	60.77 [-4.90, +4.94]	69.47 [-4.56, +4.57]	91.32 [-0.25, +0.23]	64.72 [-2.14, +2.10]	40.91 [-5.64, +5.29]
GLM 4.7 Flash 30B MoE Z.ai	65.50 [-1.76, +1.79]	48.26 [-7.06, +6.96]	75.75 [-6.47, +6.27]	70.57 [-4.49, +4.14]	72.34 [-4.67, +4.70]	88.91 [-0.31, +0.29]	66.36 [-1.97, +1.91]	36.30 [-5.05, +5.01]
SEA-LION v4.5 (Gemma E2B) 5B MoE AISG	62.28 [-1.83, +1.81]	45.42 [-6.59, +6.85]	82.06 [-6.35, +6.15]	53.28 [-5.16, +4.97]	75.93 [-4.75, +4.58]	90.30 [-0.28, +0.28]	64.40 [-2.18, +2.14]	24.58 [-4.51, +4.62]
SEA-LION v4.8 (Nemotron) 30B MoE AISG	62.09 [-1.70, +1.63]	41.47 [-6.00, +6.30]	71.33 [-7.17, +6.17]	76.10 [-4.35, +4.16]	83.37 [-3.92, +3.59]	85.86 [-0.39, +0.37]	55.17 [-1.97, +1.87]	21.35 [-4.84, +4.59]
Gemma 4 (E2B) 5B Google	61.46 [-1.88, +1.80]	40.89 [-6.75, +6.63]	80.30 [-6.85, +6.25]	53.33 [-5.12, +5.13]	76.10 [-4.94, +4.65]	90.00 [-0.29, +0.30]	64.23 [-2.21, +2.15]	25.37 [-4.30, +4.66]
Nemotron 3.5 Lightning 30B MoE NVIDIA	58.76 [-1.68, +1.75]	30.77 [-5.73, +6.14]	59.30 [-7.16, +6.65]	71.22 [-4.59, +4.61]	82.80 [-3.90, +3.92]	81.16 [-0.46, +0.45]	57.31 [-1.97, +1.95]	28.78 [-4.89, +4.93]
Olmo 3.1 32B AI2	58.14 [-2.02, +2.01]	51.19 [-6.56, +6.58]	70.87 [-8.25, +7.82]	39.05 [-5.75, +5.75]	71.11 [-5.09, +4.52]	86.14 [-0.41, +0.41]	56.10 [-2.52, +2.56]	32.55 [-6.25, +6.05]
MERaLiON 2 10B A*STAR	55.29 [-1.90, +1.95]	26.16 [-6.43, +6.75]	62.90 [-8.38, +7.10]	50.06 [-5.94, +5.89]	54.75 [-5.38, +5.38]	86.16 [-0.32, +0.32]	64.74 [-2.39, +2.30]	42.24 [-4.48, +4.01]
Tiny Aya Water 3B CohereLabs	32.71 [-1.47, +1.54]	1.49 [-1.49, +3.22]	63.94 [-6.33, +6.41]	31.66 [-4.99, +5.05]	29.21 [-4.54, +4.53]	59.17 [-0.66, +0.61]	25.67 [-1.99, +2.04]	17.83 [-3.45, +3.21]

No records

Legend:

Reasoning model

Model Size: ≤200B

Open models only