SEA Performance

Overall SEA Average

Average of 2000 bootstraps. 95% CI are shown.

1

Legend:

Reasoning model

No open weights

Model Size: All

Open/closed models

Language Performance by Model

Average of 2000 bootstraps. 95% CI are shown.

Model
SEA
MY
TL
ID
MS
TA
TH
VI
OpenAI logo
OpenAI logo
GPT 6 Astra

OpenAI

79.94

[-0.54, +0.53]

76.97

[-1.85, +1.79]

81.32

[-1.19, +1.21]

83.47

[-1.09, +1.07]

80.21

[-1.31, +1.28]

80.18

[-1.92, +1.97]

76.77

[-1.12, +1.06]

80.68

[-1.19, +1.16]

Google logo
Google logo
Gemini 4 Hydrogen

Google

79.94

[-0.54, +0.52]

78.31

[-1.81, +1.66]

80.22

[-1.17, +1.16]

83.19

[-1.06, +1.05]

81.18

[-1.31, +1.30]

79.05

[-1.79, +1.84]

76.89

[-1.02, +1.01]

80.72

[-1.14, +1.12]

AISG logo
AISG logo
SEA-LION v5 (Qwen)
397B
MoE

AISG

77.10

[-0.53, +0.53]

70.58

[-1.88, +1.69]

78.43

[-1.28, +1.27]

80.29

[-1.17, +1.21]

79.53

[-1.61, +1.52]

76.55

[-1.79, +1.60]

75.43

[-1.12, +1.09]

78.87

[-1.34, +1.17]

Anthropic logo
Anthropic logo
Opus 4.8

Anthropic

76.67

[-0.60, +0.57]

68.67

[-2.13, +2.06]

78.27

[-1.37, +1.28]

82.04

[-1.21, +1.14]

78.97

[-1.48, +1.52]

75.47

[-2.01, +1.98]

74.44

[-1.29, +1.25]

78.83

[-1.39, +1.40]

Alibaba logo
Alibaba logo
Qwen 3.5
397B
MoE

Alibaba

76.46

[-0.58, +0.54]

69.20

[-1.89, +1.86]

78.24

[-1.23, +1.23]

79.68

[-1.27, +1.22]

79.40

[-1.51, +1.42]

76.24

[-1.84, +1.61]

74.01

[-1.22, +1.21]

78.43

[-1.26, +1.22]

Thinking Machines logo
Thinking Machines logo
Inkling
975B
MoE
NVFP4

Thinking Machines

76.26

[-0.48, +0.48]

65.92

[-1.57, +1.63]

77.70

[-1.14, +1.15]

79.75

[-1.11, +1.03]

80.56

[-1.42, +1.30]

78.62

[-1.30, +1.25]

73.19

[-1.16, +1.14]

78.09

[-1.11, +1.09]

DeepSeek logo
DeepSeek logo
DeepSeek V4 Pro
1600B
MoE

DeepSeek

75.88

[-0.54, +0.54]

69.72

[-1.65, +1.69]

77.74

[-1.26, +1.28]

79.03

[-1.19, +1.11]

79.35

[-1.41, +1.36]

75.71

[-1.77, +1.60]

72.36

[-1.27, +1.24]

77.24

[-1.30, +1.26]

Google logo
Google logo
Gemma 4
31B

Google

75.71

[-0.59, +0.56]

68.91

[-1.79, +1.80]

78.91

[-1.36, +1.29]

79.07

[-1.31, +1.32]

78.32

[-1.64, +1.62]

74.07

[-1.79, +1.65]

73.20

[-1.29, +1.31]

77.48

[-1.30, +1.28]

Z.ai logo
Z.ai logo
GLM 5.3
743B
MoE

Z.ai

75.00

[-0.54, +0.53]

65.91

[-2.06, +1.84]

76.84

[-1.34, +1.27]

79.64

[-1.12, +1.04]

79.15

[-1.45, +1.39]

75.83

[-1.75, +1.68]

70.61

[-1.40, +1.34]

77.01

[-1.31, +1.24]

AISG logo
AISG logo
SEA-LION v5 (Qwen)
27B

AISG

72.82

[-0.57, +0.57]

63.36

[-1.63, +1.59]

73.82

[-1.27, +1.26]

78.41

[-1.28, +1.25]

74.11

[-1.74, +1.75]

71.84

[-1.95, +1.80]

71.93

[-1.23, +1.20]

76.24

[-1.34, +1.23]

Meta logo
Meta logo
Muse Glimmer
30B

Meta

72.70

[-0.54, +0.55]

64.71

[-1.68, +1.71]

75.47

[-1.42, +1.32]

75.36

[-1.22, +1.21]

76.59

[-1.57, +1.61]

71.72

[-1.67, +1.61]

69.40

[-1.22, +1.22]

75.66

[-1.32, +1.23]

Minimax logo
Minimax logo
MiniMax M3
428B
MoE

Minimax

71.76

[-0.60, +0.56]

61.86

[-1.84, +1.83]

74.02

[-1.54, +1.41]

75.53

[-1.25, +1.22]

75.86

[-1.53, +1.47]

72.89

[-1.63, +1.60]

68.60

[-1.36, +1.36]

73.54

[-1.41, +1.43]

Alibaba logo
Alibaba logo
Qwen 3.8
27B

Alibaba

68.48

[-0.56, +0.55]

58.13

[-1.59, +1.55]

69.99

[-1.38, +1.37]

74.13

[-1.38, +1.27]

71.13

[-1.70, +1.71]

67.11

[-1.75, +1.76]

66.42

[-1.25, +1.27]

72.46

[-1.41, +1.39]

AISG logo
AISG logo
SEA-LION v5 (Nemotron)
4B

AISG

68.39

[-0.60, +0.60]

58.91

[-1.88, +1.72]

73.53

[-1.40, +1.41]

72.09

[-1.50, +1.43]

70.26

[-1.84, +1.93]

69.13

[-1.92, +1.76]

65.33

[-1.30, +1.29]

69.46

[-1.61, +1.64]

Mistral AI logo
Mistral AI logo
Mistral Medium 3.5
128B

Mistral AI

64.90

[-0.57, +0.57]

40.14

[-1.51, +1.61]

70.30

[-1.45, +1.42]

71.20

[-1.34, +1.29]

73.97

[-1.67, +1.65]

66.20

[-1.58, +1.53]

61.76

[-1.45, +1.49]

70.69

[-1.55, +1.43]

AISG logo
AISG logo
SEA-LION v5 (Qwen)
9B

AISG

64.69

[-0.58, +0.56]

55.52

[-1.56, +1.46]

62.51

[-1.55, +1.44]

68.82

[-1.44, +1.34]

68.28

[-1.69, +1.74]

65.43

[-1.78, +1.69]

64.11

[-1.30, +1.23]

68.13

[-1.42, +1.46]

NVIDIA logo
NVIDIA logo
Nemotron 3 Ultra
550B
MoE

NVIDIA

63.94

[-0.60, +0.59]

35.03

[-1.78, +1.69]

70.76

[-1.29, +1.27]

73.39

[-1.40, +1.39]

73.71

[-1.55, +1.44]

60.92

[-1.78, +1.79]

64.94

[-1.45, +1.38]

68.83

[-1.46, +1.57]

OpenAI logo
OpenAI logo
GPT OSS
120B
MoE
mxfp4

OpenAI

63.50

[-0.64, +0.65]

45.84

[-1.87, +1.93]

70.99

[-1.47, +1.51]

68.17

[-1.50, +1.41]

68.87

[-1.81, +1.77]

62.24

[-2.02, +1.89]

62.43

[-1.35, +1.37]

65.95

[-1.51, +1.54]

AISG logo
AISG logo
SEA-LION v5 (Gemma E2B)
2B

AISG

60.77

[-0.60, +0.60]

50.67

[-1.81, +1.77]

66.14

[-1.57, +1.51]

63.50

[-1.46, +1.37]

62.14

[-1.80, +1.80]

62.39

[-1.88, +1.81]

59.02

[-1.24, +1.20]

61.52

[-1.44, +1.50]

NVIDIA logo
NVIDIA logo
Nemotron 3.5 Bolt (E4B)
4B

NVIDIA

60.56

[-0.55, +0.56]

52.65

[-1.85, +1.71]

65.69

[-1.47, +1.44]

62.56

[-1.29, +1.15]

62.27

[-1.55, +1.52]

60.66

[-1.70, +1.68]

59.16

[-1.32, +1.25]

60.91

[-1.44, +1.37]

CohereLabs logo
CohereLabs logo
Command A+
218B
MoE

CohereLabs

53.87

[-0.54, +0.57]

15.73

[-1.25, +1.21]

59.37

[-1.33, +1.33]

65.12

[-1.40, +1.34]

62.27

[-1.67, +1.77]

55.99

[-1.62, +1.56]

56.11

[-1.37, +1.30]

62.49

[-1.39, +1.45]

Google logo
Google logo
Gemma 4 (E2B)
2B

Google

53.62

[-0.60, +0.62]

40.86

[-1.85, +1.79]

58.98

[-1.57, +1.57]

56.99

[-1.27, +1.25]

55.78

[-1.62, +1.57]

54.64

[-1.71, +1.70]

52.60

[-1.44, +1.33]

55.45

[-1.44, +1.40]

Swiss AI logo
Swiss AI logo
Apertus 1.5
70B

Swiss AI

51.82

[-0.58, +0.59]

21.01

[-1.31, +1.32]

57.94

[-1.55, +1.55]

58.67

[-1.38, +1.35]

61.88

[-1.68, +1.63]

51.53

[-1.99, +1.94]

52.46

[-1.55, +1.55]

59.22

[-1.45, +1.44]

AI2 logo
AI2 logo
Olmo 3.1
32B

AI2

37.92

[-0.59, +0.60]

6.56

[-1.11, +1.15]

48.83

[-1.89, +1.87]

49.42

[-1.51, +1.52]

52.41

[-1.91, +1.92]

24.86

[-1.57, +1.53]

35.98

[-1.67, +1.68]

47.37

[-1.56, +1.54]

No records

Legend:

Reasoning model

No open weights

Model Size: All

Open/closed models