Indonesian Performance

Indonesian Scores by Model

Average of 2000 bootstraps. 95% CI are shown.

1

Legend:

Reasoning model

No open weights

Model Size: All

Open/closed models

Indonesian Competency & Task Scores

Average of 2000 bootstraps. 95% CI are shown.

Model
ID
Cultural
Inst. Following
Knowledge
Linguistic Diagnostics
Multi-Turn Chat
NLG
NLR
NLU
Safety
OpenAI logo
OpenAI logo
GPT 6 Astra

OpenAI

81.40

[-1.24, +1.22]

76.60

[-5.14, +4.80]

96.72

[-3.39, +2.56]

92.26

[-3.05, +2.66]

84.00

[-3.55, +3.33]

85.64

[-4.04, +3.48]

55.55

[-0.60, +0.61]

91.91

[-1.61, +1.47]

86.40

[-3.00, +2.60]

63.50

[-6.79, +5.90]

Anthropic logo
Anthropic logo
Opus 4.8

Anthropic

81.23

[-1.27, +1.20]

76.13

[-6.14, +6.12]

95.83

[-3.45, +2.63]

90.55

[-3.18, +2.99]

79.42

[-3.37, +3.36]

87.66

[-3.25, +3.17]

55.67

[-0.57, +0.58]

90.62

[-1.72, +1.58]

85.71

[-2.80, +2.65]

69.44

[-6.01, +5.75]

Google logo
Google logo
Gemini 3.1 Pro Preview

Google

81.21

[-1.24, +1.19]

76.33

[-5.94, +5.54]

95.25

[-3.70, +3.09]

91.35

[-3.06, +2.90]

83.38

[-2.93, +3.01]

84.55

[-3.70, +3.51]

55.88

[-0.64, +0.65]

91.07

[-1.55, +1.54]

85.46

[-2.81, +2.63]

67.59

[-6.06, +5.61]

Alibaba logo
Alibaba logo
Qwen 3.5
397B
MoE

Alibaba

79.85

[-1.21, +1.24]

74.71

[-6.23, +5.97]

97.04

[-3.11, +2.25]

87.96

[-3.55, +3.54]

78.48

[-3.47, +3.42]

88.15

[-3.07, +2.97]

55.03

[-0.59, +0.59]

90.44

[-1.70, +1.59]

81.61

[-3.25, +3.12]

65.21

[-6.09, +5.69]

Google logo
Google logo
Gemma 4
31B

Google

79.75

[-1.25, +1.20]

76.45

[-5.15, +5.24]

97.75

[-2.87, +1.90]

84.70

[-4.12, +3.76]

78.66

[-3.54, +3.41]

86.59

[-3.47, +3.39]

56.53

[-0.72, +0.71]

90.35

[-1.76, +1.57]

83.18

[-3.10, +2.85]

63.57

[-6.23, +5.79]

Alibaba logo
Alibaba logo
Qwen 3.6
27B

Alibaba

77.96

[-1.22, +1.20]

77.18

[-5.51, +5.41]

96.69

[-3.12, +2.48]

84.63

[-3.75, +3.54]

68.60

[-3.73, +3.44]

86.05

[-3.54, +3.42]

55.18

[-0.59, +0.62]

89.96

[-1.74, +1.65]

82.66

[-2.92, +2.85]

60.74

[-5.83, +5.44]

Z.ai logo
Z.ai logo
GLM 5.2
754B
MoE
FP8

Z.ai

77.79

[-1.16, +1.24]

72.58

[-5.85, +5.54]

96.08

[-3.10, +2.49]

84.51

[-3.56, +3.44]

73.58

[-3.40, +3.26]

86.70

[-3.56, +3.26]

53.70

[-0.59, +0.60]

88.43

[-1.79, +1.63]

83.79

[-2.90, +2.79]

60.70

[-5.74, +5.60]

Minimax logo
Minimax logo
MiniMax M3
428B
MoE

Minimax

76.81

[-1.23, +1.21]

75.00

[-5.27, +4.98]

90.18

[-4.71, +4.22]

84.59

[-3.80, +3.74]

67.85

[-3.02, +3.08]

85.40

[-3.52, +3.48]

55.02

[-0.54, +0.55]

88.58

[-1.75, +1.67]

80.62

[-3.00, +2.84]

64.06

[-4.71, +4.27]

DeepSeek logo
DeepSeek logo
DeepSeek V4 Pro
1600B
MoE

DeepSeek

76.59

[-1.13, +1.14]

60.27

[-5.27, +5.14]

93.44

[-3.80, +3.34]

86.54

[-3.58, +3.63]

74.13

[-3.55, +3.37]

86.50

[-3.11, +2.89]

55.77

[-0.60, +0.61]

85.87

[-1.82, +1.82]

83.40

[-3.00, +2.99]

63.37

[-4.92, +4.53]

AISG logo
AISG logo
SEA-LION v4.5 (Qwen)
27B

AISG

76.51

[-1.23, +1.23]

76.34

[-5.89, +5.62]

95.94

[-3.09, +2.39]

83.27

[-3.85, +3.69]

63.20

[-3.98, +3.83]

83.50

[-3.59, +3.46]

54.77

[-0.57, +0.59]

89.50

[-1.85, +1.82]

82.30

[-3.13, +3.03]

59.74

[-5.90, +5.54]

Meta logo
Meta logo
Muse Glimmer
30B

Meta

75.61

[-1.13, +1.10]

61.95

[-5.93, +5.89]

94.16

[-4.16, +3.22]

83.91

[-3.95, +3.76]

66.77

[-3.67, +3.63]

87.98

[-2.96, +2.80]

53.85

[-0.51, +0.50]

83.89

[-1.93, +1.93]

83.99

[-2.85, +2.73]

64.01

[-3.53, +3.32]

Thinking Machines logo
Thinking Machines logo
Inkling Small
276B
MoE

Thinking Machines

75.13

[-1.07, +1.16]

61.62

[-5.57, +5.48]

95.96

[-2.63, +2.25]

84.87

[-3.87, +3.83]

65.78

[-4.09, +3.77]

89.63

[-2.44, +2.41]

54.10

[-0.49, +0.50]

85.21

[-1.90, +1.74]

80.17

[-3.17, +3.06]

58.80

[-4.41, +3.75]

NVIDIA logo
NVIDIA logo
Nemotron 3 Ultra
550B
MoE

NVIDIA

74.86

[-1.39, +1.33]

73.29

[-5.33, +5.15]

82.65

[-6.46, +6.04]

86.81

[-3.44, +3.23]

68.34

[-3.59, +3.42]

89.02

[-2.59, +2.42]

54.19

[-0.58, +0.54]

87.82

[-1.75, +1.64]

80.69

[-3.01, +2.84]

50.88

[-6.75, +6.73]

AISG logo
AISG logo
SEA-LION v4.8 (Nemotron)
120B
MoE

AISG

73.10

[-1.16, +1.21]

70.02

[-5.50, +5.30]

86.76

[-5.10, +4.55]

83.85

[-3.69, +3.61]

56.20

[-3.92, +3.81]

86.07

[-3.45, +3.43]

53.95

[-0.52, +0.59]

87.25

[-1.84, +1.84]

78.77

[-3.13, +3.10]

54.98

[-3.48, +3.36]

OpenAI logo
OpenAI logo
GPT OSS
120B
MoE
mxfp4

OpenAI

72.28

[-1.35, +1.33]

69.72

[-4.95, +5.01]

88.90

[-5.21, +4.67]

75.92

[-4.55, +4.50]

55.90

[-4.01, +3.65]

85.28

[-4.53, +3.89]

53.81

[-0.49, +0.50]

85.70

[-2.09, +1.95]

78.74

[-3.25, +3.20]

56.58

[-5.55, +5.19]

NVIDIA logo
NVIDIA logo
Nemotron 3 Super
120B
MoE

NVIDIA

70.78

[-1.22, +1.20]

67.85

[-4.71, +5.02]

84.89

[-4.78, +4.63]

82.57

[-3.61, +3.64]

54.77

[-3.79, +3.61]

82.70

[-3.77, +3.50]

52.87

[-0.49, +0.53]

84.12

[-2.08, +1.93]

77.92

[-3.09, +3.01]

49.32

[-4.31, +3.76]

Mistral AI logo
Mistral AI logo
Mistral Medium 3.5
128B

Mistral AI

70.09

[-1.18, +1.13]

39.25

[-4.87, +4.76]

91.18

[-4.40, +3.94]

79.64

[-4.10, +4.02]

62.20

[-3.61, +3.49]

83.49

[-4.32, +4.07]

55.78

[-0.71, +0.74]

84.15

[-2.01, +1.84]

79.56

[-3.20, +3.28]

55.53

[-3.52, +3.18]

CohereLabs logo
CohereLabs logo
Command A+
218B
MoE

CohereLabs

68.66

[-1.30, +1.15]

59.90

[-4.52, +4.79]

88.71

[-4.18, +3.79]

77.91

[-4.11, +3.89]

52.63

[-3.24, +3.46]

82.75

[-3.65, +3.58]

42.42

[-0.63, +0.64]

83.40

[-1.85, +1.78]

74.20

[-3.29, +3.21]

56.06

[-5.27, +4.63]

AISG logo
AISG logo
SEA-LION v4.8 (Nemotron)
30B
MoE

AISG

65.87

[-1.29, +1.33]

58.29

[-4.64, +4.87]

80.27

[-5.27, +4.97]

76.11

[-4.32, +4.35]

46.04

[-3.82, +3.78]

84.54

[-3.84, +3.41]

52.49

[-0.63, +0.68]

82.58

[-2.22, +2.11]

66.45

[-3.38, +3.12]

46.07

[-5.69, +4.93]

Swiss AI logo
Swiss AI logo
Apertus 1.5
70B

Swiss AI

64.69

[-1.37, +1.36]

67.32

[-4.63, +4.59]

88.82

[-4.41, +3.80]

56.15

[-5.48, +5.35]

42.67

[-3.71, +3.94]

70.23

[-4.48, +4.24]

53.99

[-0.53, +0.57]

76.28

[-2.98, +2.88]

69.35

[-3.82, +3.80]

57.36

[-5.41, +4.78]

NVIDIA logo
NVIDIA logo
Nemotron 3.5 Lightning
30B
MoE

NVIDIA

64.34

[-1.36, +1.37]

48.82

[-4.68, +4.74]

75.70

[-6.05, +5.73]

75.70

[-4.33, +4.30]

46.64

[-3.97, +3.79]

82.32

[-4.19, +3.88]

50.89

[-0.58, +0.61]

80.43

[-2.33, +2.18]

63.35

[-3.34, +3.48]

55.19

[-5.68, +5.20]

Google logo
Google logo
Gemma 4 (E2B)
5B

Google

64.13

[-1.39, +1.41]

54.52

[-4.77, +4.44]

90.23

[-4.64, +4.29]

53.28

[-5.28, +4.93]

43.55

[-3.76, +3.66]

77.45

[-5.11, +4.96]

53.28

[-0.47, +0.49]

77.17

[-2.50, +2.46]

72.87

[-3.78, +3.49]

54.84

[-5.45, +5.68]

AISG logo
AISG logo
SEA-LION v4.5 (Gemma E2B)
5B
MoE

AISG

63.96

[-1.38, +1.37]

55.34

[-4.73, +4.92]

89.17

[-4.76, +4.17]

52.95

[-4.99, +4.80]

44.51

[-3.57, +3.65]

75.96

[-4.77, +4.83]

53.70

[-0.51, +0.52]

76.06

[-2.44, +2.37]

72.97

[-3.62, +3.40]

55.02

[-5.58, +5.62]

A*STAR logo
A*STAR logo
MERaLiON 2
10B

A*STAR

58.01

[-1.66, +1.65]

44.10

[-5.45, +5.64]

72.44

[-7.33, +6.97]

50.25

[-5.80, +5.70]

44.88

[-4.58, +4.72]

54.76

[-5.15, +5.14]

55.33

[-0.69, +0.67]

72.98

[-2.86, +2.74]

71.10

[-3.81, +3.60]

56.21

[-5.72, +5.10]

AI2 logo
AI2 logo
Olmo 3.1
32B

AI2

55.89

[-1.56, +1.62]

54.90

[-4.61, +4.64]

82.76

[-6.10, +5.57]

45.45

[-5.70, +5.76]

27.66

[-3.78, +4.05]

71.90

[-4.92, +4.61]

51.62

[-0.55, +0.54]

66.97

[-3.53, +3.59]

62.84

[-4.24, +3.97]

38.91

[-6.21, +7.03]

No records

Legend:

Reasoning model

No open weights

Model Size: All

Open/closed models