---
title: "How Does Nanbeige4.3-33B Score vs Qwen3.8-27B and Gemma4-31B?"
url: "https://goremy.ai/blog/articles/nanbeige-4-3-33b-benchmarks-vs-qwen-gemma"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-10-27T04:00:09.883Z"
updated: "2026-10-27T04:00:15.114Z"
topic: Open-Weight Model Benchmarks
description: Nanbeige4.3-33B leads 17 of 20 benchmarks against Qwen3.8-27B, Gemma4-31B and Muse-Glimmer-30B. See the full table and where it loses.
---

# How Does Nanbeige4.3-33B Score vs Qwen3.8-27B and Gemma4-31B?

> Nanbeige4.3-33B leads 17 of 20 benchmarks against Qwen3.8-27B, Gemma4-31B and Muse-Glimmer-30B. See the full table and where it loses.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-10-27

## How does Nanbeige4.3-33B score against Qwen3.8-27B, Gemma4-31B and Muse-Glimmer-30B?

On the model card's own table, Nanbeige4.3-33B has the top score on 17 of 20 benchmarks against Qwen3.8-27B, Muse-Glimmer-30B and Gemma4-31B. It leads clearly on agentic coding (Terminal-Bench 2.1 at 80.9, DeepSWE at 65.5) and cowork tasks (Automation Bench at 54.2). It loses narrowly to Qwen3.8-27B on SWE-Bench Pro and GDPval, and to Gemma4-31B on GPQA Diamond. All numbers are self-reported by the Nanbeige team.

## TL;DR

- Nanbeige4.3-33B is a 33B open-weight **flagship teacher model** meant to anchor distillation into smaller Nanbeige4.3 models, such as a planned 3B.
- It tops **17 of 20 listed benchmarks**, with the biggest gaps in agentic coding and agentic cowork categories.
- The three losses are **SWE-Bench Pro, GDPval and GPQA Diamond**, and each is a margin of 1.4 to 1.8 points.
- Qwen3.8-27B is the closest rival. Muse-Glimmer-30B and Gemma4-31B trail by wide margins on most agentic tests.
- Every figure comes from the **developer's own evaluation**, so treat it as a strong claim to verify on your own workload, not an independent ranking.
- The card recommends `preserve_thinking=true` for agent work, and it suggests up to **262,144 max new tokens** for reasoning and chat.

## What is Nanbeige4.3-33B?

Nanbeige4.3-33B is the flagship model of the Nanbeige4.3 series, released as open weights on Hugging Face. The team describes it as a "teacher" agent. The plan is to build a high-capacity agent first, then distill its agentic abilities into smaller models. A 3B model is named as an upcoming example.

The card claims state-of-the-art agentic performance among open models in the roughly 30B class, with a focus on complex coding and professional "cowork" tasks. Cowork here means office and workflow-style agent tasks, such as automation, planning and tool use across jobs.

## What does the full benchmark table show?

The comparison set is Qwen3.8-27B, Muse-Glimmer-30B and Gemma4-31B. All models ran at their highest thinking effort, with the same inference hyperparameters and harness per benchmark, according to the card. Bold marks the top score in each row.

### Agentic coding

| Benchmark | Nanbeige4.3-33B | Qwen3.8-27B | Muse-Glimmer-30B | Gemma4-31B |
|---|---:|---:|---:|---:|
| Terminal-Bench 2.1 | **80.9** | 73.0 | 61.8 | 47.2 |
| Terminal-Bench 4.0 | **23.8** | 6.3 | 1.6 | 0.0 |
| DeepSWE | **65.5** | 42.5 | 4.4 | 2.7 |
| SWE-Bench Pro | 60.3 | **61.7** | 36.9 | 51.2 |
| SWE-Atlas-QnA | **50.8** | 41.9 | 17.7 | 3.3 |
| NL2Repo | **53.0** | 48.3 | 27.4 | 33.8 |
| Scicode | **52.6** | 46.7 | 44.9 | 45.5 |
| SkillsBench | **61.1** | 56.4 | 39.7 | 28.9 |

### Agentic cowork

| Benchmark | Nanbeige4.3-33B | Qwen3.8-27B | Muse-Glimmer-30B | Gemma4-31B |
|---|---:|---:|---:|---:|
| Automation Bench | **54.2** | 38.3 | 15.2 | 9.7 |
| JobBench | **44.8** | 34.5 | 16.6 | 9.0 |
| GDPval | 84.8 | **86.4** | 75.8 | 71.4 |
| Claw-Eval (pass^3) | **78.3** | 75.2 | 57.3 | 44.2 |
| Toolathlon-Verified | **70.4** | 68.5 | 47.2 | 27.8 |
| DeepPlanning | **63.3** | 49.2 | 21.4 | 19.4 |
| EnterpriseOps-Gym | **41.0** | 40.2 | 30.5 | 27.3 |
| YC-bench (final funds) | **$1,025,437** | $641,290 | $513,978 | $265,152 |
| Tau3-Bench | **48.8** | 48.5 | 33.7 | 18.6 |

### Reasoning

| Benchmark | Nanbeige4.3-33B | Qwen3.8-27B | Muse-Glimmer-30B | Gemma4-31B |
|---|---:|---:|---:|---:|
| GPQA Diamond | 89.5 | 89.2 | 83.5 | **91.3** |
| AIME 2026 | **95.2** | 94.5 | 94.7 | 89.2 |
| AA-LCR | **84.0** | 82.0 | 83.3 | 72.0 |

## Where does Nanbeige4.3-33B lead by the widest margins?

The largest gaps are on long-horizon agent tests, where the model runs many tool calls and has to stay on track.

- **Terminal-Bench 4.0:** 23.8 against 6.3 for Qwen3.8-27B. Gemma4-31B scored 0.0.
- **DeepSWE:** 65.5 against 42.5 for Qwen3.8-27B. Muse-Glimmer-30B and Gemma4-31B land at 4.4 and 2.7.
- **Automation Bench:** 54.2 against 38.3 for Qwen3.8-27B.
- **DeepPlanning:** 63.3 against 49.2.
- **JobBench:** 44.8 against 34.5.
- **YC-bench:** about $1.03 million against $641,290 for Qwen3.8-27B.

Terminal-Bench 2.1 is a smaller gap but still a clear one: 80.9 against 73.0. The card also says the model beats a baseline in most of the 35 occupations on JobBench, which points to breadth across professions and not a single strong niche.

## Where does Nanbeige4.3-33B lose?

Three rows go to someone else.

- **SWE-Bench Pro:** Qwen3.8-27B scores 61.7 to Nanbeige's 60.3. Gemma4-31B is at 51.2.
- **GDPval:** Qwen3.8-27B scores 86.4 to Nanbeige's 84.8. The card notes GDPval is scored with revised rubrics.
- **GPQA Diamond:** Gemma4-31B scores 91.3, ahead of Nanbeige at 89.5 and Qwen3.8-27B at 89.2.

Several wins are also thin. Tau3-Bench is 48.8 against 48.5. EnterpriseOps-Gym is 41.0 against 40.2. AIME 2026 is 95.2 against 94.7 for Muse-Glimmer-30B. Differences under a point on benchmarks like these are often within run-to-run noise, so don't read them as a ranking.

## How reliable are these comparisons?

The table is a vendor result. A few details from the footnotes matter when you read it.

- **Thinking mode:** Nanbeige4.3-33B was evaluated with `preserve_thinking=true`, which keeps reasoning from earlier turns in multi-turn conversations. That setting suits agent loops, and it may help the model on multi-step tasks.
- **Harnesses differ by benchmark:** Terminal-Bench 2.1 uses Terminus 2. Terminal-Bench 4 and DeepSWE use mini-swe-agent v2.4.6. NL2Repo uses Claude Code. JobBench uses the Claude Code harness with a search tool. The card says each benchmark used the same harness across models, but your own scaffold may behave differently.
- **Subsets and truncation:** Claw-Eval covers only the General subset of 157 single-turn tasks. Toolathlon-Verified sets no max_tokens limit. YC-bench keeps only the most recent 10 turns of dialogue as context.
- **LLM judging:** SWE-Atlas-QnA is graded against 1,304 task-specific rubrics with GPT-6 Sol as the judge at medium reasoning effort.
- **Size:** At 33B parameters, it is somewhat larger than Qwen3.8-27B, so it is not a strict like-for-like size match.

## How do you run Nanbeige4.3-33B?

The card shows Transformers, SGLang and vLLM paths. The server examples use four-way tensor parallelism (`--tp-size 4` or `--tensor-parallel-size 4`), which hints at the hardware the team expects for serving. The card does not give a minimum VRAM figure.

```bash
vllm serve ${MODEL_PATH} \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 4 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder
```

Recommended settings from the card:

| Scenario | Temperature | Top_p | Max new tokens |
|---|---:|---:|---:|
| Agentic and tool-use tasks | 1.0 | 0.95 | 65,536 |
| Reasoning and chat tasks | 0.6 | 0.95 | 262,144 |

The card recommends `preserve_thinking=True` for multi-turn tool use, office tasks and code agents, and `False` for general chat. It also recommends `tool_call_format="xml"` for the best tool-calling results, with `json` supported for compatibility. Loading through Transformers requires `trust_remote_code=True`.

## Is Nanbeige4.3-33B worth testing?

If you build coding agents or long-running tool-use workflows on open weights, yes. The gaps on Terminal-Bench, DeepSWE, Automation Bench and DeepPlanning are large enough that a self-reported result likely points to something real. The model is a weaker bet if your work looks like SWE-Bench Pro style issue fixing or graduate-level science QA, where Qwen3.8-27B and Gemma4-31B edge it out. Run it in your own harness with `preserve_thinking` enabled before you switch anything.

## Frequently Asked Questions

### Which benchmarks does Nanbeige4.3-33B lose?

It trails Qwen3.8-27B on SWE-Bench Pro (60.3 vs 61.7) and GDPval (84.8 vs 86.4). It trails Gemma4-31B on GPQA Diamond (89.5 vs 91.3). It tops the other 17 rows in the table.

### Is Nanbeige4.3-33B open-weight?

Yes. The weights are published on Hugging Face under Nanbeige/Nanbeige4.3-33B. The card lists Transformers, SGLang and vLLM as supported runtimes.

### What is a "teacher model" in the Nanbeige4.3 series?

The team built the 33B model first as a high-capacity agent. They plan to distill its agentic behavior into smaller models, with Nanbeige4.3-3B named as an example. The 33B model is also offered for direct deployment.

### How long a context does Nanbeige4.3-33B support?

The article does not state a context window length. It recommends a 65,536 max-new-token setting for agentic tasks and 262,144 for reasoning and chat.

### Are the benchmark results independent?

No. The table comes from the Nanbeige team's own evaluation, using the harnesses listed in the footnotes. Independent reproductions on your own tasks are the best check.