How to Run Nanbeige4.3-33B Locally With vLLM or SGLang

How do you run Nanbeige4.3-33B locally?

You can run Nanbeige4.3-33B three ways: Hugging Face Transformers for quick tests, or vLLM and SGLang for serving an API. The model card's server commands both use tensor parallelism across 4 GPUs, the qwen3 reasoning parser, and the qwen3_coder tool-call parser. Context goes up to 524,288 tokens (512K). Behavior is controlled through chat-template flags: enable_thinking, preserve_thinking, and tool_call_format.

What is Nanbeige4.3-33B?

Nanbeige4.3-33B is an open-weight model from the Nanbeige team. They describe it as the flagship teacher model of the Nanbeige4.3 series, built first so it can anchor distillation into smaller models (a 3B variant is mentioned as upcoming). The card positions it as a top agentic performer in the roughly 30B class, aimed at complex coding and professional "cowork" tasks.

The card compares it with Qwen3.8-27B, Muse-Glimmer-30B and Gemma4-31B, all at highest thinking effort. It reports leading scores on most listed benchmarks, including Terminal-Bench 2.1 (80.9), DeepSWE (65.5) and AIME 2026 (95.2). It trails on a few: SWE-Bench Pro (60.3 vs 61.7 for Qwen3.8-27B), GDPval (84.8 vs 86.4) and GPQA Diamond (89.5 vs 91.3 for Gemma4-31B). These are the vendor's own numbers, and the card notes the agentic results were run with preserve_thinking=true.

How do you run it with Transformers?

Transformers is the simplest path for a smoke test. The card loads the tokenizer with use_fast=False and trust_remote_code=True, and the model with torch_dtype='auto' and device_map='auto'.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = 'Nanbeige/Nanbeige4.3-33B'
tokenizer = AutoTokenizer.from_pretrained(
  model_id,
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)

messages = [{'role': 'user', 'content': 'Which number is bigger, 9.11 or 9.8?'}]
inputs = tokenizer.apply_chat_template(
  messages,
  add_generation_prompt=True,
  return_tensors='pt'
).to(model.device)
output_ids = model.generate(inputs, max_new_tokens=1024)
response = tokenizer.decode(output_ids[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

device_map='auto' spreads layers across whatever GPUs are visible. The card does not state VRAM requirements, so check your own memory headroom before loading. Note that the example caps max_new_tokens at 1024, which is far below the recommended limits for thinking mode.

How do you serve it with vLLM?

vLLM exposes an OpenAI-compatible endpoint. The card's command:

MODEL_PATH=/path/to/your/Nanbeige4.3-33B
vllm serve ${MODEL_PATH} \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 4 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

What each flag does:

How do you serve it with SGLang?

SGLang uses near-identical settings with different flag names.

MODEL_PATH=/path/to/your/Nanbeige4.3-33B
python -m sglang.launch_server \
  --model-path ${MODEL_PATH} \
  --host 0.0.0.0 \
  --port 8000 \
  --tp-size 4 \
  --mem-fraction-static 0.8 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

--tp-size 4 sets tensor parallelism and --mem-fraction-static 0.8 reserves the static memory share. The same two parsers apply. Both commands bind to 0.0.0.0, which exposes the server on all network interfaces, so restrict access if the machine is reachable beyond your own host.

What do the chat-template flags do?

The tokenizer ships a configurable chat template with three controls:

• enable_thinking: Decides whether the model reasons before answering. It is on by default. Set it to False for non-thinking mode.
• preserve_thinking: Decides whether reasoning from earlier assistant turns stays in a multi-turn conversation. The card recommends False for general chat and question answering, and True for multi-turn tool use, office tasks, and code-agent workflows.
• tools: Passing tools turns on the tool-use template. Set tool_call_format="xml" for the best tool-calling performance. json is supported for compatibility.

Keeping prior reasoning in context costs tokens on every turn, but it is the setting the card used for its agentic benchmark results. If you build a code agent, match that configuration before judging the model.

When serving through vLLM or SGLang, these flags generally reach the template through the chat-completions request's template arguments. Check your server version's documentation for the exact field, since the card does not show a client-side example.

What sampling settings should you use?

The card gives two profiles.

Agentic runs use a higher temperature and a smaller output budget. Reasoning and chat get a lower temperature and a very large budget, which leaves room for long thinking traces. Those budgets sit inside the 512K context window, and your prompt plus history must fit alongside them.

How big is the context window, and does it affect serving?

The model supports up to 524,288 tokens. The card does not publish memory figures for using the full window, and the serve commands do not set a maximum model length. Long contexts consume KV cache memory, so if a server fails to start or runs out of memory on long requests, lowering the server's maximum context length is a standard first step. Verify that against your engine's docs rather than assuming a specific value. For measured numbers, see the Nanbeige4.3 33B VRAM requirements on vLLM. For another example of a 512K setup, see how to serve Sage-A1.5 with SGLang or vLLM at 512K context.

Is Nanbeige4.3-33B worth trying?

If you build agents, the benchmark table is the reason to test it. The card reports leads on agentic coding and cowork evaluations such as Terminal-Bench 2.1, Automation Bench (54.2) and Toolathlon-Verified (70.4). Harnesses differ across benchmarks (Terminus 2, mini-swe-agent v2.4.6, Claude Code), and the scores come from the developers, so run your own tasks. The 4-GPU recipe also tells you the practical entry cost: this is not a single consumer card setup by default.