---
title: How to Run Nanbeige4.3-33B Locally With vLLM or SGLang
url: "https://goremy.ai/blog/articles/how-to-run-nanbeige-4-3-33b-vllm-sglang"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-10-26T17:20:07.903Z"
updated: "2026-10-31T21:08:30.999Z"
topic: Open Model Run Guides
description: Run Nanbeige4.3-33B with Transformers, vLLM or SGLang. Covers the 512K context, thinking flags, tool-call format, and recommended sampling settings.
---

# How to Run Nanbeige4.3-33B Locally With vLLM or SGLang

> Run Nanbeige4.3-33B with Transformers, vLLM or SGLang. Covers the 512K context, thinking flags, tool-call format, and recommended sampling settings.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-10-26 · Updated 2026-10-31

## How do you run Nanbeige4.3-33B locally?

You can run Nanbeige4.3-33B three ways: Hugging Face Transformers for quick tests, or vLLM and SGLang for serving an API. The model card's server commands both use tensor parallelism across 4 GPUs, the `qwen3` reasoning parser, and the `qwen3_coder` tool-call parser. Context goes up to 524,288 tokens (512K). Behavior is controlled through chat-template flags: `enable_thinking`, `preserve_thinking`, and `tool_call_format`.

## TL;DR

- Nanbeige4.3-33B loads with **Transformers, vLLM, or SGLang**, and the model card gives a ready command for each.
- Both server recipes use **tensor parallelism of 4** and a GPU memory fraction of 0.8.
- Use the **`qwen3` reasoning parser and `qwen3_coder` tool parser** on either server.
- Set **`preserve_thinking` to true** for multi-turn tool use, office tasks, and code agents. Keep it false for general chat and Q&A.
- Use **temperature 1.0 for agentic work** and 0.6 for reasoning and chat, with top_p 0.95 in both cases.
- Prefer **`tool_call_format="xml"`** for tool calling. JSON works for compatibility.

## What is Nanbeige4.3-33B?

Nanbeige4.3-33B is an open-weight model from the Nanbeige team. They describe it as the flagship teacher model of the Nanbeige4.3 series, built first so it can anchor distillation into smaller models (a 3B variant is mentioned as upcoming). The card positions it as a top agentic performer in the roughly 30B class, aimed at complex coding and professional "cowork" tasks.

The card compares it with Qwen3.8-27B, Muse-Glimmer-30B and Gemma4-31B, all at highest thinking effort. It reports leading scores on most listed benchmarks, including Terminal-Bench 2.1 (80.9), DeepSWE (65.5) and AIME 2026 (95.2). It trails on a few: SWE-Bench Pro (60.3 vs 61.7 for Qwen3.8-27B), GDPval (84.8 vs 86.4) and GPQA Diamond (89.5 vs 91.3 for Gemma4-31B). These are the vendor's own numbers, and the card notes the agentic results were run with `preserve_thinking=true`.

## How do you run it with Transformers?

Transformers is the simplest path for a smoke test. The card loads the tokenizer with `use_fast=False` and `trust_remote_code=True`, and the model with `torch_dtype='auto'` and `device_map='auto'`.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = 'Nanbeige/Nanbeige4.3-33B'
tokenizer = AutoTokenizer.from_pretrained(
  model_id,
  use_fast=False,
  trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
  model_id,
  torch_dtype='auto',
  device_map='auto',
  trust_remote_code=True
)

messages = [{'role': 'user', 'content': 'Which number is bigger, 9.11 or 9.8?'}]
inputs = tokenizer.apply_chat_template(
  messages,
  add_generation_prompt=True,
  return_tensors='pt'
).to(model.device)
output_ids = model.generate(inputs, max_new_tokens=1024)
response = tokenizer.decode(output_ids[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)
```

`device_map='auto'` spreads layers across whatever GPUs are visible. The card does not state VRAM requirements, so check your own memory headroom before loading. Note that the example caps `max_new_tokens` at 1024, which is far below the recommended limits for thinking mode.

## How do you serve it with vLLM?

vLLM exposes an OpenAI-compatible endpoint. The card's command:

```bash
MODEL_PATH=/path/to/your/Nanbeige4.3-33B
vllm serve ${MODEL_PATH} \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 4 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder
```

What each flag does:

| Flag | Purpose |
|---|---|
| `--tensor-parallel-size 4` | Splits the model across 4 GPUs |
| `--gpu-memory-utilization 0.8` | Leaves 20% of GPU memory unused |
| `--enable-auto-tool-choice` | Lets the model decide when to call tools |
| `--reasoning-parser qwen3` | Separates thinking output from the final answer |
| `--tool-call-parser qwen3_coder` | Parses the model's tool calls into structured form |

## How do you serve it with SGLang?

SGLang uses near-identical settings with different flag names.

```bash
MODEL_PATH=/path/to/your/Nanbeige4.3-33B
python -m sglang.launch_server \
  --model-path ${MODEL_PATH} \
  --host 0.0.0.0 \
  --port 8000 \
  --tp-size 4 \
  --mem-fraction-static 0.8 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder
```

`--tp-size 4` sets tensor parallelism and `--mem-fraction-static 0.8` reserves the static memory share. The same two parsers apply. Both commands bind to `0.0.0.0`, which exposes the server on all network interfaces, so restrict access if the machine is reachable beyond your own host.

## What do the chat-template flags do?

The tokenizer ships a configurable chat template with three controls:

- **`enable_thinking`**: Decides whether the model reasons before answering. It is on by default. Set it to `False` for non-thinking mode.
- **`preserve_thinking`**: Decides whether reasoning from earlier assistant turns stays in a multi-turn conversation. The card recommends `False` for general chat and question answering, and `True` for multi-turn tool use, office tasks, and code-agent workflows.
- **`tools`**: Passing tools turns on the tool-use template. Set `tool_call_format="xml"` for the best tool-calling performance. `json` is supported for compatibility.

Keeping prior reasoning in context costs tokens on every turn, but it is the setting the card used for its agentic benchmark results. If you build a code agent, match that configuration before judging the model.

When serving through vLLM or SGLang, these flags generally reach the template through the chat-completions request's template arguments. Check your server version's documentation for the exact field, since the card does not show a client-side example.

## What sampling settings should you use?

The card gives two profiles.

| Scenario | Temperature | Top_p | Max new tokens |
|---|---:|---:|---:|
| Agentic and tool-use tasks | 1.0 | 0.95 | 65,536 |
| Reasoning and chat tasks | 0.6 | 0.95 | 262,144 |

Agentic runs use a higher temperature and a smaller output budget. Reasoning and chat get a lower temperature and a very large budget, which leaves room for long thinking traces. Those budgets sit inside the 512K context window, and your prompt plus history must fit alongside them.

## How big is the context window, and does it affect serving?

The model supports up to 524,288 tokens. The card does not publish memory figures for using the full window, and the serve commands do not set a maximum model length. Long contexts consume KV cache memory, so if a server fails to start or runs out of memory on long requests, lowering the server's maximum context length is a standard first step. Verify that against your engine's docs rather than assuming a specific value. For measured numbers, see the [Nanbeige4.3 33B VRAM requirements on vLLM](https://goremy.ai/blog/articles/run-nanbeige4-3-33b-vllm-vram-requirements). For another example of a 512K setup, see how to [serve Sage-A1.5 with SGLang or vLLM at 512K context](https://goremy.ai/blog/articles/how-to-serve-sage-a1-5-sglang-vllm).

## Is Nanbeige4.3-33B worth trying?

If you build agents, the benchmark table is the reason to test it. The card reports leads on agentic coding and cowork evaluations such as Terminal-Bench 2.1, Automation Bench (54.2) and Toolathlon-Verified (70.4). Harnesses differ across benchmarks (Terminus 2, mini-swe-agent v2.4.6, Claude Code), and the scores come from the developers, so run your own tasks. The 4-GPU recipe also tells you the practical entry cost: this is not a single consumer card setup by default.

## Frequently Asked Questions

### How many GPUs do I need for Nanbeige4.3-33B?

The official vLLM and SGLang commands use tensor parallelism of 4, so the reference setup is four GPUs. The card gives no minimum hardware spec or quantization guidance, so smaller setups are untested by the source. If you want a 30B-class model aimed at a single consumer card, see how [Raon-1 mini runs on an RTX 5090](https://goremy.ai/blog/articles/raon-1-mini-preview-krafton-open-model).

### Should I turn on preserve_thinking?

Turn it on for multi-turn tool use, office tasks, and code-agent workflows. Leave it off for general chat and question answering. The card's agentic benchmark results were evaluated with it enabled.

### Which tool-call format is best?

The card recommends `tool_call_format="xml"` for the best tool-calling performance. JSON is supported if you need compatibility with existing tooling. On the servers, the `qwen3_coder` parser handles tool calls.

### Can I disable reasoning?

Yes. Thinking is enabled by default, and you can set `enable_thinking` to `False` for non-thinking mode.