--- title: "How Do You Run DSpine With Qwen3.8-27B on SGLang?" url: "https://goremy.ai/blog/articles/run-dspine-qwen3-8-27b-sglang-guide" author: Luis Chavez-Mattos editor: Luis Chavez-Mattos editor_title: Director, Product published: "2026-11-09T21:20:08.335Z" updated: "2026-11-10T17:09:05.934Z" topic: Local AI description: "Set up Tencent's DSpine draft model with Qwen3.8-27B on SGLang: install steps, serve flags, block size 16, H100 memory use and the A100 problem." --- # How Do You Run DSpine With Qwen3.8-27B on SGLang? > Set up Tencent's DSpine draft model with Qwen3.8-27B on SGLang: install steps, serve flags, block size 16, H100 memory use and the A100 problem. Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-09 · Updated 2026-11-10 ## How do you run Qwen3.8-27B with DSpine on SGLang? Clone Tencent's DSpine repo, install it with pip (it pulls you toward SGLang), download the `Qwen3.8-27B-DSpine-b16` draft checkpoint, and launch `sglang serve` with `--speculative-algorithm DSPINE` and `--speculative-num-draft-tokens 16`. The main model stays `Qwen/Qwen3.8-27B`. The draft model is a five-layer helper with block size 16. In one local test, an 80 GB H100 handled it. An A100 did not work for that tester. ## TL;DR - DSpine is a **small five-layer draft model** that guesses 16 tokens at a time so Qwen3.8-27B can verify them in one pass. It is released under Apache 2.0. - Setup is **clone, `pip install -e .`, download the draft checkpoint, then serve** with the flags from the model card. - The tested hardware was a **single 80 GB H100**, using just under 79 GB of VRAM with DSpine running, including context. - Without DSpine, the same Qwen3.8-27B setup used just under 72 GB, so the **draft model costs roughly 7 GB** more in that test. - A creator reported that **the A100 attempt failed**, so plan on Hopper-class hardware. - In a quick eight-prompt test, throughput went from about 33 to about 156 tokens per second, which the tester called **roughly 4.8 times faster**. ## What is DSpine? DSpine is a speculative decoding method from Tencent. Normally a large model writes one token per full forward pass. Speculative decoding adds a small, fast draft model that proposes several upcoming tokens. The big model checks all of them in a single pass, keeps the correct prefix, fixes the first wrong token and discards the rest. The big model has the final say, so the output quality comes from the target model. The usual weakness is that parallel drafters guess each position without seeing their neighbors. Their guesses can disagree, and verification stops at the first mismatch. According to the model card, DSpine addresses this by passing predicted predecessor features to adjacent positions at every layer. Causal dependencies travel through network depth instead of through one token at a time. The released checkpoint is a five-layer draft model for Qwen3.8-27B. It uses block size 16 and an 8,192-token sliding window. ## What hardware does it need? The model card does not list a minimum GPU. The only hardware data point comes from a creator's local test on one 80 GB H100. | Setup | Reported VRAM | | --- | --- | | Qwen3.8-27B with DSpine | Just under 79 GB (including context length) | | Qwen3.8-27B without DSpine | Just under 72 GB | The tester set the static memory fraction to 0.85 to make everything fit on the card. The model card's own example uses 0.80, which leaves more headroom. Start with 0.80 and raise it only if you need more KV cache room. Expect little spare room on 80 GB. The card's example also sets a 131072-token context length, a cap of 8 running requests and a small Mamba cache, all of which affect memory. If you run out, lower the context length or the concurrency first. ## Why did it fail on an A100? The tester tried an A100 first and said it "didn't really work." No error or cause was given, so treat the reason as unknown. One plausible factor: the recommended command uses FlashAttention 3 (`fa3`) for both the target and draft attention backends. FlashAttention 3 is generally associated with Hopper GPUs like the H100, not Ampere cards like the A100. That is an inference, not something the sources confirm. If you only have A100s, expect to experiment with other attention backends, with no guarantee that DSpine will work. ## How do you install DSpine and SGLang? The model card's install steps: ```bash git clone https://github.com/Tencent/DSpine.git cd DSpine pip install -e . ``` You also need SGLang. The card's evaluation used SGLang 0.5.22, so matching that version is the safest starting point. The DSpine algorithm name has to be recognized by your SGLang install, which is why using the repo's install path matters. Then download the draft checkpoint, `tencent/Qwen3.8-27B-DSpine-b16`, from Hugging Face, for example with `huggingface-cli download`. The target model, `Qwen/Qwen3.8-27B`, downloads on first launch or can be fetched the same way. ## What is the serve command? This is the command from the model card. Replace the draft path with your local download. ```bash sglang serve \ --model-path Qwen/Qwen3.8-27B \ --speculative-algorithm DSPINE \ --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \ --speculative-num-draft-tokens 16 \ --dtype bfloat16 \ --attention-backend fa3 \ --speculative-draft-attention-backend fa3 \ --linear-attn-backend triton \ --enable-linear-replayssm-spec \ --mamba-ssm-dtype float32 \ --mamba-radix-cache-strategy extra_buffer \ --max-mamba-cache-size 64 \ --mem-fraction-static 0.80 \ --context-length 131072 \ --max-running-requests 8 \ --cuda-graph-max-bs-decode 8 \ --min-free-slots-delay 1 \ --sampling-backend flashinfer \ --reasoning-parser qwen3 \ --tool-call-parser qwen3_coder \ --trust-remote-code ``` The flags that matter most: | Flag | What it does | | --- | --- | | `--speculative-algorithm DSPINE` | Selects the DSpine method | | `--speculative-draft-model-path` | Points to the five-layer draft checkpoint | | `--speculative-num-draft-tokens 16` | Matches the checkpoint's block size of 16 | | `--attention-backend fa3` and the draft equivalent | Use FlashAttention 3 for target and draft | | `--enable-linear-replayssm-spec` | Enables ReplaySSM, which the card's evaluation used | | `--mem-fraction-static 0.80` | Sets the static memory share (0.85 in the creator's run) | | `--max-running-requests 8` | Caps concurrency, matching the card's evaluation | | `--reasoning-parser qwen3`, `--tool-call-parser qwen3_coder` | Parse reasoning blocks and tool calls | Keep `--speculative-num-draft-tokens` at 16. The checkpoint name ends in `b16` because it was trained for that block size. ## How fast is it? Two sets of numbers exist, and they measure different things. **Local test.** The creator sent eight prompts one at a time to the server and measured tokens per second. With DSpine, the run averaged about 156 tokens per second on the H100, with individual prompts between 95 and 235. Without DSpine, the same script gave about 33 tokens per second. That is roughly a 4.8x gain. It was a quick test on one card, not a rigorous benchmark. **Tencent's evaluation.** The model card reports acceptance length (tokens accepted per verification round) against the DFlash2 baseline. Higher means fewer slow passes through the big model. | Task | DFlash2 (T=0) | DSpine (T=0) | | --- | ---: | ---: | | GSM8K | 5.69 | 7.20 | | MATH-500 | 5.49 | 6.49 | | HumanEval | 4.87 | 5.71 | | MBPP | 5.04 | 6.14 | | MT-Bench | 4.11 | 4.48 | The gap narrows at temperature 1.0 (for example, GSM8K goes from 5.45 to 6.31). On agent workloads, end-to-end throughput speedups over autoregressive decoding range from 2.71x to 3.84x at low reasoning and 2.87x to 3.55x at xhigh. DFlash2 lands lower on each of those benchmarks. These runs used concurrency 8, BF16 and FlashAttention 3. Open-ended chat like MT-Bench shows the smallest gain. Structured tasks like math and code show the largest. ## Is DSpine worth trying? If you already serve Qwen3.8-27B on an H100 and latency matters, probably yes. The cost is about 7 GB of extra VRAM in the creator's test and a more involved launch command. The output still comes from the target model's verification, so you are trading memory and setup effort for speed. If you are on A100s or tight on memory, hold off. The one reported A100 attempt failed, and the 80 GB card was nearly full with DSpine running. ## Frequently Asked Questions ### What is the block size, and can I change it? The checkpoint `Qwen3.8-27B-DSpine-b16` uses block size 16, and the serve command sets `--speculative-num-draft-tokens 16` to match. The sources do not describe other block sizes for this model, so use 16. ### Does DSpine change the model's answers? The target model verifies every draft token and has the final say, so the design is meant to preserve the big model's output. Small numerical differences can occur in any serving stack, but the sources do not quantify them. ### Is DSpine free to use commercially? The model card lists the Apache 2.0 license. Check the repo's LICENSE file and the base Qwen3.8-27B license for your own use case. ### Why does the memory fraction differ between guides? The model card uses `--mem-fraction-static 0.80`. The creator used 0.85 to fit on an 80 GB H100 with a long context. Both are starting points, and the right value depends on your context length and concurrency. ### Where can I read the research behind it? The code lives at github.com/Tencent/DSpine, and the model card describes the method. The body does not give a paper link.