What Is DSpine? Tencent's Speculative Decoding Draft Model

What is DSpine?

DSpine is an open-weight speculative decoding draft model from Tencent, released under Apache 2.0. The Qwen3.8-27B checkpoint is a small helper (about 2B parameters, five layers) that guesses a block of 16 tokens at once so the 27B model can verify them in a single pass. Its twist is that each layer passes information between neighboring guess positions, so the parallel guesses agree with each other and more of them get accepted.

What is speculative decoding?

A large language model normally writes one token at a time. Each token needs a full pass through the model, and that pass is slow. Speculative decoding adds a small, fast draft model that guesses the next several tokens. The big model then checks all of those guesses in one pass.

It keeps the correct guesses from the start of the block, fixes the first wrong one, and discards the rest. The big model always has the final say, so the output quality comes from the target model. The only thing that changes is speed.

The metric that matters is acceptance length: the average number of tokens accepted per verification round. Higher means fewer slow passes through the big model.

How does DSpine work?

The core problem with fast drafters is that parallel guesses are made without seeing each other. If position 3 guesses one thing and position 4 guesses something that does not follow from it, the target model rejects everything from the first mismatch onward.

DSpine's paper title says it plainly: "Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding." The model card describes the mechanism like this: each layer passes predicted predecessor features to adjacent positions, carrying causal dependencies through network depth.

In plain terms:

1. The draft model starts all 16 positions in parallel.
2. At every layer, each position hands its current guess to the next position.
3. The guesses are refined together, so they line up as they move deeper into the network.
4. At the end, tokens are picked left to right, which keeps the final step cheap.

The dependency between tokens is still modeled. It is carried through the depth of the network rather than through sequential generation steps.

The checkpoint details from the model card:

How is DSpine different from Eagle 3, DFlash and DFlash2?

The approaches differ in when the draft tokens get to see each other.

That comparison follows the creator's summary of the field. The model card also notes that DSpine's inference uses dynamic verification that references DSpark, and it was built with thanks to DFlash, SpecForge, SGLang and FlashInfer. So it is part of a lineage, not a clean break.

One detail worth noting: in Tencent's evaluation, DSpine drafts blocks of 16 while the official DFlash2 baseline uses blocks of 8. The comparison reflects each method as configured, not an identical block size.

How fast is DSpine?

Tencent's numbers come from SGLang 0.5.22, BF16, FlashAttention 3 and concurrency 8.

Acceptance length vs DFlash2

The gain is largest on math and code, where output is more predictable, and smallest on open-ended chat (MT-Bench). That pattern is typical for speculative decoding. Temperature 1.0 sampling lowers acceptance for both methods.

Agent throughput

For agent benchmarks, Tencent replayed recorded tool-decision prefixes. Throughput is total output tokens divided by wall time. A few examples at the low reasoning setting:

DSpine leads DFlash2 in every row the card lists, at both low and xhigh reasoning. The gap in end-to-end throughput is smaller than the gap in acceptance length, which makes sense. Longer accepted runs help, but the larger draft block and verification add their own cost. A video review of the paper cited roughly 11% more throughput than DFlash2 on average, which fits that picture.

The independent single-stream test

A creator served Qwen3.8-27B with SGLang on a single 80 GB H100 and ran eight prompts one at a time. Without DSpine, the model produced about 33 tokens/s. With DSpine it produced about 156 tokens/s overall, with individual prompts ranging from roughly 95 to 235. That is about 4.8x.

This is a small test on one machine and sits well above Tencent's concurrency-8 agent figures. Batch size, prompt type and settings all change the result. Use it as evidence that the speedup reproduces, not as a number to expect on your workload.

How do you run DSpine?

The setup uses the Tencent repo and SGLang.

git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e .

Then serve the target model with the draft model attached. The key flags from the model card:

sglang serve \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DSPINE \
  --speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \
  --speculative-num-draft-tokens 16 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --speculative-draft-attention-backend fa3

The full command on the model card adds flags for linear attention, the ReplaySSM cache, memory fraction, context length (131072), max running requests (8) and the Qwen3 reasoning and tool-call parsers. Copy that complete command rather than the trimmed version above.

On hardware, the creator reported that the setup worked on an H100 but did not work when first tried on an A100. Memory use in that test was just under 79 GB with DSpine versus just under 72 GB without it, so the draft model and its caches cost several extra gigabytes.

What are the limitations?

• Narrow target. This checkpoint is built for Qwen3.8-27B. A draft model has to match its target.
• Runtime dependency. The documented path runs through SGLang with a specific algorithm flag and a repo install.
• Vendor benchmarks. The tables come from Tencent. Independent results on your prompts and batch sizes are what count.
• Memory overhead. The draft model and caches add VRAM use on top of a 27B model.