DSpine vs DFlash2: How Much Faster Is It? Benchmarks

How much faster is DSpine than DFlash2 and autoregressive decoding?

On Tencent's published agent benchmarks, DSpine runs Qwen3.8-27B about 2.7x to 3.8x faster than plain autoregressive (AR) decoding. It beats DFlash2 by roughly 8% to 21% in end-to-end throughput, depending on the task. In a separate hands-on single-GPU test on an 80 GB H100, DSpine reached about 156 tokens/s versus about 33 tokens/s without it, roughly 4.7 to 4.8x. The two setups differ, so the numbers are not interchangeable.

What is DSpine and how does it work?

DSpine is a speculative decoding method from Tencent. A small draft model guesses several tokens ahead, and the big model verifies all of them in a single forward pass. It keeps the correct prefix, fixes the first wrong token, and discards the rest. The big model has the final say, so output quality stays tied to the target model.

The weakness of block-parallel drafters is that each position guesses without seeing its neighbors' guesses. They can contradict each other, and verification stops at the first mismatch. DSpine addresses this inside the draft network. According to the model card, each layer passes predicted predecessor features to adjacent positions, so causal dependencies travel through network depth rather than across sequential steps. The paper title says the same thing: "Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding."

The released checkpoint, tencent/Qwen3.8-27B-DSpine-b16, has five layers, a block size of 16, and an 8,192-token sliding window.

How does DSpine differ from Eagle 3, DFlash, and DFlash2?

Roughly, the families split like this:

The fix-up approaches repair disagreement afterward. DSpine tries to prevent it during drafting.

What do the acceptance length benchmarks show?

Acceptance length is the average number of tokens the target model accepts per verification round. Higher means fewer slow passes through the 27B model. Tencent's general benchmark numbers, run at xhigh reasoning with at most 4,096 new tokens:

Three patterns stand out.

• Structured tasks gain the most. Math and code are predictable, so long drafts survive. GSM8K at greedy decoding improves by about 27%.
• Sampling narrows the gap. At temperature 1.0 (with top_p=0.95, top_k=20), every task gains less. MT-Bench goes from 3.89 to 4.03, about 4%.
• Open-ended chat is the hardest case. MT-Bench has the lowest acceptance for both methods.

Agent acceptance length

The agent tests replay recorded tool-decision prefixes from 20 conversations per subset, with temperature sampling.

DSpine leads in every cell. WorkBuddy Office at low reasoning shows the biggest jump, from 4.24 to 5.63.

How much throughput does DSpine add on agent workloads?

Throughput here is total output tokens divided by end-to-end concurrent replay wall time. The setup uses SGLang 0.5.22, BF16, FlashAttention 3, and concurrency 8. Speedups are relative to AR.

Dividing DSpine by DFlash2 in each row gives gains from about 8% (SWE-bench Pro and τ²-bench at low) to about 21% (WorkBuddy Web and Office at low). Most rows land between 11% and 16%. That is consistent with Tencent's reported average of about 11% more throughput than DFlash2, though the exact average depends on how the paper weights tasks.

Notice that acceptance length gains are larger than throughput gains. WorkBuddy Office at low improves acceptance by about 33% but throughput by about 21%. Longer drafts cost more to produce and verify, so not every accepted token converts to wall-clock savings.

What did the hands-on H100 test show?

A creator ran DSpine locally on a single 80 GB H100 with SGLang. The script sent eight different prompts one at a time to Qwen3.8-27B and reported tokens per second.

With DSpine, individual prompts ranged from about 95 to 235 tokens/s. That is a speedup of roughly 4.7 to 4.8x. The draft model and its cache cost about 7 GB of extra VRAM in that configuration. The creator also reported that the setup did not work on an A100 and that an H100 was required.

Why is the hands-on speedup larger than the published one?

The test conditions differ in at least three ways, and the sources do not fully explain the gap.

• Concurrency. The published agent numbers use concurrency 8. The hands-on run sent prompts one at a time, which is the regime where speculative decoding helps most, since the GPU is otherwise underused.
• Workload. The published agent tables replay tool-calling traces. The hands-on run used eight generic prompts whose content was not detailed.
• Baseline. The 33 tokens/s AR figure is a single-stream number. The AR figures in the agent table are aggregate throughput across concurrent requests.

Treat 4.8x as a best-case single-user result and 2.7x to 3.8x as a more realistic range for batched agent traffic. Neither is a guarantee for your prompts.

Is the DSpine vs DFlash2 comparison fair?

Mostly, with caveats worth knowing before you rely on the margins.

• DSpine uses a 16-token draft block. The official DFlash2 checkpoint uses 8. A longer block raises the acceptance ceiling.
• DSpine inference uses dynamic verification, which references DSpark, and has ReplaySSM enabled.
• The benchmarks come from the authors of DSpine, using their own harness. Independent replication on other hardware would add confidence.

None of this makes the results wrong. It means the gap reflects the whole package (architecture, block size, and verification), not the architecture alone.

How do you run DSpine?

Install from the repository, then serve with SGLang using the DSPINE speculative algorithm:

git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e .

The model card's serve command sets --speculative-algorithm DSPINE, --speculative-num-draft-tokens 16, the FlashAttention 3 backends, --enable-linear-replayssm-spec, --mem-fraction-static 0.80, --context-length 131072, and --max-running-requests 8, among other flags. Copy the full command from the model card rather than assembling it piecemeal. The base model is Qwen/Qwen3.8-27B, and the draft path points at the downloaded Qwen3.8-27B-DSpine-b16 checkpoint.