Blog
DSpinespeculative decoding explainedDSpine vs DFlash

What Is DSpine? Speculative Decoding Explained Simply

How speculative decoding works, and how Tencent's DSpine differs from DFlash 2: a 2B helper model, 16-token blocks, and a head-to-head speed test on one H100.

Portrait of Luis Chavez-Mattos
Edited by Luis Chavez-Mattos, Director, Product··RSS

For Engineering and IT

An editor in her late 20s holds a small computer chip above her head with both arms and looks up at it, a short line of blank tiles lined up beside the chip.

What is DSpine?

DSpine is a small helper model from Tencent, about 2 billion parameters, that sits next to a large model (Qwen 27B in the tests covered here) and makes it answer faster. The helper guesses a block of 16 upcoming tokens at once. The big model then checks all 16 guesses in a single pass, keeps the correct ones, and fixes the first wrong one. The final answer is the same as before. It arrives sooner. This technique is called speculative decoding.

TL;DR

  • Speculative decoding uses a small, fast model to guess several tokens ahead, while the large model only verifies the guesses, so the output stays the same but arrives faster.
  • DSpine drafts 16 tokens per block, while the official DFlash 2 setting drafts 8.
  • DSpine's approach is for each position in the block to pass its current guess along at every layer, then pick tokens left to right from a ready-made lookup table. DFlash blends neighboring positions with small convolutions and picks a path at the end.
  • Tencent's model card reports DSpine accepting about 7.2 tokens per round versus 5.7 for DFlash on the GSM8K math benchmark.
  • In one independent test on an NVIDIA H100 (80 GB), DSpine generated about 150 tokens per second against 129 for DFlash 2, roughly 16% faster.
  • That test used one GPU and one request at a time, so treat it as a first look, not a final verdict.

What is speculative decoding?

A large language model writes text one token at a time. A token is a chunk of text, often a short word or part of a word. Each new token needs a full, slow pass through the whole model, and that pass is the bottleneck. A 27-billion-parameter model spends most of its time waiting on those passes.

Speculative decoding adds a small helper model that is much faster. The helper guesses a few tokens ahead. The big model then checks all of those guesses in one go. It keeps the guesses that match what it would have written and corrects the first one that does not. Everything after the first wrong guess is thrown away, and the process repeats from there.

Think of an editor reviewing a draft. Checking a paragraph someone else wrote is quicker than writing it from scratch. If the draft is mostly right, the editor approves most of it in one read and fixes the odd sentence. If the draft is poor, the editor still gets a correct result, but saves little time.

The key point is that quality does not change. The big model has the final say on every token, so the answer is the one it would have produced anyway. What changes is speed, and speed depends on how many guesses survive the check.

How does a helper model draft tokens for a big model?

The simplest helper drafts one token at a time, like a smaller version of the big model. That is faster than the big model, but still sequential.

DSpine and DFlash take a different route. They draft a whole block of tokens in one pass, all in parallel, instead of one after another. DSpine drafts 16 tokens per block. The official DFlash 2 setting drafts 8.

Parallel drafting is fast, but it creates a problem. If every position in the block is guessed at the same time, the guesses can disagree with each other. Position 3 might assume the sentence is heading one way while position 4 assumes another. A block of inconsistent guesses gets rejected early, and you gain little.

So the real design question is how to make the parallel guesses agree. DSpine and DFlash answer it differently.

How does DSpine differ from DFlash?

Both are small helper models that draft a block in one pass for the same large model. The difference is in how they keep the guesses consistent.

DSpine (Tencent)DFlash 2 (Z Lab)
Helper sizeAbout 2 billion parametersA small helper model
Block size16 tokens8 tokens (official setting)
Keeping guesses consistentAt every layer, each position passes its current guess to the next positionNeighboring positions blend inside the model using small convolutions
Final selectionTokens picked left to right using a ready-made lookup tableA path picker chooses the best path at the end

DSpine's approach. The helper still guesses the whole block at once. But at every layer inside the helper, each position hands its current guess to the next position. By the end, the positions have been talking to each other repeatedly. Then the model picks tokens from left to right using a ready-made lookup table, so each choice can account for the one before it.

DFlash's approach. Neighboring positions are blended inside the model using small convolutions. A convolution is a standard operation that mixes information from nearby positions. After that, a path picker chooses the best path through the options at the end.

If you want a mental picture: DSpine has the positions pass notes along the way, then a quick lookup resolves the final choices in order. DFlash smooths the neighbors together as it goes, then chooses a route at the end. That comparison is an analogy, not a technical description, but it captures where in the process each method does its consistency work.

What does "acceptance length" mean?

Acceptance length is the average number of drafted tokens the big model accepts per round. It is the main number to watch, because it tells you how much work each expensive pass of the big model gets done.

If the big model accepts 7 tokens per round, each slow pass produces roughly 7 tokens of output instead of 1. If it accepts 3, you get about 3. A bigger block only helps if the guesses within it are good enough to survive checking.

On the model card, Tencent reports DSpine accepting about 7.2 tokens per round, against 5.7 for DFlash on GSM8K, a benchmark of grade-school math problems. Those are Tencent's own figures, which is why an independent test is worth looking at.

Is DSpine actually faster than DFlash 2?

One creator, Fahad Mirza, ran a head-to-head comparison on the same hardware: an NVIDIA H100 with 80 GB of memory, served through SGLang, with the same Qwen 27B model, memory settings, context limit and request limit for both helpers. DFlash 2 used its own recommended setting of eight draft tokens.

The test script sent 16 different prompts, eight short and eight needing longer answers or reasoning, and ran them twice, for 32 requests. It used temperature zero, which makes results repeatable, and measured tokens per second for each request.

ResultDFlash 2DSpineDSpine advantage
Short promptsabout 133 tokens/secabout 152 tokens/secabout 14%
Longer promptsabout 128 tokens/secabout 150 tokens/secabout 16%
Overallabout 129 tokens/secabout 150 tokens/secabout 16%
Per-request range91 to 16888 to 227
Memory usedabout 73 GBjust over 78 GB

Both setups handled all 32 requests without errors. In an earlier test by the same creator, DSpine took Qwen 27B from about 33 tokens per second to about 156 on the same kind of H100, which gives a sense of the gain over no helper at all.

Two cautions. First, DSpine used more memory in this setup, since it adds a few extra serving flags. If you are tight on VRAM, that matters. Second, this was one GPU handling one request at a time. Servers handling many users at once can behave differently, and the creator himself called it a first look, not a final verdict. Some long answers also hit the token limit because the model spent its budget thinking, but thinking tokens are still tokens, so the speed numbers still count.

What does this mean for you?

Most people reading this will never serve a 27B model on an H100. Still, this affects you in a few ways.

If you only use AI tools. Techniques like this are part of why AI responses keep getting faster and cheaper to run without getting worse. Because the big model checks every token, speed gains of this kind do not change answer quality. You do not need to do anything.

If you pick or buy AI infrastructure. When a vendor quotes speed, ask what helper model and setup they use, and whether the numbers come from a single request or many at once. Speedups measured on one request at a time may not hold under heavy load. Also check memory: a faster setup that needs more VRAM can cost more to host.

If you run local models. If you already serve a large open model, a helper like DSpine or DFlash 2 is worth testing on your own prompts. Your results depend on your workload. Math-heavy or highly predictable text tends to make guesses easier, though the sources here only back up the benchmark and test numbers above.

If you evaluate vendor claims. Treat model-card numbers as the maker's own. Look for an independent run on the same hardware, as in the comparison above, before you plan around a figure.

Could you build your own model speed tracker?

If your team compares models or serving setups, the results usually end up scattered across screenshots and spreadsheets. A small internal app could log each run (model, helper, hardware, prompts, tokens per second, memory used) and show the comparisons side by side. Describing that tool in plain English to Remy, which builds and deploys full-stack apps from a description, would give you a shared page with a database and sign-in behind it, instead of another subscription or a file only one person understands. It is a modest project, but it keeps benchmark history in one place you own.

Frequently Asked Questions

5 questions

Does speculative decoding change the answers the model gives?

No. The big model verifies every drafted token and corrects the first wrong one, so the final output matches what it would have produced alone. The helper only affects how fast the answer arrives.

Why does DSpine draft 16 tokens when DFlash 2 drafts 8?

Those are the block sizes each setup uses. DSpine's design drafts 16 tokens per block, and the official DFlash 2 setting is 8. A larger block can pay off only if enough of the guesses are accepted, which is what acceptance length measures.

Is DSpine always faster than DFlash 2?

Not necessarily. In the head-to-head test on one H100 it was about 16% faster overall, but that was a single GPU handling one request at a time. Results can differ with other hardware, heavier loads or different prompts.

Do I need a powerful GPU to use DSpine?

In the test, DSpine ran on an 80 GB H100 and used just over 78 GB of memory, slightly more than DFlash 2. The sources do not cover smaller GPUs, so check the model card and your own hardware before planning around it.

What is the difference between a helper model and the main model?

The helper is small, about 2 billion parameters here, and is built for speed. The main model, Qwen 27B in these tests, is larger and does the final checking. The helper proposes, and the main model decides.

How we source, write, and correct our articles. Editorial standards

Related articles

Straight to your inbox.
New pieces as they publish. No noise.