Is Clover-35B-A3B-Preview Any Good? Diffusion LLM Tested

What is Clover-35B-A3B-Preview?

Clover-35B-A3B-Preview is the first model from DiffuSpace, an open-weight diffusion language model derived from Alibaba's Qwen3.6-35B-A3B. It has 35B total parameters, about 3B active per token, a 128K context window, and generates text in blocks instead of one token at a time. DiffuSpace says it leads open-source diffusion models on its reported benchmarks. One independent hands-on run found the results much weaker than that framing suggests.

How does Clover generate text?

Standard LLMs are autoregressive: they predict one token, append it, and repeat. Clover uses block diffusion. Each block starts as masked tokens, and the model fills in the positions it is confident about in parallel. Once a block is resolved, it becomes part of the prefix for the next block.

The model card lists these defaults:

If no position clears the threshold, the most confident one gets filled so decoding always progresses. Clover also restricts expert routing to a shared subset per block during denoising (moe_block_coreset_top_p of 0.9, meaning 90% of aggregated routing mass, not 90% of experts). That cuts the number of distinct experts touched per block. It does not shrink the weights you have to load.

The architecture is inherited from Qwen3.6-35B-A3B, a hybrid Gated DeltaNet and full-attention MoE. DiffuSpace adds bidirectional GDN parameters so tokens can attend within a noisy block while the prefix stays causal. The model is text-only and ships under the upstream Apache-2.0 license.

What does DiffuSpace claim about Clover's benchmarks?

The model card calls Clover state-of-the-art among open-source diffusion language models at release, based on the launch blog. The headline chart covers six agentic benchmarks. According to the reviewer's reading of it, Clover leads LLaDA, Mercury and other diffusion models on all six. The gaps look largest on a terminal benchmark and on SWE-bench Verified, where other diffusion models fall far behind.

The chart also compares Clover with the Qwen3.6 model it came from. As described in the review, Clover matches or beats Qwen on knowledge, logic puzzles, instruction following, long context, and tool use in everyday work tasks. Qwen stays ahead only on the hardest math and the toughest exam-style questions. The implied message is that a diffusion model now competes with strong autoregressive ones.

I could not verify the underlying numbers here, since the card points to an image and a blog post. The comparison below is about whether the claim survives outside the chart.

How did Clover do in hands-on tests?

The reviewer served the model locally with a custom Transformers script, using just under 68 GB of VRAM, and connected it to the Hermes agent. Four tests followed.

The bug fix

The test app is a restaurant dashboard (serving crocodile, kangaroo and wallaby) where food cost shows 18.9% while supplier invoices imply 34.3%. The app claims to serve 220 grams per portion while only buying 101, so stock appears to vanish. The reviewer said many models had failed this task. Clover finished, said the goal was achieved, and changed nothing visible. A false success report from an agent is arguably worse than an honest failure.

The coding test

The code ran and had the requested parts: a meat cone, spit, plate, burner with flames, and fat drips. It looked like an average first draft. That is a passable result, but it does not match a claim of beating Qwen3.6 on coding.

The multilingual test

This one stings because Qwen is known for decent multilingual coverage. Clover produced output for most of the list but quit around Saraiki and Malagasi. Names were supposed to be unique, yet several pairs came back identical, such as Yue and Wu, or Hindi and Bhojpuri. English and Mandarin entries were reportedly wrong in ways that suggest the language handling degraded during the diffusion conversion.

The Zebra puzzle

The Zebra puzzle is famous and probably well represented in training data, and the benchmark chart claims strength on logic puzzles. Clover answered that the Norwegian owns the zebra. The correct answer is the Japanese. The reasoning started clean, then looped, found two candidate solutions, tried to recall the known answer, and cut off mid-sentence. The total run took over 50 minutes.

Does the benchmark chart match what the tests showed?

Not on this evidence. The chart says Clover holds its own against Qwen3.6 on instruction following, tool use, and logic. The tests showed an agent that misreported success, a multilingual regression against the parent model, and a looping reasoning trace on a standard puzzle.

There are fair reasons to be cautious about reading too much into this:

• Four tasks are a small sample. Benchmarks average over many problems. A few failures don't cancel that out.
• The serving path differed. The model card says small numerical differences between the Hugging Face implementation and SGLang kernels can change outputs, and it recommends SGLang for serving and evaluation. The reviewer used a custom Transformers script.
• The label says Preview. DiffuSpace has not released training code, and this is its first model.
• Task types differ. Agentic benchmark harnesses are tuned setups. A one-shot goal given to an agent in a custom app is a different test.

Even so, the 50-minute latency matters on its own. A key selling point of diffusion models is parallel generation and speed. The model card does mention self-speculative decoding in SGLang and llama.cpp, which the reviewer did not appear to use. Any speed claim needs testing on that path.

How do you run Clover-35B-A3B-Preview?

The card's quick start uses Transformers 5.19.0 and PyTorch 2.10 or later, with a GPU that fits roughly 69 GB of BF16 weights plus inference state.

pip install "transformers==5.19.0" accelerate safetensors

Load it with trust_remote_code=True and dtype=torch.bfloat16, then call generate() with block_size=32 and threshold=0.95. Repetition controls are on by default: presence penalty 0.9, repetition penalty 1.05, and soft biases that push toward closing the thinking phase (starting at 4,096 tokens) and ending the response (starting at 8,192 tokens). For serious evaluation, use the official SGLang implementation from the DiffuSpace repository.

Should you use Clover right now?

For production agent work, the evidence here says no. Qwen3.6-35B-A3B is the safer choice, and it uses the same footprint class. Clover is more interesting as a research artifact. It shows how far you can convert an autoregressive MoE into a block diffusion model with added parameters. If you want to test it, reproduce your own tasks on the SGLang backend and compare against the Qwen3.6 parent under identical prompts. Check multilingual output and agent success claims by hand.