---
title: "How to Run Clover-35B-A3B-Preview Locally: VRAM and Settings"
url: "https://goremy.ai/blog/articles/run-clover-35b-a3b-preview-locally"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-10-30T05:20:05.633Z"
updated: "2026-10-30T05:20:08.959Z"
topic: Open Model Run Guides
description: "Run DiffuSpace's Clover-35B-A3B diffusion model locally: about 69 GB of BF16 weights, Transformers 5.19.0 setup, block diffusion defaults, controls and SGLang."
---

# How to Run Clover-35B-A3B-Preview Locally: VRAM and Settings

> Run DiffuSpace's Clover-35B-A3B diffusion model locally: about 69 GB of BF16 weights, Transformers 5.19.0 setup, block diffusion defaults, controls and SGLang.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-10-30

## What do you need to run Clover-35B-A3B-Preview locally?

You need a GPU with enough memory for the BF16 weights, which the model card puts at roughly 69 GB, plus room for inference state. You also need PyTorch 2.10 or later with CUDA, Transformers 5.19.0, accelerate, and safetensors. In one hands-on test with a custom Transformers script, VRAM use sat just under 68 GB. Treat that as a data point, not a guarantee, since context length and generation settings change the total. For a smaller model that fits on one 80 GB GPU, see how to [run Carbon-X locally](https://goremy.ai/blog/articles/carbon-x-dna-language-model-run-locally).

## TL;DR

- Clover-35B-A3B-Preview is a **35B mixture-of-experts diffusion model** with about 3B active parameters per token and a 128K context window.
- The BF16 weights need **about 69 GB of GPU memory** before inference state, so plan for a single large card or more headroom than that.
- Install with **Transformers 5.19.0** and PyTorch 2.10 or later, and load with `trust_remote_code=True`.
- The defaults (**block_size 32, threshold 0.95**) and the repetition and length controls are already configured, so most users should not touch them.
- The coreset setting limits which experts run per block but **does not shrink the weights** you have to load.
- DiffuSpace **recommends its official SGLang implementation** for serving and evaluation. The Hugging Face path is a standalone PyTorch route.

## What is Clover-35B-A3B-Preview?

Clover is the first model from DiffuSpace. It is a text-only diffusion language model derived from Qwen3.6-35B-A3B. It keeps that model's hybrid Gated DeltaNet and full-attention MoE backbone: 40 layers, 256 routed experts, top-8 routing, and one shared expert per layer.

The difference is how it writes. A normal model emits one token at a time. Clover fills in a block of text in parallel, then moves to the next block. DiffuSpace added learned GDN parameters (including right-convolution weights and softmax controls) so tokens can attend to each other inside a noisy block while the prefix stays causal.

Context is up to 131,072 tokens, shared between the prompt and the generated response.

## How much VRAM does Clover-35B-A3B-Preview need?

The model card says approximately 69 GB for BF16 weights, plus inference state. Only 3B parameters are active at one point in time, but all 35B parameters still have to be loaded. Active-parameter count helps compute, not weight storage.

| Item | Figure | Source |
| --- | --- | --- |
| Total parameters | 35B | Model card |
| Active per token | ~3B | Model card |
| BF16 weights | ~69 GB | Model card |
| Observed VRAM in a custom Transformers run | under 68 GB | Creator demo |
| Context window | 131,072 tokens (prompt + response) | Model card |

The observed figure is slightly below the card's number. That can happen depending on what is counted and when it is measured. Long contexts add state, so budget above the weight size. The sources give no quantized variants or minimum-hardware list, so do not assume a smaller card will work.

## How do you install and load it with Transformers?

Install PyTorch with CUDA first, then the Hugging Face dependencies:

```bash
pip install "transformers==5.19.0" accelerate safetensors
```

Then load the model in BF16 onto one GPU:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Diffuspace/Clover-35B-A3B-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="cuda:0",
).eval()
```

Build the prompt with the chat template and `enable_thinking=True`, tokenize with `add_special_tokens=False`, and call `generate()` with `max_new_tokens=24576`, `block_size=32`, `threshold=0.95`, and `do_sample=False`. You can point `model_id` at a local directory instead. Defaults load from `generation_config.json`, and any keyword argument you pass to `generate()` overrides them.

Requests that exceed the 128K limit are rejected. That includes the space needed to resolve the final full block.

## How does block diffusion decoding work?

Each block starts as masked tokens. The model resolves positions whose confidence is above the threshold, in parallel. If none qualify, it fills the single most confident position so decoding always progresses. Once a block is resolved, it joins the prefix for the next block. The card calls this LowConfidence decoding.

Two settings control it:

- `block_size=32`: tokens per block.
- `threshold=0.95`: the confidence needed to fill a position in a pass.

A lower threshold would fill more positions per pass, but the card does not report the quality or speed effects, so test before changing it.

## What generation controls are set by default?

The card notes that very long generations can become repetitive. The model ships with repetition controls and gradual biases toward ending the response or the thinking phase. They also apply during greedy decoding.

| Parameter | Default | Purpose |
| --- | --- | --- |
| `presence_penalty` | 0.9 | Discourages tokens already in the response |
| `frequency_penalty` | 0.0 | Count-based penalty, off by default |
| `repetition_penalty` | 1.05 | Lowers preference for earlier tokens; 1.0 disables |
| `dllm_length_penalty` | 10.0 | Gradually favors end-of-response tokens |
| `dllm_think_end_bias` | 10.0 | Gradually encourages `</think>` |

Length control starts at 8192 response tokens and reaches full strength over 8192 more (`dllm_length_penalty_start`, `dllm_length_penalty_window`). Thinking control starts at 4096 tokens with a 4096-token window (`dllm_think_end_start`, `dllm_think_end_window`). These are soft nudges. `max_new_tokens` is the hard cap. Repetition controls look only at generated text, not the prompt.

## What is MoE coreset expert selection?

During block denoising, Clover can restrict routing to a shared subset of experts for each block. That means fewer distinct experts are used across the block.

| Setting | Default |
| --- | --- |
| `moe_block_coreset_enabled` | `true` |
| `moe_block_coreset_top_p` | `0.9` |

The selector sums the routing scores of each token's top-8 candidates across the block, ranks experts, and keeps enough to cover the requested share of that mass. A value of 0.9 means 90% of aggregated routing mass, not 90% of experts. Lower values pick fewer experts and may cost quality. This is separate from sampling `top_p`, and it does not reduce the size of the loaded weights, so it will not lower your VRAM floor.

## Should you use SGLang instead of Transformers?

DiffuSpace says it recommends its official SGLang implementation for serving and evaluation. The Hugging Face version is a standalone PyTorch path, and small numerical differences between its operations and SGLang's kernels can produce different text even with identical decoding settings. If you compare outputs or benchmark, stick to one backend.

Self-speculative decoding is available in the SGLang and llama.cpp implementations. Setup steps and supported decoding modes live in the official GitHub repository, so follow those rather than a third-party recipe. The sources do not describe llama.cpp setup or performance in detail.

## Is Clover-35B-A3B-Preview worth running?

It depends on your goal. DiffuSpace claims state-of-the-art results among open-source diffusion language models, and its chart shows leads on six agentic benchmarks. An independent hands-on test was less kind. In that run, the model failed to fix a bug in a Docker-based app, produced an average single-file HTML animation, skipped many languages in an 80-plus-language naming task with stray Chinese characters in other outputs, and got a Zebra logic puzzle half right after a very long run (over 50 minutes).

That is one creator's informal testing on a preview release, not a benchmark. It does suggest checking your own workload before committing 69 GB of VRAM. For researchers interested in diffusion language models, it is a notable open-weight release under an Apache-2.0-derived license.

## Frequently Asked Questions

### Can I run Clover-35B-A3B-Preview on a 24 GB GPU?

The sources do not support that. The BF16 weights alone are about 69 GB, and the card lists no quantized builds. Only about 3B parameters are active per token, but all weights must still be loaded.

### Do I need to change the repetition settings?

No. The card says the defaults are already configured and no extra arguments are needed for normal use. They apply even with greedy decoding. Pass keyword arguments to `generate()` only if you want to override them.

### Does lowering moe_block_coreset_top_p save memory?

No. It reduces how many distinct experts are used within a block, which may affect quality. It does not reduce the size of the stored or loaded model weights.

### Why does my output differ from SGLang results?

The Transformers path and SGLang use different kernels, and small numerical differences can change generated text even with the same settings. DiffuSpace recommends SGLang for serving and evaluation.

### What is the maximum context length?

Up to 131,072 tokens, covering the prompt and the generated response together. Requests over the limit are rejected, including space for the final full block.