How Do You Run Carbon-X, the 14B DNA Language Model, Locally?

How do you run Carbon-X locally?

Install Transformers v5 or later (or vLLM), load HuggingFaceBio/Carbon-X in bfloat16 on a single 80 GB GPU, and pass trust_remote_code=True for the tokenizer. Paste raw DNA straight into a chat message. The tokenizer detects it and converts it to 6-mer tokens. Set enable_thinking=False, because the model was post-trained and evaluated that way. For an API-style setup, run vllm serve HuggingFaceBio/Carbon-X --trust-remote-code and query it with any OpenAI client.

What is Carbon-X?

Carbon-X comes from Hugging Face's Carbon family. It starts from Qwen3-14B, adds DNA tokens to the vocabulary, and trains on 302B tokens of mid-training data. That mix is 55% raw DNA, 20% English, 15% biology text and 10% instruction data. Supervised fine-tuning and reinforcement learning with verifiable rewards on DNA tasks follow.

You ask questions in plain English with a sequence embedded in the message. It can classify variants as benign or pathogenic, identify regulatory regions, classify sequences by organism group, and predict the next bases of a sequence. It explains its answer in text, and its replies can include DNA too.

What hardware do you need?

The card says the weights fit on a single 80 GB GPU in bfloat16. Parameter count alone puts the raw weights around 30 GB, so the rest of that budget is headroom for the KV cache and long prompts. The card's cost figures assume H200 throughput at $5 per hour, which tells you the intended class of hardware.

The card also does not mention GGUF builds or other quantized releases. If you were hoping to run this through a llama.cpp-style stack on a consumer card, nothing in the documentation supports that today. Stick with Transformers or vLLM.

How do you load Carbon-X with Transformers?

Upgrade first. The card says Carbon-X requires Transformers v5 or later.

pip install -U transformers

Then load the model and tokenizer. Only the tokenizer needs remote code.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "HuggingFaceBio/Carbon-X"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")

Build a chat message, apply the template with thinking off, and generate:

messages = [{"role": "user", "content": (
    "Reference: ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG\n"
    "Variant:   ATGGTGCATCTGACTCCTGTGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG\n\n"
    "Is this variant likely to be benign or pathogenic? Explain your reasoning."
)}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True,
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

generate() picks up the sampling defaults from generation_config.json. Those are the same settings the authors used for every reported result, so leave them alone unless you have a reason.

The first tokenizer load needs Hub access, because it downloads the pinned BPE vocabulary it builds on. Plan for that if you work on an air-gapped machine. Pre-fetch the files on a connected one.

How do you serve Carbon-X with vLLM?

vLLM is the better route for batch jobs and for anything an agent will call.

pip install -U vllm
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code

The server exposes an OpenAI-compatible endpoint. Turn thinking off through chat_template_kwargs:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="HuggingFaceBio/Carbon-X",
    messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)

vLLM applies the same generation_config.json sampling defaults unless your request overrides them.

How does the DNA tokenizer behave?

This is where most surprises will come from. The tokenizer switches between BPE for text and 6-mers for DNA, so one DNA token covers six base pairs.

>>> ids = tokenizer("Describe this sequence: ACGTACGATTCA")["input_ids"]
>>> tokenizer.convert_ids_to_tokens(ids)
['Describe', 'Ġthis', 'Ġsequence', ':', 'Ġ', '<dna>', 'ACGTAC', 'GATTCA', '</dna>']

What to know:

• Auto-detection. Runs of at least 6 A/C/G/T bases get wrapped in <dna> and </dna>. Runs of at least 12 bases with up to 20% IUPAC ambiguity codes (N, R, Y) are tagged as well.
• Cleanup. Lowercase bases are uppercased, and sequences printed in space-separated blocks are joined.
• Manual control. Text you wrap in <dna>…</dna> is always read as DNA. Pass auto_dna_tags=False to the tokenizer or to apply_chat_template() to switch detection off for a call.
• Ambiguity codes cost resolution. Only A, C, G and T form 6-mers. A 6-mer containing N becomes a single <oov> token, so filter to canonical bases first if you want full resolution.
• Any length. If a sequence is not a multiple of 6, the last 1 to 5 bases use dedicated tail tokens, and decoding returns the original sequence exactly.
• Output tags. The model writes DNA inside <dna> tags. Decoding with skip_special_tokens=True strips them.

One practical consequence: token count is roughly bases divided by six, so the 40,960-token window holds far longer sequences than a character-level model would.

How should you prompt it?

The card gives three tips.

1. Put the sequence and the question in the same user message.
2. Add context you already have: gene, strand, coordinates, annotations. Without it, the model may guess a sequence's identity or function.
3. For variant questions, supply the reference and alternate sequences, or a reference with the position and base change.

Keep max_new_tokens modest. Carbon-X answers directly without extended reasoning, which is a large part of why its per-problem cost is low.

How does Carbon-X perform?

On five DNA benchmarks with thinking disabled, Carbon-X averages 66.7%. The card puts that 27 points above Qwen3-14B and ahead of every other open model it tested. GPT-6.1 Sol averages 67.5%, and the card says Carbon-X costs about 1/1,800 as much per problem.

It does not win everywhere. Frontier models lead on complex variants (insertions and deletions) and on single-base variants in long context. Within GenomeQA, splice sites (29.8%) and histone marks (34.6%) are weak, while organism group (93.2%) and motifs (90.0%) are strong. These are the authors' own evaluations, so treat them as a starting point and test on your data.

How do you use Carbon-X as a tool inside an agent?

The model card describes a pattern where a general-purpose assistant calls Carbon-X whenever a question involves DNA. You serve Carbon-X with vLLM, then wrap it in a function that takes a question, a dictionary of named sequences and optional context. The card's wrapper builds one prompt from those pieces and sends it to the local endpoint through the OpenAI client.

The point of the design is a division of labor. The general agent handles planning, file access and writing, while Carbon-X answers the DNA-specific sub-questions. Any coding agent that can run a Python function or shell command can use the same wrapper as a tool or skill. Because the endpoint is local, the sequences never leave your machine. Keep thinking disabled in the wrapper's request, and pass gene and coordinate context through when you have it.

What are the limits?

Carbon-X is a research model, and its benchmark scores are well short of perfect. The card's cost estimate for the ClinVar variants of uncertain significance (about $300 versus $32,000 to $4 million for frontier models) is the authors' estimate. It is not a clinical validation. Verify outputs before relying on them for anything consequential.