--- title: "How Do You Run Carbon-X, the 14B DNA Language Model, Locally?" url: "https://goremy.ai/blog/articles/carbon-x-dna-language-model-run-locally" author: Luis Chavez-Mattos editor: Luis Chavez-Mattos editor_title: Director, Product published: "2026-10-24T10:20:10.135Z" updated: "2026-10-24T10:20:10.135Z" topic: Open Model Run Guides description: Carbon-X is a 14.8B open DNA and text model that fits on one 80 GB GPU. Set up Transformers v5 or vLLM, handle DNA tokens, and call it from an agent. --- # How Do You Run Carbon-X, the 14B DNA Language Model, Locally? > Carbon-X is a 14.8B open DNA and text model that fits on one 80 GB GPU. Set up Transformers v5 or vLLM, handle DNA tokens, and call it from an agent. Edited by Luis Chavez-Mattos, Director, Product · Published 2026-10-24 ## How do you run Carbon-X locally? Install Transformers v5 or later (or vLLM), load `HuggingFaceBio/Carbon-X` in bfloat16 on a single 80 GB GPU, and pass `trust_remote_code=True` for the tokenizer. Paste raw DNA straight into a chat message. The tokenizer detects it and converts it to 6-mer tokens. Set `enable_thinking=False`, because the model was post-trained and evaluated that way. For an API-style setup, run `vllm serve HuggingFaceBio/Carbon-X --trust-remote-code` and query it with any OpenAI client. ## TL;DR - Carbon-X is a **14.8B-parameter decoder-only model** trained on DNA and English text together, with no separate DNA encoder. - The weights **fit on one 80 GB GPU in bfloat16**, so sensitive or unpublished sequences can stay on your own hardware. - You need **Transformers v5 or later** and `trust_remote_code=True` on the tokenizer, because the DNA tokenizer is custom code. - Keep **thinking disabled**. The model was post-trained and benchmarked that way, and the card recommends it. - The tokenizer **auto-tags DNA** (runs of A/C/G/T) and encodes it as non-overlapping 6-mers. Characters like `N` inside a 6-mer collapse into an `` token. - The model card documents Transformers and vLLM only. It does not describe GGUF or other quantized formats, so plan on the full bf16 weights. ## What is Carbon-X? Carbon-X comes from Hugging Face's Carbon family. It starts from Qwen3-14B, adds DNA tokens to the vocabulary, and trains on 302B tokens of mid-training data. That mix is 55% raw DNA, 20% English, 15% biology text and 10% instruction data. Supervised fine-tuning and reinforcement learning with verifiable rewards on DNA tasks follow. You ask questions in plain English with a sequence embedded in the message. It can classify variants as benign or pathogenic, identify regulatory regions, classify sequences by organism group, and predict the next bases of a sequence. It explains its answer in text, and its replies can include DNA too. | Spec | Value | |---|---| | Parameters | 14.8B | | Layers / hidden size | 40 / 5120 | | Attention | GQA, 40 query heads, 8 KV heads | | Context window | 40,960 tokens | | Vocabulary | 157,184 (151,669 BPE + 5,515 DNA) | | Precision | bfloat16 | | Default sampling | temperature 0.6, top-p 0.95, top-k 20 | ## What hardware do you need? The card says the weights fit on a single 80 GB GPU in bfloat16. Parameter count alone puts the raw weights around 30 GB, so the rest of that budget is headroom for the KV cache and long prompts. The card's cost figures assume H200 throughput at $5 per hour, which tells you the intended class of hardware. The card also does not mention GGUF builds or other quantized releases. If you were hoping to run this through a llama.cpp-style stack on a consumer card, nothing in the documentation supports that today. Stick with Transformers or vLLM. ## How do you load Carbon-X with Transformers? Upgrade first. The card says Carbon-X requires Transformers v5 or later. ```bash pip install -U transformers ``` Then load the model and tokenizer. Only the tokenizer needs remote code. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "HuggingFaceBio/Carbon-X" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda") ``` Build a chat message, apply the template with thinking off, and generate: ```python messages = [{"role": "user", "content": ( "Reference: ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG\n" "Variant: ATGGTGCATCTGACTCCTGTGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG\n\n" "Is this variant likely to be benign or pathogenic? Explain your reasoning." )}] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True, ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=1024) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) ``` `generate()` picks up the sampling defaults from `generation_config.json`. Those are the same settings the authors used for every reported result, so leave them alone unless you have a reason. The first tokenizer load needs Hub access, because it downloads the pinned BPE vocabulary it builds on. Plan for that if you work on an air-gapped machine. Pre-fetch the files on a connected one. ## How do you serve Carbon-X with vLLM? vLLM is the better route for batch jobs and for anything an agent will call. ```bash pip install -U vllm vllm serve HuggingFaceBio/Carbon-X --trust-remote-code ``` The server exposes an OpenAI-compatible endpoint. Turn thinking off through `chat_template_kwargs`: ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") response = client.chat.completions.create( model="HuggingFaceBio/Carbon-X", messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}], extra_body={"chat_template_kwargs": {"enable_thinking": False}}, ) print(response.choices[0].message.content) ``` vLLM applies the same `generation_config.json` sampling defaults unless your request overrides them. ## How does the DNA tokenizer behave? This is where most surprises will come from. The tokenizer switches between BPE for text and 6-mers for DNA, so one DNA token covers six base pairs. ```python >>> ids = tokenizer("Describe this sequence: ACGTACGATTCA")["input_ids"] >>> tokenizer.convert_ids_to_tokens(ids) ['Describe', 'Ġthis', 'Ġsequence', ':', 'Ġ', '', 'ACGTAC', 'GATTCA', ''] ``` What to know: - **Auto-detection.** Runs of at least 6 A/C/G/T bases get wrapped in `` and ``. Runs of at least 12 bases with up to 20% IUPAC ambiguity codes (`N`, `R`, `Y`) are tagged as well. - **Cleanup.** Lowercase bases are uppercased, and sequences printed in space-separated blocks are joined. - **Manual control.** Text you wrap in `…` is always read as DNA. Pass `auto_dna_tags=False` to the tokenizer or to `apply_chat_template()` to switch detection off for a call. - **Ambiguity codes cost resolution.** Only A, C, G and T form 6-mers. A 6-mer containing `N` becomes a single `` token, so filter to canonical bases first if you want full resolution. - **Any length.** If a sequence is not a multiple of 6, the last 1 to 5 bases use dedicated tail tokens, and decoding returns the original sequence exactly. - **Output tags.** The model writes DNA inside `` tags. Decoding with `skip_special_tokens=True` strips them. One practical consequence: token count is roughly bases divided by six, so the 40,960-token window holds far longer sequences than a character-level model would. ## How should you prompt it? The card gives three tips. 1. Put the sequence and the question in the same user message. 2. Add context you already have: gene, strand, coordinates, annotations. Without it, the model may guess a sequence's identity or function. 3. For variant questions, supply the reference and alternate sequences, or a reference with the position and base change. Keep `max_new_tokens` modest. Carbon-X answers directly without extended reasoning, which is a large part of why its per-problem cost is low. ## How does Carbon-X perform? On five DNA benchmarks with thinking disabled, Carbon-X averages 66.7%. The card puts that 27 points above Qwen3-14B and ahead of every other open model it tested. GPT-6.1 Sol averages 67.5%, and the card says Carbon-X costs about 1/1,800 as much per problem. | Benchmark | Carbon-X | GPT-6.1 Sol | Opus 5.5 | |---|---:|---:|---:| | BioReason Coding | **74.9** | 71.0 | 70.3 | | BioReason Non-SNV | 85.1 | 91.7 | **94.8** | | GenomeQA | 63.6 | **67.2** | 62.2 | | NextBase | **39.4** | 32.9 | 1.1 | | FrontierVEP | 70.4 | 74.9 | **76.2** | It does not win everywhere. Frontier models lead on complex variants (insertions and deletions) and on single-base variants in long context. Within GenomeQA, splice sites (29.8%) and histone marks (34.6%) are weak, while organism group (93.2%) and motifs (90.0%) are strong. These are the authors' own evaluations, so treat them as a starting point and test on your data. ## How do you use Carbon-X as a tool inside an agent? The model card describes a pattern where a general-purpose assistant calls Carbon-X whenever a question involves DNA. You serve Carbon-X with vLLM, then wrap it in a function that takes a question, a dictionary of named sequences and optional context. The card's wrapper builds one prompt from those pieces and sends it to the local endpoint through the OpenAI client. The point of the design is a division of labor. The general agent handles planning, file access and writing, while Carbon-X answers the DNA-specific sub-questions. Any coding agent that can run a Python function or shell command can use the same wrapper as a tool or skill. Because the endpoint is local, the sequences never leave your machine. Keep thinking disabled in the wrapper's request, and pass gene and coordinate context through when you have it. ## What are the limits? Carbon-X is a research model, and its benchmark scores are well short of perfect. The card's cost estimate for the ClinVar variants of uncertain significance (about $300 versus $32,000 to $4 million for frontier models) is the authors' estimate. It is not a clinical validation. Verify outputs before relying on them for anything consequential. ## Frequently Asked Questions ### Does Carbon-X run on a consumer GPU? The card only states that the weights fit on a single 80 GB GPU in bfloat16. It documents no quantized or GGUF builds, so smaller cards are not a supported path based on the available documentation. ### Why do I need trust_remote_code? The model uses a standard causal LM architecture, but the hybrid DNA tokenizer is custom code. `trust_remote_code=True` lets Transformers load it. With vLLM, the equivalent is the `--trust-remote-code` flag. ### Should I enable thinking mode? No. Carbon-X was post-trained, reinforcement-trained and evaluated with `enable_thinking=False`, and the card recommends keeping it off. Pass the flag in `apply_chat_template()` or through `chat_template_kwargs` on vLLM. ### What happens to sequences with N bases? Only A, C, G and T form valid 6-mers. A 6-mer containing any other character becomes one `` token, which loses information. Filter or resolve ambiguous bases first if resolution matters. ### Can I change the sampling settings? Yes. Both Transformers and vLLM default to temperature 0.6, top-p 0.95 and top-k 20 from `generation_config.json`. You can override them per request, but those defaults produced the published results.