How Accurate Is Carbon-X on DNA Tasks vs GPT and Opus?

How accurate is Carbon-X on DNA tasks compared with frontier models?

Carbon-X, an open 14.8B-parameter DNA language model, averages 66.7% across five DNA benchmarks with thinking disabled. GPT-6.1 Sol scores 67.5% and Opus 5.5 scores 60.9%. Carbon-X leads every tested model on BioReason Coding and NextBase, but trails the top frontier models on BioReason Non-SNV, GenomeQA and FrontierVEP. It also performs poorly on some sub-tasks, notably splice sites, and it has no clinical validation.

What is Carbon-X?

Carbon-X is a decoder-only model from the Carbon family, released with open weights by Hugging Face's bio team. You paste raw DNA into an ordinary chat message and it answers in English. It can classify variants as benign or pathogenic, identify regulatory regions, classify sequences by organism group, predict the next bases, and explain its reasoning.

DNA and text share one token stream. A hybrid tokenizer keeps a BPE vocabulary for text and encodes DNA as non-overlapping 6-mers, so one token covers 6 base pairs. There is no separate DNA encoder. The context window is 40,960 tokens, and the weights fit on a single 80 GB GPU in bfloat16.

How does Carbon-X score on each benchmark?

Carbon-X results use thinking disabled, default sampling and one sample per problem. Each comparison model appears at the reasoning effort that gave it the highest average. Best scores are in bold.

Two patterns stand out. Against open models, Carbon-X is far ahead on every benchmark. Against the two strongest frontier models, it wins on coding variants and next-base prediction, and gives up 4 to 10 points elsewhere. The gap is widest on Non-SNV variants (insertions, deletions and other non-single-nucleotide changes), where Opus 5.5 leads by 9.7 points.

Some frontier numbers are extreme: Gemini 4 Argon at 6.5 on BioReason Coding and Opus 5.5 at 1.1 on NextBase. The model card gives no explanation for either, so treat those cells as measured results rather than evidence of a general weakness.

What do the benchmarks actually measure?

• BioReason Coding (1,018 variants, balanced accuracy): predict benign or pathogenic from original and mutated protein-coding sequences. The gene name is removed from the prompt.
• BioReason Non-SNV (873 variants, balanced accuracy): the same call for insertions, deletions and similar changes.
• GenomeQA (2,600 questions, accuracy averaged over six tasks): multiple choice on enhancer vs. promoter, splice sites, organism group, histone marks, transcription-factor binding sites and motifs.
• NextBase (600 sequences, per-base accuracy): predict the next bases of a sequence.
• FrontierVEP (1,000 variants, accuracy): given a reference sequence up to 8,192 bases and a single-nucleotide variant, call it benign or pathogenic. The set is 500 benign and 500 pathogenic.

BioReason and GenomeQA come from the literature. The Carbon team built FrontierVEP and NextBase. That is worth remembering when reading the NextBase lead, since the task design and prompt format were chosen by the same group that trained for it. The team says training data was decontaminated against all benchmarks using whole-span DNA overlap, text n-gram overlap and sequence homology search.

How much cheaper is Carbon-X than GPT and Opus?

Carbon-X answers directly without extended reasoning, so each problem uses few output tokens. The authors put its inference cost at about 1/1,800 of GPT-6.1 Sol per problem. They compute Carbon-X cost from measured throughput per H200-hour at $5/hour, and frontier costs from standard API pricing at each model's best-scoring reasoning effort.

Their headline scenario is ClinVar, which holds about 2.4 million variants of uncertain significance:

Two caveats apply. The frontier figures reflect maximum-effort reasoning settings that produce the highest scores, so cheaper settings would narrow the gap while lowering accuracy. And the Carbon-X figure assumes you run your own GPU at good utilization. It is an estimate, not a bill.

How was Carbon-X trained?

The recipe has four stages.

1. Vocabulary extension. It starts from Qwen3-14B and adds 5,515 DNA tokens with random embeddings. About 10B tokens of training run with the transformer body frozen so only the new embeddings learn.
2. Mid-training. 302B tokens at sequence length 8,192. The mix is 55% raw DNA, 20% English, 15% biology text and 10% instruction data. The authors report that learning rate was the dominant knob: too low leaves DNA at chance, too high forgets English.
3. Supervised fine-tuning. Three epochs over 20 dataset subsets. About 30% is variant-effect reasoning over several views of the same variant. The rest covers sequence classification, basic DNA operations, next-base prediction, biology Q&A and general chat.
4. Reinforcement learning. Asynchronous GRPO with verifiable, code-checked rewards in two rounds, thinking disabled, 16 samples per prompt. The variant-effect reward penalizes ungrounded rationales.

Post-training used TRL with vLLM for generation. The model also keeps general ability, scoring 79.3% prompt-level strict accuracy on IFEval.

Where does Carbon-X fall short?

The GenomeQA breakdown shows the uneven spots:

Splice-site detection and histone-mark prediction are weak. These are multiple-choice tasks, so low scores sit well below what you would want from a tool people might rely on for regulatory biology.

Other limits to keep in mind:

• Not clinically validated. Benign or pathogenic calls on benchmark sets do not establish clinical reliability. Treat outputs as research triage, not diagnosis.
• Frontier models still lead on complex variants. Non-SNV and long-context single-base variants (FrontierVEP) favor GPT-6.1 Sol and Opus 5.5.
• Thinking is off by design. It was post-trained and evaluated with thinking disabled, and the authors recommend keeping it that way.
• Context matters. Without gene, strand, coordinates or annotations, the model may guess at a sequence's identity or function.
• Ambiguous bases lose resolution. A 6-mer containing N or any non-ACGT character becomes a single <oov> token, so filter to canonical bases first.
• Single-sample evaluation. Every number is one sample per problem, so small differences, such as the 0.8-point average gap, should not be over-read.

Is Carbon-X worth using?

It fits when you need to screen many variants or sequences cheaply, or when sequences are sensitive or unpublished and must stay on your own hardware. A single 80 GB GPU can serve it with vLLM, and it works through a standard chat interface, so it can also be wrapped as a tool that a general-purpose agent calls for DNA questions.

It fits less well when the task is splice-site or histone-mark prediction, complex indels, or anything where a clinical decision follows. In those cases a frontier model or a specialized tool is the safer baseline, and any result needs independent validation.