Dory 1.0 4B Benchmarks: How Do 1, 2 and 4 Loops Compare?

How much do extra loops improve Dory1.0-4B?

Extra loops help a lot at first and then taper off. On NVIDIA's model card, going from one loop to two lifts SWE-bench Verified from 50.3 to 59.5 and AA-LCR from 63 to 71. Going from two to four adds smaller gains, reaching 61.8 on SWE-bench Verified and 73 on AA-LCR. Some reasoning and knowledge scores do not improve at all past two loops. NVIDIA recommends two loops as the default.

What is Dory1.0-4B?

Dory1.0-4B is an open-weight looped (latent-recurrent) language model from NVIDIA. It has 4B parameters, a normalized-GPT backbone, and interleaved sliding-window and full-attention layers. Context runs up to 256K tokens, and the model supports English only. The license is OpenMDW-1.1.

The layer stack splits into three groups. Input and output blocks have 8 layers each. A shared recurrent block of 20 layers sits between them and is applied several times, with the input reinjected on every loop. More loops mean more compute per token without more parameters.

How does looping work at inference time?

You pick the loop count when you serve the model. In vLLM, you set n_recurrent_loops in --hf-overrides. In llama.cpp, you set dory.num_loops to 1, 2, 3 or 4 through --override-kv. The same weights serve every setting.

The card also describes two KV cache modes:

• Per-loop caching stores a separate cache for each loop, so memory grows with loop count.
• Last-loop caching keeps one cache per recurrent attention layer, with each loop overwriting the last. The footprint stays at the one-loop size at any loop count.

NVIDIA says accuracy with last-loop caching stays close to per-loop mode, with nearly unchanged output lengths. The card also says looping reduces the number of generated tokens, since extra compute goes into latent thinking instead of longer outputs. It shows this in a chart but gives no exact figures in the text.

How do 1, 2 and 4 loops compare on the model card?

All numbers below come from the card's BF16 results, measured by NVIDIA under one harness.

Three patterns stand out.

Front-loaded gains. SciCode jumps 7.6 points from 1 to 2 loops, then adds 0.17. AA-LCR gains 8 points, then 2. GDPval gains 131 Elo, then 21.

Agentic tasks keep improving. Terminal-Bench 2.1 adds 5.6 points from 1 to 2 loops and another 4.1 from 2 to 4. τ³-bench Banking adds 1.3, then 1.8. This matches NVIDIA's advice to consider four loops for agentic work such as Terminal-Bench and SWE-bench.

Reasoning and knowledge plateau. GPQA Diamond and HLE peak at two loops and dip slightly at four. AA-Omniscience does not move. Extra compute per token does not add facts the model never learned.

How does Dory1.0-4B stack up against Qwen3.5 and Gemma 4?

Here is the four-loop result against the card's comparison models. Gemma 4 26B A4B and Qwen3.6-35B-A3B are larger models whose names indicate smaller active parameter counts, so they are not strict size peers.

† Reported score rather than one measured in NVIDIA's harness.

Where does Dory win?

• IFBench: 79 at four loops, ahead of Gemma 4 26B A4B (77.89) and far ahead of both Qwen3.5 models. Even one loop (74.29) beats every Qwen entry.
• AA-LCR: 73 at four loops tops all four comparison models. Two loops (71) also leads. One loop (63) trails Qwen3.5-4B and Qwen3.5-9B.
• GDPval-AA-V2: 967 Elo is well above Qwen3.5-4B (424), Qwen3.5-9B (546) and Gemma 4 (751.6). It sits just under Qwen3.6-35B-A3B (987).
• τ³-bench Banking: 14.69 is the best number in the row.
• SWE-bench Verified: 61.8 narrowly passes Qwen3.5-9B (61) and clears Gemma 4 (57.40).

Where does it trail?

• GPQA Diamond: 72.16 versus 77.71 for Qwen3.5-4B, the closest comparison on size.
• HLE: 10.43 is below every comparison model.
• AA-Omniscience: about 10 against 15 to 21 for the others. This is the clearest gap, and loops do not close it.
• Terminal-Bench 2.1: 22.2 is below Qwen3.5-9B (24), Gemma 4 (35) and Qwen3.6 (43), though it beats Qwen3.5-4B (15).
• SciCode: 35.11 beats both Qwen3.5 models but trails Gemma 4 (40.32) and Qwen3.6 (37.24).

Which loop count should you use?

NVIDIA's guidance is simple:

1. Start with two loops. It captures most of the accuracy gain.
2. Use one loop when compute per token matters most.
3. Try four loops when accuracy matters more, especially for agentic coding.

The card also notes that higher loop counts can reduce generated tokens. Per-token compute rises with loops, but total output may shrink, so measure end-to-end latency and cost on your own workload instead of assuming either direction.

What should you know before trusting these numbers?

These are vendor-measured results. NVIDIA says it used a consistent harness (NeMo Gym and the Nemo Evaluator SDK) and ran Terminal-Bench 2.1 with the Terminus-2 harness for all models. That helps comparability, but it is not an independent evaluation. The card also says its AA v4.3.2 scores are internal estimates that may differ from official Artificial Analysis numbers. The AA-LCR run uses corrected answer keys with an older judge model.

Quantization matters too. The benchmarks are for BF16. The card offers MXFP8, NVFP4 and Q5_K_M formats, and the llama.cpp example uses the Q5_K_M export, so quantized accuracy may differ. Dory support in vLLM and llama.cpp was still in pull requests at the time of the card, so you need to build from those branches.