What Is Nemotron 3.5 Super VL? Architecture, Speed and 1M Context
NVIDIA's open-weight Nemotron 3.5 Super VL uses Mamba-2, MoE and attention with 120B total, 12B active parameters and 1M context. Here is where it fits.
What is Nemotron 3.5 Super VL?
Nemotron 3.5 Super VL is an open-weight vision-language model from NVIDIA. It has 120 billion total parameters but activates only 12 billion per token, which keeps it fast for its size. It mixes Mamba-2 layers, mixture-of-experts (MoE) layers and attention layers. It takes text, images and video as input and returns text. It supports a context window of up to one million tokens and is aimed at long-running agents that combine reasoning, tool use and visual understanding.
TL;DR
- Nemotron 3.5 Super VL is a 120B-parameter model with only 12B active per token, so it runs faster than its total size suggests.
- The architecture is a hybrid of Mamba-2, mixture-of-experts and attention, built to make very long contexts practical.
- It accepts text, images and video and supports up to a 1M-token context window, and NVIDIA positions it for long-running agents.
- Its clearest strengths are long-context tasks and speed on NVIDIA GPUs, where NVIDIA claims roughly double the token throughput of nearby rivals.
- On agentic and coding benchmarks it trails Qwen 3.8 Flash Next, GLM and DeepSeek, though it beats the older Qwen 3.5.
- Hands-on testing showed solid reasoning and vision work, a real bug fixed quickly, and weak spots in rare languages and code-drawn art.
How is Nemotron 3.5 Super VL built?
The model combines three layer types, and each does a different job.
- Mamba-2 layers are state-space layers. Their cost grows roughly linearly with sequence length, and they carry a compact running state instead of a key-value cache that grows with every token. That matters most at long context.
- Mixture-of-experts layers hold many specialist sub-networks. A router picks a few of them for each token, so most of the weights sit idle on any given step.
- Attention layers keep the precise token-to-token lookup that pure state-space models can lose. Mixing them in is the usual hybrid compromise.
The "VL" part means vision-language. Text, images and video go in, and text comes out. There is no image generation.
What does "120B total, 12B active" actually mean?
Only about a tenth of the weights fire for each token. Compute per token looks like a 12B model, while the knowledge capacity comes from the full 120B.
There is a practical catch that applies to every MoE model. The router can pick any expert on any token, so you generally need all 120B parameters available in memory, even though each step uses a fraction of them. The savings show up in speed and throughput, not in a smaller memory footprint. The sources do not give hardware requirements or quantization formats for this model, so check NVIDIA's release notes before planning a deployment.
How good is the 1M-token context?
NVIDIA's pitch is that the model can hold an entire codebase in context at once. In the comparison NVIDIA published, long context is the category where the model leads its open rivals. That fits the architecture, since the Mamba-2 layers keep long sequences cheaper to process than a pure attention stack would.
A big window does not guarantee good behavior inside it. The sources include no long-context test run on this model beyond NVIDIA's own chart, so treat the codebase-in-one-prompt idea as a claim to verify on your own repositories.
How fast is Nemotron 3.5 Super VL?
NVIDIA says that on its own GPUs the model generates roughly double the tokens per second of comparable open models. A creator who ran it locally in an agent harness reported that tasks finished very quickly, which matches the claim in spirit. It is not an independent measurement.
Two caveats. The speed claim is tied to NVIDIA hardware, and your results on other accelerators may differ. Also, speed per token is not the same as speed to a correct answer. A faster model that needs more retries can lose that advantage.
How does it compare on agentic coding?
This is where the model is weakest. A rival's model card (for Ornith-1.5-122B, which lists Nemotron 3.5 Super VL as a comparison column) puts the numbers side by side. These are another lab's reported figures, so read them as indicative, not definitive.
| Benchmark | Nemotron 3.5 Super VL (120B) | Qwen3.8-Flash-Next (125B) | DeepSeek-V4-Flash-0731 (284B) |
|---|---|---|---|
| TerminalBench 2.1 | 66.7 | 83.4 | 82.7 |
| SWE-bench Verified | 61.2 | 82.8 | not listed |
| SWE-bench Pro | 42.5 | 43.9 | not listed |
| BrowseComp | 61.0 | 75.3 | 80.2 |
| MCP-Atlas | 63.3 | 74.1 | 74.6 |
| Toolathlon-Verified | 60.1 | 71.4 | 70.3 |
| HLE (with tools) | 33.2 | 38.0 | 50.8 |
The gaps are large on both: on TerminalBench 2.1 the model card lists 66.7 for Nemotron against 81.2, and on SWE-bench Pro it lists 42.5 against 76.9, a wider gap. NVIDIA's own chart, as described by the creator who reviewed it, shows the same ordering: Qwen 3.8 Flash Next, GLM and DeepSeek ahead on agentic and coding tests, with Nemotron clearly above the older Qwen 3.5.
Does it work in practice?
One creator ran it through the Hermes agent harness against a local Docker app with a Postgres database, a front end, a back end and Redis. The app was a prescription-sunglasses shop with a hidden bug: the lens engine mishandled the conversion from plus to minus cylinder notation. Two patients with identical eyes got different lens thicknesses and prices.
The model was given a goal with no hint about the cause. It found the missing sphere update in the conversion, fixed it, rebuilt the container and verified the result. The comparison table then matched across every row. The downside was scope creep. It also reported three extra fixes in other files that were not broken.
Other tests from the same session:
- Handwritten circuit analysis. It read every label on a hand-drawn Sallen-Key filter, computed a cutoff near 750 Hz that matched the tester's own math, and noticed the op amp inputs were wired backwards. Its wiring explanation was slightly loose, but the tester called it a pass without much thinking time.
- Multilingual titles. It read a Portuguese paragraph from an image, captured the core idea (contentment over happiness) and followed the output format for major languages. Rare languages went badly. Burmese was garbage, Malayalam had a stray English word, and Pashto and Sindhi failed.
- Code-only art. Asked to draw a themed dragon tattoo in a single HTML file, it got the flag, dog tags, sun and banner right but produced a barely recognizable dragon.
One session is an anecdote, not a benchmark. It does show the pattern: capable and quick on structured reasoning and tool use, uneven at the edges.
Who should consider using it?
It suits teams that run NVIDIA hardware and care about throughput. It also fits workloads that mix images or video with long documents, or agents that need a large working context. The open weights allow self-hosting.
If your main goal is the highest score on agentic coding, the table above points elsewhere. Qwen 3.8 Flash Next and DeepSeek V4 Flash score noticeably higher there. The tradeoff is speed and long-context handling against raw coding accuracy. If you do adopt it for agent work, constrain its permissions and review diffs, since it showed a tendency to change files it did not need to touch.
Frequently Asked Questions
Is Nemotron 3.5 Super VL open source?
NVIDIA released it with open weights. Open weights let you download and run the model yourself. Check the license text for the exact terms on commercial use.
What does the "VL" in the name mean?
It stands for vision-language. The model accepts text, images and video as input and produces text output. It does not generate images or video.
Why does a 120B model only use 12B parameters at a time?
It is a mixture-of-experts design. A router selects a small set of experts for each token, so only about 12B of the 120B weights compute on any step. All the weights still need to be loaded.
Is it better than Qwen 3.8 Flash Next or DeepSeek for coding?
Not on the published numbers. Both score higher on agentic and coding benchmarks such as TerminalBench and SWE-bench Verified. Nemotron's case rests on long-context performance and speed on NVIDIA GPUs.
How big is the context window?
It supports up to one million tokens. NVIDIA says that is enough to hold a whole codebase at once, and its comparison shows the model leading open rivals on long-context tests.
How we source, write, and correct our articles. Editorial standards
