Edge0-145B Speed: Tokens Per Second on RTX 5090, 4090, M2 Ultra

How fast does Edge0-145B run on consumer GPUs and Macs?

Edge0-145B is an open-weight, 145-billion-parameter model that its release describes as an agent running on one consumer GPU. The headline figure reported for it is 98.3 tokens per second of agent decode on an RTX 5090 D using the NVFP4 build, alongside a reported score of 86.5 on Terminal-Bench 2.1. The model card breaks speed out by machine (RTX 5090 D, RTX 4090, M2 Ultra, RTX Spark N1X and a 16GB VRAM box), by build (NVFP4 and Q3_K), and by workload (prefill and decode). We only have a few of those numbers confirmed, so this article explains how to read the rest.

What do the speed numbers actually measure?

Four terms matter here.

Tokens per second (tok/s). A token is a chunk of text, roughly three-quarters of a word in English. If a model decodes at 98.3 tok/s, it writes about 70 words every second. A person reads at around 4 to 5 words per second, so anything above 10 or so tok/s feels live in a chat window. Agents need more, because they chain many steps together.

Prefill. Before the model can answer, it reads everything you gave it: your question, your documents, earlier messages. That reading phase is prefill, and it is usually much faster per token than writing. The card reports prefill at 4K and 16K context, meaning 4,000 and 16,000 tokens of input. Longer input means a longer wait before the first word appears.

Decode. This is the writing phase. The card separates chat decode from agent decode. Agent workloads usually carry long context and many tool calls, so they tend to be slower and more demanding than a plain chat.

Build. The same model can be saved in different file formats. Each format trades size, speed and quality differently.

Which machines does the model card cover?

The brief for this article lists the following test machines. The model card has the per-machine numbers, and we could not confirm them from the material we had, so we give only the one figure we can state.

The 16GB entry matters most to ordinary buyers. A 145-billion-parameter model is large, so seeing it listed for a 16GB card suggests the model relies on a sparse design to fit. The card's own art describes it as a "latent MoE" (mixture of experts), sparse by design and trained natively at 4 bits. In a mixture-of-experts model, only a fraction of the parameters work on each token, so it can generate faster than its total size suggests. The full weights still have to live somewhere. That is where system RAM or unified memory on a Mac comes in. We cannot say from the sources exactly how the 16GB setup splits the work.

NVFP4 vs Q3_K: which build is faster?

The answer depends on the machine, and we can't give a clean winner without the full table.

NVFP4 is a 4-bit floating-point format from Nvidia. Its hardware support arrived with the RTX 50-series (Blackwell) generation. On a card with that support, a model stored in NVFP4 can run efficiently. The model was also trained natively at 4 bits, according to the card, so the NVFP4 build is the format it was designed around. The 98.3 tok/s figure comes from this build on a 5090 D.

Q3_K is a quantization format from the llama.cpp ecosystem that squeezes weights down to roughly 3 bits. It makes the files smaller, which helps when memory is tight and works on hardware without NVFP4 support, such as older Nvidia cards and Apple chips. The tradeoff is some loss of quality, since fewer bits means less precision.

A reasonable rule of thumb, which you should check against the card:

• RTX 50-series card: start with NVFP4.
• RTX 4090, Mac or low-memory machine: the smaller Q3_K build is the likelier fit.
• Tight on memory: pick the build that fits in memory first, and chase speed second. A build that spills onto a slow disk will be slower than any table suggests.

What does the 86.5 Terminal-Bench 2.1 score mean?

Terminal-Bench tests AI agents on real tasks in a command-line terminal, the text-based interface developers and system administrators use to run commands. Tasks involve things like setting up software, fixing configurations and working with files. A higher score means the agent finished more of them correctly.

The reported score for Edge0-145B is 86.5 on version 2.1. Three cautions apply:

1. It is a self-reported number from the model's release. Independent reruns can differ.
2. It measures one skill. Terminal tasks tell you little about writing marketing copy or summarizing a contract.
3. Settings matter. Which build ran the test, how many attempts the agent got and which tools it had can all move the result. Check whether the score came from the NVFP4 build, because a smaller build may score lower.

The pairing of speed and score is what makes the release interesting. Agents run long chains of steps, so decode speed on a single machine decides whether an agent is usable or painful. A high agent decode rate alongside a strong terminal score is the combination worth watching.

What does this mean for you?

If you are a non-technical employee, you do not need to buy anything. This kind of model matters because it points to AI that runs on hardware a company already owns, without sending data to an outside service. Ask your IT team whether they are evaluating local AI for sensitive documents.

If you manage a team or budget, do not buy GPUs from a headline number. Pick the task you care about, such as summarizing long reports (prefill-heavy) or running an agent through many steps (decode-heavy). Then check the card's numbers for that workload on the machine you could realistically afford.

If you are a hobbyist or developer, work out which of the five listed machines is closest to yours. Download the build that fits your memory, run your own prompt at your real context length, and write down the tok/s you see. Your own measurement beats a table.

A simple log to keep while you test:

Machine:
Build (NVFP4 / Q3_K):
Context length (4K / 16K / other):
Prefill tok/s:
Decode tok/s:
Task and result quality (1-5):

Can you build your own benchmark tracker?

If you test several machines or builds, a shared spreadsheet gets messy fast. With Remy, you can describe a small internal app in plain English: a form where teammates log machine, build, context length and tok/s, a table that compares runs, and a chart showing which setup wins for which task. Remy builds the full-stack app, with a database and sign-in, and deploys it to a live URL. You own the code. It fits teams comparing local AI options without paying for another subscription.