---
title: "HP ZGX Fury Benchmarks: Local LLM Tokens per Second"
url: "https://goremy.ai/blog/articles/hp-zgx-fury-benchmarks-local-llm-speeds"
author: Luis Chavez-Mattos
editor: Luis Chavez-Mattos
editor_title: Director, Product
published: "2026-11-18T02:20:17.735Z"
updated: "2026-11-18T02:20:17.735Z"
topic: Local AI
description: "Measured speeds for the HP ZGX Fury: 252 GB HBM, NVFP4 models, users per model, and a 1-bit DeepSeek Flash at 80 tok/s, compared with GB10 boxes."
---

# HP ZGX Fury Benchmarks: Local LLM Tokens per Second

> Measured speeds for the HP ZGX Fury: 252 GB HBM, NVFP4 models, users per model, and a 1-bit DeepSeek Flash at 80 tok/s, compared with GB10 boxes.

Edited by Luis Chavez-Mattos, Director, Product · Published 2026-11-18

## How fast is the HP ZGX Fury?

In one creator's hands-on testing, the HP ZGX Fury ran a single user on a 320-billion-parameter GLM Flash model at about 225 tokens per second, or about 283 with speculative decoding. It served 16 users on that model at just under 80 tokens per second each. A 1-bit version of DeepSeek Flash, which is too big to fit at 4-bit, ran at about 80 tokens per second. The reason is memory: 252 GB of very fast HBM, versus about 128 GB at roughly 270 GB/s on GB10-class boxes like the ZGX Nano.

A note on sourcing: these numbers come from a single reviewer's tests, using 8,000-token prompts, and the video was sponsored by HP and Nvidia. Treat them as a realistic picture, not a lab benchmark. Your results will shift with prompt length and software setup.

## TL;DR

- The Fury's edge is **memory speed, not just memory size**: about 7.1 TB/s on the GPU's 252 GB of HBM, against roughly 270 GB/s on GB10-class boxes.
- A single user on GLM Flash got about **225 tokens per second**, and about 283 with speculative decoding.
- With 16 users on the same model, each still got **just under 80 tokens per second**.
- DeepSeek Flash served **54 concurrent users at a floor of about 40 tokens per second each**, and Nemotron 3 Nano reached 188 concurrent users.
- The full-size 4-bit DeepSeek Flash (about 420 GB) doesn't fit in HBM, but a **1-bit version does and runs at about 80 tokens per second**.
- For agents, aim for **at least 80 tokens per second per user**. Chat is fine at around 30.

## Why does memory matter more than anything else?

Running a model has two phases. In prefill, the model reads your prompt, thousands of tokens at once. That is limited by raw compute. In decode, it writes the answer, roughly one token at a time, and for each token it re-reads all the active weights from memory. That is limited by memory bandwidth.

GB10-class machines such as the ZGX Nano have good compute, so prefill is fine. But their bandwidth is only about 270 GB/s, so decode is slow. The ZGX Fury has about 7.1 TB/s of bandwidth on its 252 GB of HBM (high bandwidth memory). That is the main reason for the speedup.

The Fury also has 496 GB of unified memory attached to its Grace CPU, at about 396 GB/s. The GPU can use it, but HBM is about 18 times faster. So when the reviewer says a model "fits," he means it fits in HBM.

## How can you tell if a model will fit before downloading it?

Use this rule of thumb:

**Memory in GB = parameters (billions) × bits per weight ÷ 8**

Four-bit quantization is closer to 4.5 bits in practice, because some layers stay unquantized. At 4.5 bits, multiply billions of parameters by about 0.56. A 100B model needs roughly 56 GB.

Then leave room for the KV cache. That is the memory each conversation uses to track what has been said. It grows with context length and with the number of users. If you run several models on one GPU, their memory fractions also need to add up to slightly less than one.

The reviewer's example "jigsaw" on the Fury:

| Model | Approx. memory |
|---|---|
| GLM Flash (320B total parameters) | about 180 GB |
| Nemotron 3 Nano | 19 GB |
| A small Qwen 3.5 4B model | small |
| Embedding model plus a fine-tuned 270M decision model | about 0.5 GB for the small one |

That already lands above 212 GB, and the remaining HBM has to be kept for the KV cache. The 180 GB GLM model alone is larger than the 128 GB total on a ZGX Nano.

## What speed should you expect from the math?

You can estimate a ceiling yourself: **bandwidth ÷ active weights in memory**.

GLM Flash is a mixture of experts model. It has 320B parameters in total but only 18B active per step, which is about 10 GB per token. Dividing 7 TB/s by 10 GB gives a ceiling near 700 tokens per second. A single user actually got about 225, roughly a third of the ceiling. The reviewer says that gap is normal once attention, KV cache reads and software overhead are counted.

Speculative decoding pushed it to about 283. A small draft guesses the next few tokens and the big model checks them in one pass. That trades spare compute for fewer trips to memory. It only helps while compute is spare, so it fades as more users pile on.

## How many users can it serve at once?

Batching helps here. When many users are served together, each read of the weights serves all of them, and decode becomes a compute problem again. The Fury has about 20 petaflops of compute, which is why concurrency holds up well.

Results with 8,000-token prompts:

| Model | Concurrency result |
|---|---|
| GLM Flash | 16 users, each just under 80 tok/s |
| DeepSeek Flash | 54 concurrent users, each at least about 40 tok/s |
| Nemotron 3 Nano | up to 188 concurrent users |

Prompt length changes this a lot, and it changes differently for each model. Test with prompts that look like your real work.

## Why does per-user speed matter more for agents?

A person reads while the answer streams, so about 30 tokens per second feels fine for chat or retrieval (RAG) tools. An agent doesn't read. It writes thousands of tokens of reasoning and code before calling a tool, and it does that many times per task.

The math: a 3,000-token step at 30 tokens per second takes almost two minutes. Forty such steps is over an hour for one task. That's why the reviewer recommends at least 80 tokens per second per user for agentic work. Sub-agents running in parallel make the load heavier.

Agent workloads are also prefill heavy, because every turn resends the conversation, files and tool results. Prefix caching helps. The serving software (vLLM or SGLang, for example) reuses work it already did, and the cache hit rate shows how much it is saving.

## Does a smaller model always run faster?

No. A dense model reads all of its parameters for every token. A mixture of experts model reads only the active ones. A dense 65B model has about a fifth of GLM Flash's total parameters, yet it was capped near 200 tokens per second because it reads all 65B each time.

That isn't a knock on dense models. Many people prefer them for coding, and the Qwen 27B is one example. The catch with mixture of experts models is that all the weights must still sit in memory, even though only some are used per token.

## Can it run a model that doesn't fit, like DeepSeek Flash?

At 4-bit (NVFP4), DeepSeek Flash is about 420 GB, well over the 252 GB of HBM. It would spill into slower memory. But a 1-bit version from Unsloth does fit in HBM and ran at about 80 tokens per second.

The reviewer expected the output to be poor, as heavily compressed models usually are, and found it better than expected. Unsloth's quantization doesn't make every layer 1-bit. It keeps some parts at higher precision. He still wouldn't use it for coding or precision-heavy work, and would pick a 4-bit version of a model for that. But it shows quantization improving.

For larger teams, he points to pairing two machines over a fast network. In that setup, DeepSeek Flash hit 441 tokens per second for one user, or 40 users at 48 tokens per second each.

## What does this mean for you?

**If you are in IT or lead a team that wants private AI:** a box like this is about serving a model to many people, not one enthusiast. Estimate the memory first with the formula, reserve room for the KV cache, then decide on a per-user speed floor. If you only need chat, a lower floor and more users is fine. If you want coding agents, plan around 80 tokens per second per user.

**If you work in law, healthcare, government or security:** the reviewer ran a team of coding agents with the network cable pulled. Cyber teams containing an incident often need to cut internet access, and some document work can't go to a cloud API at all. Local inference is an answer to that, though the video's retrieval example for a 20 to 30 person law firm was cut off in the source I had, so I won't claim details.

**If you're comparing it with a ZGX Nano:** a GB10-class box can still be a fine fit for smaller models and light use. The gap shows up when a model needs more than 128 GB, or when many people or agents need fast decode at the same time.

**If you can't buy hardware:** the memory formula and the per-user speed rule apply to any setup.

## What could you build to plan a local AI rollout?

Choosing hardware is often about tracking requests and capacity, not just speeds. With [Remy](https://goremy.ai), you could describe a small internal tool in plain English, such as a model-fit calculator that applies the parameters-times-bits formula, or an intake form where teams request access and state their expected users and prompt lengths. Remy builds and deploys it as a full-stack app with sign-in and a database, so you can replace the shared spreadsheet before you spend on hardware. It builds web apps, not software that runs on your own GPU.

## Frequently Asked Questions

### What is NVFP4 and why does it matter on the ZGX Fury?

NVFP4 is a 4-bit format, and the body says DeepSeek Flash at 4-bit (NVFP4) is about 420 GB, well over the Fury's 252 GB of HBM. The body gives no block size or scale details.

### How is the ZGX Fury different from the ZGX Nano?

According to the source, GB10-class machines like the ZGX Nano top out around 128 GB of memory with about 270 GB/s of bandwidth. The Fury has 252 GB of HBM at about 7.1 TB/s, plus 496 GB of CPU-attached unified memory. Decode, which is limited by bandwidth, is much faster on the Fury.

### Can the ZGX Fury run agents without an internet connection?

Yes. The reviewer disconnected the network and ran multiple coding agents with several models on the box, with sub-agents fanning out work. The Grace CPU also handled multiple sandboxes. That matters for incident response and for total privacy.

### How many tokens per second do I need?

About 30 tokens per second is fine when a human is reading the output, such as in chat or RAG. For agents, which generate thousands of tokens per step and repeat it many times, the reviewer recommends at least 80 per user.
