What Hardware Do You Actually Need to Run LLMs Locally?

The hardware requirements for local LLM inference come down to one number more than any other: how much VRAM (or unified memory) you have, and how fast that memory can feed the GPU. A 27B-class model like Qwen3 needs somewhere between 15GB and 56GB depending on quantization, and that range is the difference between a $1,600 gaming GPU and a rack of A100s.¹ Compute matters too, but for most enterprise inference workloads, memory capacity and bandwidth are the real bottleneck. Everything below maps to that framework.

1. What does Qwen 27B actually need?

Start with the model, not the hardware. A dense 27B model needs:

• BF16 (full precision): ~56GB VRAM.¹
• FP8: ~28GB VRAM.¹
• Q4_K_M (4-bit quantized): roughly 14-18GB VRAM before KV cache and context overhead.¹²

That last number is why this conversation changed in the last year. A quantized 27B model now fits on hardware a mid-size team can just buy, not requisition. Community testing backs this up directly: one setup running Qwen 27B Q4_K_M on a 16GB RTX 4070 Ti SUPER hit 130K context at roughly 20 tokens/sec.² Drop to 12GB or 8GB and you're trading context length or quantization quality, but the model still runs.²

2. Consumer GPU tier: RTX 4090 / 5090

This is the entry point for anyone serious about running a 27B model in-house. The RTX 4090 launched at $1,599 with 24GB of GDDR6X.³ At that VRAM ceiling, Qwen 27B fits entirely in memory at Q4-Q5 quantization with room for about 16K of context.⁴ Step up to a 32GB RTX 5090 and you can push context out to 131K tokens while running around 90-175 tokens/sec depending on the exact model variant.⁵

This tier is the sweet spot for a single-model, single-team deployment. You're not fighting bandwidth limits the way unified-memory machines do, because a discrete GPU's dedicated GDDR memory sits close to the compute die and is built for exactly this job.

3. Unified-memory tier: Mac Studio M3 Ultra

The Mac Studio M3 Ultra is the quiet, single-box alternative. It configures up to 512GB of unified memory, which means it can load models that would need multiple discrete GPUs to fit anywhere else.⁶ Bandwidth on that memory pool is 819GB/s.⁷

That number sounds high until you compare it to a discrete GPU rig, where dedicated VRAM moves data faster per watt. Benchmarks on Mac hardware show throughput dropping as a model chews through serious token counts, because unified-memory architecture can't move data as fast as dedicated VRAM once generation ramps up.⁶ You get capacity. You don't get speed. For a 27B model that fits on a $1,600 GPU anyway, the Mac Studio's real value is running much larger models you couldn't otherwise touch on consumer hardware, not running a 27B model faster.

4. Purpose-built desktop AI: DGX Spark and Strix Halo-class boxes

NVIDIA's DGX Spark is the newest entrant in the "AI mini supercomputer" category: 128GB of unified LPDDR5x memory and a Blackwell GPU with NVFP4 support, marketed as capable of running models up to 200B parameters locally.⁸

The price tells its own story. NVIDIA raised the Founders Edition price 18%, from $3,999 to $4,699, in February 2026 due to memory supply constraints.⁸ The benchmarks complicate the pitch further: on GPT-OSS 120B, the DGX Spark generates about 38.6 tokens/sec, while a three-GPU RTX 3090 rig costing roughly $2,500-3,000 hits 124 tokens/sec on the same test.⁸ The Spark's ~273GB/s bandwidth is the limiter, same story as the Mac Studio, just at a higher price point.⁸

If you need one quiet box that can technically load a very large model, this tier delivers. If you need throughput for real workloads, a used multi-GPU rig will beat it for less money.

5. Data-center grade: A100/H100 multi-GPU racks

None of the above applies once you're running mixture-of-experts giants like Kimi-K2 (1T parameters), GLM-4.5 (355B), or Qwen3-235B. Those models need 4 to 16 A100-80GB GPUs, with total hardware costs running from $60,000 to $240,000.⁹

That's real infrastructure, and it's worth naming clearly because it's not the tier most teams need. A dense 27B model fits on a single consumer GPU costing around $2,000.⁹ Don't buy a rack because you saw a benchmark for a trillion-parameter model. Match the hardware tier to the model you're actually running.

6. When does local hardware beat the API bill?

This is the question that actually decides the purchase, and a peer-reviewed Carnegie Mellon study puts real numbers on it.⁹ For small open-source models in the 27B-32B class, running on a single ~$2,000 GPU like an RTX 5090, break-even against commercial APIs can happen in as little as 0.3 months when compared against a premium model like Claude-4 Opus.⁹ Even against cheaper baselines, most small-model deployments break even within three months.⁹

The math gets murkier as models scale up:

• Medium models (70B-class): break-even stretches to 3.8-31.2 months, depending on hardware cost ($15k-$30k) and which API you're comparing against.⁹
• Large models (235B+): break-even ranges from 3.5 months to nearly 9 years, again almost entirely dependent on the comparison API.⁹

That spread matters because API pricing itself spans a 600x range, from $0.10 per million tokens on the cheap end to $60 per million tokens for frontier reasoning models.¹⁰ Enterprise LLM API spend hit $8.4 billion in 2025, more than doubling in six months, and inference now outweighs training as the dominant AI budget line.¹⁰ If your team is burning serious token volume against a mid-to-premium API, a 27B-class self-hosted model is not a moonshot. It is often the cheaper option within a single quarter. If you're a low-volume team already on a cheap-tier API, the math flips and buying hardware is a slower payback.

This is the same tension we covered in The Race to the Bottom: falling API prices move the break-even line constantly, so this isn't a one-time calculation. Re-run it whenever your provider changes pricing tiers.

7. What's next: processing-in-memory chips

Every tier above hits the same wall eventually: memory bandwidth, not raw compute, caps how fast you can generate tokens. Processing-in-memory (PIM) chips are the industry's answer, and they're not vaporware anymore.

Samsung's LPDDR5X-PIM adds compute logic directly into the memory banks. Standard LPDDR5X-9600 tops out at 76.8GB/s; the PIM version hits 614GB/s, an eightfold bandwidth increase, and measured 3.01x faster tokens-per-second in AI inference benchmarks.¹¹ SK Hynix is chasing the same problem with its own approach, which it calls Accelerator-in-Memory, or AiM.¹² Both companies are pursuing this because HBM is expensive and memory now makes up the bulk of AI chip cost. PIM is the lower-cost, lower-power path to similar speedups.¹²

If PIM memory reaches consumer and prosumer hardware, the bandwidth bottleneck that limits Mac Studios and DGX Sparks today gets a lot less punishing. That would shift the local-vs-API break-even math again, likely in favor of local deployment, since the gap between "can fit" and "runs fast" starts to close.

Bottom line: a hardware decision tree

Strip out the marketing and the decision is fairly mechanical:

• Running a 27B-class model, single team, need speed: get a 24-32GB consumer GPU (RTX 4090/5090). Cheapest path to full throughput.
• Need to load much larger models occasionally, quiet office setup, throughput isn't critical: Mac Studio M3 Ultra, but budget for slower generation.
• Want a compact dedicated box and can absorb a premium for convenience: DGX Spark or a Strix Halo-class machine, understanding you're paying more for less throughput than a discrete-GPU rig.
• Running trillion-parameter MoE models in production: you need the multi-GPU data-center tier and the budget that comes with it.
• High token volume against a mid-to-premium API and predictable workloads: run the CMU break-even math yourself. For 27B-class models, payback is often under three months.⁹
• Handling regulated data or contractual data-residency requirements: local deployment answers the compliance question before the cost math even matters, which is a big part of why teams evaluate platforms like Remy alongside the raw hardware calculus.

A 27B model is no longer a data-center problem. It's a purchasing decision you can make with a corporate card and a weekend. The bigger models still need the bigger checks. Know which one you're actually running before you buy hardware for the wrong tier.