The $13B Reason You Should Run Local AI Models

The short answer

Running a local model instead of a cloud API means the weights, the data, and the uptime are yours, not rented from a vendor who can get acquired, breached, or change terms overnight. That tradeoff used to cost real capability. It mostly doesn't anymore.

I have spent the last few years watching engineering teams build on other people's infrastructure and call it a strategy. Then the strategy's owner gets bought.

Nvidia just bought the neutral ground

In August 2026, Nvidia agreed to buy Hugging Face for $12.9 billion, days after reports surfaced that the company was fielding offers around $13 billion or more.¹ Hugging Face was valued at $4.5 billion in its 2023 Series D, a round Nvidia itself participated in.² Three years later, Nvidia is buying the whole thing outright, nearly tripling that price.¹

Hugging Face is not just another startup. It is the default place developers go to discover, download, and share open AI models, datasets, and tools. People call it the GitHub of AI, and for once the nickname undersells it.¹ Its value came from being neutral ground, a hub that hosted competing models and worked across every hardware vendor's stack, including Nvidia's rivals.¹

That neutrality is the thing at risk now. Nvidia already controls most of the hardware that trains and runs the models Hugging Face hosts. Owning the distribution layer too puts one company in the middle of model discovery, model access, and the chips underneath both.¹ Nvidia has its own reasons to keep open models healthy, since a thriving open ecosystem is a hedge against closed AI labs trying to reduce their Nvidia dependence.¹ But "a hedge that serves Nvidia's interests" and "neutral infrastructure you can build a company on" are not the same promise.

If your AI stack depends on pulling models from a hub, or on an API from a lab that could just as easily get bought, folded, repriced, or deprecated, you don't own your infrastructure. You're leasing it from whoever wins the next acquisition fight.

The cloud's security record isn't reassuring either

The acquisition risk would be easier to shrug off if cloud AI infrastructure had a clean security record. It doesn't.

IBM's 2025 Cost of a Data Breach report found that 13% of organizations reported a breach of their AI models or applications, and 97% of those breached had no proper AI access controls in place.³ Of the AI-related incidents, 60% led to compromised data and 31% caused operational disruption.³ That's not a hypothetical risk analysts warn about. It's what already happened to over one in eight organizations running AI in the cloud.

Separately, a Salesforce-linked breach tied to Gainsight's connected app exposed data across more than 200 companies, confirmed by Google in November 2025.⁴ The attackers, identified as the group ShinyHunters, got in not through a Salesforce vulnerability but through a third-party integration with broad access.⁴ That's the exact shape of risk you take on every time you wire another SaaS or API vendor into your stack: you inherit their blast radius.

None of this means cloud AI is reckless or that every vendor is negligent. It means the attack surface of "AI running on someone else's infrastructure, accessed through someone else's access controls" is large, growing, and mostly outside your control. We've made this case before when writing about cloud bugs and usage fees forcing a return to self-hosting, and the acquisition news just adds a second axis of risk: not just what a vendor charges you, but who owns them next year.

The capability gap closed faster than most engineers expected

For years, the honest argument against local models was simple: they weren't good enough. That argument is aging badly.

Qwen3.8-27B, Alibaba's dense 27-billion-parameter open model released in August 2026, scores around 51 to 52 on Artificial Analysis's Intelligence Index, putting it near Claude 4.8 on max settings and within striking distance of much larger mixture-of-experts systems, despite running on a single consumer GPU.⁵⁶ The full weights compress to roughly 17 to 18 GB at 4-bit quantization, small enough to run on a 24 GB card like an RTX 3090 or 4090, or a mid-range Apple Silicon Mac.⁶ It ships Apache 2.0 licensed, with a 262,144-token context window and native image and video understanding.⁶

On Qwen's own agentic-coding benchmarks, the model jumped from 63.4 to 73.0 on Terminal-Bench 2.1 over its predecessor.⁶ Independent hands-on testing found it running at 15 to 20 tokens per second on single-thread inference on a dual DGX Spark setup, scaling to 60 to 70 tokens per second under concurrency, using the quantized version.

Those numbers deserve a caveat I'd want if I were reading this instead of writing it: vendor-reported benchmarks are directional, not gospel, and your mileage depends heavily on your hardware and your task.⁶ But the direction is unmistakable. A file you can download tonight and run on a gaming GPU is now doing agentic coding and reasoning work that needed a hosted frontier API a year ago. That is a genuinely new fact on the ground, not a marketing claim.

What this actually changes for engineering leaders

I'm not going to tell you to rip out every API integration tomorrow. Plenty of workloads genuinely benefit from a hosted frontier model, especially anything that needs the very top of the capability curve or bursts unpredictably. Buy vs. build is a real tradeoff, not a moral position, and we've written at length about when renting cloud LLM tokens beats owning quantized models and vice versa.

But the calculus for what belongs on your own hardware just shifted. Three things changed at once:

The distribution layer consolidated. One company now sits astride the chips and the model hub most developers use to find and deploy open models.¹

The security data caught up with the anecdotes. AI-related breaches are no longer rare edge cases; they show up in double digits across surveyed organizations, and third-party integrations remain the weak point.³⁴

The capability floor rose enormously. Work that required a frontier API a year ago now runs on hardware you can buy at an electronics store.⁶

Put those together and the case for running your own models for your core workloads, the ones tied to customer data, proprietary logic, or anything you'd rather not explain to a new corporate parent, gets a lot stronger. Ownership isn't about avoiding all cloud dependency. It's about deciding which pieces of your AI stack you actually control, and which ones you're renting from whoever wins the next $13 billion bidding war. Platforms like Remy exist for exactly this reason, giving teams a straightforward way to run capable open models on infrastructure they actually control instead of infrastructure they hope stays neutral.

If you haven't audited which of your AI workloads sit on infrastructure you don't own, this is a good week to start. Acquisitions move faster than procurement cycles, and the next one might not come with two days' warning.