The API Matryoshka: What Renting AI Actually Costs

Renting a frontier AI API costs less than self-hosting an open-source model for most companies, full stop. But that answer hides a bigger problem: most of the "AI-native" software you're renting is itself just renting the same API, with a markup stacked on top, and you rarely see where your dollar actually lands.

The tell: Muse's hidden Azure lane

A developer poking through session logs from Meta's flagship consumer AI agent, Muse, found something odd. One subagent had routed through a model labeled azure/muse-special. The response signature was tagged gpt_responses_v1 and carried an encrypted payload starting with gAAAAA, a format OpenAI uses. The conclusion: Muse, built by one of the best-funded AI labs on earth, appears able to fall back to an OpenAI model served through Microsoft's Azure cloud rather than Meta's own in-house models.¹

That alone would be a curiosity. What makes it useful is the context. Muse's runtime ships with client code and model IDs for multiple third-party providers: Claude Opus, Sonnet, and Haiku, several GPT-5 variants routed through OpenAI, Azure, and Codex, and Kimi K3 through Fireworks. Most production traffic still runs on Meta's own models. But the capability to quietly reach for someone else's model is built into the product, not bolted on as an emergency patch.¹

Meta has form here. Reuters reported the company was testing a "human concierge" layer for Muse, with contractors quietly handling some of the phone calls the agent supposedly placed on its own, a pattern that echoes Facebook's old "M" assistant, where most tasks were done by people, not code.² Strip away the branding and you get the same lesson twice: the automation you're sold is often a thinner layer than it looks, wrapped around infrastructure, or people, that someone else is providing.

If a company with Meta's compute budget and its own trained models still reaches for a rented one, it's worth asking what your own AI vendor is actually running underneath your subscription.

What is an "AI wrapper," really?

An AI wrapper is a software product built on top of an existing foundation model, accessed through API calls, rather than a company that trains its own models from scratch.³ That used to be an insult in venture circles. It's now closer to an architecture diagram of most of the industry.

The distinction that matters isn't wrapper versus "AI-native." It's whether the product does something the foundation-model provider can't simply replicate with a feature update. Investors increasingly look for accumulated proprietary data, deep embedding into a specific vertical workflow, or some other moat the underlying model vendor can't ship overnight.³ Legal-AI company Harvey and coding tool Cursor are commonly cited as having cleared that bar, despite Cursor's margin troubles below. Horizontal writing tools like Jasper got squeezed hard once OpenAI added native competing features to its own products.³

A Google Cloud VP overseeing the company's startup organization put it more bluntly: startups that are "almost white-labeling" someone else's model with no proprietary layer on top have their "check engine light" on, because the industry has stopped rewarding thin wrappers with thin moats.⁴

Tracing a subscription dollar through the stack

Here's where the matryoshka metaphor earns its keep. A typical enterprise AI subscription dollar passes through several nested layers before it does any actual work, and each layer keeps a cut:

1. The enterprise pays a SaaS subscription for an "AI-native" tool, priced per seat or per credit.
2. The SaaS company pays a foundation-model API per token, for every request its product makes on the customer's behalf.
3. The foundation lab pays a GPU cloud by the hour to run that model, and keeps the spread as its own compute margin.
4. The GPU cloud pays Nvidia, and its own margin, for the hardware underneath all of it.

The economics of the middle layers have shifted fast. OpenAI's compute margin, revenue minus infrastructure cost, rose from roughly 35% in January 2024 to about 70% by October 2025, according to The Information's reporting.⁵⁶ The foundation-model layer now runs close to traditional software margins, largely because enterprise buyers pay a premium for reliability they can't get from a hobbyist deployment.

The application layer sitting above it tells a very different story. Bessemer's 2025 dataset on fast-growing AI application companies, its "Supernovas," shows average early-stage gross margins around 25%, many of them negative, against the 75-80% margins considered healthy in traditional SaaS.⁵⁶ The markups compound as they move down the stack, but the company closest to the customer, the wrapper, is often the one eating the loss, not collecting the biggest cut.

Cursor's negative gross margin, in real numbers

The clearest documented case of what this does to a business is Cursor, the AI coding tool from Anysphere. The venture firm Foundamental ran the numbers on its cost structure and found Cursor was paying Anthropic roughly $650 million a year for model inference against roughly $500 million in annualized revenue, a gross margin of about negative 30% before support, hosting, or anything else.⁵⁷ Foundamental's phrase for the dynamic stuck: "scaling yourself into hell."⁷

The mechanics are simple. Cursor sells flat-rate subscriptions. Its customers' token usage is variable and, in practice, largely outside Cursor's control, set by how much code a developer asks the model to generate. When usage grows faster than the price the company can charge, the wrapper's margin runs the wrong direction. Cursor's public response has been to push its own model, Composer, into the product, trying to move some of that inference cost in-house rather than paying Anthropic's rate on every call.

GitHub Copilot lost money on purpose, for a while

Microsoft ran the same experiment at a different scale. Reported figures showed Microsoft losing an average of $20 per user per month on GitHub Copilot, priced at $10 a month, in 2023, with its heaviest users costing the company as much as $80 a month.⁸⁵ A company with Microsoft's balance sheet could absorb that as a strategic loss leader. Most SaaS companies renting the same underlying models cannot, which is why usage-based billing, rather than flat per-seat pricing, has become the industry's standard fix for wrapper economics that don't survive contact with heavy users.

Is it cheaper to rent an API or self-host an open-source model?

Hosted API pricing for input tokens runs roughly $0.20 to $10 per million tokens depending on the model, with output priced several times higher.⁹ Self-hosting looks cheap on paper by comparison, but the GPU rental market alone spans a four-to-fivefold range, from $2.69 an hour for an H100 on RunPod's community cloud to the equivalent of $12.29 an hour on Azure.⁹ And that's before utilization. Real production clusters rarely run near peak: a study of a 20,000-plus GPU ByteDance cluster found compute utilization below 60% on more than 99% of GPUs measured, which quietly inflates the effective cost of every token a self-hosted model actually produces.⁹

Run the full comparison and the crossover point moves a long way from where the sticker prices suggest. One practitioner cost breakdown put the raw breakeven for self-hosting Llama 3.3 70B on a single A100 GPU, versus GPT-5 API pricing, at around 256 million tokens a month. Once you add the DevOps and infrastructure overhead that self-hosting actually requires, that breakeven jumps to roughly 576 million tokens a month, about 19 million tokens a day. Against cheaper hosted models like Gemini 2.5 Flash-Lite, the breakeven climbs to nearly the GPU's entire theoretical capacity.¹⁰ For most workloads under roughly 50 to 100 million tokens a day, renting the API is still the cheaper, less risky option. This is the real crossover math that self-hosting advocates rarely put on the slide, and it's why, even as enterprise AI infrastructure spend roughly doubled, enterprise usage of open-source models actually fell, from 19% of total usage to 11%.⁹ Owning the model isn't automatically owning the economics. It's a different break-even calculation entirely, and it only pays off past a volume threshold most teams never reach.

Paying a markup to route your markup

There's now a layer sitting between the enterprise and the foundation model that exists purely to manage which foundation model gets used. Aggregators like OpenRouter route a request across multiple providers to find the best price or availability for a given call. Stripe agreed to acquire OpenRouter in August 2026 for a reported $7.5 billion, a stunning jump from the roughly $1.3 billion valuation OpenRouter had raised at less than three months earlier.¹¹ Stripe's stated rationale was to help businesses "maximize profitability by routing their requests intelligently and spending their tokens efficiently."¹¹

That's a real service. It's also another nested layer of the matryoshka, one more party that needs to make a margin between your enterprise and the GPU actually running the model. The same Google Cloud VP who warned about thin wrappers extended the warning to aggregators specifically: "stay out of the aggregator business," he said, because routing traffic between other people's models is an even thinner value proposition than wrapping one model directly.⁴

What enterprises are actually paying for

None of this shows up cleanly on an invoice, which is exactly the problem. Large enterprises (10,000-plus employees) now spend an average of $4.7 million a year on AI-native application subscriptions, and 78% of IT leaders reported unexpected charges tied to consumption-based AI pricing in the past year.¹² Some of that comes from a governance gap that has nothing to do with which model is best: 68% of employees reported accessing generative AI assistants through personal accounts rather than approved platforms, and 57% admitted entering confidential information into public AI tools.¹³ Organizations with high levels of this kind of unmanaged, shadow AI usage face average data breach costs of $4.74 million, versus $4.07 million for organizations with low or no shadow AI, a $670,000 premium per incident that rarely gets attributed back to the AI line item on the budget.¹³ It's the same waste pattern that shows up across the broader SaaS stack, just with a newer coat of paint, and the same lock-in dynamics that make renewal conversations so lopsided.

A due-diligence checklist before your next renewal

Before signing or renewing an AI-native SaaS contract, ask these questions directly:

1. Which model actually runs this? Ask the vendor to name the underlying foundation model and provider, not just describe capability.
2. Whose infrastructure serves it? Determine whether the vendor's compute sits on infrastructure they control, or a cloud they're renting by the hour, since that spread is where their margin, or loss, lives.
3. Per-seat or per-token? Understand whether your price is fixed regardless of usage, and if so, ask what happens to the vendor's margin when your usage grows, because that's the Cursor problem in miniature.
4. How many layers are between us and the model? Check for aggregator or routing middleware, since each hop adds a margin you're funding indirectly.
5. What would it cost to run this ourselves? Even if the answer is "more than renting," running the breakeven math puts a real number on what you're paying for convenience versus what you'd pay for control.

That last question is really the point of this whole exercise. Most companies should keep renting frontier model access below the volume thresholds where self-hosting wins. But the choice to rent should be a decision made with real numbers, not a default nobody examined. Tools like Remy exist for teams that want to treat their AI pipeline as an asset they own and can inspect, rather than a subscription they hope is priced fairly. The matryoshka doesn't close until someone actually opens each layer and checks what's inside.