Self-Hosted AI Gateway vs. SaaS: The Real Build vs. Buy Math

A self-hosted AI gateway (LiteLLM, Portkey OSS, Bifrost, Kong) is free to license and gives you full control over routing, but it dumps infrastructure, uptime, and DevOps labor onto your team. A managed SaaS gateway (Portkey Pro, OpenRouter, Kong Konnect) charges a monthly fee and takes that labor off your plate, at the cost of usage-based billing and some vendor lock-in. Neither wins outright. The right call depends on your request volume, your governance requirements, and how much ops capacity you want to spend on plumbing instead of product.

What does an AI gateway actually do?

Any team running more than one model in production needs a layer between the application and the LLM providers. That's the AI gateway. It typically handles:

• Unified API access. One interface instead of separate integrations for OpenAI, Anthropic, Google, and open-weight models.
• Failover and routing. Automatic rerouting when a provider rate-limits or goes down.
• Cost tracking. Per-request, per-team, or per-model spend visibility.
• Observability. Logs, latency metrics, and traces across every model call.
• Guardrails. Prompt filtering, PII redaction, and policy enforcement before a request leaves the building.

Once a company runs more than one model in production, this layer stops being optional. Kong's data shows 62% of organizations on its AI Gateway already run two or more models simultaneously, and 20% run six or more.¹ The gateway isn't a nice-to-have anymore. It's plumbing.

The two paths: self-hosted open source vs. managed SaaS

On one side: the open-source stack. LiteLLM, Portkey's open-source core, Bifrost, and Kong's self-managed AI Gateway. You run the code, you own the servers, and you pay nothing for the license.

On the other side: the managed layer. Portkey Pro and Enterprise, OpenRouter, TrueFoundry, and Kong Konnect. You pay a subscription, and the vendor runs the infrastructure, the upgrades, and the on-call rotation.

It's a straightforward Build vs. Buy decision, the same one that shows up across open-source data infrastructure and internal AI agent workflows. The gateway layer is just the latest place it's landed.

Has open source caught up on features and performance?

Mostly, yes, on features. Performance is a different story.

OpenRouter's own comparison shows LiteLLM covering 2,500+ models across 100+ providers for free at the OSS tier, with Portkey close behind at 1,600+ models and 250+ providers, plus built-in HIPAA, SOC2, and ISO27001 compliance tooling that LiteLLM doesn't ship out of the box.² Feature parity is close enough that most teams won't hit a capability wall picking either one.

Performance is where the gap shows. In a head-to-head benchmark under identical 12-CPU allocations with no policies enabled, Kong's Konnect data planes processed requests 228% faster than Portkey OSS and 859% faster than LiteLLM, with 65% and 86% lower p95/p99 latency respectively.³ That's not a knock on LiteLLM's functionality. It's a reminder that free software still carries an engineering cost: tuning a self-hosted gateway for throughput takes real work a managed platform has already done for you.

Sticker price vs. total cost of ownership

Most teams get this math wrong. They compare LiteLLM's $0 license fee to Portkey's $49/month sticker price and conclude self-hosting is free money. It isn't.

A realistic self-hosted LiteLLM deployment, once you add Postgres, Redis, load balancing, and roughly 20% of a DevOps engineer's time, runs $500 to $2,500 a month, climbing to $3,000-$8,000 a month at enterprise scale above 10 million requests monthly.⁴ Compare that to Portkey Pro at a flat $49/month for unlimited logs, or its usage-based tiers scaling from a free 10K-log Dev plan upward.⁵ Kong Konnect Plus starts at $25/month per control plane, but a real production deployment with data planes, Postgres, and Kong-specific expertise runs an estimated $575-$1,425/month.⁴ The same pattern shows up in general-purpose API gateways too: self-hosting can run $50,000+ a year in infrastructure and engineering time, versus roughly $30,000 a year for a fully managed, production-ready gateway.⁶

The license is free. The operation isn't. That gap is the entire TCO argument, and it's the same one covered in the API trap piece on cloud cost overruns driving self-hosting: the sticker price and the total cost of ownership are two different numbers, and only one of them shows up on the vendor's pricing page.

When does self-hosting the gateway pay off?

The break-even point isn't about the gateway alone. It's about the volume flowing through it and what sits behind it.

• Under 5-10 million tokens/month. Stick with a managed gateway and commercial APIs. Hidden engineering, monitoring, and compliance costs routinely erase any savings below this threshold.⁷
• Above 100 million tokens/month. Self-hosting the inference layer behind your gateway can save $5M-$50M annually.⁷ One Fortune 500 legal-document case study replaced $1.05M/month in Claude Sonnet API spend with a self-hosted 8xH100 cluster running Llama 3.1 405B, cutting costs 75% and saving $9.4M a year with a 4.5-month payback.⁷
• DevOps capacity. Without engineers who can own Postgres, Redis, and load balancer tuning as a standing responsibility, the $500-$2,500/month self-hosted estimate will run higher in practice.⁴
• Data sovereignty. If prompts and responses can't leave your infrastructure for legal or contractual reasons, self-hosting stops being optional regardless of the cost math.
• Existing platform standardization. If you're already running Kong for API management, adding its AI Gateway plugin is a much smaller lift than adopting Kong from zero just for AI routing.⁴

The governance wildcard

Cost math assumes you're optimizing for price. Often you're actually optimizing for control, and that changes the calculus entirely.

Across organizations running AI traffic through Kong, less than 1% have any AI-specific governance controls in place: no prompt guards, no request or response filtering, no route-level policy.¹ Meanwhile 97.8% of enterprise AI requests go straight to third-party proprietary APIs, meaning prompts and data leave the company's perimeter with essentially no oversight.¹ Enterprise AI traffic through Kong grew 7x as a share of total platform traffic in nine months.¹ The volume is arriving faster than the governance is.

This is the same shadow-AI pattern covered in the shadow AI tech stack piece and the rules for AI-generated code security: employees and teams route around whatever infrastructure doesn't exist yet. A gateway, self-hosted or SaaS, is what turns ungoverned AI traffic into traffic you can actually see and control. If your real problem is that nobody knows where prompts are going, buying a managed gateway with built-in guardrails might solve that faster than building your own policy layer from scratch. If the problem is that those controls need to live entirely inside your own perimeter for compliance reasons, self-hosting is the only real answer.

Decision framework: build, buy, or hybrid

• Self-host if: you're above the 5-10 million token/month threshold, you already have DevOps capacity to own Postgres/Redis/load balancing, and data sovereignty or vendor lock-in avoidance matters more than convenience.⁴⁷
• Buy managed SaaS if: you're under that volume threshold, you lack dedicated ops headcount, or you need compliance certifications (HIPAA, SOC2, ISO27001) out of the box rather than built in-house.²⁷
• Go hybrid if: you self-host the inference layer for cost control at scale but keep a lightweight managed gateway in front for governance, observability, and multi-provider failover, capturing the savings of ownership without rebuilding every guardrail from scratch.

Treat the gateway the way you'd treat any other piece of infrastructure you're deciding whether to rent or own: as an asset with a real total cost, not a line item you can eyeball from a pricing page. The license fee tells you almost nothing. The ops bill, the governance gap, and your actual token volume tell you everything. Teams evaluating this trade-off for their own agent and application infrastructure, including platforms like Remy, are increasingly finding that the real decision point isn't capability parity anymore. It's whether you want to own the operational curve or pay someone else to own it for you.