How to Self-Host Your Company's AI Stack: Replacing SaaS APIs with Local Models

The short answer

You self-host AI for your business by running open-weight models on GPUs you control, using an inference layer (like the open-source Superlinked Inference Engine, or a lighter edge runtime like FreeToken) instead of calling a metered API for every request. It makes sense once your usage is steady and high volume, or once compliance rules say your data cannot leave your network. Below that line, a hosted API is still cheaper, and pretending otherwise is how self-hosting projects fail.

This is a build-vs-buy decision with a real number attached, not a vibe. Work the number first, then build.

Start with the number that actually decides this

Self-hosting trades a per-token bill for a fixed GPU cost. An API meters every token. A GPU costs the same whether it sits idle or runs flat out, so the entire question is whether you can keep it busy enough to beat the metered price.

Superlinked's own break-even math, presented at Infer() Summit 2026, puts the crossover for a dense 32B model on one GPU at roughly 10 million requests a month against a $0.50-per-million-token managed API.¹ Below that, the GPU sits idle and the API wins by a wide margin. Mixture-of-experts and smaller 4B-class models cross sooner; an idle GPU never crosses at all.¹

Against budget-tier frontier APIs, the line moves even further out. DeepSeek V4-Flash currently prices at roughly $0.14 per million input tokens and $0.28 per million output tokens, with cache hits near $0.003 per million.² One industry analysis puts the self-hosting break-even against that pricing at around 5.7 billion tokens a month on a single H100, a volume almost no mid-market company generates.³ Against a premium frontier API, the same analysis puts the crossover closer to 100 to 256 million tokens a month, and even that assumes 60 to 70 percent sustained GPU utilization.³

The practical takeaway: self-hosting a big generalist chat model to replace GPT-class API calls rarely pays off for a normal business. Self-hosting the specific, high-frequency, repetitive tasks buried in your agent stack pays off constantly.

Where the crossover actually happens

Step 1: Separate frontier reasoning from document plumbing

Most of what an AI-powered business actually runs is not frontier reasoning. It is OCR, parsing, entity extraction, embeddings, and reranking, the plumbing that feeds an agent before it ever generates a sentence. This work is high-volume and repetitive, which is the worst possible shape for per-token pricing and the best possible shape for a GPU you own.⁴

Superlinked's SIE (Superlinked Inference Engine) was built specifically for this layer. It is an open-source, Apache 2.0 inference server that runs OCR, document parsing, structured extraction, embeddings, and reranking from one cluster, reached through a single SDK.⁴ Superlinked reports that its batch-then-route design reaches about 89 percent GPU efficiency, versus roughly 51 percent for the naive route-then-batch pattern most teams build by hand, which works out to roughly 1.8x the throughput per GPU at the same latency.¹

The savings are not hypothetical. Intercom's Chief AI Officer, Fergal Reid, reported cutting about $250,000 a month by replacing a hosted GPT call with a fine-tuned 14B Qwen model on a single pipeline task inside its Fin AI product.⁴

Step 2: Pick models that match the job, not your ego

You do not need a 671-billion-parameter model to clean up a transcript or extract a vendor name from an invoice. Match model size to task:

• Frontier-class reasoning, self-hosted: DeepSeek V3 and V3.1 are open-weight, MIT-licensed, 671B-parameter mixture-of-experts models with 37B activated parameters per token, deployable via vLLM, SGLang, or LMDeploy on your own GPU cluster.⁵ Running the full model requires datacenter-class hardware, typically 8x H100 or H200 GPUs. Distilled versions, from 1.5B to 70B parameters, run on much smaller hardware for teams that need strong reasoning without a data center.
• Small, task-specific cleanup: S1-mini from Superwhisper is a 0.6B, 484MB open-weight model, released under Apache 2.0 with a naming clause, that reformats and cleans raw transcripts entirely on-device with zero network calls.⁶ It reached 94.8 percent token accuracy on a held-out set of 7,519 cases in Superwhisper's own evaluation.⁷ That is the model class you want for narrow, repetitive jobs: small, cheap to run, good enough.
• Edge and consumer-hardware inference: FreeToken, from research group FlashML, is an edge-native serving system that treats a laptop or gaming desktop as a unified inference platform rather than a small GPU. It splits mixture-of-experts model execution between GPU cache and CPU based on measured PCIe and CPU bandwidth, letting a single workstation GPU run models as large as 753B parameters and a laptop run 35B-class models.⁸ It is a research-grade tool, not enterprise infrastructure, but it shows how far consumer hardware has moved.

Step 3: Size the hardware to the workload, not the marketing

A sub-40B model at FP8 quantization typically fits on a single rentable GPU today. As of mid-2026, an entry-tier L4 or A10 (24GB) rents for roughly $0.39 to $1.00 an hour and handles 4B-class models and embeddings. A workhorse A100 80GB or RTX PRO 6000 (96GB) runs $0.90 to $1.80 an hour and covers 30B-class models or mixture-of-experts. An H100 80GB, for genuine headroom, runs $1.80 to $4 an hour.¹ Full-precision DeepSeek V3-class models need multiple H100s or H200s; Northflank's own deployment guide recommends 8x H200 GPUs for DeepSeek-V3.1 with vLLM.

Embeddings are the cheapest entry point. Superlinked's benchmarks show a self-hosted bge-m3 embedder running on a single GPU at roughly $0.0036 per million tokens, against $0.12 to $0.13 per million tokens for hosted providers like Voyage or OpenAI, a difference of more than 30x at scale.¹

Step 4: Adapt the model instead of replacing it

Open weights are not fixed. Superlinked's own LoRA experiment on a 568M-parameter multilingual embedder took a single adapter, trained on one L4 GPU in 41 minutes for about 80 cents, and lifted German legal retrieval accuracy from 0.2613 to 0.3129, a 19.7 percent gain, with a small trade-off on general-domain tasks that a mixed training set mostly recovered.¹ That is the actual argument for owning your model layer: you can tune it for your domain for the price of a coffee, something no API vendor will do for you.

Step 5: Deploy air-gapped if compliance requires it

If your data cannot leave your network, self-hosting stops being optional. SIE ships the same Docker image, Helm chart, and Terraform modules from a laptop to a Kubernetes cluster, with model-weight snapshots for fully air-gapped deployment.⁴ This matters more than it sounds. On January 30, 2025, Italy's data protection authority blocked DeepSeek's chatbot from processing Italian users' personal data after the company failed to give an adequate account of its data handling, a decision that put a fine point on why document pipelines carrying personal or contractual data attract regulatory attention.⁴ Flexera's 2025 State of the Cloud Report found about 70 percent of enterprises now run hybrid infrastructure across an average of 2.4 public clouds, which is the practical argument for an inference layer that installs the same way anywhere rather than living inside one vendor's API.⁴

Step 6: Rent the API everywhere else

Self-hosting is not a purity test. Rent the API when volume is low or spiky, when you need the absolute frontier model for the hardest reasoning tasks, or when your whole inference bill is still small enough that GPU operations are not worth the engineering time.¹ This is the same build-vs-buy logic that applies to any internal tool: own the software an asset sits underneath rather than renting metered access to it, but only once the volume and the operating capacity justify the switch.

The bottom line

Most companies calling this decision wrong are comparing the sticker price of a GPU to their API bill and stopping there. The real comparison is fully loaded self-hosting cost, hardware, electricity, DevOps time, redundancy, against sustained token volume, not a peak estimate. Below roughly 10 million requests a month for mid-sized open models, or 100 million-plus tokens a month against a premium frontier API, keep renting. Above it, or the moment compliance takes the decision out of your hands, start moving the plumbing in-house first, then the reasoning.