Models and Routing

Every AI model call on the platform, from an app's method, from the agent, from the editor's coding agent, or from the voice worker, passes through one model service. The catalog of models is code; each model declares who made it, who serves it, what it costs, and where its data goes; and a workspace controls which providers it uses and with whose keys. This chapter covers the catalog, the routing of a call, and the controls around it.

#The Catalog

The catalog lives in code, one file per model, keyed by publisher: who made the model. Each entry declares the model's id, type, capabilities, pricing, the adapter that speaks to its serving vendor, and the product facts the picker shows. A generated registry indexes the whole catalog at build time; adding a model means adding a file and regenerating the registry, never calling an admin API.

Adapters are keyed by provider: who serves the model. The two are deliberately different trees. Many publishers have no serving API of their own; they are reached through a hosting vendor. One publisher's model can be served by several providers. Adapters extend a base class and take a resolved credential in their constructor, so an adapter never reads a key from configuration.

Model types cover chat and reasoning, embeddings, image generation and editing, video, speech-to-text, text-to-speech, realtime voice, document extraction, reranking, and more, each with its own input and output vocabulary.

#Routes

A model can declare several routes: serving arrangements for the same product. Each route names its adapter, the vendor's wire name for the model, the service it egresses through, the capabilities it supports, and its facts: the data boundary and the region where it is processed. A route can carry its own rate card when the vendor's price differs from the product's.

A call picks one route, in this order:

1. A pin. A caller can name a route explicitly with model@route. A pin is hard: if the route is not usable for this workspace, the call fails rather than silently rerouting.
2. The workspace's own key. A route whose service the workspace holds a key for is preferred.
3. The capability filter. A request that needs server-side tools, a hosted connector, or image input drops routes that cannot serve it.
4. The preferred route set by the operator for the model, or the primary route when none is set.

A route whose service the workspace has disabled is removed before selection; the model as a whole is unavailable only when no route survives. One deterministic route per model per workspace also keeps prompt caches coherent: a conversation never splits across providers.

The facts on each route are shown to the user in the model picker, so a workspace can see and choose where inference runs.

#Where Inference Goes

Every provider is a surface in the counterparty registry, with its hosts, data classes, retention mode, training policy, and agreements. When a provider's agreement excludes a feature, the adapter strips that feature from requests instead of attempting it. See Counterparties and Egress.

#The Model Service

The model service is the single chokepoint. For every call it:

• resolves the model and route as above, through the model factory and the clearinghouse;
• checks the workspace's credit and plan state;
• admits the call under the capacity gate: a lease keyed to whose key, which provider, and which model, shared fleet-wide through Redis; it shrinks on a vendor refusal and regrows on success, with bulk loads waiting behind a reserve while interactive calls fail fast rather than hang;
• executes through the adapter, streaming where the caller streams;
• bills the workspace: every adapter declares the billing events it can emit, each event has a declared price in the catalog, and a call under a workspace's own key bills under distinct own-key events at the platform's own-key rate;
• records metrics with the route as a dimension, so cost, latency, and error rates are readable per app, per model, and per serving route;
• classifies errors so a workspace-state condition, such as a disabled model, is reported to the caller as exactly that, and never logged as a platform failure.

#Workspace Controls

• Enable and disable any provider, or a single model of a provider.
• Bring your own key for a provider or a single model, honored by every route through that provider.
• Default models per workspace for the agent, the editor, and the picker.
• Capability overrides granted by the operator, additive per workspace, for models or features outside a plan.
• Custom and local models: a workspace can register its own endpoint as a model, billed at no platform charge.