Voice
How an app answers the phone and holds a realtime voice conversation, with LiveKit for media, Telnyx for telephony, and a voice worker that bridges to the model and exposes the app's methods as tools.
An app can answer the phone, place calls, and hold a realtime voice conversation in the browser, with its methods available to the conversation as tools. The media runs through LiveKit and telephony through Telnyx; a dedicated voice worker bridges the two to the realtime model providers and back to the platform. This chapter covers the components and a call's lifecycle.
#The Components
The voice worker is its own image and Deployment, a LiveKit Agents process that registers with LiveKit Cloud and receives a job per session. It imports nothing from the platform's code; the whole boundary is HTTP, typed as a wire contract that the two sides evolve together. It holds no keys of its own either: its one secret is a standing token, with which it fetches the deployment's LiveKit and model keys from the platform, once at boot to register and again as each call starts, so a key rotated in the admin reaches the next call. Its lifecycle is call-shaped: surge-only rollouts, a long drain on SIGTERM so a live call finishes, and scaling on CPU.
LiveKit carries the media. A browser joins a room directly over WebRTC. A phone call arrives from Telnyx over SIP into a room. The worker joins the room as the agent.
Telnyx provides the phone numbers. The platform searches, orders, and configures numbers on the workspace's behalf, binds them to apps, sets caller-ID names, and bills monthly rental for dedicated numbers. A sandbox tier of shared numbers serves development.
The platform mints every session, holds the configuration, resolves the engine, receives tool calls, stores transcripts, and bills usage.
#Engines
A voice interface declares its engine in its interface config. Two shapes are supported: a native speech-to-speech model with a voice, or a cascaded pipeline of a language model, a speech-to-text model, a text-to-speech model, and a voice. The platform normalizes either into a canonical engine and resolves each slot against the model catalog. Slot credentials follow the workspace's vendor controls: a workspace's own key for a provider is used when present, and the platform's otherwise. The same resolution runs at build time to reject an engine that is not wired, and at billing time to price what ran.
#A Call's Lifecycle
- Admit. A browser calls the app's voice routes, a method places an outbound call, or an inbound call arrives on a bound number. One admission sequence serves all three: the workspace must be able to pay (balance and budgets, with one auto-refill), and the app and the caller must be under their concurrent-call limits. The platform then creates the session row, with its channel recorded explicitly as web, phone-in, or phone-out, and mints the worker's per-session credential.
- Dispatch. The platform sends the worker into the room itself, with dispatch metadata carrying the credential, the caller's identity, and, for a phone call, how the caller may verify. The browser's LiveKit token only joins the room. An inbound call is the exception: LiveKit's dispatch rule starts the job, and the worker registers the call with the platform using its standing token, which answers with the same things. From then on the worker authenticates every request with the session's own credential.
- Configure. The worker fetches the runtime configuration: the resolved engine, the tools the interface exposes, and the app's instructions.
- Converse. The worker bridges the room's audio to the realtime model. When the model calls a tool, the worker posts the call to the platform, which runs the method through the normal execution path with the voice session as its originating identity, and returns the result to the model. Tools can be fast, slow, or background; a background tool releases the turn and resolves later.
- Transcribe. Spoken turns and tool rounds append to the session's transcript as they happen, with previews rather than full payloads for tool traffic.
- Finalize. When the call ends the worker finalizes the session; the platform closes the transcript, records duration and model usage per slot, and bills the workspace. A call the platform ends itself (a suspension, a deletion, a transfer), or whose worker died, is billed from the usage snapshot the worker flushes every few seconds.
Outbound calls start from a method through the SDK, authenticated with the hook token, and follow the same lifecycle. A caller can list, read, and delete their own finished calls; the app's workspace sees every call its app has had.
``` Left: a browser (WebRTC) and a phone (PSTN → Telnyx → SIP) both entering a LiveKit room. Center: the voice worker joins the room as the agent. Arrows: worker ⇄ realtime model provider (audio/text) worker → platform: the deployment's keys (standing token, at boot and each call's start) · register (standing token, inbound only) · runtime config · tool calls · transcript appends · finalize (all with the per-session credential) platform → method execution for each tool call (voice session as identity) Right: platform: session row (channel: web | phone-in | phone-out) · engine resolution (S2S or LLM+STT+TTS+voice, catalog-backed, workspace keys honored) · transcript · billing. Bottom: Telnyx number management: search → order → bind to app → caller-ID → monthly billing.