Process Roles
The single container image, the roles it splits into, and the reasons behind that split.
The platform is one container image. Every process on the trusted cluster is that image started with a role flag, and each role is its own Kubernetes Deployment. This chapter names the roles, describes what each one does, and explains why the split falls where it does.
The image is built from a single TypeScript codebase. The domain lives in three groups. The foundation group imports no domain and covers logging, errors, config, cache and the database base. The platform group is the business of running the product: organizations, users, auth, billing, the model catalog, storage and the queue contract. The customer-apps group is what customers build and host: releases, data sources, SDK actions and app users. The v1 codebase is its own peer beside them. Each role is a folder whose entry file is the process. A role holds only its own transport code and imports the domain. No role imports another, so anything two roles share lives in the domain. In development, the same image runs every role in one process.
#The Roles
| Role | What it does |
|---|---|
| HTTP | The REST API, app serving, method dispatch, hook endpoints, admin, and the app-events and presence streams (Server-Sent Events) |
| WebSocket | The WebSocket server: editor sockets and dev-session events |
| DB socket | The socket sandboxes run their database queries over. It subscribes to nothing on the event bus, so traffic there cannot slow a query |
| General worker | Drains the step, ingest, file-scan, email-send and both SES queues |
| Bulk worker | Drains the bulk-ingest queue for corpus loads |
| Git | Smart HTTP git server: each app's remote, mirrored to S3 |
| App database | Owns and serves each app's SQLite databases |
| Mail inbound | SMTP listener that turns inbound mail into method invocations |
| Sandbox proxy | The only public path to a dev box: preview, control, language server |
| Provisioner | Reconciles dedicated resources; hosts the platform's batch crons |
| Sandbox orchestrator | Release pods, job runners and dev boxes on the untrusted cluster |
The voice worker is a separate image, because its LiveKit bindings need a different base. It registers with the HTTP role and deploys alongside it.
#Why the Split Falls Where It Does
Each role exists for one of three reasons: it holds long-lived connections, it carries rights no request pod should hold, or its lifecycle cannot ride the API's rollout cadence.
Long-lived connections. The WebSocket role and the sandbox proxy role hold hour-long sockets: an editor's control channel, a language server session. A rolling deploy of the API tier must not cut these off, and a pod holding one of these sockets must not block the request tier from scaling down. Both roles roll on their own cadence, with their own grace periods. The app-events and presence streams that an app's visitors hold are the exception by design. They are Server-Sent Events on the HTTP role, and an API rollout ends them within its drain. The client SDK reconnects and reconciles, because delivery was only ever at most once.
Rights that request pods must not carry. Two roles talk to a Kubernetes API. The provisioner creates StatefulSets for dedicated retrieval instances on the trusted cluster; the sandbox orchestrator creates pods on the untrusted cluster. Each runs under its own ServiceAccount, scoped to exactly those rights, and the HTTP pods carry none of them. The HTTP role reaches the orchestrator over an internal route with a server token, never with cluster credentials.
Lifecycle. The app database role holds open SQLite connections and flushes them on a debounce, so it drains carefully and hands ownership to a peer on shutdown. The git role holds repositories on local NVMe. The mail inbound role listens on SMTP behind a Network Load Balancer. The provisioner is a singleton, which also makes it the right home for the batch crons that must run exactly once. Two of these are long: the billing-events loader and the revenue rebuild. Each is a minutes-long crawl with a scratch file, and both stay off the pods that serve requests.
Worker pools. The general worker drains queues whose jobs run seconds to minutes: steps, document ingest, file scans, email sends. The bulk worker drains corpus loads, where one batch can saturate a JavaScript thread for minutes at a time. They run on separate node groups, so a bulk load never takes a core from interactive work.
#How Roles Find Each Other
Roles do not import each other. They meet through the data plane and through internal routes.
- Postgres is the shared state. Every role opens the same three pools: writer, reader, and a report reader with a longer timeout.
- Valkey carries cache namespaces, pub/sub events for cache invalidation, ownership records for app databases, the warm-pool registry, and the live audit tail. It is encrypted at rest and requires TLS on every connection. It admits one user, the platform's, which signs in with short-lived IAM tokens; unauthenticated connections are refused.
- The job queue carries work from the HTTP role to the workers. It is SQS in the cloud and pgmq in the platform's own Postgres on a self-host. The message catalog has one module per message type, with its shape, its producer helper, and the handler the worker registers.
- Internal HTTP routes connect the HTTP role to the app database, git, and sandbox orchestrator roles. Each is guarded by a server token that never leaves the cluster.
- The callback origin is the one hostname a sandbox reaches back on. Every managed capability an app uses arrives there as a request with a hook token.
#Workload Identity
Every pod runs under an EKS Pod Identity that maps its ServiceAccount to an IAM role with explicit resource ARNs. None of the roles that touch data carry a wildcard.
| Used by | Grants |
|---|---|
| HTTP, WebSocket, workers, mail inbound | S3 in every region, SQS, Cognito, Bedrock, KMS (the one key), Secrets Manager (the org bundles), Firehose, SES |
| Git | Both app buckets in every region; the audit stream |
| App database | The private bucket in every region |
| Sandbox proxy | The audit stream |
| Provisioner | Enqueue rebuilds; a namespaced Role for the dedicated namespace |
| Sandbox orchestrator | Describe the untrusted cluster; an access entry on that cluster bound to a pod-CRUD Role |
| Model trainer | The training queue and its artifacts |
| Model serving | Tuned-model artifacts |
Worker pods disable the Kubernetes ServiceAccount token mount. Their only identity is the AWS one.
#The Rest of the Repository
The domain groups are the platform, not utilities. One module builds every DAO and holds the three Postgres pools. The model catalog, pricing, adapters and runtime sit in the platform group, alongside billing, auth, audit, storage and the counterparty registry. Data sources, releases and the SDK actions sit in the customer-apps group. The boundaries between groups are checkable rules: the foundation group imports no domain, and the platform group imports the customer-apps group only through a short named list.
Every deployable is its own package with its own build. These are the API (the one image every role runs), the voice worker, model serving and training, the six sandbox images the orchestrator runs, and the edge, which is made up of seven Workers. The repository root holds only what spans them: the Kubernetes manifests, CI, the local compose stack and the docs.
The manifests are one per role, with kustomize overlays per environment. Build-time code generation and the validators that run in CI live beside the API. Every schema change is kept as raw SQL migrations.
``` Figure 02, "Process Roles": a vertical internals figure. The image is a left vertical artifact spine, the platform image launched with a different role flag; its eleven role peers sit in two rows inside one dashed "ONE IMAGE" zone, surrounded by the three planes they touch. Front-facing roles (top row): HTTP (REST, :3000, plus SSE streams), WebSocket (realtime, sockets), Sandbox proxy (HTTP + WS, dev box), Mail inbound (SMTP to methods), Provisioner (crons, singleton). Internal and worker roles (bottom row): App database (internal, SQLite), DB socket (sandbox queries), Git (smart HTTP, NVMe), General worker (queue), Bulk worker (queue, corpus loads), Sandbox orchestrator (untrusted pods). Above the front-facing roles, four dashed off-figure endpoint chips mark the external parties that reach them: web and API clients, browser sockets, the editor and its dev box, and inbound SMTP. Each points down into its role, crossing the top of the image zone. To the right, a dashed "CLUSTER APIS" zone holds the two Kubernetes APIs only two roles may call: Provisioner reaches the Trusted K8s API and Sandbox orchestrator reaches the Untrusted K8s API ("rights no request pod holds"). The Voice worker sits above the zone as a labeled exception: a separate image on a LiveKit base, deployed alongside HTTP. Along the bottom, a "SHARED DATA PLANE" floor of Postgres, Valkey, the job queue (SQS in the cloud, pgmq on a self-host), and S3. Roles do not import each other and are not wired individually to these stores; a note reads "every role opens the same pools". The one drawn data-plane path is the queue: HTTP produces into it and the General and Bulk workers drain it. Inside the image, HTTP is the hub of the guarded internal routes ("internal routes · server token"), reaching the App database, Git, and Sandbox orchestrator roles. Every connector is grey and orthogonal with one arrowhead style; there is no trust boundary in this figure, so there is no crimson.