ReferenceAsk
Remy Reference/Architecture/Architecture/Process Roles
02Architecture

Process Roles

The single container image, the roles it splits into, and the reasons behind that split.

The platform is one container image. Every process on the trusted cluster is that image started with a role flag, and each role is its own Kubernetes Deployment. This chapter names the roles, describes what each one does, and explains why the split falls where it does.

The image is built from a single TypeScript codebase. The domain lives in three groups. The foundation group imports no domain and covers logging, errors, config, cache and the database base. The platform group is the business of running the product: organizations, users, auth, billing, the model catalog, storage and the queue contract. The customer-apps group is what customers build and host: releases, data sources, SDK actions and app users. The v1 codebase is its own peer beside them. Each role is a folder whose entry file is the process. A role holds only its own transport code and imports the domain. No role imports another, so anything two roles share lives in the domain. In development, the same image runs every role in one process.


#The Roles

RoleWhat it does
HTTPThe REST API, app serving, method dispatch, hook endpoints, admin, and the app-events and presence streams (Server-Sent Events)
WebSocketThe WebSocket server: editor sockets and dev-session events
DB socketThe socket sandboxes run their database queries over. It subscribes to nothing on the event bus, so traffic there cannot slow a query
General workerDrains the step, ingest, file-scan, email-send and both SES queues
Bulk workerDrains the bulk-ingest queue for corpus loads
GitSmart HTTP git server: each app's remote, mirrored to S3
App databaseOwns and serves each app's SQLite databases
Mail inboundSMTP listener that turns inbound mail into method invocations
Sandbox proxyThe only public path to a dev box: preview, control, language server
ProvisionerReconciles dedicated resources; hosts the platform's batch crons
Sandbox orchestratorRelease pods, job runners and dev boxes on the untrusted cluster

The voice worker is a separate image, because its LiveKit bindings need a different base. It registers with the HTTP role and deploys alongside it.


#Why the Split Falls Where It Does

Each role exists for one of three reasons: it holds long-lived connections, it carries rights no request pod should hold, or its lifecycle cannot ride the API's rollout cadence.

Long-lived connections. The WebSocket role and the sandbox proxy role hold hour-long sockets: an editor's control channel, a language server session. A rolling deploy of the API tier must not cut these off, and a pod holding one of these sockets must not block the request tier from scaling down. Both roles roll on their own cadence, with their own grace periods. The app-events and presence streams that an app's visitors hold are the exception by design. They are Server-Sent Events on the HTTP role, and an API rollout ends them within its drain. The client SDK reconnects and reconciles, because delivery was only ever at most once.

Rights that request pods must not carry. Two roles talk to a Kubernetes API. The provisioner creates StatefulSets for dedicated retrieval instances on the trusted cluster; the sandbox orchestrator creates pods on the untrusted cluster. Each runs under its own ServiceAccount, scoped to exactly those rights, and the HTTP pods carry none of them. The HTTP role reaches the orchestrator over an internal route with a server token, never with cluster credentials.

Lifecycle. The app database role holds open SQLite connections and flushes them on a debounce, so it drains carefully and hands ownership to a peer on shutdown. The git role holds repositories on local NVMe. The mail inbound role listens on SMTP behind a Network Load Balancer. The provisioner is a singleton, which also makes it the right home for the batch crons that must run exactly once. Two of these are long: the billing-events loader and the revenue rebuild. Each is a minutes-long crawl with a scratch file, and both stay off the pods that serve requests.

Worker pools. The general worker drains queues whose jobs run seconds to minutes: steps, document ingest, file scans, email sends. The bulk worker drains corpus loads, where one batch can saturate a JavaScript thread for minutes at a time. They run on separate node groups, so a bulk load never takes a core from interactive work.


#How Roles Find Each Other

Roles do not import each other. They meet through the data plane and through internal routes.

  • Postgres is the shared state. Every role opens the same three pools: writer, reader, and a report reader with a longer timeout.
  • Valkey carries cache namespaces, pub/sub events for cache invalidation, ownership records for app databases, the warm-pool registry, and the live audit tail. It is encrypted at rest and requires TLS on every connection. It admits one user, the platform's, which signs in with short-lived IAM tokens; unauthenticated connections are refused.
  • The job queue carries work from the HTTP role to the workers. It is SQS in the cloud and pgmq in the platform's own Postgres on a self-host. The message catalog has one module per message type, with its shape, its producer helper, and the handler the worker registers.
  • Internal HTTP routes connect the HTTP role to the app database, git, and sandbox orchestrator roles. Each is guarded by a server token that never leaves the cluster.
  • The callback origin is the one hostname a sandbox reaches back on. Every managed capability an app uses arrives there as a request with a hook token.

#Workload Identity

Every pod runs under an EKS Pod Identity that maps its ServiceAccount to an IAM role with explicit resource ARNs. None of the roles that touch data carry a wildcard.

Used byGrants
HTTP, WebSocket, workers, mail inboundS3 in every region, SQS, Cognito, Bedrock, KMS (the one key), Secrets Manager (the org bundles), Firehose, SES
GitBoth app buckets in every region; the audit stream
App databaseThe private bucket in every region
Sandbox proxyThe audit stream
ProvisionerEnqueue rebuilds; a namespaced Role for the dedicated namespace
Sandbox orchestratorDescribe the untrusted cluster; an access entry on that cluster bound to a pod-CRUD Role
Model trainerThe training queue and its artifacts
Model servingTuned-model artifacts

Worker pods disable the Kubernetes ServiceAccount token mount. Their only identity is the AWS one.


#The Rest of the Repository

The domain groups are the platform, not utilities. One module builds every DAO and holds the three Postgres pools. The model catalog, pricing, adapters and runtime sit in the platform group, alongside billing, auth, audit, storage and the counterparty registry. Data sources, releases and the SDK actions sit in the customer-apps group. The boundaries between groups are checkable rules: the foundation group imports no domain, and the platform group imports the customer-apps group only through a short named list.

Every deployable is its own package with its own build. These are the API (the one image every role runs), the voice worker, model serving and training, the six sandbox images the orchestrator runs, and the edge, which is made up of seven Workers. The repository root holds only what spans them: the Kubernetes manifests, CI, the local compose stack and the docs.

The manifests are one per role, with kustomize overlays per environment. Build-time code generation and the validators that run in CI live beside the API. Every schema change is kept as raw SQL migrations.

What This Gives an Operator
One artifact to build, scan, and version. A deploy is one tag.
Each role scales, rolls, and fails independently.
Rights are attached to the role that needs them, and to nothing else.
Adding a role is a folder, a manifest, and a ServiceAccount.
DiagramFIG. 02 — PROCESS ROLES
platform imagedifferent --roleweb · API clientsbrowser socketseditor ⇄ sandboxinbound SMTPHTTPREST · SSE · :3000WebSocketeditor socketsSandbox proxyHTTP+WS · dev boxMail inboundSMTP → methodsProvisionercrons · singletonApp databaseinternal · SQLiteGitsmart HTTP · NVMeGeneral workerSQS · step · scanBulk workerSQS · corpus loadsSandbox orch.untrusted podsTrusted K8s APIprovisioner SA · scopedUntrusted K8s APIorchestrator SA · scopedVoice workerseparate image · LiveKitPostgreswriter · reader · reportValkeycache · TLS · IAMSQSjobs → workersS3objects · git mirrorONE IMAGECLUSTER APISSHARED DATA PLANEevery role opens the same poolsrights no request pod holds/_internal · token
container image (artifact)running role / backing serviceoff-figure endpointgroupingroute · work path
One image, ten roles. Every process on the trusted cluster runs the same platform image, launched with a different --role, each in its own Deployment. Roles never import each other; they meet only through the shared data plane and guarded /_internal routes. Three planes surround them: public traffic from the internet, the cluster APIs only two roles may call, and the pools every role opens. The voice worker is the exception: a separate image, deployed alongside the HTTP role.

``` Figure 02, "Process Roles": a vertical internals figure. The image is a left vertical artifact spine, the platform image launched with a different role flag; its eleven role peers sit in two rows inside one dashed "ONE IMAGE" zone, surrounded by the three planes they touch. Front-facing roles (top row): HTTP (REST, :3000, plus SSE streams), WebSocket (realtime, sockets), Sandbox proxy (HTTP + WS, dev box), Mail inbound (SMTP to methods), Provisioner (crons, singleton). Internal and worker roles (bottom row): App database (internal, SQLite), DB socket (sandbox queries), Git (smart HTTP, NVMe), General worker (queue), Bulk worker (queue, corpus loads), Sandbox orchestrator (untrusted pods). Above the front-facing roles, four dashed off-figure endpoint chips mark the external parties that reach them: web and API clients, browser sockets, the editor and its dev box, and inbound SMTP. Each points down into its role, crossing the top of the image zone. To the right, a dashed "CLUSTER APIS" zone holds the two Kubernetes APIs only two roles may call: Provisioner reaches the Trusted K8s API and Sandbox orchestrator reaches the Untrusted K8s API ("rights no request pod holds"). The Voice worker sits above the zone as a labeled exception: a separate image on a LiveKit base, deployed alongside HTTP. Along the bottom, a "SHARED DATA PLANE" floor of Postgres, Valkey, the job queue (SQS in the cloud, pgmq on a self-host), and S3. Roles do not import each other and are not wired individually to these stores; a note reads "every role opens the same pools". The one drawn data-plane path is the queue: HTTP produces into it and the General and Bulk workers drain it. Inside the image, HTTP is the hub of the guarded internal routes ("internal routes · server token"), reaching the App database, Git, and Sandbox orchestrator roles. Every connector is grey and orthogonal with one arrowhead style; there is no trust boundary in this figure, so there is no crimson.