Caches, Queues and Events
The cache and coordination layer every role shares, the queues that carry work to the workers, and the realtime channel that reaches into an app's browser.
Three things carry state between platform roles, none of it in the database: Valkey for cache and coordination, SQS for work, and a realtime event primitive that reaches all the way into an app's browser. This chapter covers each, and why the queues are split the way they are.
#Valkey
One ElastiCache Valkey replication group, Multi-AZ with automatic failover, serves every role. It is never a system of record; everything in it can be rebuilt from Postgres, S3 or the pods themselves. What lives there:
- Cache namespaces. Every cached read is under a versioned key prefix, with tag-based invalidation, so a schema change in what is cached is a prefix bump rather than a flush.
- Pub/sub events. Cache invalidation and cross-process signals ride pub/sub, so a write on one pod invalidates every pod's view.
- Ownership records. Which app database pod owns which database, with heartbeats and a reverse index. See App Databases.
- The warm-pool registry. The atomic pop that hands out sandboxes, the leader lease, and the pool's demand samples. See Release Sandboxes.
- Locks and leases. Per-repository push and sync locks on the git server, dev-session leases, migration status.
- Interface sessions and hook tokens. Short-lived credentials for app users and for executions.
- The live audit tail. The most recent events per workspace, for an instant dashboard view ahead of the durable pipeline.
- Adaptive vendor limits. Per-counterparty concurrency ceilings that shrink on refusal and regrow on success.
#The Queues
Work moves from the HTTP role to the workers over SQS. Each queue exists because sharing it with another kind of work would let one kind sit in front of the other.
| Queue | Carries | Why it is separate |
|---|---|---|
| Steps | v1 workflow steps with a caller waiting | The interactive queue |
| Ingest | Data-source documents, minutes each | A corpus upload must not sit in front of a step |
| Bulk ingest | Batches of documents for corpus loads | A backfill must not sit in front of an interactive add; its own worker pool and node group |
| File scan | S3 object-created events | Feeds the AV and PII scanners; latency-insensitive |
| Email send | Outbound email chunks | Fan-out sends off the request path |
| SES events | Per-message delivery events | Delivery tracking |
| SES reputation | Reputation notifications | Account-level signals |
| Training | Jewel training jobs | Consumed only by the trainer's own identity |
Ingest and bulk-ingest carry dead-letter queues: a document that reliably kills its handler is parked for inspection after a bounded number of attempts rather than retried forever, and a batch whose long copy chain was interrupted by a rollout is handed back rather than cut.
The message contract is one module per message type, carrying its shape, the producer helper any role calls, and the handler the worker registers. Producer and consumer read the same definition. A message that arrives at a processor with no handler is thrown back to the queue rather than deleted, so a routing bug leaves evidence.
The processor long-polls, runs up to a per-queue concurrency, and extends each message's visibility with a heartbeat while its job runs, up to a thirty-minute ceiling per job. On SIGTERM it stops taking new messages and lets in-flight work finish within the pod's grace period. Every processor runs in the same worker image; the general pool drains everything but bulk, the bulk pool drains only bulk, and a development process drains all of them.
The worker fleets scale on queue depth through KEDA. See Scale and Reliability.
#Realtime Events
Apps get a server-to-client realtime primitive with two invariants.
Authorization happens in a method. A method publishes to channels, plain app-chosen strings, and grants a client the right to hear a list of them; a grant can only be minted under a hook token, from inside an execution. Who may hear what is always app code, under the platform's normal auth rules. The client holds a platform-served event stream that delivers matching publishes, and the stream applies no policy beyond the grant it was minted with.
Channels are scoped by environment. Publisher and grantor both derive the scope from their hook token, so a method running in a dev session can never publish into live users' browsers, and a preview never crosses into live.
Grants are short-lived: fifteen minutes by default, an hour at most. The stream closes at expiry, and the client re-mints through the app's method, which re-evaluates access. A grant may also permit a client to publish, within deliberately small bounds, for ephemeral signals like cursors and typing indicators.
The WS role carries the platform's own realtime traffic: editor sockets, dev-session events, and the WebSocket over which a sandbox's database queries reach the app database role.