Caches, Queues and Events
The cache and coordination layer every role shares, the queues that carry work to the workers, and the realtime channel that reaches into an app's browser.
Three things carry state between platform roles that is not the database: Valkey for cache and coordination, SQS for work, and a realtime event primitive that reaches all the way into an app's browser. This chapter covers each, and why the work is split across the queues it is.
#Valkey
One ElastiCache Valkey replication group, Multi-AZ with automatic failover, serves every role. It is encrypted at rest and requires TLS on every connection. It admits one user, the platform's, which signs in with short-lived IAM tokens, so a connection that does not authenticate is refused. It is never a system of record; everything in it can be rebuilt from Postgres, S3 or the pods themselves. What lives there:
- Cache namespaces. Every cached read is under a versioned key prefix, with tag-based invalidation, so a schema change in what is cached is a prefix bump rather than a flush.
- Pub/sub events. Cache invalidation and cross-process signals ride pub/sub, so a write on one pod invalidates every pod's view.
- Ownership records. Which app database pod owns which database, with heartbeats and a reverse index. See App Databases.
- The warm-pool registry. The atomic pop that hands out sandboxes, the leader lease, and the pool's demand samples. See Release Sandboxes.
- Locks and leases. Per-repository push and sync locks on the git server, dev-session leases, migration status.
- Interface sessions and hook tokens. Short-lived credentials for app users and for executions.
- The live audit tail. The most recent events per workspace, for an instant dashboard view ahead of the durable pipeline.
- Adaptive vendor limits. Per-counterparty concurrency ceilings that shrink on refusal and regrow on success.
#The Queues
Work moves over SQS to the worker fleets. It comes from the roles that hand it off, from the workers themselves (a slice queuing the next, the object mover, a moving app's deferred work), and, for three queues, directly from S3, SES and EventBridge. Each queue exists because sharing it with another kind of work would let one kind sit in front of the other.
| Queue | Carries | Why it is separate |
|---|---|---|
| Steps | Every SDK action a Remy method calls, and MindStudio workflow runs, with a caller waiting | The interactive queue |
| Ingest | Data-source documents, minutes each | A corpus upload must not sit in front of a step |
| Bulk ingest | Batches of documents for corpus loads | A backfill must not sit in front of an interactive add; its own worker pool and node group |
| File scan | S3 object-created events, and the hourly sweep's due scans | Feeds the AV and PII scanners; latency-insensitive |
| Email send | Outbound email chunks | Fan-out sends off the request path |
| SES events | Per-message delivery events | Delivery tracking |
| SES reputation | Per-app tenant reputation findings, through EventBridge | How an app's SES pause becomes a visible state |
| Training | Jewel training jobs | Consumed only by the trainer's own identity |
Ingest, bulk ingest, file scan and both SES queues carry dead-letter queues: a message that reliably kills its handler is parked for inspection after a bounded number of attempts rather than retried forever. A failed message comes back after a delay rather than at once, so a brief blip does not spend those attempts in a second. The step queue has no dead-letter queue, and neither does email send, deliberately: a chunk that cannot send yet is normal there, and a dead-letter queue would write off a paused app's campaign. A batch whose long copy chain was interrupted by a rollout is handed back rather than cut.
The message catalog has one module per message type, carrying its shape, the producer helper any role calls, and the handler the worker registers. Producer and consumer read the same definition. A message that arrives at a processor with no handler is thrown back to the queue rather than deleted, so a routing bug leaves evidence.
The processor long-polls, runs up to a per-queue concurrency, and extends each message's visibility with a heartbeat while its job runs, up to a thirty-minute ceiling per job. On SIGTERM it stops taking new messages and gives in-flight work up to 270 seconds on the general fleet and 540 seconds on the bulk fleet, inside each pod's grace period. It then hands back what is still running: a step answers its caller, and other work returns to the queue. Every processor runs in the same worker image; the general pool drains everything but bulk, the bulk pool drains only bulk, and a development process drains all of them.
The worker fleets scale on queue depth through KEDA. See Scale and Reliability.
The queues sit on a seam that hides the transport. In the cloud it is SQS. On a self-host it is pgmq in the platform's own Postgres, installed by migrations of its own, with the same holds, redelivery and dead letters and nothing more to run. A self-host sizes its workers by hand, with a concurrency per queue, rather than scaling them on depth.
#Realtime Events
Apps get a server-to-client realtime primitive with two invariants.
Authorization happens in a method. A method publishes to channels, which are plain app-chosen strings. A method also grants a client the right to hear a list of channels, and a grant can only be minted under a hook token, from inside an execution. Who may hear what is always app code, under the platform's normal auth rules. The client holds a platform-served event stream that delivers matching publishes, and the stream applies no policy beyond the grant it was minted with.
Channels are scoped by environment. Publisher and grantor both derive the scope from their hook token, so a method running in a dev session can never publish into live users' browsers, and a preview never crosses into live.
Grants are short-lived by design: fifteen minutes by default, an hour at most. The stream closes at expiry, and the client re-mints through the app's method, which re-evaluates access. A grant may also permit a client to publish, within deliberately small bounds, for ephemeral signals like cursors and typing indicators.
The WS role carries the platform's own realtime traffic: editor sockets and dev-session events. Every WS pod subscribes to the bus, so every pod parses every publish in the fleet. For that reason the WebSocket over which a sandbox's database queries reach the app database role is a role of its own, the DB socket. It subscribes to nothing, so a flood on the bus cannot slow an app's queries.