ReferenceAsk
Remy Reference/Architecture/Architecture/Portability
35Architecture

Portability

What separates the platform from the substrate it runs on (the sandbox provider, hosts, routing, the capability manifest, and sizing), the compose stack that runs the whole platform on a laptop, and the three deployment profiles of one artifact.

The cloud estate described in the preceding chapters is one deployment of a platform built to run in more than one place. This chapter names what is built for that: the seams that separate the platform from the substrate it runs on, and the local stack that already runs the whole thing on a laptop. It closes with where the platform is going.


#What Is Built

One image with roles. Every platform process is the same image launched with a role, and a development process runs every role at once. Nothing about the split into Deployments is baked into the code; it is a property of the manifests.

The sandbox provider seam. The orchestrator talks to sandboxes through the contract in Execution Islands: seven methods and a warm pool, declarative, with no exec and no file writes. Two providers implement it. The Kubernetes provider runs Kata pods on the untrusted cluster in the cloud. The Docker Engine provider runs containers under whatever runtime the operator registers, and is chosen whenever no cluster is configured, so it is the laptop default. Remote sandbox services fit the same interface. The release manager, the job runners and the dev boxes above it never know which one they are on.

One difference is invisible to them but matters to an operator. On Docker, every sandbox shares one bridge network with every other, and the worker's port is unauthenticated. One Docker host is therefore one trust domain, fit for builders who would let each other's apps run side by side.

Hosts as configuration. Every hostname the platform reads or emits comes from configuration with development defaults derived in code: the app apex, the sandbox apex, the callback origin sandboxes reach back on, the git host, the platform apexes. No personal or environment-specific hostname exists in code.

The frontend is one build for every deployment. The Worker that serves it writes the api's origin and the platform assets' base into each page, and the api's public deployment document supplies the rest, from the WebSocket and git hosts to the session cookie's domain.

Four audiences read a hostname: the browser, a sandbox, an external sender, and in-network services. The split between them is a divergence in values only, never in code paths. The one exception is deliberate: a dedicated host for outside senders serves only their receivers. A laptop's tunnel is the case for it, publishing one host to the internet for webhooks while the rest of the stack stays on the machine. In the cloud the callback host is the api's own, and the exception never applies.

The routing contract. The platform reads the app, host and path from query parameters and never routes on the Host header. Any reverse proxy that maps a hostname to those parameters is a valid edge for routing. The features above the mapping, such as image transformation on an app's own host, travel with the Workers. See Edge and Custom Domains.

The counterparty registry as a capability manifest. Every external service is a surface that declares where its credential lives. A deployment that sets the key has the capability. A deployment that does not gets a clean "not configured" answer rather than an error at a vendor. The admin surface that renders the registry is the manifest of what this deployment can do.

Not every capability gets a stand-in off the cloud. One small enough to go without is simply off until its vendor is configured. Browser rendering is the example: the PDFs and images an app generates from HTML, and the snapshots of app pages handed to crawlers, belong to Cloudflare Browser Run, so a deployment without it has none.

The compose stack. The repository's compose stack runs the whole platform from images built out of the checkout, the way a deployment runs it:

  • Postgres, which also carries the job queues as pgmq.
  • Valkey, the cloud's Redis engine.
  • Qdrant.
  • Fluent Bit, collecting the audit log into the stack's own object store.
  • The npm registry.
  • The API and worker roles, and the voice worker.
  • ClamAV and Presidio, as an optional profile.
  • A local edge.

A development overlay layers source mounts, file watching, the frontends' development servers and Mailpit on top. Mailpit takes the stack's mail over SMTP and delivers none of it. The base compose file alone is the run mode a self-hosted deployment grows from. New workspaces' files go to SeaweedFS, the stack's own storage region.

The local edge is Caddy terminating TLS for a wildcard development domain, with CoreDNS answering the wildcard. Caddy sends the platform's own hosts straight to their roles. Every app host goes to the production app Worker itself, run on workerd, so local app routing is the Worker's own code rather than a copy of it. In run mode the frontends are served the same way, by their own static-site Workers over their compiled bundles.

The SDKs come from the deployment. Every sandbox resolves the platform's own npm packages through the deployment's own registry, set by one setting that the sandbox images' npm configuration reads. A deployment that publishes its builds there, as a laptop's watchers do, runs its apps on those. One that publishes nothing passes npm's releases through.

Sizing in one place. Every sandbox lifecycle and pool setting lives in one configuration file, and every scaling threshold lives in the manifests and Terraform, so a deployment of a different size changes numbers, not code.


#Three Profiles of One Artifact

ProfileRuns onSubstrateOperated by
CloudThe platform's AWS accountThe Terraform in this repository's sibling: two VPCs, EKS with Kata, Aurora, Valkey, S3 in four regions, CloudflareThe platform team
DedicatedAn AWS account the customer owns, or one the platform team runs for themThe same Terraform, stamped into another accountThe platform team, through a cross-account role
Self-hostedA laptop or a single serverCompose: Postgres, Valkey, an S3-compatible object store (SeaweedFS), local Qdrant, Docker sandboxes, the Workers on workerd behind CaddyThe customer

The cloud profile is the fully built reference deployment this folder documents. The dedicated profile is the same Terraform in another account. The self-hosted profile is the direction. The compose stack runs the platform end to end today, and the remaining work is the pieces that are multi-tenant operational machinery in the cloud, made pluggable so a single-tenant operator can run them. Several are done:

  • Sandbox provider. A single-tenant operator runs plain containers where the cloud runs microVMs.
  • Object storage. A self-host's store is a storage region like the cloud's four.
  • Stored secrets. A self-host encrypts them under its own key.
  • Email. A self-host sends through Resend or Postmark, with its apps' own sending domains and delivery events, or through any SMTP relay, every app from the relay's one address. The cloud's per-app tenants stay the cloud's, and a deployment with no transport sends no email and says so.
  • Sign-in. A self-host signs its people in with built-in accounts or its own identity provider over OpenID Connect, and can turn password sign-in off for everyone but its owner (see Identity and Auth).
  • Billing. A deployment with no payment processor sells no plans and no credits. Its workspaces get every feature, their usage is recorded at list prices rather than drawn from a balance, and the budgets their owners set are what cap spending. The cloud cannot drift into that by losing a key, because a payment processor named without its keys stops the platform from starting.

The rest of the edge remains.

Three pieces of that direction have a settled shape:

  • Object storage is a region. The regions are configuration. Each is a row that names its driver (S3 in an AWS region, or any S3-compatible store, SeaweedFS in the compose stack), its buckets, its CDN apex and whether the internet reaches it. References, public URLs and moves between regions behave the same whatever backs a region. No region is special in code; the cloud's four are rows in its deployment configuration. A self-host runs one region or several, and a laptop runs its own beside the cloud's, so its existing data keeps working while new workspaces stay on the machine. A model vendor is handed an image as a URL only from a region the internet reaches. A self-host's images go to the vendor as bytes, read inside the deployment's own network, or are uploaded to the vendor first, since no vendor can fetch from a laptop or a company's network.
  • The edge travels as code. Off Cloudflare, the same Workers run on workerd behind a TLS proxy, with image transformation handed to imgproxy. A self-hosted app keeps its on-domain image transforms and the file, image and video hosts.
  • Stored secrets are sealed under a key from a registry. App secrets, connected accounts' tokens and every vendor key, the platform's own and each workspace's, are sealed by one module into the platform database. In the cloud the key is a KMS key; off AWS it is a key the deployment generates at setup. Each value names the key that sealed it, so a laptop reads what it stored under KMS beside what it stores now. Losing a self-host's key loses what is sealed under it, the same bargain every self-hosted product makes with its master key. See Secrets.

Moving between profiles is a migration into a deployment someone operates, never an in-place upgrade of a box.

What This Gives a Customer
Confidence that the architecture is not welded to one account or one vendor's edge.
A path to a dedicated deployment on the same code and the same infrastructure definitions.
A platform whose local development environment is the product running whole, which is what makes a self-hosted edition a configuration surface rather than a fork.
DiagramFIG. 35 — PORTABILITY
The platform artifactone image, many rolesworkers, sandbox imagesthe same code everywhereFive pluggable pointssandbox providerhosts as configurationrouting contractcapability manifestsizing in one placeCloudthe platform's AWS accountEKS + Kata, Aurora, Valkey, S3, CloudflareDedicatedan AWS account the customer ownsthe same Terraform, another accountSelf-hosteda laptop or a single serverCompose: Postgres, Redis, Qdrant, Docker, CaddyPORTABILITY · ONE ARTIFACT, THREE PROFILESeach profile plugs into the same pointsthe sandbox contract · Fig 05the direction: pluggable identity, sandbox, ingress
artifact · point · profileone artifact, three profiles
One artifact, three profiles. The platform is one container image launched as many roles, with its workers, its sandbox images, and the same code in every deployment. Five pluggable points separate it from the substrate it runs on: the sandbox provider (six methods and a warm pool), the hostnames it reads and emits (all configuration), the routing contract (the app, host, and path from query parameters, never the Host header), the counterparty registry as a capability manifest (a service with no key configured simply reports that it is not configured), and the sizing that lives in one place. Three profiles implement those points on different substrate: the cloud profile is the fully built reference this material documents, on the platform’s AWS account with EKS, Kata, Aurora, Valkey, S3, and Cloudflare; the dedicated profile is the same Terraform stamped into an account the customer owns; and the self-hosted profile is the compose stack that already runs the whole platform on a laptop, with the remaining multi-tenant machinery (identity, the sandbox provider, and ingress) being made pluggable. That last profile is the direction. No trust boundary is drawn here, so there is no crimson.

``` Figure 35, "Portability": one artifact, five seams, three profiles. Center: the platform artifact (the heavy node): one image launched as many roles, plus the Workers and sandbox images, the same code everywhere. Around it, five seams drawn as sockets: the sandbox provider, hosts as configuration, the routing contract, the capability manifest (the counterparty registry), and sizing in one place. A cross-reference points to Figure 5 for the sandbox contract. Three deployments plugged into the sockets, each implementing those seams on a different substrate: Cloud (EKS with Kata, Aurora, Valkey, S3 in four regions, Cloudflare Workers), Dedicated (the same Terraform in an account the customer owns), and Self-hosted (compose: Postgres, Valkey, SeaweedFS, Qdrant, Docker sandboxes, the Workers on workerd behind Caddy, imgproxy). The self-hosted deployment is labelled the direction. No trust boundary is drawn in this figure, so there is no crimson.