The Trust Boundary
The boundary between the platform and customer code, what enforces it, and what crosses it.
The platform and the code it runs for customers live in different VPCs, on different Kubernetes clusters, with a peering between them that admits three ports in one direction. This chapter draws that line and everything on either side of it.
#Two VPCs, Two Clusters
The trusted VPC holds the platform: every role from Process Roles, the load balancers, Aurora, Valkey, the shared and dedicated Qdrant instances, the GPU pools, and the VPC endpoints. Nodes sit in private subnets with no public IPs. The node security group admits traffic only from inside the VPC; everything that reaches a node arrives with an in-VPC source, whether from a load balancer target or another node. NAT gateways provide egress.
The untrusted VPC holds the execution islands and nothing else: an EKS cluster whose only workloads are Kata sandbox pods and the small set of system add-ons they need. Its Kubernetes API endpoint is private, and its nodes carry encryption at rest. Its flow logs record every accepted and rejected flow, because this is where customer code runs and forensics matter. CloudWatch Container Insights runs here, configured for metrics only, with no container-log shipping off the cluster.
The two VPCs have non-overlapping address space and are connected by a VPC peering with routes in both directions. Routing is not reachability: peered traffic gets no security-group rule for free.
#The Three Ports
Three security-group rules on the untrusted cluster admit traffic from the trusted VPC. Nothing admits traffic in the other direction.
| Port | Source | Destination | Who uses it |
|---|---|---|---|
| Worker port | Trusted VPC | Release sandbox pods | The HTTP role, to configure a sandbox and execute methods |
| Dev-box port | Trusted VPC | Dev box pods | The sandbox-proxy role, for preview, control channel, language server and HMR |
| 443 | Trusted VPC | The untrusted Kubernetes API | The sandbox orchestrator, to create, watch, label and delete pods |
Identity is enforced separately from reachability. Reaching the Kubernetes API over port 443 grants nothing; the orchestrator authenticates with an EKS access entry for its Pod Identity IAM role, mapped to a Kubernetes group that is bound to one namespaced Role. That Role allows: pod create, get, list, watch, patch and delete in the sandbox namespace; log reads; pod metrics; event listing; and ConfigMap create and delete for mounting runner scripts. Nothing cluster-scoped.
The bearer token the orchestrator presents is a SigV4-presigned STS identity request, minted in-process. No kubeconfig and no long-lived cluster credential exists anywhere.
#What a Sandbox Can Reach
A NetworkPolicy in the sandbox namespace selects every pod carrying the sandbox label and allows egress only to the public internet. Every private range is denied: the trusted VPC, the untrusted VPC itself, the Kubernetes service network, and the instance metadata endpoint. The VPC CNI's network-policy agent is enabled, so the policy is enforced in the kernel and not merely logged.
Sandbox pods set dnsPolicy: None with public resolvers, so a sandbox cannot resolve a cluster-internal name. They mount no ServiceAccount token, so there is no credential to present to anything in the cluster.
The one path back into the platform is the callback origin: a public hostname, over HTTPS, through the edge, carrying a hook token scoped to one execution. App code reaches its database, the hook endpoints and the model service exactly as any client on the internet would, and is authorized the same way.
#The Front Door
An internet-facing Application Load Balancer terminates TLS for the HTTP, WebSocket, git and sandbox-proxy roles. It listens on 443 with the TLS 1.3 FIPS security policy and redirects 80 to 443. It is dualstack and targets pod IPs directly. A separate internet-facing Network Load Balancer carries SMTP to the mail-inbound role, also targeting pod IPs, so client addresses are preserved for the SMTP conversation.
Every app request arrives at the ALB from a Cloudflare Worker, which has already resolved the hostname to an app and rewritten the request so the platform reads the app, host and path from query parameters. The platform never routes on the Host header. See Edge and Custom Domains.
#Private Paths to AWS Services
Two kinds of VPC endpoint keep platform traffic off the public internet.
S3 gateway endpoints in both VPCs route S3 traffic over the AWS network. This covers every object read and write. It also covers container image layers, since ECR stores them in S3.
A Bedrock runtime interface endpoint in the trusted VPC has private DNS enabled, so the SDK's calls to the regional Bedrock hostname resolve to a private address. Model inference through Bedrock never crosses the public internet. The endpoint's security group admits HTTPS from the VPC; the real gate is the IAM policy on the pod identity.
#Administrative Access
Neither cluster is reachable from the public internet for administration. Each VPC has a bastion managed through AWS Systems Manager with no public IP and no inbound SSH. The bastions act as subnet routers on a private mesh network, so an operator's laptop reaches private endpoints through an authenticated tunnel. On the untrusted side the security group scopes API access to the bastion, so only the management router and the orchestrator can reach it.
Deploy automation reaches the private untrusted API the same way: a GitHub Actions job opens an SSM port-forward through the bastion with an IAM policy that permits exactly that session and nothing broader.
#Detection and Logging on the Network
- VPC flow logs on both VPCs.
- GuardDuty with EKS runtime monitoring on the clusters, managed by GuardDuty itself.
- CloudTrail for every control-plane call in the account.
- Container Insights on the untrusted cluster for fleet metrics over time.
Figure 04, "The Trust Boundary": two VPCs side by side, a crimson trust boundary down the middle, three one-way crossings, and a callback that re-enters through the public front door. Left, the Trusted VPC. Private subnets (no public IPs) hold the platform data plane (Aurora, Valkey, Qdrant), the VPC endpoints (S3 gateway, Bedrock over private DNS), and the three roles that reach across: the Sandbox orchestrator, the HTTP role, and the Sandbox-proxy. Public subnets hold the front door: an ALB on 443 with a TLS 1.3 FIPS policy, targeting pod IPs, and an NLB for SMTP. Admin is over an SSM bastion with no inbound SSH. Right, the Untrusted VPC. The Untrusted Kubernetes API (a private endpoint) manages a sandbox namespace holding the Kata pods: a release sandbox on kata_apps, a dev box on kata_dev, and job runners on kata_jobs. Two grey governing annotations sit beside them: the orchestrator's identity (an EKS access entry bound to one namespaced Role, a SigV4 STS token, no kubeconfig) and the NetworkPolicy (egress to the public internet only, all private ranges denied, no ServiceAccount token, dnsPolicy None). Flow logs record every accepted and rejected flow. Down the gutter runs the crimson boundary, labeled ONE-WAY TRUST BOUNDARY (VPC PEERED), the same label the Topology figure uses. The two VPCs are peered for connectivity, but each crossing is a separate security-group admit. Three crimson crossings cross it, trusted to untrusted only, each labeled with its port: 443 to the Kubernetes API, the worker port to the release sandbox, the dev-box port to the dev box. Nothing crosses back. Along the bottom, the callback loop: a sandbox's only path back leaves to an off-figure public internet and re-enters at the public ALB over HTTPS with a per-execution hook token, never back across the boundary. Crimson is spent only on the boundary, its label, and the three crossings.