What Changes in Your Infrastructure Contract When Personal AI Agents Go From Hundreds to Millions of Tenants
Scaling agents to millions of tenants rewrites four contract terms.

Per-user persistent AI agent deployment at scale requires a new infrastructure contract. Moving from hundreds to millions of tenants changes four terms, which are isolation, state management, billing, and resource guarantees. A platform that gives every customer an agent that remembers context must enforce all those terms together and keep model choice separate from the computer the agent runs on.
Why agents at scale are an infrastructure problem, not an AI problem
A chatbot’s infrastructure contract is built for a short exchange. A request arrives, the model answers, and the resources are released. An agent instead acts as a persistent, resumable worker, with a session that may last minutes or hours. It needs state that survives pauses and restarts, plus a browser, an OS environment, credentials, scheduling, and storage. Killing that worker mid-task is not equivalent to ending a completed web request.
That mismatch is there even when the fleet is small. A better model does not preserve a worker’s memory when its process disappears, and more servers do not define how a paused task resumes. Cloudflare’s April 2026 Agent Cloud expansion says that the infrastructure in use today, including always-on virtual servers and isolated sandboxes, cannot scale to a world where every user and employee has dozens of personal agents running at the same time. Most teams discover the mismatch at the wrong moment, when they are already past hundreds of tenants.
What "hundreds of tenants" actually means in infrastructure terms
At a few hundred tenants, the usual arrangement is shared containers and one orchestration service. The service combines the inbound API handler, LLM client, session store, tool execution engine, and scheduling logic. Redis and Postgres hold state outside the running process, while long tasks go to a background worker queue. Tenants pay a flat-rate bill or a rough share of the total. This is enough to support a working demo and a small fleet.
The control plane can be quite small. An orchestration service written in Rust or Go can run on under 2 vCPUs and 2 GB of RAM, with the inbound gateway needing only 32–128 MB. At this stage, LLM API calls dominate infrastructure spending. Docker/runc containers are familiar to operate and quick to provision, but they share the host kernel. A kernel exploit in one container can therefore compromise the node.
With fewer co-tenants, fewer tenants are at risk from that exploit. Resource headroom also masks contention, and state stored in a database can cover straightforward sessions without a more involved persistence contract. Fairness, tenant-level billing, and isolation guarantees remain informal or absent. Teams manage them by convention. That is the starting contract the larger fleet inherits.
The isolation clause that breaks first when tenant count climbs
Higher density enlarges the blast radius of a single compromise. The relevant number is the co-tenants on the affected host. As that number rises, a shared-kernel failure exposes more tenants at once. Agents make this boundary especially important because they execute shell commands, make API calls, and write files. Code that is LLM-generated or user-supplied must be treated as hostile.
The isolation hierarchy starts with standard Docker/runc containers, which share the host kernel. gVisor adds a user-space kernel called Sentry that intercepts syscalls before they reach the host, strengthening the boundary without full hardware VM overhead. Firecracker microVMs go further through hardware virtualization and a separate kernel for each tenant. For arbitrary code execution, that distinction belongs in the contract.
A stronger boundary does not solve dirty recycling. After a sandbox is released, it can still hold workspace files, environment variables with injected secrets, or background processes from its last tenant. The replacement contract must require environment scrubbing and SIGKILL for all non-system processes before reuse. Infrastructure must perform these duties. Cloudflare’s Sandboxes GA provides persistent, isolated Linux environments with a shell, filesystem, and background processes per agent, but any platform still needs explicit terms for tenant separation and reuse.
How state management must be renegotiated at scale
Even with correct isolation, a long-running task can lose its state. Redis, Postgres, and a worker queue can hold application records, but that alone does not preserve an agent’s running machine when a worker is preempted or a host fails. A container restart loses in-memory state. The persistence contract must therefore cover the execution environment as well as the records the application chose to save. Those are different forms of state.
Firecracker’s snapshot restore resumes the machine. It memory-maps the snapshot file, loads the CPU state, and continues execution from the pause point without booting a kernel, running init, or restarting the agent. The VM does not know it was stopped. Preserving in-memory state provides a basis for sleeping between tasks.
However, a machine snapshot captures machine state, and external effects already produced through API calls, record writes, or sent messages require workflow-level checkpointing. Restoring memory alone cannot bring those effects into line with the task as it resumes. Workflow-level checkpointing must address external context, nondeterministic replay, and persistent effects as well as internal state. Temporal, Inngest, DBOS, and LangGraph’s checkpointer belong in this discussion because the new contract requires coordination between machine recovery and workflow recovery, a choice to use both.
Why per-tenant billing shape breaks before infrastructure does
The ability to sleep changes the price contract. A personal agent may work for seconds, sit idle for hours, and then resume. An always-on VM allocation bills across that interval regardless of use. Flat-rate or coarse-grained allocation can be affordable when tenants are few and predictable, but idle billing destroys unit economics as that pattern spreads across the fleet.
Per-second metering with no compute charges for sleeping machines matches this workload. The infrastructure layer must record sleep and wake events accurately because application-level estimates are not enough to establish what compute was consumed. Persistent identity need not mean permanent active allocation. Cloudflare’s Dynamic Workers framing addresses this distinction by arguing that a separate container for every agent is too expensive for mainstream adoption. The cost of isolation must be separated from the cost of keeping resources continuously allocated.
LLM spending needs tenant-level attribution as well. Sticky routing uses consistent hashing to keep prompt prefixes warm in a provider’s cache, so routing choices can change the cost of serving a tenant. Round-robin routing obscures which tenants drive that spending. A tenant’s bill must be based on the LLM API calls they actually use. Yet accurate attribution is only half the bargain, because charging tenants separately does not stop one high-volume tenant from consuming resources others need.
What resource guarantees must look like when agent density is high
At low density, spare capacity conceals resource conflicts. At high density, that spare capacity disappears, and one noisy tenant can degrade every co-located agent. CPU, DRAM, SSD, and network are all contention surfaces for agent workloads. Unlike GPU-bound inference, these workers use conventional compute across all four. A resource contract limited to CPU does not cover the full problem.
Fairness must therefore become an infrastructure primitive. The scheduler must enforce a minimum allocation per tenant regardless of what other tenants are doing. Otherwise, a customer running their first few sessions competes on best-effort terms against a customer running very high agent volumes. Billing cannot repair that imbalance after the fact. Per-tenant cost attribution is meaningful only when the allocation behind it is also enforceable.
Warm pools make this guarantee concrete at startup. Pre-booted idle sandboxes can be acquired without waiting for a sandbox to cold-start at the moment of demand. For eval and RL workloads that need many identical environments at once, copy-on-write snapshot forking avoids a full allocation from scratch for each environment. These mechanisms make different parts of the resource promise affordable. The control plane’s own footprint must also stay bounded and predictable so that orchestration does not become another source of contention.
How the isolation, state, billing, and resource contracts change together when regulated industries are involved
Healthcare, lending, and insurance add obligations that engineering preference cannot override. The exact legal scope still needs care. HHS’s official CMS guidance states that organizations outside the definitions of covered entity and business associate do not have to comply with HIPAA rules. HHS’s Business Associates FAQ says who the covered entities are and explains that some of the rules also apply to business associates. Within that scope, an agent accessing protected health information does not remove the organization’s duties under HIPAA’s Privacy Rule, Security Rule, and Breach Notification Rule.
The record must show what the agent did. Logging only that an agent was invoked does not describe its access to a patient record, retrieval of a lab result, or generation of a clinical summary. That calls for trajectory-level observability, with traces that can branch, merge, pause for human input, and span many steps. Preserved, queryable workflow checkpoints support immutable audit trails of AI-assisted decisions tied to the EHR record. A VM snapshot alone is not an audit record.
Per-tenant microVM isolation supplies a boundary, while recycling controls address the risk of one tenant’s PHI appearing in another tenant’s environment. HIPAA’s scope does not itself establish a universal requirement for a named virtualization technology. The engineering requirement is to make protection enforceable. In the same way, sleep/wake billing must remain compatible with preserved workflow records, while resource guarantees must support the workflows those records describe.
Legacy applications add a further constraint on the environment. EHR, LOS, DMS, and Citrix/VDI automation may require the agent to use the same GUI a person uses because there is no API alternative. That requires a full OS environment, such as Linux, Windows, or Android. Regulated workflows also need live desktop handoff so a human can take over at any point. This is a compliance control, and it belongs in the infrastructure agreement.
The revised infrastructure contract, term by term
For a team choosing infrastructure, the first requirement is an explicit tenant boundary. Each tenant must have its own kernel boundary through a microVM or isolation that gives the same protection. When a sandbox is released, its processes must be stopped and environment scrubbing must be done before another tenant uses it. The provider must own that lifecycle. A dedicated agent should guarantee enforceable separation of environments.
The second requirement is coordinated persistence across machine and workflow state. The contract must provide VM-level snapshot and restore so agents can sleep and wake without a cold restart. It must also provide workflow-level checkpointing that tracks external side effects and coordinates them with restored execution. Both primitives need to be specified together. For regulated deployments, the agreement must additionally cover preserved, queryable workflow records and any required human handoff.
The third requirement is metering that follows actual use. Compute must be billed per second, with no charges for sleeping machines, and LLM costs must be charged to the tenant whose API use caused them. There must be a clear fairness mechanism to protect smaller workloads from high-volume tenants. Cost attribution and access must agree. A bill that identifies each tenant accurately is incomplete if the platform cannot enforce that tenant’s share.
The fourth requirement is a minimum resource allocation enforced by infrastructure. The agreement must cover the contention surfaces the agent uses, with warm pools so startup has no cold-start delay and copy-on-write snapshot forking when many identical environments are needed at once. The control plane must have a bounded footprint.
Cloudflare’s April 2026 expansion offers one public example through Dynamic Workers, Sandboxes GA, Artifacts, and Think. Teams building their own stack can use that articulation to distinguish baseline primitives from differentiators, while evaluating each contract term explicitly. Model choice remains a separate decision throughout. The customer brings their own model and keys, because the infrastructure runs the computer.


