- Published on
Rethinking the Deployment Unit for AI Agents on Kubernetes
- Authors
- Name
- DeepHorus
In a recent CNCF blog post, Lin Sun argues that Kubernetes Pods should be treated as execution workers for AI agents rather than the deployment unit itself. Drawing on experience from the kagent project, Sun highlights that as agent fleets scale, traditional Kubernetes abstractions become mismatched with agent lifecycle patterns. The post proposes a layered approach where a control plane above Kubernetes manages agent identity, lifecycle, and placement. This shift matters because scaling AI agents from a handful to thousands demands rethinking fundamental deployment concepts.
The challenges emerge when agent counts grow: isolating agents, assigning identities, enforcing access policies, and tracking individual agent activity. These are fundamentally agent platform problems, even if they must be expressed on Kubernetes. Initially, kagent ran many agents inside a single runtime, but later moved to a model where each agent became a full workload with its own Pod, Service, and ServiceAccount. That provided process isolation, identity integration, network policy enforcement, and per-agent observability. Such an approach brings Kubernetes-native scheduling and resource management directly to agents.
However, agents do not behave like microservices. They may wake only for a task, run for seconds, then idle; they can spawn subagents in parallel or pause indefinitely waiting for human approval. A dedicated Pod per potential agent is wasteful and inefficient. This has pushed the community toward an alternative: a control plane that abstracts agents as logical actors scheduled onto a pool of long-lived Pods. kagent subsequently added support for stronger isolation through the Kubernetes Agent Sandbox project.
Agent Substrate, introduced by Google alongside Agent Sandbox, exemplifies this shift. Kubernetes continues to manage Pods, Services, storage, and compute, while Agent Substrate handles logical agent placement and movement. Its abstractions mirror platform engineering concepts: a WorkerPool maps to a NodePool, Workers to Nodes, and an ActorTemplate to a Pod specification. Workers are real Pods; Actors are logical AI entities that are scheduled, suspended, or resumed as needed.
Kubernetes is only aware of WorkerPools and ActorTemplates. Workers and Actors live in Agent Substrate's API, with each Worker mapped to one Pod. This allows a fixed pool of Pods to support far more logical agents than would be practical with one continuously running Pod per agent. Pods become the execution worker layer, not the deployment model. The efficiency gain is significant for short-lived, bursty agent workloads.
The implications extend beyond scheduling. If an Actor can run on any Worker, identity may belong to the ActorTemplate, namespace, tenant, and version rather than a Pod. Access control, network policy, and runtime permissions must be expressed at the template level with per-Actor overrides. Ownership, quotas, and billing become harder to reason about when execution is not one-to-one with Pods. Observability must follow the logical agent across Pods, associating logs, traces, and audit records with the Actor wherever it was scheduled.
Industry reaction has been positive, with early adopters noting that this model aligns well with the bursty, event-driven nature of agent workloads. Kubernetes remains the industry standard for microservices and inference workloads, but for agent orchestration, a higher-level abstraction seems necessary. Expect further evolution in agent platform engineering, with kagent and Google's Agent Sandbox/Substrate leading the way. The next step is likely broader community adoption and refinement of these abstractions.
