Kubernetes Sandbox
Run your agent’s commands and file edits in a sandbox pod on your own Kubernetes cluster, claimed from a kubernetes-sigs/agent-sandbox warm pool.
While Pydantic AI Harness is on 0.x releases, the API may change between minor releases; when it does, deprecation warnings and release-note migration guidance tell you (or your agent) exactly how to upgrade. See the version policy.
flowchart TB
accTitle: Kubernetes Sandbox: your agent's commands run in a sandbox pod on your cluster
accDescr: On your machine, the model calls Coder, Shell or FileSystem tools such as run_command. Via ctx.workspace, KubernetesSandbox's get_workspace claims a pod from a SandboxWarmPool through the Kubernetes API with your kubeconfig, and runs commands through the pod's sandboxd over a kubectl port-forward or an in-cluster address. The claim deletes itself after sandbox_timeout or when destroy(ref) is called. A later run reattaches with the WorkspaceRef in the message history or result.workspace.ref.
subgraph machine["Your machine"]
model(["model"])
tools["Coder, Shell or<br/>FileSystem tools,<br/>such as run_command"]
ref["WorkspaceRef<br/>in the message history,<br/>result.workspace.ref"]
k8s["KubernetesSandbox()<br/>get_workspace claims<br/>a pod for the run"]:::capability
end
subgraph cluster["Your Kubernetes cluster"]
pool["SandboxWarmPool<br/>of ready pods"]
pod["sandbox pod running sandboxd<br/>deleted after sandbox_timeout<br/>or by destroy(ref)"]:::external
end
model <-->|"tool call, result"| tools
tools -->|"ctx.workspace"| k8s
ref -.->|"a later run<br/>reattaches"| k8s
k8s -->|"claim, Kubernetes API,<br/>your kubeconfig"| pool
pool --> pod
k8s <-->|"commands, kubectl port-forward<br/>or in-cluster address"| pod
classDef capability stroke:#E520E9,stroke-width:2px
classDef external stroke:#0E9C86,stroke-width:2px
pip install "pydantic-ai-harness[coder,kubernetes,anthropic]"
uv add "pydantic-ai-harness[coder,kubernetes,anthropic]"
The coder extra is there because the examples use Coder; the sandbox itself doesn’t need it. The anthropic extra is there because the examples use an Anthropic model; swap it for your model provider’s extra. The client uses your kubeconfig, or the pod’s service account inside the cluster. The default port-forward connection also needs kubectl on your PATH.
Install agent-sandbox (v1.0.7 or later) with its extensions:
kubectl apply --server-side -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v1.0.11/sandbox-with-extensions.yaml
Each sandbox pod must run sandboxd, agent-sandbox’s in-pod server. Commands run in the container that runs sandboxd, so that container’s image decides which tools the agent has. This template copies sandboxd into python:3.12 and keeps three sandboxes warm:
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxTemplate
metadata:
name: python
spec:
podTemplate:
spec:
automountServiceAccountToken: false
securityContext: {runAsNonRoot: true, fsGroup: 1000}
initContainers:
- name: install-sandboxd
image: registry.k8s.io/agent-sandbox/sandboxd:v1.0.11
command: [/bin/sh, -c, cp /usr/local/bin/sandboxd /sandboxd-bin/sandboxd]
securityContext: {runAsUser: 1000}
volumeMounts: [{name: sandboxd-bin, mountPath: /sandboxd-bin}]
containers:
- name: app
image: python:3.12
command: [/sandboxd-bin/sandboxd, --root-dir=/workspace]
env: [{name: HOME, value: /workspace}]
securityContext: {runAsUser: 1000, allowPrivilegeEscalation: false}
readinessProbe: {httpGet: {path: /v1/health, port: 8080}}
volumeMounts:
- {name: sandboxd-bin, mountPath: /sandboxd-bin, readOnly: true}
- {name: workspace, mountPath: /workspace}
volumes:
- {name: sandboxd-bin, emptyDir: {}}
- {name: workspace, emptyDir: {}}
---
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
name: python-pool
spec:
replicas: 3
sandboxTemplateRef: {name: python}
For stronger isolation, set runtimeClassName (gVisor or Kata) on the pod template and connect from inside the cluster.
from pydantic_ai import Agent
from pydantic_ai_harness.coder import Coder
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
agent = Agent('anthropic:claude-opus-5-5', capabilities=[KubernetesSandbox(warm_pool='python-pool'), Coder()])
result = agent.run_sync('Clone https://github.com/pydantic/pydantic-ai and summarize how capabilities work.')
Coder’s shell and file tools now run in a sandbox pod, not on your machine. The run claims an already-running pod from the warm pool the first time it needs the workspace; with Coder, RepoContext does that when the run starts, even without a tool call. The sandbox keeps running after the run ends; see Clean up.
from pydantic_ai import Agent
from pydantic_ai_harness.coder import Coder
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
agent = Agent('anthropic:claude-opus-5-5', capabilities=[KubernetesSandbox(warm_pool='python-pool'), Coder()])
result = agent.run_sync('Clone https://github.com/pydantic/pydantic-ai and summarize how capabilities work.')
followup = agent.run_sync(
'Which capability would you add next, and where would it live?',
message_history=result.all_messages(),
)
The follow-up run finds the sandbox in the message history and works in it, so the clone is still there. If the claim has been deleted, the run raises WorkspaceUnavailableError instead of starting over in an empty sandbox.
For a narrower agent, use Shell and FileSystem instead of Coder, or write your own tool that runs in the sandbox:
from pydantic_ai import Agent, RunContext
from pydantic_ai_harness.file_system import FileSystem
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
from pydantic_ai_harness.shell import Shell
agent = Agent('anthropic:claude-opus-5-5', capabilities=[KubernetesSandbox(warm_pool='python-pool'), Shell(), FileSystem()])
@agent.tool
async def run_python(ctx: RunContext, code: str) -> str:
"""Run a Python snippet in the sandbox."""
result = await ctx.workspace.run(['python', '-c', code], timeout=10)
return result.stdout + result.stderr
Commands start in sandboxd’s root directory, /workspace in the template above, and run as the container’s user.
Claiming or reattaching gives up with TimeoutError after 190 seconds: the 180 seconds the SDK waits for a new claim to become Ready, plus one API server request. A claim that times out is deleted. A command timeout starts once the sandbox is claimed. A timeout or a cancelled run kills the command’s process group, and the sandbox keeps running for other commands. WorkspaceTimeoutError carries the output so far. A job that detaches with setsid may keep running.
A background job that keeps the command’s output open delays the result by about 5 seconds. A command whose combined output passes 10 MiB is stopped and raises WorkspaceOutputLimitError, with the first 64 KiB of each stream in stdout and stderr. Redirect large or background output to a file instead; Shell.start_command manages its own output log.
To come back to the sandbox without the message history, pass its ref back as workspace=:
from pydantic_ai import Agent
from pydantic_ai_harness.coder import Coder
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
agent = Agent('anthropic:claude-opus-5-5', capabilities=[KubernetesSandbox(warm_pool='python-pool'), Coder()])
result = agent.run_sync('Clone https://github.com/pydantic/pydantic-ai and summarize how capabilities work.')
ref = result.workspace.ref # store this, e.g. in your database
later = agent.run_sync('Which capability would you add next, and where would it live?', workspace=ref)
The ref is <namespace>/<claim name>. It holds no credentials, so the process that reattaches needs its own access to the cluster. KubernetesSandbox reattaches only to claims it made in its namespace; see Security model. Pass workspace='new' to claim a fresh sandbox even when the message history names one.
warm_pool, namespace and sandbox_timeout only shape a new claim; connection, working_dir and env apply to every command, including after you reattach. A new claim gets working_dir created for it; on an attached claim it must already exist, or commands fail with WorkspaceError.
KubernetesSandbox(...).backend(ref) returns a backend for a stored ref that connects on first use. A run leaves a backend you built open: close it with await backend.aclose(), which closes the connection (and any kubectl port-forward) but not the sandbox.
To seed files before the agent starts, build the backend yourself, write through a Workspace, and pass the backend to the run:
from pydantic_ai import Agent
from pydantic_ai.workspaces import Workspace
from pydantic_ai_harness.coder import Coder
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox, KubernetesSandboxBackend
agent = Agent('anthropic:claude-opus-5-5', capabilities=[KubernetesSandbox(warm_pool='python-pool'), Coder()])
async def run_in_prepared_sandbox() -> str:
backend = KubernetesSandboxBackend(warm_pool='python-pool')
try:
await Workspace(backend).write_text('TASK.md', 'Add a `--version` flag to cli.py.\n')
result = await agent.run('Do the task in TASK.md.', workspace=backend)
return result.output
finally:
await backend.aclose()
ref = backend.ref # set once the sandbox is claimed
if ref is not None:
await KubernetesSandbox(warm_pool='python-pool').destroy(ref)
The run uses the backend as it is, so KubernetesSandbox’s settings don’t apply to it, and leaves it open: close it with aclose(). To keep the sandbox instead, store backend.ref and pass it as workspace= to reattach later.
By default the client reaches each sandbox through kubectl port-forward, which suits running the agent on your machine. An agent running in the cluster connects directly:
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
sandbox = KubernetesSandbox(warm_pool='python-pool', namespace='agents', connection='in-cluster-pod-ip')
'in-cluster-service' connects through the sandbox’s Service instead, which needs service: true on the template. Either way, agent-sandbox’s default NetworkPolicy blocks traffic from other pods, so add a networkPolicy to the template that lets your agent’s pods reach TCP ports 8080 and 9090. Port-forwarding doesn’t work on gVisor or Kata, so use an in-cluster connection there.
For sandboxd on other ports than 8080 and 9090, or a port-forward that needs longer to start, pass the SDK’s connection config:
from k8s_agent_sandbox.models import SandboxdPodTunnelConnectionConfig
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
sandbox = KubernetesSandbox(
warm_pool='python-pool',
connection=SandboxdPodTunnelConnectionConfig(grpc_port=9191, rest_port=8181, port_forward_ready_timeout=60),
)
SandboxdInClusterConnectionConfig takes the same ports, with mode='in-cluster-pod-ip' or mode='in-cluster-service'. For sandboxd started with --auth-token-sha256 or --auth-token-file, pass the token as either config’s auth_token; every command sends it.
Ending a run never deletes the sandbox. A sandbox claimed by KubernetesSandbox is deleted after sandbox_timeout (24 hours by default) whether or not anyone still uses it; delete it sooner with the ref you stored:
from pydantic_ai.workspaces import WorkspaceRef
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
async def delete_sandbox(ref: WorkspaceRef) -> None:
await KubernetesSandbox(warm_pool='python-pool').destroy(ref)
destroy(ref) deletes the claim, and with it the pod; a claim that is already gone returns quietly.
Commands start through /bin/sh and file operations run as shell commands, so the image needs a POSIX shell with coreutils (base64, dd, find).
If a sandbox’s pod is lost while its claim remains, for example when its node fails, agent-sandbox starts a new pod. Commands fail until it is ready, with an error durable execution retries. The new pod starts empty except for what the template keeps on a persistent volume; the template above uses emptyDir, so files are lost.
The agent-sandbox SDK runs on asyncio only, so under Trio workspace operations and destroy raise UserError.
KubernetesSandbox makes one check of its own: it reattaches to and deletes only claims in its namespace named pydantic-ai- and 32 hex digits, the names it gives its claims. A ref from message history, workspace=ref, backend(ref) or destroy(ref) that names any other claim raises ValueError before anything reaches the cluster. The name is not proof of origin, and a KubernetesSandboxBackend(ref=...) you build yourself attaches to any claim it names, so keep refs server-side.
Everything else is set by your cluster, not by this capability: the pod’s isolation (it shares its node’s kernel unless runtimeClassName selects a runtime such as gVisor or Kata), which claims the agent’s kubeconfig or service account can reach, which pods can reach sandboxd’s ports, and whether sandboxd requires an auth token. The SDK talks to sandboxd over plaintext gRPC: connection='port-forward' tunnels it through the API server, and an in-cluster connection does not.
| Option | What it does |
|---|---|
warm_pool | Name of the SandboxWarmPool to claim new sandboxes from. Required. |
namespace | Namespace of the warm pool and the claims. Default: 'default'. |
connection | 'port-forward' (default), 'in-cluster-pod-ip' or 'in-cluster-service'; or the SDK’s SandboxdPodTunnelConnectionConfig or SandboxdInClusterConnectionConfig, for other ports and timeouts. |
sandbox_timeout | Seconds after which a newly claimed sandbox is deleted. Default: 86_400 (24 hours); None keeps it until destroy(ref). |
working_dir | Absolute directory commands start in and relative paths resolve against. Default: None, sandboxd’s root directory. Created on a new claim; on an attached claim it must already exist. |
env | Environment variables every command gets, on top of the container’s own. Default: None. |
authority | The authority of the sandbox’s files: 'none' for files nobody on your side wrote, such as a cloned third-party repository, 'user' for the user’s own. The workspace reports it through authority_of, and a capability that checks it sends a file’s text to the model with no more authority than that. Default: 'operator'. |
path_authority | Levels for some directories and files, overriding authority for the paths under them; the longest matching path wins. Paths are sandbox paths, relative to the working directory or absolute; ~ isn’t expanded. Default: None. |
Outside a run, every command opens its own connection (with the default connection, a kubectl port-forward) and closes it when it ends, so the worker has nothing to close. File operations are shell commands, and one may run a few, so each activity takes a little longer than it does inside a plain run. Run a Temporal dev server on localhost:7233 first.
import asyncio
import uuid
from pydantic_ai import Agent
from pydantic_ai.durable_exec.temporal import PydanticAIPlugin, PydanticAIWorkflow, TemporalDurability
from pydantic_ai_harness.coder import Coder
from pydantic_ai_harness.kubernetes_sandbox import KubernetesSandbox
from temporalio import workflow
from temporalio.client import Client
from temporalio.worker import Worker
agent = Agent(
'anthropic:claude-opus-5-5',
name='kubernetes_coder',
capabilities=[KubernetesSandbox(warm_pool='python-pool'), Coder(), TemporalDurability()],
)
@workflow.defn
class SandboxWorkflow(PydanticAIWorkflow):
__pydantic_ai_agents__ = [agent]
@workflow.run
async def run(self, prompt: str) -> str:
return (await agent.run(prompt)).output
async def main() -> None:
client = await Client.connect('localhost:7233', plugins=[PydanticAIPlugin()])
async with Worker(client, task_queue='sandbox', workflows=[SandboxWorkflow]):
print(
await client.execute_workflow(
SandboxWorkflow.run,
'Use the shell tool to run pwd.',
id=f'sandbox-{uuid.uuid4()}', task_queue='sandbox',
)
)
if __name__ == '__main__':
asyncio.run(main())
Coder, Shell, and FileSystem work under DBOS, Temporal and Prefect. See the Coder, Shell, and FileSystem guides for engine-specific limits.
Removing a capability while workflows using it are still running changes their replay history. Drain those workflows or use Temporal worker versioning before deploying the change.
KubernetesSandbox emits no spans of its own. Core’s instrumentation records the sandbox on the agent run span as pydantic_ai.workspace.provider and pydantic_ai.workspace.id, and each command or file operation a tool makes runs inside that tool call’s span; operations at run start, such as RepoContext loading repo instructions, run in the agent run span. Claiming a sandbox logs its ref at INFO (Claimed Kubernetes sandbox <namespace>/<claim name>) on the pydantic_ai_harness.kubernetes_sandbox._backend logger, so a sandbox that outlives its run can still be found.
See Workspaces for more.
Bases: AbstractCapability[AgentDepsT]
Run the agent’s workspace in a Kubernetes agent sandbox.
A run with no reference claims a sandbox from warm_pool on first use; pass a WorkspaceRef
to attach to an existing claim. Ending a run does not delete the sandbox; destroy(ref) does,
and sandbox_timeout, unless None, deletes it after that many seconds. Commands run in the
pod’s sandboxd container, so its image decides which tools are available.
This capability supplies the workspace only. Compose it with capabilities that use it, such
as Coder, Shell, and FileSystem. See
Workspaces for more.
Name of the SandboxWarmPool new sandboxes are claimed from. Its SandboxTemplate must run sandboxd.
Type: str
Kubernetes namespace of the warm pool and of the claims made from it.
Type: str Default: 'default'
How to reach the sandbox. 'port-forward' (default) runs kubectl port-forward from outside the
cluster and doesn’t work on gVisor or Kata. 'in-cluster-pod-ip' and 'in-cluster-service' connect
directly from a pod in the cluster. For other ports or timeouts, pass the SDK’s
SandboxdPodTunnelConnectionConfig or SandboxdInClusterConnectionConfig.
Type: Connection Default: 'port-forward'
Seconds after which a newly claimed sandbox is deleted, whether or not anyone still uses it;
None keeps it until destroy(ref). Defaults to 24 hours.
Type: int | None Default: DEFAULT_SANDBOX_TIMEOUT
Absolute directory commands start in and relative paths resolve against; None is sandboxd’s
root directory (its --root-dir, /workspace by default). Created on a new claim; an attached claim must
already have it.
Type: str | None Default: None
Environment variables every command gets, on top of the container’s own; a command’s env is layered on top.
Type: Mapping[str, str] | None Default: field(default=None, repr=False)
The authority of the sandbox’s files: 'none' for files nobody on the
application’s side wrote, such as a cloned third-party repository, and 'user' for the user’s own files.
Anything below 'operator' wraps the workspace in an AuthorityWorkspace.
Type: Authority Default: 'operator'
Levels for some directories and files, overriding authority for the paths under them; the longest matching
path wins. Paths are sandbox paths: relative ones resolve against the working directory, and ~ isn’t expanded.
See Workspace.authority_of.
Type: Mapping[str, Authority] | None Default: None
@classmethod
def from_spec(
cls,
*,
id: str | None = None,
description: str | None = None,
warm_pool: str,
namespace: str = 'default',
connection: ConnectionMode = 'port-forward',
sandbox_timeout: int | None = DEFAULT_SANDBOX_TIMEOUT,
working_dir: str | None = None,
env: Mapping[str, str] | None = None,
authority: Authority = 'operator',
path_authority: Mapping[str, Authority] | None = None,
**kwargs: Any,
) -> KubernetesSandbox[Any]
Build from an agent spec, naming the fields a spec can express.
Core reads this signature to generate the spec’s JSON schema. Inherited, core falls back
to the variadic base from_spec, because the connection field’s SDK config types are a
TYPE_CHECKING import, and publishes only the bare 'KubernetesSandbox'. A spec names a
connection by its mode string. **kwargs reaches the constructor unchanged, so
defer_loading keeps its refusal.
KubernetesSandbox[Any]
def backend(ref: WorkspaceRef) -> KubernetesSandboxBackend
Construct a lazy backend for a claim this capability made; close its connection with aclose().
KubernetesSandboxBackend
@async
def destroy(ref: WorkspaceRef) -> None
Delete the claim, and with it the sandbox, without running anything in it.
A claim that no longer exists is already gone, so destroying it returns quietly. Only claims
this capability made in its namespace are accepted.
def get_workspace(
ctx: RunContext[AgentDepsT],
*,
ref: WorkspaceRef | None,
) -> WorkspaceBackend | None
The backend for this run, in an AuthorityWorkspace when authority or path_authority asks for one.
WorkspaceBackend | None
Bases: WorkspaceBackend, SupportsCommands
A sandbox claimed from an agent-sandbox SandboxWarmPool, or attached to by its claim.
Construction does no I/O. Without ref, the first operation claims a sandbox from warm_pool;
with one, it attaches to that claim and raises WorkspaceUnavailableError if it is gone. The
backend never deletes the claim; call KubernetesSandbox.destroy(ref) or rely on sandbox_timeout.
Commands run through the pod’s sandboxd, and file operations run as shell commands, so they see
the same filesystem. The image needs /bin/sh and coreutils.
ref : WorkspaceRef | None Default: None
Identity of an existing claim to attach to on first use, from a previous backend’s ref.
The SandboxWarmPool a new claim is made from; required without ref.
namespace : str Default: 'default'
Namespace of the warm pool and the claims made from it.
How commands reach the pod’s sandboxd: 'port-forward' (a kubectl port-forward),
'in-cluster-pod-ip' or 'in-cluster-service', or an SDK sandboxd connection config.
Seconds after which a new claim deletes itself, or None to keep it until
KubernetesSandbox.destroy(ref).
Absolute directory commands run in, created on a new claim; an attached claim must
already have it. None runs them in sandboxd’s root directory.
Environment variables for every command, under each call’s own env.
WorkspaceRef(provider='kubernetes', id='<namespace>/<claim>'), or None until a claim is made.
Type: WorkspaceRef | None
@async
def get_sandbox() -> AsyncSandbox
Return the SDK’s AsyncSandbox handle, claiming or attaching first like any operation.
The handle’s connection belongs to this backend: after aclose(), call this again.
AsyncSandbox
@async
def aclose() -> None
Close the connection to the sandbox (including a kubectl port-forward); the sandbox keeps running.