diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1808541c..36cdf19d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -44,9 +44,11 @@ jobs: - sdk-versions: minimum openai: openai==2.34.0 anthropic: anthropic==0.83.0 + claude-agent-sdk: claude-agent-sdk==0.2.144 - sdk-versions: latest openai: openai>=2.34.0,<4.0.0 anthropic: anthropic>=0.83.0,<2.0.0 + claude-agent-sdk: claude-agent-sdk>=0.2.144,<0.3 name: Provider SDKs (${{ matrix.sdk-versions }}) steps: - uses: actions/checkout@v4 @@ -64,6 +66,7 @@ jobs: uv pip install --upgrade '${{ matrix.openai }}' '${{ matrix.anthropic }}' + '${{ matrix.claude-agent-sdk }}' - run: uv run --no-sync mypy src tests - run: uv run --no-sync ty check @@ -71,3 +74,27 @@ jobs: uv run --no-sync pytest tests/providers/openai tests/providers/anthropic + tests/harnesses + + no-extras: + # `pip install ai` with no extras: the harness and sandbox pieces must + # import, and fail with InstallationError when built, not on first use. + runs-on: ubuntu-latest + name: Without optional extras + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - uses: astral-sh/setup-uv@v5 + + - run: uv sync --no-default-groups + + - run: uv pip install pytest + + - run: >- + uv run --no-sync pytest + tests/harnesses/experimental/test_extras.py + tests/workspaces/experimental/test_extras.py diff --git a/.gitignore b/.gitignore index f766fe51..f9198ecf 100644 --- a/.gitignore +++ b/.gitignore @@ -216,3 +216,4 @@ __marimo__/ .claude/ .codex +.env* diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7449113d..8762eec9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,3 +1,115 @@ +## Harnesses and workspaces + +`ai.harnesses.experimental` drives coding-agent CLIs. Adapters live in +`src/ai/harnesses/experimental/_adapters/` (`claude.py` on +`claude-agent-sdk`, `codex.py` on `codex app-server`). Workspaces live in +`src/ai/workspaces/experimental/` (`_local.py`, `_sandbox.py`). + +Rules that apply to both: + +- **Layering.** Caller, then `Harness`, then adapter, then workspace. Each + layer knows only the one below. A workspace never imports harness names + and never learns a CLI's dialect. A new CLI touches one adapter; a new + place to run touches one workspace. +- **Capability honesty.** Every `Capabilities` field is a claim you watched + be true. When a capability is absent, raise `UnsupportedError`. Never a + silent no-op. +- **Translate errors at the boundary.** Callers never catch a CLI library's + or provider's exceptions. Map them to `ai.harnesses.experimental.errors` + or `ai.workspaces.experimental.errors`. + +### Adding a harness + +1. Pick the CLI protocol first. It must offer, in one mode: a long-lived + process or resume by id, input while a turn runs (steer), an interrupt, + and a way to ask the client about tool calls. Codex had to move from + `codex exec --json` to `app-server` for this. +2. Implement the `Adapter` protocol in `_adapters/base.py`. `new_session()` + returns the CLI's own id, never one you mint. +3. Never spawn a process yourself. Call `workspace.spawn(argv)`. If the CLI + library insists on spawning, give it a transport backed by the workspace + (see `_adapters/remote_cli.py`). +4. Yield `ai.types.events` only, ending with `StreamEnd`. Text, reasoning, + and tool calls need Start / Delta / End triples. Emit tool arguments as a + `ToolDelta`, even when the call arrives complete. Tool results are + `ToolCallResult` events. +5. `approve is None` means approve everything. Answer the CLI's approval + requests yourself; never fall back to asking a terminal that is not there. + Exceptions inside a permission callback do not reach the caller: park them + on the adapter and re-raise from `turn()`. +6. Hold the workspace session lock (`_session_lock.SessionLock`) for every + conversation you open, so a second writer gets `SessionBusyError` on every + CLI. +7. Store `history=` natively with the rules in `_handoff.py`: text verbatim, + tool calls with their original names, reasoning as a note or dropped with + a warning. Never drop context silently. +8. Spell the gateway in the adapter (`gateway_env`, `gateway_config`). When + `workspace.injects_credentials_for(gateway.host)`, hand the CLI + `gateway.brokered()`. +9. Add a factory in `_factories.py` (and to `BUILDERS`), then add the kind to + `SPECS` and `HARNESS_CLI` in `tests/harnesses/experimental/conftest.py`. + That runs the whole e2e suite against the new adapter. + +### Adding a workspace + +1. Subclass `Workspace` from `_base.py`. Set `kind`, `owner` (`"provider"` + only for a box the SDK may install software into), and `duplex_spawn`. + `env` belongs to the work, not the agent. +2. Do not override `contains()`. It is pure-path on purpose: a remote path + does not exist on this machine. +3. Move files through `read_text`, `write_text`, `_copy_in`, and `_copy_out`. + Filter both directions with `walk_uploadable` and `copy_tree`. Batch + uploads; round trips are the cost on a remote box. +4. Map failures: a program that cannot run is `exit_code=127` from `exec` + and `WorkspaceError` from `spawn`; a missing file is `FileNotFoundError`; + a workspace that no longer exists is `WorkspaceGoneError`. +5. `Process.terminate()` returns only when the program is gone, and raises + when it could not make that true. +6. If the workspace can outlive the client, support reconnect by `name=`, + carry that identity in `coords`, and add a `keep` flag. Live state is + found by asking a harness inside the workspace, never by the workspace. +7. For ptys, reuse `_pty_holder.py` and `_pty.FramedPty` over your byte + channel. Never name a module `pty.py`. +8. Export it from `src/ai/workspaces/experimental/__init__.py`, and add it to + the `any_workspace` fixtures in `tests/workspaces/experimental/conftest.py` + and `tests/harnesses/experimental/conftest.py` as + `pytest.param("yours", marks=pytest.mark.yours)`. It must present the same + `project` the local branch does. + +### Tests + +Live tests drive real CLIs and real sandboxes. No fakes, no recorded doubles. +A missing CLI skips; it never passes silently. Markers: + +- `live`: drives a real local `claude` or `codex`. +- `sandbox`: needs Vercel Sandbox credentials. +- `slow`: minutes, not seconds. + +```bash +uv run pytest # offline (default) +uv run pytest tests/harnesses tests/workspaces -m "not sandbox" # plus local CLIs +uv run pytest tests/harnesses tests/workspaces -m "" # everything +``` + +Plain `uv run pytest` deselects `live` and `sandbox` (see `addopts` in +`pyproject.toml`); passing `-m` replaces that filter. + +The live suite needs `claude` and `codex` CLIs that can reach a model: a +login, or the gateway variables in `.env.local`. The sandbox suite +also needs Vercel credentials and `AI_GATEWAY_API_KEY` in a `.env.local` at +the repo root (`vercel link`, then `vercel env pull`). Every test is capped at +300 seconds. + +When writing tests, never branch on native tool names, wait on events instead +of sleeping, assert exact answers, and put anything on the control surface +on `any_harness` so it runs on both harnesses and both workspaces. + +Run the harness examples with: + +```bash +uv run examples/.test_scripts/run-examples.py --harnesses +``` + ## Releasing Every PR needs to be labeled as `breaking`, `feature`, `fix`, or diff --git a/README.md b/README.md index f2346d64..c5a8ba67 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ official SDK lazily. Vercel OIDC for AI Gateway also uses an optional extra: uv add "ai[openai]" # OpenAI-compatible providers uv add "ai[anthropic]" # Anthropic-compatible providers uv add "ai[vercel]" # Vercel OIDC for AI Gateway +uv add "ai[claude-code]" # Claude Code harness (experimental) +uv add "ai[sandbox]" # Vercel Sandbox workspaces (experimental) ``` ```python @@ -156,6 +158,24 @@ approval = await ai.hook( ai.resolve_hook("approve_send_email", {"granted": True, "reason": "approved"}) ``` +## Harnesses (experimental) + +Harnesses drive coding-agent CLIs you already have, Claude Code and Codex, +on this machine or in a Vercel Sandbox. A turn streams the same `ai.events` +as an agent: + +```python +from ai.harnesses import experimental as harnesses +from ai.workspaces import experimental as workspaces + +async with harnesses.claude_code(workspace=workspaces.Local(".")) as harness: + result = await harness.run("What does this repo do?") +``` + +This API is experimental and may change or be removed. See +[Harnesses](https://ai-python.dev/docs/basics/harnesses) and +[Workspaces](https://ai-python.dev/docs/basics/workspaces). + ## Examples Focused samples live in category directories under `examples/`. @@ -164,6 +184,8 @@ Focused samples live in category directories under `examples/`. - `examples/media/` - image, video, speech, transcription, embeddings, reranking, and multimodal input/output - `examples/models/` - streaming, structured output, and provider examples +- `examples/harnesses/` - driving Claude Code and Codex, locally and in a + sandbox (experimental) - `examples/apps/` - end-to-end demos End-to-end demos: @@ -173,3 +195,7 @@ End-to-end demos: - `examples/apps/durable_agent_temporal/` - durable agent with Temporal - `examples/apps/durable_agent_workflows/` - durable agent with Workflows - `examples/apps/slack_agent/` - Slack agent +- `examples/apps/afk/` - move a coding-agent conversation into a sandbox and + back (experimental harnesses) +- `examples/apps/deepysec/` - security-scanning pipeline on harnesses + (experimental) diff --git a/docs/ai-python/content/docs/basics/harnesses.mdx b/docs/ai-python/content/docs/basics/harnesses.mdx new file mode 100644 index 00000000..e5f673cc --- /dev/null +++ b/docs/ai-python/content/docs/basics/harnesses.mdx @@ -0,0 +1,693 @@ +--- +title: Harnesses +description: Drive Claude Code and Codex from Python. +type: guide +summary: Run turns on coding-agent CLIs, stream events, steer, stop, approve tool calls, resume conversations, and open the CLI's own TUI. +--- + +A harness drives a coding-agent CLI you already have, `claude` or `codex`. +Your program sends prompts. The CLI does the agent work with its own tools, +login, and session store. The same code runs on either CLI, on this machine or +in a [workspace](/docs/basics/workspaces) such as a Vercel Sandbox. + +This API is experimental and may change or be removed. + +Each harness needs its CLI installed. See [Harnesses](#harnesses). + +## Run a prompt + +```python title="harness_run.py" +import asyncio + +from ai.harnesses import experimental as harnesses +from ai.workspaces import experimental as workspaces + + +async def main() -> None: + async with harnesses.claude_code(workspace=workspaces.Local(".")) as harness: + result = await harness.run("What does this repo do? One sentence.") + + print(result.text) + print(result.finish_reason, result.usage.output_tokens) + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +`claude_code()` and `codex()` return a `Harness` and share these arguments, +so swapping one for the other is a one-word edit: + +```python +harness = harnesses.codex(workspace=workspaces.Local("."), effort="high") +``` + +| Argument | Description | Default | +|---|---|---| +| `workspace` | Where the CLI runs. | required | +| `approve` | Approval hook. See [Approve tool calls](#approve-tool-calls). | `None`: every call runs | +| `approval_timeout` | Seconds a hook may take before the call is denied. | `300.0` | +| `model` | Model name passed to the CLI. | `None`: the CLI's default | +| `effort` | `"low"`, `"medium"`, `"high"`, or `"max"`. | `None`: the CLI's default | +| `gateway` | Upstream model provider LLM Gateway. See [Workspaces](/docs/basics/workspaces#reach-a-model-through-ai-gateway). | `None`: the workspace's gateway | +| `executable` | CLI name or path. | `"claude"` or `"codex"` | +| `instructions` | Text added to the CLI's own system prompt, for every turn. | `None` | +| `mcp_servers` | MCP servers to connect: `{"name": {"command": ..., "args": [...], "env": {...}}}`, or `{"name": {"url": ...}}` for a remote one. | `None` | + +Each harness adds its own arguments; see [Harnesses](#harnesses). + +Every harness runs with full access by default: it may run any command and +edit any file the workspace lets it. Nobody is at a terminal to answer a +permission prompt, so there is no narrower default to fall back on. Restrict +it with an [approval hook](#approve-tool-calls), with each harness's own +options below, or by running it in a +[Vercel Sandbox](/docs/basics/workspaces#vercel-sandbox). + +`run()` returns a `Result`: + +| Field | Description | +|---|---| +| `text` | The agent's final message, after its last tool call. | +| `finish_reason` | `"stop"`, `"cancelled"`, or `"length"` (Claude Code hit its turn limit). | +| `messages` | The conversation so far, as `ai.messages.Message` values, in order: each tool call, then its result. | +| `usage` | Token usage. | +| `output` | The validated value when you pass `output_type`; otherwise the same as `text`. | +| `prompt_sent` | Exactly what the agent received. | +| `approval_errors` | Approval hooks that raised or timed out. | + +## Harnesses + +A missing extra raises `ai.errors.InstallationError` when you call the +factory. A missing CLI raises `ExecutableMissingError` when the harness +opens. + +Each harness was verified against one CLI version: `claude` 2.1.289 and +`codex` 0.160.1. An older CLI warns when the harness opens. A harness error +carries the CLI version in its notes, so a failure on a newer CLI says so. + +On your own machine the SDK uses whatever CLI you installed and never +installs or updates one. In a [Vercel Sandbox](/docs/basics/workspaces#vercel-sandbox) +it installs the latest release from npm when the harness opens, and +upgrades an older one, so a sandbox always runs the newest CLI. It does +not upgrade while a copy of the CLI is running there, such as an agent you +detached from and came back to. + +### Claude Code + +`harnesses.claude_code()` drives the `claude` CLI through +`claude-agent-sdk`. It needs the `claude-code` extra: + +```bash title="Terminal" +uv add "ai[claude-code]" +``` + +It takes the [shared arguments](#run-a-prompt), plus: + +| Argument | Description | Default | +|---|---|---| +| `tools` | The built-in tools the agent has, such as `["Read", "Grep"]`. `[]` for none. | `None`: all of them | +| `disallowed_tools` | Tools taken away, such as `["Bash"]`. | `None` | +| `max_turns` | Most agent steps in one turn. A turn that hits it settles with `finish_reason="length"`. | `None`: no cap | + +- Tool names are Claude Code's own, such as `Bash`, `Write`, and `Edit`. +- `result.usage.raw["cost_usd"]` reports the turn's cost. + +### Codex + +`harnesses.codex()` drives the `codex` CLI over `codex app-server`. It needs +no extra, only the `codex` CLI. + +It takes the [shared arguments](#run-a-prompt), plus: + +| Argument | Description | Default | +|---|---|---| +| `sandbox` | Codex's own sandbox mode: `"read-only"`, `"workspace-write"`, or `"danger-full-access"`. | `"danger-full-access"` | +| `config` | Codex `-c` config overrides. Wins over values derived from `gateway`, `effort`, and `mcp_servers`, key by key. | `None` | + +- On `Local`, pass `sandbox="read-only"` or `"workspace-write"` to restrict + Codex with its own sandbox. In `"read-only"` mode it declines writes. +- `instructions` become Codex's developer instructions. +- `codex()` also accepts Claude Code's `tools`, `disallowed_tools`, and + `max_turns`, so swapping the two factories stays a one-word edit, but + raises `UnsupportedError` when one is set: Codex has no equivalent. +- In a Vercel Sandbox, leave `sandbox` unset. Codex's own sandbox cannot + start inside the VM, so any other mode raises `UnsupportedError` when the + harness opens. +- Tool names are Codex item kinds, such as `commandExecution` and + `fileChange`. The shell command is in the call's arguments, never the name. +- `effort="max"` is sent to Codex as `"xhigh"`. + +## Keep a conversation + +`harness.run()` uses a throwaway conversation. Use a session for several turns +that share context: + +```python +session = harness.session() +await session.run("Read utils.py.") +result = await session.run("Which function is the slowest?") +``` + +The CLI creates the conversation on the first turn, so `session.session_id` +is empty until then. Read `session.handle` after a turn, not before. + +`harness.run()` also takes a whole conversation, the way `Agent.run()` does. +The last message must be a user message; it becomes the prompt, and the +messages before it become the conversation's history: + +```python +result = await harness.run([*earlier_messages, ai.user_message("Now add tests.")]) +``` + +A session turn takes one prompt: a string, a list of strings and parts, or +one `ai.user_message(...)`. + +## Send files + +A prompt is a string, or a list of strings and parts. Add images, PDFs, or +text files as `FilePart`s: + +```python +import ai + +result = await session.run( + ["What is wrong in this screenshot?", ai.file_part(png_bytes)] +) +``` + +| File | Claude Code | Codex | +|---|---|---| +| Images (`image/*`) | yes | yes | +| PDFs (`application/pdf`) | yes | no | +| Text files (`text/*`) | yes, as text | yes, as text | + +- A file the harness cannot take raises `UnsupportedError` when you call + `run()` or `stream()`, before anything runs. Check + `harness.capabilities.images` and `harness.capabilities.documents`. +- A file given as a URL is downloaded on your machine and sent as data, so + it works in a sandbox whose outbound traffic is locked down. +- The file stays in the conversation: `result.messages` and `history()` + hold it as a `FilePart`. + +## Stream events + +`session.stream()` returns a `Turn`. Use it like `agent.run()`: iterate it +for events, then read the result: + +```python +import ai + +async with session.stream("Refactor utils.py into a package.") as turn: + async for event in turn: + if isinstance(event, ai.events.TextDelta): + print(event.chunk, end="", flush=True) + elif isinstance(event, ai.events.ToolEnd): + print(event.tool_call.tool_name, event.tool_call.tool_args) + elif isinstance(event, ai.events.ToolCallResult): + print(event.results[0].result) + +print(turn.output) +``` + +Leaving the `async with` block settles the turn, and stops the agent if it +is still working. `async with` is optional: a bare `async for` works too. + +| Attribute | Description | +|---|---| +| `messages` | The conversation so far. Updated while the turn runs. | +| `blocked` | `True` while the agent waits on your approval hook. | +| `deferred_hooks` | The approvals it is waiting on. | +| `result` | The `Result`, once the turn has settled. | +| `output`, `usage` | Shortcuts for `result.output` and `result.usage`. | + +A turn yields `ai.events.AgentEvent` values: model events plus +`ToolCallResult`. Every event carries `.message`, the assistant message so +far. Consumers of `ai.Agent` streams, such as +[AI SDK UI](/docs/basics/ai-sdk-ui), work here too. The events are coarser, +though; see [Harnesses and agents](#harnesses-and-agents). + +## Steer and stop + +Control verbs live on the session, because only one turn runs at a time: + +```python +turn = session.stream("Refactor utils.py into a package.") +steered = False +async for event in turn: + if isinstance(event, ai.events.ToolStart) and not steered: + await session.steer("Keep it one module; just split the functions.") + steered = True +``` + +- `steer(text)` sends guidance to the running turn. The agent acts on it + before the turn ends. The steer is added to the transcript as a user + message. +- `stop()` ends the turn. It settles as `finish_reason == "cancelled"`, keeps + what was streamed, and the session stays usable. Call it from the loop + body or from another task, such as a Stop button. It returns once the + turn has settled, so `turn.result` is ready right after. +- `run(..., timeout=seconds)` stops the turn at the deadline and returns a + cancelled `Result`. It does not raise, except with `output_type` (see + [Structured output](#structured-output)). + +A bare `break` ends the turn too: the agent is stopped, and the turn +settles as cancelled with what was streamed. The next `run()` or `stream()` +starts cleanly. Only one turn runs at a time; a second `run()` from another +task waits for the first. + +## Approve tool calls + +Pass `approve=` to decide which tool calls run: + +```python +SHELL = {"Bash", "commandExecution"} # Claude Code's and Codex's shell tools + + +async def approve(ctx: harnesses.ApprovalContext) -> harnesses.Decision: + if ctx.call.name in SHELL: + command = str(ctx.call.input.get("command", "")) + if "git push" in command or "rm -rf" in command: + return harnesses.Deny("not that command") + return harnesses.Allow() + target = ctx.call.input.get("file_path") + if target: + real = await ctx.workspace.resolve(target) # follows symlinks + if not ctx.workspace.contains(real): + return harnesses.Deny("stay inside the workspace") + return harnesses.Allow() + + +async with harnesses.claude_code( + workspace=workspaces.Local("."), approve=approve +) as harness: + ... +``` + +Without a hook, every tool call is allowed. Nobody is at a terminal to +answer, so a hook is how you restrict the agent. + +The hook receives an `ApprovalContext`: + +| Field | Description | +|---|---| +| `call` | The `ToolCall`: the CLI's own tool name (`call.name`) and arguments (`call.input`), untranslated. See [Harnesses](#harnesses) for each CLI's names. | +| `history` | The conversation so far. | +| `workspace` | The workspace the CLI runs in. | +| `harness` | `"claude-code"` or `"codex"`. | +| `session_id` | The conversation's id. | +| `prompt` | The turn's prompt on Claude Code; `None` on Codex. | + +It returns a decision: + +| Decision | Effect | +|---|---| +| `Allow()` | The call runs. | +| `Allow(input={...})` | The call runs with rewritten arguments. Claude Code only. | +| `Deny(reason)` | The call is blocked. On Claude Code the agent reads the reason; on Codex it only learns the call was refused. | +| `Deny(reason, stop=True)` | The call is blocked and the turn ends. | + +- A decision the harness cannot honor, such as `Allow(input=...)` on Codex + or `Allow(remember=True)` on either, denies the call and raises + `UnsupportedError` from the turn. +- A hook that raises or runs past `approval_timeout` denies the call. The + failure is listed in `result.approval_errors`. +- On Claude Code every tool call reaches the hook, reads included. On + Codex the CLI's policy decides which calls to ask about; shell commands + and file changes do. See `approval` in [Capabilities](#capabilities). +- A shell command is a string the agent wrote. Matching on it, as above, is + a speed bump, not isolation. For isolation, run the harness in a + [Vercel Sandbox](/docs/basics/workspaces#vercel-sandbox). +- `workspace.contains(path)` checks the spelling of a path only. Use + `contains(await workspace.resolve(path))` so a symlink cannot lead out. +- The hook runs in the context of the turn it decides for, so contextvars + that the turn's caller set are visible inside it. + +While the hook decides, the turn's stream reports it: a `HookEvent` with +status `"pending"`, then `RunBlocked`. When the hook returns, a `HookEvent` +follows with status `"resolved"` and the decision as its `resolution`, or +`"cancelled"` if the hook failed. `turn.blocked` follows these events. Their +`hook_type` is `harnesses.APPROVAL_HOOK_TYPE`. Your hook answers them, not +`ai.resolve_hook()`, so AI SDK UI does not show approval buttons for them. + +## Capabilities + +Each harness declares what it can do in `harness.capabilities`, before +anything runs. Using a capability a harness lacks raises `UnsupportedError`; +the one exception is `deny_reason`, whose absence only drops the reason. + +| Capability | Description | Claude Code | Codex | +|---|---|---|---| +| `run` | Run a prompt and wait for the result: `run()`. | yes | yes | +| `stream` | Iterate a turn's events as they arrive: `stream()`. | yes | yes | +| `steer` | Send guidance to the turn that is running: `session.steer()`. | yes | yes | +| `stop` | Interrupt the running turn; it settles as cancelled: `session.stop()`. | yes | yes | +| `resume` | Continue a conversation this process did not create: `resume()`, `harness_from()`. | yes | yes | +| `fork` | Branch a conversation into a new one with the same past: `fork()`. | yes | yes | +| `history` | Read stored transcripts in full, tool calls included: `history()`, `sessions()`. | yes | yes | +| `images` | Images in a prompt. See [Send files](#send-files). | yes | yes | +| `documents` | PDFs in a prompt. | yes | no | +| `rewrite_tool_input` | Rewrite a call's arguments before it runs: `Allow(input={...})`. | yes | no | +| `deny_reason` | The reason in `Deny(reason)` reaches the agent. Without it the call is still denied, but the reason is dropped. | yes | no | +| `remember_decisions` | Keep an approval for the rest of the session: `Allow(remember=True)`. | no | no | +| `approval` | Which tool calls reach the hook. `"all"`: every call, reads included. `"policy"`: the CLI's own policy decides, so not every call may. | `"all"` | `"policy"` | + +## Resume and fork + +A `Handle` names a conversation as plain data. Save it and continue later, +from any process: + +```python +saved = session.handle.model_dump_json() + +handle = harnesses.Handle.model_validate_json(saved) +async with harnesses.harness_from(handle, approve=approve) as session: + await session.run("Continue.") +``` + +A handle holds the harness kind, the CLI's own session id, the workspace, and +the options. It never holds a hook, a key, or MCP servers (their `env` often +holds tokens), so pass `approve=`, `gateway=`, and `mcp_servers=` again. The SDK keeps no registry of conversations. Store handles in your own +database. + +A harness can also read its CLI's store, including conversations a person +started in a terminal: + +```python +infos = await harness.sessions() +messages = await harness.history(infos[0].session_id) +branch = await harness.fork(infos[0].session_id) +same = await harness.resume(infos[0].session_id) +``` + +- `resume()` and `fork()` return a `Session` whose `messages` already hold + the past. Run or stream on it exactly as on a new session; the new turns + follow that past. +- `history(session_id, limit=, offset=)` pages through a stored + conversation. `limit` and `offset` count the messages it returns. +- `fork()` starts a new conversation with the same past. The original is + never written. +- `resume()` continues the original. Only one client may write a + conversation. If another SDK client or a `harness.tui()` holds it, + `resume()` raises `SessionBusyError`, and `SessionInfo.running` is `True`. +- A `claude` or `codex` that a person started in a terminal holds no claim, + so `resume()` does not refuse it. `sessions()` still shows it as + `running`, with its `pid`: any `claude`, and a `codex` started as + `codex resume `. A fresh `codex` names no conversation, so it is not + seen. Fork conversations you did not start. + +`harness_from(handle, fork=True)` forks instead of resuming. + +## Hand off a conversation + +Pass `history=` to start a new conversation from an earlier one. It works +across harnesses and machines: + +```python +history = await claude_agent.history(session_id) # or result.messages +result = await codex_agent.run("Carry on.", history=history) +session = sandbox_agent.session(history=history) +``` + +The receiving CLI stores the history natively before the first turn. Tool +calls keep their original names. Two limits: + +- Reasoning cannot go into Claude Code. It is dropped with a warning. +- Paths point to the source machine. Copy the files and tell the agent where + they are now. + +## Open the CLI's TUI + +`harness.tui()` opens the CLI's own interactive UI on a pseudo-terminal in the +workspace. Attach your terminal with `tty.bridge`: + +```python +from ai.workspaces.experimental import tty + +tui = await harness.tui(session_id, name="work") +await tty.bridge(tui.pty) # Ctrl-] detaches + +await tty.bridge(await workspace.attach("work")) # later, from any process +``` + +- A `Tui` has `session_id`, `handle`, `pty`, `detach()`, and `close()`. It has + no `run`, `steer`, or `stop`, because a person drives it. +- A `name` keeps the TUI running after you detach. Without one it ends with + your connection. +- `tui(history=...)` starts a new TUI conversation with a past. +- A bare `tui()` on Codex has `session_id` and `handle` of `None`, because + Codex picks its own id. Find the conversation later with `sessions()`. +- `tty` is POSIX only and needs a real terminal on stdin. Import it + explicitly. + +## Structured output + +Pass `output_type` to `run()`: + +```python +import pydantic + + +class Finding(pydantic.BaseModel): + file: str + line: int + title: str + + +result = await session.run(prompt, output_type=list[Finding], retries=2) +findings: list[Finding] = result.output +``` + +The SDK adds the JSON schema to the prompt, validates the reply, and asks +again with the error up to `retries` times. Then it raises pydantic's +`ValidationError`. With `timeout=`, a cancelled reply that does not validate +raises too. `result.prompt_sent` shows the edited prompt. + +## Errors + +Harness errors live in `ai.harnesses.experimental.errors`. They subclass +`HarnessError`, which subclasses `ai.errors.AIError`. `NotAuthenticatedError` +is shared with workspaces and subclasses `WorkspaceError`. + +| Error | Meaning | +|---|---| +| `UnsupportedError` | This harness lacks the capability. | +| `HarnessClosedError` | The harness is not open. | +| `ExecutableMissingError` | The CLI is not runnable in the workspace. | +| `SessionBusyError` | Another client writes this conversation. Fork it. | +| `ResumeFailedError` | The conversation could not be reopened. | +| `AgentCrashedError` | The CLI process died. | +| `TurnFailedError` | The CLI reported the turn as failed. | +| `NotAuthenticatedError` | No credentials where the CLI runs. | + +A refusal or a cancelled turn is a normal `Result`, not an error. + +## Harnesses and agents + +A harness is an agent, not a model. The CLI runs its own loop and tools, so a +harness turn is like a whole `agent.run()`, not one model call. You get fewer +controls and coarser events than with `ai.Agent`. You also get some controls +that `ai.Agent` does not have. + +| You want to | `ai.Agent` | Harness | +|---|---|---| +| Change the loop | Override `loop()`. | No. | +| Run or gate each tool | Your tools, `require_approval`. | `approve=`, for the calls the CLI asks about. | +| Change the model or params per step | Yes. | No. | +| Approve while nobody is connected | Defer the hook and run again later. | No: the CLI waits for the answer. | +| Stop a run | Leave the `async with` block, or cancel. | `stop()` from any task, `break`, `timeout=`. | +| Add guidance mid-run | No. | `steer()`. | +| Continue later | Pass the messages again. | `resume()`, `fork()`, a `Handle`, or `history=`. | +| Let a person drive it | No. | `tui()`. | + +What the event stream holds: + +| Event detail | `ai.Agent` | Claude Code | Codex | +|---|---|---|---| +| Text deltas | yes | yes | yes | +| Tool calls and their results | yes | yes | yes | +| Tool arguments while they stream | yes | no: sent whole | no: sent whole | +| One `StreamStart`/`StreamEnd` per model call | yes | no: one per turn | no: one per turn | +| Usage per model call | yes | no: per turn, with cost | no: per turn | + +Code that reads events should not need more than this: text, tool calls and +results, the final `StreamEnd`, and `messages`. The SDK does not invent +detail the CLI did not report. + +### Use a harness as a tool + +A harness fits where a sub-agent does. Give an `ai.Agent` a tool that yields +the turn's events: + +```python +session = harness.session() + + +@ai.tool +async def code(task: str) -> ai.SubAgentTool: + """Do a coding task in the project; returns what was done.""" + async with session.stream(task) as turn: + async for event in turn: + yield event + + +async with ai.Agent(tools=[code]).run(model, messages) as stream: + async for event in stream: + ... +``` + +- The harness's events reach the agent's stream as `PartialToolCallResult` + values. +- The agent's model sees the harness's final answer as the tool result. +- Closing the agent's run stops the harness's turn. +- The tool calls inside the harness were run by the CLI, not by the agent. + Do not run them again from a custom loop. + +### Combine and customize + +A turn is an async iterable of `ai` events, so the tools you use with agent +streams work on it. + +**Run turns side by side.** A session runs one turn at a time, so use one +session per task. `ai.util.merge` interleaves their events. Tag each stream +if you need to know where an event came from: + +```python +async def tagged(label, turn): + async with turn: + async for event in turn: + yield label, event + + +poem = harness.session().stream("Write a poem into poem.txt.") +limerick = harness.session().stream("Write a limerick into limerick.txt.") +async for label, event in ai.util.merge( + tagged("poem", poem), tagged("limerick", limerick) +): + ... +``` + +Inside an agent loop, `ai.yield_from` does the same with labels: each event +reaches the consumer as a `PartialToolCallResult` whose `label` names its +source. + +```python +class FanOut(ai.Agent): + async def loop(self, context): + left, right = harness.session(), harness.session() + async with left.stream("Fix the parser.") as a, right.stream("Write the docs.") as b: + fixed, documented = await asyncio.gather( + ai.yield_from(a, aggregator=ai.agents.MessageAggregator, label="parser"), + ai.yield_from(b, aggregator=ai.agents.MessageAggregator, label="docs"), + ) + context.add(ai.assistant_message(f"{fixed}\n\n{documented}")) + return + yield +``` + +**Make a turn one step of your own agent loop.** Override `Agent.loop()` +and yield the turn's events. You decide which events to pass on, what joins +the agent's context, and what runs next: + +```python +class CodeThenReview(ai.Agent): + async def loop(self, context): + async with harness.session().stream(context.messages[-1].text) as turn: + async for event in turn: + if not isinstance(event, ai.events.ReasoningDelta): + yield event # pass on, drop, or rewrite + context.add(ai.assistant_message(turn.output)) + context.add(ai.user_message("Review that. Start with YES or NO.")) + async with ai.stream(context=context) as review: # then a model call + async for event in review: + yield event + context.add(review.message) +``` + +Add the harness's answer to the context, not its whole transcript: its tool +calls name tools the agent's model does not have. + +**Loop around turns in plain Python.** Run a turn, check the result, and +prompt again. An async generator can pass the turn's events on and add +events of its own: + +```python +async def until_passes(harness, task): + session = harness.session() + prompt = task + for attempt in range(3): + async with session.stream(prompt) as turn: + async for event in turn: + yield event + run = await harness.workspace.exec(["python3", "-m", "pytest", "-q"]) + yield Checked(attempt, run.exit_code == 0) # an event type of your own + if run.exit_code == 0: + return + prompt = f"The tests fail:\n{run.stdout}\nFix them." +``` + +What you cannot change: the harness runs its own loop, so its events report +what already happened. Rewriting an event changes what your consumers see, +not what the agent did. To change what the agent does, use the controls the +CLI offers: + +| When | How | +|---|---| +| Before a tool call runs | `approve=`: allow, deny, deny and stop, or `Allow(input=...)` on Claude Code. | +| While a turn runs | `session.steer(text)`; `session.stop()`, `break`, or leaving the `async with` block. | +| Between turns | Your own loop: the next prompt, the history, or another harness. | + +## How it works + +- **Layering.** Caller, then `Harness`, then one adapter per CLI, then the + workspace. Adapters never start processes; they ask the workspace. That is + why one adapter runs locally and in a microVM. A workspace knows nothing + about harnesses. +- **A borrowed conversation model.** Messages, parts, events, and usage are + the `ai` types. `result.messages` works anywhere `ai` messages do, such as + [AI SDK UI](/docs/basics/ai-sdk-ui). +- **Structured output is a layer.** It only calls `run()`. It is the one + place the SDK edits your prompt, which is why `prompt_sent` exists. + +## Examples + +One runnable file per feature: + +- [`examples/harnesses/basic.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/basic.py): + one prompt. +- [`examples/harnesses/streaming.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/streaming.py): + events as they arrive. +- [`examples/harnesses/as_tool.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/as_tool.py): + a harness as a tool of an `ai.Agent`. +- [`examples/harnesses/combine.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/combine.py): + two turns at once, merged into one stream. +- [`examples/harnesses/custom_loop.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/custom_loop.py): + a harness turn as one step of your own `Agent.loop()`. +- [`examples/harnesses/until_done.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/until_done.py): + your own loop around turns: prompt, check, prompt again. +- [`examples/harnesses/steering.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/steering.py): + redirect a running turn. +- [`examples/harnesses/tool_approval.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/tool_approval.py): + an approval hook. +- [`examples/harnesses/take_over.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/take_over.py): + fork a conversation you did not start. +- [`examples/harnesses/resume.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/resume.py): + save a handle, continue in another process. +- [`examples/harnesses/handoff.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/handoff.py): + move a conversation between harnesses. +- [`examples/harnesses/sandbox.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/sandbox.py): + the same code in a Vercel Sandbox. +- [`examples/harnesses/copy_files.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/copy_files.py): + copy a project in and results out. +- [`examples/harnesses/reconnect.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/reconnect.py): + detach, then come back to a sandbox. +- [`examples/harnesses/tui.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/tui.py): + the CLI's own TUI, attached to your terminal. + +Apps: + +- [`examples/apps/afk`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/afk): + a CLI that pushes local conversations into a sandbox TUI and brings them + back. +- [`examples/apps/deepysec`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/deepysec): + a security-scanning pipeline with fan-out across sandboxes and structured + output. diff --git a/docs/ai-python/content/docs/basics/index.mdx b/docs/ai-python/content/docs/basics/index.mdx index 290c00a8..3d5a2b92 100644 --- a/docs/ai-python/content/docs/basics/index.mdx +++ b/docs/ai-python/content/docs/basics/index.mdx @@ -22,6 +22,8 @@ Start with the model call, then add the pieces your app needs: - Pause tools for approval with hooks. - Run subagents as tools and stream their output. - Bridge agent streams to AI SDK UI clients over Server-Sent Events (SSE). +- Drive Claude Code and Codex from Python, locally or in a Vercel Sandbox + (experimental). ## Main primitives @@ -32,6 +34,8 @@ Start with the model call, then add the pieces your app needs: - `@ai.tool` exposes Python functions to models. - `Agent` runs the stream -> tool -> stream loop. - `ai.hook` pauses a workflow for external input. +- `ai.harnesses.experimental` drives coding-agent CLIs, and + `ai.workspaces.experimental` says where they run. ## Example map @@ -47,10 +51,14 @@ Focused samples live in category directories under `examples/`: - `examples/agents/custom_loop.py` overrides `Agent.loop`. - `examples/agents/streaming_tool.py` streams partial output from a tool. - `examples/agents/subagent.py` runs a subagent as a tool. +- [`examples/harnesses/basic.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/basic.py) + runs one Claude Code turn. End-to-end demos live in [`examples/apps/web_agent`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/web_agent), [`examples/apps/coding_agent`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/coding_agent), [`examples/apps/durable_agent_temporal`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/durable_agent_temporal), [`examples/apps/durable_agent_workflows`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/durable_agent_workflows), -and [`examples/apps/slack_agent`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/slack_agent). +[`examples/apps/slack_agent`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/slack_agent), +[`examples/apps/afk`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/afk), +and [`examples/apps/deepysec`](https://github.com/vercel-labs/ai-python/tree/main/examples/apps/deepysec). diff --git a/docs/ai-python/content/docs/basics/meta.json b/docs/ai-python/content/docs/basics/meta.json index e5c30479..07fec1b2 100644 --- a/docs/ai-python/content/docs/basics/meta.json +++ b/docs/ai-python/content/docs/basics/meta.json @@ -13,6 +13,8 @@ "durable-execution", "subagents-and-multi-agent", "human-in-the-loop", + "harnesses", + "workspaces", "ai-sdk-ui", "telemetry" ] diff --git a/docs/ai-python/content/docs/basics/workspaces.mdx b/docs/ai-python/content/docs/basics/workspaces.mdx new file mode 100644 index 00000000..215a4f31 --- /dev/null +++ b/docs/ai-python/content/docs/basics/workspaces.mdx @@ -0,0 +1,268 @@ +--- +title: Workspaces +description: Choose where a harness runs and its files live. +type: guide +summary: Run harnesses in a local directory or a Vercel Sandbox, copy files, run commands and terminals, and reach a model through AI Gateway. +--- + +A workspace is where a [harness](/docs/basics/harnesses) runs. It owns every +path and process: the CLI, the commands the agent runs, and the files it +reads and writes. Change the workspace and the rest of your code stays the +same. + +This API is experimental and may change or be removed. + +## Local + +`Local` is a directory on this machine. It needs no extra. + +```python +from ai.harnesses import experimental as harnesses +from ai.workspaces import experimental as workspaces + +async with harnesses.codex(workspace=workspaces.Local("./project")) as harness: + await harness.run("Summarize the README.") +``` + +On your own machine a missing CLI raises `ExecutableMissingError`. The SDK +never installs software there. + +## Vercel Sandbox + +`VercelSandbox` is a Vercel Sandbox microVM. It needs the `sandbox` extra: + +```bash title="Terminal" +uv add "ai[sandbox]" +``` + +The extra needs a `vercel-sandbox` build with interactive PTY support +([vercel/vercel-py#415](https://github.com/vercel/vercel-py/pull/415), not +released yet). A missing or older package raises +`ai.errors.InstallationError` at construction. + +```python +gateway = workspaces.vercel_ai_gateway() # AI_GATEWAY_API_KEY + +async with workspaces.VercelSandbox(gateway=gateway) as sandbox: + await workspaces.copy("./my-project", sandbox / "project") + async with harnesses.codex(workspace=sandbox) as harness: + await harness.run("Fix the failing test in project/.") + await workspaces.copy(sandbox / "project", "./out") +``` + +- The SDK installs the latest release of the CLI into the sandbox when a + harness opens, and upgrades an older one, unless a copy of it is running + there. See [Harnesses](/docs/basics/harnesses#harnesses). +- The VM is the isolation boundary, so Codex runs there with full access. See + [Codex](/docs/basics/harnesses#codex). +- A harness opens its workspace but never closes it. Open the sandbox with + `async with`, as above, so it stops when you are done. + +### Credentials + +`VercelSandbox()` takes Vercel credentials from, in this order: + +- `token`, `team_id`, and `project_id` arguments. A project (OIDC) token + alone is enough: it names its own team and project. +- `VERCEL_TOKEN`, `VERCEL_TEAM_ID`, and `VERCEL_PROJECT_ID`, or + `VERCEL_OIDC_TOKEN`, in the environment. +- `.env.local` from `vercel link` and `vercel env pull`, found by searching + up from the current directory to the repository root. Pass `env_file=` or + `project_root=` to look elsewhere. + +The OIDC token is short-lived. When it expires, sandbox creation raises +`NotAuthenticatedError` telling you to pull it again. + +Only the Vercel credentials are read from `.env.local`; everything else in it +stays on your machine. Pass what the work needs with `env={...}`, or send the +whole file with `forward_project_env=True`. `env` applies to everything run +in the workspace, including the commands the agent runs. + +### Reach a model through AI Gateway + +A CLI authenticates where it runs. Your terminal login does not exist inside +a VM, so give the workspace a gateway: + +```python +gateway = workspaces.vercel_ai_gateway() +sandbox = workspaces.VercelSandbox(gateway=gateway) +``` + +`vercel_ai_gateway()` reads `AI_GATEWAY_API_KEY` from the environment, or +takes `api_key=`. It raises `NotAuthenticatedError` when there is no key. + +With a gateway, the sandbox locks its outbound traffic to an allowlist: + +- The gateway host. The sandbox firewall writes the key into requests to it, + so the VM only ever holds a placeholder. +- `registry.npmjs.org`, so the CLI can be installed. +- Any hosts in `allow_hosts=[...]`. + +Everything else is refused. + +`Local(path, gateway=gateway)` works too. There is no firewall locally, so +the real key goes in the CLI's environment. You can also pass `gateway=` to a +harness factory; it wins over the workspace's. + +### Options and reconnect + +| Option | Description | Default | +|---|---|---| +| `execution_time_limit` | How long the VM lives, in seconds. The platform default is short, so set it for long work. | `None`: the platform's default | +| `image` | The image a new VM boots from: a [Vercel Container Registry](https://vercel.com/docs/sandbox/concepts/images) reference such as `"vercel/sandbox/python:3.14"` or your own `"my-repo:v1"`. Not with `name` or `snapshot`. Harnesses need `python3` in the image, and `npm` to install their CLI; the default image has both. | `None`: the platform's default image | +| `keep` | Whether `close()` leaves the VM running. Not kept, `close()` destroys the sandbox and its snapshots. | `None`: `False` for a sandbox you created, `True` for one you reconnected to | +| `name` | Reconnect to an existing sandbox instead of creating one. `sandbox.name` is set once the VM exists. | `None`: create one | +| `ports` | Ports to expose. Get a URL with `await sandbox.reachable_url(port)`. | `None` | +| `snapshot` | Start from a sandbox snapshot. | `None` | +| `workdir` | The working directory. | `"/vercel/sandbox"` | + +```python +async with workspaces.VercelSandbox(keep=True, gateway=gateway) as sandbox: + harness = harnesses.claude_code(workspace=sandbox) + await harness.open() + session = harness.session() + await session.run("Start the migration.") + saved = session.handle.model_dump_json() + await harness.detach() # the CLI keeps running in the VM + name = sandbox.name + +# later, anywhere: +async with workspaces.VercelSandbox(name=name, gateway=gateway) as sandbox: + ... +``` + +- `harness.detach()` releases your client and leaves the CLI running. + `harness.close()` ends it. +- Without an approval hook, a detached agent runs to completion. With one, it + waits at its next tool call until a client reconnects. +- Outbound rules are fixed when the VM is created. Reconnecting with + `gateway=` to a sandbox made without one raises `WorkspaceError`. + +### Stop, resume, and what it costs + +A sandbox keeps its files when its VM stops: + +```python +await sandbox.stop() # the VM stops; its files are kept as a snapshot + +info = await workspaces.VercelSandbox(name=name).info() +print(info.status, info.snapshot.size_bytes) # "stopped", without resuming + +async with workspaces.VercelSandbox(name=name) as sandbox: + ... # resumed from the snapshot: files back, processes not +``` + +- A sandbox stops on `stop()` and when its `execution_time_limit` runs out. + Either way it is kept with a snapshot, and reconnecting resumes it. A + snapshot is billed as storage until it expires (30 days by default). +- `close()` on a sandbox you do not keep destroys it, snapshots included, + so nothing is left to pay for. +- `info()` describes a sandbox without resuming it: `status`, `region`, + `vcpus`, `memory_mb`, its `snapshot`, and its `sessions`. Each stopped + session reports what it is billed on: `duration_ms`, `active_cpu_ms`, + `egress_bytes`. A running session reports these when it stops. +- `WorkspaceGoneError` means the sandbox no longer exists, or stopped with + no snapshot to resume from. + +## Copy files + +`copy()` moves a directory between your machine and a workspace. Mark the +workspace side with `workspace / "path"`: + +```python +await workspaces.copy("./project", sandbox / "project") # in +await workspaces.copy(sandbox / "reviews", "./reviews") # out +await workspaces.copy(one / "out", other / "in") # between workspaces +``` + +`workspace / "path"` builds a `WorkspacePath`. Two plain paths raise +`TypeError`; use `shutil` for local copies. + +`copy()` skips what the tree's `.gitignore` files exclude, and common noise +such as `.git`, `node_modules`, and `.venv`. Pass `ignore=[...]` to set your +own glob patterns, or `gitignore=False` to ignore `.gitignore`. It returns +the number of files copied. + +For single files, use the workspace methods. Never use `pathlib`, which +breaks as soon as the workspace is remote: + +```python +text = await sandbox.read_text("project/notes.md") +await sandbox.write_text("project/todo.md", "- ship it\n") +data = await sandbox.read_bytes("project/logo.png") +found = await sandbox.exists("project/setup.py") +``` + +`read_text` raises `FileNotFoundError` for a missing file. + +## Run commands + +```python +result = await sandbox.exec(["pytest", "-q"], timeout=120) +print(result.exit_code, result.stdout, result.stderr) +``` + +`exec` returns an `ExecResult`. On `Local`, a program that cannot start +returns `exit_code=127` instead of raising, and a command that runs past +`timeout` is killed and raises `TimeoutError`. + +`spawn(argv)` starts a long-running `Process` with `write`, `readline`, +`read_stderr`, `wait`, `terminate`, and `detach`. `terminate()` returns only +once the process is gone. + +`workspace.contains(path)` checks whether an absolute path is inside the +workspace, without touching any filesystem. Relative paths return `False`. +Use it in approval hooks. + +## Terminals + +`pty(argv)` runs a program on a pseudo-terminal and returns a `Pty`: + +```python +pty = await sandbox.pty(["bash"], name="shell", size=(120, 40)) +await pty.send(b"ls\n") +print(await pty.receive()) +await pty.detach() + +pty = await sandbox.attach("shell") # from any process +print(await sandbox.ptys()) # [PtyInfo(name="shell", pid=..., attached=False)] +await pty.close() +``` + +- A named pty keeps running after `detach()`. `attach(name)` reconnects to + it, and `ptys()` lists them. An unnamed pty ends with your connection. +- `close()` ends the program. `resize(cols, rows)` and `wait()` are also + available. +- To attach your own terminal, use `tty.bridge(pty)` from + `ai.workspaces.experimental.tty`. It is POSIX only and not imported by the + package. + +## Coordinates + +`workspace.coords` is a `WorkspaceCoords` with `provider`, `location`, and +`options`. A harness `Handle` carries it, and +`harnesses.workspace_from(coords)` rebuilds the workspace. A sandbox is +rebuilt by reconnecting to it by name. + +## Errors + +Workspace errors live in `ai.workspaces.experimental.errors` and subclass +`WorkspaceError`, which subclasses `ai.errors.AIError`. + +| Error | Meaning | +|---|---| +| `WorkspaceGoneError` | The sandbox is stopped or never existed. | +| `WorkspaceError` | The workspace could not do what was asked, for example a sandbox that could not be reached. | +| `NotAuthenticatedError` | No usable credentials where the CLI runs, or for the sandbox itself. | + +Only forget a saved sandbox on `WorkspaceGoneError`. Any other +`WorkspaceError` may mean the VM is still running. + +## Examples + +- [`examples/harnesses/sandbox.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/sandbox.py) + runs both harnesses in a Vercel Sandbox. +- [`examples/harnesses/copy_files.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/copy_files.py) + copies a project in and the results out. +- [`examples/harnesses/reconnect.py`](https://github.com/vercel-labs/ai-python/blob/main/examples/harnesses/reconnect.py) + leaves work running in a sandbox and comes back to it. diff --git a/docs/ai-python/content/docs/reference/ai/index.mdx b/docs/ai-python/content/docs/reference/ai/index.mdx index b70997f0..ccbd0e2a 100644 --- a/docs/ai-python/content/docs/reference/ai/index.mdx +++ b/docs/ai-python/content/docs/reference/ai/index.mdx @@ -49,6 +49,10 @@ These module aliases are also re-exported from `ai`. - [`util`](/docs/reference/util): Async utility helpers. - [`experimental_telemetry`](/docs/reference/telemetry): Tracing and observability APIs. +- [`harnesses`](/docs/reference/harnesses): Coding-agent CLI drivers, under + `harnesses.experimental`. +- [`workspaces`](/docs/reference/workspaces): Places harnesses run, under + `workspaces.experimental`. ## Models diff --git a/docs/ai-python/content/docs/reference/harnesses.mdx b/docs/ai-python/content/docs/reference/harnesses.mdx new file mode 100644 index 00000000..b784bc91 --- /dev/null +++ b/docs/ai-python/content/docs/reference/harnesses.mdx @@ -0,0 +1,442 @@ +--- +title: "ai.harnesses.experimental" +description: Reference for driving coding-agent CLIs. +type: reference +summary: Reference for claude_code, codex, Harness, Session, Turn, Tui, Result, Handle, approvals, capabilities, and harness errors. +--- + +`ai.harnesses.experimental` drives coding-agent CLIs, Claude Code and Codex, +on a [workspace](/docs/reference/workspaces). See +[Harnesses](/docs/basics/harnesses) for a guide. + +This API is experimental and may change or be removed. + +```python +from ai.harnesses import experimental as harnesses +from ai.workspaces import experimental as workspaces + +async with harnesses.claude_code(workspace=workspaces.Local(".")) as harness: + result = await harness.run("What does this repo do?") +``` + +## Harnesses + +Each factory returns a `Harness`. They share these arguments: + +| Argument | Description | Default | +|---|---|---| +| `workspace` | `Workspace` the CLI runs in. | required | +| `approve` | Async function that takes an `ApprovalContext` and returns a `Decision`. | `None`: every tool call is allowed | +| `approval_timeout` | Seconds a hook may take before the call is denied. | `300.0` | +| `model` | Model name passed to the CLI. | `None`: the CLI's default | +| `gateway` | Upstream model provider LLM Gateway, as a `Gateway`. Overrides the workspace's gateway. | `None`: the workspace's gateway | +| `effort` | `"low"`, `"medium"`, `"high"`, or `"max"`. | `None`: the CLI's default | +| `executable` | CLI name or path. | `"claude"` or `"codex"` | +| `instructions` | Text added to the CLI's own system prompt. | `None` | +| `mcp_servers` | MCP servers: `{"name": {"command": ..., "args": [...], "env": {...}}}` or `{"name": {"url": ...}}`. Not stored in a `Handle`. | `None` | + +Every harness runs with full access unless an approval hook or its own +options restrict it. + +### claude_code + +```python +harnesses.claude_code( + *, + workspace, + approve=None, + approval_timeout=300.0, + model=None, + gateway=None, + effort=None, + executable="claude", + instructions=None, + mcp_servers=None, + tools=None, + disallowed_tools=None, + max_turns=None, +) +``` + +Claude Code. Needs the `claude-code` extra, and raises +`ai.errors.InstallationError` when it is missing. Takes the shared +arguments, plus: + +| Argument | Description | Default | +|---|---|---| +| `tools` | The built-in tools the agent has. `[]` for none. | `None`: all | +| `disallowed_tools` | Tools taken away. | `None` | +| `max_turns` | Most agent steps in one turn; hitting it settles as `"length"`. | `None` | + +### codex + +```python +harnesses.codex( + *, + workspace, + approve=None, + approval_timeout=300.0, + model=None, + sandbox=None, + config=None, + gateway=None, + effort=None, + executable="codex", + instructions=None, + mcp_servers=None, + tools=None, + disallowed_tools=None, + max_turns=None, +) +``` + +Codex over `codex app-server`. Needs only the `codex` CLI. Takes the shared +arguments, plus: + +| Argument | Description | Default | +|---|---|---| +| `sandbox` | Codex's own sandbox mode: `"read-only"`, `"workspace-write"`, or `"danger-full-access"`. On a provider-owned workspace, other values raise `UnsupportedError` on open. | `"danger-full-access"` | +| `config` | Codex config overrides. Wins over values derived from `gateway`, `effort`, and `mcp_servers`, key by key. | `None` | + +`instructions` become Codex's developer instructions. It also takes +`tools`, `disallowed_tools`, and `max_turns`, and raises `UnsupportedError` +when one is set. + +`effort="max"` is sent to Codex as `"xhigh"`. + +## Harness + +One configured CLI, open on one workspace. Build it with `claude_code()` or +`codex()`. Use it as an async context manager, or call `open()` and +`close()`. + +```python +harness.kind # "claude-code" or "codex" +harness.name # same as kind +harness.version # what the CLI reported, or None +harness.capabilities +harness.workspace +harness.options # model, executable, sandbox, effort +harness.approve # the hook, or None +harness.approval_timeout +harness.is_open +harness.process_ids # local pids; empty in a sandbox +``` + +Methods: + +- `await open()`: Open the workspace, then probe the CLI. Idempotent. +- `await close()`: End the CLI and its conversations. The workspace stays + open; close it yourself. +- `await detach()`: Release this client and leave the CLI running. On a + workspace whose processes end with the client, this is `close()`. +- `session(*, history=None)`: Start a new `Session`. With `history`, it + begins with that past. +- `await run(prompt, *, history=None, output_type=None, retries=1, + timeout=None)`: Run one prompt in a throwaway conversation. Returns a + `Result`. `prompt` may also be a whole conversation, `list[Message]`, + as `Agent.run()` takes it: the last message must be a user message and + becomes the prompt; the rest becomes `history`. Passing a conversation + and `history` together raises `ValueError`. +- `await sessions()`: List the CLI's conversations in this workspace as + `SessionInfo` values, including ones started outside the SDK. +- `await history(session_id, *, limit=None, offset=0)`: Return a stored + transcript as `ai.messages.Message` values, tool calls and files + included. `limit` and `offset` count those messages. +- `await resume(session_id)`: Continue an existing conversation. The + returned `Session`'s `messages` hold its past; run or stream on it as on + a new one. Raises + `SessionBusyError` when another client holds it, and `ResumeFailedError` + when it does not exist. +- `await fork(session_id)`: Start a new conversation with the same past. The + original is never written. +- `await tui(session_id=None, *, history=None, name=None, size=None)`: Open + the CLI's own TUI on a pty in the workspace. Returns a `Tui`. Passing both + `session_id` and `history` raises `ValueError`. + +Methods that need a capability the harness lacks raise `UnsupportedError`. +Methods called before `open()` raise `HarnessClosedError`. + +## Session + +One conversation. + +```python +session.session_id +session.messages +session.handle +``` + +`session_id` is the CLI's own id. It is empty until the first turn creates +the conversation. + +Methods: + +- `await run(prompt, *, output_type=None, retries=1, timeout=None)`: Send a + prompt and wait for the turn to settle. Returns a `Result`. +- `stream(prompt)`: Send a prompt and return a `Turn`. + +`prompt` is a `str`, a user `Message` (`ai.user_message(...)`), or a list +of `str`, `TextPart`, and `FilePart`. A conversation (`list[Message]`) +raises `TypeError` here; pass it to `Harness.run()` or as +`Harness.session(history=...)`. A file the harness cannot take (see `images` and `documents` in +`Capabilities`) raises `UnsupportedError` before anything runs. Text files +are sent as text on every harness. A URL is downloaded on this machine. +- `await steer(text)`: Send guidance to the running turn. Raises + `RuntimeError` when no turn is running. +- `await stop()`: End the running turn. It settles as `"cancelled"`. Safe + from any task; returns once the turn has settled. No-op when no turn runs. +- `await reclaim()`: Settle a turn whose consumer stopped iterating. `run()` + and `stream()` already do this, so you rarely need it. +- `await close()`: Release the conversation so another client can resume it. + +- `output_type`: The SDK adds the type's JSON schema to the prompt, + validates the reply, and asks again up to `retries` times before raising + pydantic's `ValidationError`. +- `timeout`: The turn is stopped at the deadline and returns a cancelled + `Result`. With `output_type` too, a cancelled reply that does not validate + raises. + +## Turn + +A running turn, the harness's peer of `ai.AgentStream`. Iterate it for +`ai.events.AgentEvent` values: model events, `ToolCallResult`, and the +approval events below. + +```python +async with session.stream(prompt) as turn: + async for event in turn: + ... +turn.result +turn.settled +turn.output # result.output +turn.usage # result.usage +turn.messages # live while it runs +turn.blocked +turn.deferred_hooks +await turn.aclose() +``` + +Leaving the `async with` block calls `aclose()`: the turn settles, and the +agent is stopped if it is still working. `result`, `output`, and `usage` +raise `RuntimeError` until the turn settles. A turn can be iterated once. + +The whole turn is one `StreamStart` ... `StreamEnd`, with tool calls and +their results inside: the CLI runs the loop and does not report each model +call. + +Approval events, when the harness has an `approve` hook: + +- `HookEvent`, status `"pending"`, when the CLI asks about a call. Its + `hook_id` is `"approve_"`, its `tool_call_id` the call's id, its + `metadata` `{"tool": name, "kwargs": input}`, and its `hook_type` + `APPROVAL_HOOK_TYPE`. +- `RunBlocked`, when the first pending approval starts. `blocked` is `True` + until no approval is pending. +- `HookEvent`, status `"resolved"` with the decision as its `resolution`, or + `"cancelled"` when the hook raised or timed out. + +These are answered by the hook, not by `ai.resolve_hook()`, and are not part +of `messages`. + +## Result + +One settled turn. + +| Field | Description | +|---|---| +| `finish_reason` | `"stop"`, `"cancelled"`, or `"length"` (Claude Code hit its turn limit). | +| `text` | The agent's final message, after its last tool call. | +| `messages` | The conversation as of settling: each tool call in an assistant message, then its result in a `"tool"` message. | +| `usage` | [`ai.types.usage.Usage`](/docs/reference/types). Claude Code reports cost in `usage.raw["cost_usd"]`. | +| `output` | The validated value when the turn ran with `output_type`; otherwise `text`. | +| `prompt_sent` | Exactly what the agent received. | +| `approval_errors` | Approval hooks that raised or timed out. Those calls were denied. | +| `cancelled` | `True` when `finish_reason` is `"cancelled"`. | + +## Tui + +A conversation a person drives in the CLI's own TUI. + +```python +tui.session_id +tui.handle +tui.pty +await tui.detach() +await tui.close() +``` + +- `session_id` and `handle` are `None` for a bare `tui()` on Codex, which + picks its own id. Find the conversation later with `sessions()`. +- `pty` is the `Pty` to attach with `ai.workspaces.experimental.tty.bridge`. +- `detach()` drops the connection. A named TUI keeps running. +- `close()` ends the TUI and releases the conversation. + +## SessionInfo + +One conversation in the CLI's store. + +| Field | Description | +|---|---| +| `kind` | `"claude-code"` or `"codex"`. | +| `session_id` | The CLI's own id. | +| `title` | The conversation's title or first prompt, or `None`. | +| `cwd` | The directory it ran in, or `None`. | +| `updated_at`, `created_at` | Timestamps as the CLI reports them, or `None`. | +| `running` | `True` when a process has the conversation open. An SDK client or `harness.tui()` holds it, so `resume()` would raise `SessionBusyError`. A CLI started in a terminal shows as running too: any `claude`, and a `codex` started as `codex resume `. `resume()` does not check for those. | +| `pid` | The process that has it open, in the workspace, or `None`. | + +## Handle + +A conversation as plain, serializable data. + +| Field | Description | +|---|---| +| `kind` | `"claude-code"` or `"codex"`. | +| `session_id` | The CLI's own id. | +| `workspace` | `WorkspaceCoords`. | +| `options` | Configuration the conversation was created with. | +| `harness_version` | CLI version, or `None`. | + +A handle carries no approval hook and no key. + +## harness_from + +```python +async with harnesses.harness_from( + handle, + *, + fork=False, + approve=None, + approval_timeout=300.0, + gateway=None, +) as session: + ... +``` + +Rebuilds the harness and workspace a `Handle` names, and yields its +conversation: resumed, or forked with `fork=True`. The harness closes when +the block ends; the rebuilt workspace is not closed. Raises +`ResumeFailedError` for an unknown harness kind. + +## workspace_from + +```python +workspace = harnesses.workspace_from(coords) +``` + +Rebuilds a `Local` or `VercelSandbox` from `WorkspaceCoords`. A sandbox is +reconnected by name. Other providers raise `WorkspaceError`. + +## Approvals + +### ApprovalContext + +What an approval hook decides with. It has no control verbs: the agent is +waiting on the decision. + +| Field | Description | +|---|---| +| `call` | The `ToolCall` being decided. | +| `history` | The conversation so far. | +| `workspace` | The workspace the CLI runs in. | +| `harness` | `"claude-code"` or `"codex"`. | +| `session_id` | The conversation's id. | +| `prompt` | The turn's prompt on Claude Code; `None` on Codex. | + +### ToolCall + +One tool call, as the CLI described it. + +| Field | Description | +|---|---| +| `id` | Call id. | +| `name` | The CLI's native tool name, such as `Bash` or `fileChange`. Codex names shell calls `commandExecution`; the command is in `input`. | +| `input` | Native arguments. | +| `title` | Display text. Do not key policy on it. | +| `kind` | A hint the CLI supplied, or `None`. | +| `agent_id` | Set when a subagent made the call. | +| `hints` | Extras the CLI supplied. | +| `raw` | The untouched native payload. | + +### Allow + +```python +harnesses.Allow(input=None, remember=False) +``` + +Lets the call run. `input` rewrites the arguments where +`capabilities.rewrite_tool_input` is true (Claude Code only). `remember` is +not supported on either harness. An unsupported option denies the call and +raises `UnsupportedError` from the turn. + +### Deny + +```python +harnesses.Deny(reason="", *, stop=False) +``` + +Blocks the call. `Deny("stay in the repo")` and `Deny(reason="stay in the +repo")` are the same. `reason` reaches the agent where +`capabilities.deny_reason` is true (Claude Code only). `stop=True` also ends +the turn. + +### Decision + +`Allow | Deny`. + +### APPROVAL_HOOK_TYPE + +`"harness_approval"`: the `hook_type` of the approval events a `Turn` +reports. Not `ToolApproval`, so AI SDK UI does not offer to answer them. + +The hook runs in a copy of the context of the turn it decides for, so +contextvars the turn's caller set are visible inside it. + +## Capabilities + +What a harness can do, declared before anything runs. Read it from +`harness.capabilities`. + +| Capability | Description | Claude Code | Codex | +|---|---|---|---| +| `run` | Run a prompt and wait for the result: `run()`. | yes | yes | +| `stream` | Iterate a turn's events as they arrive: `stream()`. | yes | yes | +| `steer` | Send guidance to the turn that is running: `session.steer()`. | yes | yes | +| `stop` | Interrupt the running turn; it settles as cancelled: `session.stop()`. | yes | yes | +| `resume` | Continue a conversation this process did not create: `resume()`, `harness_from()`. | yes | yes | +| `fork` | Branch a conversation into a new one with the same past: `fork()`. | yes | yes | +| `history` | Read stored transcripts in full, tool calls included: `history()`, `sessions()`. | yes | yes | +| `images` | Images in a prompt. | yes | yes | +| `documents` | PDFs in a prompt. | yes | no | +| `rewrite_tool_input` | Rewrite a call's arguments before it runs: `Allow(input={...})`. | yes | no | +| `deny_reason` | The reason in `Deny(reason)` reaches the agent. Without it the call is still denied, but the reason is dropped. | yes | no | +| `remember_decisions` | Keep an approval for the rest of the session: `Allow(remember=True)`. | no | no | +| `approval` | Which tool calls reach the hook. `"all"`: every call, reads included. `"policy"`: the CLI's own policy decides, so not every call may. | `"all"` | `"policy"` | + +`approval` is an `ApprovalCoverage`: + +| Value | Meaning | +|---|---| +| `"all"` | Every tool call reaches the hook. | +| `"policy"` | The CLI's own policy decides which calls reach the hook. | +| `"none"` | No approval channel. | + +`capabilities.approve` is `True` unless `approval` is `"none"`. + +## Errors + +`ai.harnesses.experimental.errors` contains the harness errors. They +subclass `HarnessError`, which subclasses `ai.errors.AIError`, except +`NotAuthenticatedError`. + +| Error | Meaning | Attributes | +|---|---|---| +| `UnsupportedError` | The harness lacks a capability. | `harness`, `capability` | +| `HarnessClosedError` | The harness is not open. | | +| `ExecutableMissingError` | The CLI is not runnable in the workspace. | `kind`, `executable`, `install` | +| `SessionBusyError` | Another client holds the conversation. Claims live in the workspace under `~/.ai-python/locks/`. | `session_id`, `holder_pid` | +| `ResumeFailedError` | The conversation could not be reopened. | `session_id` | +| `AgentCrashedError` | The CLI process died. | `harness`, `detail`, `exit_code` | +| `TurnFailedError` | The CLI reported the turn as failed. | `harness`, `detail` | +| `NotAuthenticatedError` | No credentials where the CLI runs. Re-exported from `ai.workspaces.experimental.errors`, so it subclasses `WorkspaceError`. | `harness`, `detail`, `hint` | diff --git a/docs/ai-python/content/docs/reference/index.mdx b/docs/ai-python/content/docs/reference/index.mdx index 5b306b0a..ae1ca701 100644 --- a/docs/ai-python/content/docs/reference/index.mdx +++ b/docs/ai-python/content/docs/reference/index.mdx @@ -9,5 +9,6 @@ Reference pages are organized by public module. Start with `ai` for names imported directly from the top-level package, such as `ai.stream`, `ai.Agent`, and `ai.Tool`. Use sibling module sections such as -`ai.events`, `ai.messages`, `ai.ops`, `ai.providers`, `ai.testing`, and +`ai.events`, `ai.messages`, `ai.ops`, `ai.providers`, `ai.testing`, +`ai.harnesses.experimental`, `ai.workspaces.experimental`, and `ai.agents` for public submodule APIs. diff --git a/docs/ai-python/content/docs/reference/meta.json b/docs/ai-python/content/docs/reference/meta.json index a361672f..46e7faa7 100644 --- a/docs/ai-python/content/docs/reference/meta.json +++ b/docs/ai-python/content/docs/reference/meta.json @@ -13,6 +13,8 @@ "events", "errors", "mcp", + "harnesses", + "workspaces", "types", "util", "telemetry" diff --git a/docs/ai-python/content/docs/reference/workspaces.mdx b/docs/ai-python/content/docs/reference/workspaces.mdx new file mode 100644 index 00000000..8a355511 --- /dev/null +++ b/docs/ai-python/content/docs/reference/workspaces.mdx @@ -0,0 +1,336 @@ +--- +title: "ai.workspaces.experimental" +description: Reference for places where harnesses run. +type: reference +summary: Reference for Local, VercelSandbox, Workspace, copy, Process, Pty, Gateway, tty, and workspace errors. +--- + +`ai.workspaces.experimental` contains the places a +[harness](/docs/reference/harnesses) runs: a local directory or a Vercel +Sandbox. See [Workspaces](/docs/basics/workspaces) for a guide. + +This API is experimental and may change or be removed. + +```python +from ai.workspaces import experimental as workspaces + +async with workspaces.Local("./project") as workspace: + result = await workspace.exec(["ls"]) +``` + +## Local + +```python +workspaces.Local(path, *, env=None, gateway=None) +``` + +A directory on this machine. `open()` creates it when missing. `close()` +ends every process it spawned. + +| Argument | Description | Default | +|---|---|---| +| `path` | Directory path. `~` is expanded. | required | +| `env` | Environment for everything run here, on top of this process's environment. | `None` | +| `gateway` | Upstream model provider LLM Gateway, as a `Gateway`. The credential goes in the CLI's environment. | `None` | + +## VercelSandbox + +```python +workspaces.VercelSandbox( + *, + token=None, + team_id=None, + project_id=None, + workdir="/vercel/sandbox", + snapshot=None, + image=None, + env=None, + forward_project_env=False, + env_file=None, + project_root=None, + execution_time_limit=None, + ports=None, + name=None, + keep=None, + gateway=None, + allow_hosts=(), +) +``` + +A Vercel Sandbox microVM. Needs the `sandbox` extra with interactive PTY +support, and raises `ai.errors.InstallationError` when it is missing. + +| Argument | Description | Default | +|---|---|---| +| `token`, `team_id`, `project_id` | Vercel credentials. A project (OIDC) token alone is enough: it names its own team and project. | `None`: read from the environment or `.env.local` | +| `workdir` | Working directory in the VM. | `"/vercel/sandbox"` | +| `snapshot` | Snapshot id to start from. | `None` | +| `image` | Vercel Container Registry image a new VM boots from (`"repo:tag"`, `"team/project/repo@sha256:..."`). Raises `ValueError` with `name` or `snapshot`. Harnesses need `python3` in the image, and `npm` to install their CLI. | `None`: the platform's default image | +| `env` | Environment for everything run in the VM. | `None` | +| `forward_project_env` | Also send every non-credential variable from `.env.local`. | `False` | +| `env_file` | Path to the env file. | `None`: the nearest `.env.local`, searching up from `project_root` to the repository root | +| `project_root` | Where to start searching for `.env.local`. | `None`: the current directory | +| `execution_time_limit` | VM lifetime in seconds. | `None`: the platform's default | +| `ports` | Ports to expose when the VM is created. | `None` | +| `name` | Reconnect to an existing sandbox instead of creating one. | `None`: create one | +| `keep` | Whether `close()` leaves the VM running. Not kept, `close()` destroys the sandbox and its snapshots. | `None`: `False` for a sandbox you created, `True` for one you reconnected to | +| `gateway` | Upstream model provider LLM Gateway, as a `Gateway`. Locks outbound traffic to an allowlist and injects the credential at the firewall; see [Workspaces](/docs/basics/workspaces#reach-a-model-through-ai-gateway). | `None` | +| `allow_hosts` | Extra hosts the VM may reach when `gateway` is set. | `()` | + +Beyond `Workspace`, a sandbox has `name`: the platform's name for it, or +`None` before it exists; and `image`: once open, the digest-pinned reference +the platform reports for the VM (`"vercel/sandbox/node@sha256:..."` for +`"vercel/sandbox/node:24"`; the tag is not kept), also after reconnecting by +`name`, and before that the one passed in, or `None` for the default. + +`open()` with `name` raises `WorkspaceGoneError` when the sandbox is stopped +or missing, `WorkspaceError` when it cannot be reached or was created without +credential injection for the requested gateway, and `NotAuthenticatedError` +when the Vercel credentials are rejected. + + +`VercelSandbox` methods beyond `Workspace`: + +- `await stop()`: Stop the VM now and keep its files as a snapshot. Closes + this client. `VercelSandbox(name=...)` resumes it later. +- `await info()`: Return a `SandboxInfo` without resuming the sandbox: + `name`, `status`, `region`, `vcpus`, `memory_mb`, `created_at`, + `snapshot` (a `SandboxSnapshot`: `id`, `size_bytes`, `created_at`, + `expires_at`), `sessions` (a list of `SandboxSession`: `id`, `status`, + `started_at`, `stopped_at`, `duration_ms`, `active_cpu_ms`, + `egress_bytes`, `ingress_bytes`, `vcpus`, `memory_mb`, `region`), and + `resumable`. Usage is reported once a session stops. Raises + `WorkspaceGoneError` when the sandbox no longer exists. +- Opening by `name` a sandbox that stopped resumes it from its snapshot. + +## Workspace + +The base class both workspaces implement. Use it as an async context +manager, or call `open()` and `close()`. + +```python +workspace.kind +workspace.owner +workspace.duplex_spawn +workspace.path +workspace.coords +workspace.env +workspace.gateway +``` + +| Property | Description | +|---|---| +| `kind` | `"local"` or `"sandbox"`. | +| `owner` | `"user"` or `"provider"`. The SDK installs a missing CLI only on a provider-owned workspace. | +| `duplex_spawn` | Whether `spawn()` gives a writable stdin. `True` on both. | +| `path` | The working directory as the agent sees it. | +| `coords` | `WorkspaceCoords` for this workspace. A `VercelSandbox` raises `WorkspaceError` before it is open. | +| `env` | Environment for everything run here. | +| `gateway` | The workspace's `Gateway`, or `None`. | + +Methods: + +- `await open()` and `await close()`. +- `await exec(argv, *, env=None, timeout=None)`: Run a command to + completion. Returns `ExecResult`. On `Local`, a program that cannot start + gives `exit_code=127`, and `timeout` kills the command and raises + `TimeoutError`. +- `await spawn(argv, *, env=None)`: Start a process with duplex stdio. + Returns `Process`. +- `await pty(argv, *, env=None, name=None, size=None)`: Run a program on a + pseudo-terminal. Returns `Pty`. A `name` keeps it running after you + detach; a name already running raises `WorkspaceError`. `size` is + `(cols, rows)`, `(80, 24)` by default. +- `await attach(name)`: Reconnect to a named pty. Returns `Pty`. Raises + `WorkspaceError` when none is running. +- `await ptys()`: List named ptys still running. Returns `list[PtyInfo]`. +- `await read_text(path)`, `await read_bytes(path)`: Read a file. Relative + paths resolve against `path`. Raise `FileNotFoundError` when it is + missing. +- `await write_text(path, data)`: Write a file, creating parent directories. +- `await exists(path)`: Whether a path exists. +- `await home()`: The home directory on the workspace's machine. The SDK + keeps its state there, under `~/.ai-python/` (session locks, named ptys). +- `await reachable_url(port)`: A URL the agent can reach for a port. +- `contains(candidate)`: Whether an absolute path is inside `path`. Pure-path + check; relative paths return `False`. A symlink inside that points out + still looks inside: pair it with `resolve()`. +- `await resolve(path)`: The real path `path` names on the workspace's + machine: relative to `path`, symlinks followed, `..` folded. A path that + does not exist yet resolves as far as it exists. +- `injects_credentials_for(host)`: Whether requests to `host` get the + credential at egress. +- `workspace / "path"`: Build a `WorkspacePath`. + +## copy + +```python +await workspaces.copy(source, dest, *, ignore=None, gitignore=True) +``` + +Copy a directory between your machine and a workspace, or between two +workspaces. Mark the workspace side with `workspace / "path"`. Two plain paths +raise `TypeError`. + +| Argument | Description | Default | +|---|---|---| +| `source`, `dest` | `str`, `Path`, or `WorkspacePath`. | required | +| `ignore` | Glob patterns to skip. | `None`: common noise such as `.git`, `node_modules`, `.venv`, and caches | +| `gitignore` | Also skip what the tree's `.gitignore` files exclude. | `True` | + +Returns the number of files copied. + +## contains + +```python +workspaces.contains(root, candidate) +``` + +Pure-path containment. Returns `True` when `candidate` is an absolute path +inside `root`. Does not touch the filesystem. + +## WorkspacePath + +A path and the workspace it lives in. Built with `workspace / "path"`. +Supports `/` for sub-paths. `str()` gives `"kind:path"`. + +| Field | Description | +|---|---| +| `workspace` | The `Workspace`. | +| `path` | The path inside it. | + +## WorkspaceCoords + +A serializable locator for a workspace. A harness `Handle` carries it. + +| Field | Description | +|---|---| +| `provider` | `"local"` or `"sandbox"`. | +| `location` | The directory, or the sandbox's name. | +| `options` | Provider options, such as the sandbox `workdir`. | + +## ExecResult + +A finished command. + +| Field | Description | +|---|---| +| `exit_code` | The exit status. | +| `stdout` | Standard output, as text. | +| `stderr` | Standard error, as text. | + +## Process + +A live process from `spawn()`. + +```python +process.pid +await process.write(data) +await process.readline() +await process.read_stderr() +await process.os_pid() +await process.wait() +await process.detach() +await process.terminate() +await process.kill() +``` + +- `pid`: Local pid, or `None` in a sandbox. +- `os_pid()`: The pid where the process runs, or `None` when unknown. +- `detach()`: Release the handle and leave the process running, where the + workspace allows it. On `Local` this is `terminate()`. +- `terminate()`: Returns only once the process is gone. In a sandbox it + raises `WorkspaceError` when the process survives `SIGKILL`. +- `wait()`: The exit status. + +## Pty + +The master side of a pseudo-terminal. + +```python +pty.name +pty.pid +await pty.send(data) +await pty.receive() +await pty.resize(cols, rows) +await pty.detach() +await pty.close() +await pty.wait() +``` + +- `name`: Set for a named pty, otherwise `None`. +- `pid`: The program's pid where it runs. +- `send(data)` and `receive()` take and return `bytes`. `receive()` returns + `b""` when the program has ended. +- `resize(cols, rows)` changes the window size. +- `detach()` drops the connection. A named program keeps running; an unnamed + one ends. +- `close()` ends the program. +- `wait()` returns the exit status, or `None` when the connection ended + first. + +## PtyInfo + +A named pty listed by `ptys()`. + +| Field | Description | +|---|---| +| `name` | What `attach()` takes. | +| `pid` | The program's pid. | +| `attached` | Whether a client is connected. Attaching anyway takes it over. | + +## Gateway + +Where a CLI reaches its model: a base URL and a credential. + +```python +gateway = workspaces.Gateway(base_url="https://...", credential="...") +gateway.host # hostname of base_url +gateway.is_brokered # True when credential is the placeholder +gateway.brokered() # a copy with the placeholder credential +``` + +The placeholder is `"ai-python-brokered-credential"`. Harnesses hand it to +the CLI on a workspace that injects the real key at egress. + +## vercel_ai_gateway + +```python +workspaces.vercel_ai_gateway( + *, + api_key=None, + base_url="https://ai-gateway.vercel.sh", +) +``` + +Returns a `Gateway` for Vercel AI Gateway. `api_key` defaults to +`AI_GATEWAY_API_KEY`. Raises `NotAuthenticatedError` when there is no key. + +## tty + +`ai.workspaces.experimental.tty` attaches this process's terminal to a +`Pty`. It is POSIX only and is not imported by the package: + +```python +from ai.workspaces.experimental import tty + +status = await tty.bridge(pty, detach_key=tty.DETACH_KEY) +``` + +- `bridge` puts the terminal in raw mode, forwards input, output, and window + size, and restores the terminal on the way out. +- It returns the program's exit status, or `None` when you detach or the + connection ends. +- `DETACH_KEY` is Ctrl-] (`b"\x1d"`). Pass `detach_key=None` to disable it. +- It raises `RuntimeError` when stdin is not a terminal. + +## Errors + +`ai.workspaces.experimental.errors` contains the workspace errors. All +subclass `WorkspaceError`, which subclasses `ai.errors.AIError`. + +| Error | Meaning | Attributes | +|---|---|---| +| `WorkspaceError` | The workspace could not do what was asked. | | +| `WorkspaceGoneError` | The workspace no longer exists. Any other `WorkspaceError` may mean it still does. | | +| `NotAuthenticatedError` | No usable credentials where the CLI runs, or for the sandbox. | `harness`, `detail`, `hint` | diff --git a/examples/.test_scripts/check-examples.py b/examples/.test_scripts/check-examples.py index 7bdad9e0..9f75ec8b 100755 --- a/examples/.test_scripts/check-examples.py +++ b/examples/.test_scripts/check-examples.py @@ -51,6 +51,16 @@ _EXAMPLES_DIR / "apps" / "slack_agent", ["."], ), + ( + "afk", + _EXAMPLES_DIR / "apps" / "afk", + ["."], + ), + ( + "deepysec", + _EXAMPLES_DIR / "apps" / "deepysec", + ["."], + ), ] diff --git a/examples/.test_scripts/run-examples.py b/examples/.test_scripts/run-examples.py index 2b12b5d6..7b145d86 100755 --- a/examples/.test_scripts/run-examples.py +++ b/examples/.test_scripts/run-examples.py @@ -6,6 +6,9 @@ uv run examples/.test_scripts/run-examples.py --image # also image samples uv run examples/.test_scripts/run-examples.py --video # also video samples uv run examples/.test_scripts/run-examples.py --e2e # also e2e test scripts + uv run examples/.test_scripts/run-examples.py --harnesses + # also harness samples: need logged-in `claude` and `codex` CLIs, and + # AI_GATEWAY_API_KEY plus Vercel credentials for the sandbox ones uv run examples/.test_scripts/run-examples.py --all # everything uv run examples/.test_scripts/run-examples.py --parallel # in parallel uv run examples/.test_scripts/run-examples.py models/stream.py @@ -74,6 +77,25 @@ class Sample: Sample("media/speech_generation.py"), ] +# These drive real coding-agent CLIs, not a model, so --model and --protocol +# do not apply. harnesses/tui.py is left out: it needs an interactive terminal. +HARNESS_SAMPLES: list[Sample] = [ + Sample("harnesses/basic.py", timeout=300.0), + Sample("harnesses/streaming.py", timeout=300.0), + Sample("harnesses/combine.py", timeout=300.0), + Sample("harnesses/as_tool.py", timeout=300.0), + Sample("harnesses/custom_loop.py", timeout=300.0), + Sample("harnesses/until_done.py", timeout=600.0), + Sample("harnesses/steering.py", timeout=300.0), + Sample("harnesses/tool_approval.py", timeout=300.0), + Sample("harnesses/take_over.py", timeout=300.0), + Sample("harnesses/resume.py", timeout=300.0), + Sample("harnesses/handoff.py", timeout=600.0), + Sample("harnesses/sandbox.py", timeout=600.0), + Sample("harnesses/copy_files.py", timeout=600.0), + Sample("harnesses/reconnect.py", timeout=900.0), +] + BROKEN_SAMPLES: list[Sample] = [] # E2E tests pick non-default ports so they don't collide with a running @@ -104,6 +126,7 @@ class Sample: *IMAGE_SAMPLES, *VIDEO_SAMPLES, *AUDIO_SAMPLES, + *HARNESS_SAMPLES, *BROKEN_SAMPLES, *E2E_TESTS, ] @@ -248,6 +271,11 @@ def main() -> None: parser.add_argument( "--audio", action="store_true", help="include audio samples" ) + parser.add_argument( + "--harnesses", + action="store_true", + help="include harness samples (real claude/codex CLIs)", + ) parser.add_argument( "--broken", action="store_true", help="include broken samples" ) @@ -290,6 +318,7 @@ def main() -> None: or args.image or args.video or args.audio + or args.harnesses or args.broken or args.e2e ) @@ -310,6 +339,8 @@ def main() -> None: samples.extend(VIDEO_SAMPLES) if not args.examples and (args.audio or args.all): samples.extend(AUDIO_SAMPLES) + if not args.examples and (args.harnesses or args.all): + samples.extend(HARNESS_SAMPLES) if not args.examples and (args.broken or args.all): samples.extend(BROKEN_SAMPLES) if not args.examples and (args.e2e or args.all): diff --git a/examples/README.md b/examples/README.md index 41527cfd..79223072 100644 --- a/examples/README.md +++ b/examples/README.md @@ -7,6 +7,9 @@ Examples are grouped by the main API surface they demonstrate. - `media/` contains image, video, speech, transcription, multimodal, embedding, and reranking examples. - `models/` contains model streaming, structured output, and provider examples. +- `harnesses/` contains examples that drive Claude Code and Codex, locally and + in a Vercel Sandbox (experimental; run with + `run-examples.py --harnesses`). Provider-specific model examples live under `models//`, such as `models/gateway/`, `models/openai/`, and `models/anthropic/`. diff --git a/examples/apps/afk/README.md b/examples/apps/afk/README.md new file mode 100644 index 00000000..fb6ea115 --- /dev/null +++ b/examples/apps/afk/README.md @@ -0,0 +1,257 @@ +# afk — step away; your agents don't + +A small CLI on `ai.harnesses` for one everyday situation: you have been working +with `claude` or `codex` in a directory, and you need to leave. `afk` lists +the conversations in that directory across both harnesses, opens one in the +harness's own TUI in a sandbox, lets you detach and close the laptop, +pushes work to run unattended, and comes back later to attach, watch, bring +home, or stop. Nothing has to be started "the afk way": it reads the +harnesses' own stores, so a conversation you began by typing `claude` an hour +ago is listed like any other. + +``` +$ afk +~/proj · claude, codex + + here + a3f5 claude migrate the billing tests to pytest active 2m + b81c claude add rate limiting to the webhook idle 6h + 01a0 codex audit the auth middleware idle 3h + + remote + c2d1 claude billing · migrate the billing tests… running $0.031 ($0.085/h) ← from a3f5 · amber-fox + 9e07 claude add rate limiting to the webhook finished $0.012 ($0.085/h) ← new, was running · ← from b81c · teal-owl + 4b2a codex audit the auth middleware expired $0.054 snapshot 1.1 GB, kept to Nov 18 · ← from 01a0 · red-elk + /h: memory, every hour it runs; CPU adds while it computes + +afk push [--bg] · attach · peek · pull [--files] · stop · rm +``` + +On a terminal that list is a picker, drawn under your prompt rather than +over the whole screen: arrow keys choose a conversation, Enter or a letter +runs the command shown for it as if you had typed it (`stop` asks first), +and `q` leaves the list in your scrollback. Piped, `afk` prints the list. + +A key means the same thing on every row it appears on: + +| key | | +|---|---| +| Enter | into the conversation: one here resumes in its harness's own TUI (`claude --resume ` or `codex resume `), a pushed one is attached. One here that is open in another terminal is taken over, after a y: that process ends first, then the TUI opens here (`afk stop && claude --resume `). Where afk cannot end what drives it, Enter peeks | +| `p` | peek | +| `s` | stop: a pushed one's sandbox (its files are kept), or the process a conversation here is open in | +| `r` | rm: delete a pushed one's sandbox and snapshot | +| `u`, `b` | push, push `--bg` (a conversation here) | +| `l`, `f` | pull, pull `--files` (a pushed one) | + +On a stopped or expired one, Enter resumes it. `stop` and `rm` ask first. + +A conversation here that is open in another terminal is `in use`. afk sees +any `claude`, but a `codex` only when it was started as `codex resume `: +a fresh `codex` names no conversation afk can find. + +## The verbs + +| | | +|---|---| +| `afk [--all]` | this directory's conversations, and the ones pushed elsewhere, with a status; `--all` lists every one, not just the recent 15 | +| `afk push [id] [--as NAME] [--bg [--prompt TEXT]] [--hours H] [--git-history] [--no-network]` | copy it and this directory into a sandbox of its own and open the TUI there, or run it unattended with `--bg`. `--prompt` tells the unattended agent what to do instead of "continue from where it left off". The sandbox runs up to `H` hours (default 5); in a git repository the work goes on a branch, and `--git-history` gives the sandbox the repository's history too. The network is open; `--no-network` leaves only the model gateway | +| `afk attach ` | your terminal, back on its TUI; if you quit the TUI, attach opens it again in the sandbox. A stopped or expired one is resumed from its snapshot first. One pushed with `--bg` opens in the TUI there: its unattended agent is ended first, after a y if it is still mid-turn | +| `afk peek ` | follow a conversation's transcript like `tail -f`, read-only: a pushed one, or one here open in another terminal. A status line says it is following; it ends when the agent finishes its turn or stops running, or, for one a person drives, after 30s with nothing new. Ctrl-C stops watching, never the agent. A stopped one is resumed to read it and stopped again after | +| `afk pull [--files]` | bring a conversation home and open it in the TUI here; `--files` also brings the work: in a git repository as a branch, otherwise the files it changed, after showing them and asking | +| `afk wait [id] [--every S]` | poll until it (or every pushed one that is running) finishes, stops, expires or goes; then ring the terminal bell and say what happened. Ctrl-C stops waiting, never the agent | +| `afk stop ` | stop its sandbox, keeping its files as a snapshot (`attach` resumes it; it is billed as storage until `rm` or its expiry, 30 days); for a conversation here, end the process it is open in and wait until it has exited (that terminal gets its prompt back; the conversation stays) | +| `afk rm ` | delete its sandbox and snapshot: nothing is kept, nothing more is billed | +| `afk setup` | choose the Vercel team afk's sandboxes use | + +**Ctrl-]** detaches from a sandbox TUI and leaves it running. A pulled +conversation opens in an ordinary local TUI: quit it as you always do. + +An id is whatever the list showed: a label you gave with `--as`, a word from a +title, or a short id prefix. With no id, `afk` chooses only when exactly one +conversation is clearly in use; with several in play it shows them and asks. +It never guesses. + +## Three flows + +**You were just working, normally.** In another tab: + +``` +$ afk push --as billing # copies the conversation into a sandbox, opens its TUI + # work in it; Ctrl-] and close the laptop +$ afk attach billing # next day, any machine: same TUI, scrollback intact +``` + +A sandbox afk creates runs up to five hours — the platform's own default is +five *minutes*, which is shorter than any time away — and stops on its own +after that; `--hours` sets it (Hobby plans allow up to 0.75). Stopping loses +nothing: the sandbox is kept with a snapshot of its files, listed as +`expired`, and `afk attach` resumes it, conversation and files intact. + +Push *copies*: the local conversation stays. Keep typing here and the list +shows both, the pushed one marked `← from a3f5`. + +**Let it work while you're gone.** + +``` +$ afk push --bg --as tests --prompt "make the failing tests pass" +$ afk wait tests # or come back later: `afk` marks what changed +$ afk peek tests # tail what it is doing +$ afk pull tests --files # bring the result home, its edits too, in the TUI +``` + +Unattended means no approval gate, so it never blocks. Without `--prompt` +it is asked to continue from where the conversation left off — fine when you +had just asked for something, useless when the last thing said was "not +yet". + +To step in, `afk attach tests` opens it in the TUI in its sandbox, on the +same conversation. The unattended agent still holds that conversation after +its turn, so attach ends it first; mid-turn, it asks, since that ends the +turn. From then on it is a TUI like any other. + +**Bring the work home.** `pull` alone brings only the conversation: your +files are never touched. `pull --files` also brings what the agent did. + +In a git repository (pushed from its top), it comes home as a branch. Push +records your working tree as it was — uncommitted and untracked files +included — as a commit on top of HEAD, without touching your branch, index +or files, and the sandbox works on `afk/