diff --git a/.changeset/exec-tool-single-backend.md b/.changeset/exec-tool-single-backend.md new file mode 100644 index 00000000..0b63d0f6 --- /dev/null +++ b/.changeset/exec-tool-single-backend.md @@ -0,0 +1,5 @@ +--- +"@cloudflare/computer": minor +--- + +The `exec` tool has no `backend` argument when only one backend is configured. It always runs there, its description no longer talks about choosing a backend, and `defaultBackend` becomes optional. A single shell backend also drops the `input` argument it could never accept, and a single callable backend describes `command` as ES module source. With more than one backend, nothing changes and `defaultBackend` is still required. diff --git a/.changeset/unified-modules.md b/.changeset/unified-modules.md new file mode 100644 index 00000000..42f091ec --- /dev/null +++ b/.changeset/unified-modules.md @@ -0,0 +1,11 @@ +--- +"@cloudflare/computer": minor +--- + +`WorkerJavaScriptBackend` takes a single `modules` option. A string is bundled source, as before. An object of functions is a host module that runs in the Durable Object under a `ws:*` specifier, and each function becomes a named export: `modules: { "ws:weather": { forecast } }` lets code write `import { forecast } from "ws:weather"`. A factory, `(host) => ({ ... })`, builds a host module from the Workspace's Git client, Artifacts client, or runtime. Each function receives `(args, { signal, deadline, access, resolvePath })` and may return any JSON-compatible value. + +`ws:git` and `ws:artifacts` are no longer installed automatically. Add `createGitModule()` from `@cloudflare/computer/modules/git` and `createArtifactsModule()` from `@cloudflare/computer/modules/artifacts`. `createContainerModule()` from `@cloudflare/computer/modules/container` runs shell commands in the Workspace's container backend. The container shares the Workspace's files, a canceled execution kills the command, and it refuses to run on a read-only backend. `node:fs` and `node:fs/promises` stay built in. + +The backend now describes its source language and every module for a model, and the `exec` tool shows that text. A backend's `description` in the tool options becomes optional when the backend describes itself, so the module list the model reads always matches what is installed. The tool also stops offering `input` when no configured backend accepts it. + +To migrate, move `trustedModules` entries into `modules`, replacing any `call(method, args)` handler with one function per method. Replace `allowGitNetwork: true` with `createGitModule({ allowNetwork: true })` and `allowArtifactNetwork: true` with `createArtifactsModule({ allowNetwork: true })`. diff --git a/README.md b/README.md index dd124e41..cb7daa64 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,8 @@ SQLite and exposes one pluggable execution surface through Workers RPC, so there is no second store or sync round trip. - **Isolate JavaScript** runs an ECMAScript module in a fresh Dynamic Worker with structured input/results, durable relative imports, - configured libraries, Workspace-backed `node:fs/promises`, and trusted `ws:git` and - `ws:artifacts` modules. + configured libraries, Workspace-backed `node:fs/promises`, and host modules such as + `ws:git`, `ws:artifacts`, and `ws:container`. A Workspace may register multiple backends under stable IDs. `workspace.runtime.exec(source, { backend })` is the single execution diff --git a/docs/09_tool_interface.md b/docs/09_tool_interface.md index a8629a1c..181a2ba7 100644 --- a/docs/09_tool_interface.md +++ b/docs/09_tool_interface.md @@ -55,7 +55,16 @@ export class Agent { Pass the returned AI SDK `ToolSet` to `generateText`, `streamText`, or an agent framework hook such as `getTools()`. -Pass `shell` only when the Workspace has matching backend ids: +Pass `shell` only when the Workspace has matching backend ids. With one backend, `exec` has no `backend` argument and always runs there: + +```ts +const tools = createAITools({ + workspace, + shell: { backends: { "worker-javascript": {} } }, +}); +``` + +With more than one, pass `defaultBackend` and the model picks a backend per call: ```ts const tools = createAITools({ @@ -244,7 +253,19 @@ The tool uses forced removal, so deleting a missing path succeeds. Set `recursiv ## `exec` -`exec` is opt-in. It calls `workspace.runtime.exec` with the configured backend and streams bounded output. Backend descriptions are included in the model-facing tool description, so describe capabilities and startup cost in plain language. +`exec` is opt-in. It calls `workspace.runtime.exec` with the configured backend and streams bounded output. + +Each backend's entry in the tool description joins two parts: the `description` you pass, and what the backend says about itself (`backend.description`, read through `workspace.runtime.describe(id)`). `WorkerJavaScriptBackend` describes its source language and every module code can import, so `{ "worker-javascript": {} }` is enough and the list stays in step with `modules`. A backend that does not describe itself needs a `description`. Describe capabilities and startup cost in plain language. + +The tool offers only the arguments that can work: + +| Backends | Arguments | +| --- | --- | +| One shell backend | `command`, `cwd`, `env` | +| One callable backend | `command`, `cwd`, `env`, `input` | +| More than one | `command`, `cwd`, `backend`, `env`, plus `input` when any is callable. `defaultBackend` is required. | + +A `backend` value the model sends anyway is dropped when only one backend is configured. The output still names the backend that ran. Wire this tool carefully: it executes arbitrary shell commands inside the configured backend. Treat its output as untrusted text when including it in later model input. Omit `shell` or use `readonly: true` when command execution is not part of the agent's job. diff --git a/docs/16_code_execution.md b/docs/16_code_execution.md index 26371512..34c316a2 100644 --- a/docs/16_code_execution.md +++ b/docs/16_code_execution.md @@ -17,7 +17,7 @@ The selected backend defines how it interprets `source`. | --- | --- | --- | | `container-shell` | shell command | Full Linux, native binaries, installed packages, processes | | `worker-shell` | just-bash command | Fast text tools and Workspace Git without a Container | -| `worker-javascript` | ECMAScript module | Isolated structured JavaScript with trusted Workspace modules | +| `worker-javascript` | ECMAScript module | Isolated structured JavaScript with Workspace host modules | Applications may register additional command or module backends under their own IDs. Backend IDs are part of the execution contract: changing the backend may change the source language. @@ -85,4 +85,4 @@ new WorkerJavaScriptBackend({ The backend argument is never itself authorization. -See [17. Isolate JavaScript](./17_isolate_javascript.md) for module and trusted-package behavior. +See [17. Isolate JavaScript](./17_isolate_javascript.md) for module behavior. diff --git a/docs/17_isolate_javascript.md b/docs/17_isolate_javascript.md index 2db84ecf..93555d4a 100644 --- a/docs/17_isolate_javascript.md +++ b/docs/17_isolate_javascript.md @@ -88,7 +88,7 @@ Completed execution records remain available for replay for sixty minutes by def Cancellation stops new host capability calls, disposes the Dynamic Worker, and waits for host calls that were already accepted. Exit 130 is published only after those calls settle. Normal completion uses the same drain rule, so an unawaited capability call cannot mutate the workspace after exit 0. -Host calls have a caller-visible deadline, controlled by `maxHostCallMs` and defaulting to `maxTimeoutMs`. Missing the deadline fails the capability call and marks the execution failed, even if caller code catches that error. Execution still waits for the accepted host operation itself before publishing a terminal event because many host APIs cannot roll back an external side effect after dispatch. Trusted modules receive an optional `{ signal, deadline }` context and must stop promptly when the signal aborts. A trusted module that ignores cancellation and never settles will keep execution in its finalizing state. `compatibilityDate` and `compatibilityFlags` control the Dynamic Worker runtime and default to the package-tested settings. +Host calls have a caller-visible deadline, controlled by `maxHostCallMs` and defaulting to `maxTimeoutMs`. Missing the deadline fails the capability call and marks the execution failed, even if caller code catches that error. Execution still waits for the accepted host operation itself before publishing a terminal event because many host APIs cannot roll back an external side effect after dispatch. Host module functions receive a `signal` in their context and must stop promptly when it aborts. A host module that ignores cancellation and never settles will keep execution in its finalizing state. `compatibilityDate` and `compatibilityFlags` control the Dynamic Worker runtime and default to the package-tested settings. ## Environment, standard input, and the `process` shim @@ -119,22 +119,54 @@ const handle = await workspace.runtime.exec( ); ``` -## Configured modules +## Modules -Bare imports are installed at backend construction, not passed on individual executions: +Caller source can import three kinds of module, and all of them are fixed when the backend is constructed: + +| Kind | Configured with | Runs in | Example | +| --- | --- | --- | --- | +| Built in | Always installed | The isolate, backed by the Workspace | `node:fs`, `node:fs/promises` | +| Source | `modules: { name: "source" }` | The isolate | a bundled library | +| Host | `modules: { "ws:name": { fn } }`, or a factory | The Durable Object | `ws:git`, `ws:container`, your own | ```ts +import { createArtifactsModule } from "@cloudflare/computer/modules/artifacts"; +import { createContainerModule } from "@cloudflare/computer/modules/container"; +import { createGitModule } from "@cloudflare/computer/modules/git"; + new WorkerJavaScriptBackend({ loader: env.LOADER, modules: { "tar-stream": TAR_STREAM_BUNDLE, + "ws:git": createGitModule(), + "ws:artifacts": createArtifactsModule(), + "ws:container": createContainerModule(), + "ws:weather": { + forecast: ([city]) => lookUpForecast(String(city)), + }, }, }); ``` -Unknown bare imports fail before Worker creation. `node:fs` and `node:fs/promises` are host-installed exceptions backed by the durable Workspace. Configured modules are code, not host authority, and may not use the reserved `ws:` namespace or shadow either filesystem specifier. +An import that is not built in, configured, or a relative Workspace path fails before the Worker is created. Caller source and durable files cannot shadow a configured or built-in module. + +The backend describes its modules for a model in `backend.description`, and the `exec` tool shows that text, so the list the model reads always matches what is installed: + +```text +`command` is ECMAScript module source, run in an isolated JavaScript runtime. Relative imports resolve from `cwd` in the workspace. +Code has no direct network access. + +Modules code can import: +- `node:fs/promises` (also `node:fs`): the workspace's files. ... +- `tar-stream`: a bundled library. +- `ws:git`: The workspace's Git repository tools: `status({ dir })`, ... +- `ws:container`: Runs shell commands in a full Linux container that shares this workspace's files. ... +- `ws:weather`: exports `forecast`. +``` + +A factory adds its own text through a `description` property, as the prebuilt modules do. An object of functions is listed by its export names; say more about it in the `exec` tool's backend description if the model needs it. -## Trusted Workspace modules +### Built-in filesystem Filesystem access uses the familiar asynchronous Node API, but is backed by the durable Workspace rather than an isolate-local filesystem. Both forms are installed automatically: @@ -148,7 +180,56 @@ await fs.writeFile("/workspace/output.txt", text.toUpperCase()); Supported promise APIs are `readFile`, `writeFile`, `mkdir`, `rm`, `chmod`, `symlink`, `readlink`, `readdir`, `stat`, `lstat`, and `access`. `readFile` returns bytes when encoding is omitted and supports `"utf8"` / `"utf-8"` for text; other encodings are rejected. `writeFile` supports the default `"w"` flag and exclusive `"wx"`; other Node flags are rejected, and—as in Node—the parent directory must already exist. Relative symlink targets are preserved by `readlink`, while reads and writes through symlinks are rejected by the Workspace confinement boundary. Synchronous and callback-style Node filesystem APIs are intentionally unavailable because every operation crosses the isolate-to-Workspace capability boundary. -The entire `ws:` namespace remains reserved for other Workspace-maintained host capabilities. The built-in runtime installs `ws:git` and `ws:artifacts`. +Path confinement rejects lexical escapes and every symlink component before an operation. These checks are not an atomic inode-style “resolve beneath root” primitive: do not treat one isolate capability as a security boundary against a separate, more privileged principal concurrently replacing paths in the same mutable Workspace. Deployments requiring that adversarial concurrency need a future transactional DOFS primitive or separate Workspace identities. + +### Source modules + +A string value is JavaScript source installed as a bare import, such as a bundled library. It is plain code with no host access, and it cannot use the `ws:` namespace or replace `node:fs` or `node:fs/promises`. + +### Host modules + +A host module runs in the Durable Object, and each of its functions becomes a named export in the isolate. Host modules must use a simple `ws:*` specifier. Nothing under `ws:` is installed unless you configure it. + +Pass an object of functions: + +```ts +modules: { + "ws:weather": { + forecast: ([city]) => lookUpForecast(String(city)), + }, +} +``` + +When the functions need the Workspace's Git client, Artifacts client, or runtime, pass a factory instead. The backend calls it once when it connects to its Workspace. This is how the prebuilt modules work: + +```ts +modules: { + "ws:repo": (host) => ({ + async recent(args, context) { + const dir = await context.resolvePath(String(args[0] ?? ".")); + return host.git.log({ dir, depth: 5 }); + }, + }), +} +``` + +```js +import { recent } from "ws:repo"; +export default () => recent("/workspace/app"); +``` + +Each function receives the arguments the isolate passed, as an array of JSON-compatible values, and a context: + +| Field | Meaning | +| --- | --- | +| `signal` | Aborts when the call passes its deadline or the execution is cancelled. | +| `deadline` | Epoch milliseconds after which the isolate stops waiting. | +| `access` | The backend's `"read"` or `"read-write"` access. Check it before any write. | +| `resolvePath(path, { allowMissing })` | Confines a caller path to the backend root and rejects symlinks. | + +The arguments come from caller code, so parse them before use. A function may return a value or a promise. The result must be JSON-compatible, and the bridge checks it at runtime: `undefined` becomes `null` and `undefined` object fields are dropped, as with `JSON.stringify`. It fits within the same capability byte limits as every other host call. A function that ignores `signal` and never settles keeps the execution in its finalizing state. + +Specifiers and the export names of an object are checked at construction. A factory's export names are checked when the backend connects and the factory runs. A module must export at least one function, and every export name must be a JavaScript identifier name other than `default` or `then`. A reserved word such as `delete` is allowed, and caller code renames it on import: `import { delete as remove } from "ws:files"`. Importing a name the module does not export fails when the module graph links, before any code runs. ### `ws:git` @@ -156,25 +237,57 @@ The entire `ws:` namespace remains reserved for other Workspace-maintained host import { clone, diff, status, log, cli } from "ws:git"; ``` -`ws:git` is explicit host authority rather than ambient isolate networking. Clone, fetch, pull, push, `ls-remote`, and submodule commands can perform host-side requests even when the Dynamic Worker has `globalOutbound: null`, so they are denied by default. Enable them only on a trusted backend construction with `allowGitNetwork: true`; local Git operations remain available without that authority. Remote `ws:artifacts.importArtifact()` is independently denied unless backend construction sets `allowArtifactNetwork: true`. +`createGitModule()` from `@cloudflare/computer/modules/git` wraps the Workspace's Git client. Every `dir` and `cwd` is confined to the backend root, `clone` and `cli` need a read-write backend, and `cli` rejects `-C`, `--git-dir`, and `--work-tree`. Clone, fetch, pull, push, `ls-remote`, and submodule commands run from the host, even when the Dynamic Worker has `globalOutbound: null`, so they are denied unless you pass `createGitModule({ allowNetwork: true })`. ### `ws:artifacts` ```js -import { - create, - get, - list, - importArtifact, - deleteArtifact, -} from "ws:artifacts"; +import { create, get, list, importArtifact, deleteArtifact } from "ws:artifacts"; ``` -These modules are sandbox-side shims over host RPC. Loader bindings, credentials, Durable Object storage, and unrestricted Workspace objects never enter user code. The host bridge checks the backend's fixed read/read-write authority on every mutation. Artifacts methods fail clearly when no Artifacts binding is configured. +`createArtifactsModule()` from `@cloudflare/computer/modules/artifacts` wraps the Workspace's Artifacts client. Calls that change Artifacts need a read-write backend. `importArtifact()` fetches from a caller-chosen URL on the host, so it is denied unless you pass `createArtifactsModule({ allowNetwork: true })`. Every call fails clearly when no Artifacts binding is configured. -Caller modules and durable files cannot shadow `node:fs`, `node:fs/promises`, or `ws:*`. +### `ws:container` -Path confinement rejects lexical escapes and every symlink component before an operation. These checks are not an atomic inode-style “resolve beneath root” primitive: do not treat one isolate capability as a security boundary against a separate, more privileged principal concurrently replacing paths in the same mutable Workspace. Deployments requiring that adversarial concurrency need a future transactional DOFS primitive or separate Workspace identities. +`createContainerModule()` from `@cloudflare/computer/modules/container` lets JavaScript run shell commands in the Workspace's container backend. With it, JavaScript is the only backend the model sees, and the container is something that JavaScript can call: + +```ts +this.workspace = new Workspace({ + storage: ctx.storage, + backends: [ + new WorkerJavaScriptBackend({ + loader: env.LOADER, + access: "read-write", + modules: { "ws:container": createContainerModule() }, + }), + new CloudflareContainerBackend({ /* ... */ }), + ], +}); + +const tools = createAITools({ + workspace: this.workspace, + shell: { backends: { "worker-javascript": {} } }, +}); +``` + +```js +import { exec } from "ws:container"; + +export default async function () { + const { exitCode, stdout, stderr } = await exec("npm test", { cwd: "/workspace/app" }); + return { passed: exitCode === 0, stdout, stderr }; +} +``` + +`exec(command, { cwd, env, stdin, timeoutMs })` runs through `workspace.runtime.exec` on the container backend (`"container-shell"` unless you pass `backend`). The container shares the Workspace's files: writes the module made before the call are pushed to the container, and the container's changes are pulled back before `exec` returns. A non-zero exit code comes back as a value, not as an error. + +A few limits follow from `exec` being a host call: + +- Output comes back when the command finishes, not while it runs. Each stream is cut at `maxOutputBytes` (64 KiB by default), which must stay well under the backend's `maxCapabilityBytes`. +- The command's timeout is capped at the time left before the host call deadline (`maxHostCallMs`, which defaults to `maxTimeoutMs`). Raise `defaultTimeoutMs`, `maxTimeoutMs`, and `maxHostCallMs` for slow installs and builds, and remember the container's first start. +- Cancelling the execution kills the running command. + +A container command can write to the Workspace and reach the network, whatever the JavaScript backend's egress settings say. `exec` refuses to run on a read-only backend. ## Isolation and lifecycle @@ -190,7 +303,3 @@ Each execution receives a fresh Dynamic Worker with: - retained events and result rows in the Workspace database. Standard output and standard error stream live. The Dynamic Worker hands the readable end of its output stream to the host through the `attachOutput` bridge call, and the host drains it frame by frame while user code is still running, appending each chunk to the execution event stream as it arrives rather than buffering the run and publishing at the end. The structured result and the exit event settle once the output stream closes, so the terminal events always follow the last output. Output remains bounded by `maxStdioBytes` across both streams. Completed writes are durable immediately. Failure or cancellation does not roll back filesystem effects already completed. - -## Trusted integrations - -A host can configure additional reserved capability modules through `WorkerJavaScriptBackend.trustedModules`; these modules are fixed when the backend is constructed and cannot be supplied or replaced by caller source. diff --git a/docs/README.md b/docs/README.md index 25b76eb7..2775b03e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,7 +20,7 @@ It provides: - R2-backed mounts for pre-filling read-only data into the workspace tree. - Durability over DO restarts for all file operations. - Pluggable execution backends selected through `workspace.runtime`: a Cloudflare Container shell, a just-bash Dynamic Worker, or an isolated ECMAScript-module Dynamic Worker. - - Isolated JavaScript with structured input/results, durable relative imports, configured libraries, durable `node:fs/promises`, trusted `ws:git` / `ws:artifacts`, and managed execution records. + - Isolated JavaScript with structured input/results, durable relative imports, configured libraries, durable `node:fs/promises`, host modules such as `ws:git` and `ws:container`, and managed execution records. - Workspace constructable without a backend, for filesystem-only use cases. - Out-of-the-box AI SDK tools for `@cloudflare/agents` through `@cloudflare/computer/tools`. @@ -45,9 +45,12 @@ The package ships several entrypoints: | `@cloudflare/computer` | The Workspace wrapper, first-class `workspace.runtime`, stub types, the R2 mount, and proxy classes. | | `@cloudflare/computer/backends/container` | `CloudflareContainerBackend` and `withWorkspaceContainer`. Pulls in the computerd / capnweb sync plumbing. | | `@cloudflare/computer/backends/worker-shell` | `WorkerShellBackend` and the bundled just-bash command runtime. | -| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable relative imports, `node:fs/promises`, and trusted `ws:git` / `ws:artifacts`. | +| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable relative imports, `node:fs/promises`, and host modules. | | `@cloudflare/computer/git` | Opt-in isomorphic-git glue for working with checkouts inside the workspace. Bundled lazily, with `pako` replaced by Workers `node:zlib`, and kept out of the default `@cloudflare/computer` graph. | | `@cloudflare/computer/artifacts` | `createArtifact`, an optionally session-scoped wrapper over the Cloudflare Artifacts Workers binding, plus its argv CLI. | +| `@cloudflare/computer/modules/container` | `createContainerModule()` for `ws:container`: run container commands from isolate JavaScript. | +| `@cloudflare/computer/modules/git` | `createGitModule()` for `ws:git`: confined Git from isolate JavaScript. | +| `@cloudflare/computer/modules/artifacts` | `createArtifactsModule()` for `ws:artifacts`: Artifacts from isolate JavaScript. | | `@cloudflare/computer/tools` | AI SDK tools for agents: read, write, edit, ls, optional exec, and optional publish. | A consumer that only uses the container backend never imports the @@ -240,7 +243,7 @@ above, then dive into the area you're working on. | [14. Assets interface](./14_assets_interface.md) | `share` a workspace file to R2 and get back a presigned URL. | | [15. Artifacts interface](./15_artifacts_interface.md) | `createArtifact` and the `artifacts` CLI, an optionally session-scoped wrapper over the Cloudflare Artifacts binding. | | [16. Execution runtime architecture](./16_code_execution.md) | One runtime entry point over command and module backends. | -| [17. Isolate JavaScript runtime](./17_isolate_javascript.md) | ECMAScript modules, durable imports, configured libraries, durable `node:fs/promises`, trusted `ws:git` / `ws:artifacts`, and managed lifecycle. | +| [17. Isolate JavaScript runtime](./17_isolate_javascript.md) | ECMAScript modules, durable imports, configured libraries, durable `node:fs/promises`, host modules, and managed lifecycle. | | [18. Runtime migration](./18_runtime_migration.md) | Breaking preview-API mappings from public shell and script-execution surfaces to `workspace.runtime`. | | [19. Performance](./19_performance.md) | Filesystem benchmarks: `fs-bench` numbers, an `npm install` comparison, and how to reproduce them. | diff --git a/examples/rlm/README.md b/examples/rlm/README.md index 7d08e6db..899533f3 100644 --- a/examples/rlm/README.md +++ b/examples/rlm/README.md @@ -66,7 +66,7 @@ const backend = new WorkerJavaScriptBackend({ root: "/workspace", access: "read", egress: { mode: "none" }, - trustedModules: { + modules: { "ws:model": modelCapability, }, }); @@ -77,13 +77,13 @@ const workspace = new Workspace({ }); ``` -The important line is `trustedModules`. Generated code cannot read model credentials or call the network directly. It can only use the host-owned `ws:model` interface. +The important line is `modules`. Generated code cannot read model credentials or call the network directly. It can only use the host-owned `ws:model` interface. A generated module follows this shape: ```js import fs from "node:fs/promises"; -import { call as callModel } from "ws:model"; +import { batch } from "ws:model"; export default async function () { const manifest = JSON.parse( @@ -96,7 +96,7 @@ export default async function () { })), ); - const mapped = await callModel("batch", requests); + const mapped = await batch(requests); const totals = validateAndSum(mapped); return { answer: largestLabel(totals) }; } @@ -150,7 +150,7 @@ The reducer is exact relative to its inputs. Model classifications can still be The shortest path through the example is: 1. [`worker/rlm-agent.ts`](worker/rlm-agent.ts) wires together the model, Workspace, Worker JavaScript backend, executor tool, and `ws:model`. -2. [`worker/capability.ts`](worker/capability.ts) implements the bounded `ws:model("batch", requests)` interface. +2. [`worker/capability.ts`](worker/capability.ts) implements the bounded `batch(requests)` function behind `ws:model`. 3. [`worker/structured-rlm.ts`](worker/structured-rlm.ts) describes the map result and JavaScript reduction for each task family. 4. [`worker/agent-common.ts`](worker/agent-common.ts) writes the same corpus into each Computer Workspace. 5. [`worker/executor-tool.ts`](worker/executor-tool.ts) creates the native Computer executor tool and keeps its browser-facing result small. diff --git a/examples/rlm/worker/capability.test.ts b/examples/rlm/worker/capability.test.ts index d6bc9fcf..c902cdea 100644 --- a/examples/rlm/worker/capability.test.ts +++ b/examples/rlm/worker/capability.test.ts @@ -1,3 +1,4 @@ +import type { WorkspaceModuleCallContext } from "@cloudflare/computer"; import type { LanguageModel } from "ai"; import { beforeEach, describe, expect, it, vi } from "vitest"; @@ -30,6 +31,15 @@ function successfulResult(text = "ok") { }; } +function context(signal = new AbortController().signal): WorkspaceModuleCallContext { + return { + signal, + deadline: Date.now() + 1_000, + access: "read", + resolvePath: async (path) => path, + }; +} + async function waitForCalls(count: number): Promise { for (let attempt = 0; attempt < 50; attempt += 1) { if (generateTextMock.mock.calls.length >= count) return; @@ -44,28 +54,25 @@ beforeEach(() => { }); describe("recursive model batch capability", () => { - it("exposes only the batch method", async () => { + it("exposes only the batch function", () => { const capability = createModelCapability(model, hooks()); - await expect(capability.call("generate", [[]])).rejects.toThrow( - "Unknown model capability method", - ); - expect(generateTextMock).not.toHaveBeenCalled(); + expect(Object.keys(capability)).toEqual(["batch"]); }); it("strictly validates the external argument and request shapes", async () => { const capability = createModelCapability(model, hooks()); - await expect(capability.call("batch", [])).rejects.toThrow("exactly one argument"); - await expect(capability.call("batch", [null])).rejects.toThrow("non-empty array"); - await expect(capability.call("batch", [[]])).rejects.toThrow("non-empty array"); + await expect(capability.batch([], context())).rejects.toThrow("exactly one argument"); + await expect(capability.batch([null], context())).rejects.toThrow("non-empty array"); + await expect(capability.batch([[]], context())).rejects.toThrow("non-empty array"); await expect( - capability.call("batch", [[{ prompt: "classify", input: null, extra: true }], null]), + capability.batch([[{ prompt: "classify", input: null, extra: true }], null], context()), ).rejects.toThrow("exactly one argument"); await expect( - capability.call("batch", [[{ prompt: "classify", input: null, extra: true }]]), + capability.batch([[{ prompt: "classify", input: null, extra: true }]], context()), ).rejects.toThrow("Invalid batch request"); - await expect(capability.call("batch", [[{ prompt: "", input: null }]])).rejects.toThrow( + await expect(capability.batch([[{ prompt: "", input: null }]], context())).rejects.toThrow( "requires a prompt", ); expect(generateTextMock).not.toHaveBeenCalled(); @@ -78,7 +85,9 @@ describe("recursive model batch capability", () => { input: null, })); - await expect(capability.call("batch", [requests])).rejects.toThrow("cannot exceed 24 requests"); + await expect(capability.batch([requests], context())).rejects.toThrow( + "cannot exceed 24 requests", + ); expect(generateTextMock).not.toHaveBeenCalled(); }); @@ -88,7 +97,7 @@ describe("recursive model batch capability", () => { const capability = createModelCapability(model, { ...observed, admit }); const requests = [{ prompt: "classify", input: "evidence" }]; - await expect(capability.call("batch", [requests])).rejects.toThrow( + await expect(capability.batch([requests], context())).rejects.toThrow( "exhausted its child-model call budget", ); expect(admit).toHaveBeenCalledWith(1); @@ -99,7 +108,7 @@ describe("recursive model batch capability", () => { const capability = createModelCapability(model, hooks()); await expect( - capability.call("batch", [[{ prompt: "é".repeat(8 * 1024 + 1), input: null }]]), + capability.batch([[{ prompt: "é".repeat(8 * 1024 + 1), input: null }]], context()), ).rejects.toThrow("16384 bytes"); expect(generateTextMock).not.toHaveBeenCalled(); }); @@ -109,10 +118,10 @@ describe("recursive model batch capability", () => { const maximumBody = "x".repeat(48 * 1024 - 11); await expect( - capability.call("batch", [[{ prompt: "classify", input: { body: maximumBody } }]]), + capability.batch([[{ prompt: "classify", input: { body: maximumBody } }]], context()), ).resolves.toHaveLength(1); await expect( - capability.call("batch", [[{ prompt: "classify", input: { body: `${maximumBody}x` } }]]), + capability.batch([[{ prompt: "classify", input: { body: `${maximumBody}x` } }]], context()), ).rejects.toThrow("49152 bytes"); }); @@ -123,7 +132,7 @@ describe("recursive model batch capability", () => { input: { body: "x".repeat(48 * 1024 - 12) }, })); - await expect(capability.call("batch", [requests])).rejects.toThrow( + await expect(capability.batch([requests], context())).rejects.toThrow( "Model batch request cannot exceed", ); expect(generateTextMock).not.toHaveBeenCalled(); @@ -145,9 +154,10 @@ describe("recursive model batch capability", () => { }), ); const capability = createModelCapability(model, hooks()); - const resultPromise = capability.call("batch", [ - Array.from({ length: 8 }, (_, index) => ({ prompt: `request ${index}`, input: null })), - ]); + const resultPromise = capability.batch( + [Array.from({ length: 8 }, (_, index) => ({ prompt: `request ${index}`, input: null }))], + context(), + ); await waitForCalls(4); expect(generateTextMock).toHaveBeenCalledTimes(4); @@ -186,10 +196,7 @@ describe("recursive model batch capability", () => { const signal = new AbortController().signal; await expect( - capability.call("batch", [[{ prompt: "classify", input: { evidence: "safe" } }]], { - signal, - deadline: Date.now() + 1_000, - }), + capability.batch([[{ prompt: "classify", input: { evidence: "safe" } }]], context(signal)), ).resolves.toEqual([ { id: expect.any(String), index: 0, ok: true, text: "classification", error: null }, ]); @@ -225,7 +232,7 @@ describe("recursive model batch capability", () => { const capability = createModelCapability(model, observer); await expect( - capability.call("batch", [[{ prompt: "classify", input: null }]]), + capability.batch([[{ prompt: "classify", input: null }]], context()), ).resolves.toEqual([ { id: expect.any(String), @@ -255,10 +262,9 @@ describe("recursive model batch capability", () => { const observer = hooks(); const capability = createModelCapability(model, observer); const controller = new AbortController(); - const result = capability.call( - "batch", + const result = capability.batch( [Array.from({ length: 8 }, (_, index) => ({ prompt: `request ${index}`, input: null }))], - { signal: controller.signal, deadline: Date.now() + 1_000 }, + context(controller.signal), ); await waitForCalls(4); @@ -274,9 +280,10 @@ describe("recursive model batch capability", () => { const observer = hooks(); const capability = createModelCapability(model, observer); - await capability.call("batch", [ - [{ prompt: "SECRET_PROMPT", input: { evidence: "SECRET_INPUT" } }], - ]); + await capability.batch( + [[{ prompt: "SECRET_PROMPT", input: { evidence: "SECRET_INPUT" } }]], + context(), + ); const synchronizedHookData = JSON.stringify({ started: observer.started.mock.calls, diff --git a/examples/rlm/worker/capability.ts b/examples/rlm/worker/capability.ts index bc8b0e05..7ad79083 100644 --- a/examples/rlm/worker/capability.ts +++ b/examples/rlm/worker/capability.ts @@ -1,4 +1,4 @@ -import type { WorkspaceRuntimeValue, WorkspaceTrustedModule } from "@cloudflare/computer"; +import type { WorkspaceModuleFunction, WorkspaceRuntimeValue } from "@cloudflare/computer"; import { generateText, type LanguageModel } from "ai"; import { z } from "zod"; @@ -73,18 +73,20 @@ interface ChildHooks { failed(metadata: FailedMetadata): void; } -export function createModelCapability( - model: LanguageModel, - hooks: ChildHooks, -): WorkspaceTrustedModule { +/** The `ws:model` trusted module: one `batch` function over bounded child requests. */ +export type ModelCapability = { + /** Run up to 24 child model requests and return one result per request. */ + readonly batch: WorkspaceModuleFunction; +}; + +export function createModelCapability(model: LanguageModel, hooks: ChildHooks): ModelCapability { return { - async call(method, args, context) { - if (method !== "batch") throw new Error(`Unknown model capability method: ${method}`); + async batch(args, context) { const requests = parseBatchArgs(args); if (hooks.admit && !hooks.admit(requests.length)) { throw new Error("This run has exhausted its child-model call budget."); } - const signal = context?.signal; + const signal = context.signal; throwIfAborted(signal); const results: Array = new Array(requests.length); @@ -165,7 +167,7 @@ async function runChild( } } -function parseBatchArgs(args: WorkspaceRuntimeValue[]): ChildRequest[] { +function parseBatchArgs(args: readonly WorkspaceRuntimeValue[]): ChildRequest[] { if (args.length !== 1) throw new Error("Model batch requires exactly one argument."); const value = args[0]; if (!Array.isArray(value) || value.length === 0) { diff --git a/examples/rlm/worker/prompts.ts b/examples/rlm/worker/prompts.ts index 3243e9c0..ca554246 100644 --- a/examples/rlm/worker/prompts.ts +++ b/examples/rlm/worker/prompts.ts @@ -16,11 +16,11 @@ The executor command must be a complete ES module with export default async func export const RLM_SYSTEM_PROMPT = `Solve the Oolong benchmark with one comprehensive recursive Computer execution, then answer briefly. -The executor command must be a complete ES module with export default async function. Use import fs from "node:fs/promises" and import { call as callModel } from "ws:model". Read /workspace/oolong-real/manifest.json. Its contextChunks contain the long corpus. +The executor command must be a complete ES module with export default async function. Use import fs from "node:fs/promises" and import { batch } from "ws:model". Read /workspace/oolong-real/manifest.json. Its contextChunks contain the long corpus. Use Computer as an RLM. Do not make manifest-inspection, schema-discovery, or diagnostic-only calls: 1. Read the bounded corpus chunks. -2. You have one child-call budget of 24 total requests. Call callModel("batch", requests) exactly once with at most 24 focused requests. Each request is { prompt, input }; keep each input to one chunk and ask for structured, question-specific evidence. +2. You have one child-call budget of 24 total requests. Call batch(requests) exactly once with at most 24 focused requests. Each request is { prompt, input }; keep each input to one chunk and ask for structured, question-specific evidence. 3. Each child result is { index, ok, text, error }. Aggregate successful findings in JavaScript and tolerate failed workers. For first/last-event questions, retain chunk indexes and choose evidence by transcript position; never replace an earlier event with a later, more salient one. 4. Parse the child text, aggregate findings in corpus chunk order, and return { answer } from the default function. If a later execution is needed to finalize from prior findings, it must still return { answer }. diff --git a/examples/rlm/worker/rlm-agent.ts b/examples/rlm/worker/rlm-agent.ts index 7d0fd706..bd34039e 100644 --- a/examples/rlm/worker/rlm-agent.ts +++ b/examples/rlm/worker/rlm-agent.ts @@ -107,7 +107,7 @@ export class RlmAgent extends AIChatAgent { root: WORKSPACE_ROOT, access: "read", egress: { mode: "none" }, - trustedModules: { "ws:model": modelCapability }, + modules: { "ws:model": modelCapability }, maxConcurrentExecutions: 1, maxConcurrentCapabilityCalls: 4, // One manifest read + 24 chunk reads + one bounded ws:model batch. diff --git a/examples/rlm/worker/rlm-interfaces.txt b/examples/rlm/worker/rlm-interfaces.txt index 7e876c9a..139ce825 100644 --- a/examples/rlm/worker/rlm-interfaces.txt +++ b/examples/rlm/worker/rlm-interfaces.txt @@ -1,7 +1,7 @@ Module interface: import fs from "node:fs/promises"; -import { call as callModel } from "ws:model"; +import { batch } from "ws:model"; export default async function () { // Put every file read, batch call, reduction, and return statement inside this function. } -No asynchronous operation or return statement may appear at module scope. Call recursive inference exactly once as callModel("batch", requests). +No asynchronous operation or return statement may appear at module scope. Call recursive inference exactly once as batch(requests). diff --git a/examples/rlm/worker/step-settings.ts b/examples/rlm/worker/step-settings.ts index 7e91f873..443bcfbb 100644 --- a/examples/rlm/worker/step-settings.ts +++ b/examples/rlm/worker/step-settings.ts @@ -5,7 +5,7 @@ export const COMPUTER_FINALIZATION_STEP = 1; const FINALIZATION_INSTRUCTION = 'Return the benchmark answer now. The next executor module must return a typed { answer } object using evidence already seen. Do not inspect more data, return diagnostics, or call ws:model again. The final module must contain no imports or file I/O. Use exactly this shape: export default async function () { return { answer: "derived answer" }; }'; const RECURSION_RETRY_INSTRUCTION = - 'The previous execution did not use recursive inference. Call ws:model("batch", requests) now with question-specific requests over the Workspace chunks, then return the child findings.'; + "The previous execution did not use recursive inference. Call batch(requests) from ws:model now with question-specific requests over the Workspace chunks, then return the child findings."; export function requiredComputerStep( stepNumber: number, diff --git a/examples/rlm/worker/structured-rlm.test.ts b/examples/rlm/worker/structured-rlm.test.ts index 4c76af39..789808c8 100644 --- a/examples/rlm/worker/structured-rlm.test.ts +++ b/examples/rlm/worker/structured-rlm.test.ts @@ -114,7 +114,7 @@ describe("structured RLM strategy", () => { expect(prompt).toContain("structured-v1"); expect(prompt).toContain("last_spell_by_episode"); - expect(prompt).toContain('callModel("batch", requests)'); + expect(prompt).toContain("batch(requests)"); expect(prompt).not.toContain("SECRET_GOLD"); }); }); diff --git a/packages/computer/README.md b/packages/computer/README.md index 14833957..3f0115b8 100644 --- a/packages/computer/README.md +++ b/packages/computer/README.md @@ -254,8 +254,8 @@ Alongside `exec`, the runtime exposes `getExec`, `killExec`, and [`examples/worker-shell`](../../examples/worker-shell). - **Worker JavaScript** evaluates a module with structured input/results, durable relative imports, configured libraries, - Workspace-backed `node:fs/promises`, and trusted `ws:git` / - `ws:artifacts` modules. It runs after `runtime.exec()` returns; the + Workspace-backed `node:fs/promises`, and host modules such as + `ws:git`, `ws:artifacts`, and `ws:container`. It runs after `runtime.exec()` returns; the run stays alive while its event stream is consumed. See [`docs/17_isolate_javascript.md`](../../docs/17_isolate_javascript.md) and [`examples/worker-javascript`](../../examples/worker-javascript). @@ -417,7 +417,10 @@ on a computerd instance. | `@cloudflare/computer` | The `Workspace` wrapper, `workspace.runtime`, stub types, the R2 mount, and proxy classes. | | `@cloudflare/computer/backends/container` | `CloudflareContainerBackend` and `withWorkspaceContainer`. Pulls in the computerd / capnweb sync plumbing. | | `@cloudflare/computer/backends/worker-shell` | `WorkerShellBackend` and the bundled just-bash runtime. | -| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable imports, `node:fs/promises`, and trusted `ws:git` / `ws:artifacts`. | +| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable imports, `node:fs/promises`, and host modules. | +| `@cloudflare/computer/modules/container` | `createContainerModule()` for `ws:container`: run container commands from isolate JavaScript. | +| `@cloudflare/computer/modules/git` | `createGitModule()` for `ws:git`: confined Git from isolate JavaScript. | +| `@cloudflare/computer/modules/artifacts` | `createArtifactsModule()` for `ws:artifacts`: Artifacts from isolate JavaScript. | | `@cloudflare/computer/tools` | AI SDK tools for agents: `read`, `ls`, `find`, `grep`, `write`, `edit`, `delete`, and optional `exec` and `publish`. | | `@cloudflare/computer/git` | Opt-in `isomorphic-git` glue for checkouts inside the workspace. | | `@cloudflare/computer/assets` | `createAssets` — share a workspace file to R2 as a presigned URL. | diff --git a/packages/computer/package.json b/packages/computer/package.json index 42898ddf..7baeb171 100644 --- a/packages/computer/package.json +++ b/packages/computer/package.json @@ -31,6 +31,18 @@ "types": "./dist/artifacts/index.d.ts", "import": "./dist/artifacts/index.js" }, + "./modules/container": { + "types": "./dist/modules/container.d.ts", + "import": "./dist/modules/container.js" + }, + "./modules/git": { + "types": "./dist/modules/git.d.ts", + "import": "./dist/modules/git.js" + }, + "./modules/artifacts": { + "types": "./dist/modules/artifacts.d.ts", + "import": "./dist/modules/artifacts.js" + }, "./tools": { "types": "./dist/tools/index.d.ts", "import": "./dist/tools/index.js" diff --git a/packages/computer/rolldown.config.ts b/packages/computer/rolldown.config.ts index 7f21309e..7179b638 100644 --- a/packages/computer/rolldown.config.ts +++ b/packages/computer/rolldown.config.ts @@ -31,6 +31,9 @@ export default defineConfig({ "artifacts/index": "src/artifacts/index.ts", "assets/index": "src/assets/index.ts", "tools/index": "src/tools/index.ts", + "modules/container": "src/modules/container.ts", + "modules/git": "src/modules/git.ts", + "modules/artifacts": "src/modules/artifacts.ts", "backends/container/index": "src/backends/container/index.ts", "backends/worker-javascript/index": "src/backends/worker-javascript/index.ts", "backends/worker-shell/index": "src/backends/worker-shell/index.ts", diff --git a/packages/computer/src/backend.ts b/packages/computer/src/backend.ts index dac310eb..fd6c3fd9 100644 --- a/packages/computer/src/backend.ts +++ b/packages/computer/src/backend.ts @@ -60,6 +60,11 @@ export interface WorkspaceBackend { // too. Defaults to false when omitted. readonly callable?: boolean; + // What the backend tells a model about itself, such as the language + // it runs and what that code can use. The exec tool shows it next to + // the caller's own description. Omit when there is nothing to add. + readonly description?: string; + // Materialise a connection. Called lazily on first use, once // per backend per workspace lifetime. The Workspace caches the // resulting handle by `id`; subsequent exec / push / pull diff --git a/packages/computer/src/backends/worker-javascript/module-graph.ts b/packages/computer/src/backends/worker-javascript/module-graph.ts index 8280e7b5..66cdcae3 100644 --- a/packages/computer/src/backends/worker-javascript/module-graph.ts +++ b/packages/computer/src/backends/worker-javascript/module-graph.ts @@ -1,7 +1,12 @@ import { parse } from "acorn"; import type { WorkspaceRuntimeCapability } from "../../runtime/capability.js"; -import type { WorkspaceRuntimeLoader } from "../../runtime/types.js"; +import type { + WorkspaceModule, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceRuntimeLoader, +} from "../../runtime/types.js"; export type JavaScriptModuleMap = WorkspaceRuntimeLoader extends { load(code: { modules: infer Modules }): unknown; @@ -12,14 +17,119 @@ export type JavaScriptModuleMap = WorkspaceRuntimeLoader extends { const ENTRY_BASENAME = "__workspace_entry__.js"; const RUNNER_MODULE = "workspace-runtime-runner.js"; const CAPABILITIES_MODULE = "workspace-capabilities.js"; -const TRUSTED_MODULES = ["node:fs", "node:fs/promises", "ws:git", "ws:artifacts"] as const; +// Installed in every execution and backed by the Workspace. No module +// in the `modules` option may use these names. +const BUILT_IN_MODULES = ["node:fs", "node:fs/promises"] as const; +const HOST_SPECIFIER = /^ws:[A-Za-z0-9][A-Za-z0-9._-]*$/; +const EXPORT_NAME = /^[A-Za-z_$][A-Za-z0-9_$]*$/; +// `default` would turn the function into the default export, and a +// `then` export makes the module namespace look like a promise to +// `await import(...)`. +const RESERVED_EXPORT_NAMES = new Set(["default", "then"]); + +/** The `modules` option, parsed once when the backend is constructed. */ +export interface ParsedModules { + readonly source: Readonly>; + readonly host: ReadonlyMap; + /** One markdown bullet per importable module, for a model. */ + readonly description: string; +} + +const FILESYSTEM_DESCRIPTION = + "- `node:fs/promises` (also `node:fs`): the workspace's files. `readFile`, `writeFile`, `mkdir`, `rm`, `readdir`, `stat`, `lstat`, `readlink`, `symlink`, `chmod`, and `access`. Async only."; + +/** + * Parse the backend's `modules` option: split it into bundled source + * and host module factories, check every specifier and every object's + * export names, and describe each module for a model. A factory's + * export names are checked when the backend connects and it runs. + * + * @param modules - The modules passed to the backend. + * @returns Source modules, host module factories, and their description. + * @throws When a specifier or an object's export names are not allowed. + * The host configured the backend wrongly and no execution can use it. + */ +export function parseModules(modules: Readonly>): ParsedModules { + const source: Record = Object.create(null); + const host = new Map(); + const lines = [FILESYSTEM_DESCRIPTION]; + for (const [specifier, module] of Object.entries(modules)) { + const name = `\`${specifier}\``; + if (BUILT_IN_MODULES.some((builtIn) => builtIn === specifier)) { + throw new Error(`Module ${JSON.stringify(specifier)} is built in and cannot be replaced.`); + } + if (typeof module === "string") { + if (specifier.startsWith("ws:")) { + throw new Error( + `Module ${JSON.stringify(specifier)} uses the ws:* namespace, which is only for host modules.`, + ); + } + if (specifier.includes("/") || isInternalModuleName(specifier)) { + throw new Error(`Module ${JSON.stringify(specifier)} uses a reserved module name.`); + } + source[specifier] = module; + lines.push(`- ${name}: a bundled library.`); + continue; + } + if (!HOST_SPECIFIER.test(specifier)) { + throw new Error( + `Host module ${JSON.stringify(specifier)} must use a simple ws:* name, such as "ws:container".`, + ); + } + if (typeof module === "function") { + host.set(specifier, module); + lines.push(`- ${name}: ${module.description ?? "a host module."}`); + continue; + } + if (module === null || typeof module !== "object" || Array.isArray(module)) { + throw new Error( + `Module ${JSON.stringify(specifier)} must be source text, an object of functions, or a factory.`, + ); + } + assertHostModuleExports(specifier, module); + host.set(specifier, () => module); + const exports = Object.keys(module).map((key) => `\`${key}\``); + lines.push(`- ${name}: exports ${exports.join(", ")}.`); + } + return { source, host, description: lines.join("\n") }; +} + +/** + * Check the functions a host module exports. + * + * @param specifier - The module's specifier, for error messages. + * @param functions - The module's functions. + * @throws When the module exports nothing, a name is not allowed, or a + * value is not a function. + */ +export function assertHostModuleExports( + specifier: string, + functions: WorkspaceModuleFunctions, +): void { + const names = Object.keys(functions); + if (names.length === 0) { + throw new Error(`Host module ${JSON.stringify(specifier)} must export a function.`); + } + for (const name of names) { + if (!EXPORT_NAME.test(name) || RESERVED_EXPORT_NAMES.has(name)) { + throw new Error( + `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a JavaScript identifier other than "default" or "then".`, + ); + } + if (typeof functions[name] !== "function") { + throw new Error( + `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a function.`, + ); + } + } +} export interface BuildModuleGraphOptions { source: string; cwd: string; capability: WorkspaceRuntimeCapability; - configuredModules: Record; - trustedModuleNames?: string[]; + configuredModules: Readonly>; + hostModules: ReadonlyMap; maxSourceBytes: number; maxCapabilityBytes: number; maxModules?: number; @@ -39,18 +149,10 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { let totalBytes = new TextEncoder().encode(options.source).byteLength; const maxModules = options.maxModules ?? 128; const maxDepth = options.maxDepth ?? 32; - const trustedModuleNames = new Set(TRUSTED_MODULES); - for (const name of options.trustedModuleNames ?? []) { - if ( - !/^ws:[A-Za-z0-9][A-Za-z0-9._-]*$/.test(name) || - TRUSTED_MODULES.includes(name as (typeof TRUSTED_MODULES)[number]) - ) { - throw new Error( - `Trusted module ${JSON.stringify(name)} must use a unique simple reserved ws:* name.`, - ); - } - trustedModuleNames.add(name); - } + const importableModuleNames = new Set([ + ...BUILT_IN_MODULES, + ...options.hostModules.keys(), + ]); async function visit(path: string, source: string, depth: number): Promise { if (depth > maxDepth) throw new Error(`Workspace JavaScript import depth exceeds ${maxDepth}.`); @@ -63,12 +165,14 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { } for (const specifier of imports(source)) { - if (trustedModuleNames.has(specifier)) continue; + if (importableModuleNames.has(specifier)) continue; if (specifier === CAPABILITIES_MODULE) { throw new Error(`Module ${JSON.stringify(specifier)} is reserved for Workspace internals.`); } if (specifier.startsWith("ws:")) { - throw new Error(`Unknown trusted Workspace module ${JSON.stringify(specifier)}.`); + throw new Error( + `Module ${JSON.stringify(specifier)} is not configured. Add it to the backend's modules option.`, + ); } if (specifier.startsWith(".")) { const resolved = resolveRelative(path, specifier); @@ -111,21 +215,6 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { await visit(entryPath, options.source, 0); - for (const specifier of Object.keys(options.configuredModules)) { - if ( - trustedModuleNames.has(specifier) || - specifier.startsWith("ws:") || - specifier === ENTRY_BASENAME || - specifier === RUNNER_MODULE || - specifier === CAPABILITIES_MODULE || - specifier.includes("/") - ) { - throw new Error( - `Configured module ${JSON.stringify(specifier)} uses a reserved module name.`, - ); - } - } - // node:* specifiers use protocol-style resolution and therefore need exact // module-map keys rather than the importer-directory aliases used by ws:*. modules["node:fs/promises"] = { js: nodeFsPromisesModule() }; @@ -134,11 +223,9 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { for (const directory of directories) { const prefix = directory ? `${directory}/` : ""; const toCapabilities = relativeModule(directory, CAPABILITIES_MODULE); - modules[`${prefix}ws:git`] = { js: gitModule(toCapabilities) }; - modules[`${prefix}ws:artifacts`] = { js: artifactsModule(toCapabilities) }; - for (const specifier of options.trustedModuleNames ?? []) { + for (const [specifier, functions] of options.hostModules) { modules[`${prefix}${specifier}`] = { - js: trustedModule(toCapabilities, specifier), + js: hostModule(toCapabilities, specifier, Object.keys(functions)), }; } for (const [specifier, source] of Object.entries(options.configuredModules)) { @@ -297,17 +384,15 @@ function capabilitiesModule(maxCapabilityBytes: number) { `; } -function proxyModule(capabilitiesImport: string, namespace: string, methods: string[]) { +// Exports go through `export { local as name }` rather than +// `export const name`, so a reserved word such as `delete` still works +// as an export name. +function hostModule(capabilitiesImport: string, specifier: string, names: readonly string[]) { + const namespace = JSON.stringify(`host/${specifier}`); return ` import { call } from ${JSON.stringify(capabilitiesImport)}; - ${methods.map((method) => `export const ${method} = (...args) => call(${JSON.stringify(namespace)}, ${JSON.stringify(method)}, args);`).join("\n")} - `; -} - -function trustedModule(capabilitiesImport: string, specifier: string) { - return ` - import { call as hostCall } from ${JSON.stringify(capabilitiesImport)}; - export const call = (method, ...args) => hostCall(${JSON.stringify(`trusted/${specifier}`)}, "call", [method, ...args]); + ${names.map((name, index) => `const fn${index} = (...args) => call(${namespace}, ${JSON.stringify(name)}, args);`).join("\n")} + export { ${names.map((name, index) => `fn${index} as ${name}`).join(", ")} }; `; } @@ -370,17 +455,3 @@ function nodeFsPromisesModule() { function nodeFsModule() { return `${nodeFsPromisesModule()}\nexport { default as promises } from "node:fs/promises";`; } - -function gitModule(capabilitiesImport: string) { - return proxyModule(capabilitiesImport, "git", ["clone", "diff", "status", "log", "cli"]); -} - -function artifactsModule(capabilitiesImport: string) { - return proxyModule(capabilitiesImport, "artifacts", [ - "create", - "get", - "list", - "importArtifact", - "deleteArtifact", - ]); -} diff --git a/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts b/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts index f75ef9bc..e75b00fb 100644 --- a/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts +++ b/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts @@ -244,7 +244,13 @@ describe("WorkerJavaScriptBackend", () => { ); const fs = new WorkspaceFilesystem(db); const backend = new WorkerJavaScriptBackend({ loader: throwingLoader("unused") }); - await backend.connect({ db, fs, git: undefined as never, artifacts: undefined as never }); + await backend.connect({ + db, + fs, + git: undefined as never, + artifacts: undefined as never, + runtime: undefined as never, + }); const columns = db.all<{ name: string }>("PRAGMA table_info(workspace_runtime_executions)"); expect(columns.map((column) => column.name)).toEqual( expect.arrayContaining(["created_at", "finished_at"]), @@ -310,6 +316,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = handle.exec({ source: `import task from "./task.js"; export default task;`, @@ -564,6 +571,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "successful-host-call", source: "export default 1" }); let settled = false; @@ -628,6 +636,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "live-stream", source: "export default 1" }); const reader = execution.events.getReader(); @@ -682,6 +691,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "kill-mid-stream", source: "export default 1" }); const reader = execution.events.getReader(); @@ -744,6 +754,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "no-exit", source: "export default 1" }); const events = []; @@ -754,7 +765,7 @@ describe("WorkerJavaScriptBackend", () => { await handle.close(); }); - it("aborts cooperative trusted-module calls at their deadline", async () => { + it("aborts cooperative host module calls at their deadline", async () => { const db = new Database(new SQLiteTestStorage()); initializeSchema(db, () => 0); const fs = new WorkspaceFilesystem(db); @@ -762,11 +773,11 @@ describe("WorkerJavaScriptBackend", () => { let aborted = false; const backend = new WorkerJavaScriptBackend({ maxHostCallMs: 5, - trustedModules: { + modules: { "ws:test": { - call(_method, _args, context) { + run(_args, context) { return new Promise((_resolve, reject) => { - context?.signal.addEventListener("abort", () => { + context.signal.addEventListener("abort", () => { aborted = true; reject(context.signal.reason); }); @@ -783,7 +794,7 @@ describe("WorkerJavaScriptBackend", () => { _input: unknown, host: { call(name: string, args: string): Promise }, ) { - await host.call("trusted/ws:test.call", JSON.stringify(["run"])); + await host.call("host/ws:test.run", JSON.stringify([])); }, }; }, @@ -796,6 +807,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "trusted-timeout", source: "export default 1" }); const events = []; @@ -847,6 +859,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "cancel-host-call", source: "export default 1" }); await started; @@ -989,6 +1002,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); await handle.exec({ id: "subscribers", source: "export default 1" }); await handle.getExec({ id: "subscribers", after: "tail" }); @@ -997,28 +1011,163 @@ describe("WorkerJavaScriptBackend", () => { await handle.close(); }); - it("rejects malformed host trusted-module names", async () => { - const workspace = new Workspace({ - storage: new SQLiteTestStorage(), - backends: [ + it.each([ + ["a host module with a path", { "ws:bad/path": { run: async () => null } }, /simple ws:\*/], + ["a host module outside ws:*", { container: { run: async () => null } }, /simple ws:\*/], + ["source under ws:*", { "ws:lib": "export const x = 1;" }, /only for host modules/], + ["a replacement node:fs", { "node:fs": "export default {};" }, /built in/], + [ + "a replacement node:fs host module", + { "node:fs/promises": { run: async () => null } }, + /built in/, + ], + ["a number", { "ws:test": 42 }, /source text, an object of functions, or a factory/], + ["an object with no functions", { "ws:test": {} }, /must export a function/], + [ + "an object with a non-identifier name", + { "ws:test": { "not-a-name": async () => null } }, + /JavaScript identifier/, + ], + [ + "an object with a default export", + { "ws:test": { default: async () => null } }, + /JavaScript identifier/, + ], + [ + "an object with a then export", + // biome-ignore lint/suspicious/noThenProperty: The case checks that the backend rejects a `then` export. + { "ws:test": { then: async () => null } }, + /JavaScript identifier/, + ], + ["an object with a non-function export", { "ws:test": { run: "nope" } }, /must be a function/], + ])("rejects %s at construction", (_label, modules, message) => { + expect( + () => new WorkerJavaScriptBackend({ loader: throwingLoader("must not load"), - trustedModules: { - "ws:bad/path": { - async call() { - return null; - }, - }, - } as never, + // SAFETY: Each case hands the constructor a shape the types may forbid, to check its runtime guard. + modules: modules as never, }), - ], + ).toThrow(message); + }); + + it("rejects a factory whose functions are not allowed when it connects", async () => { + const db = new Database(new SQLiteTestStorage()); + initializeSchema(db, () => 0); + const backend = new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + modules: { "ws:test": () => ({ "not-a-name": async () => null }) }, }); - await workspace.fs.mkdir("/workspace", { recursive: true }); await expect( - workspace.runtime.exec(`import { call } from "ws:bad/path"; export default call;`, { - backend: "worker-javascript", + backend.connect({ + db, + fs: new WorkspaceFilesystem(db), + git: undefined as never, + artifacts: undefined as never, + runtime: undefined as never, }), - ).rejects.toThrow(/simple reserved ws:\*/); + ).rejects.toThrow(/JavaScript identifier/); + }); + + it("describes its modules for a model", () => { + const backend = new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + access: "read", + modules: { + lib: "export const x = 1;", + "ws:weather": { forecast: () => null, alerts: () => null }, + "ws:described": Object.assign(() => ({ run: () => null }), { + description: "Does a thing.", + }), + "ws:plain": () => ({ run: () => null }), + }, + }); + + expect(backend.description).toContain("ECMAScript module source"); + expect(backend.description).toContain("read-only"); + expect(backend.description).toContain("- `lib`: a bundled library."); + expect(backend.description).toContain("- `ws:weather`: exports `forecast`, `alerts`."); + expect(backend.description).toContain("- `ws:described`: Does a thing."); + expect(backend.description).toContain("- `ws:plain`: a host module."); + }); + + it("builds host modules from the Workspace services when it connects", async () => { + const db = new Database(new SQLiteTestStorage()); + initializeSchema(db, () => 0); + const git = { marker: "git" }; + let seen: unknown; + const backend = new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + modules: { + "ws:test": (host) => { + seen = host.git; + return { run: async () => null }; + }, + }, + }); + await backend.connect({ + db, + fs: new WorkspaceFilesystem(db), + git: git as never, + artifacts: undefined as never, + runtime: undefined as never, + }); + expect(seen).toBe(git); + }); + + it("does not dispatch inherited members of a host module", async () => { + const db = new Database(new SQLiteTestStorage()); + initializeSchema(db, () => 0); + const fs = new WorkspaceFilesystem(db); + await fs.mkdir("/workspace", { recursive: true }); + let response = ""; + const backend = new WorkerJavaScriptBackend({ + modules: { "ws:test": { run: async () => null } }, + loader: { + load() { + return { + getEntrypoint() { + return { + async evaluate( + _input: unknown, + host: { call(name: string, args: string): Promise }, + ) { + response = await host.call("host/ws:test.toString", JSON.stringify([])); + }, + }; + }, + }; + }, + }, + }); + const handle = await backend.connect({ + db, + fs, + git: undefined as never, + artifacts: undefined as never, + runtime: undefined as never, + }); + const execution = await handle.exec({ id: "inherited", source: "export default 1" }); + for await (const _event of execution.events) { + // Drain the run so the host call settles. + } + expect(JSON.parse(response)).toMatchObject({ + error: { message: expect.stringContaining("Unknown Workspace host module call") }, + }); + await handle.close?.(); + }); + + it("does not install ws:git or ws:artifacts unless they are configured", async () => { + const workspace = new Workspace({ + storage: new SQLiteTestStorage(), + backends: [new WorkerJavaScriptBackend({ loader: throwingLoader("must not load") })], + }); + await workspace.fs.mkdir("/workspace", { recursive: true }); + for (const specifier of ["ws:git", "ws:artifacts"]) { + await expect( + workspace.runtime.exec(`import * as m from "${specifier}"; export default () => m;`), + ).rejects.toThrow(/is not configured/); + } }); it("rejects relative imports that collide with internal Loader modules", async () => { @@ -1036,24 +1185,16 @@ describe("WorkerJavaScriptBackend", () => { expect(load).not.toHaveBeenCalled(); }); - it("rejects configured module names that collide with generated modules", async () => { - const load = vi.fn(); - const workspace = new Workspace({ - storage: new SQLiteTestStorage(), - backends: [ - new WorkerJavaScriptBackend({ - loader: { load }, - modules: { - "__workspace_entry__.js": "export default 42", - "node:fs": "export default {};", - }, - }), - ], - }); - await workspace.fs.mkdir("/workspace", { recursive: true }); - await expect( - workspace.runtime.exec("export default 1", { backend: "worker-javascript" }), - ).rejects.toThrow(/reserved module name/); - expect(load).not.toHaveBeenCalled(); - }); + it.each(["__workspace_entry__.js", "workspace-capabilities.js", "nested/lib"])( + "rejects a source module named %s at construction", + (specifier) => { + expect( + () => + new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + modules: { [specifier]: "export default 42" }, + }), + ).toThrow(/reserved module name/); + }, + ); }); diff --git a/packages/computer/src/backends/worker-javascript/worker-javascript.ts b/packages/computer/src/backends/worker-javascript/worker-javascript.ts index 4bd5fcfe..5f211e6e 100644 --- a/packages/computer/src/backends/worker-javascript/worker-javascript.ts +++ b/packages/computer/src/backends/worker-javascript/worker-javascript.ts @@ -4,29 +4,50 @@ import { dynamicWorkerEgress, type WorkspaceEgressPolicy } from "../../runtime/e import type { ModuleExecutionEnvelope, ModuleExecutionInput, + WorkspaceModule, WorkspaceModuleBackend, WorkspaceModuleBackendHandle, WorkspaceModuleBackendHost, + WorkspaceModuleFunctions, WorkspaceRuntimeAccess, WorkspaceRuntimeEvent, WorkspaceRuntimeLoader, WorkspaceRuntimeValue, - WorkspaceTrustedModule, } from "../../runtime/types.js"; import { decodeRuntimeFrames, type RuntimeFrame } from "./frames.js"; -import { buildModuleGraph } from "./module-graph.js"; +import { + assertHostModuleExports, + buildModuleGraph, + type ParsedModules, + parseModules, +} from "./module-graph.js"; export interface WorkerJavaScriptBackendOptions { loader: WorkspaceRuntimeLoader; id?: string; root?: string; access?: WorkspaceRuntimeAccess; - modules?: Record; /** - * Host-owned capability modules installed under reserved ws:* specifiers. - * Caller source may import them, but cannot provide or replace them. + * Modules caller source can import by specifier. + * + * A string value is JavaScript source bundled into the isolate, such as + * a library build. An object of functions, or a factory that builds + * one, is a host module: it runs in the Durable Object under a `ws:*` + * specifier, and each function becomes a named export. + * + * ```ts + * modules: { + * "tar-stream": TAR_STREAM_BUNDLE, + * "ws:container": createContainerModule(), + * "ws:weather": { forecast: ([city]) => lookUpForecast(String(city)) }, + * } + * ``` + * + * `node:fs` and `node:fs/promises` are always installed and cannot be + * replaced. The constructor throws when a specifier is not allowed, and + * connecting throws when a host module's export names are not allowed. */ - trustedModules?: Record<`ws:${string}`, WorkspaceTrustedModule>; + modules?: Record; defaultTimeoutMs?: number; maxTimeoutMs?: number; maxSourceBytes?: number; @@ -56,10 +77,6 @@ export interface WorkerJavaScriptBackendOptions { compatibilityFlags?: string[]; egress?: WorkspaceEgressPolicy; globalOutbound?: Fetcher | null; - /** Allow ws:git operations that can perform host-side network requests. */ - allowGitNetwork?: boolean; - /** Allow ws:artifacts imports from caller-selected remote URLs. */ - allowArtifactNetwork?: boolean; } type ResolvedWorkerJavaScriptBackendOptions = Required< @@ -90,8 +107,9 @@ type ResolvedWorkerJavaScriptBackendOptions = Required< | "compatibilityFlags" > > & - Omit & { + Omit & { egress: WorkspaceEgressPolicy; + modules: ParsedModules; }; interface WorkspaceExecutionContext { @@ -139,6 +157,8 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { readonly protocol = "module" as const; readonly type = "worker-javascript"; readonly callable = true; + /** What this backend tells a model: the source language and every importable module. */ + readonly description: string; readonly id: string; readonly #options: ResolvedWorkerJavaScriptBackendOptions; @@ -198,6 +218,7 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { this.#options = { ...backendOptions, egress: resolvedEgress, + modules: parseModules(options.modules ?? {}), root: options.root ?? "/workspace", access: options.access ?? "read-write", defaultTimeoutMs, @@ -222,6 +243,14 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { compatibilityDate, compatibilityFlags: options.compatibilityFlags ?? ["nodejs_compat"], }; + this.description = [ + "`command` is ECMAScript module source, run in an isolated JavaScript runtime. Relative imports resolve from `cwd` in the workspace.", + ...(resolvedEgress.mode === "none" ? ["Code has no direct network access."] : []), + ...(this.#options.access === "read" ? ["The workspace is read-only here."] : []), + "", + "Modules code can import:", + this.#options.modules.description, + ].join("\n"); } async connect(host: WorkspaceModuleBackendHost): Promise { @@ -232,6 +261,7 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { readonly #options: ResolvedWorkerJavaScriptBackendOptions; readonly #host: WorkspaceModuleBackendHost; + readonly #hostModuleFunctions: ReadonlyMap; readonly #records = new Map(); readonly #pendingIds = new Set(); #closed = false; @@ -243,6 +273,13 @@ class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { constructor(options: ResolvedWorkerJavaScriptBackendOptions, host: WorkspaceModuleBackendHost) { this.#options = options; this.#host = host; + const functions = new Map(); + for (const [specifier, factory] of options.modules.host) { + const built = factory({ git: host.git, artifacts: host.artifacts, runtime: host.runtime }); + assertHostModuleExports(specifier, built); + functions.set(specifier, built); + } + this.#hostModuleFunctions = functions; host.db.run(` CREATE TABLE IF NOT EXISTS workspace_runtime_executions ( backend TEXT NOT NULL, @@ -353,8 +390,8 @@ class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { source: input.source, cwd: input.cwd ?? this.#options.root, capability, - configuredModules: this.#options.modules ?? {}, - trustedModuleNames: Object.keys(this.#options.trustedModules ?? {}), + configuredModules: this.#options.modules.source, + hostModules: this.#hostModuleFunctions, maxSourceBytes: this.#options.maxSourceBytes, maxCapabilityBytes: this.#options.maxCapabilityBytes, }); @@ -389,11 +426,7 @@ class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { this.#records.set(id, record); try { const bridge = new WorkspaceRuntimeBridge(capability, { - git: this.#host.git, - artifacts: this.#host.artifacts, - trustedModules: this.#options.trustedModules, - allowGitNetwork: this.#options.allowGitNetwork ?? false, - allowArtifactNetwork: this.#options.allowArtifactNetwork ?? false, + hostModules: this.#hostModuleFunctions, maxPayloadBytes: this.#options.maxCapabilityBytes, maxCallDurationMs: this.#options.maxHostCallMs, maxConcurrentCalls: this.#options.maxConcurrentCapabilityCalls, diff --git a/packages/computer/src/index.ts b/packages/computer/src/index.ts index 587c2053..b98ab49a 100644 --- a/packages/computer/src/index.ts +++ b/packages/computer/src/index.ts @@ -73,9 +73,15 @@ export type { WorkspaceEgressPolicy } from "./runtime/egress.js"; export type { ModuleExecutionEnvelope, ModuleExecutionInput, + WorkspaceModule, WorkspaceModuleBackend, WorkspaceModuleBackendHandle, WorkspaceModuleBackendHost, + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunction, + WorkspaceModuleFunctions, + WorkspaceModuleHost, WorkspaceRegisteredBackend, WorkspaceRuntimeAccess, WorkspaceRuntimeDisposeOptions, @@ -88,7 +94,6 @@ export type { WorkspaceRuntimeResult, WorkspaceRuntimeStatus, WorkspaceRuntimeValue, - WorkspaceTrustedModule, } from "./runtime/types.js"; export { decodeRuntimeEvents, encodeRuntimeEvent } from "./runtime/wire.js"; export { type RawShellValue, type ShellValue, sh, shellQuote } from "./sh.js"; diff --git a/packages/computer/src/modules/artifacts.ts b/packages/computer/src/modules/artifacts.ts new file mode 100644 index 00000000..e577c5ef --- /dev/null +++ b/packages/computer/src/modules/artifacts.ts @@ -0,0 +1,83 @@ +// `ws:artifacts`: the Workspace's Artifacts client for isolate JavaScript. +// +// import { create, get, list, importArtifact, deleteArtifact } from "ws:artifacts"; +// +// Calls that change Artifacts need a read-write backend. Importing from +// a caller-chosen URL is denied unless the module is created with +// `allowNetwork: true`: the request runs from the host, so the +// isolate's own egress settings do not stop it. + +import type { ArtifactClient } from "../artifacts/index.js"; +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceModuleHost, +} from "../runtime/types.js"; + +/** Options for {@link createArtifactsModule}. */ +export interface ArtifactsModuleOptions { + /** + * Allow `importArtifact` from a caller-chosen remote URL. Defaults to + * `false`. The request runs from the host, so the JavaScript + * backend's egress settings do not apply to it. + */ + readonly allowNetwork?: boolean; +} + +/** + * Build the `ws:artifacts` host module over the Workspace's Artifacts client. + * + * It exports `create`, `get`, `list`, `importArtifact`, and + * `deleteArtifact`, with the same arguments as the matching + * `ArtifactClient` methods. + * + * @param options - Whether remote imports are allowed. + * @returns The module to pass as `modules["ws:artifacts"]`. + */ +export function createArtifactsModule( + options: ArtifactsModuleOptions = {}, +): WorkspaceModuleFactory { + const allowNetwork = options.allowNetwork ?? false; + + // SAFETY for the casts below: the isolate's arguments pass through to the Artifacts client, as they did when ws:artifacts was built in. The client checks its own inputs. + const create = (host: WorkspaceModuleHost): WorkspaceModuleFunctions => ({ + create([name, createOptions], context) { + requireWrite(context, "Artifacts create"); + return host.artifacts.create( + String(name), + createOptions as unknown as Parameters[1], + ); + }, + get([name]) { + return host.artifacts.get(String(name)); + }, + list() { + return host.artifacts.list(); + }, + importArtifact([name, source, importOptions], context) { + requireWrite(context, "Artifacts import"); + if (!allowNetwork) { + throw new Error("Artifacts import requires createArtifactsModule({ allowNetwork: true })."); + } + return host.artifacts.import( + String(name), + source as unknown as Parameters[1], + importOptions as unknown as Parameters[2], + ); + }, + deleteArtifact([name], context) { + requireWrite(context, "Artifacts delete"); + return host.artifacts.delete(String(name)); + }, + }); + return Object.assign(create, { + description: `Git repositories stored in Cloudflare Artifacts: \`create(name)\`, \`get(name)\`, \`list()\`, \`importArtifact(name, source)\`, and \`deleteArtifact(name)\`.${allowNetwork ? "" : " Importing from a remote URL is not allowed."}`, + }); +} + +function requireWrite(context: WorkspaceModuleCallContext, operation: string) { + if (context.access !== "read-write") { + throw new Error(`${operation} requires Workspace write access.`); + } +} diff --git a/packages/computer/src/modules/container.test.ts b/packages/computer/src/modules/container.test.ts new file mode 100644 index 00000000..c6fdd2a4 --- /dev/null +++ b/packages/computer/src/modules/container.test.ts @@ -0,0 +1,208 @@ +import { describe, expect, it } from "vitest"; + +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFunction, + WorkspaceModuleHost, +} from "../runtime/types.js"; +import { createContainerModule } from "./container.js"; + +interface ExecOptions { + readonly backend: string; + readonly encoding: "utf8"; + readonly cwd?: string; + readonly env?: Record; + readonly stdin?: string; + readonly timeoutMs: number; +} + +interface Run { + readonly command: string; + readonly options: ExecOptions; + killed: boolean; +} + +// An in-memory Workspace runtime that records each command and finishes +// it with the given output, or holds it open until it is killed. +function fakeRuntime(output: { + exitCode?: number; + stdout?: string; + stderr?: string; + hang?: boolean; +}) { + const runs: Run[] = []; + const runtime = { + async exec(command: string, options: ExecOptions) { + const run: Run = { command, options, killed: false }; + runs.push(run); + let stop: () => void = () => undefined; + const stopped = new Promise((resolve) => { + stop = resolve; + }); + return { + async result() { + if (output.hang) await stopped; + return { + exitCode: run.killed ? 130 : (output.exitCode ?? 0), + stdout: output.stdout ?? "", + stderr: output.stderr ?? "", + }; + }, + async kill() { + run.killed = true; + stop(); + }, + }; + }, + }; + return { runtime, runs }; +} + +// Build the module's functions the way the backend does when it connects. +function build( + runtime: ReturnType["runtime"], + options?: Parameters[0], +): { readonly exec: WorkspaceModuleFunction } { + // SAFETY: The module only calls runtime.exec, and the fake implements the part of WorkspaceRuntime it uses. + const host = { runtime, git: undefined, artifacts: undefined } as unknown as WorkspaceModuleHost; + const functions = createContainerModule(options)(host); + const exec = functions.exec; + if (!exec) throw new Error("ws:container must export exec"); + return { exec }; +} + +function callContext( + overrides: Partial = {}, +): WorkspaceModuleCallContext { + return { + signal: new AbortController().signal, + deadline: Date.now() + 60_000, + access: "read-write", + resolvePath: async (path) => path, + ...overrides, + }; +} + +describe("createContainerModule", () => { + it("runs the command on the container backend and returns its output", async () => { + const { runtime, runs } = fakeRuntime({ exitCode: 3, stdout: "out", stderr: "err" }); + const container = build(runtime); + + await expect( + container.exec( + ["npm test", { cwd: "/workspace/app", env: { CI: "1" }, stdin: "y\n" }], + callContext(), + ), + ).resolves.toEqual({ exitCode: 3, stdout: "out", stderr: "err" }); + expect(runs).toHaveLength(1); + expect(runs[0]).toMatchObject({ + command: "npm test", + options: { + backend: "container-shell", + encoding: "utf8", + cwd: "/workspace/app", + env: { CI: "1" }, + stdin: "y\n", + }, + }); + }); + + it("uses the configured backend id and omits unset options", async () => { + const { runtime, runs } = fakeRuntime({}); + const container = build(runtime, { backend: "linux" }); + + await container.exec(["ls"], callContext()); + expect(Object.keys(runs[0]?.options ?? {}).sort()).toEqual([ + "backend", + "encoding", + "timeoutMs", + ]); + expect(runs[0]?.options.backend).toBe("linux"); + }); + + it("refuses to run on a read-only backend", async () => { + const { runtime, runs } = fakeRuntime({}); + const container = build(runtime); + + await expect(container.exec(["ls"], callContext({ access: "read" }))).rejects.toThrow( + /write access/, + ); + expect(runs).toHaveLength(0); + }); + + it("caps the timeout at the time left before the host call deadline", async () => { + const { runtime, runs } = fakeRuntime({}); + const container = build(runtime); + + await container.exec( + ["sleep 1", { timeoutMs: 600_000 }], + callContext({ deadline: Date.now() + 5_000 }), + ); + await container.exec(["sleep 1", { timeoutMs: 1_000 }], callContext()); + expect(runs[0]?.options.timeoutMs).toBeLessThanOrEqual(5_000); + expect(runs[1]?.options.timeoutMs).toBe(1_000); + }); + + it("kills the command when the call is aborted", async () => { + const { runtime, runs } = fakeRuntime({ hang: true }); + const container = build(runtime); + const controller = new AbortController(); + + const pending = container.exec(["sleep 100"], callContext({ signal: controller.signal })); + await new Promise((resolve) => setTimeout(resolve, 0)); + controller.abort(new Error("cancelled")); + + await expect(pending).resolves.toMatchObject({ exitCode: 130 }); + expect(runs[0]?.killed).toBe(true); + }); + + it("does not start a command once the call is aborted", async () => { + const { runtime, runs } = fakeRuntime({}); + const container = build(runtime); + const controller = new AbortController(); + controller.abort(new Error("cancelled")); + + await expect( + container.exec(["ls"], callContext({ signal: controller.signal })), + ).rejects.toThrow("cancelled"); + expect(runs).toHaveLength(0); + }); + + it("truncates each stream on UTF-8 boundaries", async () => { + const { runtime } = fakeRuntime({ stdout: "a🙂b", stderr: "🙂🙂" }); + const container = build(runtime, { maxOutputBytes: 5 }); + + await expect(container.exec(["echo"], callContext())).resolves.toEqual({ + exitCode: 0, + stdout: "a🙂\n\n[truncated, 1 more bytes]", + stderr: "🙂\n\n[truncated, 4 more bytes]", + }); + }); + + it.each([ + ["no arguments", [], /takes a command/], + ["too many arguments", ["ls", {}, {}], /takes a command/], + ["an empty command", [" "], /non-empty string/], + ["a non-string command", [["ls"]], /non-empty string/], + ["non-object options", ["ls", "fast"], /options must be an object/], + ["an unknown option", ["ls", { shell: "zsh" }], /unknown option "shell"/], + ["a non-string cwd", ["ls", { cwd: 1 }], /cwd must be a string/], + ["a non-string env value", ["ls", { env: { A: 1 } }], /env "A" must be a string/], + ["a non-positive timeout", ["ls", { timeoutMs: 0 }], /timeoutMs must be a positive number/], + ])("rejects %s without running anything", async (_label, args, message) => { + const { runtime, runs } = fakeRuntime({}); + const container = build(runtime); + + // SAFETY: Each case hands exec arguments that isolate code could send; the cast only widens the test table's inferred type. + await expect(container.exec(args as never, callContext())).rejects.toThrow(message); + expect(runs).toHaveLength(0); + }); + + it("rejects a bad maxOutputBytes at construction", () => { + expect(() => createContainerModule({ maxOutputBytes: 0 })).toThrow(/maxOutputBytes/); + }); + + it("describes itself for a model", () => { + expect(createContainerModule().description).toContain("full Linux container"); + }); +}); diff --git a/packages/computer/src/modules/container.ts b/packages/computer/src/modules/container.ts new file mode 100644 index 00000000..06720b7a --- /dev/null +++ b/packages/computer/src/modules/container.ts @@ -0,0 +1,190 @@ +// `ws:container`: lets isolate JavaScript run shell commands in the +// Workspace's container backend. +// +// Installed on a WorkerJavaScriptBackend, it turns the container into a +// library the JavaScript backend calls, rather than a second backend +// the model has to choose between: +// +// import { exec } from "ws:container"; +// const { exitCode, stdout } = await exec("npm test", { cwd: "/workspace" }); +// +// Each call goes through `workspace.runtime.exec`, so the container +// sees the same files as the isolate: the usual sync bracket pushes +// pending Workspace writes before the command and pulls the +// container's changes after it. + +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceModuleHost, + WorkspaceRuntimeValue, +} from "../runtime/types.js"; +import { truncateText } from "../text-truncation.js"; + +const DEFAULT_BACKEND = "container-shell"; +const DEFAULT_MAX_OUTPUT_BYTES = 64 * 1024; +const EXEC_OPTION_KEYS = new Set(["cwd", "env", "stdin", "timeoutMs"]); + +/** Options for {@link createContainerModule}. */ +export interface ContainerModuleOptions { + /** Id of the container backend. Defaults to `"container-shell"`. */ + readonly backend?: string; + /** + * Largest standard output and standard error returned to the + * isolate, in bytes per stream. Output past it is cut and ends with + * a truncation marker. Defaults to 64 KiB. Keep both streams well + * under the backend's `maxCapabilityBytes`. + */ + readonly maxOutputBytes?: number; +} + +/** + * Build the `ws:container` host module over the Workspace's container + * backend. + * + * It exports `exec(command, { cwd, env, stdin, timeoutMs })`, which + * returns `{ exitCode, stdout, stderr }` once the command finishes. A + * non-zero exit code is a normal result, not an error. Cancelling the + * execution kills the command. + * + * A container command can write to the Workspace and reach the network, + * so `exec` refuses to run on a read-only backend. Egress settings on + * the JavaScript backend do not apply to the container. + * + * @param options - Which backend to use and how much output to return. + * @returns The module to pass as `modules["ws:container"]`. Its + * `description` tells the model how to use it. + * @throws When `maxOutputBytes` is not a positive integer. The host + * configured the module wrongly. + */ +export function createContainerModule( + options: ContainerModuleOptions = {}, +): WorkspaceModuleFactory { + const backend = options.backend ?? DEFAULT_BACKEND; + const maxOutputBytes = options.maxOutputBytes ?? DEFAULT_MAX_OUTPUT_BYTES; + if (!Number.isInteger(maxOutputBytes) || maxOutputBytes <= 0) { + throw new Error("createContainerModule: maxOutputBytes must be a positive integer."); + } + + const create = (host: WorkspaceModuleHost): WorkspaceModuleFunctions => ({ + async exec(args, context) { + if (context.access !== "read-write") { + throw new Error("ws:container exec requires Workspace write access."); + } + const request = parseExecArgs(args); + const timeoutMs = remainingTime(request.timeoutMs, context); + context.signal.throwIfAborted(); + + const handle = await host.runtime.exec(request.command, { + backend, + encoding: "utf8", + timeoutMs, + ...(request.cwd === undefined ? {} : { cwd: request.cwd }), + ...(request.env === undefined ? {} : { env: request.env }), + ...(request.stdin === undefined ? {} : { stdin: request.stdin }), + }); + // Cancelling the isolate execution, or passing the host call + // deadline, stops the command instead of leaving it running. + const kill = () => void handle.kill().catch(() => undefined); + if (context.signal.aborted) kill(); + else context.signal.addEventListener("abort", kill, { once: true }); + try { + const result = await handle.result(); + return { + exitCode: result.exitCode, + stdout: truncateText(result.stdout, maxOutputBytes), + stderr: truncateText(result.stderr, maxOutputBytes), + }; + } finally { + context.signal.removeEventListener("abort", kill); + } + }, + }); + return Object.assign(create, { description: DESCRIPTION }); +} + +const DESCRIPTION = [ + "Runs shell commands in a full Linux container that shares this workspace's files.", + "Use it for npm, node, python, package managers, native binaries, and network access. The container can take a while to start on first use.", + 'Call `const { exitCode, stdout, stderr } = await exec("npm test", { cwd: "/workspace" })`. Options are `cwd`, `env`, `stdin`, and `timeoutMs`.', + "Output comes back when the command finishes, and long output is truncated. A non-zero `exitCode` is returned, not thrown.", +].join(" "); + +interface ExecRequest { + readonly command: string; + readonly cwd: string | undefined; + readonly env: Record | undefined; + readonly stdin: string | undefined; + readonly timeoutMs: number | undefined; +} + +// Arguments come from isolate code. A malformed call throws, and the +// bridge hands that error back to the isolate as a rejected promise. +function parseExecArgs(args: readonly WorkspaceRuntimeValue[]): ExecRequest { + if (args.length === 0 || args.length > 2) { + throw new TypeError("exec(command, options?) takes a command and an optional options object."); + } + const [command, options] = args; + if (typeof command !== "string" || command.trim().length === 0) { + throw new TypeError("exec: command must be a non-empty string."); + } + if (options === undefined || options === null) { + return { command, cwd: undefined, env: undefined, stdin: undefined, timeoutMs: undefined }; + } + if (typeof options !== "object" || Array.isArray(options)) { + throw new TypeError("exec: options must be an object."); + } + for (const key of Object.keys(options)) { + if (!EXEC_OPTION_KEYS.has(key)) { + throw new TypeError( + `exec: unknown option ${JSON.stringify(key)}. Use cwd, env, stdin, or timeoutMs.`, + ); + } + } + return { + command, + cwd: optionalString(options.cwd, "cwd"), + env: optionalEnv(options.env), + stdin: optionalString(options.stdin, "stdin"), + timeoutMs: optionalTimeout(options.timeoutMs), + }; +} + +function optionalString(value: WorkspaceRuntimeValue | undefined, name: string) { + if (value === undefined || value === null) return undefined; + if (typeof value !== "string") throw new TypeError(`exec: ${name} must be a string.`); + return value; +} + +function optionalEnv(value: WorkspaceRuntimeValue | undefined) { + if (value === undefined || value === null) return undefined; + if (typeof value !== "object" || Array.isArray(value)) { + throw new TypeError("exec: env must be an object of strings."); + } + const env: Record = {}; + for (const [key, entry] of Object.entries(value)) { + if (typeof entry !== "string") { + throw new TypeError(`exec: env ${JSON.stringify(key)} must be a string.`); + } + env[key] = entry; + } + return env; +} + +function optionalTimeout(value: WorkspaceRuntimeValue | undefined) { + if (value === undefined || value === null) return undefined; + if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) { + throw new TypeError("exec: timeoutMs must be a positive number."); + } + return value; +} + +// The command must finish before the host call deadline, or the +// isolate stops waiting while the container keeps working. Cap the +// requested timeout at the time left. +function remainingTime(requested: number | undefined, context: WorkspaceModuleCallContext) { + const remaining = context.deadline - Date.now(); + if (remaining <= 0) throw new Error("exec: the host call deadline has already passed."); + return requested === undefined ? remaining : Math.min(requested, remaining); +} diff --git a/packages/computer/src/modules/git.ts b/packages/computer/src/modules/git.ts new file mode 100644 index 00000000..837ce108 --- /dev/null +++ b/packages/computer/src/modules/git.ts @@ -0,0 +1,127 @@ +// `ws:git`: the Workspace's Git client for isolate JavaScript. +// +// import { status, diff, log, clone, cli } from "ws:git"; +// +// Every path the isolate passes is confined to the JavaScript backend's +// root. Commands that change the repository need a read-write backend, +// and commands that reach the network are denied unless the module is +// created with `allowNetwork: true`: they run from the host, so the +// isolate's own egress settings do not stop them. + +import type { GitClient } from "../git/index.js"; +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceModuleHost, + WorkspaceRuntimeValue, +} from "../runtime/types.js"; + +const NETWORK_COMMANDS = new Set(["clone", "fetch", "pull", "push", "ls-remote", "submodule"]); + +/** Options for {@link createGitModule}. */ +export interface GitModuleOptions { + /** + * Allow `clone` and network `cli` commands such as `fetch` and `push`. + * Defaults to `false`. These requests run from the host, so the + * JavaScript backend's egress settings do not apply to them. + */ + readonly allowNetwork?: boolean; +} + +/** + * Build the `ws:git` host module over the Workspace's Git client. + * + * It exports `clone`, `diff`, `status`, `log`, and `cli`. Each takes the + * same options object as the matching `GitClient` method, with `dir` or + * `cwd` resolved against the backend root. + * + * @param options - Whether network commands are allowed. + * @returns The module to pass as `modules["ws:git"]`. + */ +export function createGitModule(options: GitModuleOptions = {}): WorkspaceModuleFactory { + const allowNetwork = options.allowNetwork ?? false; + const requireNetwork = (operation: string) => { + if (!allowNetwork) { + throw new Error(`${operation} requires createGitModule({ allowNetwork: true }).`); + } + }; + + const create = (host: WorkspaceModuleHost): WorkspaceModuleFunctions => ({ + async clone([value], context) { + requireWrite(context, "Git clone"); + requireNetwork("Git clone"); + // SAFETY: The isolate's options object passes through to the Git client, as it did when ws:git was built in. The client checks its own options; only the path is rewritten here. + return host.git.clone( + (await withDir(value, context, true)) as unknown as Parameters[0], + ); + }, + async diff([value], context) { + // SAFETY: As for clone. + return host.git.diff((await withDir(value, context)) as Parameters[0]); + }, + async status([value], context) { + // SAFETY: As for clone. + return host.git.status((await withDir(value, context)) as Parameters[0]); + }, + async log([value], context) { + // SAFETY: As for clone. + return host.git.log((await withDir(value, context)) as Parameters[0]); + }, + async cli([value], context) { + requireWrite(context, "Git CLI"); + // SAFETY: As for clone. + const input = (value ?? {}) as unknown as Parameters[0]; + assertSafeCliArguments(input.argv); + if (input.argv?.some((argument) => NETWORK_COMMANDS.has(argument.toLowerCase()))) { + requireNetwork("Git CLI network command"); + } + return host.git.cli({ + ...input, + cwd: await context.resolvePath(input.cwd ?? ".", { allowMissing: true }), + }); + }, + }); + return Object.assign(create, { + description: `The workspace's Git repository tools: \`status({ dir })\`, \`diff({ dir })\`, \`log({ dir, depth })\`, \`clone({ url, dir })\`, and \`cli({ argv, cwd })\` for any other git subcommand.${allowNetwork ? "" : " Network commands such as clone, fetch, and push are not allowed."}`, + }); +} + +function requireWrite(context: WorkspaceModuleCallContext, operation: string) { + if (context.access !== "read-write") { + throw new Error(`${operation} requires Workspace write access.`); + } +} + +async function withDir( + value: WorkspaceRuntimeValue | undefined, + context: WorkspaceModuleCallContext, + allowMissing = false, +): Promise> { + const options = value !== null && typeof value === "object" && !Array.isArray(value) ? value : {}; + return { + ...options, + dir: await context.resolvePath(typeof options.dir === "string" ? options.dir : ".", { + allowMissing, + }), + }; +} + +// Git path overrides would let a command escape the confined directory. +function assertSafeCliArguments(argv: string[] | undefined) { + if ( + argv?.some( + (argument) => + argument === "-C" || + argument.startsWith("-C") || + argument === "--git-dir" || + argument.startsWith("--git-dir=") || + argument === "--work-tree" || + argument.startsWith("--work-tree="), + ) + ) { + throw new Error( + "Git CLI path overrides are not available inside a confined Workspace runtime.", + ); + } +} diff --git a/packages/computer/src/runtime/bridge.test.ts b/packages/computer/src/runtime/bridge.test.ts index 18f330a0..94d16a4f 100644 --- a/packages/computer/src/runtime/bridge.test.ts +++ b/packages/computer/src/runtime/bridge.test.ts @@ -4,7 +4,7 @@ import { WorkspaceRuntimeBridge } from "./bridge.js"; import type { WorkspaceRuntimeCapability } from "./capability.js"; const encoder = new TextEncoder(); -const args = JSON.stringify(["run", "value"]); +const args = JSON.stringify(["value"]); function bridge(limits: { maxCalls?: number; @@ -13,13 +13,7 @@ function bridge(limits: { }) { return new WorkspaceRuntimeBridge({} as WorkspaceRuntimeCapability, { ...limits, - trustedModules: { - "ws:test": { - async call() { - return "ok"; - }, - }, - }, + hostModules: new Map([["ws:test", { run: async () => "ok" }]]), }); } @@ -30,9 +24,9 @@ async function message(response: Promise) { describe("WorkspaceRuntimeBridge cumulative limits", () => { it("accepts the configured call count and rejects the next call", async () => { const target = bridge({ maxCalls: 2 }); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toContain( + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toContain( "exceeds 2 capability calls", ); }); @@ -40,20 +34,20 @@ describe("WorkspaceRuntimeBridge cumulative limits", () => { it("accepts requests at the cumulative byte boundary and rejects the next request", async () => { const bytes = encoder.encode(args).byteLength; const target = bridge({ maxTotalRequestBytes: bytes * 2 }); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toContain( + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toContain( `requests exceed ${bytes * 2} bytes`, ); }); it("accepts responses at the cumulative byte boundary and rejects the next response", async () => { - const sample = await bridge({}).call("trusted/ws:test.call", args); + const sample = await bridge({}).call("host/ws:test.run", args); const bytes = encoder.encode(sample).byteLength; const target = bridge({ maxTotalResponseBytes: bytes * 2 }); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toContain( + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toContain( `responses exceed ${bytes * 2} bytes`, ); }); diff --git a/packages/computer/src/runtime/bridge.ts b/packages/computer/src/runtime/bridge.ts index 552e28f0..cfe4a6e8 100644 --- a/packages/computer/src/runtime/bridge.ts +++ b/packages/computer/src/runtime/bridge.ts @@ -1,17 +1,11 @@ import { RpcTarget } from "cloudflare:workers"; -import type { ArtifactClient } from "../artifacts/index.js"; -import type { GitClient } from "../git/index.js"; import { assertRuntimeValue, type WorkspaceRuntimeCapability } from "./capability.js"; -import type { WorkspaceTrustedModule } from "./types.js"; +import type { WorkspaceModuleCallContext, WorkspaceModuleFunctions } from "./types.js"; export class WorkspaceRuntimeBridge extends RpcTarget { readonly #capability: WorkspaceRuntimeCapability; - readonly #git: GitClient | undefined; - readonly #artifacts: ArtifactClient | undefined; - readonly #trustedModules: Record; - readonly #allowGitNetwork: boolean; - readonly #allowArtifactNetwork: boolean; + readonly #hostModules: ReadonlyMap; readonly #maxPayloadBytes: number; readonly #maxCallDurationMs: number; readonly #maxConcurrentCalls: number; @@ -31,11 +25,7 @@ export class WorkspaceRuntimeBridge extends RpcTarget { constructor( capability: WorkspaceRuntimeCapability, integrations: { - git?: GitClient; - artifacts?: ArtifactClient; - trustedModules?: Record; - allowGitNetwork?: boolean; - allowArtifactNetwork?: boolean; + hostModules?: ReadonlyMap; maxPayloadBytes?: number; maxCallDurationMs?: number; maxConcurrentCalls?: number; @@ -48,11 +38,7 @@ export class WorkspaceRuntimeBridge extends RpcTarget { ) { super(); this.#capability = capability; - this.#git = integrations.git; - this.#artifacts = integrations.artifacts; - this.#trustedModules = integrations.trustedModules ?? {}; - this.#allowGitNetwork = integrations.allowGitNetwork ?? false; - this.#allowArtifactNetwork = integrations.allowArtifactNetwork ?? false; + this.#hostModules = integrations.hostModules ?? new Map(); this.#maxPayloadBytes = integrations.maxPayloadBytes ?? 1024 * 1024; this.#maxCallDurationMs = integrations.maxCallDurationMs ?? 30_000; this.#maxConcurrentCalls = integrations.maxConcurrentCalls ?? 16; @@ -114,10 +100,14 @@ export class WorkspaceRuntimeBridge extends RpcTarget { const operation = encodeCall(async () => { const encodedArgs = JSON.parse(argsJson) as unknown[]; const args = encodedArgs.map(decodeBridgeValue); - if (name.startsWith("git.")) return this.#callGit(name.slice(4), args); - if (name.startsWith("artifacts.")) return this.#callArtifacts(name.slice(10), args); - if (name.startsWith("trusted/")) { - return this.#callTrusted(name, args, { signal: abort.signal, deadline }); + if (name.startsWith("host/")) { + return this.#callHostModule(name, args, { + signal: abort.signal, + deadline, + access: this.#capability.access, + resolvePath: (path, options) => + this.#capability.resolveConfined(path, options?.allowMissing ?? false), + }); } const operation = name.startsWith("fs.") ? name.slice(3) : name; switch (operation) { @@ -221,140 +211,28 @@ export class WorkspaceRuntimeBridge extends RpcTarget { } } - async #callTrusted( - name: string, - args: unknown[], - context: { signal: AbortSignal; deadline: number }, - ) { - const suffix = ".call"; - const specifier = name.endsWith(suffix) ? name.slice("trusted/".length, -suffix.length) : ""; - const trusted = this.#trustedModules[specifier]; - if (!trusted) { - throw new Error(`Unknown trusted Workspace module call ${JSON.stringify(name)}.`); + // `name` is `host/.`. Function names are + // identifiers and never contain a dot, so the last dot splits them + // from a specifier such as `ws:a.b`. Own-property checks keep + // isolate code from reaching `toString` or other inherited members. + async #callHostModule(name: string, args: unknown[], context: WorkspaceModuleCallContext) { + const target = name.slice("host/".length); + const dot = target.lastIndexOf("."); + const specifier = dot === -1 ? "" : target.slice(0, dot); + const functionName = dot === -1 ? "" : target.slice(dot + 1); + const functions = this.#hostModules.get(specifier); + const fn = + functions !== undefined && Object.hasOwn(functions, functionName) + ? functions[functionName] + : undefined; + if (typeof fn !== "function") { + throw new Error(`Unknown Workspace host module call ${JSON.stringify(name)}.`); } - const method = String(args[0]); - const callArgs = args.slice(1); - assertBridgeValues(callArgs); - const result = await trusted.call(method, callArgs, context); + assertBridgeValues(args); + const result = (await fn(args, context)) ?? null; assertBridgeValues([result]); return result; } - - async #callGit(name: string, args: unknown[]) { - if (!this.#git) throw new Error("Workspace Git is not configured for this execution."); - switch (name) { - case "clone": - this.#requireWrite("Git clone"); - this.#requireGitNetwork("Git clone"); - return this.#git - .clone( - (await this.#gitOptions(args[0], true)) as unknown as Parameters[0], - ) - .then(() => null); - case "diff": - return this.#git.diff( - (await this.#gitOptions(args[0])) as Parameters[0], - ); - case "status": - return this.#git.status( - (await this.#gitOptions(args[0])) as Parameters[0], - ); - case "log": - return this.#git.log((await this.#gitOptions(args[0])) as Parameters[0]); - case "cli": { - this.#requireWrite("Git CLI"); - const input = (args[0] ?? {}) as Parameters[0]; - assertSafeGitCliArguments(input.argv); - if (isGitNetworkCommand(input.argv)) this.#requireGitNetwork("Git CLI network command"); - return this.#git.cli({ - ...input, - cwd: await this.#capability.resolveConfined(input.cwd ?? ".", true), - }); - } - default: - throw new Error(`Unknown Workspace Git operation ${JSON.stringify(name)}.`); - } - } - - #callArtifacts(name: string, args: unknown[]) { - if (!this.#artifacts) - throw new Error("Workspace Artifacts are not configured for this execution."); - switch (name) { - case "create": - this.#requireWrite("Artifacts create"); - return this.#artifacts.create( - String(args[0]), - args[1] as Parameters[1], - ); - case "get": - return this.#artifacts.get(String(args[0])); - case "list": - return this.#artifacts.list(); - case "importArtifact": - this.#requireWrite("Artifacts import"); - if (!this.#allowArtifactNetwork) { - throw new Error( - "Artifacts import requires WorkerJavaScriptBackend allowArtifactNetwork: true.", - ); - } - return this.#artifacts.import( - String(args[0]), - args[1] as Parameters[1], - args[2] as Parameters[2], - ); - case "deleteArtifact": - this.#requireWrite("Artifacts delete"); - return this.#artifacts.delete(String(args[0])); - default: - throw new Error(`Unknown Workspace Artifacts operation ${JSON.stringify(name)}.`); - } - } - - async #gitOptions(value: unknown, allowMissing = false): Promise> { - const options = (value ?? {}) as Record; - return { - ...options, - dir: await this.#capability.resolveConfined( - typeof options.dir === "string" ? options.dir : ".", - allowMissing, - ), - }; - } - - #requireGitNetwork(operation: string) { - if (!this.#allowGitNetwork) { - throw new Error(`${operation} requires WorkerJavaScriptBackend allowGitNetwork: true.`); - } - } - - #requireWrite(operation: string) { - if (this.#capability.access !== "read-write") { - throw new Error(`${operation} requires Workspace write access.`); - } - } -} - -function assertSafeGitCliArguments(argv: string[] | undefined) { - if ( - argv?.some( - (argument) => - argument === "-C" || - argument.startsWith("-C") || - argument === "--git-dir" || - argument.startsWith("--git-dir=") || - argument === "--work-tree" || - argument.startsWith("--work-tree="), - ) - ) { - throw new Error( - "Git CLI path overrides are not available inside a confined Workspace runtime.", - ); - } -} - -function isGitNetworkCommand(argv: string[] | undefined) { - const networkCommands = new Set(["clone", "fetch", "pull", "push", "ls-remote", "submodule"]); - return argv?.some((argument) => networkCommands.has(argument.toLowerCase())) ?? false; } function assertBridgeValues( @@ -369,17 +247,19 @@ function assertBridgeValues( (typeof value === "number" && Number.isFinite(value)) ) return; - if (typeof value !== "object") - throw new Error("Trusted module values must be JSON-compatible."); - if (seen.has(value)) throw new Error("Trusted module values must be acyclic."); + if (typeof value !== "object") throw new Error("Host module values must be JSON-compatible."); + if (seen.has(value)) throw new Error("Host module values must be acyclic."); seen.add(value); if (Array.isArray(value)) for (const item of value) visit(item); else { const prototype = Object.getPrototypeOf(value); if (prototype !== Object.prototype && prototype !== null) { - throw new Error("Trusted module values must contain only plain objects."); + throw new Error("Host module values must contain only plain objects."); + } + // An undefined field is absent, as in JSON. encodeBridgeValue drops it. + for (const item of Object.values(value as Record)) { + if (item !== undefined) visit(item); } - for (const item of Object.values(value as Record)) visit(item); } seen.delete(value); }; @@ -398,7 +278,9 @@ function encodeBridgeValue(value: unknown): unknown { if (Array.isArray(value)) return wrap("array", { items: value.map(encodeBridgeValue) }); if (value && typeof value === "object") { return wrap("object", { - entries: Object.entries(value).map(([key, child]) => [key, encodeBridgeValue(child)]), + entries: Object.entries(value) + .filter(([, child]) => child !== undefined) + .map(([key, child]) => [key, encodeBridgeValue(child)]), }); } return value; diff --git a/packages/computer/src/runtime/runtime.test.ts b/packages/computer/src/runtime/runtime.test.ts index f91ca460..7cdb3a23 100644 --- a/packages/computer/src/runtime/runtime.test.ts +++ b/packages/computer/src/runtime/runtime.test.ts @@ -52,7 +52,7 @@ function replayBackend(events: WorkspaceRuntimeEvent[]): WorkspaceModuleBackendH function runtimeFor(handle: WorkspaceModuleBackendHandle): WorkspaceRuntime { return new WorkspaceRuntime({ - callableBackendIds: new Set(), + backends: new Map(), backendHandle: async () => handle, resolveBackendId: () => "backend", }); @@ -181,7 +181,7 @@ describe("WorkspaceRuntime utf8 encoding", () => { describe("WorkspaceRuntime callable gate", () => { it("rejects structured input for a non-callable backend", async () => { const runtime = new WorkspaceRuntime({ - callableBackendIds: new Set(), + backends: new Map(), backendHandle: async () => moduleHandleStub(), resolveBackendId: () => "worker-shell", }); @@ -194,7 +194,7 @@ describe("WorkspaceRuntime callable gate", () => { it("accepts structured input for a callable module backend", async () => { const handle = moduleHandleStub(); const runtime = new WorkspaceRuntime({ - callableBackendIds: new Set(["worker-javascript"]), + backends: new Map([["worker-javascript", { callable: true }]]), backendHandle: async () => handle, resolveBackendId: () => "worker-javascript", }); diff --git a/packages/computer/src/runtime/runtime.ts b/packages/computer/src/runtime/runtime.ts index ec43d5df..f74965f8 100644 --- a/packages/computer/src/runtime/runtime.ts +++ b/packages/computer/src/runtime/runtime.ts @@ -14,7 +14,8 @@ import type { } from "./types.js"; interface WorkspaceRuntimeRouterOptions { - callableBackendIds: ReadonlySet; + // What each registered backend says about itself. + backends: ReadonlyMap; backendHandle: (id: string) => Promise; resolveBackendId: (id: string | undefined) => string; } @@ -39,7 +40,15 @@ export class WorkspaceRuntime { // this to know whether a backend is callable without the caller // having to declare it a second time. isCallable(id: string): boolean { - return this.#options.callableBackendIds.has(id); + return this.#options.backends.get(id)?.callable === true; + } + + // What the named backend says about itself for a model: its source + // language and, for the JavaScript backend, the modules code can + // import. The exec tool adds it to the backend's entry so a caller + // does not have to repeat it. + describe(id: string): string | undefined { + return this.#options.backends.get(id)?.description; } exec(source: string): Promise>; diff --git a/packages/computer/src/runtime/types.ts b/packages/computer/src/runtime/types.ts index c92a9f64..ae6e4934 100644 --- a/packages/computer/src/runtime/types.ts +++ b/packages/computer/src/runtime/types.ts @@ -4,15 +4,83 @@ import type { ExecEncoding, ExecSyncResult, KillSignal } from "../shell.js"; export type WorkspaceRuntimeAccess = "read" | "read-write"; -export interface WorkspaceTrustedModule { - /** Dispatch a call made through a host-installed reserved ws:* module. */ - call( - method: string, - args: WorkspaceRuntimeValue[], - context?: { signal: AbortSignal; deadline: number }, - ): Promise; +/** Per-call context the backend passes to every host module function. */ +export interface WorkspaceModuleCallContext { + /** Aborts when the call passes its deadline or the execution is cancelled. */ + readonly signal: AbortSignal; + /** Epoch milliseconds after which the caller stops waiting for this call. */ + readonly deadline: number; + /** Access level of the backend running the call. */ + readonly access: WorkspaceRuntimeAccess; + /** + * Resolve a path the isolate passed against the backend's root. + * Rejects paths that escape the root or pass through a symlink. + * + * @param path - An absolute path, or one relative to the backend root. + * @param options - Set `allowMissing` when the path may not exist yet. + * @returns The confined absolute path. + */ + resolvePath(path: string, options?: { readonly allowMissing?: boolean }): Promise; } +/** + * One host function exported by a host module. + * + * `args` holds the arguments the isolate passed, decoded from the wire. + * They come from untrusted code, so parse them before use. The function + * may return a value or a promise of one. The result must be + * JSON-compatible: the bridge checks it at runtime, treats `undefined` + * as `null`, and drops `undefined` object fields, the way + * `JSON.stringify` does. + */ +export type WorkspaceModuleFunction = ( + args: readonly WorkspaceRuntimeValue[], + context: WorkspaceModuleCallContext, +) => unknown; + +/** Named functions a host module exports into the isolate. */ +export type WorkspaceModuleFunctions = Readonly>; + +/** Workspace services a host module factory can build its functions from. */ +export interface WorkspaceModuleHost { + /** The Workspace's Git client. Throws on use when Git is not configured. */ + readonly git: import("../git/index.js").GitClient; + /** The Workspace's Artifacts client. Throws on use when Artifacts is not configured. */ + readonly artifacts: import("../artifacts/index.js").ArtifactClient; + /** The Workspace runtime, for running commands on other backends. */ + readonly runtime: import("./runtime.js").WorkspaceRuntime; +} + +/** + * Builds a host module's functions from the Workspace's services. The + * backend calls it once when it connects to its Workspace. The + * prebuilt modules in `@cloudflare/computer/modules/*` are factories. + */ +export interface WorkspaceModuleFactory { + (host: WorkspaceModuleHost): WorkspaceModuleFunctions; + /** + * What the module does and how to call it, for a model. The + * JavaScript backend adds it to its own description, which the exec + * tool shows. Objects of functions are listed by their export names. + */ + readonly description?: string; +} + +/** + * A module caller source can import. + * + * - A string is JavaScript source bundled into the isolate, with no host access. + * - An object of functions is a host module. Its functions run in the + * Durable Object and each becomes a named export: + * `{ "ws:weather": { forecast } }` lets code write + * `import { forecast } from "ws:weather"`. + * - A factory is a host module that needs the Workspace's Git client, + * Artifacts client, or runtime, such as `createContainerModule()`. + * + * Host modules must use a `ws:*` specifier. + */ +export type WorkspaceModule = string | WorkspaceModuleFunctions | WorkspaceModuleFactory; + export type WorkspaceRuntimeValue = | null | boolean @@ -181,13 +249,19 @@ export interface WorkspaceModuleBackendHandle { close?(): Promise; } -export type WorkspaceModuleBackendHost = import("../backend.js").WorkspaceBackendHost; +/** What a module backend receives when it connects to its Workspace. */ +export type WorkspaceModuleBackendHost = import("../backend.js").WorkspaceBackendHost & { + /** The Workspace runtime, handed to host modules. */ + readonly runtime: import("./runtime.js").WorkspaceRuntime; +}; export interface WorkspaceModuleBackend { readonly protocol: "module"; readonly id: string; readonly type: string; readonly callable?: boolean; + /** What the backend tells a model about itself. Shown by the exec tool. */ + readonly description?: string; connect(host: WorkspaceModuleBackendHost): Promise; } diff --git a/packages/computer/src/text-truncation.ts b/packages/computer/src/text-truncation.ts new file mode 100644 index 00000000..8100d812 --- /dev/null +++ b/packages/computer/src/text-truncation.ts @@ -0,0 +1,37 @@ +const encoder = new TextEncoder(); + +/** + * Cut text to at most `maxBytes` UTF-8 bytes without splitting a + * character, and say how much was left out. + * + * @param value - The text to cut. + * @param maxBytes - The largest number of UTF-8 bytes to keep. + * @returns The text unchanged when it fits, or its longest whole-character + * prefix followed by a `[truncated, N more bytes]` marker. + */ +export function truncateText(value: string, maxBytes: number): string { + const totalBytes = encoder.encode(value).byteLength; + if (totalBytes <= maxBytes) return value; + const { text, bytes } = utf8Prefix(value, maxBytes); + return `${text}\n\n[truncated, ${totalBytes - bytes} more bytes]`; +} + +/** + * The longest whole-character prefix of `value` that fits in `maxBytes` + * UTF-8 bytes. + * + * @param value - The text to cut. + * @param maxBytes - The largest number of UTF-8 bytes to keep. + * @returns The prefix and its size in bytes. + */ +export function utf8Prefix(value: string, maxBytes: number): { text: string; bytes: number } { + let bytes = 0; + let end = 0; + for (const char of value) { + const charBytes = encoder.encode(char).byteLength; + if (bytes + charBytes > maxBytes) break; + bytes += charBytes; + end += char.length; + } + return { text: value.slice(0, end), bytes }; +} diff --git a/packages/computer/src/tools/ai.test.ts b/packages/computer/src/tools/ai.test.ts index 5d283781..93506a95 100644 --- a/packages/computer/src/tools/ai.test.ts +++ b/packages/computer/src/tools/ai.test.ts @@ -1,5 +1,8 @@ import { SQLiteTestStorage } from "@cloudflare/dofs/testing"; import { describe, expect, it } from "vitest"; +import { z } from "zod"; +import { WorkerJavaScriptBackend } from "../backends/worker-javascript/worker-javascript.js"; +import { createContainerModule } from "../modules/container.js"; import type { WorkspaceRuntimeExecHandle, WorkspaceRuntimeResult } from "../runtime/types.js"; import { Workspace } from "../workspace.js"; import { @@ -85,6 +88,17 @@ function toolDescription(tool: unknown): string { return description; } +function inputSchema(tool: unknown): z.ZodType { + const schema = (tool as { inputSchema?: unknown }).inputSchema; + if (!(schema instanceof z.ZodType)) throw new Error("tool has no zod input schema"); + return schema; +} + +function inputProperties(tool: unknown): string[] { + const json = z.toJSONSchema(inputSchema(tool)) as { properties?: Record }; + return Object.keys(json.properties ?? {}).sort(); +} + function makeWorkspace(): Workspace { return new Workspace({ storage: new SQLiteTestStorage(), now: () => 1_700_000_000_000 }); } @@ -1697,7 +1711,167 @@ describe("createAITools callable exec", () => { }, }); - expect(toolDescription(tools.exec)).toContain("callable"); + expect(toolDescription(tools.exec)).toContain("Run code in the workspace"); + expect(toolDescription(tools.exec)).toContain("`result` field"); + expect(toolDescription(tools.exec)).toContain("JavaScript module runtime"); + }); + + it("adds what the backend says about itself after the caller's description", () => { + const workspace = { + runtime: { + async exec() { + throw new Error("not used"); + }, + isCallable: () => true, + describe: (id: string) => + id === "js" ? "Modules: `ws:weather` exports `forecast`." : undefined, + }, + }; + const withBoth = createAITools({ + workspace, + shell: { backends: { js: { description: "Use for data work." } } }, + }); + const withBackendOnly = createAITools({ workspace, shell: { backends: { js: {} } } }); + + expect(toolDescription(withBoth.exec)).toContain( + "Use for data work.\n\nModules: `ws:weather` exports `forecast`.", + ); + expect(toolDescription(withBackendOnly.exec)).toContain("`ws:weather` exports `forecast`"); + }); + + it("requires a description for a backend that does not describe itself", () => { + const workspace = { + runtime: { + async exec() { + throw new Error("not used"); + }, + }, + }; + + expect(() => createAITools({ workspace, shell: { backends: { shell: {} } } })).toThrow( + /does not describe itself/, + ); + }); +}); + +describe("createAITools exec against a real JavaScript backend", () => { + it("lists the backend's modules in the tool description", () => { + const workspace = new Workspace({ + storage: new SQLiteTestStorage(), + backends: [ + new WorkerJavaScriptBackend({ + loader: { load: () => ({ getEntrypoint: () => ({}) }) }, + modules: { + "tar-stream": "export default {};", + "ws:weather": { forecast: ([city]) => ({ city, sky: "clear" }) }, + "ws:container": createContainerModule(), + }, + }), + ], + }); + const tools = createAITools({ + workspace, + shell: { backends: { "worker-javascript": {} } }, + }); + const description = toolDescription(tools.exec); + + expect(description).toContain("`node:fs/promises`"); + expect(description).toContain("- `tar-stream`: a bundled library."); + expect(description).toContain("- `ws:weather`: exports `forecast`."); + expect(description).toContain( + "- `ws:container`: Runs shell commands in a full Linux container", + ); + expect(description).toContain("no direct network access"); + expect(inputProperties(tools.exec)).toEqual(["command", "cwd", "env", "input"]); + }); +}); + +describe("createAITools exec with one backend", () => { + function recordingWorkspace(callable: boolean) { + const calls: Array<{ command: string; backend: string | undefined; input: unknown }> = []; + const workspace = { + runtime: { + async exec( + command: string, + options: { encoding: "utf8"; backend?: string; input?: unknown }, + ) { + calls.push({ command, backend: options.backend, input: options.input }); + return { result: async () => ({ exitCode: 0, stdout: "", stderr: "", value: 1 }) }; + }, + isCallable: (id: string) => callable && id === "worker-javascript", + }, + }; + return { calls, workspace }; + } + + it("has no backend argument and runs on the only backend without defaultBackend", async () => { + const { calls, workspace } = recordingWorkspace(true); + const tools = createAITools({ + workspace, + shell: { backends: { "worker-javascript": { description: "Isolated JavaScript." } } }, + }); + + expect(inputProperties(tools.exec)).toEqual(["command", "cwd", "env", "input"]); + const parsed = inputSchema(tools.exec).parse({ + command: "export default () => 1", + backend: "container", + }); + expect(parsed).toEqual({ command: "export default () => 1" }); + await executeTool(tools.exec, parsed); + expect(calls).toEqual([ + { command: "export default () => 1", backend: "worker-javascript", input: undefined }, + ]); + }); + + it("does not mention backends in the description", () => { + const { workspace } = recordingWorkspace(true); + const tools = createAITools({ + workspace, + shell: { backends: { "worker-javascript": { description: "Isolated JavaScript." } } }, + }); + + expect(toolDescription(tools.exec)).not.toMatch(/backend/i); + }); + + it("drops input for a single shell backend", () => { + const { workspace } = recordingWorkspace(false); + const tools = createAITools({ + workspace, + shell: { backends: { shell: { description: "Fast shell." } } }, + }); + + expect(inputProperties(tools.exec)).toEqual(["command", "cwd", "env"]); + expect(toolDescription(tools.exec)).toContain("Run a shell command"); + expect(toolDescription(tools.exec)).not.toMatch(/backend/i); + }); + + it("keeps the backend argument when more than one backend is configured", () => { + const { workspace } = recordingWorkspace(true); + const tools = createAITools({ + workspace, + shell: { + defaultBackend: "worker-javascript", + backends: { + "worker-javascript": { description: "Isolated JavaScript." }, + container: { description: "Full Linux." }, + }, + }, + }); + + expect(inputProperties(tools.exec)).toEqual(["backend", "command", "cwd", "env", "input"]); + }); + + it("requires defaultBackend when more than one backend is configured", () => { + const { workspace } = recordingWorkspace(false); + + expect(() => + createAITools({ + workspace, + shell: { + backends: { shell: { description: "Fast shell." }, container: { description: "Linux." } }, + }, + }), + ).toThrow(/defaultBackend/); }); }); diff --git a/packages/computer/src/tools/exec.ts b/packages/computer/src/tools/exec.ts index 5a58fb36..620d8e56 100644 --- a/packages/computer/src/tools/exec.ts +++ b/packages/computer/src/tools/exec.ts @@ -3,6 +3,7 @@ import { z } from "zod"; import { notCallableMessage } from "../runtime/runtime.js"; import type { WorkspaceRuntimeValue } from "../runtime/types.js"; +import { truncateText, utf8Prefix } from "../text-truncation.js"; // A finite JSON value: what a callable backend accepts as `input` and // returns as `result`. Declared as a concrete recursive schema rather @@ -64,17 +65,29 @@ export interface ExecWorkspaceLike { // are callable; the runtime derives it from each backend's // `callable` flag. Omit when no backend is callable. isCallable?(id: string): boolean; + // What a backend says about itself for a model, such as the + // language it runs and the modules that code can import. The tool + // shows it after the caller's own description. + describe?(id: string): string | undefined; }; } export interface ExecBackendDescription { - description: string; + // Guidance for the model about this backend, shown before whatever + // the backend says about itself. Required only when the backend does + // not describe itself. + description?: string; } export interface ExecToolOptions { workspace: ExecWorkspaceLike; + // Backends the model may run on. With exactly one, the tool has no + // `backend` argument and always runs there, so the model never has + // to reason about backends. backends: Record; - defaultBackend: string; + // Backend used when the model omits `backend`. Required when more + // than one backend is configured; with one it defaults to that one. + defaultBackend?: string; // Per-snapshot display cap for each of stdout and stderr, in bytes. // Output past it is shown as a truncation marker. Defaults to 64 KiB. maxBytes?: number; @@ -110,16 +123,15 @@ export type ExecToolOutput = } | { command: string; cwd: string | null; backend: string; error: string }; -export function createExecTool(options: ExecToolOptions): Tool< - { - command: string; - cwd?: string; - backend?: string; - env?: Record; - input?: WorkspaceRuntimeValue; - }, - ExecToolOutput -> { +type ExecToolInput = { + command: string; + cwd?: string; + backend?: string; + env?: Record; + input?: WorkspaceRuntimeValue; +}; + +export function createExecTool(options: ExecToolOptions): Tool { const maxBytes = options.maxBytes ?? DEFAULT_MAX_BYTES; const streamMaxBytes = options.streamMaxBytes ?? DEFAULT_STREAM_MAX_BYTES; const now = options.now ?? Date.now; @@ -127,74 +139,66 @@ export function createExecTool(options: ExecToolOptions): Tool< if (backendIds.length === 0) { throw new Error("createExecTool: pass at least one backend in `backends`"); } - if (!backendIds.includes(options.defaultBackend)) { + const single = backendIds.length === 1; + const defaultBackend = options.defaultBackend ?? (single ? backendIds[0] : undefined); + if (defaultBackend === undefined || !backendIds.includes(defaultBackend)) { throw new Error( - `createExecTool: defaultBackend ${JSON.stringify(options.defaultBackend)} is not one of ${backendIds.map((id) => JSON.stringify(id)).join(", ")}`, + `createExecTool: pass a defaultBackend that is one of ${backendIds.map((id) => JSON.stringify(id)).join(", ")}`, ); } - const isCallable = options.workspace.runtime.isCallable?.bind(options.workspace.runtime); - const callableBackendIds = new Set(backendIds.filter((id) => isCallable?.(id) === true)); - const backendGuidance = backendIds - .map((id) => { - const suffix = callableBackendIds.has(id) ? " (callable)" : ""; - return `- ${JSON.stringify(id)}${suffix}: ${options.backends[id].description}`; - }) - .join("\n"); - const callableGuidance = - callableBackendIds.size > 0 - ? [ - "", - `Callable backends (${[...callableBackendIds].map((id) => JSON.stringify(id)).join(", ")}) run \`command\` as module source rather than a shell command. Pass \`input\` to hand the module a structured value, and read the module's returned value back from the \`result\` field. Other backends reject \`input\`.`, - ].join("\n") - : ""; - const description = [ - "Run a shell command in the workspace. The workspace exposes multiple backends, each with different capabilities.", - "Pick the cheapest backend that can run the command; fall back to a heavier one only when the lighter backend's command set doesn't cover what you need.", - "", - "Backends:", - backendGuidance, - "", - `Default backend: ${JSON.stringify(options.defaultBackend)}. Try this first for any command you're not sure about; if it fails with a "command not found" or a similar capability error, retry on a backend whose description covers the missing tool.`, - "Use for builds, test runs, typechecks, formatters, and git plumbing. Prefer the dedicated read, write, and edit tools for file operations. Long output is truncated to keep tool replies small.", - callableGuidance, - ].join("\n"); - - const backendSchema = z - .enum(backendIds as [string, ...string[]]) - .optional() - .describe( - [ - "Which backend to run on. Omit to use the default", - `(${JSON.stringify(options.defaultBackend)}). Set explicitly when the`, - "default backend is not capable of running the command. If a command fails because the backend lacks that tool, retry on a backend whose description covers it.", - ].join(" "), - ); + const runtime = options.workspace.runtime; + const backends = backendIds.map((id) => { + const text = [options.backends[id]?.description, runtime.describe?.(id)] + .filter((part) => part !== undefined && part !== "") + .join("\n\n"); + if (text === "") { + throw new Error( + `createExecTool: backend ${JSON.stringify(id)} does not describe itself; pass a description`, + ); + } + return { id, text, callable: runtime.isCallable?.(id) === true }; + }); + const callableBackendIds = new Set(backends.filter((b) => b.callable).map((b) => b.id)); + const description = describeTool(backends, defaultBackend); + // Offer only the fields that can work: `backend` when there is a + // choice, `input` when some backend accepts it. + const shape: Record = { + command: z.string().describe(commandHint(backends)), + cwd: z.string().optional().describe("Working directory. Defaults to the workspace root."), + env: z + .record(z.string(), z.string()) + .optional() + .describe( + "Environment variables for this run only. Values override the base environment without affecting later runs.", + ), + }; + if (!single) { + shape.backend = z + // SAFETY: createExecTool checked that backendIds has at least one entry. + .enum(backendIds as [string, ...string[]]) + .optional() + .describe( + `Which backend to run on. Omit to use the default (${JSON.stringify(defaultBackend)}). If a command fails because the backend lacks that tool, retry on a backend whose description covers it.`, + ); + } + if (callableBackendIds.size > 0) { + shape.input = jsonValueSchema + .optional() + .describe( + single + ? "Structured value handed to the module." + : "Structured value handed to a callable backend's module. Other backends reject it.", + ); + } + // SAFETY: Every field in `shape` has the type ExecToolInput gives it, and the fields left out are optional there. + const inputSchema = z.object(shape) as unknown as z.ZodType; return tool({ description, - inputSchema: z.object({ - command: z - .string() - .describe( - "Shell command, e.g. 'npm test' or 'git diff HEAD'. For a callable backend this is the module source to run.", - ), - cwd: z.string().optional().describe("Working directory. Defaults to the workspace root."), - backend: backendSchema, - env: z - .record(z.string(), z.string()) - .optional() - .describe( - "Environment variables for this run only. Values override the backend's base environment without affecting later runs.", - ), - input: jsonValueSchema - .optional() - .describe( - "Structured value handed to a callable backend's module. Only callable backends accept it; other backends reject it.", - ), - }), + inputSchema, execute: async function* ({ command, cwd, backend, env, input }, { abortSignal }) { - const selectedBackend = backend ?? options.defaultBackend; + const selectedBackend = backend ?? defaultBackend; const base = { command, cwd: cwd ?? null, backend: selectedBackend }; if (input !== undefined && !callableBackendIds.has(selectedBackend)) { yield { ...base, error: notCallableMessage(selectedBackend) }; @@ -285,8 +289,8 @@ export function createExecTool(options: ExecToolOptions): Tool< yield { ...base, exitCode: result.exitCode, - stdout: truncate(result.stdout, maxBytes), - stderr: truncate(result.stderr, maxBytes), + stdout: truncateText(result.stdout, maxBytes), + stderr: truncateText(result.stderr, maxBytes), ...(result.value === undefined ? {} : { result: result.value }), }; } catch (err) { @@ -297,6 +301,62 @@ export function createExecTool(options: ExecToolOptions): Tool< }); } +const FILE_TOOLS_HINT = + "Prefer the dedicated read, write, and edit tools for file operations. Long output is truncated to keep tool replies small."; +const SHELL_HINT = "Use for builds, test runs, typechecks, formatters, and git plumbing."; +const CALLABLE_HINT = + "Pass `input` to hand the module a structured value, and read its return value back from the `result` field."; + +interface DescribedBackend { + readonly id: string; + readonly text: string; + readonly callable: boolean; +} + +// With one backend the description is about what it does. With several +// it lists them and explains how to choose. +function describeTool(backends: readonly DescribedBackend[], defaultBackend: string): string { + const [only, ...others] = backends; + if (only !== undefined && others.length === 0) { + return only.callable + ? ["Run code in the workspace.", "", only.text, "", CALLABLE_HINT, FILE_TOOLS_HINT].join("\n") + : [ + "Run a shell command in the workspace.", + "", + only.text, + "", + `${SHELL_HINT} ${FILE_TOOLS_HINT}`, + ].join("\n"); + } + const callable = backends.filter((backend) => backend.callable).map((b) => JSON.stringify(b.id)); + return [ + "Run a shell command in the workspace. The workspace exposes multiple backends, each with different capabilities.", + "Pick the cheapest backend that can run the command; fall back to a heavier one only when the lighter backend's command set doesn't cover what you need.", + "", + "Backends:", + ...backends.map( + (b) => `- ${JSON.stringify(b.id)}${b.callable ? " (callable)" : ""}: ${b.text}`, + ), + "", + `Default backend: ${JSON.stringify(defaultBackend)}. Try this first for any command you're not sure about; if it fails with a "command not found" or a similar capability error, retry on a backend whose description covers the missing tool.`, + `${SHELL_HINT} ${FILE_TOOLS_HINT}`, + ...(callable.length === 0 + ? [] + : [ + "", + `Callable backends (${callable.join(", ")}) run \`command\` as module source rather than a shell command. ${CALLABLE_HINT} Other backends reject \`input\`.`, + ]), + ].join("\n"); +} + +function commandHint(backends: readonly DescribedBackend[]): string { + if (backends.every((backend) => backend.callable)) return "Module source to run."; + if (backends.every((backend) => !backend.callable)) { + return "Shell command, e.g. 'npm test' or 'git diff HEAD'."; + } + return "Shell command, e.g. 'npm test' or 'git diff HEAD'. For a callable backend this is the module source to run."; +} + function errorMessage(err: unknown): string { return err instanceof Error ? err.message : String(err); } @@ -329,47 +389,16 @@ class StreamBuffer { } // The chunk crosses the cap: keep the largest whole-character // prefix that fits, then stop growing the head. - let used = this.#headBytes; - let end = 0; - for (const char of chunk) { - const charBytes = encoder.encode(char).byteLength; - if (used + charBytes > this.#cap) break; - used += charBytes; - end += char.length; - } - this.#head += chunk.slice(0, end); - this.#headBytes = used; + const prefix = utf8Prefix(chunk, this.#cap - this.#headBytes); + this.#head += prefix.text; + this.#headBytes += prefix.bytes; } render(maxBytes: number): string { if (this.#totalBytes <= maxBytes && this.#totalBytes === this.#headBytes) { return this.#head; } - let used = 0; - let end = 0; - for (const char of this.#head) { - const charBytes = encoder.encode(char).byteLength; - if (used + charBytes > maxBytes) break; - used += charBytes; - end += char.length; - } - return `${this.#head.slice(0, end)}\n\n[truncated, ${this.#totalBytes - used} more bytes]`; + const shown = utf8Prefix(this.#head, maxBytes); + return `${shown.text}\n\n[truncated, ${this.#totalBytes - shown.bytes} more bytes]`; } } - -function truncate(value: string, maxBytes: number): string { - if (!value) return value; - const totalBytes = encoder.encode(value).byteLength; - if (totalBytes <= maxBytes) return value; - - let usedBytes = 0; - let endOffset = 0; - for (const char of value) { - const charBytes = encoder.encode(char).byteLength; - if (usedBytes + charBytes > maxBytes) break; - usedBytes += charBytes; - endOffset += char.length; - } - - return `${value.slice(0, endOffset)}\n\n[truncated, ${totalBytes - usedBytes} more bytes]`; -} diff --git a/packages/computer/src/workspace.ts b/packages/computer/src/workspace.ts index 25f797a2..d8d4b325 100644 --- a/packages/computer/src/workspace.ts +++ b/packages/computer/src/workspace.ts @@ -219,8 +219,7 @@ export class Workspace { readonly #backends: WorkspaceBackend[]; readonly #backendsById: Map; readonly #moduleBackendsById: Map; - readonly #registeredBackendIds: Set; - readonly #callableBackendIds: Set; + readonly #registeredBackends: Map; readonly #defaultBackendId: string | undefined; readonly #observer: WorkspaceObserver; readonly #syncLogger: SyncLogger; @@ -305,19 +304,16 @@ export class Workspace { this.#moduleBackendsById = new Map( registered.filter(isModuleBackend).map((backend) => [backend.id, backend]), ); - this.#registeredBackendIds = new Set(); - this.#callableBackendIds = new Set( - registered.filter((backend) => backend.callable === true).map((backend) => backend.id), - ); + this.#registeredBackends = new Map(); for (const backend of registered) { - if (this.#registeredBackendIds.has(backend.id)) { + if (this.#registeredBackends.has(backend.id)) { throw new Error( `Workspace: duplicate backend id ${JSON.stringify(backend.id)}. ` + "Pass an explicit `id` on each backend's constructor options to " + "distinguish them.", ); } - this.#registeredBackendIds.add(backend.id); + this.#registeredBackends.set(backend.id, backend); } this.#defaultBackendId = registered[0]?.id; this.#observer = options.observer ?? noopObserver; @@ -429,7 +425,7 @@ export class Workspace { get runtime(): WorkspaceRuntime { if (!this.#runtime) { this.#runtime = new WorkspaceRuntime({ - callableBackendIds: this.#callableBackendIds, + backends: this.#registeredBackends, backendHandle: (id) => this.#backendHandleFor(id), resolveBackendId: (id) => this.#resolveBackendId(id) ?? "", }); @@ -861,13 +857,13 @@ export class Workspace { // workspace; throws on an unknown id. Omitted ids fall through // to the first backend in the list (the default). #resolveBackendId(id: string | undefined): string | undefined { - if (this.#registeredBackendIds.size === 0) return undefined; + if (this.#registeredBackends.size === 0) return undefined; const target = id ?? this.#defaultBackendId; if (target === undefined) return undefined; - if (!this.#registeredBackendIds.has(target)) { + if (!this.#registeredBackends.has(target)) { throw new Error( `Workspace: no backend with id ${JSON.stringify(target)}. ` + - `Configured backends: ${[...this.#registeredBackendIds].map((key) => JSON.stringify(key)).join(", ") || ""}.`, + `Configured backends: ${[...this.#registeredBackends.keys()].map((key) => JSON.stringify(key)).join(", ") || ""}.`, ); } return target; @@ -998,6 +994,7 @@ export class Workspace { fs: this.#fs, git: this.#gitFactory ? this.git : DISABLED_GIT_CLIENT, artifacts: this.#artifacts, + runtime: this.runtime, }), ) .then(async (handle) => { diff --git a/packages/computer/tests/script-runner-worker.ts b/packages/computer/tests/script-runner-worker.ts index 244b548b..86fac5fc 100644 --- a/packages/computer/tests/script-runner-worker.ts +++ b/packages/computer/tests/script-runner-worker.ts @@ -1,4 +1,6 @@ import { DurableObject, RpcTarget, WorkerEntrypoint } from "cloudflare:workers"; +import type { ShellRPC, SyncRPC } from "@cloudflare/computer-rpc"; +import type { WorkspaceBackend } from "../src/backend.js"; import { WorkerJavaScriptBackend } from "../src/backends/worker-javascript/index.js"; import { createGitClient } from "../src/git/index.js"; import type { @@ -7,12 +9,55 @@ import type { WorkspaceStub, } from "../src/index.js"; import { Workspace } from "../src/index.js"; +import { createArtifactsModule } from "../src/modules/artifacts.js"; +import { createContainerModule } from "../src/modules/container.js"; +import { createGitModule } from "../src/modules/git.js"; export interface Env { HOST: DurableObjectNamespace; LOADER: WorkerLoader; } +// A command backend that stands in for the container. It echoes the +// command, working directory, one environment variable, and standard +// input, and exits with the length of the command. +function fakeContainerBackend(): WorkspaceBackend { + const encoder = new TextEncoder(); + const shell: ShellRPC = { + async exec(input) { + const id = input.id ?? crypto.randomUUID(); + const stdin = input.stdin ? new TextDecoder().decode(input.stdin) : ""; + const stdout = `ran ${input.source} in ${input.cwd ?? "?"} with ${input.env?.WHO ?? "-"} and ${stdin || "-"}\n`; + return { + id, + events: new ReadableStream({ + start(controller) { + controller.enqueue({ id, seq: 1, name: "stdout", value: encoder.encode(stdout) }); + controller.enqueue({ id, seq: 2, name: "stderr", value: encoder.encode("warn\n") }); + controller.enqueue({ id, seq: 3, name: "exit", code: input.source.length % 256 }); + controller.close(); + }, + }), + }; + }, + getExec: () => Promise.reject(new Error("not used")), + killExec: () => Promise.resolve(), + disposeExec: () => Promise.resolve(), + }; + // SAFETY: The fake backend declares sync "none", so the Workspace never calls these methods. + const sync = new Proxy( + {}, + { get: () => () => Promise.reject(new Error("sync: none")) }, + ) as SyncRPC; + return { + id: "container-shell", + type: "fake-container", + async connect() { + return { rpc: { sync, shell }, sync: "none", close: async () => {} }; + }, + }; +} + export class HostDO extends DurableObject { readonly #workspace: Workspace; @@ -30,27 +75,43 @@ export class HostDO extends DurableObject { maxConcurrentCapabilityCalls: 2, modules: { "math-kit": "export const double = (value) => value * 2;", - }, - trustedModules: { + "ws:git": createGitModule(), + "ws:artifacts": createArtifactsModule(), + "ws:container": createContainerModule(), "ws:test-host": { - async call(method, args) { - if (method === "invalid-result") return new Date() as never; - if (method === "large-error") throw new Error("x".repeat(5000)); - if (method === "slow") { - await new Promise((resolve) => setTimeout(resolve, 20)); - return null; - } - if (method === "marker") { - return { - __workspace_codec__: { version: 1, type: "bytes", data: [1] }, - keep: true, - }; - } - return { method, args }; + async echo(args) { + return { args: [...args] }; + }, + async sum(args) { + return args.reduce( + (total, value) => total + (typeof value === "number" ? value : 0), + 0, + ); + }, + async delete(args) { + return { deleted: args[0] ?? null }; + }, + async invalidResult() { + // SAFETY: The test hands the bridge a non-JSON value on purpose to check that it rejects it. + return new Date() as never; + }, + async largeError() { + throw new Error("x".repeat(5000)); + }, + async slow() { + await new Promise((resolve) => setTimeout(resolve, 20)); + return null; + }, + async marker() { + return { + __workspace_codec__: { version: 1, type: "bytes", data: [1] }, + keep: true, + }; }, }, }, }), + fakeContainerBackend(), ], }); } diff --git a/packages/computer/tests/script-runner.test.ts b/packages/computer/tests/script-runner.test.ts index 61c8b63f..5fea0805 100644 --- a/packages/computer/tests/script-runner.test.ts +++ b/packages/computer/tests/script-runner.test.ts @@ -49,14 +49,14 @@ describe("WorkspaceRuntime", () => { }); }); - it("executes an ES module with configured and trusted modules", async () => { + it("executes an ES module with source and host modules", async () => { const response = await runtime({ source: ` import { double } from "math-kit"; import fs from "node:fs/promises"; import { promises as nodeFs } from "node:fs"; import * as git from "ws:git"; - import { call } from "ws:test-host"; + import { echo, sum, delete as remove } from "ws:test-host"; export default async function main(input) { const value = double(input.value); await fs.writeFile("/workspace/runtime-result.txt", String(value)); @@ -65,7 +65,9 @@ describe("WorkspaceRuntime", () => { value, persisted: await fs.readFile("/workspace/runtime-result.txt", "utf8"), gitExitCode: initialized.exitCode, - trusted: await call("echo", input.value), + trusted: await echo(input.value, "second"), + summed: await sum(1, 2, 3), + removed: await remove("/workspace/gone.txt"), nodeFs: { isFile: (await nodeFs.stat("/workspace/runtime-result.txt")).isFile(), entries: await nodeFs.readdir("/workspace"), @@ -86,7 +88,9 @@ describe("WorkspaceRuntime", () => { value: 42, persisted: "42", gitExitCode: 0, - trusted: { method: "echo", args: [21] }, + trusted: { args: [21, "second"] }, + summed: 6, + removed: { deleted: "/workspace/gone.txt" }, nodeFs: { isFile: true, entries: expect.arrayContaining(["runtime-result.txt"]), @@ -100,14 +104,14 @@ describe("WorkspaceRuntime", () => { const response = await runtime({ source: ` import fs from "node:fs/promises"; - import { call } from "ws:test-host"; + import { marker } from "ws:test-host"; export default async () => { await fs.writeFile("/workspace/bytes.bin", new Uint8Array([0, 127, 255])); const value = await fs.readFile("/workspace/bytes.bin"); return { isBytes: value instanceof Uint8Array, bytes: Array.from(value), - marker: await call("marker"), + marker: await marker(), }; }; `, @@ -230,11 +234,11 @@ describe("WorkspaceRuntime", () => { expect(payload.result.stdout).toContain("stdio truncated"); }); - it("bounds oversized trusted-module error responses", async () => { + it("bounds oversized host module error responses", async () => { const response = await runtime({ source: ` - import { call } from "ws:test-host"; - export default () => call("large-error"); + import { largeError } from "ws:test-host"; + export default () => largeError(); `, cwd: "/workspace", }); @@ -261,9 +265,9 @@ describe("WorkspaceRuntime", () => { it("bounds concurrent host capability calls", async () => { const response = await runtime({ source: ` - import { call } from "ws:test-host"; + import { slow } from "ws:test-host"; export default async () => { - const settled = await Promise.allSettled([call("slow"), call("slow"), call("slow")]); + const settled = await Promise.allSettled([slow(), slow(), slow()]); return settled.map((item) => item.status); }; `, @@ -274,11 +278,11 @@ describe("WorkspaceRuntime", () => { expect(JSON.parse(text).result.value).toEqual(["fulfilled", "fulfilled", "rejected"]); }); - it("rejects non-plain results from host trusted modules", async () => { + it("rejects non-plain results from host modules", async () => { const response = await runtime({ source: ` - import { call } from "ws:test-host"; - export default () => call("invalid-result"); + import { invalidResult } from "ws:test-host"; + export default () => invalidResult(); `, cwd: "/workspace", }); @@ -292,6 +296,85 @@ describe("WorkspaceRuntime", () => { }); }); + it("exposes only the functions a host module declares", async () => { + const response = await runtime({ + source: ` + import * as host from "ws:test-host"; + export default () => Object.keys(host).sort(); + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text).result.value).toEqual([ + "delete", + "echo", + "invalidResult", + "largeError", + "marker", + "slow", + "sum", + ]); + }); + + it("fails to link an import the host module does not export", async () => { + const response = await runtime({ + source: ` + import { call } from "ws:test-host"; + export default () => call("echo"); + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text), text).toMatchObject({ + result: { status: "failed", stderr: expect.stringContaining("does not provide an export") }, + }); + }); + + it("runs container commands from isolate code through ws:container", async () => { + const response = await runtime({ + source: ` + import { exec } from "ws:container"; + export default () => + exec("npm test", { cwd: "/workspace/app", env: { WHO: "isolate" }, stdin: "y" }); + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text), text).toMatchObject({ + result: { + status: "completed", + value: { + exitCode: 8, + stdout: "ran npm test in /workspace/app with isolate and y\n", + stderr: "warn\n", + }, + }, + }); + }); + + it("rejects a malformed ws:container call inside the isolate", async () => { + const response = await runtime({ + source: ` + import { exec } from "ws:container"; + export default async () => { + try { + await exec("ls", { shell: "zsh" }); + return "ran"; + } catch (error) { + return error.message; + } + }; + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text).result.value).toContain('unknown option "shell"'); + }); + it("does not expose unrestricted host operations through the node:fs dispatcher", async () => { const response = await runtime({ source: ` @@ -356,7 +439,7 @@ describe("WorkspaceRuntime", () => { }); }); - it("confines trusted Git operations to the backend root", async () => { + it("confines ws:git operations to the backend root", async () => { const response = await runtime({ source: ` import { status } from "ws:git"; @@ -405,7 +488,7 @@ describe("WorkspaceRuntime", () => { expect(JSON.parse(text), text).toMatchObject({ result: { status: "failed", - stderr: expect.stringContaining("allowArtifac"), + stderr: expect.stringContaining("createArtifactsModule"), }, }); }); @@ -423,12 +506,12 @@ describe("WorkspaceRuntime", () => { expect(JSON.parse(text), text).toMatchObject({ result: { status: "failed", - stderr: expect.stringContaining("allowGitNetwork"), + stderr: expect.stringContaining("createGitModule"), }, }); }); - it("rejects trusted Git paths that traverse a symlink", async () => { + it("rejects ws:git paths that traverse a symlink", async () => { await write("/outside/repository/README.md", "outside"); await symlink("/outside/repository", "/workspace/linked-repository"); const response = await runtime({