diff --git a/.changeset/modules.md b/.changeset/modules.md new file mode 100644 index 00000000..52f178a3 --- /dev/null +++ b/.changeset/modules.md @@ -0,0 +1,11 @@ +--- +"@cloudflare/computer": minor +--- + +`WorkerJavaScriptBackend` takes a single `modules` option. A string is bundled source, as before. An object of functions is a host module that runs in the Durable Object under a `ws:*` specifier, and each function becomes a named export: `modules: { "ws:weather": { forecast } }` lets code write `import { forecast } from "ws:weather"`. A factory, `(host) => ({ ... })`, builds a host module from the Workspace's Git client, Artifacts client, or runtime. Each function receives `(args, { signal, deadline, access, resolvePath })` and may return any JSON-compatible value. + +`ws:git` and `ws:artifacts` are no longer installed automatically. Add `createGitModule()` from `@cloudflare/computer/modules/git` and `createArtifactsModule()` from `@cloudflare/computer/modules/artifacts`. `node:fs` and `node:fs/promises` stay built in. + +The backend describes its source language and every importable module for a model in `backend.description`, which `workspace.runtime.describe(id)` returns. + +To migrate, move `trustedModules` entries into `modules`, replacing any `call(method, args)` handler with one function per method. Replace `allowGitNetwork: true` with `createGitModule({ allowNetwork: true })` and `allowArtifactNetwork: true` with `createArtifactsModule({ allowNetwork: true })`. diff --git a/README.md b/README.md index 8e1ea38e..c8b8adfa 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,8 @@ SQLite and exposes one pluggable execution surface through Workers RPC, so there is no second store or sync round trip. - **Isolate JavaScript** runs an ECMAScript module in a fresh Dynamic Worker with structured input/results, durable relative imports, - configured libraries, Workspace-backed `node:fs/promises`, and trusted `ws:git` and - `ws:artifacts` modules. + configured libraries, Workspace-backed `node:fs/promises`, and host modules such as + `ws:git` and `ws:artifacts`. A Workspace may register multiple backends under stable IDs. `workspace.runtime.exec(source, { backend })` is the single execution diff --git a/docs/16_code_execution.md b/docs/16_code_execution.md index 26371512..34c316a2 100644 --- a/docs/16_code_execution.md +++ b/docs/16_code_execution.md @@ -17,7 +17,7 @@ The selected backend defines how it interprets `source`. | --- | --- | --- | | `container-shell` | shell command | Full Linux, native binaries, installed packages, processes | | `worker-shell` | just-bash command | Fast text tools and Workspace Git without a Container | -| `worker-javascript` | ECMAScript module | Isolated structured JavaScript with trusted Workspace modules | +| `worker-javascript` | ECMAScript module | Isolated structured JavaScript with Workspace host modules | Applications may register additional command or module backends under their own IDs. Backend IDs are part of the execution contract: changing the backend may change the source language. @@ -85,4 +85,4 @@ new WorkerJavaScriptBackend({ The backend argument is never itself authorization. -See [17. Isolate JavaScript](./17_isolate_javascript.md) for module and trusted-package behavior. +See [17. Isolate JavaScript](./17_isolate_javascript.md) for module behavior. diff --git a/docs/17_isolate_javascript.md b/docs/17_isolate_javascript.md index 2db84ecf..7db07cf6 100644 --- a/docs/17_isolate_javascript.md +++ b/docs/17_isolate_javascript.md @@ -88,7 +88,7 @@ Completed execution records remain available for replay for sixty minutes by def Cancellation stops new host capability calls, disposes the Dynamic Worker, and waits for host calls that were already accepted. Exit 130 is published only after those calls settle. Normal completion uses the same drain rule, so an unawaited capability call cannot mutate the workspace after exit 0. -Host calls have a caller-visible deadline, controlled by `maxHostCallMs` and defaulting to `maxTimeoutMs`. Missing the deadline fails the capability call and marks the execution failed, even if caller code catches that error. Execution still waits for the accepted host operation itself before publishing a terminal event because many host APIs cannot roll back an external side effect after dispatch. Trusted modules receive an optional `{ signal, deadline }` context and must stop promptly when the signal aborts. A trusted module that ignores cancellation and never settles will keep execution in its finalizing state. `compatibilityDate` and `compatibilityFlags` control the Dynamic Worker runtime and default to the package-tested settings. +Host calls have a caller-visible deadline, controlled by `maxHostCallMs` and defaulting to `maxTimeoutMs`. Missing the deadline fails the capability call and marks the execution failed, even if caller code catches that error. Execution still waits for the accepted host operation itself before publishing a terminal event because many host APIs cannot roll back an external side effect after dispatch. Host module functions receive a `signal` in their context and must stop promptly when it aborts. A host module that ignores cancellation and never settles will keep execution in its finalizing state. `compatibilityDate` and `compatibilityFlags` control the Dynamic Worker runtime and default to the package-tested settings. ## Environment, standard input, and the `process` shim @@ -119,22 +119,51 @@ const handle = await workspace.runtime.exec( ); ``` -## Configured modules +## Modules -Bare imports are installed at backend construction, not passed on individual executions: +Caller source can import three kinds of module, and all of them are fixed when the backend is constructed: + +| Kind | Configured with | Runs in | Example | +| --- | --- | --- | --- | +| Built in | Always installed | The isolate, backed by the Workspace | `node:fs`, `node:fs/promises` | +| Source | `modules: { name: "source" }` | The isolate | a bundled library | +| Host | `modules: { "ws:name": { fn } }`, or a factory | The Durable Object | `ws:git`, `ws:artifacts`, your own | ```ts +import { createArtifactsModule } from "@cloudflare/computer/modules/artifacts"; +import { createGitModule } from "@cloudflare/computer/modules/git"; + new WorkerJavaScriptBackend({ loader: env.LOADER, modules: { "tar-stream": TAR_STREAM_BUNDLE, + "ws:git": createGitModule(), + "ws:artifacts": createArtifactsModule(), + "ws:weather": { + forecast: ([city]) => lookUpForecast(String(city)), + }, }, }); ``` -Unknown bare imports fail before Worker creation. `node:fs` and `node:fs/promises` are host-installed exceptions backed by the durable Workspace. Configured modules are code, not host authority, and may not use the reserved `ws:` namespace or shadow either filesystem specifier. +An import that is not built in, configured, or a relative Workspace path fails before the Worker is created. Caller source and durable files cannot shadow a configured or built-in module. + +The backend describes its modules for a model in `backend.description`, which `workspace.runtime.describe(id)` returns. It is built from the same `modules` option the backend runs with, so it always matches what is installed: -## Trusted Workspace modules +```text +`command` is ECMAScript module source, run in an isolated JavaScript runtime. Relative imports resolve from `cwd` in the workspace. +Code has no direct network access. + +Modules code can import: +- `node:fs/promises` (also `node:fs`): the workspace's files. ... +- `tar-stream`: a bundled library. +- `ws:git`: The workspace's Git repository tools: `status({ dir })`, ... +- `ws:weather`: exports `forecast`. +``` + +A factory adds its own text through a `description` property, as the prebuilt modules do. An object of functions is listed by its export names. + +### Built-in filesystem Filesystem access uses the familiar asynchronous Node API, but is backed by the durable Workspace rather than an isolate-local filesystem. Both forms are installed automatically: @@ -148,7 +177,56 @@ await fs.writeFile("/workspace/output.txt", text.toUpperCase()); Supported promise APIs are `readFile`, `writeFile`, `mkdir`, `rm`, `chmod`, `symlink`, `readlink`, `readdir`, `stat`, `lstat`, and `access`. `readFile` returns bytes when encoding is omitted and supports `"utf8"` / `"utf-8"` for text; other encodings are rejected. `writeFile` supports the default `"w"` flag and exclusive `"wx"`; other Node flags are rejected, and—as in Node—the parent directory must already exist. Relative symlink targets are preserved by `readlink`, while reads and writes through symlinks are rejected by the Workspace confinement boundary. Synchronous and callback-style Node filesystem APIs are intentionally unavailable because every operation crosses the isolate-to-Workspace capability boundary. -The entire `ws:` namespace remains reserved for other Workspace-maintained host capabilities. The built-in runtime installs `ws:git` and `ws:artifacts`. +Path confinement rejects lexical escapes and every symlink component before an operation. These checks are not an atomic inode-style “resolve beneath root” primitive: do not treat one isolate capability as a security boundary against a separate, more privileged principal concurrently replacing paths in the same mutable Workspace. Deployments requiring that adversarial concurrency need a future transactional DOFS primitive or separate Workspace identities. + +### Source modules + +A string value is JavaScript source installed as a bare import, such as a bundled library. It is plain code with no host access, and it cannot use the `ws:` namespace or replace `node:fs` or `node:fs/promises`. + +### Host modules + +A host module runs in the Durable Object, and each of its functions becomes a named export in the isolate. Host modules must use a simple `ws:*` specifier. Nothing under `ws:` is installed unless you configure it. + +Pass an object of functions: + +```ts +modules: { + "ws:weather": { + forecast: ([city]) => lookUpForecast(String(city)), + }, +} +``` + +When the functions need the Workspace's Git client, Artifacts client, or runtime, pass a factory instead. The backend calls it once when it connects to its Workspace. This is how the prebuilt modules work: + +```ts +modules: { + "ws:repo": (host) => ({ + async recent(args, context) { + const dir = await context.resolvePath(String(args[0] ?? ".")); + return host.git.log({ dir, depth: 5 }); + }, + }), +} +``` + +```js +import { recent } from "ws:repo"; +export default () => recent("/workspace/app"); +``` + +Each function receives the arguments the isolate passed, as an array of JSON-compatible values, and a context: + +| Field | Meaning | +| --- | --- | +| `signal` | Aborts when the call passes its deadline or the execution is cancelled. | +| `deadline` | Epoch milliseconds after which the isolate stops waiting. | +| `access` | The backend's `"read"` or `"read-write"` access. Check it before any write. | +| `resolvePath(path, { allowMissing })` | Confines a caller path to the backend root and rejects symlinks. | + +The arguments come from caller code, so parse them before use. A function may return a value or a promise. The result must be JSON-compatible, and the bridge checks it at runtime: `undefined` becomes `null` and `undefined` object fields are dropped, as with `JSON.stringify`. It fits within the same capability byte limits as every other host call. A function that ignores `signal` and never settles keeps the execution in its finalizing state. + +Specifiers and the export names of an object are checked at construction. A factory's export names are checked when the backend connects and the factory runs. A module must export at least one function, and every export name must be a JavaScript identifier name other than `default` or `then`. A reserved word such as `delete` is allowed, and caller code renames it on import: `import { delete as remove } from "ws:files"`. Importing a name the module does not export fails when the module graph links, before any code runs. ### `ws:git` @@ -156,25 +234,15 @@ The entire `ws:` namespace remains reserved for other Workspace-maintained host import { clone, diff, status, log, cli } from "ws:git"; ``` -`ws:git` is explicit host authority rather than ambient isolate networking. Clone, fetch, pull, push, `ls-remote`, and submodule commands can perform host-side requests even when the Dynamic Worker has `globalOutbound: null`, so they are denied by default. Enable them only on a trusted backend construction with `allowGitNetwork: true`; local Git operations remain available without that authority. Remote `ws:artifacts.importArtifact()` is independently denied unless backend construction sets `allowArtifactNetwork: true`. +`createGitModule()` from `@cloudflare/computer/modules/git` wraps the Workspace's Git client. Every `dir` and `cwd` is confined to the backend root, `clone` and `cli` need a read-write backend, and `cli` treats a leading `-C ` as its working directory, confined the same way, while rejecting any other `-C`, `--git-dir`, and `--work-tree`. Clone, fetch, pull, push, `ls-remote`, and submodule commands run from the host, even when the Dynamic Worker has `globalOutbound: null`, so they are denied unless you pass `createGitModule({ allowNetwork: true })`. ### `ws:artifacts` ```js -import { - create, - get, - list, - importArtifact, - deleteArtifact, -} from "ws:artifacts"; +import { create, get, list, importArtifact, deleteArtifact } from "ws:artifacts"; ``` -These modules are sandbox-side shims over host RPC. Loader bindings, credentials, Durable Object storage, and unrestricted Workspace objects never enter user code. The host bridge checks the backend's fixed read/read-write authority on every mutation. Artifacts methods fail clearly when no Artifacts binding is configured. - -Caller modules and durable files cannot shadow `node:fs`, `node:fs/promises`, or `ws:*`. - -Path confinement rejects lexical escapes and every symlink component before an operation. These checks are not an atomic inode-style “resolve beneath root” primitive: do not treat one isolate capability as a security boundary against a separate, more privileged principal concurrently replacing paths in the same mutable Workspace. Deployments requiring that adversarial concurrency need a future transactional DOFS primitive or separate Workspace identities. +`createArtifactsModule()` from `@cloudflare/computer/modules/artifacts` wraps the Workspace's Artifacts client. Calls that change Artifacts need a read-write backend. `importArtifact()` fetches from a caller-chosen URL on the host, so it is denied unless you pass `createArtifactsModule({ allowNetwork: true })`. Every call fails clearly when no Artifacts binding is configured. ## Isolation and lifecycle @@ -190,7 +258,3 @@ Each execution receives a fresh Dynamic Worker with: - retained events and result rows in the Workspace database. Standard output and standard error stream live. The Dynamic Worker hands the readable end of its output stream to the host through the `attachOutput` bridge call, and the host drains it frame by frame while user code is still running, appending each chunk to the execution event stream as it arrives rather than buffering the run and publishing at the end. The structured result and the exit event settle once the output stream closes, so the terminal events always follow the last output. Output remains bounded by `maxStdioBytes` across both streams. Completed writes are durable immediately. Failure or cancellation does not roll back filesystem effects already completed. - -## Trusted integrations - -A host can configure additional reserved capability modules through `WorkerJavaScriptBackend.trustedModules`; these modules are fixed when the backend is constructed and cannot be supplied or replaced by caller source. diff --git a/docs/README.md b/docs/README.md index 06ca0d7a..f9cfb30f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,7 +20,7 @@ It provides: - R2-backed mounts for pre-filling read-only data into the workspace tree. - Durability over DO restarts for all file operations. - Pluggable execution backends selected through `workspace.runtime`: a Cloudflare Container shell, a just-bash Dynamic Worker, or an isolated ECMAScript-module Dynamic Worker. - - Isolated JavaScript with structured input/results, durable relative imports, configured libraries, durable `node:fs/promises`, trusted `ws:git` / `ws:artifacts`, and managed execution records. + - Isolated JavaScript with structured input/results, durable relative imports, configured libraries, durable `node:fs/promises`, host modules such as `ws:git` and `ws:artifacts`, and managed execution records. - Workspace constructable without a backend, for filesystem-only use cases. - Out-of-the-box AI SDK tools for `@cloudflare/agents` through `@cloudflare/computer/tools`. @@ -46,9 +46,11 @@ The package ships several entrypoints: | `@cloudflare/computer/backends/container` | `ContainerBackend` and `withWorkspaceContainer`, for a container the durable object schedules (`scheduling_policy: "durable_object"`). Same sync plumbing; the launch names the image and the instance size. | | `@cloudflare/computer/backends/container-legacy` | `LegacyContainerBackend` and `withLegacyWorkspaceContainer`, for a container the platform schedules and sizes from the containers block. | | `@cloudflare/computer/backends/worker-shell` | `WorkerShellBackend` and the bundled just-bash command runtime. | -| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable relative imports, `node:fs/promises`, and trusted `ws:git` / `ws:artifacts`. | +| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable relative imports, `node:fs/promises`, and host modules. | | `@cloudflare/computer/git` | Opt-in isomorphic-git glue for working with checkouts inside the workspace. Bundled lazily, with `pako` replaced by Workers `node:zlib`, and kept out of the default `@cloudflare/computer` graph. | | `@cloudflare/computer/artifacts` | `createArtifact`, an optionally session-scoped wrapper over the Cloudflare Artifacts Workers binding, plus its argv CLI. | +| `@cloudflare/computer/modules/git` | `createGitModule()` for `ws:git`: confined Git from isolate JavaScript. | +| `@cloudflare/computer/modules/artifacts` | `createArtifactsModule()` for `ws:artifacts`: Artifacts from isolate JavaScript. | | `@cloudflare/computer/tools` | AI SDK tools for agents: read, write, edit, ls, optional exec, and optional publish. | A consumer that only uses the container backend never imports the @@ -247,7 +249,7 @@ above, then dive into the area you're working on. | [14. Assets interface](./14_assets_interface.md) | `share` a workspace file to R2 and get back a presigned URL. | | [15. Artifacts interface](./15_artifacts_interface.md) | `createArtifact` and the `artifacts` CLI, an optionally session-scoped wrapper over the Cloudflare Artifacts binding. | | [16. Execution runtime architecture](./16_code_execution.md) | One runtime entry point over command and module backends. | -| [17. Isolate JavaScript runtime](./17_isolate_javascript.md) | ECMAScript modules, durable imports, configured libraries, durable `node:fs/promises`, trusted `ws:git` / `ws:artifacts`, and managed lifecycle. | +| [17. Isolate JavaScript runtime](./17_isolate_javascript.md) | ECMAScript modules, durable imports, configured libraries, durable `node:fs/promises`, host modules, and managed lifecycle. | | [18. Runtime migration](./18_runtime_migration.md) | Breaking preview-API mappings from public shell and script-execution surfaces to `workspace.runtime`. | | [19. Performance](./19_performance.md) | Filesystem benchmarks: `fs-bench` numbers, an `npm install` comparison, and how to reproduce them. | diff --git a/examples/rlm/README.md b/examples/rlm/README.md index 7d08e6db..899533f3 100644 --- a/examples/rlm/README.md +++ b/examples/rlm/README.md @@ -66,7 +66,7 @@ const backend = new WorkerJavaScriptBackend({ root: "/workspace", access: "read", egress: { mode: "none" }, - trustedModules: { + modules: { "ws:model": modelCapability, }, }); @@ -77,13 +77,13 @@ const workspace = new Workspace({ }); ``` -The important line is `trustedModules`. Generated code cannot read model credentials or call the network directly. It can only use the host-owned `ws:model` interface. +The important line is `modules`. Generated code cannot read model credentials or call the network directly. It can only use the host-owned `ws:model` interface. A generated module follows this shape: ```js import fs from "node:fs/promises"; -import { call as callModel } from "ws:model"; +import { batch } from "ws:model"; export default async function () { const manifest = JSON.parse( @@ -96,7 +96,7 @@ export default async function () { })), ); - const mapped = await callModel("batch", requests); + const mapped = await batch(requests); const totals = validateAndSum(mapped); return { answer: largestLabel(totals) }; } @@ -150,7 +150,7 @@ The reducer is exact relative to its inputs. Model classifications can still be The shortest path through the example is: 1. [`worker/rlm-agent.ts`](worker/rlm-agent.ts) wires together the model, Workspace, Worker JavaScript backend, executor tool, and `ws:model`. -2. [`worker/capability.ts`](worker/capability.ts) implements the bounded `ws:model("batch", requests)` interface. +2. [`worker/capability.ts`](worker/capability.ts) implements the bounded `batch(requests)` function behind `ws:model`. 3. [`worker/structured-rlm.ts`](worker/structured-rlm.ts) describes the map result and JavaScript reduction for each task family. 4. [`worker/agent-common.ts`](worker/agent-common.ts) writes the same corpus into each Computer Workspace. 5. [`worker/executor-tool.ts`](worker/executor-tool.ts) creates the native Computer executor tool and keeps its browser-facing result small. diff --git a/examples/rlm/worker/capability.test.ts b/examples/rlm/worker/capability.test.ts index d6bc9fcf..c902cdea 100644 --- a/examples/rlm/worker/capability.test.ts +++ b/examples/rlm/worker/capability.test.ts @@ -1,3 +1,4 @@ +import type { WorkspaceModuleCallContext } from "@cloudflare/computer"; import type { LanguageModel } from "ai"; import { beforeEach, describe, expect, it, vi } from "vitest"; @@ -30,6 +31,15 @@ function successfulResult(text = "ok") { }; } +function context(signal = new AbortController().signal): WorkspaceModuleCallContext { + return { + signal, + deadline: Date.now() + 1_000, + access: "read", + resolvePath: async (path) => path, + }; +} + async function waitForCalls(count: number): Promise { for (let attempt = 0; attempt < 50; attempt += 1) { if (generateTextMock.mock.calls.length >= count) return; @@ -44,28 +54,25 @@ beforeEach(() => { }); describe("recursive model batch capability", () => { - it("exposes only the batch method", async () => { + it("exposes only the batch function", () => { const capability = createModelCapability(model, hooks()); - await expect(capability.call("generate", [[]])).rejects.toThrow( - "Unknown model capability method", - ); - expect(generateTextMock).not.toHaveBeenCalled(); + expect(Object.keys(capability)).toEqual(["batch"]); }); it("strictly validates the external argument and request shapes", async () => { const capability = createModelCapability(model, hooks()); - await expect(capability.call("batch", [])).rejects.toThrow("exactly one argument"); - await expect(capability.call("batch", [null])).rejects.toThrow("non-empty array"); - await expect(capability.call("batch", [[]])).rejects.toThrow("non-empty array"); + await expect(capability.batch([], context())).rejects.toThrow("exactly one argument"); + await expect(capability.batch([null], context())).rejects.toThrow("non-empty array"); + await expect(capability.batch([[]], context())).rejects.toThrow("non-empty array"); await expect( - capability.call("batch", [[{ prompt: "classify", input: null, extra: true }], null]), + capability.batch([[{ prompt: "classify", input: null, extra: true }], null], context()), ).rejects.toThrow("exactly one argument"); await expect( - capability.call("batch", [[{ prompt: "classify", input: null, extra: true }]]), + capability.batch([[{ prompt: "classify", input: null, extra: true }]], context()), ).rejects.toThrow("Invalid batch request"); - await expect(capability.call("batch", [[{ prompt: "", input: null }]])).rejects.toThrow( + await expect(capability.batch([[{ prompt: "", input: null }]], context())).rejects.toThrow( "requires a prompt", ); expect(generateTextMock).not.toHaveBeenCalled(); @@ -78,7 +85,9 @@ describe("recursive model batch capability", () => { input: null, })); - await expect(capability.call("batch", [requests])).rejects.toThrow("cannot exceed 24 requests"); + await expect(capability.batch([requests], context())).rejects.toThrow( + "cannot exceed 24 requests", + ); expect(generateTextMock).not.toHaveBeenCalled(); }); @@ -88,7 +97,7 @@ describe("recursive model batch capability", () => { const capability = createModelCapability(model, { ...observed, admit }); const requests = [{ prompt: "classify", input: "evidence" }]; - await expect(capability.call("batch", [requests])).rejects.toThrow( + await expect(capability.batch([requests], context())).rejects.toThrow( "exhausted its child-model call budget", ); expect(admit).toHaveBeenCalledWith(1); @@ -99,7 +108,7 @@ describe("recursive model batch capability", () => { const capability = createModelCapability(model, hooks()); await expect( - capability.call("batch", [[{ prompt: "é".repeat(8 * 1024 + 1), input: null }]]), + capability.batch([[{ prompt: "é".repeat(8 * 1024 + 1), input: null }]], context()), ).rejects.toThrow("16384 bytes"); expect(generateTextMock).not.toHaveBeenCalled(); }); @@ -109,10 +118,10 @@ describe("recursive model batch capability", () => { const maximumBody = "x".repeat(48 * 1024 - 11); await expect( - capability.call("batch", [[{ prompt: "classify", input: { body: maximumBody } }]]), + capability.batch([[{ prompt: "classify", input: { body: maximumBody } }]], context()), ).resolves.toHaveLength(1); await expect( - capability.call("batch", [[{ prompt: "classify", input: { body: `${maximumBody}x` } }]]), + capability.batch([[{ prompt: "classify", input: { body: `${maximumBody}x` } }]], context()), ).rejects.toThrow("49152 bytes"); }); @@ -123,7 +132,7 @@ describe("recursive model batch capability", () => { input: { body: "x".repeat(48 * 1024 - 12) }, })); - await expect(capability.call("batch", [requests])).rejects.toThrow( + await expect(capability.batch([requests], context())).rejects.toThrow( "Model batch request cannot exceed", ); expect(generateTextMock).not.toHaveBeenCalled(); @@ -145,9 +154,10 @@ describe("recursive model batch capability", () => { }), ); const capability = createModelCapability(model, hooks()); - const resultPromise = capability.call("batch", [ - Array.from({ length: 8 }, (_, index) => ({ prompt: `request ${index}`, input: null })), - ]); + const resultPromise = capability.batch( + [Array.from({ length: 8 }, (_, index) => ({ prompt: `request ${index}`, input: null }))], + context(), + ); await waitForCalls(4); expect(generateTextMock).toHaveBeenCalledTimes(4); @@ -186,10 +196,7 @@ describe("recursive model batch capability", () => { const signal = new AbortController().signal; await expect( - capability.call("batch", [[{ prompt: "classify", input: { evidence: "safe" } }]], { - signal, - deadline: Date.now() + 1_000, - }), + capability.batch([[{ prompt: "classify", input: { evidence: "safe" } }]], context(signal)), ).resolves.toEqual([ { id: expect.any(String), index: 0, ok: true, text: "classification", error: null }, ]); @@ -225,7 +232,7 @@ describe("recursive model batch capability", () => { const capability = createModelCapability(model, observer); await expect( - capability.call("batch", [[{ prompt: "classify", input: null }]]), + capability.batch([[{ prompt: "classify", input: null }]], context()), ).resolves.toEqual([ { id: expect.any(String), @@ -255,10 +262,9 @@ describe("recursive model batch capability", () => { const observer = hooks(); const capability = createModelCapability(model, observer); const controller = new AbortController(); - const result = capability.call( - "batch", + const result = capability.batch( [Array.from({ length: 8 }, (_, index) => ({ prompt: `request ${index}`, input: null }))], - { signal: controller.signal, deadline: Date.now() + 1_000 }, + context(controller.signal), ); await waitForCalls(4); @@ -274,9 +280,10 @@ describe("recursive model batch capability", () => { const observer = hooks(); const capability = createModelCapability(model, observer); - await capability.call("batch", [ - [{ prompt: "SECRET_PROMPT", input: { evidence: "SECRET_INPUT" } }], - ]); + await capability.batch( + [[{ prompt: "SECRET_PROMPT", input: { evidence: "SECRET_INPUT" } }]], + context(), + ); const synchronizedHookData = JSON.stringify({ started: observer.started.mock.calls, diff --git a/examples/rlm/worker/capability.ts b/examples/rlm/worker/capability.ts index bc8b0e05..7ad79083 100644 --- a/examples/rlm/worker/capability.ts +++ b/examples/rlm/worker/capability.ts @@ -1,4 +1,4 @@ -import type { WorkspaceRuntimeValue, WorkspaceTrustedModule } from "@cloudflare/computer"; +import type { WorkspaceModuleFunction, WorkspaceRuntimeValue } from "@cloudflare/computer"; import { generateText, type LanguageModel } from "ai"; import { z } from "zod"; @@ -73,18 +73,20 @@ interface ChildHooks { failed(metadata: FailedMetadata): void; } -export function createModelCapability( - model: LanguageModel, - hooks: ChildHooks, -): WorkspaceTrustedModule { +/** The `ws:model` trusted module: one `batch` function over bounded child requests. */ +export type ModelCapability = { + /** Run up to 24 child model requests and return one result per request. */ + readonly batch: WorkspaceModuleFunction; +}; + +export function createModelCapability(model: LanguageModel, hooks: ChildHooks): ModelCapability { return { - async call(method, args, context) { - if (method !== "batch") throw new Error(`Unknown model capability method: ${method}`); + async batch(args, context) { const requests = parseBatchArgs(args); if (hooks.admit && !hooks.admit(requests.length)) { throw new Error("This run has exhausted its child-model call budget."); } - const signal = context?.signal; + const signal = context.signal; throwIfAborted(signal); const results: Array = new Array(requests.length); @@ -165,7 +167,7 @@ async function runChild( } } -function parseBatchArgs(args: WorkspaceRuntimeValue[]): ChildRequest[] { +function parseBatchArgs(args: readonly WorkspaceRuntimeValue[]): ChildRequest[] { if (args.length !== 1) throw new Error("Model batch requires exactly one argument."); const value = args[0]; if (!Array.isArray(value) || value.length === 0) { diff --git a/examples/rlm/worker/prompts.ts b/examples/rlm/worker/prompts.ts index 3243e9c0..ca554246 100644 --- a/examples/rlm/worker/prompts.ts +++ b/examples/rlm/worker/prompts.ts @@ -16,11 +16,11 @@ The executor command must be a complete ES module with export default async func export const RLM_SYSTEM_PROMPT = `Solve the Oolong benchmark with one comprehensive recursive Computer execution, then answer briefly. -The executor command must be a complete ES module with export default async function. Use import fs from "node:fs/promises" and import { call as callModel } from "ws:model". Read /workspace/oolong-real/manifest.json. Its contextChunks contain the long corpus. +The executor command must be a complete ES module with export default async function. Use import fs from "node:fs/promises" and import { batch } from "ws:model". Read /workspace/oolong-real/manifest.json. Its contextChunks contain the long corpus. Use Computer as an RLM. Do not make manifest-inspection, schema-discovery, or diagnostic-only calls: 1. Read the bounded corpus chunks. -2. You have one child-call budget of 24 total requests. Call callModel("batch", requests) exactly once with at most 24 focused requests. Each request is { prompt, input }; keep each input to one chunk and ask for structured, question-specific evidence. +2. You have one child-call budget of 24 total requests. Call batch(requests) exactly once with at most 24 focused requests. Each request is { prompt, input }; keep each input to one chunk and ask for structured, question-specific evidence. 3. Each child result is { index, ok, text, error }. Aggregate successful findings in JavaScript and tolerate failed workers. For first/last-event questions, retain chunk indexes and choose evidence by transcript position; never replace an earlier event with a later, more salient one. 4. Parse the child text, aggregate findings in corpus chunk order, and return { answer } from the default function. If a later execution is needed to finalize from prior findings, it must still return { answer }. diff --git a/examples/rlm/worker/rlm-agent.ts b/examples/rlm/worker/rlm-agent.ts index 7d0fd706..bd34039e 100644 --- a/examples/rlm/worker/rlm-agent.ts +++ b/examples/rlm/worker/rlm-agent.ts @@ -107,7 +107,7 @@ export class RlmAgent extends AIChatAgent { root: WORKSPACE_ROOT, access: "read", egress: { mode: "none" }, - trustedModules: { "ws:model": modelCapability }, + modules: { "ws:model": modelCapability }, maxConcurrentExecutions: 1, maxConcurrentCapabilityCalls: 4, // One manifest read + 24 chunk reads + one bounded ws:model batch. diff --git a/examples/rlm/worker/rlm-interfaces.txt b/examples/rlm/worker/rlm-interfaces.txt index 7e876c9a..139ce825 100644 --- a/examples/rlm/worker/rlm-interfaces.txt +++ b/examples/rlm/worker/rlm-interfaces.txt @@ -1,7 +1,7 @@ Module interface: import fs from "node:fs/promises"; -import { call as callModel } from "ws:model"; +import { batch } from "ws:model"; export default async function () { // Put every file read, batch call, reduction, and return statement inside this function. } -No asynchronous operation or return statement may appear at module scope. Call recursive inference exactly once as callModel("batch", requests). +No asynchronous operation or return statement may appear at module scope. Call recursive inference exactly once as batch(requests). diff --git a/examples/rlm/worker/step-settings.ts b/examples/rlm/worker/step-settings.ts index 7e91f873..443bcfbb 100644 --- a/examples/rlm/worker/step-settings.ts +++ b/examples/rlm/worker/step-settings.ts @@ -5,7 +5,7 @@ export const COMPUTER_FINALIZATION_STEP = 1; const FINALIZATION_INSTRUCTION = 'Return the benchmark answer now. The next executor module must return a typed { answer } object using evidence already seen. Do not inspect more data, return diagnostics, or call ws:model again. The final module must contain no imports or file I/O. Use exactly this shape: export default async function () { return { answer: "derived answer" }; }'; const RECURSION_RETRY_INSTRUCTION = - 'The previous execution did not use recursive inference. Call ws:model("batch", requests) now with question-specific requests over the Workspace chunks, then return the child findings.'; + "The previous execution did not use recursive inference. Call batch(requests) from ws:model now with question-specific requests over the Workspace chunks, then return the child findings."; export function requiredComputerStep( stepNumber: number, diff --git a/examples/rlm/worker/structured-rlm.test.ts b/examples/rlm/worker/structured-rlm.test.ts index 4c76af39..789808c8 100644 --- a/examples/rlm/worker/structured-rlm.test.ts +++ b/examples/rlm/worker/structured-rlm.test.ts @@ -114,7 +114,7 @@ describe("structured RLM strategy", () => { expect(prompt).toContain("structured-v1"); expect(prompt).toContain("last_spell_by_episode"); - expect(prompt).toContain('callModel("batch", requests)'); + expect(prompt).toContain("batch(requests)"); expect(prompt).not.toContain("SECRET_GOLD"); }); }); diff --git a/packages/computer/README.md b/packages/computer/README.md index dc9b81b4..0412c6fb 100644 --- a/packages/computer/README.md +++ b/packages/computer/README.md @@ -255,8 +255,8 @@ Alongside `exec`, the runtime exposes `getExec`, `killExec`, and [`examples/worker-shell`](../../examples/worker-shell). - **Worker JavaScript** evaluates a module with structured input/results, durable relative imports, configured libraries, - Workspace-backed `node:fs/promises`, and trusted `ws:git` / - `ws:artifacts` modules. It runs after `runtime.exec()` returns; the + Workspace-backed `node:fs/promises`, and host modules such as + `ws:git` and `ws:artifacts`. It runs after `runtime.exec()` returns; the run stays alive while its event stream is consumed. See [`docs/17_isolate_javascript.md`](../../docs/17_isolate_javascript.md) and [`examples/worker-javascript`](../../examples/worker-javascript). @@ -418,7 +418,9 @@ on a computerd instance. | `@cloudflare/computer` | The `Workspace` wrapper, `workspace.runtime`, stub types, the R2 mount, and proxy classes. | | `@cloudflare/computer/backends/container-legacy` | `LegacyContainerBackend` and `withLegacyWorkspaceContainer`. Pulls in the computerd / capnweb sync plumbing. | | `@cloudflare/computer/backends/worker-shell` | `WorkerShellBackend` and the bundled just-bash runtime. | -| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable imports, `node:fs/promises`, and trusted `ws:git` / `ws:artifacts`. | +| `@cloudflare/computer/backends/worker-javascript` | `WorkerJavaScriptBackend`, configured libraries, durable imports, `node:fs/promises`, and host modules. | +| `@cloudflare/computer/modules/git` | `createGitModule()` for `ws:git`: confined Git from isolate JavaScript. | +| `@cloudflare/computer/modules/artifacts` | `createArtifactsModule()` for `ws:artifacts`: Artifacts from isolate JavaScript. | | `@cloudflare/computer/tools` | AI SDK tools for agents: `read`, `ls`, `find`, `grep`, `write`, `edit`, `delete`, and optional `exec` and `publish`. | | `@cloudflare/computer/git` | Opt-in `isomorphic-git` glue for checkouts inside the workspace. | | `@cloudflare/computer/assets` | `createAssets` — share a workspace file to R2 as a presigned URL. | diff --git a/packages/computer/package.json b/packages/computer/package.json index 516e38d4..333f145a 100644 --- a/packages/computer/package.json +++ b/packages/computer/package.json @@ -31,6 +31,14 @@ "types": "./dist/artifacts/index.d.ts", "import": "./dist/artifacts/index.js" }, + "./modules/git": { + "types": "./dist/modules/git.d.ts", + "import": "./dist/modules/git.js" + }, + "./modules/artifacts": { + "types": "./dist/modules/artifacts.d.ts", + "import": "./dist/modules/artifacts.js" + }, "./tools": { "types": "./dist/tools/index.d.ts", "import": "./dist/tools/index.js" diff --git a/packages/computer/rolldown.config.ts b/packages/computer/rolldown.config.ts index 0a78e602..70b3ec7a 100644 --- a/packages/computer/rolldown.config.ts +++ b/packages/computer/rolldown.config.ts @@ -31,6 +31,8 @@ export default defineConfig({ "artifacts/index": "src/artifacts/index.ts", "assets/index": "src/assets/index.ts", "tools/index": "src/tools/index.ts", + "modules/git": "src/modules/git.ts", + "modules/artifacts": "src/modules/artifacts.ts", "backends/container-legacy/index": "src/backends/container-legacy/index.ts", "backends/container/index": "src/backends/container/index.ts", "backends/worker-javascript/index": "src/backends/worker-javascript/index.ts", diff --git a/packages/computer/src/backend.ts b/packages/computer/src/backend.ts index dac310eb..fd6c3fd9 100644 --- a/packages/computer/src/backend.ts +++ b/packages/computer/src/backend.ts @@ -60,6 +60,11 @@ export interface WorkspaceBackend { // too. Defaults to false when omitted. readonly callable?: boolean; + // What the backend tells a model about itself, such as the language + // it runs and what that code can use. The exec tool shows it next to + // the caller's own description. Omit when there is nothing to add. + readonly description?: string; + // Materialise a connection. Called lazily on first use, once // per backend per workspace lifetime. The Workspace caches the // resulting handle by `id`; subsequent exec / push / pull diff --git a/packages/computer/src/backends/worker-javascript/module-graph.ts b/packages/computer/src/backends/worker-javascript/module-graph.ts index 8280e7b5..2e94ca65 100644 --- a/packages/computer/src/backends/worker-javascript/module-graph.ts +++ b/packages/computer/src/backends/worker-javascript/module-graph.ts @@ -1,7 +1,12 @@ import { parse } from "acorn"; import type { WorkspaceRuntimeCapability } from "../../runtime/capability.js"; -import type { WorkspaceRuntimeLoader } from "../../runtime/types.js"; +import type { + WorkspaceModule, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceRuntimeLoader, +} from "../../runtime/types.js"; export type JavaScriptModuleMap = WorkspaceRuntimeLoader extends { load(code: { modules: infer Modules }): unknown; @@ -12,14 +17,119 @@ export type JavaScriptModuleMap = WorkspaceRuntimeLoader extends { const ENTRY_BASENAME = "__workspace_entry__.js"; const RUNNER_MODULE = "workspace-runtime-runner.js"; const CAPABILITIES_MODULE = "workspace-capabilities.js"; -const TRUSTED_MODULES = ["node:fs", "node:fs/promises", "ws:git", "ws:artifacts"] as const; +// Installed in every execution and backed by the Workspace. No module +// in the `modules` option may use these names. +const BUILT_IN_MODULES = ["node:fs", "node:fs/promises"] as const; +const HOST_SPECIFIER = /^ws:[A-Za-z0-9][A-Za-z0-9._-]*$/; +const EXPORT_NAME = /^[A-Za-z_$][A-Za-z0-9_$]*$/; +// `default` would turn the function into the default export, and a +// `then` export makes the module namespace look like a promise to +// `await import(...)`. +const RESERVED_EXPORT_NAMES = new Set(["default", "then"]); + +/** The `modules` option, parsed once when the backend is constructed. */ +export interface ParsedModules { + readonly source: Readonly>; + readonly host: ReadonlyMap; + /** One markdown bullet per importable module, for a model. */ + readonly description: string; +} + +const FILESYSTEM_DESCRIPTION = + "- `node:fs/promises` (also `node:fs`): the workspace's files. `readFile`, `writeFile`, `mkdir`, `rm`, `readdir`, `stat`, `lstat`, `readlink`, `symlink`, `chmod`, and `access`. Async only."; + +/** + * Parse the backend's `modules` option: split it into bundled source + * and host module factories, check every specifier and every object's + * export names, and describe each module for a model. A factory's + * export names are checked when the backend connects and it runs. + * + * @param modules - The modules passed to the backend. + * @returns Source modules, host module factories, and their description. + * @throws When a specifier or an object's export names are not allowed. + * The host configured the backend wrongly and no execution can use it. + */ +export function parseModules(modules: Readonly>): ParsedModules { + const source: Record = Object.create(null); + const host = new Map(); + const lines = [FILESYSTEM_DESCRIPTION]; + for (const [specifier, module] of Object.entries(modules)) { + const name = `\`${specifier}\``; + if (BUILT_IN_MODULES.some((builtIn) => builtIn === specifier)) { + throw new Error(`Module ${JSON.stringify(specifier)} is built in and cannot be replaced.`); + } + if (typeof module === "string") { + if (specifier.startsWith("ws:")) { + throw new Error( + `Module ${JSON.stringify(specifier)} uses the ws:* namespace, which is only for host modules.`, + ); + } + if (specifier.includes("/") || isInternalModuleName(specifier)) { + throw new Error(`Module ${JSON.stringify(specifier)} uses a reserved module name.`); + } + source[specifier] = module; + lines.push(`- ${name}: a bundled library.`); + continue; + } + if (!HOST_SPECIFIER.test(specifier)) { + throw new Error( + `Host module ${JSON.stringify(specifier)} must use a simple ws:* name, such as "ws:git".`, + ); + } + if (typeof module === "function") { + host.set(specifier, module); + lines.push(`- ${name}: ${module.description ?? "a host module."}`); + continue; + } + if (module === null || typeof module !== "object" || Array.isArray(module)) { + throw new Error( + `Module ${JSON.stringify(specifier)} must be source text, an object of functions, or a factory.`, + ); + } + assertHostModuleExports(specifier, module); + host.set(specifier, () => module); + const exports = Object.keys(module).map((key) => `\`${key}\``); + lines.push(`- ${name}: exports ${exports.join(", ")}.`); + } + return { source, host, description: lines.join("\n") }; +} + +/** + * Check the functions a host module exports. + * + * @param specifier - The module's specifier, for error messages. + * @param functions - The module's functions. + * @throws When the module exports nothing, a name is not allowed, or a + * value is not a function. + */ +export function assertHostModuleExports( + specifier: string, + functions: WorkspaceModuleFunctions, +): void { + const names = Object.keys(functions); + if (names.length === 0) { + throw new Error(`Host module ${JSON.stringify(specifier)} must export a function.`); + } + for (const name of names) { + if (!EXPORT_NAME.test(name) || RESERVED_EXPORT_NAMES.has(name)) { + throw new Error( + `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a JavaScript identifier other than "default" or "then".`, + ); + } + if (typeof functions[name] !== "function") { + throw new Error( + `Host module ${JSON.stringify(specifier)} export ${JSON.stringify(name)} must be a function.`, + ); + } + } +} export interface BuildModuleGraphOptions { source: string; cwd: string; capability: WorkspaceRuntimeCapability; - configuredModules: Record; - trustedModuleNames?: string[]; + configuredModules: Readonly>; + hostModules: ReadonlyMap; maxSourceBytes: number; maxCapabilityBytes: number; maxModules?: number; @@ -39,18 +149,10 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { let totalBytes = new TextEncoder().encode(options.source).byteLength; const maxModules = options.maxModules ?? 128; const maxDepth = options.maxDepth ?? 32; - const trustedModuleNames = new Set(TRUSTED_MODULES); - for (const name of options.trustedModuleNames ?? []) { - if ( - !/^ws:[A-Za-z0-9][A-Za-z0-9._-]*$/.test(name) || - TRUSTED_MODULES.includes(name as (typeof TRUSTED_MODULES)[number]) - ) { - throw new Error( - `Trusted module ${JSON.stringify(name)} must use a unique simple reserved ws:* name.`, - ); - } - trustedModuleNames.add(name); - } + const importableModuleNames = new Set([ + ...BUILT_IN_MODULES, + ...options.hostModules.keys(), + ]); async function visit(path: string, source: string, depth: number): Promise { if (depth > maxDepth) throw new Error(`Workspace JavaScript import depth exceeds ${maxDepth}.`); @@ -63,12 +165,14 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { } for (const specifier of imports(source)) { - if (trustedModuleNames.has(specifier)) continue; + if (importableModuleNames.has(specifier)) continue; if (specifier === CAPABILITIES_MODULE) { throw new Error(`Module ${JSON.stringify(specifier)} is reserved for Workspace internals.`); } if (specifier.startsWith("ws:")) { - throw new Error(`Unknown trusted Workspace module ${JSON.stringify(specifier)}.`); + throw new Error( + `Module ${JSON.stringify(specifier)} is not configured. Add it to the backend's modules option.`, + ); } if (specifier.startsWith(".")) { const resolved = resolveRelative(path, specifier); @@ -111,21 +215,6 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { await visit(entryPath, options.source, 0); - for (const specifier of Object.keys(options.configuredModules)) { - if ( - trustedModuleNames.has(specifier) || - specifier.startsWith("ws:") || - specifier === ENTRY_BASENAME || - specifier === RUNNER_MODULE || - specifier === CAPABILITIES_MODULE || - specifier.includes("/") - ) { - throw new Error( - `Configured module ${JSON.stringify(specifier)} uses a reserved module name.`, - ); - } - } - // node:* specifiers use protocol-style resolution and therefore need exact // module-map keys rather than the importer-directory aliases used by ws:*. modules["node:fs/promises"] = { js: nodeFsPromisesModule() }; @@ -134,11 +223,9 @@ export async function buildModuleGraph(options: BuildModuleGraphOptions) { for (const directory of directories) { const prefix = directory ? `${directory}/` : ""; const toCapabilities = relativeModule(directory, CAPABILITIES_MODULE); - modules[`${prefix}ws:git`] = { js: gitModule(toCapabilities) }; - modules[`${prefix}ws:artifacts`] = { js: artifactsModule(toCapabilities) }; - for (const specifier of options.trustedModuleNames ?? []) { + for (const [specifier, functions] of options.hostModules) { modules[`${prefix}${specifier}`] = { - js: trustedModule(toCapabilities, specifier), + js: hostModule(toCapabilities, specifier, Object.keys(functions)), }; } for (const [specifier, source] of Object.entries(options.configuredModules)) { @@ -297,17 +384,15 @@ function capabilitiesModule(maxCapabilityBytes: number) { `; } -function proxyModule(capabilitiesImport: string, namespace: string, methods: string[]) { +// Exports go through `export { local as name }` rather than +// `export const name`, so a reserved word such as `delete` still works +// as an export name. +function hostModule(capabilitiesImport: string, specifier: string, names: readonly string[]) { + const namespace = JSON.stringify(`host/${specifier}`); return ` import { call } from ${JSON.stringify(capabilitiesImport)}; - ${methods.map((method) => `export const ${method} = (...args) => call(${JSON.stringify(namespace)}, ${JSON.stringify(method)}, args);`).join("\n")} - `; -} - -function trustedModule(capabilitiesImport: string, specifier: string) { - return ` - import { call as hostCall } from ${JSON.stringify(capabilitiesImport)}; - export const call = (method, ...args) => hostCall(${JSON.stringify(`trusted/${specifier}`)}, "call", [method, ...args]); + ${names.map((name, index) => `const fn${index} = (...args) => call(${namespace}, ${JSON.stringify(name)}, args);`).join("\n")} + export { ${names.map((name, index) => `fn${index} as ${name}`).join(", ")} }; `; } @@ -370,17 +455,3 @@ function nodeFsPromisesModule() { function nodeFsModule() { return `${nodeFsPromisesModule()}\nexport { default as promises } from "node:fs/promises";`; } - -function gitModule(capabilitiesImport: string) { - return proxyModule(capabilitiesImport, "git", ["clone", "diff", "status", "log", "cli"]); -} - -function artifactsModule(capabilitiesImport: string) { - return proxyModule(capabilitiesImport, "artifacts", [ - "create", - "get", - "list", - "importArtifact", - "deleteArtifact", - ]); -} diff --git a/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts b/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts index f75ef9bc..e75b00fb 100644 --- a/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts +++ b/packages/computer/src/backends/worker-javascript/worker-javascript.test.ts @@ -244,7 +244,13 @@ describe("WorkerJavaScriptBackend", () => { ); const fs = new WorkspaceFilesystem(db); const backend = new WorkerJavaScriptBackend({ loader: throwingLoader("unused") }); - await backend.connect({ db, fs, git: undefined as never, artifacts: undefined as never }); + await backend.connect({ + db, + fs, + git: undefined as never, + artifacts: undefined as never, + runtime: undefined as never, + }); const columns = db.all<{ name: string }>("PRAGMA table_info(workspace_runtime_executions)"); expect(columns.map((column) => column.name)).toEqual( expect.arrayContaining(["created_at", "finished_at"]), @@ -310,6 +316,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = handle.exec({ source: `import task from "./task.js"; export default task;`, @@ -564,6 +571,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "successful-host-call", source: "export default 1" }); let settled = false; @@ -628,6 +636,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "live-stream", source: "export default 1" }); const reader = execution.events.getReader(); @@ -682,6 +691,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "kill-mid-stream", source: "export default 1" }); const reader = execution.events.getReader(); @@ -744,6 +754,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "no-exit", source: "export default 1" }); const events = []; @@ -754,7 +765,7 @@ describe("WorkerJavaScriptBackend", () => { await handle.close(); }); - it("aborts cooperative trusted-module calls at their deadline", async () => { + it("aborts cooperative host module calls at their deadline", async () => { const db = new Database(new SQLiteTestStorage()); initializeSchema(db, () => 0); const fs = new WorkspaceFilesystem(db); @@ -762,11 +773,11 @@ describe("WorkerJavaScriptBackend", () => { let aborted = false; const backend = new WorkerJavaScriptBackend({ maxHostCallMs: 5, - trustedModules: { + modules: { "ws:test": { - call(_method, _args, context) { + run(_args, context) { return new Promise((_resolve, reject) => { - context?.signal.addEventListener("abort", () => { + context.signal.addEventListener("abort", () => { aborted = true; reject(context.signal.reason); }); @@ -783,7 +794,7 @@ describe("WorkerJavaScriptBackend", () => { _input: unknown, host: { call(name: string, args: string): Promise }, ) { - await host.call("trusted/ws:test.call", JSON.stringify(["run"])); + await host.call("host/ws:test.run", JSON.stringify([])); }, }; }, @@ -796,6 +807,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "trusted-timeout", source: "export default 1" }); const events = []; @@ -847,6 +859,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); const execution = await handle.exec({ id: "cancel-host-call", source: "export default 1" }); await started; @@ -989,6 +1002,7 @@ describe("WorkerJavaScriptBackend", () => { fs, git: undefined as never, artifacts: undefined as never, + runtime: undefined as never, }); await handle.exec({ id: "subscribers", source: "export default 1" }); await handle.getExec({ id: "subscribers", after: "tail" }); @@ -997,28 +1011,163 @@ describe("WorkerJavaScriptBackend", () => { await handle.close(); }); - it("rejects malformed host trusted-module names", async () => { - const workspace = new Workspace({ - storage: new SQLiteTestStorage(), - backends: [ + it.each([ + ["a host module with a path", { "ws:bad/path": { run: async () => null } }, /simple ws:\*/], + ["a host module outside ws:*", { container: { run: async () => null } }, /simple ws:\*/], + ["source under ws:*", { "ws:lib": "export const x = 1;" }, /only for host modules/], + ["a replacement node:fs", { "node:fs": "export default {};" }, /built in/], + [ + "a replacement node:fs host module", + { "node:fs/promises": { run: async () => null } }, + /built in/, + ], + ["a number", { "ws:test": 42 }, /source text, an object of functions, or a factory/], + ["an object with no functions", { "ws:test": {} }, /must export a function/], + [ + "an object with a non-identifier name", + { "ws:test": { "not-a-name": async () => null } }, + /JavaScript identifier/, + ], + [ + "an object with a default export", + { "ws:test": { default: async () => null } }, + /JavaScript identifier/, + ], + [ + "an object with a then export", + // biome-ignore lint/suspicious/noThenProperty: The case checks that the backend rejects a `then` export. + { "ws:test": { then: async () => null } }, + /JavaScript identifier/, + ], + ["an object with a non-function export", { "ws:test": { run: "nope" } }, /must be a function/], + ])("rejects %s at construction", (_label, modules, message) => { + expect( + () => new WorkerJavaScriptBackend({ loader: throwingLoader("must not load"), - trustedModules: { - "ws:bad/path": { - async call() { - return null; - }, - }, - } as never, + // SAFETY: Each case hands the constructor a shape the types may forbid, to check its runtime guard. + modules: modules as never, }), - ], + ).toThrow(message); + }); + + it("rejects a factory whose functions are not allowed when it connects", async () => { + const db = new Database(new SQLiteTestStorage()); + initializeSchema(db, () => 0); + const backend = new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + modules: { "ws:test": () => ({ "not-a-name": async () => null }) }, }); - await workspace.fs.mkdir("/workspace", { recursive: true }); await expect( - workspace.runtime.exec(`import { call } from "ws:bad/path"; export default call;`, { - backend: "worker-javascript", + backend.connect({ + db, + fs: new WorkspaceFilesystem(db), + git: undefined as never, + artifacts: undefined as never, + runtime: undefined as never, }), - ).rejects.toThrow(/simple reserved ws:\*/); + ).rejects.toThrow(/JavaScript identifier/); + }); + + it("describes its modules for a model", () => { + const backend = new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + access: "read", + modules: { + lib: "export const x = 1;", + "ws:weather": { forecast: () => null, alerts: () => null }, + "ws:described": Object.assign(() => ({ run: () => null }), { + description: "Does a thing.", + }), + "ws:plain": () => ({ run: () => null }), + }, + }); + + expect(backend.description).toContain("ECMAScript module source"); + expect(backend.description).toContain("read-only"); + expect(backend.description).toContain("- `lib`: a bundled library."); + expect(backend.description).toContain("- `ws:weather`: exports `forecast`, `alerts`."); + expect(backend.description).toContain("- `ws:described`: Does a thing."); + expect(backend.description).toContain("- `ws:plain`: a host module."); + }); + + it("builds host modules from the Workspace services when it connects", async () => { + const db = new Database(new SQLiteTestStorage()); + initializeSchema(db, () => 0); + const git = { marker: "git" }; + let seen: unknown; + const backend = new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + modules: { + "ws:test": (host) => { + seen = host.git; + return { run: async () => null }; + }, + }, + }); + await backend.connect({ + db, + fs: new WorkspaceFilesystem(db), + git: git as never, + artifacts: undefined as never, + runtime: undefined as never, + }); + expect(seen).toBe(git); + }); + + it("does not dispatch inherited members of a host module", async () => { + const db = new Database(new SQLiteTestStorage()); + initializeSchema(db, () => 0); + const fs = new WorkspaceFilesystem(db); + await fs.mkdir("/workspace", { recursive: true }); + let response = ""; + const backend = new WorkerJavaScriptBackend({ + modules: { "ws:test": { run: async () => null } }, + loader: { + load() { + return { + getEntrypoint() { + return { + async evaluate( + _input: unknown, + host: { call(name: string, args: string): Promise }, + ) { + response = await host.call("host/ws:test.toString", JSON.stringify([])); + }, + }; + }, + }; + }, + }, + }); + const handle = await backend.connect({ + db, + fs, + git: undefined as never, + artifacts: undefined as never, + runtime: undefined as never, + }); + const execution = await handle.exec({ id: "inherited", source: "export default 1" }); + for await (const _event of execution.events) { + // Drain the run so the host call settles. + } + expect(JSON.parse(response)).toMatchObject({ + error: { message: expect.stringContaining("Unknown Workspace host module call") }, + }); + await handle.close?.(); + }); + + it("does not install ws:git or ws:artifacts unless they are configured", async () => { + const workspace = new Workspace({ + storage: new SQLiteTestStorage(), + backends: [new WorkerJavaScriptBackend({ loader: throwingLoader("must not load") })], + }); + await workspace.fs.mkdir("/workspace", { recursive: true }); + for (const specifier of ["ws:git", "ws:artifacts"]) { + await expect( + workspace.runtime.exec(`import * as m from "${specifier}"; export default () => m;`), + ).rejects.toThrow(/is not configured/); + } }); it("rejects relative imports that collide with internal Loader modules", async () => { @@ -1036,24 +1185,16 @@ describe("WorkerJavaScriptBackend", () => { expect(load).not.toHaveBeenCalled(); }); - it("rejects configured module names that collide with generated modules", async () => { - const load = vi.fn(); - const workspace = new Workspace({ - storage: new SQLiteTestStorage(), - backends: [ - new WorkerJavaScriptBackend({ - loader: { load }, - modules: { - "__workspace_entry__.js": "export default 42", - "node:fs": "export default {};", - }, - }), - ], - }); - await workspace.fs.mkdir("/workspace", { recursive: true }); - await expect( - workspace.runtime.exec("export default 1", { backend: "worker-javascript" }), - ).rejects.toThrow(/reserved module name/); - expect(load).not.toHaveBeenCalled(); - }); + it.each(["__workspace_entry__.js", "workspace-capabilities.js", "nested/lib"])( + "rejects a source module named %s at construction", + (specifier) => { + expect( + () => + new WorkerJavaScriptBackend({ + loader: throwingLoader("must not load"), + modules: { [specifier]: "export default 42" }, + }), + ).toThrow(/reserved module name/); + }, + ); }); diff --git a/packages/computer/src/backends/worker-javascript/worker-javascript.ts b/packages/computer/src/backends/worker-javascript/worker-javascript.ts index 4bd5fcfe..280ba395 100644 --- a/packages/computer/src/backends/worker-javascript/worker-javascript.ts +++ b/packages/computer/src/backends/worker-javascript/worker-javascript.ts @@ -4,29 +4,50 @@ import { dynamicWorkerEgress, type WorkspaceEgressPolicy } from "../../runtime/e import type { ModuleExecutionEnvelope, ModuleExecutionInput, + WorkspaceModule, WorkspaceModuleBackend, WorkspaceModuleBackendHandle, WorkspaceModuleBackendHost, + WorkspaceModuleFunctions, WorkspaceRuntimeAccess, WorkspaceRuntimeEvent, WorkspaceRuntimeLoader, WorkspaceRuntimeValue, - WorkspaceTrustedModule, } from "../../runtime/types.js"; import { decodeRuntimeFrames, type RuntimeFrame } from "./frames.js"; -import { buildModuleGraph } from "./module-graph.js"; +import { + assertHostModuleExports, + buildModuleGraph, + type ParsedModules, + parseModules, +} from "./module-graph.js"; export interface WorkerJavaScriptBackendOptions { loader: WorkspaceRuntimeLoader; id?: string; root?: string; access?: WorkspaceRuntimeAccess; - modules?: Record; /** - * Host-owned capability modules installed under reserved ws:* specifiers. - * Caller source may import them, but cannot provide or replace them. + * Modules caller source can import by specifier. + * + * A string value is JavaScript source bundled into the isolate, such as + * a library build. An object of functions, or a factory that builds + * one, is a host module: it runs in the Durable Object under a `ws:*` + * specifier, and each function becomes a named export. + * + * ```ts + * modules: { + * "tar-stream": TAR_STREAM_BUNDLE, + * "ws:git": createGitModule(), + * "ws:weather": { forecast: ([city]) => lookUpForecast(String(city)) }, + * } + * ``` + * + * `node:fs` and `node:fs/promises` are always installed and cannot be + * replaced. The constructor throws when a specifier is not allowed, and + * connecting throws when a host module's export names are not allowed. */ - trustedModules?: Record<`ws:${string}`, WorkspaceTrustedModule>; + modules?: Record; defaultTimeoutMs?: number; maxTimeoutMs?: number; maxSourceBytes?: number; @@ -56,10 +77,6 @@ export interface WorkerJavaScriptBackendOptions { compatibilityFlags?: string[]; egress?: WorkspaceEgressPolicy; globalOutbound?: Fetcher | null; - /** Allow ws:git operations that can perform host-side network requests. */ - allowGitNetwork?: boolean; - /** Allow ws:artifacts imports from caller-selected remote URLs. */ - allowArtifactNetwork?: boolean; } type ResolvedWorkerJavaScriptBackendOptions = Required< @@ -90,8 +107,9 @@ type ResolvedWorkerJavaScriptBackendOptions = Required< | "compatibilityFlags" > > & - Omit & { + Omit & { egress: WorkspaceEgressPolicy; + modules: ParsedModules; }; interface WorkspaceExecutionContext { @@ -139,6 +157,8 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { readonly protocol = "module" as const; readonly type = "worker-javascript"; readonly callable = true; + /** What this backend tells a model: the source language and every importable module. */ + readonly description: string; readonly id: string; readonly #options: ResolvedWorkerJavaScriptBackendOptions; @@ -198,6 +218,7 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { this.#options = { ...backendOptions, egress: resolvedEgress, + modules: parseModules(options.modules ?? {}), root: options.root ?? "/workspace", access: options.access ?? "read-write", defaultTimeoutMs, @@ -222,6 +243,14 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { compatibilityDate, compatibilityFlags: options.compatibilityFlags ?? ["nodejs_compat"], }; + this.description = [ + "`command` is ECMAScript module source, run in an isolated JavaScript runtime. Relative imports resolve from `cwd` in the workspace.", + ...(resolvedEgress.mode === "none" ? ["Code has no direct network access."] : []), + ...(this.#options.access === "read" ? ["The workspace is read-only here."] : []), + "", + "Modules code can import:", + this.#options.modules.description, + ].join("\n"); } async connect(host: WorkspaceModuleBackendHost): Promise { @@ -232,6 +261,7 @@ export class WorkerJavaScriptBackend implements WorkspaceModuleBackend { class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { readonly #options: ResolvedWorkerJavaScriptBackendOptions; readonly #host: WorkspaceModuleBackendHost; + readonly #hostModuleFunctions: ReadonlyMap; readonly #records = new Map(); readonly #pendingIds = new Set(); #closed = false; @@ -243,6 +273,13 @@ class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { constructor(options: ResolvedWorkerJavaScriptBackendOptions, host: WorkspaceModuleBackendHost) { this.#options = options; this.#host = host; + const functions = new Map(); + for (const [specifier, factory] of options.modules.host) { + const built = factory({ git: host.git, artifacts: host.artifacts, runtime: host.runtime }); + assertHostModuleExports(specifier, built); + functions.set(specifier, built); + } + this.#hostModuleFunctions = functions; host.db.run(` CREATE TABLE IF NOT EXISTS workspace_runtime_executions ( backend TEXT NOT NULL, @@ -353,8 +390,8 @@ class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { source: input.source, cwd: input.cwd ?? this.#options.root, capability, - configuredModules: this.#options.modules ?? {}, - trustedModuleNames: Object.keys(this.#options.trustedModules ?? {}), + configuredModules: this.#options.modules.source, + hostModules: this.#hostModuleFunctions, maxSourceBytes: this.#options.maxSourceBytes, maxCapabilityBytes: this.#options.maxCapabilityBytes, }); @@ -389,11 +426,7 @@ class JavaScriptBackendHandle implements WorkspaceModuleBackendHandle { this.#records.set(id, record); try { const bridge = new WorkspaceRuntimeBridge(capability, { - git: this.#host.git, - artifacts: this.#host.artifacts, - trustedModules: this.#options.trustedModules, - allowGitNetwork: this.#options.allowGitNetwork ?? false, - allowArtifactNetwork: this.#options.allowArtifactNetwork ?? false, + hostModules: this.#hostModuleFunctions, maxPayloadBytes: this.#options.maxCapabilityBytes, maxCallDurationMs: this.#options.maxHostCallMs, maxConcurrentCalls: this.#options.maxConcurrentCapabilityCalls, diff --git a/packages/computer/src/index.ts b/packages/computer/src/index.ts index 70a134c7..fc863c0a 100644 --- a/packages/computer/src/index.ts +++ b/packages/computer/src/index.ts @@ -73,9 +73,15 @@ export type { WorkspaceEgressPolicy } from "./runtime/egress.js"; export type { ModuleExecutionEnvelope, ModuleExecutionInput, + WorkspaceModule, WorkspaceModuleBackend, WorkspaceModuleBackendHandle, WorkspaceModuleBackendHost, + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunction, + WorkspaceModuleFunctions, + WorkspaceModuleHost, WorkspaceRegisteredBackend, WorkspaceRuntimeAccess, WorkspaceRuntimeDisposeOptions, @@ -88,7 +94,6 @@ export type { WorkspaceRuntimeResult, WorkspaceRuntimeStatus, WorkspaceRuntimeValue, - WorkspaceTrustedModule, } from "./runtime/types.js"; export { decodeRuntimeEvents, encodeRuntimeEvent } from "./runtime/wire.js"; export { type RawShellValue, type ShellValue, sh, shellQuote } from "./sh.js"; diff --git a/packages/computer/src/modules/artifacts.ts b/packages/computer/src/modules/artifacts.ts new file mode 100644 index 00000000..e577c5ef --- /dev/null +++ b/packages/computer/src/modules/artifacts.ts @@ -0,0 +1,83 @@ +// `ws:artifacts`: the Workspace's Artifacts client for isolate JavaScript. +// +// import { create, get, list, importArtifact, deleteArtifact } from "ws:artifacts"; +// +// Calls that change Artifacts need a read-write backend. Importing from +// a caller-chosen URL is denied unless the module is created with +// `allowNetwork: true`: the request runs from the host, so the +// isolate's own egress settings do not stop it. + +import type { ArtifactClient } from "../artifacts/index.js"; +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceModuleHost, +} from "../runtime/types.js"; + +/** Options for {@link createArtifactsModule}. */ +export interface ArtifactsModuleOptions { + /** + * Allow `importArtifact` from a caller-chosen remote URL. Defaults to + * `false`. The request runs from the host, so the JavaScript + * backend's egress settings do not apply to it. + */ + readonly allowNetwork?: boolean; +} + +/** + * Build the `ws:artifacts` host module over the Workspace's Artifacts client. + * + * It exports `create`, `get`, `list`, `importArtifact`, and + * `deleteArtifact`, with the same arguments as the matching + * `ArtifactClient` methods. + * + * @param options - Whether remote imports are allowed. + * @returns The module to pass as `modules["ws:artifacts"]`. + */ +export function createArtifactsModule( + options: ArtifactsModuleOptions = {}, +): WorkspaceModuleFactory { + const allowNetwork = options.allowNetwork ?? false; + + // SAFETY for the casts below: the isolate's arguments pass through to the Artifacts client, as they did when ws:artifacts was built in. The client checks its own inputs. + const create = (host: WorkspaceModuleHost): WorkspaceModuleFunctions => ({ + create([name, createOptions], context) { + requireWrite(context, "Artifacts create"); + return host.artifacts.create( + String(name), + createOptions as unknown as Parameters[1], + ); + }, + get([name]) { + return host.artifacts.get(String(name)); + }, + list() { + return host.artifacts.list(); + }, + importArtifact([name, source, importOptions], context) { + requireWrite(context, "Artifacts import"); + if (!allowNetwork) { + throw new Error("Artifacts import requires createArtifactsModule({ allowNetwork: true })."); + } + return host.artifacts.import( + String(name), + source as unknown as Parameters[1], + importOptions as unknown as Parameters[2], + ); + }, + deleteArtifact([name], context) { + requireWrite(context, "Artifacts delete"); + return host.artifacts.delete(String(name)); + }, + }); + return Object.assign(create, { + description: `Git repositories stored in Cloudflare Artifacts: \`create(name)\`, \`get(name)\`, \`list()\`, \`importArtifact(name, source)\`, and \`deleteArtifact(name)\`.${allowNetwork ? "" : " Importing from a remote URL is not allowed."}`, + }); +} + +function requireWrite(context: WorkspaceModuleCallContext, operation: string) { + if (context.access !== "read-write") { + throw new Error(`${operation} requires Workspace write access.`); + } +} diff --git a/packages/computer/src/modules/git.ts b/packages/computer/src/modules/git.ts new file mode 100644 index 00000000..cce42eed --- /dev/null +++ b/packages/computer/src/modules/git.ts @@ -0,0 +1,145 @@ +// `ws:git`: the Workspace's Git client for isolate JavaScript. +// +// import { status, diff, log, clone, cli } from "ws:git"; +// +// Every path the isolate passes is confined to the JavaScript backend's +// root. Commands that change the repository need a read-write backend, +// and commands that reach the network are denied unless the module is +// created with `allowNetwork: true`: they run from the host, so the +// isolate's own egress settings do not stop them. + +import type { GitClient } from "../git/index.js"; +import type { + WorkspaceModuleCallContext, + WorkspaceModuleFactory, + WorkspaceModuleFunctions, + WorkspaceModuleHost, + WorkspaceRuntimeValue, +} from "../runtime/types.js"; + +const NETWORK_COMMANDS = new Set(["clone", "fetch", "pull", "push", "ls-remote", "submodule"]); + +/** Options for {@link createGitModule}. */ +export interface GitModuleOptions { + /** + * Allow `clone` and network `cli` commands such as `fetch` and `push`. + * Defaults to `false`. These requests run from the host, so the + * JavaScript backend's egress settings do not apply to them. + */ + readonly allowNetwork?: boolean; +} + +/** + * Build the `ws:git` host module over the Workspace's Git client. + * + * It exports `clone`, `diff`, `status`, `log`, and `cli`. Each takes the + * same options object as the matching `GitClient` method, with `dir` or + * `cwd` resolved against the backend root. + * + * @param options - Whether network commands are allowed. + * @returns The module to pass as `modules["ws:git"]`. + */ +export function createGitModule(options: GitModuleOptions = {}): WorkspaceModuleFactory { + const allowNetwork = options.allowNetwork ?? false; + const requireNetwork = (operation: string) => { + if (!allowNetwork) { + throw new Error(`${operation} requires createGitModule({ allowNetwork: true }).`); + } + }; + + const create = (host: WorkspaceModuleHost): WorkspaceModuleFunctions => ({ + async clone([value], context) { + requireWrite(context, "Git clone"); + requireNetwork("Git clone"); + // SAFETY: The isolate's options object passes through to the Git client, as it did when ws:git was built in. The client checks its own options; only the path is rewritten here. + return host.git.clone( + (await withDir(value, context, true)) as unknown as Parameters[0], + ); + }, + async diff([value], context) { + // SAFETY: As for clone. + return host.git.diff((await withDir(value, context)) as Parameters[0]); + }, + async status([value], context) { + // SAFETY: As for clone. + return host.git.status((await withDir(value, context)) as Parameters[0]); + }, + async log([value], context) { + // SAFETY: As for clone. + return host.git.log((await withDir(value, context)) as Parameters[0]); + }, + async cli([value], context) { + requireWrite(context, "Git CLI"); + // SAFETY: As for clone. + const input = (value ?? {}) as unknown as Parameters[0]; + const { argv, cwd } = leadingDirectory(input.argv ?? [], input.cwd ?? "."); + assertSafeCliArguments(argv); + if (argv.some((argument) => NETWORK_COMMANDS.has(argument.toLowerCase()))) { + requireNetwork("Git CLI network command"); + } + return host.git.cli({ + ...input, + argv, + cwd: await context.resolvePath(cwd, { allowMissing: true }), + }); + }, + }); + return Object.assign(create, { + description: `The workspace's Git repository tools: \`status({ dir })\`, \`diff({ dir })\`, \`log({ dir, depth })\`, \`clone({ url, dir })\`, and \`cli({ argv, cwd })\` for any other git subcommand.${allowNetwork ? "" : " Network commands such as clone, fetch, and push are not allowed."}`, + }); +} + +function requireWrite(context: WorkspaceModuleCallContext, operation: string) { + if (context.access !== "read-write") { + throw new Error(`${operation} requires Workspace write access.`); + } +} + +async function withDir( + value: WorkspaceRuntimeValue | undefined, + context: WorkspaceModuleCallContext, + allowMissing = false, +): Promise> { + const options = value !== null && typeof value === "object" && !Array.isArray(value) ? value : {}; + return { + ...options, + dir: await context.resolvePath(typeof options.dir === "string" ? options.dir : ".", { + allowMissing, + }), + }; +} + +// Agents often run `git -C `. A leading `-C` becomes +// the working directory, so it goes through the same confinement as +// `cwd` instead of reaching the Git client as a path override. +function leadingDirectory(argv: string[], cwd: string): { argv: string[]; cwd: string } { + if (argv[0] !== "-C") return { argv, cwd }; + const directory = argv[1]; + if (directory === undefined || directory === "") { + throw new Error("Git CLI option '-C' requires a value."); + } + return { + argv: argv.slice(2), + cwd: directory.startsWith("/") ? directory : `${cwd.replace(/\/+$/, "")}/${directory}`, + }; +} + +// Any other path override would let a command escape the confined +// directory. +function assertSafeCliArguments(argv: string[] | undefined) { + if ( + argv?.some( + (argument) => + argument === "-C" || + argument.startsWith("-C") || + argument === "--git-dir" || + argument.startsWith("--git-dir=") || + argument === "--work-tree" || + argument.startsWith("--work-tree="), + ) + ) { + throw new Error( + "Git CLI path overrides are not available inside a confined Workspace runtime.", + ); + } +} diff --git a/packages/computer/src/runtime/bridge.test.ts b/packages/computer/src/runtime/bridge.test.ts index 18f330a0..94d16a4f 100644 --- a/packages/computer/src/runtime/bridge.test.ts +++ b/packages/computer/src/runtime/bridge.test.ts @@ -4,7 +4,7 @@ import { WorkspaceRuntimeBridge } from "./bridge.js"; import type { WorkspaceRuntimeCapability } from "./capability.js"; const encoder = new TextEncoder(); -const args = JSON.stringify(["run", "value"]); +const args = JSON.stringify(["value"]); function bridge(limits: { maxCalls?: number; @@ -13,13 +13,7 @@ function bridge(limits: { }) { return new WorkspaceRuntimeBridge({} as WorkspaceRuntimeCapability, { ...limits, - trustedModules: { - "ws:test": { - async call() { - return "ok"; - }, - }, - }, + hostModules: new Map([["ws:test", { run: async () => "ok" }]]), }); } @@ -30,9 +24,9 @@ async function message(response: Promise) { describe("WorkspaceRuntimeBridge cumulative limits", () => { it("accepts the configured call count and rejects the next call", async () => { const target = bridge({ maxCalls: 2 }); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toContain( + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toContain( "exceeds 2 capability calls", ); }); @@ -40,20 +34,20 @@ describe("WorkspaceRuntimeBridge cumulative limits", () => { it("accepts requests at the cumulative byte boundary and rejects the next request", async () => { const bytes = encoder.encode(args).byteLength; const target = bridge({ maxTotalRequestBytes: bytes * 2 }); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toContain( + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toContain( `requests exceed ${bytes * 2} bytes`, ); }); it("accepts responses at the cumulative byte boundary and rejects the next response", async () => { - const sample = await bridge({}).call("trusted/ws:test.call", args); + const sample = await bridge({}).call("host/ws:test.run", args); const bytes = encoder.encode(sample).byteLength; const target = bridge({ maxTotalResponseBytes: bytes * 2 }); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toBeUndefined(); - await expect(message(target.call("trusted/ws:test.call", args))).resolves.toContain( + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toBeUndefined(); + await expect(message(target.call("host/ws:test.run", args))).resolves.toContain( `responses exceed ${bytes * 2} bytes`, ); }); diff --git a/packages/computer/src/runtime/bridge.ts b/packages/computer/src/runtime/bridge.ts index 552e28f0..cfe4a6e8 100644 --- a/packages/computer/src/runtime/bridge.ts +++ b/packages/computer/src/runtime/bridge.ts @@ -1,17 +1,11 @@ import { RpcTarget } from "cloudflare:workers"; -import type { ArtifactClient } from "../artifacts/index.js"; -import type { GitClient } from "../git/index.js"; import { assertRuntimeValue, type WorkspaceRuntimeCapability } from "./capability.js"; -import type { WorkspaceTrustedModule } from "./types.js"; +import type { WorkspaceModuleCallContext, WorkspaceModuleFunctions } from "./types.js"; export class WorkspaceRuntimeBridge extends RpcTarget { readonly #capability: WorkspaceRuntimeCapability; - readonly #git: GitClient | undefined; - readonly #artifacts: ArtifactClient | undefined; - readonly #trustedModules: Record; - readonly #allowGitNetwork: boolean; - readonly #allowArtifactNetwork: boolean; + readonly #hostModules: ReadonlyMap; readonly #maxPayloadBytes: number; readonly #maxCallDurationMs: number; readonly #maxConcurrentCalls: number; @@ -31,11 +25,7 @@ export class WorkspaceRuntimeBridge extends RpcTarget { constructor( capability: WorkspaceRuntimeCapability, integrations: { - git?: GitClient; - artifacts?: ArtifactClient; - trustedModules?: Record; - allowGitNetwork?: boolean; - allowArtifactNetwork?: boolean; + hostModules?: ReadonlyMap; maxPayloadBytes?: number; maxCallDurationMs?: number; maxConcurrentCalls?: number; @@ -48,11 +38,7 @@ export class WorkspaceRuntimeBridge extends RpcTarget { ) { super(); this.#capability = capability; - this.#git = integrations.git; - this.#artifacts = integrations.artifacts; - this.#trustedModules = integrations.trustedModules ?? {}; - this.#allowGitNetwork = integrations.allowGitNetwork ?? false; - this.#allowArtifactNetwork = integrations.allowArtifactNetwork ?? false; + this.#hostModules = integrations.hostModules ?? new Map(); this.#maxPayloadBytes = integrations.maxPayloadBytes ?? 1024 * 1024; this.#maxCallDurationMs = integrations.maxCallDurationMs ?? 30_000; this.#maxConcurrentCalls = integrations.maxConcurrentCalls ?? 16; @@ -114,10 +100,14 @@ export class WorkspaceRuntimeBridge extends RpcTarget { const operation = encodeCall(async () => { const encodedArgs = JSON.parse(argsJson) as unknown[]; const args = encodedArgs.map(decodeBridgeValue); - if (name.startsWith("git.")) return this.#callGit(name.slice(4), args); - if (name.startsWith("artifacts.")) return this.#callArtifacts(name.slice(10), args); - if (name.startsWith("trusted/")) { - return this.#callTrusted(name, args, { signal: abort.signal, deadline }); + if (name.startsWith("host/")) { + return this.#callHostModule(name, args, { + signal: abort.signal, + deadline, + access: this.#capability.access, + resolvePath: (path, options) => + this.#capability.resolveConfined(path, options?.allowMissing ?? false), + }); } const operation = name.startsWith("fs.") ? name.slice(3) : name; switch (operation) { @@ -221,140 +211,28 @@ export class WorkspaceRuntimeBridge extends RpcTarget { } } - async #callTrusted( - name: string, - args: unknown[], - context: { signal: AbortSignal; deadline: number }, - ) { - const suffix = ".call"; - const specifier = name.endsWith(suffix) ? name.slice("trusted/".length, -suffix.length) : ""; - const trusted = this.#trustedModules[specifier]; - if (!trusted) { - throw new Error(`Unknown trusted Workspace module call ${JSON.stringify(name)}.`); + // `name` is `host/.`. Function names are + // identifiers and never contain a dot, so the last dot splits them + // from a specifier such as `ws:a.b`. Own-property checks keep + // isolate code from reaching `toString` or other inherited members. + async #callHostModule(name: string, args: unknown[], context: WorkspaceModuleCallContext) { + const target = name.slice("host/".length); + const dot = target.lastIndexOf("."); + const specifier = dot === -1 ? "" : target.slice(0, dot); + const functionName = dot === -1 ? "" : target.slice(dot + 1); + const functions = this.#hostModules.get(specifier); + const fn = + functions !== undefined && Object.hasOwn(functions, functionName) + ? functions[functionName] + : undefined; + if (typeof fn !== "function") { + throw new Error(`Unknown Workspace host module call ${JSON.stringify(name)}.`); } - const method = String(args[0]); - const callArgs = args.slice(1); - assertBridgeValues(callArgs); - const result = await trusted.call(method, callArgs, context); + assertBridgeValues(args); + const result = (await fn(args, context)) ?? null; assertBridgeValues([result]); return result; } - - async #callGit(name: string, args: unknown[]) { - if (!this.#git) throw new Error("Workspace Git is not configured for this execution."); - switch (name) { - case "clone": - this.#requireWrite("Git clone"); - this.#requireGitNetwork("Git clone"); - return this.#git - .clone( - (await this.#gitOptions(args[0], true)) as unknown as Parameters[0], - ) - .then(() => null); - case "diff": - return this.#git.diff( - (await this.#gitOptions(args[0])) as Parameters[0], - ); - case "status": - return this.#git.status( - (await this.#gitOptions(args[0])) as Parameters[0], - ); - case "log": - return this.#git.log((await this.#gitOptions(args[0])) as Parameters[0]); - case "cli": { - this.#requireWrite("Git CLI"); - const input = (args[0] ?? {}) as Parameters[0]; - assertSafeGitCliArguments(input.argv); - if (isGitNetworkCommand(input.argv)) this.#requireGitNetwork("Git CLI network command"); - return this.#git.cli({ - ...input, - cwd: await this.#capability.resolveConfined(input.cwd ?? ".", true), - }); - } - default: - throw new Error(`Unknown Workspace Git operation ${JSON.stringify(name)}.`); - } - } - - #callArtifacts(name: string, args: unknown[]) { - if (!this.#artifacts) - throw new Error("Workspace Artifacts are not configured for this execution."); - switch (name) { - case "create": - this.#requireWrite("Artifacts create"); - return this.#artifacts.create( - String(args[0]), - args[1] as Parameters[1], - ); - case "get": - return this.#artifacts.get(String(args[0])); - case "list": - return this.#artifacts.list(); - case "importArtifact": - this.#requireWrite("Artifacts import"); - if (!this.#allowArtifactNetwork) { - throw new Error( - "Artifacts import requires WorkerJavaScriptBackend allowArtifactNetwork: true.", - ); - } - return this.#artifacts.import( - String(args[0]), - args[1] as Parameters[1], - args[2] as Parameters[2], - ); - case "deleteArtifact": - this.#requireWrite("Artifacts delete"); - return this.#artifacts.delete(String(args[0])); - default: - throw new Error(`Unknown Workspace Artifacts operation ${JSON.stringify(name)}.`); - } - } - - async #gitOptions(value: unknown, allowMissing = false): Promise> { - const options = (value ?? {}) as Record; - return { - ...options, - dir: await this.#capability.resolveConfined( - typeof options.dir === "string" ? options.dir : ".", - allowMissing, - ), - }; - } - - #requireGitNetwork(operation: string) { - if (!this.#allowGitNetwork) { - throw new Error(`${operation} requires WorkerJavaScriptBackend allowGitNetwork: true.`); - } - } - - #requireWrite(operation: string) { - if (this.#capability.access !== "read-write") { - throw new Error(`${operation} requires Workspace write access.`); - } - } -} - -function assertSafeGitCliArguments(argv: string[] | undefined) { - if ( - argv?.some( - (argument) => - argument === "-C" || - argument.startsWith("-C") || - argument === "--git-dir" || - argument.startsWith("--git-dir=") || - argument === "--work-tree" || - argument.startsWith("--work-tree="), - ) - ) { - throw new Error( - "Git CLI path overrides are not available inside a confined Workspace runtime.", - ); - } -} - -function isGitNetworkCommand(argv: string[] | undefined) { - const networkCommands = new Set(["clone", "fetch", "pull", "push", "ls-remote", "submodule"]); - return argv?.some((argument) => networkCommands.has(argument.toLowerCase())) ?? false; } function assertBridgeValues( @@ -369,17 +247,19 @@ function assertBridgeValues( (typeof value === "number" && Number.isFinite(value)) ) return; - if (typeof value !== "object") - throw new Error("Trusted module values must be JSON-compatible."); - if (seen.has(value)) throw new Error("Trusted module values must be acyclic."); + if (typeof value !== "object") throw new Error("Host module values must be JSON-compatible."); + if (seen.has(value)) throw new Error("Host module values must be acyclic."); seen.add(value); if (Array.isArray(value)) for (const item of value) visit(item); else { const prototype = Object.getPrototypeOf(value); if (prototype !== Object.prototype && prototype !== null) { - throw new Error("Trusted module values must contain only plain objects."); + throw new Error("Host module values must contain only plain objects."); + } + // An undefined field is absent, as in JSON. encodeBridgeValue drops it. + for (const item of Object.values(value as Record)) { + if (item !== undefined) visit(item); } - for (const item of Object.values(value as Record)) visit(item); } seen.delete(value); }; @@ -398,7 +278,9 @@ function encodeBridgeValue(value: unknown): unknown { if (Array.isArray(value)) return wrap("array", { items: value.map(encodeBridgeValue) }); if (value && typeof value === "object") { return wrap("object", { - entries: Object.entries(value).map(([key, child]) => [key, encodeBridgeValue(child)]), + entries: Object.entries(value) + .filter(([, child]) => child !== undefined) + .map(([key, child]) => [key, encodeBridgeValue(child)]), }); } return value; diff --git a/packages/computer/src/runtime/runtime.test.ts b/packages/computer/src/runtime/runtime.test.ts index f91ca460..7cdb3a23 100644 --- a/packages/computer/src/runtime/runtime.test.ts +++ b/packages/computer/src/runtime/runtime.test.ts @@ -52,7 +52,7 @@ function replayBackend(events: WorkspaceRuntimeEvent[]): WorkspaceModuleBackendH function runtimeFor(handle: WorkspaceModuleBackendHandle): WorkspaceRuntime { return new WorkspaceRuntime({ - callableBackendIds: new Set(), + backends: new Map(), backendHandle: async () => handle, resolveBackendId: () => "backend", }); @@ -181,7 +181,7 @@ describe("WorkspaceRuntime utf8 encoding", () => { describe("WorkspaceRuntime callable gate", () => { it("rejects structured input for a non-callable backend", async () => { const runtime = new WorkspaceRuntime({ - callableBackendIds: new Set(), + backends: new Map(), backendHandle: async () => moduleHandleStub(), resolveBackendId: () => "worker-shell", }); @@ -194,7 +194,7 @@ describe("WorkspaceRuntime callable gate", () => { it("accepts structured input for a callable module backend", async () => { const handle = moduleHandleStub(); const runtime = new WorkspaceRuntime({ - callableBackendIds: new Set(["worker-javascript"]), + backends: new Map([["worker-javascript", { callable: true }]]), backendHandle: async () => handle, resolveBackendId: () => "worker-javascript", }); diff --git a/packages/computer/src/runtime/runtime.ts b/packages/computer/src/runtime/runtime.ts index ec43d5df..f74965f8 100644 --- a/packages/computer/src/runtime/runtime.ts +++ b/packages/computer/src/runtime/runtime.ts @@ -14,7 +14,8 @@ import type { } from "./types.js"; interface WorkspaceRuntimeRouterOptions { - callableBackendIds: ReadonlySet; + // What each registered backend says about itself. + backends: ReadonlyMap; backendHandle: (id: string) => Promise; resolveBackendId: (id: string | undefined) => string; } @@ -39,7 +40,15 @@ export class WorkspaceRuntime { // this to know whether a backend is callable without the caller // having to declare it a second time. isCallable(id: string): boolean { - return this.#options.callableBackendIds.has(id); + return this.#options.backends.get(id)?.callable === true; + } + + // What the named backend says about itself for a model: its source + // language and, for the JavaScript backend, the modules code can + // import. The exec tool adds it to the backend's entry so a caller + // does not have to repeat it. + describe(id: string): string | undefined { + return this.#options.backends.get(id)?.description; } exec(source: string): Promise>; diff --git a/packages/computer/src/runtime/types.ts b/packages/computer/src/runtime/types.ts index c92a9f64..84bebae0 100644 --- a/packages/computer/src/runtime/types.ts +++ b/packages/computer/src/runtime/types.ts @@ -4,15 +4,83 @@ import type { ExecEncoding, ExecSyncResult, KillSignal } from "../shell.js"; export type WorkspaceRuntimeAccess = "read" | "read-write"; -export interface WorkspaceTrustedModule { - /** Dispatch a call made through a host-installed reserved ws:* module. */ - call( - method: string, - args: WorkspaceRuntimeValue[], - context?: { signal: AbortSignal; deadline: number }, - ): Promise; +/** Per-call context the backend passes to every host module function. */ +export interface WorkspaceModuleCallContext { + /** Aborts when the call passes its deadline or the execution is cancelled. */ + readonly signal: AbortSignal; + /** Epoch milliseconds after which the caller stops waiting for this call. */ + readonly deadline: number; + /** Access level of the backend running the call. */ + readonly access: WorkspaceRuntimeAccess; + /** + * Resolve a path the isolate passed against the backend's root. + * Rejects paths that escape the root or pass through a symlink. + * + * @param path - An absolute path, or one relative to the backend root. + * @param options - Set `allowMissing` when the path may not exist yet. + * @returns The confined absolute path. + */ + resolvePath(path: string, options?: { readonly allowMissing?: boolean }): Promise; } +/** + * One host function exported by a host module. + * + * `args` holds the arguments the isolate passed, decoded from the wire. + * They come from untrusted code, so parse them before use. The function + * may return a value or a promise of one. The result must be + * JSON-compatible: the bridge checks it at runtime, treats `undefined` + * as `null`, and drops `undefined` object fields, the way + * `JSON.stringify` does. + */ +export type WorkspaceModuleFunction = ( + args: readonly WorkspaceRuntimeValue[], + context: WorkspaceModuleCallContext, +) => unknown; + +/** Named functions a host module exports into the isolate. */ +export type WorkspaceModuleFunctions = Readonly>; + +/** Workspace services a host module factory can build its functions from. */ +export interface WorkspaceModuleHost { + /** The Workspace's Git client. Throws on use when Git is not configured. */ + readonly git: import("../git/index.js").GitClient; + /** The Workspace's Artifacts client. Throws on use when Artifacts is not configured. */ + readonly artifacts: import("../artifacts/index.js").ArtifactClient; + /** The Workspace runtime, for running commands on other backends. */ + readonly runtime: import("./runtime.js").WorkspaceRuntime; +} + +/** + * Builds a host module's functions from the Workspace's services. The + * backend calls it once when it connects to its Workspace. The + * prebuilt modules in `@cloudflare/computer/modules/*` are factories. + */ +export interface WorkspaceModuleFactory { + (host: WorkspaceModuleHost): WorkspaceModuleFunctions; + /** + * What the module does and how to call it, for a model. The + * JavaScript backend adds it to its own description, which the exec + * tool shows. Objects of functions are listed by their export names. + */ + readonly description?: string; +} + +/** + * A module caller source can import. + * + * - A string is JavaScript source bundled into the isolate, with no host access. + * - An object of functions is a host module. Its functions run in the + * Durable Object and each becomes a named export: + * `{ "ws:weather": { forecast } }` lets code write + * `import { forecast } from "ws:weather"`. + * - A factory is a host module that needs the Workspace's Git client, + * Artifacts client, or runtime, such as `createGitModule()`. + * + * Host modules must use a `ws:*` specifier. + */ +export type WorkspaceModule = string | WorkspaceModuleFunctions | WorkspaceModuleFactory; + export type WorkspaceRuntimeValue = | null | boolean @@ -181,13 +249,19 @@ export interface WorkspaceModuleBackendHandle { close?(): Promise; } -export type WorkspaceModuleBackendHost = import("../backend.js").WorkspaceBackendHost; +/** What a module backend receives when it connects to its Workspace. */ +export type WorkspaceModuleBackendHost = import("../backend.js").WorkspaceBackendHost & { + /** The Workspace runtime, handed to host modules. */ + readonly runtime: import("./runtime.js").WorkspaceRuntime; +}; export interface WorkspaceModuleBackend { readonly protocol: "module"; readonly id: string; readonly type: string; readonly callable?: boolean; + /** What the backend tells a model about itself. Shown by the exec tool. */ + readonly description?: string; connect(host: WorkspaceModuleBackendHost): Promise; } diff --git a/packages/computer/src/workspace.ts b/packages/computer/src/workspace.ts index 25f797a2..d8d4b325 100644 --- a/packages/computer/src/workspace.ts +++ b/packages/computer/src/workspace.ts @@ -219,8 +219,7 @@ export class Workspace { readonly #backends: WorkspaceBackend[]; readonly #backendsById: Map; readonly #moduleBackendsById: Map; - readonly #registeredBackendIds: Set; - readonly #callableBackendIds: Set; + readonly #registeredBackends: Map; readonly #defaultBackendId: string | undefined; readonly #observer: WorkspaceObserver; readonly #syncLogger: SyncLogger; @@ -305,19 +304,16 @@ export class Workspace { this.#moduleBackendsById = new Map( registered.filter(isModuleBackend).map((backend) => [backend.id, backend]), ); - this.#registeredBackendIds = new Set(); - this.#callableBackendIds = new Set( - registered.filter((backend) => backend.callable === true).map((backend) => backend.id), - ); + this.#registeredBackends = new Map(); for (const backend of registered) { - if (this.#registeredBackendIds.has(backend.id)) { + if (this.#registeredBackends.has(backend.id)) { throw new Error( `Workspace: duplicate backend id ${JSON.stringify(backend.id)}. ` + "Pass an explicit `id` on each backend's constructor options to " + "distinguish them.", ); } - this.#registeredBackendIds.add(backend.id); + this.#registeredBackends.set(backend.id, backend); } this.#defaultBackendId = registered[0]?.id; this.#observer = options.observer ?? noopObserver; @@ -429,7 +425,7 @@ export class Workspace { get runtime(): WorkspaceRuntime { if (!this.#runtime) { this.#runtime = new WorkspaceRuntime({ - callableBackendIds: this.#callableBackendIds, + backends: this.#registeredBackends, backendHandle: (id) => this.#backendHandleFor(id), resolveBackendId: (id) => this.#resolveBackendId(id) ?? "", }); @@ -861,13 +857,13 @@ export class Workspace { // workspace; throws on an unknown id. Omitted ids fall through // to the first backend in the list (the default). #resolveBackendId(id: string | undefined): string | undefined { - if (this.#registeredBackendIds.size === 0) return undefined; + if (this.#registeredBackends.size === 0) return undefined; const target = id ?? this.#defaultBackendId; if (target === undefined) return undefined; - if (!this.#registeredBackendIds.has(target)) { + if (!this.#registeredBackends.has(target)) { throw new Error( `Workspace: no backend with id ${JSON.stringify(target)}. ` + - `Configured backends: ${[...this.#registeredBackendIds].map((key) => JSON.stringify(key)).join(", ") || ""}.`, + `Configured backends: ${[...this.#registeredBackends.keys()].map((key) => JSON.stringify(key)).join(", ") || ""}.`, ); } return target; @@ -998,6 +994,7 @@ export class Workspace { fs: this.#fs, git: this.#gitFactory ? this.git : DISABLED_GIT_CLIENT, artifacts: this.#artifacts, + runtime: this.runtime, }), ) .then(async (handle) => { diff --git a/packages/computer/tests/script-runner-worker.ts b/packages/computer/tests/script-runner-worker.ts index 244b548b..399cd780 100644 --- a/packages/computer/tests/script-runner-worker.ts +++ b/packages/computer/tests/script-runner-worker.ts @@ -7,6 +7,8 @@ import type { WorkspaceStub, } from "../src/index.js"; import { Workspace } from "../src/index.js"; +import { createArtifactsModule } from "../src/modules/artifacts.js"; +import { createGitModule } from "../src/modules/git.js"; export interface Env { HOST: DurableObjectNamespace; @@ -30,23 +32,37 @@ export class HostDO extends DurableObject { maxConcurrentCapabilityCalls: 2, modules: { "math-kit": "export const double = (value) => value * 2;", - }, - trustedModules: { + "ws:git": createGitModule(), + "ws:artifacts": createArtifactsModule(), "ws:test-host": { - async call(method, args) { - if (method === "invalid-result") return new Date() as never; - if (method === "large-error") throw new Error("x".repeat(5000)); - if (method === "slow") { - await new Promise((resolve) => setTimeout(resolve, 20)); - return null; - } - if (method === "marker") { - return { - __workspace_codec__: { version: 1, type: "bytes", data: [1] }, - keep: true, - }; - } - return { method, args }; + async echo(args) { + return { args: [...args] }; + }, + async sum(args) { + return args.reduce( + (total, value) => total + (typeof value === "number" ? value : 0), + 0, + ); + }, + async delete(args) { + return { deleted: args[0] ?? null }; + }, + async invalidResult() { + // SAFETY: The test hands the bridge a non-JSON value on purpose to check that it rejects it. + return new Date() as never; + }, + async largeError() { + throw new Error("x".repeat(5000)); + }, + async slow() { + await new Promise((resolve) => setTimeout(resolve, 20)); + return null; + }, + async marker() { + return { + __workspace_codec__: { version: 1, type: "bytes", data: [1] }, + keep: true, + }; }, }, }, diff --git a/packages/computer/tests/script-runner.test.ts b/packages/computer/tests/script-runner.test.ts index 61c8b63f..1e67d66d 100644 --- a/packages/computer/tests/script-runner.test.ts +++ b/packages/computer/tests/script-runner.test.ts @@ -49,14 +49,14 @@ describe("WorkspaceRuntime", () => { }); }); - it("executes an ES module with configured and trusted modules", async () => { + it("executes an ES module with source and host modules", async () => { const response = await runtime({ source: ` import { double } from "math-kit"; import fs from "node:fs/promises"; import { promises as nodeFs } from "node:fs"; import * as git from "ws:git"; - import { call } from "ws:test-host"; + import { echo, sum, delete as remove } from "ws:test-host"; export default async function main(input) { const value = double(input.value); await fs.writeFile("/workspace/runtime-result.txt", String(value)); @@ -65,7 +65,9 @@ describe("WorkspaceRuntime", () => { value, persisted: await fs.readFile("/workspace/runtime-result.txt", "utf8"), gitExitCode: initialized.exitCode, - trusted: await call("echo", input.value), + trusted: await echo(input.value, "second"), + summed: await sum(1, 2, 3), + removed: await remove("/workspace/gone.txt"), nodeFs: { isFile: (await nodeFs.stat("/workspace/runtime-result.txt")).isFile(), entries: await nodeFs.readdir("/workspace"), @@ -86,7 +88,9 @@ describe("WorkspaceRuntime", () => { value: 42, persisted: "42", gitExitCode: 0, - trusted: { method: "echo", args: [21] }, + trusted: { args: [21, "second"] }, + summed: 6, + removed: { deleted: "/workspace/gone.txt" }, nodeFs: { isFile: true, entries: expect.arrayContaining(["runtime-result.txt"]), @@ -100,14 +104,14 @@ describe("WorkspaceRuntime", () => { const response = await runtime({ source: ` import fs from "node:fs/promises"; - import { call } from "ws:test-host"; + import { marker } from "ws:test-host"; export default async () => { await fs.writeFile("/workspace/bytes.bin", new Uint8Array([0, 127, 255])); const value = await fs.readFile("/workspace/bytes.bin"); return { isBytes: value instanceof Uint8Array, bytes: Array.from(value), - marker: await call("marker"), + marker: await marker(), }; }; `, @@ -230,11 +234,11 @@ describe("WorkspaceRuntime", () => { expect(payload.result.stdout).toContain("stdio truncated"); }); - it("bounds oversized trusted-module error responses", async () => { + it("bounds oversized host module error responses", async () => { const response = await runtime({ source: ` - import { call } from "ws:test-host"; - export default () => call("large-error"); + import { largeError } from "ws:test-host"; + export default () => largeError(); `, cwd: "/workspace", }); @@ -261,9 +265,9 @@ describe("WorkspaceRuntime", () => { it("bounds concurrent host capability calls", async () => { const response = await runtime({ source: ` - import { call } from "ws:test-host"; + import { slow } from "ws:test-host"; export default async () => { - const settled = await Promise.allSettled([call("slow"), call("slow"), call("slow")]); + const settled = await Promise.allSettled([slow(), slow(), slow()]); return settled.map((item) => item.status); }; `, @@ -274,11 +278,11 @@ describe("WorkspaceRuntime", () => { expect(JSON.parse(text).result.value).toEqual(["fulfilled", "fulfilled", "rejected"]); }); - it("rejects non-plain results from host trusted modules", async () => { + it("rejects non-plain results from host modules", async () => { const response = await runtime({ source: ` - import { call } from "ws:test-host"; - export default () => call("invalid-result"); + import { invalidResult } from "ws:test-host"; + export default () => invalidResult(); `, cwd: "/workspace", }); @@ -292,6 +296,42 @@ describe("WorkspaceRuntime", () => { }); }); + it("exposes only the functions a host module declares", async () => { + const response = await runtime({ + source: ` + import * as host from "ws:test-host"; + export default () => Object.keys(host).sort(); + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text).result.value).toEqual([ + "delete", + "echo", + "invalidResult", + "largeError", + "marker", + "slow", + "sum", + ]); + }); + + it("fails to link an import the host module does not export", async () => { + const response = await runtime({ + source: ` + import { call } from "ws:test-host"; + export default () => call("echo"); + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text), text).toMatchObject({ + result: { status: "failed", stderr: expect.stringContaining("does not provide an export") }, + }); + }); + it("does not expose unrestricted host operations through the node:fs dispatcher", async () => { const response = await runtime({ source: ` @@ -356,7 +396,7 @@ describe("WorkspaceRuntime", () => { }); }); - it("confines trusted Git operations to the backend root", async () => { + it("confines ws:git operations to the backend root", async () => { const response = await runtime({ source: ` import { status } from "ws:git"; @@ -374,7 +414,7 @@ describe("WorkspaceRuntime", () => { }); }); - it("rejects Git CLI path overrides that bypass the runtime root", async () => { + it("confines a leading Git CLI -C to the runtime root", async () => { const response = await runtime({ source: ` import { cli } from "ws:git"; @@ -384,6 +424,43 @@ describe("WorkspaceRuntime", () => { }); const text = await response.text(); expect(response.status, text).toBe(200); + expect(JSON.parse(text), text).toMatchObject({ + result: { + status: "failed", + stderr: expect.stringContaining("must stay under /workspace"), + }, + }); + }); + + it("runs a Git CLI command in a leading -C directory", async () => { + const response = await runtime({ + source: ` + import { cli } from "ws:git"; + export default async () => { + await cli({ cwd: "/workspace", argv: ["init", "c-repo"] }); + return cli({ cwd: "/workspace", argv: ["-C", "c-repo", "rev-parse", "--show-toplevel"] }); + }; + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); + expect(JSON.parse(text).result, text).toMatchObject({ + status: "completed", + value: { exitCode: 0, stdout: expect.stringContaining("/workspace/c-repo") }, + }); + }); + + it("rejects Git CLI path overrides after the subcommand", async () => { + const response = await runtime({ + source: ` + import { cli } from "ws:git"; + export default () => cli({ cwd: "/workspace", argv: ["status", "--git-dir=/outside"] }); + `, + cwd: "/workspace", + }); + const text = await response.text(); + expect(response.status, text).toBe(200); expect(JSON.parse(text), text).toMatchObject({ result: { status: "failed", @@ -405,7 +482,7 @@ describe("WorkspaceRuntime", () => { expect(JSON.parse(text), text).toMatchObject({ result: { status: "failed", - stderr: expect.stringContaining("allowArtifac"), + stderr: expect.stringContaining("createArtifactsModule"), }, }); }); @@ -423,12 +500,12 @@ describe("WorkspaceRuntime", () => { expect(JSON.parse(text), text).toMatchObject({ result: { status: "failed", - stderr: expect.stringContaining("allowGitNetwork"), + stderr: expect.stringContaining("createGitModule"), }, }); }); - it("rejects trusted Git paths that traverse a symlink", async () => { + it("rejects ws:git paths that traverse a symlink", async () => { await write("/outside/repository/README.md", "outside"); await symlink("/outside/repository", "/workspace/linked-repository"); const response = await runtime({