rlm.org

Autolith RLM

Recursive inference

A recursive language model treats long context as an external environment instead of a prompt. The model decomposes a task, runs bounded inference over selected snippets, and composes the returned values. Autolith realizes this as a middle layer between a tool call and a child agent, so recursion becomes an ordinary Lisp operation:

(infer "List the invariants this file maintains."
       :context (list #p"src/conversation/store.lisp")
       :contract '(:type :object
                   :properties
                   (("invariants" (:type :array :items (:type :string))))
                   :required ("invariants"))
       :budget (rlm-budget-create :calls 16 :tokens 80000 :depth 3))

Autolith already has the outer level. Task children are persistent agents with conversations, tool registries, and orchestrator scheduling. RLM adds the two inner levels plus the paper-defining root operation, rlm-complete, described below:

• An inference call. One bounded inference operation over explicit context. Returns a Lisp value. Never touches the caller’s conversation.
• An inference frame. One task with a private ephemeral conversation, a contract, and a budget subtree. A frame may recurse into child frames until the budget runs out. Only the frame’s value and a trace reference escape.

A frame never inherits the parent conversation, the agent identity, or unrestricted capabilities. It receives the task, the materialized context views, the contract, and the remaining budget. This containment is what makes recursion cheap and safe: a hundred frames cannot pollute the primary conversation or the prompt cache. The frame’s request puts the views first, then the answer shape, and the task last, so frames asking different questions of the same views share a cacheable prefix.

Context views

Context is passed explicitly, and the designator set is split by authority. Programmatic Lisp callers in the active image may pass:

• a string, passed verbatim,
• a pathname, read at call time,
• a plist (:label ... :content ...) for labeled literals.

The model-visible tools never accept raw filesystem paths. Their views carry literal text, a resource URI resolved through the resource protocol under the caller’s authority (so restricted frames stay confined to their readable scheme set), or a stored context object reference as context:<sha256>. A resource view reads the resource whole, up to *rlm-view-maximum-bytes* (4 MiB), so a conversation: view carries the complete durable history rather than the paged window resource.read returns. Each view materializes into a labeled block with a content digest, so a trace records exactly what the frame saw. Views are read-only and fixed for the frame’s lifetime.

Contracts

A contract names the shape of the value a frame must return. Contracts reuse the task output schema language documented for task agent roles; task-output-schema-normalize validates the definition and task-output-schema-valid-p validates the value. Two contract forms exist:

• :text, the default. The frame returns its answer as a string.
• a schema, for example (:type :object :properties (("claims" (:type :array :items (:type :string)))) :required ("claims")). The frame is instructed to answer with exactly one JSON value of any type, object and array included, and the parsed, validated value is returned as portable tagged native data.

A malformed answer is repaired by re-asking within the remaining budget. On exhaustion, infer signals rlm-partial-result, a subtype of rlm-budget-exhausted. Read rlm-partial-result-observation for the explicit :incomplete status, exhausted allowance, remaining budget, trace identifier, and bounded recent assistant and tool outputs. These outputs are evidence, not a validated answer. Read inference:<trace-id> for the complete record. The tool response contains this observation rather than a bare failure.

Budgets

Every inference runs under an rlm-budget: remaining calls, tokens, and recursion depth, with thread-safe accounting. Recursive frames share the parent’s counters and decrement depth. Every provider request atomically reserves, in one critical section, one call and an output token tranche; the tranche doubles as the request’s provider output ceiling where the wire protocol accepts one (elsewhere it stays an accounting reservation) and is settled against the reported usage afterwards, with failed or usage-less requests refunding the output tranche in full while the admitted call stays consumed. Settlement discounts prompt-cache reads: cached input tokens are repeated context served from the provider cache, so a long run over a shared prefix drains the pool only for its genuinely new work. A tranche takes at most a share of the remaining pool (a quarter by default, capped at 16000 tokens) and never more than the pool holds, so concurrent siblings keep headroom and combined reservations cannot oversubscribe the allocation; only input tokens stay post hoc, gating the next reservation rather than the current one. Exhaustion stops further provider calls; incomplete observations preserve the work already performed.

Root completions

rlm-complete is the paper-faithful realization: the complete input remains outside the root model context, and only selected bounded slices are supplied to sub-inference contexts. The context designator is interned as an immutable content-addressed object under data/inferences/objects/ and the root model receives only its label, size, and digest. A dedicated heap-isolated Lisp environment holds the content, every intermediate value, and the recursion:

(rlm-complete "Summarize every incident in this log."
              :context #p"/var/log/huge.log"
              :budget (rlm-budget-create :calls 24 :tokens 240000
                                         :depth 2))

The root model drives the environment through the env.eval tool, one form per call, observing only bounded printed values and output. Inside the environment, (context-length), (context-slice start end), and (context-search pattern) inspect the external object; (infer ...), (rlm-map ...), and (rlm-run ...) proxy sub-inferences back to the host over an authenticated loopback endpoint, where budgets, contracts, and traces are enforced host-side because environment code is model-authored and untrusted; the run proxy also makes installed distilled decomposition policies reachable from root runs; (finish value) records the final answer. Every proxied call descends the run’s shared budget subtree, so model-authored loops can never outspend the root allocation. Finish is terminal: the first finish wins, later operations are refused, and the value must be portable readable data because it crosses the environment boundary as printed text. The run returns the recorded Lisp value plus the root trace identifier. The model reaches the same operation through the rlm.complete tool.

Every proxied operation also appends one replay-safe :rlm-call metadata record to the root trace, carrying the operation, the child trace identifiers, each child’s settled billable token spend, and the remaining budget, so cost reporting can split a conversation’s own usage from its recursive spend. The root trace therefore holds the run’s machine-readable invocation tree even when environment code discards the trace identifiers it receives, which is what trace-driven policy distillation reads.

The defining litmus property is tested end to end: an input larger than the provider window is processed almost completely, with the decomposition authored by the root model as Lisp, no provider request exceeding the window, intermediate values staying in the environment, and the exact answer returned.

After completion or budget exhaustion, the worker is retained for another run over the same context under the same originating configuration. Per-call model routing does not change that ownership. The next request includes earlier names; inspect (environment-names) to see them live.

An incomplete rlm.complete response includes a context: URI and continuation instructions. Call it again with that URI in context.object and a fresh budget to continue from the retained variables. In Lisp, resolve the digest with rlm-context-object-find and pass the object as :context.

Retention is in memory, not a saved image: at most two idle workers are kept. Eviction, shutdown, and other failures stop them. If a retained worker dies, the next run starts a fresh environment and reports the loss. Trace and context files remain available independently of the worker.

Parallel maps

rlm-map fans a list of tasks out as frames sharing one budget subtree, each task a string or a (:task ... :context ...) plist. Results keep task order. Finished frames retain their validated values; budget-exhausted frames carry :status :incomplete with their own evidence and trace identifiers. Other failures carry :error. The model reaches the same fan-out through the rlm.map tool.

Policies

Decomposition and synthesis are generic functions dispatching on a policy keyword, so live self modification can promote strategies distilled from successful traces as new eql-specialized methods:

• rlm-decompose-inference-task returns subtask plists for a task, or NIL to run it directly.
• rlm-synthesize-inference-results composes the final answer from the subtask results; the default method for every policy runs one more frame whose views are the results, failures included.

rlm-run drives one task through a policy: decompose, fan the subtasks out through rlm-map, and synthesize under the caller’s contract, all inside one budget subtree. The default :direct policy never decomposes.

rlm-distill automates the promotion path from traces to policies. A gate frame reads bounded head-and-tail trace views and prefers declining, so an empty review stays cheap; an approved review runs a planner frame whose proposal is structurally validated with evaluation disabled: exactly one rlm-decompose-inference-task defmethod, eql-specialized on a fresh policy keyword, never :direct, keeping the generic’s parameter specializers. A rejected proposal is re-asked with its rejection reason within a bounded attempt count. Distillation installs nothing: the returned proposal goes through the ordinary durable mutation pipeline, which journals, checks, commits, and can discard it. The model reaches it through the rlm.distill tool, available only to the primary agent.

Traces

Frames persist their private conversations under data/inferences/ in the same append-only readable format as regular conversations, one file per frame. The caller’s conversation only ever sees the returned value. The trace answers “why did this inference conclude that” after the fact: task, views with digests, model parameters, raw exchanges, and usage are all in the file.

Traces stay addressable after the fact: inference:<trace-id> URIs read a bounded trace log back through resource.read, and the compaction instructions tell summaries to reference frames by trace identifier instead of restating their content, so compaction keeps evidence reachable instead of inlining it.

Recursion surfaces

The lisp.* tools run in heap-isolated workers with no provider access, so model-authored worker code cannot call infer. The surfaces are:

• The infer, rlm-map, rlm-run, and rlm-complete functions in the active image, for user REPL forms and for self-modification code (policies, agenda automation).
• The rlm.infer, rlm.map, and rlm.complete tools for the model, taking tasks, views or one external context, an optional JSON Schema contract for the frame tools, and call, token, and depth allowances.

Background RLM tool jobs appear in a separate RLM section above the prompt, beside commands and child agents. Rows show the job ID, operation, queued or running state, elapsed time, and latest phase. rlm.infer reports request counts and remaining calls; rlm.map adds frame index and fan width. rlm.complete reports environment startup and nested direct, map, and synthesis phases.

Provider stream arrivals produce content-free receiving updates at most once per second for each request.

Spinners animate between phase reports. The no update timer measures the absence of real progress reports, not animation. These rows are transient; task text and source contents belong in the private inference traces.

Both surfaces accept :capabilities (the tools’ capabilities argument). A frame without capabilities is a pure call over its views. A read frame runs as a restricted agent turn: it may use resource.read (workspace scheme only), the search tools, and nested rlm.infer and rlm.map sharing the same budget subtree, with tool rounds bounded per round. Its last request keeps the same tool definitions but disables calls, so the answer reuses the cached prefix; a call returned anyway fails the frame. Inside a frame the nested tools ignore allowance arguments and descend the enclosing budget, so fan-out can never outspend the root allocation.

Design decisions

Settled trade-offs, recorded so review rounds do not relitigate them.

The environment is privileged and unsandboxed

env.eval runs model-authored Lisp in a heap-isolated worker with the user’s privileges, exactly like lisp.* always has; Autolith’s stance is that sandboxing is no substitute for oversight. Opaque context handles with host-proxied context-slice and context-search were considered and rejected: they add a socket round trip per operation while confining nothing, because arbitrary worker Lisp can still open any file the user can. Confinement is why frames may not launch environments, not a property handles would provide. The store is protected differently: objects are read-only on disk, host-side lookup re-hashes their content and rejects corruption, and re-interning the original content repairs a corrupted object. Revisit only alongside real OS sandboxing of the worker.

Finish and shutdown do not block

Admission shares one lock with the terminal state, so nothing starts after a finish commits, but a finish never waits for admitted operations: they only overlap when environment code spawned worker threads, they stay budget-bounded, and a blocking finish could deadlock the run. Endpoint shutdown likewise joins handlers briefly and then abandons stragglers; an abandoned handler is a budget-bounded provider call whose response write fails harmlessly, which is cheaper than any safe cancellation.

Context objects live in the heap

Interning and the environment’s context functions materialize whole objects in memory. This is a documented capacity bound, not a design error: the RLM property only requires the content to stay outside model contexts, and content addressing already provides stable identity. Streaming, mmap, or indexes arrive when a workload actually exceeds the heap, not before.

That heap bound is also the only size limit on a root context. The rlm.complete tool reads a workspace file named by its context URI whole, up to *rlm-context-maximum-bytes* (64 MiB), rather than through the editing snapshot that resource.read keeps, so an input far larger than the model’s window, and larger than an editing snapshot, is still one context object.

Token accounting is exact for output, soft for input

Reservations bound what concurrent requests may produce, but exact input usage is only known after a response, so input tokens settle post hoc and gate the next reservation. Providers that report no usage consume reservations but settle as free; deterministic test providers depend on this, and real providers report usage.

Implementation

• Budgets: src/inference/budget.lisp.
• Context views and materialization: src/inference/view.lisp.
• The inference frame and infer: src/inference/frame.lisp, with the compact frame system prompts and trace persistence.
• The rlm.infer and rlm.map tools, frame registries, and the contract conversion from JSON Schema: src/inference/tools.lisp.
• Parallel maps: src/inference/map.lisp.
• Policies and rlm-run: src/inference/policy.lisp.
• Trace-driven policy distillation and rlm.distill: src/inference/distill.lisp.
• The inference: trace resource: src/inference/resource.lisp, with the trace reference guidance in the compaction instructions.
• Content-addressed context objects: src/inference/object.lisp.
• The loopback host endpoint for environment calls: src/inference/endpoint.lisp.
• Root completions, the environment prelude, and env.eval: src/inference/environment.lisp.
• Model-decided auto-mode command permissions: src/inference/permission.lisp.

Per-call model routing

Pass optional model and effort arguments to rlm.infer, rlm.map, or rlm.complete to select a model and supported reasoning effort for the entire subtree. For a map, this selection applies to every frame; for a root completion, it applies to the environment and child inferences. Make this selection in the outermost call. Nested calls use the enclosing selection without overrides.