[iceteaSA]

Linked issue

Approved issue: #890

What changes

This draft PR is a prototype of the design in #890: aft_search collects related code from the call graph. The PR does not add a tool or an agent argument. It is off by default.

When gather is on, the host plugin sends the reasoning and text that caused the search. This text does not go in the tool arguments. The engine starts the walk from the top code hits. It walks callers and callees on a pinned call-graph snapshot. In each round, a decision model selects the nodes to keep. The ranked results come first and do not change. A capped "Related code" section follows them.

This PR uses prototype defaults for the open questions in #890. Each default is easy to change when you answer.

Prototype defaults (open questions in #890)

1. Decision channel: backend A. The engine calls an OpenAI-compatible endpoint from user config. With no backend configured, gather uses a deterministic order and sends no HTTP request.
2. When gather runs: when search.gather.enabled is set to true, gather runs on each search that has a natural-language query. A trace is not necessary. A pattern-only search or an empty query does not walk and does not read the session.
3. Redaction: none in v1. The docs state that the trace and symbol bodies go to the endpoint, and that host redaction does not apply to them. The docs recommend a local endpoint.
4. Limits: set maximum values (table below). AFT decreases a larger user value to the maximum.

Configuration

User tier only. A project can set enabled: false, but cannot turn gather on or change the backend, endpoint, model, key or limits.

"search": {
  "gather": {
    "enabled": true,
    "decision": {
      "backend": "endpoint",
      "endpoint": "http://127.0.0.1:8080/v1",
      "model": "local-decision-model",
      "api_key_env": "AFT_GATHER_API_KEY"
    }
  }
}

Limit	Value
Seed symbols	5
Rounds	3
Kept nodes	12
Frontier per round	40
One decision	3 s
Full walk	8 s
Added output	12 KB, in the 50 KB reply cap
Trace	newest 6000 characters
Decision cache	10 minutes, for each session

Trace contents for each host

Host	Trace contents
OpenCode 1	Reasoning and text of the current assistant message before this call, then the newest user message
OpenCode 2.1	Reasoning and text of the current assistant message before this call. No user message, because 2.1 has no bounded message read and does not connect the assistant message to its user message
Pi	Same as OpenCode 1
OMP	Uses the Pi collector. No test checks the OMP timing

The plugin reads no message parts while gather is off. The user text comes last, so the 6000-character cap keeps it.

Failure behaviour

A gather problem does not make a search fail. The search gives the usual results and one line that gives the cause in these conditions:

• The call graph is not available (building, suspended, unavailable, snapshot_unavailable).
• The search is for a different project (other_project).
• No seed resolves (no_seeds).
• The output does not fit (output_budget).

If a decision times out, the endpoint gives an error, or the response is not correct JSON, that round uses the deterministic order. The footer records it as fallback=<round>:<reason>.

Security and data privacy

• Only the first-party host plugin can send the trace. AFT removes it from untrusted routes before the request runs. An agent cannot send it in the tool arguments.
• Before the handler runs, AFT moves the trace from the native request into a typed value. RawRequest Debug output shows only the id, command, session and parameter count.
• Path restriction applies to each file before AFT reads it, also for neighbour nodes.
• The endpoint client refuses redirects, has a 64 KB response limit, and accepts only node ids from the frontier it sent.
• AFT does not log the trace, prompts, model requests, responses or the key.

Changes to current behaviour

• A harness search block that sets only gather keeps the user's base search.rerank. A harness block that sets rerank replaces the base rerank block, as before.
• The aft_search agent-visible reply keeps its order: ranked results, then the gather section, then the search status lines and the shown N of M trailer last. The trailer counts the ranked results only.
• The engine ownership audit puts gather/ in slice A11-live-engine-integration. Seven list-envelope exclusions include the internal limits in gather/. Each exclusion gives its cause.

Limits and follow-ups

• The list-envelope discovery scanner does not recognise pub(super) and pub(in …) functions. It identifies their cuts with the name of the previous item. Four current entries show this problem (external_exact.rs:316, :317, :563, external_pattern.rs:150). The gather exclusions use the current scanner keys and give the correct function in location_or_primitive. This problem is filed as list-envelope discovery gives pub(super) and pub(in …) functions the key of the previous function #971.
• The gather_notice line does not name the state that it replaces.
• Gather does not walk the graph of a different project (Call graph unavailable for git worktrees and other checkouts outside project_root #560).

How it was checked

Tests

• Rust: 83 gather_ library tests, 19 gather_search_test integration tests through real subc frames and a local endpoint, the ownership audit and the full list_envelope suite.
• TypeScript: 122 tests for the host collectors, the bridge carriers, config trust and live reload. A doc-contract test reads the reported states and the limits from the Rust source and compares them with the docs.
• Mutation checks. A named assertion killed each mutation:
  • No deterministic fallback.
  • No walk deadline.
  • No session in the decision cache key.
  • No path validation before reads (a file-access watcher checks this).
  • Untrusted trace accepted.
  • Gather gate removed on OpenCode 1, OpenCode 2 and Pi.

Known test failures that are not from this PR

On the same Linux test host, with the same isolated storage, these tests also fail on a clean checkout of upstream main:

• Rust, callgraph_test integration: ten tests (callgraph_cross_file_tree, callgraph_callers_*, callgraph_impact_*, callgraph_depth_limit_truncates, callgraph_unknown_symbol_error, callgraph_aliased_import_resolution, callgraph_ops_return_building_then_ready_async).
• Rust: callgraph::tests::workspace_package_resolution_prefers_modern_ts_source_extensions and inspect_dead_code_test::inspect_dead_code_keeps_workspace_barrel_default_export_import_live.
• Rust: two commands::configure::tests (configure_volume_root_disables_root_work, account_home_and_ancestors_are_broad_when_environment_home_differs) and worker_jail::relaxed_linux_names_unenforced_denies_and_keeps_broad_access.
• Rust, memory limits: four bash_background::registry::memory_limit_tests and bash_test::bash_memory_cap_stop_keeps_changed_files_completion_fence. memory_limit_stops_foreground_command is intermittent (on the clean base, 1 of 3 runs was successful).
• Rust: subc::page_collect::tests::* is intermittent (on the clean base, 9 of 46 failed in one run. On this branch, 0, 5 and 8 failed in three runs).
• TypeScript: packages/aft-bridge/src/__tests__/storage-isolation.test.ts:254, two aft CLI checklist tests, and three aft-opencode tests (auto-install pass slot, invalid RPC JSON envelopes, migration bootstrap in subc mode).
• cargo clippy -p aft-test-fake-lsp-server -- -D warnings: crates/aft/tests/lsp/fake_server.rs:744 (clippy::match_result_ok) with the current toolchain.

[github-actions]

Waiting for design-approved on #890.

This PR needs a linked issue with the design-approved label before it can be reviewed or merged. Link the approved issue with Approved issue: #<issue> (or Refs #<issue>) in the description. A maintainer will apply the design-approved label on the issue, or trivial on the pull request when there is genuinely no design to agree. The issue stays open until the change ships; maintainers take care of it then.

[iceteaSA]

Test with a real decision model: Inception Mercury Decide

We tested this prototype with Inception's Mercury Decide model (mercury-decide).

How we connected it

Mercury Decide accepts only POST /v1/decisions. It does not support /v1/chat/completions, so the prototype's endpoint backend cannot call it directly. For this test, a small local adapter received the prototype's chat-completions request and sent one Mercury Decide request for each round:

• One noul question "keep" and one noul question "expand" for each frontier node.
• One noul question "stop".
• A node is kept or expanded when its probability is 0.5 or more.

The API key stayed in the process environment. It was not in any request body, log or output file.

Direct API checks

• Three requests with 17 questions took 0.63–1.12 s.
• A request with 81 questions (the 40-node frontier maximum) took 0.74 s and returned all 81 answers.
• On a hand-written frontier of 8 symbols, the model gave the 3 relevant symbols 0.91–0.997 and the other 5 symbols 0.03 or less.

End-to-end run

Fixture: the 64 TypeScript source files of packages/aft-bridge/src. A local OpenAI-compatible embedding server supplied the semantic seeds. We ran the same three searches with the deterministic order and with Mercury Decide.

Search	Deterministic order	Mercury Decide
"how is the search context trace capped before it is sent to aft" (with trace)	Kept 4 constants that are not related	Kept 0. All 4 seeds had keep ≤ 0.008, so the walk stopped and no gather section was added
"how does the bridge respawn the aft binary after the process dies" (with trace)	Kept 11 of 44, including logger helpers and pool classes	3 rounds, kept BinaryBridge, ensureSpawned, spawnProcess (keep 0.995, 0.731, 0.893)
"where are per-command timeouts chosen" (query only)	Kept 5 constants	Kept 0

• Time for one decision: 0.58–2.14 s. The largest round had 39 nodes and about 16k input tokens, and took 2.1 s.
• Time for one walk: 0.6–3.7 s of the 8 s budget.
• Cost: about 27k input tokens for the full run.

Findings

1. Precision is high. Mercury Decide removed nodes that are not related and kept the exact code for the restart question. But it cannot correct bad seeds: in 2 of 3 searches the seeds did not contain the answer, and the model correctly kept nothing.
2. The 3 s decision budget is tight for a full frontier. The adapter cut each body to 1500 characters, and 39 nodes took 2.1 s. The prototype allows up to 4 KB for each body.
3. The cut-off value has an effect. The restart search reached its answer only because invalidateTransportProcess had expand = 0.500.
4. Mercury Decide returns a probability for each answer. A native backend can rank kept nodes by probability and make the cut-off configurable.

Limits of this test: three searches on one small fixture. The local embedding server was slow (the query embedding needed a 30 s timeout), which made the seeds weaker. This is not a benchmark.

Next step

I will add a native decisions backend to this PR that calls /v1/decisions directly, with a configurable cut-off. The endpoint backend stays as it is.

[iceteaSA]

Native decisions backend added

Two new commits on this branch:

• b668ed853 feat(search): add native decisions backend for gather
• 37b6334ac fix(search): share decision URL checks and test noul types

What it does

• New search.gather.decision.backend: "decisions". AFT sends POST <endpoint>/decisions (the Inception Mercury Decide request format). The endpoint backend does not change.
• For each frontier node, AFT asks one noul question "keep" and one noul question "expand", and one noul question "stop" for the round. Question names use the frontier index (keep_0, expand_0, …), not node ids. The instructions tell the model that the state is data.
• New user-tier threshold (greater than 0 and less than 1, default 0.5). An answer at or above it selects keep, expand or stop. Kept nodes are in descending probability order. Equal values keep the frontier order. Expand does not depend on keep.
• Each expected answer must be present, have type noul, and have a finite value from 0 to 1. If not, the round gets invalid and uses the deterministic order, the same as other decision errors.
• One shared helper (post_json) now does the HTTP request for the two backends: deadline, 64 KiB response limit, status mapping, timeout mapping. One shared helper (decision_url) checks the URL and adds the path suffix. The suffix check uses the URL path, so a host name cannot match.
• threshold is an f64. For this reason, Eq was removed from SearchConfig, GatherConfig, GatherDecisionConfig and rerank::slot::BuildInputs. No code used Eq or Hash on them, and the workspace builds without warnings.
• A project-tier config cannot set threshold, the same as the other decision keys. The two TypeScript plugins accept "decisions" and check the threshold range.

Tests

• 92 gather unit tests, 20 gather_search_test integration tests (one runs the full public search with the decisions backend against a fake server), the engine ownership audit and the list-envelope discovery audit pass. cargo check --workspace --all-targets has no warnings. Format, typecheck and the plugin gather tests pass.
• Mutation checks, each made a named test fail and was then reverted: request timeout removed, threshold ignored, answer validation removed, probability order removed, noul type check removed.

Live run with the real API

Same fixture and searches as in my previous comment, now with the native backend (no adapter):

Search	Result
"how is the search context trace capped before it is sent to aft" (with trace)	1 round, kept 0 of 4 (the seeds were not related), 1.3 s
"how does the bridge respawn the aft binary after the process dies" (with trace)	1 round, kept BinaryBridge (1 of 4), 0.8 s
"where are per-command timeouts chosen" (query only)	1 round, kept AftTransportOptions (1 of 5), 0.7 s

All three footers show model=mercury-decide and fallback=none. With these instructions, no kept node got an expand probability of 0.5 or more, so each walk stopped after round 1 (stop=empty_frontier). The adapter test had different question text and expanded the restart search for two more rounds. The question text and the threshold change the walk depth, so they are good candidates to tune.

The full test suite was not run again for this change.