Skip to content

feat(aikit): declare tool changes as transcript system messages - #110

Merged
sanchitrk merged 2 commits into
canaryfrom
claude/aikit-system-messages-61f8b5
Oct 27, 2026
Merged

sanchitrk merged 2 commits into
canaryfrom
claude/aikit-system-messages-61f8b5

Conversation

@sanchitrk

Copy link
Copy Markdown
Contributor

Summary

Tools can now be added or removed mid-conversation with a transcript SystemMessage (toolsAdded, toolsRemoved, metadata). Context.tools is the initial declaration. Transcript helpers resolve the current tool set and decide, per provider, whether a change is declared in place or folded into the top-level tool list.

Codex and Copilot Responses

  • Both run on one shared ResponsesLanguageModel (providers/responses/, moved from openai-codex/). A ResponsesRequestProfile captures each backend's differences:
    • where the system prompt goes;
    • whether max_output_tokens is sent;
    • when encrypted reasoning is requested;
    • zstd compression;
    • tool-field defaults.
  • Models that support it declare every tool in additional_tools items, starting at the head of the input. The incremental-update notice precedes later declarations, and removals are announced as text. The request input only grows, so the cached prefix survives tool changes.
  • Models that only support tool search get later additions as a tool-search call/output pair. Everything else gets the current tools top-level.
  • Stream items are tracked by output_index, because Copilot rotates item ids on every event (which split one tool call into an empty call plus a call with arguments).

Copilot

  • model.list shows only the models the login's availableModelIds allows, matching each model's API id.
  • A Claude model that can't disable thinking no longer gets thinking: disabled, which Copilot rejected with a 400.
  • GPT models that accept reasoning.effort: "none" get it when thinking is off (checked live).
  • tool_choice and parallel_tool_calls are sent only when the call names a tool choice; Grok rejected them on requests without tools.
  • The model generator's long-context and thinking override lists for Copilot are updated.

Other changes

  • Errors: when an adapter can't parse a provider's error, the message and code are read from the raw response body. This fixes Copilot errors that showed only "Bad Request".
  • Grammar tools: calls are replayed using the declaration of every tool the conversation declared, so calls to a tool that was later removed still replay correctly.
  • activeTools: a request with activeTools sends its tools top-level instead of declaring them in place.
  • Removed: ToolCall.addedToolNames; the token estimate now counts toolsAdded.
  • Harness: stored context messages are narrowed to user and assistant, and decoding a stored system message fails with an explicit error. Storing system messages comes later.

Test plan

  • vp check and vp run typecheck
  • aikit unit tests, including the new transcript.test.ts (prefix stability across removals and loads, the tool-search boundary, redefinition, re-declaration) and the Copilot Responses tests (rotating ids, tool fields, off effort)
  • Live E2E tools.added.e2e.test.ts on Codex, Copilot Responses, Copilot Messages, Copilot Chat, Anthropic, OpenAI, Google and OpenRouter, with an artifact for each
  • Codework model.list Copilot filter test
  • Five unrelated live tests failed in the last full run, on OpenAI gpt-5.4-nano, OpenRouter glm images, and the Anthropic and OpenRouter thinking-level artifacts. One nano test passed when re-run.

Add a transcript SystemMessage that adds or removes tools mid-conversation.
Context.tools is the initial declaration; each provider takes changes in
place where its transport allows and folds them into the top-level tool
list otherwise.

- Codex and Copilot Responses share one Responses model with a per-backend
  request profile, declare every tool in additional_tools items and
  announce removals as text, keeping the cached prefix across changes
- track Responses stream items by output_index, since Copilot rotates item
  ids on every event
- limit Copilot models to the ones the login can reach
- leave thinking out for Copilot Claude models that cannot disable it, and
  send the off effort to Copilot GPT models that take one
- send tool_choice and parallel_tool_calls by default for Codex only
- read the error message from the raw body when the adapter cannot parse it
- replay grammar tool calls from every declared tool, and keep in-place
  declarations out of requests limited by activeTools
- remove ToolCall.addedToolNames; the harness does not store system
  messages yet
@sanchitrk
sanchitrk merged commit 251f096 into canary Oct 27, 2026
2 checks passed
@sanchitrk
sanchitrk deleted the claude/aikit-system-messages-61f8b5 branch October 27, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant