Skip to content

fix(tool): record locally executed tools in OpenCode V2 - #134

Open
pedropombeiro wants to merge 3 commits into
DEVtheOPS:mainfrom
pedropombeiro:fix/local-tool-telemetry
Open

pedropombeiro wants to merge 3 commits into
DEVtheOPS:mainfrom
pedropombeiro:fix/local-tool-telemetry

Conversation

@pedropombeiro

Copy link
Copy Markdown

Description

With OpenCode V2, the plugin drops telemetry for every tool that OpenCode runs locally, such as shell, read, edit, MCP tools, Code Mode, and subagents:

  • opencode.tool.duration is never recorded.
  • No opencode.tool.* spans are created.
  • opencode.commit.count and the commit log event never fire.
  • Every tool_result log event reports duration_ms: 0.

Root cause: In V2 tool events, executed means the model provider ran the tool, not that OpenCode ran it. OpenCode publishes session.tool.called with executed: tool.providerExecuted (publish-llm-event.ts#L458-L476). It then publishes the terminal event for a locally executed tool with the same false value (#L573-L591). Only provider-hosted tools, such as OpenAI web_search or Anthropic code_execution, report executed: true (#L477-L513). handleToolCalled and finishTool gated spans, durations, and commit detection on executed, so the plugin measured only hosted tools.

Fix: A tool counts as executed once the plugin observes session.tool.called. OpenCode starts a local execution only after that event is published (step.ts#L100-L128). Calls that never ran don't publish it:

Neither case is recorded as an execution.

  • Failures after the call: A tool that fails after session.tool.called, including a permission rejection or user decline (step.ts#L199-L207), is recorded as a failed execution (success=false). Its duration includes any permission wait.
  • Provider execution: The flag is now the tool.provider_executed span attribute. It accepts both the current top-level executed and the nested provider.executed shape, which OpenCode dev uses (session-event.ts#L312-L372). That covers only the payload shape: dev also renames the events to session.next.tool.*, which this PR doesn't address.
  • Subagents: Subagent dispatch spans now start at session.tool.called instead of at the first progress event that carries a child session ID. Child-session linking is unchanged.

Tests now cover local success and failure, local git commit detection, permission rejection, malformed input without a call, provider-hosted success and failure, the nested provider.executed shape, and a subagent that fails before creating a child. Two existing tests encoded the old assumption that executed: false meant the tool didn't run, so I updated them.

I also checked it end to end with OpenCode 2.0.20 and a local OTLP/HTTP JSON sink. A read call produced a 33 ms span, histogram sample, and duration_ms. A shell call running sleep 2 recorded 2,834 ms. A git commit through shell incremented opencode.commit.count and emitted the commit log event.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Chore (dependency updates, etc.)

Checklist

  • I have read the CONTRIBUTING.md document
  • My code follows the style guidelines of this project
  • bun run lint passes with no errors
  • bun run check:jsdoc-coverage passes with no errors
  • bun run typecheck passes with no errors
  • bun test passes with no errors
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation accordingly
  • My commits follow the Conventional Commits specification

Related issues

Independent of #112, which covers V1 message.part.updated timestamp restamping and orphan spans. This PR fixes V2 tool events that were never measured.

Additional context

The executed gating was introduced in #132, which treated executed: false calls as invalid. That holds for malformed calls but also matches every successful local call.

In OpenCode V2 tool events, `executed` means the model provider ran the
tool. Locally executed tools always report `executed: false`, so the
plugin skipped their spans, duration histogram, and commit detection, and
reported `duration_ms: 0` in `tool_result`.

Treat a tool as executed when the plugin observes `session.tool.called`.
Malformed-input rejections and calls cancelled before publication never
emit that event, so they are still not recorded as executions. Record
provider execution as the `tool.provider_executed` span attribute, and
accept both the top-level `executed` field and the nested
`provider.executed` shape.
@dialupdisaster

Copy link
Copy Markdown
Contributor

Non-blocking test coverage suggestion: strengthen the disabled tool-tracing test in tests/handlers/disabled.test.ts. It currently only sends session.tool.input.started, which does not create a span even when tracing is enabled.

Add a local tool sequence (input.started → called with executed: false → success) with tool tracing disabled, and assert that:

  • No tool span is created.
  • The tool duration histogram and tool_result.duration_ms still record the elapsed time.
  • For a successful shell git commit call, the commit counter and commit log event still fire.

This would protect the behavior changed here: execution metadata and command capture must happen before the tracing-disabled early return, so disabling tool spans does not also disable metrics or commit detection.

Pedro Pombeiro added 2 commits October 4, 2026 00:17
…bled

Send a complete local shell call with tool tracing disabled, and assert that the plugin creates no tool span but still records duration, the tool_result log, and the git commit counter and log.
@pedropombeiro

Copy link
Copy Markdown
Author

Thanks @dialupdisaster 🏓 Good catch: the old test only sent session.tool.input.started, so it passed whether or not tracing was enabled.

I replaced it in 23a6c22 and 7ae54b2 with a full local shell git commit call (input.started → called with executed: false → success) with tool tracing disabled. It asserts that:

  • No tool span is created after called or success.
  • The duration histogram and tool_result.duration_ms record 250 ms, measured from called.
  • The commit counter and commit log event don't fire after called, then fire exactly once after success.

I confirmed the test fails if the tracing-disabled early return moves above the metadata capture in handleToolCalled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants