Skip to content

fix(cohere): trace v2 clients and streaming lifecycle - #350

Merged
nik13 merged 8 commits into
devfrom
fix/cohere-typescript-v2-streams
Nov 2, 2026
Merged

nik13 merged 8 commits into
devfrom
fix/cohere-typescript-v2-streams

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Keep and repair @traceai/cohere; do not delete it. The old integration missed generated v2 clients and all chatStream calls, and treated SDK camelCase fields as wire-format snake_case.

  • Patch the actual generated v2 prototype reached through CohereClient.v2, covering both client.v2 and the methods bound by CohereClientV2. Keep CohereInstrumentation / manuallyInstrument working.
  • Trace streaming until terminal usage, exhaustion, failure, iterator return or abort. Aggregate text and tool-call deltas without changing events, controllers, errors, or HttpResponsePromise.withRawResponse().
  • Fix chat history, tool calls, token usage and rerank score extraction. Preserve EMBEDDING spans and use RERANKER for reranking instead of treating everything as an LLM call.
  • Respect suppression, parent context, concurrent calls, multiple SDK copies, idempotent patching and disable/enable, including already-bound v2 methods.
  • Apply privacy controls to raw payloads, Cohere-specific fields, tool content, error details and embedding vectors. Do not invent model names.
  • Bound the SDK peer range to 7.x/8.x; pin offline fixtures to 7.0.0 and 8.3.0. Fix the package's native ESM entry and replace the README's broken exporter example with maintained package setup.

Why this route

Cohere 8.3.0 binds v2 methods to a separate generated client, rather than the v1 prototype: SDK source. The public client.v2 accessor exposes that same generated implementation: SDK source. Prototype discovery constructs an inert client with no I/O or environment credential lookup. Instrumentation must be configured before constructing customer clients.

The existing package API is retained instead of requiring customers to copy helper code or migrate to a different client wrapper. Endpoint scope remains chat, chatStream, embed, and rerank.

Verification

Before the implementation change, the real-SDK reproducer failed all three cases: v2 emitted zero spans; v1 history extraction omitted earlier messages; the global patched flag also prevented a second SDK copy from being instrumented.

After the change:

  • Frozen filtered dependency install passed; unrelated lockfile resolutions are preserved.
  • CJS, ESM and ESNext TypeScript builds passed.
  • 71 tests passed (35 new): real SDKs 7.0.0 and 8.3.0 against localhost, v1/v2 streaming, embeddings/rerank typing, request-option forwarding, HTTP/parser/iterator errors, early return, abort, suppression, parenting, concurrency and enable/disable.
  • Fresh-process CommonJS automatic instrumentation and native ESM manual instrumentation passed.
  • Actual OTLP/HTTP export assertions passed for span kinds, usage, hidden content/errors and vector redaction.
  • Package tarball built successfully.

All local build/test/package commands ran under the supplied outbound-network-denied sandbox, with localhost permitted. No vendor/model API calls or real credentials were used. The dedicated CI job repeats the tests inside a network namespace with only loopback enabled.

pnpm install --frozen-lockfile --ignore-scripts --filter '@traceai/cohere...'
pnpm --filter '@traceai/cohere...' --workspace-concurrency=1 run build
pnpm --filter '@traceai/cohere' exec jest --runInBand
pnpm --filter '@traceai/cohere' pack --pack-destination /tmp/cohere-package

E2E coverage

not applicable (no product e2e/ harness) · flows hit: n/a · marker: n/a · verdict: not applicable

The real-SDK localhost and OTLP wire tests are the integration regression gate. No live Cohere verification is claimed. SDK 7.0.0 has no chatStream; stream coverage uses 8.3.0.

Release and docs

Release-please owns package versioning; no versions or changelog entries were hand-bumped. Package README changes ship with this fix. No separate docs-site PR is opened against an unreleased package change. This PR does not merge, tag or publish a release.

🤖 Generated with Claude Code

nik13 and others added 8 commits October 31, 2026 05:49
Preserve the instrumentor API and SDK promise/iterator contracts while tracing generated v2 clients, v1 camelCase payloads, embeddings and reranking. Add offline real-SDK coverage, OTLP privacy assertions and a dedicated CI gate.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Keep generated outputs visible while omitting echoed history, embedding texts, reranked documents and citation sources when inputs are hidden. Treat EOF without a terminal event as incomplete without changing consumer iteration. Cover SDK floor and current release with independent privacy flags and OTLP assertions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve v2 content-start text, share finish classification across chat paths, and keep SDK requests and responses unchanged. Cover credential OTLP export, terminal statuses with content hidden, and partial start-event output.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…v2-streams

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve indexed v2 thinking blocks in streaming output and input-hidden response projections. Accumulate v1 tool-call deltas without parsing partial argument strings, and replace them with authoritative final calls.

Add 45 real-SDK regressions with OTLP checks for completion, interruption, multiple content/tool indices, input hiding, and all output hiding controls.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 10b91e1 into dev Nov 2, 2026
88 checks passed
@nik13
nik13 deleted the fix/cohere-typescript-v2-streams branch November 2, 2026 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant