Skip to content

fix(instructor): preserve TypeScript extraction stream lifecycle - #390

Merged
nik13 merged 3 commits into
devfrom
fix/instructor-typescript-lifecycle
Nov 2, 2026
Merged

nik13 merged 3 commits into
devfrom
fix/instructor-typescript-lifecycle

Conversation

@nik13

@nik13 nik13 commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Keep and repair @traceai/instructor. Preserve the public InstructorInstrumentation entry point while making extraction spans cover the actual request, validation and stream lifecycle.

  • Emit one CHAIN extraction parent, not a second model-call span with provider=instructor. Provider instrumentors retain ownership of model/provider/token/cost accounting and validation-retry child spans.
  • Keep structured streams open through consumption; run next, return and throw under the extraction context. Preserve errors and cancellation, including closing a never-started stream, without eagerly consuming it.
  • Capture the last cumulative parsed output without _meta; map supplied numeric usage into instructor.usage.* rather than duplicating provider gen_ai.* totals. Do not estimate usage missing from the SDK.
  • Capture messages rather than serializing the schema, SDK client, or request-option headers. Respect suppression and existing FI privacy controls.
  • Replace the process-wide patch guard with module-scoped ownership. Restore owned factories on disable, leave later third-party wrappers intact, make existing clients inert, and let in-flight operations finish. Do not retain all created clients.
  • Initialize configuration before enabling loader hooks, and resolve the current tracer at request time.
  • Replace stale mock-only tests with real floor/current SDK contracts, add executable README examples using published OpenAI instrumentation and a loopback OTLP collector, and enroll this package in the existing integration CI matrix.

Why?

The final 36-test suite against the original dev implementation failed 26 tests (10 passed). It reproduced premature successful stream completion, missing stream context/errors/output, fake provider attribution, usage loss, passthrough duplication, and non-reversible patching.

Existing PR #385 repairs Python only; it does not overlap this TypeScript package. No dedicated competitor TypeScript Instructor implementation was established, so this repair follows the actual JavaScript SDK rather than importing assumptions from Python integrations. No third-party implementation code was copied.

Pinned upstream contracts:

How was it tested?

All local tests ran under an OS sandbox denying outbound networking except localhost. No real credentials, vendor/model API, or production backend was used.

  • 36/36 tests pass across real Instructor 1.0.0 and 1.7.0, Zod 3.25.76, OpenAI 4.58.0, and real OpenTelemetry SDKs. Source-baseline countercheck: 26 failed / 10 passed with the same final tests.
  • Nonstream extraction, real validation retry, lazy/eager stream behavior, iteration and cleanup errors, break/return/throw before and after first yield, recovered throw, empty stream, suppression, concurrency, multiple module copies, disable/re-enable, wrapper ownership, result identity and request options.
  • Real OTLP JSON serialization checks cover span kind, output shape, namespaced usage, and content/error redaction.
  • Both README examples execute actual auto-patching with published @traceai/openai@0.1.14, real OpenAI/Instructor/Zod SDKs, and loopback HTTP transports. The collector receives exactly one CHAIN parent plus one LLM child, with matching trace/parent IDs and no captured synthetic content or credentials.
  • Re-ran all 36 tests with the unit tests mapped to published @traceai/fi-core@1.1.0; passed. The README subprocess uses its installed dependency graph.
  • All CJS/ESM/ESNext builds and tarball creation pass. Frozen lockfile installation passes; unrelated workspace resolutions are retained.
  • Existing integration planner/result-gate suite: 15 passed. No Ruff configuration, migrations, frontend contracts, or product UI is touched. No lint result is claimed.

Re-run from typescript:

pnpm --filter @traceai/instructor... install --frozen-lockfile --ignore-scripts
pnpm run build:deps
pnpm --filter @traceai/instructor run build
pnpm --filter @traceai/instructor test --runInBand

E2E coverage

Not applicable (no product e2e/scripts/e2e-coverage.mjs harness). The SDK contract tests and README loopback collector cover this library's end-to-end path; no live backend-ingestion claim is made.

Compatibility and rollout

  • Branch/target are dev, per this repair's instructions (not the generic template's main). No package version or CHANGELOG version section was hand-edited. Release-please owns the release.
  • Instructor 1.0 opens the provider stream eagerly and omits usage in its parsed result; the integration preserves those upstream semantics. Instructor 1.7 returns lazy generators and can supply _meta.usage. Streaming usage remains conditional on what the framework provides; result usage is not a sum of retry costs.
  • Existing instrumentor/configuration entry points remain. Clients created before instrumentation cannot be retroactively patched. After disable/re-enable, construct new clients. Finish or close streams before shutting down the tracer provider.
  • The README starts the published OpenAI instrumentor with enabled: false and lets registerInstrumentations enable it, ensuring privacy configuration is initialized first.
  • A standalone TypeScript site guide is held until this package fix is released. The existing site page is Python-only; a JavaScript package README already existed. No docs PR promises unreleased behavior.
  • No merge, tag, registry publication, or package deletion is included. Disabling the instrumentor is the safe way to turn off tracing; reverting would restore known incorrect trace semantics.

Review

Self-reviewed against internal correctness, contracts, failure-mode, performance, security and test-scenario standards, including the whitespace-insensitive diff. No unresolved blocking finding in the final diff. The current package owns no new service, migration or enterprise boundary. Human review remains required.

🤖 Generated with Claude Code

nik13 and others added 3 commits October 31, 2026 07:47
Keep one extraction parent through validation and iterator consumption, restore owned factory patches, and verify floor/current SDKs and documented provider composition over OTLP.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Preserve the repaired Instructor contract tests and union lockfile entries without re-resolving unrelated packages.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Retain the repaired Instructor importer and union the added SDK lock entries.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit eb09d4c into dev Nov 2, 2026
102 checks passed
@nik13
nik13 deleted the fix/instructor-typescript-lifecycle branch November 2, 2026 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant