[MCC-1554793] - Python Lib - Trace ID Propagation

[glondhe-mdsol]

feature

MCC-1554793 - Python Lib - Trace ID Propagation

Checklist

• Review the pull request to fix typos and ensure variable/function names are intuitive, etc.
• Make sure the pull request does not have excessive number of unnecessary commits. Utilize the git commit --amend --no-edit command to reduce commit messages when making small file changes (like changing linespacing).
• Make sure you have added unit tests for the code changes. Tests should be added in the tests/ folder.
• Modify docs, if required. Add any new documentation in the doc/ folder.
• Rebase on latest active development branch (develop/main).

Changes Summary

Why: Users need the trace ID for support reports, without changing return types.

Changes

• A client middleware reads x-dataconnect-trace-id. The payload is the fallback: app_metadata for list_flights, schema metadata for do_get, and the JSON trace_id for do_put and do_action. Malformed metadata is ignored.
• New read-only DataConnectClient.trace_id holds the most recent call's ID. It is shared state, so it is not reliable under concurrent use of one client.
• trace_id is carried on transport and SDK errors and printed as Trace ID: ….
• New extract_error_payload() handles earlier :: in gRPC text, such as IPv6.
• get_datetime_formats accepts both the old list and the new {"formats", "trace_id"} shape.
• Return types are unchanged.

Deployment Notes

In addition to IAC/code, add any notes for reviewer should know.

[Copilot]

Copilot review overview

🟡 Changes recommended

Moderate API compatibility and payload-decoding issues remain unresolved.

Review effort: Lite
Findings: 4

Open (4)

• Breaking change: dataset versions result is no longer iterable · New
• Fetch results break documented DataFrame usage · New
• Handle non-object response metadata safely · New
• Support legacy list-shaped response metadata · New

What changed in this PR

Adds server trace ID propagation across transport, service, and client layers, with trace-aware result models and lazy dataset frames.

Changes:

• Extracts trace IDs from Flight metadata and JSON responses.
• Adds trace-aware models for datasets, fetches, publishing, and datetime formats.
• Updates lazy frames and related tests for wrapped results.

File	Summary
tests/​test_transport_trace_metadata.py	Tests Flight metadata decoding.
tests/​test_publish.py	Tests publish trace propagation.
tests/​test_get_datetime_formats.py	Tests datetime trace propagation.
tests/​test_get_datasets_paginated.py	Tests dataset and frame result changes.
tests/​test_fetch_data.py	Tests fetch trace propagation.
tests/​test_dry_publish.py	Tests dry-publish trace propagation.
dataconnect/​transport/​models.py	Adds trace-aware transport result models.
dataconnect/​transport/​base.py	Updates transport contracts.
dataconnect/​transport/​arrow_flight/​transport.py	Extracts trace IDs; moderate compatibility issues remain for legacy payloads and non-object metadata, plus a documentation nit.
dataconnect/​service/​mappers.py	Maps publish trace IDs.
dataconnect/​service/​default.py	Propagates trace IDs through service methods.
dataconnect/​service/​base.py	Updates service interfaces.
dataconnect/​models.py	Adds public trace-aware results; wrapped fetch results break documented DataFrame usage.
dataconnect/​client.py	Exposes updated result APIs; documented iterable and DataFrame compatibility changes require migration or compatibility preservation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues affect malformed metadata, structured error parsing, error fallback propagation, and publish payload trace capture.

Review effort: Lite
Findings: 2

Open (2)

• Defensively decode server-provided raw trace IDs · New
• Capture trace IDs from do_put response payloads · New

Resolved since last review (4)

• Support legacy list-shaped response metadata
• Handle non-object response metadata safely
• Fetch results break documented DataFrame usage
• Breaking change: dataset versions result is no longer iterable

[Copilot]

Copilot review overview

🔵 Needs a closer look

Three moderate unresolved error-path issues can lose trace IDs or error classifications.

Review effort: Lite
Findings: None

Resolved since last review (2)

• Capture trace IDs from do_put response payloads
• Defensively decode server-provided raw trace IDs

[Copilot]

Copilot review overview

🟡 Changes recommended

Authentication normalization, header-only errors, and streaming error conversion can lose trace IDs.

Review effort: Balanced
Findings: 3

Open (3)

• Preserve trace IDs from original structured authentication errors · New
• Avoid reparsing TransportError instances in streaming fetch failures · New
• Propagate captured trace IDs through all error conversion paths · New

[Copilot]

🟡 Changes recommended

Interrupted streams can lose already-received trace IDs, and the new datetime-format envelope lacks transport tests.

3 open findings

• Capture trace ID before interrupted flight stream · New
• Preserve trace ID when chunk reading is interrupted · New
• Add transport coverage for response envelope trace IDs · New

3 resolved since last review

• Propagate captured trace IDs through all error conversion paths
• Avoid reparsing TransportError instances in streaming fetch failures
• Preserve trace IDs from original structured authentication errors

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟡 Changes recommended

Targeted checks confirmed trace loss in quoted wrapped errors and interrupted datetime-format streams.

2 open findings

• Quoted gRPC payloads are misparsed and lose error details · New
• Streaming completion errors lose the previously received trace ID · New

3 resolved since last review

• Add transport coverage for response envelope trace IDs
• Preserve trace ID when chunk reading is interrupted
• Capture trace ID before interrupted flight stream

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🔵 Needs a closer look

Three moderate issues remain in error parsing and multi-result trace metadata handling.

0 open findings

2 resolved since last review

• Quoted gRPC payloads are misparsed and lose error details
• Streaming completion errors lose the previously received trace ID

🧠 Review effort: Lite

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟡 Changes recommended

Locally rejected requests can leave trace_id reporting a stale value from the preceding request.

1 open finding

• Clear stale trace_id before validating each public request · New

🧠 Review effort: Balanced