Skip to content

fix(engine): keep parse-and-retry with langchain-openai 1.7's refusal check - #450

Open
nadeem4 wants to merge 2 commits into
mainfrom
fix/pydantic-2-14
Open

nadeem4 wants to merge 2 commits into
mainfrom
fix/pydantic-2-14

Conversation

@nadeem4

@nadeem4 nadeem4 commented Nov 4, 2026

Copy link
Copy Markdown
Owner

What

structured() and streaming_structured() now treat everything before the parser as the model, so the parse-and-retry wrapper applies to langchain-openai 1.7's three-step chain (model, refusal check, parser).

Why

CI started failing (20 unit tests, test (3.12), test (3.13), integration) after dependencies moved to langchain-core 1.7.5 / langchain-openai 1.7.4. It is not pydantic 2.14: ValidationError is still a ValueError there. langchain-openai 1.7 inserts a _check_refusal step between the model and the parser, so _split (which required exactly two steps) returned None, the retry wrapper was skipped, and a bad answer's ValidationError escaped instead of being retried. The extra steps run directly under the caller's run (not as a nested sequence) so the trace still attaches the parsed answer to the model call.

How to test

pytest packages/nl2sql/tests/unit/test_parse_and_retry.py test_chat_tools.py test_conversations.py test_engine_events.py test_cost_and_caps.py test_limits_and_join_keys.py test_llm_providers_v2.py. Passes on pydantic 2.14.2 + langchain-openai 1.7.4 and on pydantic 2.13.5 + langchain-openai 1.6.2.

Checklist

  • Failing tests first (the 20 CI failures reproduced locally)
  • Boundary tests pass
  • No doc change needed (behaviour unchanged)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant