Conversation
…enants [AE-2294] Tool and input simulation fell back to a hardcoded gpt-4.1-mini whenever the strategy had no model. Coded agents never carry a model in their runtime schema, so every coded agent simulation hit the UiPath-hosted default, which BYOM-only governance policies block. UIPATH_SIMULATION_MODEL now fills that gap, and the resolved model is actually sent instead of only being logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
github-actions
Bot
added
test:uipath-langchain
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
No unresolved blocking issues were identified.
Review effort: Lite
Findings: None
What changed in this PR
Enables coded-agent simulations to select BYOM models through configurable model resolution, with uipath bumped to 2.14.26.
Changes:
- Resolves models from strategy settings,
UIPATH_SIMULATION_MODEL, or the legacy default. - Applies resolved models to mocker requests and cache keys.
- Adds coverage for resolution and BYOM request propagation.
| File | Description |
|---|---|
packages/uipath/uv.lock |
Updates locked package version. |
packages/uipath/tests/cli/eval/mocks/test_simulation_model.py |
Tests model resolution and BYOM propagation. |
packages/uipath/src/uipath/eval/mocks/_simulation_model.py |
Resolves simulation models. |
packages/uipath/src/uipath/eval/mocks/_llm_mocker.py |
Uses resolved tool-simulation models. |
packages/uipath/src/uipath/eval/mocks/_input_mocker.py |
Uses resolved input-generation models. |
packages/uipath/pyproject.toml |
Bumps the package version. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Coded agent tool simulation always ran on
gpt-4.1-mini-2025-04-14, so BYOM-only tenants get a governance 403 and the whole run dies withUiPathMockResponseGenerationError. Siemens Mobility is blocked on this (SF 03028630).Why it happens
LLMMockerandLLMInputMockerfall back toChatModels.gpt_4_1_mini_2025_04_14when the strategy has no model.cli_run/cli_debugfill the model fromschema.metadata["settings"]["model"], which only the low-code runtime sets fromagent.json. Coded agents never set it, so every coded agent simulation goes to the UiPath-hosted default and skips AI Trust Layer BYOM. Turning simulation off works because the real tool path uses the customer's own model.The fallback was also only logged. When no model was configured,
completion_kwargswent out without one, so the LLM gateway client's own default decided the model.What changed
_simulation_model.pywithsimulation_completion_kwargs, used by both mockers. Model order: the strategy's model (simulation config oragent.json), thenUIPATH_SIMULATION_MODEL, then the old default.completion_kwargs, so what we log is what we send.For Siemens the fix is one line in the coded agent's
.env:UIPATH_SIMULATION_MODEL=<their SDC alias>.Call outs
requesting_product/agenthub_configheaders the mocker sends. If their policy is scoped by product, that is a separate follow-up.register_llm_mocker, defaulting to the LLMGW mocker) is the longer-term fix and will come as a follow-up. This PR unblocks the customer without a new public API.Testing
tests/cli/eval/mocks/test_simulation_model.py: resolver order, and an end-to-end mockable call with no model configured assertingX-UiPath-LlmGateway-NormalizedApi-ModelNamecarries the env model.pytest tests/cli/evalgreen,ruff check,ruff format --check,mypyclean.Jira: AE-2294
🤖 Generated with Claude Code