Repository navigation
Conversation
The integration pinned `foundry-local-sdk>=0.5.1,<0.5.2`, which excludes every
SDK release currently published (1.2.4, 2.1.0, 2.2.1) and forces applications to
either downgrade the SDK or drop Foundry Local.
SDK 2.x replaced the v1 Python surface, so the adapter is migrated to it:
- The manager is now a process-wide singleton created from a `Configuration`.
`_resolve_foundry_local_manager()` reuses it so several clients and models can
share one manager, catalog, and web service instead of raising on the second
construction.
- Model resolution goes through the catalog. `get_model()` covers aliases and
`get_model_variant()` covers full model IDs, which the alias lookup misses.
- `device` now selects a variant by `variant.info.runtime.device_type` instead of
being passed to `get_model_info()`/`download_model()`/`load_model()`.
- `prepare_model` maps to `model.download()` and `model.load()`.
- The web service is the OpenAI-compatible transport, so it is started when
`bootstrap` is set and its bound URL becomes the client base URL
(`{url}/v1`). When neither bootstrap nor an already-running service yields a
URL, initialization fails with an actionable error instead of building an
OpenAI client against a placeholder.
- `timeout` is applied to the OpenAI client, which is where request timeouts
actually take effect now that inference is reached over HTTP.
The dependency range becomes `foundry-local-sdk>=2.0.0,<3`. SDK 2.x ships a
`cp311-abi3` wheel, so Foundry Local now requires Python 3.11+ in practice.
Tests are updated to the new SDK surface and cover the new behavior: web service
startup, the missing-endpoint error, base URL construction, timeout propagation,
model-ID resolution, and device variant selection.
GitHub Actions
Inactive
GitHub Actions
Error
agent-framework-automation
Bot
added
the
python
There was a problem hiding this comment.
🟡 Changes recommended
Interpreter compatibility, concurrent singleton initialization, and hardware-provider setup need correction before approval.
3 open issues
What changed in this PR
Migrates the Python Foundry Local integration to SDK 2.x to address #9321 while retaining Agent Framework’s OpenAI transport and middleware.
Changes:
- Adopts singleton manager initialization, catalog-based model selection, and the built-in web service.
- Updates the SDK dependency range and lockfile.
- Adapts unit tests, client documentation, and the sample to the new API.
| File | Description |
|---|---|
python/uv.lock |
Locks SDK 2.2.1 and its native-runtime dependencies. |
python/samples/02-agents/providers/foundry/foundry_local_agent.py |
Uses the new catalog and model metadata APIs. |
python/packages/foundry_local/tests/test_foundry_local_client.py |
Updates coverage for model resolution, endpoint startup, and timeouts. |
python/packages/foundry_local/tests/conftest.py |
Provides SDK 2.x manager and model mocks. |
python/packages/foundry_local/pyproject.toml |
Changes the supported SDK range to >=2.0.0,<3. |
python/packages/foundry_local/agent_framework_foundry_local/_foundry_local_client.py |
Migrates manager, model preparation, and HTTP endpoint setup. |
Review effort: Balanced
| manager = FoundryLocalManager.instance | ||
| if manager is not None: | ||
| return manager | ||
| FoundryLocalManager.initialize(Configuration(app_name=_APP_NAME)) |
There was a problem hiding this comment.
Fixed. The check-and-initialize sequence now runs under a module-level lock and re-checks the singleton inside it, so the second thread returns the existing manager instead of calling initialize() a second time.
The regression test mocks the SDK boundary and drives two threads through it. I confirmed it actually catches the race by temporarily removing the lock: initialize() then runs twice and the test fails with assert 2 == 1, which is exactly what the real SDK surfaces as a rejected second initialization.
| "agent-framework-core>=1.13.0,<2", | ||
| "agent-framework-openai>=1.6.0,<2", | ||
| "foundry-local-sdk>=0.5.1,<0.5.2", | ||
| "foundry-local-sdk>=2.0.0,<3", |
There was a problem hiding this comment.
Fixed and aligned in all four places. requires-python is now >=3.11 and the 3.10 and 3.15 classifiers are gone, matching the SDK's own Requires-Python: >=3.11,<3.15; the mypy target moved to 3.11; core's all extra now carries the marker python_version >= '3.11' and python_version < '3.15'; and the test matrix excludes the integration on both 3.10 and 3.15.
One correction on the lockfile: uv.lock already declared requires-python = ">=3.11, <3.15" upstream, before this PR, so regenerating it is a no-op and uv lock --check passes as-is. I did not force a regeneration because that would churn unrelated pinned versions.
| raise ValueError(message) | ||
| base_url = f"{manager.urls[0]}/v1" | ||
|
|
||
| model_info = _resolve_model(manager, model_setting, device=device) |
There was a problem hiding this comment.
Fixed. bootstrap now calls download_and_register_eps() before the catalog is read, so accelerated variants are registered and therefore discoverable instead of reporting "not found" or silently falling back to CPU.
I scoped it to skip when device is explicitly DeviceType.CPU, so those callers do not download providers they will not use. Two tests cover it: one asserts the call happens for a GPU variant, one asserts it does not happen for CPU while the CPU variant is still selected.
… providers Follow-up to the SDK 2.x migration, addressing review feedback. The singleton check-and-initialize sequence was racy: two threads creating their first clients concurrently could both observe an uninitialized manager, and the SDK rejects the second `initialize()` call, so one of them would fail instead of sharing the manager. The sequence now runs under a lock and re-checks the singleton inside it. The regression test drives two threads at the SDK boundary; without the lock the second `initialize()` call is reached and the test fails, so the race is covered rather than assumed. Catalog resolution for accelerated devices also needed preparation that was missing. A variant is only listed once its execution provider has been downloaded and registered, so on a fresh installation asking for GPU or NPU (or letting the SDK auto-select) could report "not found" or silently fall back to CPU. `bootstrap` now calls `download_and_register_eps()` before the catalog is read, and explicitly requesting CPU skips it so those callers do not download providers they will not use. Finally, the distribution metadata advertised interpreters the new dependency set cannot support: `foundry-local-sdk` 2.x declares `Requires-Python` `>=3.11,<3.15`, while this package still declared `>=3.10` and listed 3.10 and 3.15 classifiers. The requirement, the classifiers, and the mypy target move to 3.11, the `all` extra in core gains the matching environment marker, and the test matrix excludes the integration on 3.10 and 3.15. `uv lock --check` confirms the lockfile still satisfies the workspace.
GitHub Actions
Inactive
|
@microsoft-github-policy-service agree |
|
lisry-1 Thank you for the work here. This PR substantially duplicates the open SDK 2.x migration in #9322: both address #9321, and all eight files changed here are also changed there. #9322 is the broader team-owned migration and already covers the core upgrade path. I’m closing this newer PR to consolidate the work there; the automatic execution-provider registration idea can still be proposed as a focused follow-up if needed. |
Motivation & Context
agent-framework-foundry-localpinsfoundry-local-sdk>=0.5.1,<0.5.2, which excludes every SDK release published since (1.2.4, 2.1.0, 2.2.1). Users either downgrade the SDK or drop Foundry Local, and the v1 Python surface (FoundryLocalManager(bootstrap=..., timeout=...),get_model_info,download_model,load_model,endpoint,api_key) no longer exists in the current SDK, so simply widening the pin is not enough.This migrates the adapter to the SDK 2.x API while keeping Agent Framework's existing OpenAI transport and middleware intact.
Description & Review Guide
What are the major changes?
python/packages/foundry_local/agent_framework_foundry_local/_foundry_local_client.py:Configuration. A new_resolve_foundry_local_manager()reusesFoundryLocalManager.instancewhen present and otherwise initializes it, so several clients and models share one manager, catalog, and web service instead of raising on the second construction._resolve_model()looks the model up withcatalog.get_model()(aliases) and falls back tocatalog.get_model_variant()(full model IDs, which the alias lookup misses). This is stricter than before about reporting a miss and is covered by tests.device: previously forwarded toget_model_info/download_model/load_model. It now selects a variant byvariant.info.runtime.device_typeand callsselect_variant(); an unmatched device raises the same "not found" error, including the device in the message as before.prepare_model: maps tomodel.download()andmodel.load().bootstrapis set and the service is not up, it is started, and{manager.urls[0]}/v1becomes the OpenAI client base URL (api_key="none", matching the SDK's ownsamples/python/web-serverexample). If no URL is available, initialization now fails with an actionable error rather than building an OpenAI client against a placeholder.timeout: applied to theAsyncOpenAIclient, which is where request timeouts take effect now that inference is reached over HTTP.Dependency range in
pyproject.tomlbecomesfoundry-local-sdk>=2.0.0,<3;python/uv.lockregenerated withuv 0.13.5(addscffi/onnxruntime/onnxruntime-genai-core/flatbuffers, dropstqdmandhttpxfrom the SDK's dependency set).python/samples/02-agents/providers/foundry/foundry_local_agent.pyupdated tocatalog.list_models()and themodel.info.*metadata fields.What is the impact of these changes?
Foundry Local works with current SDK releases. SDK 2.x ships a
cp311-abi3wheel, so the integration now requires Python 3.11+ in practice even though the package still declares>=3.10; if you want that declared explicitly, I can bumprequires-pythonin a follow-up.DeviceTypenow comes fromfoundry_local_sdkrather thanfoundry_local.models, andclient.manager.download_model(...)/load_model(...)are replaced bycatalog.get_model(...)thenmodel.download()/model.load(). Both are documented in the client docstring.What do you want reviewers to focus on?
The endpoint lifecycle: whether reusing an already-running web service (rather than always starting one) matches the intended
bootstrapsemantics, and whetherbootstrap=Falsewith no running service should raise (current behavior) or fall back to starting it.Related Issue
Fixes #9321
The issue asks maintainers to confirm the supported SDK range and backward-compatibility policy before implementation. The range chosen here is
>=2.0.0,<3(current release 2.2.1). If the intended range differs, or ifbootstrap/deviceshould keep a different compatibility shape, I am happy to adjust.AI Assistance
AI assistance details:
AI tooling assisted with issue/source analysis, the implementation, the tests, and this PR description. Verification performed by the contributor: the package's unit tests (21) pass locally against
foundry-local-sdk==2.2.1,ruff checkandruff format --checkare clean, and the API surface used (FoundryLocalManager.initialize,Configuration,catalog.get_model/get_model_variant,IModel.download/load/select_variant,start_web_service,urls) was read from the installed 2.2.1 package and the SDK's published samples. Not verified: live inference against the native runtime, because no Foundry Local service or model weights are available in this environment — the endpoint format{url}/v1andapi_key="none"come from the SDK's own web-server sample rather than an end-to-end run.Contribution Checklist