Skip to content

Bundle evaluations and dataset extensions in the Foundry meta-package - #10085

Open
Mohamed Hessien (m7md7sien) wants to merge 1 commit into
Azure:mainfrom
m7md7sien:feat/foundry-pack-eval-dataset
Open

Mohamed Hessien (m7md7sien) wants to merge 1 commit into
Azure:mainfrom
m7md7sien:feat/foundry-pack-eval-dataset

Conversation

@m7md7sien

@m7md7sien Mohamed Hessien (m7md7sien) commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Fixes #10086

Adds azure.ai.evaluations and azure.ai.dataset to the microsoft.foundry meta-package so a single azd extension install microsoft.foundry brings the evaluation authoring and dataset versioning commands along with the rest of the Foundry surface.

Both extensions are already on main at 1.0.0-beta.1 (#9500 and #9499), and both ship their own release pipeline (eng/pipelines/release-ext-azure-ai-evaluations.yml, eng/pipelines/release-ext-azure-ai-dataset.yml).

Changes

File Change
extension.yaml Two new dependencies, inserted in the existing alphabetical order; version 1.0.0-beta.31.0.0-beta.4
version.txt 1.0.0-beta.31.0.0-beta.4
CHANGELOG.md New 1.0.0-beta.4 entry
dependencies:
  - id: azure.ai.agents
    version: "~1.0.0-beta.16"
  - id: azure.ai.connections
    version: "~1.0.0-beta.7"
  - id: azure.ai.dataset          # new
    version: "~1.0.0-beta.1"
  - id: azure.ai.evaluations      # new
    version: "~1.0.0-beta.1"
  - id: azure.ai.inspector
    version: "~1.0.0-beta.7"
  ...

The pins use the same ~<version> style as every existing dependency, so the pack picks up patch releases of both extensions without another manifest change.

⚠️ Release sequencing

This PR is safe to merge now — it only edits extension.yaml, version.txt and CHANGELOG.md, not registry.json.

But the meta-package must not be released until azure.ai.evaluations and azure.ai.dataset are published to the registry. Neither is in registry.json yet (#10099 is still open).

Correction. An earlier revision of this description claimed ext-registry-ci would reject the pack's registry entry for the unpublished pins. That is wrong, and thanks to the Copilot review for catching it. Per validateDependencies, a dependency whose id is absent from the registry is reported as a warning, and TestRegistryFileIsValid fails only on errors — so registry validation would pass. An error is raised only when the id is present but the version constraint matches no published version.

So the gate is not CI, it is install-time resolution: azd resolves an extension's dependencies from the parent's own source when installing, so releasing the pack now would produce a microsoft.foundry that fails to resolve two of its dependencies for anyone installing it.

Correct order:

  1. Merge this PR.
  2. Release azure.ai.evaluations and azure.ai.dataset (their release pipelines → registry update PRs).
  3. Only then release microsoft.foundry 1.0.0-beta.4.

Worth noting for step 3: because the pins are ~1.0.0-beta.1 and the published versions will be exactly 1.0.0-beta.1, the constraint resolves cleanly. If the pack were ever released with a pin ahead of what is published (say ~1.0.0-beta.2), that would be a hard registry validation error rather than a warning.

Validation

  • Manifest parses; all nine dependency ids resolve to extension directories under cli/azd/extensions/, and every pinned version matches the version that extension declares.
  • No Go code touched.

Add `azure.ai.evaluations` and `azure.ai.dataset` to the `microsoft.foundry`
dependency list so installing the meta-package brings the evaluation
authoring and dataset versioning commands along with the rest of the
Foundry surface. Both extensions landed on main at 1.0.0-beta.1 in Azure#9500
and Azure#9499.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
3 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Correct the release-sequencing note to distinguish registry validation from install-time resolution.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates the microsoft.foundry meta-package to bundle dataset and evaluation extensions.

Changes:

  • Adds azure.ai.dataset and azure.ai.evaluations dependencies.
  • Bumps the package to 1.0.0-beta.4.
  • Documents the bundled extensions in the changelog.
File summaries
File Description
cli/azd/extensions/microsoft.foundry/extension.yaml Adds dataset and evaluations dependencies.
cli/azd/extensions/microsoft.foundry/version.txt Updates the package version.
cli/azd/extensions/microsoft.foundry/CHANGELOG.md Records the new release.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cli/azd/extensions/microsoft.foundry/extension.yaml
@@ -1,5 +1,11 @@
# Release History

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking comment untill the release of azure.ai.evaluations and azure.ai.dataset.
Feel free to resolve the comment when the two packages are released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bundle the evaluations and dataset extensions in the Foundry meta-package

2 participants