docs(ai-agent): standardize help and explain environments in azd ai agent - #10057
Open
FanH (Siglud) wants to merge 2 commits into
Open
FanH (Siglud) wants to merge 2 commits into
FanH (Siglud) wants to merge 2 commits into
Conversation
Document core azd help conventions and apply inherited styled help to all agent commands. Add environment context and help snapshots without changing command behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
FanH (Siglud)
requested review from
JeffreyCA,
Rick Winter (RickWinter),
Glenn Harper (glharper),
Marina He (hemarina),
Hui Miao (huimiu),
Zhijie Huang (hund030),
Wei Meng (m5i-work),
Richard Park (richardpark-msft),
tg-msft,
John Miller (therealjohn),
Travis Angevine (trangevi),
JianW (v1212) and
Victor Vazquez (vhvb1989)
as code owners
September 16, 2026 09:34
|
Azure Pipelines: Successfully started running 2 pipeline(s). 21 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Two moderate findings and one nit remain unresolved.
Pull request overview
This PR standardizes azd ai agent help output and documents environment behavior across the agent extension.
Changes:
- Adds inherited, color-aware help formatting with full usage paths, aligned flags, and examples.
- Adds environment, deployment, and runtime-variable guidance.
- Updates command help, snapshots, tests, documentation, spelling configuration, and dependencies.
File summaries
| File | Summary |
|---|---|
docs/guides/adding-a-new-command.md |
Links contributor guidance to help conventions. |
cli/azd/extensions/azure.ai.agents/tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yaml |
Updates root-help scenario expectations. |
cli/azd/extensions/azure.ai.agents/internal/helpformat/helpformat.go |
Implements shared help rendering. moderate (1 vote): shell-style placeholders are not highlighted. moderate (1 vote): uncaptained eval generate examples render without captions. |
cli/azd/extensions/azure.ai.agents/internal/helpformat/helpformat_test.go |
Adds formatter coverage. |
cli/azd/extensions/azure.ai.agents/internal/cmd/version.go |
Updates version-command help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/update.go |
Updates command help and examples. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-version.snap |
Updates version help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-toolbox.snap |
Updates toolbox help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-toolbox_add.snap |
Updates toolbox-add help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-show.snap |
Updates show help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sessions.snap |
Updates sessions help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sessions_stop.snap |
Updates session-stop help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sessions_show.snap |
Updates session-show help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sessions_list.snap |
Updates session-list help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sessions_delete.snap |
Updates session-delete help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sessions_create.snap |
Updates session-create help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sample.snap |
Updates sample help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-sample_list.snap |
Updates sample-list help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-run.snap |
Updates run help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-root.snap |
Updates root help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-publish.snap |
Updates publish help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-pack.snap |
Updates pack help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-optimize.snap |
Updates optimize help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-optimize_status.snap |
Updates optimize-status help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-optimize_list.snap |
Updates optimize-list help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-optimize_deploy.snap |
Updates optimize-deploy help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-optimize_cancel.snap |
Updates optimize-cancel help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-optimize_apply.snap |
Updates optimize-apply help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-monitor.snap |
Updates monitor help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-invoke.snap |
Updates invoke help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-invocations.snap |
Updates invocations help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-invocations_show.snap |
Updates invocation-show help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-invocations_follow.snap |
Updates invocation-follow help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-invocations_cancel.snap |
Updates invocation-cancel help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-init.snap |
Updates init help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files.snap |
Updates files help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files_upload.snap |
Updates file-upload help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files_stat.snap |
Updates file-stat help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files_mkdir.snap |
Updates file-mkdir help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files_list.snap |
Updates file-list help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files_download.snap |
Updates file-download help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-files_delete.snap |
Updates file-delete help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-eval.snap |
Updates evaluation help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-eval_update.snap |
Updates evaluation-update help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-eval_show.snap |
Updates evaluation-show help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-eval_run.snap |
Updates evaluation-run help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-eval_list.snap |
Updates evaluation-list help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-eval_generate.snap |
Updates evaluation-generation help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-endpoint.snap |
Updates endpoint help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-endpoint_update.snap |
Updates endpoint-update help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-endpoint_show.snap |
Updates endpoint-show help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-doctor.snap |
Updates doctor help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-delete.snap |
Updates delete help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-connection.snap |
Updates connection help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-connection_add.snap |
Updates connection-add help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-code.snap |
Updates code help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/testdata/TestAgentHelp-code_download.snap |
Updates code-download help snapshot. |
cli/azd/extensions/azure.ai.agents/internal/cmd/session.go |
Updates session command help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/sample.go |
Updates sample command help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/root.go |
Installs the shared help renderer. |
cli/azd/extensions/azure.ai.agents/internal/cmd/invoke.go |
Updates invoke command help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/invocations.go |
Updates invocation lifecycle help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go |
Updates initialization help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/help.go |
Adds environment guidance. nit (1 vote): azd env set should use <key> <value> notation. |
cli/azd/extensions/azure.ai.agents/internal/cmd/help_test.go |
Adds help snapshot and color coverage. |
cli/azd/extensions/azure.ai.agents/internal/cmd/files.go |
Updates file-operation help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/eval.go |
Updates evaluation help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/eval_update.go |
Updates evaluator-update help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/eval_show.go |
Updates evaluation-display help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/eval_run.go |
Updates evaluation-run help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/eval_list.go |
Updates evaluation-list help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/doctor.go |
Updates diagnostic help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/code.go |
Updates code-management help. |
cli/azd/extensions/azure.ai.agents/internal/cmd/add.go |
Updates dependency help. |
cli/azd/extensions/azure.ai.agents/go.sum |
Records dependency checksums. |
cli/azd/extensions/azure.ai.agents/go.mod |
Adds snapshot-testing dependency metadata. |
cli/azd/extensions/azure.ai.agents/cspell.yaml |
Adds extension spelling configuration. |
cli/azd/docs/style-guidelines/azd-style-guide.md |
Documents help standards and renderer usage. |
cli/azd/.vscode/cspell.yaml |
Adds shared spelling terms. |
Review details
Suppressed comments (3)
cli/azd/extensions/azure.ai.agents/internal/cmd/help.go:12
azd env settakes an environment-variable key as its first positional argument, not an environment name; the selected environment is chosen separately viaselector--environment. Using<name>here conflicts with the preceding environment-name examples and can lead users to run the wrong command. Please use the same<key> <value>notation as the core command help.
Use 'azd env set <name> <value>' to save a value and 'azd env get-values' to
cli/azd/extensions/azure.ai.agents/internal/helpformat/helpformat.go:50
- The new root footer contains
${FOUNDRY_PROJECT_ENDPOINT}and${VARIABLE_NAME}, but this token pattern only recognizes angle- and square-bracket placeholders. As a result, shell-style environment substitutions remain unhighlighted in colored help even though the renderer promises placeholder highlighting and styles the surrounding command/flag tokens. Add an environment-variable placeholder token and route it throughWithWarningFormat(with coverage for the footer form).
cli/azd/extensions/azure.ai.agents/internal/helpformat/helpformat.go:181 - The new help conventions require a short
#caption above each copyable command, but this fallback renders every non-caption line as a command. The existingeval generatemetadata has four command lines with no captions, so its new snapshot is an undifferentiated block unlike the other public commands. Add captions to that example metadata (and refresh its snapshot), or otherwise reject/handle uncaptained examples consistently.
- Files reviewed: 78/79 changed files
- Comments generated: 0
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Resolve the invoke help conflict by retaining upstream platform latency diagnostics and the existing long-running invocation guidance. Refresh the invoke help snapshot for the added flag and example. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Two review follow-ups remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 78/79 changed files
- Comments generated: 2
- Review effort level: Lite (auto)
Note
Copilot is running an experiment and ran this review at Lite.
| Use 'azd env new <name>' to create one, 'azd env select <name>' to change the | ||
| default, or --environment <name> (-e) to select one for a command. | ||
|
|
||
| Use 'azd env set <name> <value>' to save a value and 'azd env get-values' to |
Comment on lines
+33
to
+37
| Examples | ||
| azd ai agent eval generate | ||
| azd ai agent eval generate --gen-instruction "This agent handles restaurant reservations." --eval-model gpt-4o --max-samples 50 | ||
| azd ai agent eval generate --gen-instruction-file ./instructions.md --eval-model gpt-4o | ||
| azd ai agent eval generate --dataset ./tests/golden.jsonl --evaluator builtin.intent_resolution |
Contributor
azure.ai.agents PR buildNote This is an unsigned development build. Install it only if you trust this PR. Install the extension: azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/10057/azure-ai-agents.zip"
|
Hui Miao (huimiu)
approved these changes
Sep 17, 2026
FanH (Siglud)
enabled auto-merge (squash)
September 17, 2026 07:57
FanH (Siglud)
disabled auto-merge
September 17, 2026 07:57
John Miller (therealjohn)
approved these changes
Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8506 Part.1
Summary
Partially addresses #8506, scoped to
azd ai agentand its descendants. Otherazd ainamespaces are intentionally unchanged.azd --help, in the CLI style guide and link them from the contributor command guide.azd ai agentusage paths, and examples for every public command. Retain the root banner and simplify duplicated eval command guidance..azure/<environment>/.env,azd env set/get-values, Foundry project and per-service deployment values, and service-level runtimeenvsubstitutions.The renderer keeps Cobra metadata intact and routes through the SDK's
UsageFunc, preserving per-command output formats/defaults. Examples retain command ordering, quoting, and multiline continuations. Help does not require Azure authentication or inspect user configuration.Verification
Using Go 1.26.4 from the agent extension directory:
go buildgo test ./internal/helpformat ./internal/cmd -short -count=1golangci-linton the formatter and changed command code; scoped spelling checks.Reused azd's snapshot helper, adding its existing
cupaloydependency to this extension's module.Interactive scenario
Updated
tests/cli-interactive-tester-scenarios/tier0/0.02-help-root.yamlto cover consistent formatting and environment guidance. It was not run because these scenarios require the interactive tester setup. For terminal visual validation, use thefoundry-extension-scenario-orchestratoragent with thefoundry-extension-scenario-pr-regressionskill.Comparison
Before
After