Skip to content

OfficeQA smoke variant; deepseek routed to Azure by default - #96

Merged
varunursekar merged 2 commits into
mainfrom
officeqa-e2e-azure
Sep 13, 2026
Merged

varunursekar merged 2 commits into
mainfrom
officeqa-e2e-azure

Conversation

@varunursekar

@varunursekar varunursekar commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator
  • officeqa/baseline/build.e2e.yaml and officeqa/partitions-e2e/ (2 / 3 / 3 cases): a reduced variant with the same seed, target and gateway scopes, six-run budgets and a 25-minute agent clock, mirroring gaia/baseline/build.shell.e2e.yaml. For exercising the machinery end to end after a change; never reported.
  • The OfficeQA and BrowseComp-Plus builds now route deepseek-v4-flash to azure_ai/DeepSeek-V4-Flash by default. --param target_model_route= still selects any other deployment.

The variant compiles, and the benchmark-config tests pass.

🤖 Generated with Claude Code

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with no concrete correctness, security, or repository-rule failures identified.

Summary

  • Adds disjoint 2/3/3 development, validation, and test subsets for quickly exercising the complete OfficeQA pipeline.
  • Configures six evaluation runs per accessible partition and a 25-minute optimizer-agent clock.
  • Routes evaluation and finalization requests to azure_ai/DeepSeek-V4-Flash by default while retaining the target_model_route override.
  • Documents the smoke variant as non-reportable benchmark infrastructure.

Reviews (1) · Last reviewed commit: "OfficeQA README: list the smoke variant ..."

varunursekar and others added 2 commits September 13, 2026 01:34
…fault

build.e2e.yaml mirrors GAIA's: same seed, target and gateway scopes, eight cases,
six-run budgets, a 25-minute agent clock. Fireworks stays reachable through
--param target_model_route.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@varunursekar
varunursekar merged commit d140001 into main Sep 13, 2026
7 checks passed
@varunursekar
varunursekar deleted the officeqa-e2e-azure branch September 13, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant