Repository navigation
Flake batch 7: slow-embedding budget test asserts the budget, not runner speed - #1057
Merged
Merged
Conversation
…ses, not runner speed The 500 ms budget fired on time on a loaded Windows runner, but the lexical fallback after it took ~495 ms and tripped the single 1 s wall-clock bound. The test now checks that the deadline fired near the budget and that the reply never waited for the held response (now 10 s, released when the client disconnects), with the wall-clock bound at half the hold. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test-only fix.
aft_search_contract_test::slow_query_embedding_degrades_within_budget_and_next_query_retries_freshfailed on a loaded Windows runner (PR #1056's shard): the 500 ms query budget fired on time (516 ms), but the lexical fallback after it took ~495 ms and tripped the test's single 1 s wall-clock bound.The test now asserts from its recorded phases that the deadline fired near the budget (400 ms to 2.5 s) and bounds the reply at half the fake server's hold. The hold is now 10 s and ends early when the client disconnects, so a normal run still takes about 0.6 s. Next-query retry, request count and phase order assertions are unchanged.
Build server: test passed 4 of 4 runs; Windows target check with -D warnings.
Summary by cubic
Test-only fix for the flaky
slow_query_embedding_degrades_within_budget_and_next_query_retries_freshcontract test, which failed on a loaded Windows runner when the lexical fallback after the 500 ms budget took ~495 ms and tripped the single 1 s wall-clock bound.The test now asserts from its recorded phases that the deadline fired near the budget (400 ms to 2.5 s), and bounds the total reply at half the fake server's hold. The server hold is now 10 s and ends early when the client disconnects, so a normal run still takes about 0.6 s. Next-query retry, request count, and phase order assertions are unchanged.
Written for commit a5bdd6f. Summary will update on new commits.