Skip to content

revert: remove cross-target render prefetch (#582) - #584

Merged
timtofan merged 1 commit into
masterfrom
atimofieiev/revert-render-prefetch
Oct 22, 2026
Merged

timtofan merged 1 commit into
masterfrom
atimofieiev/revert-render-prefetch

Conversation

@timtofan

Copy link
Copy Markdown
Collaborator

Why

Revert the runtime change from #582 while continuing the tail-latency investigation. Overlapping backend reads helped multi-target requests in the canary, but the measurements did not establish an overall p99/p99.9 improvement. Follow-up observations still do not justify retaining the extra speculative backend work across the fleet.

This is a conservative rollback of request scheduling, not a claim that prefetch caused the backend deadline failures. Those failures were also observed with the previous implementation. Restoring the prior scheduling provides a known baseline for a controlled comparison.

Architecture

The render handler again parses, fetches, and evaluates top-level targets in request order through the existing worker queues. It no longer starts request-scoped futures for the static inputs of later targets. Remove the prefetch implementation and its dedicated tests along with the handler integration.

The application runtime source returns exactly to the pre-#582 revision. The separate test-only commit from that PR is deliberately retained: d4101ee increased the race-test package timeout from 10s to 60s because the existing streaming-decoder oracle exceeded 10s on CI runners. Reverting that timeout would restore an unrelated CI failure and would not affect the deployed binary.

The five existing backend/render performance switches, their implementations, backend timeouts, worker limits, and configured batching remain unchanged. The defaults change in #583 is separate and can be reviewed independently; its diff also applies cleanly after this revert.

Before / after

With #582 prefetch After this revert
In the eligible multi-target path, static inputs of later targets can begin fetching before their evaluation Each top-level target fetches its data when evaluated
Request-scoped futures retain prefetched results and deduplicate missing inputs The previous handler/cache behavior is restored
Expressions still evaluate in target order, with prefetch cancellation/draining on exit Expressions evaluate in target order with the previous request lifecycle
Potential multi-target latency benefit, with additional overlapping backend work Forgo that benefit while restoring the prior scheduling baseline
Race-test package deadline is 60s Deadline remains 60s; no production behavior depends on it

Validation

No production rollback has been performed by this PR. After merge, deploy through the normal release process and compare warmed p99/p99.9, errors, traffic, and per-request resource use. Keep the five existing =1 service overrides until a defaults-enabled binary is deployed.

@timtofan
timtofan merged commit 9bea4ce into master Oct 22, 2026
6 checks passed
@timtofan
timtofan deleted the atimofieiev/revert-render-prefetch branch October 22, 2026 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants