Repository navigation
feat(cache): run the handler once for concurrent misses with coalesce=True - #440
Merged
Merged
Conversation
…=True `@cache(coalesce=True)` coalesces concurrent misses of one key within a process. The first GET to miss renders as usual; requests that miss the same key meanwhile wait for it, then read the backend again and are served the stored entry. If nothing was stored, each waiting request renders itself, all at once. HEAD waits for a running GET but never leads, and a request whose backend read fails does not wait. Off by default. coalesce must be a bool, needs a positive ttl and is rejected with private or no_cache; no_store ignores it with the usual warning. Closes #252
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #252
What
@cache(coalesce=True)(off by default) coalesces concurrent misses of one cache key within a process:asyncio.shield, so a cancelled follower does not cancel the others), then read the backend again and are served the stored entry, withETag,Ageand 304 handling like any hit.private, error status, streamed response, handler raised, cancelled), each waiting request runs the handler itself, concurrently, with no queueing.finally, however the request ended.The registry lives in the new
fastapi_cachex/_coalesce.pyand is keyed by event loop, backend and cache key, so different apps or loops never wait on each other. Each worker process still runs the handler once per cold key; the docs point toget_or_set()stampede protection for a cross-worker lock.Validation
coalescemust be abool(CacheXError).ttland is rejected withprivateorno_cache, which never serve a stored entry (CacheXError).no_storeit is listed in the existing "ignored arguments"UserWarning.Docs
HTTP_CACHING.md: new "Concurrent misses" section.APP_CACHE.md: cross-link from stampede protection.CACHE_FLOW.md: module list, decision logic and the validation list.changelog.d/252.added.md.Tests
tests/test_cache_coalesce.py(19 tests) covers:coalesce=False;Each guard was mutation-checked: removing
finish(), the shield, the re-read, the registry cleanup, the HEAD rule, the failed-read rule or the validation makes a test fail. The tests wait on events with a 5 s timeout instead of sleeping.