Skip to content

fix: restore exact PostgreSQL list counts at scale - #160

Merged
rrrodzilla merged 1 commit into
mainfrom
fix/159-bounded-authorized-counts
Sep 10, 2026
Merged

rrrodzilla merged 1 commit into
mainfrom
fix/159-bounded-authorized-counts

Conversation

@rrrodzilla

Copy link
Copy Markdown
Contributor

Fixes #159.

Default PostgreSQL lists on large collections timed out while counting every matching row through Cedar. Generated Read policies can now use exact database counts after comparison of the applicable policy ASTs and storage certification establish equivalent authorization, including tenant isolation. Default totals remain exact and enabled.

Custom applicable Read predicates, explicit operator policies, unsupported storage shapes, and other backends retain the exact authorized scan. Each list captures its policy snapshot and prepares principal/action state once; the default Cedar adapter is no longer evaluated twice. count=false skips storage certification and stops after filling the readable page. Projection, field restrictions, filters, offsets, sorting, and relation enrichment keep their existing behavior.

The storage proof checks schema identity, physical columns, strict Cedar representation, and decoder limits under one PostgreSQL transaction snapshot. Unproven shapes fall back; I/O failures remain errors. Conservative string checks avoid the expensive regular expressions found during scale testing.

Validation completed:

  • 105 PostgreSQL library tests and five real PostgreSQL certification regressions passed.
  • Focused authorization/context/paging regressions passed, including custom policies, escaped identities, stale schema shapes, and captured policy reloads.
  • A real PostgreSQL HTTP regression passed for exact counts and projection, including malformed required data outside the requested page forcing fallback.
  • Affected PostgreSQL, acton, and CLI Clippy checks passed with warnings denied. The added regressions are included in PostgreSQL CI.
  • Independent synthetic collections of 202,628 rows passed nine HTTP checks. Simple GET/POST exact totals returned in 0.58 to 0.60 seconds; rich records, filtered counts, deep/past-end offsets, and projection returned in 0.82 to 0.91 seconds. Restricted fields and relation labels stayed correct. Page-only rich records returned in 0.12 seconds. The official v0.44.0 baseline returned HTTP 408 after 30.02 seconds.

Residual custom-policy work remains proportional to matching rows. A debug-build sparse-policy query over 20,000 candidates returned an exact total of 20, while the 202,628-candidate custom scan reached the request deadline. This patch does not claim globally bounded exact counts or infer release-build custom-policy timing from the debug build.

Includes CLI v0.44.1, integration crate v0.43.1, backend v0.17.1, PostgreSQL v0.12.1, and release notes so the fix needs only one PR validation cycle before tagging.

@rrrodzilla
rrrodzilla merged commit 2e95095 into main Sep 10, 2026
6 checks passed
@rrrodzilla
rrrodzilla deleted the fix/159-bounded-authorized-counts branch September 10, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Authorized total_count scans the entire collection, so list endpoints on large schemas return 408

1 participant