Skip to content

test(reporting): run the row-upload lease test at the wall clock - #1767

Merged
bokelley merged 1 commit into
nextfrom
claude/issue-1695-row-lease-clock
Nov 1, 2026
Merged

bokelley merged 1 commit into
nextfrom
claude/issue-1695-row-lease-clock

Conversation

@bokelley

@bokelley bokelley commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Unblocks CI on next. Since 2026-11-01, Postgres conformance tests (Postgres 16, core) has failed on every PR (#1764, #1765) with:

FAILED tests/conformance/reporting/test_reporting_row_storage_object_pg.py::test_the_producer_stages_object_uploads_outside_its_publication_transaction
ReportingRowStoreError: Reporting row store: STATE_UNAVAILABLE: no live worker lease covers this row upload

Cause

The test runs one producer turn with the settling harness, whose clock is fixed at 2026-11-01T02:10Z. The PG ledger store stamps the worker lease from that injected clock. The row upload (adcp_reporting_storage/ledger/row_write.py) then checks that lease against the database's clock_timestamp(). That passed while the real time was before the harness instant plus the lease. It fails on every run since the wall clock passed it. The last green next run was on 2026-10-31.

Fix

The test sets the harness clock to the wall clock before the turn, so the lease lies in the real future. The test still checks what it checked before:

  • the upload is staged outside the publication transaction;
  • one object is written;
  • rows read back, and verified rows read back;
  • no write intent is left.

Production code is unchanged.

Reproduced and verified against a local PostgreSQL 16: the test failed before the change, and the whole file passes after it (23 passed).

🤖 Generated with Claude Code

The producer row-upload test stamps its worker lease with the settling
harness clock, fixed at 2026-11-01T02:10Z, while the row store checks the
lease against the database's clock_timestamp. Once the wall clock passed that
instant plus the lease, every run failed with STATE_UNAVAILABLE: no live
worker lease covers this row upload, on next and on every pull request. The
test now runs its turn at the wall clock, so the lease lies in the real
future.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean test-only change.

Verified: single-file diff (7 net lines) in tests/conformance/reporting/test_reporting_row_storage_object_pg.py:443-457. The change captures the harness clock and advances clock[0] = datetime.now(timezone.utc) after setup and before run_worker(), so the stamped worker lease lands in the real future relative to the DB clock_timestamp() check — fixing the STATE_UNAVAILABLE failure that appeared once the wall clock passed the fixed harness instant. Production code, assertions, wire shape, public surface, generated types, and governance are all untouched.

No critical/high/medium findings. gated_paths is false, high_risk is false, no no-auto-approve team match, no prior decision. None of rows 1–8 fire; falls through to row 9.

@bokelley
bokelley merged commit c1c6d6e into next Nov 1, 2026
41 checks passed
@bokelley
bokelley deleted the claude/issue-1695-row-lease-clock branch November 1, 2026 22:10
bokelley added a commit that referenced this pull request Nov 2, 2026
Backport of #1767 (c1c6d6e) from `next`. Since 2026-11-01 every
`main`-based CI run fails
`test_the_producer_stages_object_uploads_outside_its_publication_transaction`
with `STATE_UNAVAILABLE: no live worker lease covers this row upload`
(#1792): the settling harness pins its clock at 2026-11-01T02:10Z, the
producer stamps `lease_expires_at` from that clock, and `row_write.py`
checks the lease against the database's `clock_timestamp()`. Once real
time passed the frozen instant plus the lease, no upload was covered.

Run the producer's turn at the wall clock so the stamped lease lies in
the database's real future. The lease check keeps its production
semantics; only the test's clock moves.

Refs #1792

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant