Skip to content

Record the verdict when publication decides it; pin contract v1.0.4 (#390) - #392

Merged
lbacik merged 2 commits into
mainfrom
fix/issue-390-recorded-verdict
Nov 11, 2026
Merged

lbacik merged 2 commits into
mainfrom
fix/issue-390-recorded-verdict

Conversation

@lbacik

@lbacik lbacik commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

Fixes #390

The engine now records an attempt's verdict when publication decides it, and no recovery path recomputes it afterwards (contract §14). With the suite pinned at v1.0.4, the producer job passes recovery-release.

Changes

  • Recording at publication. As soon as the publisher returns, the engine writes the outcome and a short reason into the attempt checkpoint (new optional verdict field) and the outcome into the archive record. This happens before the result-comment check, the issue release and cleanup. Only the first recording counts. Old checkpoints without the field read as "no recorded verdict".
  • outcome_recorded once per attempt. It is emitted at the recording moment. _commit_finalization emits it only when no verdict was recorded, so an attempt interrupted before publication returned still announces its outcome once. The Contract Event order is unchanged.
  • recovery release finalizes with the recorded verdict and lifecycle_state: released. The confirmed release comment names the verdict and --saved-at. It is followed by a plain release (never round-finished), and the branch is retained.
  • recovery retry republishes towards the recorded verdict and finalizes with it. It answers not_fulfilled and keeps the hold when the verdict is unreachable or the publisher answers another outcome. Two cases are unreachable: a handoff whose note is gone, and a complete the checkpoint no longer shows as publishing, which would push the branch first.
  • Startup reconciliation after a restart steers the same way. Before this change it re-inferred the verdict and could finalize a recorded infrastructure_error as complete. When the verdict cannot be reached, the attempt stays retained for the operator and the engine emits recorded_verdict_held.
  • No recorded verdict → today's inference, unchanged, including Recover and release retained implementation attempts #84's rule that an unpublished branch is never a success.
  • Docs. ADR 0010 records the decision and its relation to Recover and release retained implementation attempts #84. CONTEXT.md gains "Recorded verdict" and "Retained Attempt". docs/conformance-harness.md now says the after-outcome hold takes effect before the result comment is confirmed.
  • Pin. pyproject.toml names agent-operator-contract v1.0.4. That release fixes recovery-release scenario expects a claim to open without a release verb agent-operator-contract#39: the last step of recovery-release used to expect a Claim without a release verb, and it now opens the later Claim with set-issue followed by release. Harness protocol stays 1.1. uv.lock is gitignored, so only the local lockfile was regenerated. It names rev=v1.0.4#5e17e8c.
  • Tests. The new tests/test_recorded_verdict.py covers release, retry, restart and once-only emission. The checkpoint tests cover the new field. The xfail is removed from test_recovery_release_finalizes_a_retained_attempt. One event-order test now expects outcome_recorded before the error-limit halt. Two retry tests now advance the checkpoint to publishing, as the production publisher does, because the fake publisher does not.

Verification

  • Producer suite v1.0.4 in Docker, --require-all with all capabilities: 41 PASS, exit 0, including recovery-retry and recovery-release.
  • The same with --capabilities "": 41 PASS, exit 0.
  • At v1.0.3, before the pin bump, both runs were 40 PASS and 1 FAIL. The only failure was the last step of recovery-release (a later eligible issue must open a claim after the release). Every verdict check already passed.
  • tests/test_conformance_harness.py and the contract tests on v1.0.4: 128 passed.
  • uv run pytest locally on macOS: 2017 passed, 10 failed. The 10 (test_attempt_processes.py, test_control_offline.py) fail identically on a clean main on that machine.

🤖 Generated with Claude Code

lbacik and others added 2 commits November 11, 2026 10:11
The engine now writes an attempt's verdict (outcome plus a short reason)
durably into the attempt checkpoint as soon as the publisher returns,
before the result comment is confirmed, the issue released or the
workspace cleaned up, and emits `outcome_recorded` at that moment, once
per attempt. Finalization announces the outcome only when no verdict was
recorded (an attempt interrupted before publication returned, or an
older checkpoint).

A Retained Attempt now keeps that verdict:

- `recovery release` finalizes with the recorded verdict instead of
  inferring one from the workspace, and its release comment names it.
- `recovery retry` and startup reconciliation after a restart republish
  towards the recorded verdict. When that cannot be done safely, or the
  publisher answers another outcome, retry answers not fulfilled and
  reconciliation keeps the attempt retained (`recorded_verdict_held`).
- Without a recorded verdict, today's inference is unchanged, including
  the rule from #84 that an unpublished branch is never a success.

ADR 0010 records the decision and its relation to #84. CONTEXT.md gains
"Recorded verdict" and "Retained Attempt", and the harness docs describe
the earlier `after-outcome` hold. The harness test for `recovery release`
loses its xfail marker.

The pinned producer suite v1.0.3 now passes every verdict check of
`recovery-release`. Its last step still fails, because it expects a
claim to open without a `release` verb; that is reported upstream as
lbacik/agent-operator-contract#39.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v1.0.4 opens the later Claim of the recovery-release scenario through
`set-issue` and `release`, as every other scenario does
(lbacik/agent-operator-contract#39). With the recorded verdict in place,
the producer suite now passes recovery-release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lbacik lbacik changed the title Record the verdict when publication decides it (#390) Record the verdict when publication decides it; pin contract v1.0.4 (#390) Nov 11, 2026
@lbacik
lbacik merged commit 886d71c into main Nov 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant