Qualify long-lived workflow histories on published Server artifacts

[rmcdaniel]

Scope

Qualifies Server #237 against the exact published Server 2.4.26 image, PHP SDK 2.1.6, Python SDK 2.3.7, Rust SDK 2.1.2, Workflow 2.2.18, Waterline 2.0.7 and CLI 2.1.2. This PR adds reproducible synthetic evidence and Server operating guidance. The product defects found during qualification were fixed and released in their owning repositories.

Results

• MySQL 8.4.5 and PostgreSQL 16.15 completed histories below, around and just beyond the 8,000/10,000-event guidance, including repeated 10,010-event runs. Results were exact, events contiguous and final backlog empty. The repeat report records task latency, worker memory, database growth, attempts and repair counts: event threshold and repetitions.
• A separate 1,030-event matrix crossed the 4 MiB warning and 5 MiB recommendation on both databases. A completed 10,010-event PostgreSQL history replayed offline three times in 0.334, 0.340 and 0.351 seconds; its full worker run took 439.95 seconds. Size and replay report with raw artifacts.
• Each published PHP, Python and Rust SDK completed a 4,000-signal/4,000-side-effect workflow with activities, timers and continuation. The exact published tuple passed 31/31 replay conformance scenarios. PHP, Python and conformance, Rust.
• A published Python worker recovered after PostgreSQL interruption and worker replacement without duplicate task attempts or final backlog. Targeted retention removed a completed run while preserving external payload bytes referenced by an active run, which then resumed. Recovery and retention report.

Operating decision

Keep the default warning at 8,000 events or 4 MiB and the continue-as-new recommendation at 10,000 events or 5 MiB. Server reference now explains the counters, continuation, recovery and retention practice. The evidence supports the tested published tuple and limits; it does not claim a universal capacity number. Automatic age-based expiry and remote object-store cleanup were not exercised by the targeted retention run.

Checks

• Repository contracts and public-boundary scans passed on this head.
• PHPUnit feature suite is running on this head.
• The exact published tuple passed 31/31 replay conformance scenarios; all bounded synthetic stacks, volumes and probe images were removed.

[rmcdaniel]

Mixed-history candidate slice is now in commit bda3e5c3. The evidence README links the raw probe logs, durable snapshots, task timings, offer timestamps and PHP allocation logs.

Published Server 2.4.21 (index sha256:3985988df14263102056cfc65cb4a8beb9e4ad2335ee91a38bcea74e214fe807) ran with Workflow #568 candidate 141eeb15 mounted read-only, published Python SDK 2.3.5, MySQL 8.0, Redis 7, PHP 128 MiB and synthetic data. A cold worker restart followed 4,000 acknowledged signals. Eight activity/timer pairs ran before continue-as-new. The first processing worker timed out after a bounded 900-second window during the fifth activity; its lease expired and a fresh SDK worker retried and completed it once. Final SDK verification retrieved 4,049 ordered events across two runs and the exact result {count:4000,total:7998000}. Initial and successor history/timeline row counts were 4,047/4,047 and 2/2. There were no open tasks, rejected signals or failed jobs. The initial summary crossed the 5 MiB continue-as-new recommendation at 5,429,147 bytes.

This is correctness and recovery evidence with a serious latency finding, not a qualified limit. The first-to-last durable signal span was 441.225 seconds. The first run finished about 34 minutes 50 seconds after the last signal event, including timeout and lease wait. For the 16 post-offer workflow tasks, created-to-completed p50/p95/p99 were 15.74/160.00/160.00 seconds. A MySQL snapshot showed a scheduler transaction modifying 3,308 rows and holding 4,601 locks while an HTTP completion transaction waited on a task lock. Workflow #573 and draft PR #574 own a bounded repair path. The highest logged PHP request allocation was 92 MiB. Eight queue-worker container restarts occurred without Docker OOM; the exact recycling cause was not established. Active SDK-worker peak RSS and per-signal API latency for this timed-out run were not captured.

The updated fixture emits offer metrics before worker processing, accepts a bounded worker window and resume run ID, and verifies legitimate activity retry starts separately from single durable completions. A fresh 10-signal default-mode smoke passed. This branch remains draft pending the same-shape comparison with the watchdog fix, PostgreSQL, external payloads, retention and published-package conformance.