Qualify fresh H200 replicas from a network volume below 60 seconds #68

Qualify the published warmctl/DMTCP releases for scaling a fresh H200 replica from a prepared high-performance network volume in less than 60 seconds to the first external streaming token. Earlier 20.83-second local measurements excluded provisioning and network reads.

The acceptance path uses a cached stock Runpod image, a preverified Python runtime on the volume, the independently released DMTCP bundled with warmctl, and one immutable snapshot. Create the snapshot locally, deep-verify, copy with verified/fsynced files, deep-verify the volume copy, then reuse it for every replica.

Observed bottlenecks:

• Per-pod checking/extraction of the full Python archive cost 43.90 s; connecting its prepared tree took 0.40–0.58 s.
• More prefetch readers can slow the critical path: 64 readers improved aggregate prefetch but delayed filesystem setup and increased total restore time.
• The 20.4-core placement showed substantial variation; the tested 27-vCPU placement has 22.95 cores of quota and 287 GB RAM.
• Harness failures with cgroup v1, a default-Python-User-Agent rejection and an initial streaming HTTP 404 remain excluded from performance claims.

Two final controlled runs reached the first external token in 52.1408 s and 52.9356 s, including 5.5727 s and 7.2785 s of test-only client-cache advice. warmctl serving took 38.4739 s and 38.6915 s. Both used 16 prefetch workers with process/weight lanes together, THP and 16 DMTCP readers. An ordinary fresh pod without cache advice took 32.9252 s, with warm host cache. Weight fingerprints, streaming READY, tool calls and 32 requests per replica passed; the pods were deleted afterward.

Acceptance:

• Publish and verify standalone DMTCP packages and the warmctl release.
• Preserve one verified snapshot on the network volume.
• Repeat fresh-pod creation to external first token below 60 seconds.
• Check streaming, tools, fingerprints and 32 requests; tear down after each run.
• Publish the measured startup recipe and cache/resource boundaries: merged in Document sub-minute H200 startup from a network volume #69.

This is functional/performance qualification for this model/hardware configuration, not an SLA. Numerical canary equivalence remains unqualified at unchanged 0.001 tolerance. Direct-publication recovery is separately tracked in #67.