warmctl is GPU checkpoint/restore for vLLM. It snapshots a warmed-up engine and restores it in seconds on any host with the same GPUs and NVIDIA driver branch, so a new replica skips weight loading, torch.compile, kernel autotuning and CUDA graph capture. GLM-5.3-Flash on 8×H100: vLLM cold start 650 s, warmctl up 37 s, then stock serving speed. That makes scale to zero, fast autoscaling and spot GPU replacement practical for large models.

Docs · Catalog · Blog · Status and limits · Alternatives

Quickstart

Linux GPU host, Docker with the NVIDIA runtime, root:

curl -fsSL https://warmctl.io/install.sh | sh  # newest release in /opt/warmctl
sysctl -w kernel.io_uring_disabled=2           # CRIU cannot dump io_uring
warmctl doctor                                 # check the host and its GPUs
warmctl catalog                                # models this host can restore
warmctl up qwen3-8b --listen 0.0.0.0:8000      # pull, restore, serve

The public catalog has 21 models on H100, H200, B200, B300, L40S and RTX PRO 6000. Any other model vLLM serves with sleep mode, build yourself (tutorial):

warmctl build --profile qwen3-8b-tp1 --model-dir /models/qwen3-8b --gpus 0 --output /snaps/qwen3-8b
warmctl push /snaps/qwen3-8b ghcr.io/you/qwen3-8b
warmctl up ghcr.io/you/qwen3-8b

Measured

warmctl up to serving, snapshot in page cache, against a vLLM cold start (weights on local disk, empty compile caches).

Model	GPUs	Image	warmctl up	vLLM cold start
GLM-5.3-Flash	8×H100	404 GiB	37 s	650 s
DeepSeek-V4.1-Flash	8×H100	678 GiB	78 s	357 s
DeepSeek-V4.1-Flash-1017	8×B300	729 GiB	76 s	~1026 s
DeepSeek-V4.1-Flash-1017, DEP8	8×B200	809 GiB	84 s	1077 s
DeepSeek-V4.1-Flash-1017, DEP8	8×B300	851 GiB	88 s	930 s
gpt-oss-120b	1×B200	78 GiB	13 s	351 s
Gemma 4 26B-A4B	1×B200	60 GiB	12 s	~330 s
Qwen3-32B	1×B200	72 GiB	13 s	~300 s
Qwen3.6-35B-A3B	1×B200	46 GiB	10 s	~240 s
Qwen3-8B	1×H100	23 GiB	9 s	~80 s

~ is the time to a ready engine while building the snapshot; where we measured both, a plain cold start was 1–24% shorter. On B200, GLM-5.3-Flash restores in 44 s (8×B200) and DeepSeek-V4.1-Flash in 73 s (4×B200). Every platform is in the catalog.

GLM-5.3-Flash on 8×H100 by phase, from page cache and from a cold network file share (about 10 GB/s):

Phase	Page cache	File share, cold
CRIU restores the process tree	8.1 s	15.3 s
GPU state restored	10.5 s	13.3 s
Weights reloaded (320 GiB)	12.3 s	19.3 s
Serving	37.3 s	56.8 s

Restored engines serve at stock vLLM speed or faster (GLM-5.3-Flash: 100–136% of stock throughput).

How it works

build starts vLLM in a container, warms it over every shape in the profile and checkpoints the processes (CRIU), the GPUs (cuda-checkpoint) and the weights. push stores the snapshot as an OCI artifact, one per platform. up checks the host, restores everything at the same GPU addresses, runs a canary against the original engine and opens an OpenAI-compatible API. Multi-GPU engines keep NCCL and symmetric memory through NVIDIA cuInterpose.

Details: how a restore works, architecture, shared GPU memory, registries and caches.

Status and limits

warmctl is in beta: releases are GitHub pre-releases, and every user-facing change is in the CHANGELOG. Each restore is checked before it serves: the host must match the snapshot before anything is downloaded, every file carries its SHA-256, every weights allocation is compared by a GPU fingerprint, and a canary prompt must get the original engine's answer (bitwise for large models).

Works today	Not yet
vLLM	SGLang, TensorRT-LLM, TGI, LMDeploy (planned)
1 to 8 GPUs on one node, tensor and expert parallel	Engines that span several nodes
H100, H200, B200, B300, L40S, RTX PRO 6000	Another GPU model or count, or another driver branch: build a snapshot for that platform
Docker hosts, Kubernetes, KServe	GPUs in confidential-computing mode (cuda-checkpoint does not support it)
Any OCI registry, or a shared file system

Kubernetes

The warmctl runtime image, registry.warmctl.io/warmctl/runtime:latest (anonymous pull; pin a release tag such as :v2.0.0-beta.15 for a cluster that must not change under you), is the catalog's vLLM image with warmctl on top (make pod-image builds it). In a privileged pod it runs the same warmctl up NAME and restores in place, with readiness on /health:

kubectl apply -f deploy/kubernetes/warmctl.yaml

deploy/kubernetes also holds node preparation, a KEDA autoscaler and a KServe LLMInferenceService preset; details in docs/kubernetes.md. Tutorials: Kubernetes, KServe. Measured on GPU clusters: GLM-5.3-Flash on 8×H100 restores in 58 s from a shared file system (post), and a new KServe replica of Qwen3-8B on an L40S serves in 14 s (post).

Profiles

schema: warmctl.profile/v1
id: llama31-8b-tp1
model: {id: meta-llama/Llama-3.1-8B-Instruct, revision: 0e9e39f2}
parallel: {tp: 1}
hardware: [h100-80gb-sxm]
vllm: {max-model-len: 8192, kv-cache-memory-bytes: 21474836480}

vllm: takes any vllm serve flag; the rest comes from profiles/_defaults.yaml.

Requirements

Linux x86-64 with glibc, Docker with the NVIDIA runtime, root, driver 580+, kernel.io_uring_disabled=2. A snapshot restores under any release of the driver branch it was built on (580.x, 610.x). Proven on H100 SXM, H200, B200, B300, L40S and RTX PRO 6000. warmctl hardware lists the GPU models warmctl knows and marks this host's; another model is a row in a hardware.yaml next to your profiles (--profile-dir). Keep warmctl, cuinterpose/ and the versioned dmtcp/ directory from the linux-amd64 archive side by side; the installer does this automatically. Linux arm64 archives include DMTCP but no cuInterpose yet and are CPU-tested, not GPU-qualified. The native DMTCP packages require Debian 12-compatible glibc (>= 2.36), libstdc++ and libatomic (libstdc++6 and libatomic1 on Debian/Ubuntu). macOS builds run the registry side: catalog, images, mirror, push, pull. Source builds use make build cuinterpose; see DMTCP runtime installation and upgrades. Every release is signed and has build provenance (verify).

Develop

make test · make lint · make e2e-test (CPU, Docker, root) · make release (the release archives as a local snapshot). Releases: RELEASING.md. GPU acceptance: docs/gpu-runbook.md.

License

Apache-2.0, including the bundled NVIDIA cuInterpose build (NOTICE).