feat(compose): install app images into dockerd's containerd

[mike-sul]

Why

On a dockerd that uses the containerd image store, composectl install goes through docker load, which keeps a second compressed copy of every app image in containerd's content store next to the one in composectl's store (about 380 MB extra for a 380 MB image, plus the unpacked snapshots). This PR streams the images from the store straight into dockerd's containerd under an install lease, unpacks them, and deletes the lease so containerd's GC drops the compressed layers. The store copy stays the only persistent compressed one and remains the source for offline repair and reinstall after aktualizr-lite's image prune.

Design note with the decision record, the rejected options and the board measurements: docs/containerd-install-plan-2026-10-08.md.

What changes

• Discovery (pkg/compose/docker_client.go): the containerd socket, namespace and snapshotter are read from /info. On a local unix socket a raw GET through the client's transport is decoded into the few fields needed, since the bundled Docker v25 client has no Containerd field in its Info type. The snapshotter comes from the driver-type entry of DriverStatus instead of the overlayfs hardcode. The server version is compared numerically with a leading v stripped (the Arduino build reports v29.5.2).
• Install into containerd (pkg/compose/ctrd_install.go): per image, layers above the first missing chain-ID snapshot are written with content.WriteBlob, then config, manifest (no layer labels, so only the lease holds the layers) and index; Image.Unpack on the platform manifest; syncfs on the snapshotter root before the image records are created; records for the pinned digest ref and the tag ref, with rollback of the first record if the second fails; lease deleted synchronously. Leases left by an interrupted install are deleted by the next install, so a broken snapshot is never reused. A flock on <StoreRoot>/.install.lock serializes install, uninstall, rm, prune, update cancel and update complete's prune.
• Host gate: the route is used only with the containerd image store, on a local unix socket, and at least Docker 29.5.2, the oldest version whose docker image ls lists images whose layers are not in containerd. If containerd cannot be reached on such a daemon the install fails rather than falling back to docker load. Other daemons keep docker load.
• Storage precheck (pkg/compose/storage.go): the bytes now land on up to four paths (store, compose root, content root, snapshotter root). check sizes each term, merges the ones on the same filesystem by st_dev and checks each against its own free space; check --format json gains a storage array (path, required, free, ok) for aktualizr-lite. An image whose config or manifest cannot be read is sized conservatively from the app tree. update init shares the sizing and counts only the remaining bytes of partly fetched blobs.
• go.mod: importing the root package of the containerd v1.7.15 client adds 17 indirect modules and makes golang.org/x/sys direct. No new direct library; no Go bump (directive stays 1.22.12).

What is lost

docker save of an installed image no longer yields its layers on the containerd route, since the compressed layers are not in the content store. aktualizr-lite's size reporting should read the per-filesystem storage array.

Review and verification

• Reviewed in four rounds by an independent GPT model acting as an adversarial reviewer (Codex, read-only, high effort). Round 1: three blockers (tcp hosts could not install, check --local failed on a missing config, merge gates undemonstrated) and four should-fix items. Round 2: degraded sizing undercounted, record rollback defeated by cancellation. Round 3: snapshot dedupe without chain IDs undercounted. Round 4: ready to merge, no remaining code findings. All findings were fixed except two that were rejected with reasons (the inherited length of UninstallApps; the CLI prints in the install check).
• make check, make test-unit, and make test-e2e pass in the dev container in all four cells of the matrix: CONTAINERD_SNAPSHOTTER false/true × stock dockerd / dockerd with the digest-identity patch. The e2e target now runs packages serially (-p 1) because the integration and pkg/compose tests share the daemons.
• On the board (Docker v29.5.2, containerd 2.3.3, overlayfs, ext4 eMMC): install of a 17-layer, 208 MB image took 3m12s with snapshots 0 → 17 and content blobs 4 → 7 (index, manifest, config, no layers); docker image ls --digests, check --install, ps and run worked with no pull; an update sharing base layers took 2m30s; uninstall --prune and rm returned the board to 0 images, 0 snapshots, 4 blobs and no composectl leases.

Power-cut trials on the board (2026-10-13)

Eleven sysrq-b cuts with nothing synced, on the Arduino UNO Q with the real composectl, each followed by a reboot and a recovery with no manual step: three during unpack (two left torn snapshots, metadata ahead of the disk), three between the first image record and the lease delete (probe-triggered), five right after install returned. All recovered through the next install: stale lease deleted, torn snapshots dropped, image re-unpacked or kept, container ran, check --install clean. docker image prune -a with a stopped container keeps the image; without one it removes it and the reinstall from the store brings it back. Details in the design note.

Two pre-existing limits the trials made visible, both listed as follow-ups in the note:

• The compose bundle is untarred without a sync on both install routes, so a cut inside the writeback window after install loses it; check --install reports it and the next install rewrites it. A syncfs on the compose root would close it.
• A cut between containerd's GC commit and its unlinks leaves the compressed layers as orphan files that containerd only removes at its next content cleanup (the next image removal on the device). Bounded to one install's layers and self-healing.

Still unproven

• Real power cuts and erofs on a deployed fleet (deployment gate).
• The precheck on a host where the store, content root and snapshotter root are on separate or nearly full mounts.

Commits

Each commit builds and passes go vet on its own: design note, discovery, install, storage sizing, locks in the CLI, integration tests, then the note update with the power-cut trials.

🤖 Generated with Claude Code