Skip to content

test(integration): fix docker-in-docker tests on ubuntu-26.04 runners - #149

Merged
bilby91 merged 2 commits into
mainfrom
fix/ci-dind-ubuntu-2604
Nov 4, 2026
Merged

bilby91 merged 2 commits into
mainfrom
fix/ci-dind-ubuntu-2604

Conversation

@bilby91

@bilby91 bilby91 commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

TestComposeSource_DockerInDocker* started failing on every runner. That includes a rerun of main's last green commit (8800af2), so this isn't caused by any open PR.

Cause

Between 2026-10-28 and 2026-11-04, ubuntu-latest moved from ubuntu-24.04 (Docker 28.0.4) to ubuntu-26.04 (kernel 7.0, Docker 29.4.2).

The tests used ghcr.io/devcontainers/features/docker-in-docker:2, which resolves to 2.17.0. On a Debian image, v2's install.sh switches to iptables-legacy unconditionally at install time. Kernel 7.0 no longer supports legacy iptables, so the inner dockerd exits. Its log (surfaced by the first commit) shows:

failed to start daemon: ... failed to create NAT chain DOCKER: iptables failed:
iptables --wait -t nat -N DOCKER: iptables v1.8.9 (legacy): ...
Perhaps iptables or your kernel needs to be upgraded.

v4 (4.1.3) checks for legacy support at install time and re-selects at container start (iptablesSwitchAtRuntime, default true). That makes it choose nft on these hosts.

Changes

  1. Diagnostics: when the inner dockerd never comes up, the test now reports the kernel, the selected iptables alternative and the tail of dockerd's and containerd's logs. Before, it said only that the socket was missing. The first test's inline copy of the wait loop now uses waitInnerDockerd.
  2. Feature version: dindFeatureRef moves to :4.
    • v4 still declares the /var/lib/docker volume, privileged and the docker-init.sh entrypoint that these tests cover.
    • It also adds a /var/lib/containerd volume. The first test now cleans up with cleanupComposeWorkspace, which removes every project volume.

Not taken: pinning runs-on: ubuntu-24.04. It would only hide the incompatibility until that runner image is retired.

🤖 Generated with Claude Code

…ils to start

Both docker-in-docker tests waited for the inner dockerd and, on
timeout, reported only that its socket was absent. The feature's
docker-init.sh logs dockerd and containerd to /tmp; include their tail,
the kernel and the selected iptables alternative in the failure. The
first test's inline copy of the wait loop now uses waitInnerDockerd.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — 9547928 · CLEAN

I found no concrete defect in this one-commit, one-file change at 9547928f308de63df56e600ef2bb2fd7ff65808c; the bounded, best-effort diagnostics only run after the existing dockerd readiness timeout and preserve the test's failure outcome.

What I reviewed

Coverage: Reviewed the changed integration-test helper and its callers, including Engine.Exec's context and output behavior, against the core review dimensions and repository guidance at base 8800af2aab1f9180f8edf5b90000158717b756c0. Checked correctness/failure paths and test coverage under R8; no new test invariant or production behavior is introduced. .dap/code-review.yml and AGENTS.md/CLAUDE.md are absent at the governing SHA; applied .dap/review/engineering.md and CONTRIBUTING.md instead. Other conditional dimensions do not apply to this diagnostic-only test change. Did not run tests or the system under review.


CLEAN — this round found nothing. Reply in a finding's own thread to argue it. On the pull request I answer @dap-code-review-by-crunchloop review, pause and resume.

ghost
ghost previously approved these changes Nov 4, 2026

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No open findings on 9547928.

The DAP Code Review check run remains the verdict of record. This approval is dismissed if a finding reopens or a later head is not clean.

ubuntu-latest moved to ubuntu-26.04 (kernel 7.0) and both
docker-in-docker tests began failing on every runner, main included.
The inner dockerd log, now reported, shows why: docker-in-docker:2
(2.17.0) switches a Debian image to iptables-legacy unconditionally at
install time, and on a kernel without legacy iptables dockerd exits
with "failed to create NAT chain DOCKER: iptables v1.8.9 (legacy)".
v4 checks at install time and re-selects at container start, choosing
nft there.

Move dindFeatureRef to :4. v4 declares a second volume
(/var/lib/containerd); the first test now cleans up with
cleanupComposeWorkspace, which removes every project volume, instead
of only the /var/lib/docker one. Pinning the runner to ubuntu-24.04
was not taken: it hides the incompatibility until that image retires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ghost
ghost dismissed their stale review November 4, 2026 15:48

Dismissed by DAP code review: fe7d636 has not been reviewed clean (the pull request was pushed to).

@bilby91 bilby91 changed the title ci: docker-in-docker tests fail on ubuntu-26.04 runners test(integration): fix docker-in-docker tests on ubuntu-26.04 runners Nov 4, 2026

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — fe7d636 · CLEAN

The change updates the Docker-in-Docker integration tests to use feature v4 and shared diagnostics/cleanup helpers; I found no concrete defects introduced by these changes.

What I reviewed

Coverage: Reviewed fe7d636dd1e1ef8b136bf3eb66f2cf508419f17e (2 commits, 1 changed file), including integration-test correctness and coverage, cleanup/resource lifecycle, and failure diagnostics. Checked the shared cleanup helper and its callers, the compose volume creation and labels, and the CI integration-test configuration. Read .dap/review/engineering.md, CONTRIBUTING.md, and README.md at the governing base SHA 8800af2aab1f9180f8edf5b90000158717b756c0; .dap/code-review.yml is absent, so the repository’s engineering standard and core review dimensions guided this review. No findings or suspicions remain. Conditional dimensions for transactions, redelivery, schemas or published interfaces do not apply.


CLEAN — this round found nothing. Reply in a finding's own thread to argue it. On the pull request I answer @dap-code-review-by-crunchloop review, pause and resume.

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No open findings on fe7d636.

The DAP Code Review check run remains the verdict of record. This approval is dismissed if a finding reopens or a later head is not clean.

@bilby91
bilby91 marked this pull request as ready for review November 4, 2026 16:14

bilby91 commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

All six integration shards passed on ubuntu-26.04 (both Go versions), including TestComposeSource_DockerInDockerRunsContainers and TestComposeSource_DockerInDockerRestartsWithoutRegistry; each shard's log ends in ok and every step, including Complete job, finished with success by 15:54 UTC. Four jobs (run 41861504913) still show in_progress because of the ongoing GitHub Actions incident (githubstatus.com, "Incident with Actions and Pages"), so the checks will read pending until GitHub reconciles them or the run is re-run after the incident.

@bilby91
bilby91 merged commit 8f740f1 into main Nov 4, 2026
10 checks passed
@bilby91
bilby91 deleted the fix/ci-dind-ubuntu-2604 branch November 4, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant