chore(infra): add scripts/lab, a Linux lab for macOS development

CONTRIBUTING.md

@@ -61,6 +61,10 @@ Everything goes through one seam, so CI and your machine cannot disagree:
 scripts/task check     # fmt:check, lint, test, build
 ```
 
+On macOS, Linux-only checks run in an [OrbStack](https://orbstack.dev) machine:
+`scripts/lab setup` once, then prefix the command, as in
+`scripts/lab scripts/task build:static`. Firecracker needs KVM, so boots stay in CI.
+
 Never use `--no-verify`. If a check is wrong, fix the check in its own pull
 request. A bug fix arrives with the test that would have caught it.
 

cairn/items/0097-linux-lab-for-development-on-macos.md

@@ -0,0 +1,75 @@
+---
+id: 97
+uid: fcf55ec2-86a4-4d81-9d93-8eb637cc9992
+title: Linux lab for development on macOS
+type: chore
+status: done
+milestone: v0.1
+created: 2026-10-02
+updated: 2026-10-02
+closed_at: 2026-10-02
+priority: p0
+effort: s
+area: infra
+---
+
+## Purpose
+
+The maintainer works on macOS (Apple silicon). Everything liam ships is Linux: static musl binaries, io_uring, and a kernel image. Without a Linux machine at hand, every Linux-only check (`scripts/task build:static`, io_uring tests, kernel builds) waits on a CI round trip. A local lab makes those a one-line command, and gives parallel agents a shared place to run them without sharing a target directory.
+
+## Approach
+
+- An OrbStack machine named `liam-lab` (Ubuntu, aarch64). OrbStack mounts `/Users` at the same path, so the worktree is visible inside the VM unchanged.
+- `scripts/lab <cmd...>` runs a command in the VM at the caller's directory. `CARGO_TARGET_DIR` is set under the VM user's home, derived from the worktree path, so macOS builds, Linux builds and parallel worktrees never share a target directory.
+- `scripts/lab setup` provisions idempotently: build-essential, musl-tools, the x86_64 cross gcc and binutils, flex, bison, bc, libelf-dev, libssl-dev, cpio, file, curl, qemu-system-x86; rustup with the toolchain from `rust-toolchain.toml` plus the x86_64 and aarch64 musl targets. Cargo is configured so the x86_64 musl target links with the cross toolchain.
+- The VM has no KVM, so nothing here boots Firecracker. That stays on CI.
+
+## Acceptance criteria
+
+- [x] `scripts/lab --help` describes the commands, and a missing `orb` or a missing `liam-lab` machine fails with an error naming the fix.
+- [x] `scripts/lab setup` provisions a fresh machine, and a second run succeeds without reinstalling anything.
+- [x] `scripts/lab cargo test --workspace` passes in the lab.
+- [x] `scripts/lab scripts/task build:static` produces static x86_64 `liamd` and `liam-init`.
+- [x] io_uring works in the lab: a program calls `io_uring_setup` and completes a request.
+- [x] Two worktrees get different target directories, neither of them the macOS `target/`.
+- [x] CONTRIBUTING.md tells macOS contributors how to use the lab.
+
+## 2026-10-02
+
+Cross-compiling C for x86_64 musl on an aarch64 machine: x86_64-linux-gnu-gcc alone fails on mimalloc (it finds the aarch64 glibc headers: bits/libc-header-start.h missing). Ubuntu ships musl for the native architecture only, so setup builds musl 1.2.5 (the version in Rust's bundled x86_64 musl libc.a) from upstream source for x86_64 with --enable-wrapper=gcc, into ~/.local/x86_64-linux-musl. The tarball's sha256 a9a118bb...c75e4 matches Ubuntu's musl_1.2.5.orig.tar.gz on both archive.ubuntu.com and ports.ubuntu.com. Cargo config: linker x86_64-linux-gnu-gcc, CC_x86_64_unknown_linux_musl the musl-gcc wrapper. scripts/task build:static now honours CARGO_TARGET_DIR, which scripts/lab sets.
+
+## 2026-10-02
+
+Evidence 1: scripts/lab --help prints usage, exit 0; no args exits 2. With PATH=/usr/bin:/bin: 'scripts/lab: orb not found. Install OrbStack (brew install orbstack), then: orb create ubuntu liam-lab', exit 1. With a stub orb listing another machine: 'scripts/lab: no OrbStack machine named liam-lab. Create it: orb create ubuntu liam-lab && scripts/lab setup', exit 1. A command's exit status passes through (sh -c 'exit 7' gave 7).
+
+## 2026-10-02
+
+Evidence 2: a throwaway machine (orb create ubuntu:resolute liam-lab-fresh) provisioned from nothing by this script with only the machine name changed: exit 0 in 1m33s, installing all packages, rust 1.98.1 with both musl targets and building musl; build:static then passed there. The second setup on it: exit 0 in 0.74s, 'all installed', toolchain 'unchanged', musl not rebuilt. liam-lab itself: setup re-run several times, each exit 0 with nothing reinstalled. The throwaway machine was deleted.
+
+## 2026-10-02
+
+Evidence 3: scripts/lab cargo test --workspace in liam-lab, exit 0: six test binaries, 'test result: ok' for each (2 + 2 + 2 passed, three with 0 tests).
+
+## 2026-10-02
+
+Evidence 4: scripts/lab scripts/task build:static, exit 0: liamd 543392 bytes and liam-init 397880 bytes, both 'ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), static-pie linked, stripped'. CI's static job measured 543,400 and 397,880 for the same code. Both run in the lab under OrbStack's Rosetta binfmt: 'liamd 0.0.0', 'liam-init 0.0.0', which exercises mimalloc's C built by the cross musl-gcc. The aarch64-unknown-linux-musl target also builds both binaries statically and they run.
+
+## 2026-10-02
+
+Evidence 5: kernel 7.0.14-orbstack, /proc/sys/kernel/io_uring_disabled = 0, 149 io_uring symbols in /proc/kallsyms including __arm64_sys_io_uring_setup. A 50-line C program (raw syscalls, no liburing) set up a 4-entry ring, submitted IORING_OP_READ of /etc/hostname and reaped it: 'io_uring: features=0x3ffff user_data=42 res=9 data=liam-lab', exit 0.
+
+## 2026-10-02
+
+Evidence 6: CARGO_TARGET_DIR in the lab is /home/oddurs/.cache/liam-lab/target/Users/oddurs/Code/.worktrees/liam/chore/linux-lab for this worktree (and for its crates/liamd subdirectory), and /home/oddurs/.cache/liam-lab/target/Users/oddurs/Code/liam for the primary checkout. After the lab builds the worktree has no target/ directory at all.
+
+## 2026-10-02
+
+Evidence 7: CONTRIBUTING.md, under Checks, says in three lines that on macOS Linux-only checks run in an OrbStack machine via scripts/lab setup and a prefixed command, and that Firecracker boots stay in CI.
+
+## 2026-10-02
+
+Review follow-up: the musl prefix is now versioned (~/.local/x86_64-linux-musl-1.2.5), so bumping musl_version rebuilds rather than keeping old headers; rustup-init is downloaded to a file before it runs, so a failed download fails setup with curl's error. Re-verified: setup on liam-lab built the versioned prefix, the old one was removed, and a clean build:static gave liamd 543392 and liam-init 397880 bytes, static-pie x86-64. A second throwaway machine was provisioned from nothing by the revised script (exit 0), build:static passed on it, and it was deleted.
+
+## Result
+
+scripts/lab runs any command in the liam-lab OrbStack machine (Ubuntu 26.04 aarch64, kernel 7.0 with io_uring enabled) at the caller's directory, with CARGO_TARGET_DIR under ~/.cache/liam-lab/target/<worktree path> so worktrees and macOS never share a target. scripts/lab setup provisions it idempotently in about 1.5 minutes from nothing. x86_64 static builds work: linker x86_64-linux-gnu-gcc, and C dependencies compile against an x86_64 musl 1.2.5 built from source, because Ubuntu has no cross musl. scripts/lab scripts/task build:static gives the same sizes as CI to within 8 bytes. x86_64 binaries run in the lab under Rosetta. No KVM: Firecracker boots stay in CI. Run scripts/task check on macOS, because site/node_modules belongs to macOS.

scripts/lab

@@ -0,0 +1,154 @@
+#!/bin/sh
+# A Linux machine for the checks macOS cannot run: static musl builds, io_uring,
+# kernel builds. It is an OrbStack machine, which mounts /Users at the same path,
+# so a worktree is the same directory on both sides and nothing is copied.
+#
+# The VM has no KVM. Anything that boots Firecracker still runs in CI.
+set -eu
+
+machine=liam-lab
+repo=$(cd "$(dirname "$0")/.." && pwd)
+
+die() {
+  printf 'scripts/lab: %s\n' "$1" >&2
+  exit 1
+}
+
+say() {
+  printf '▸ %s\n' "$1"
+}
+
+usage() {
+  cat <<'USAGE'
+usage: scripts/lab <command...>
+       scripts/lab setup
+
+  <command...>  run a command in the liam-lab machine, in the current directory
+  setup         install the build toolchain in liam-lab; safe to run again
+
+Builds in the lab get a target directory of their own under the VM user's home,
+one per worktree, so they never collide with the macOS target/ or with each
+other. Example:
+
+  scripts/lab scripts/task build:static
+  scripts/lab cargo test --workspace
+
+Firecracker needs KVM, which the lab does not have; that runs in CI. Run
+scripts/task check on macOS: the site's node_modules in the worktree are macOS's.
+USAGE
+}
+
+require_machine() {
+  command -v orb >/dev/null 2>&1 ||
+    die 'orb not found. Install OrbStack (brew install orbstack), then: orb create ubuntu liam-lab'
+  orb list --quiet | grep -qx "$machine" ||
+    die "no OrbStack machine named $machine. Create it: orb create ubuntu $machine && scripts/lab setup"
+}
+
+# Runs in the VM through `sh -c`, so $HOME is the VM user's home. The target
+# directory is keyed on the worktree's path: parallel worktrees stay apart, and
+# the same worktree reuses its incremental build.
+run() {
+  require_machine
+  top=$(git rev-parse --show-toplevel 2>/dev/null || pwd)
+  # shellcheck disable=SC2016 # expanded in the VM, not here
+  exec orb -m "$machine" -w "$(pwd)" sh -c '
+    key=$1
+    shift
+    export PATH="$HOME/.cargo/bin:$PATH"
+    export CARGO_TARGET_DIR="$HOME/.cache/liam-lab/target$key"
+    exec "$@"
+  ' lab "$top" "$@"
+}
+
+musl_version=1.2.5
+musl_sha256=a9a118bbe84d8764da0ea0d28b3ab3fae8477fc7e4085d90102b8596fc7c75e4
+
+packages='build-essential musl-tools gcc-x86-64-linux-gnu binutils-x86-64-linux-gnu
+flex bison bc libelf-dev libssl-dev cpio file curl qemu-system-x86'
+
+setup() {
+  require_machine
+  channel=$(sed -n 's/^channel *= *"\(.*\)"/\1/p' "$repo/rust-toolchain.toml")
+  [ -n "$channel" ] || die "no channel in $repo/rust-toolchain.toml"
+
+  say "packages"
+  # shellcheck disable=SC2016,SC2086 # expanded in the VM; the list is split on purpose
+  orb -m "$machine" -u root sh -c '
+    set -eu
+    missing=
+    for p in "$@"; do
+      dpkg-query -W -f="\${Status}" "$p" 2>/dev/null | grep -q "ok installed" ||
+        missing="$missing $p"
+    done
+    if [ -z "$missing" ]; then
+      echo "  all installed"
+      exit 0
+    fi
+    echo "  installing:$missing"
+    export DEBIAN_FRONTEND=noninteractive
+    apt-get update -qq
+    apt-get install -y -qq --no-install-recommends $missing
+  ' setup $packages
+
+  say "rust $channel and an x86_64 musl toolchain"
+  # shellcheck disable=SC2016 # expanded in the VM
+  orb -m "$machine" sh -c '
+    set -eu
+    channel=$1 musl_version=$2 musl_sha256=$3
+    if [ ! -x "$HOME/.cargo/bin/rustup" ]; then
+      # Downloaded first: piped into sh, a failed download is an empty script
+      # that exits 0.
+      init=$(mktemp)
+      curl --proto =https --tlsv1.2 -sSf https://sh.rustup.rs -o "$init"
+      sh "$init" -y --quiet --no-modify-path --default-toolchain none >/dev/null
+      rm -f "$init"
+    fi
+    PATH="$HOME/.cargo/bin:$PATH"
+    rustup toolchain install "$channel" --profile minimal \
+      --component clippy,rustfmt \
+      --target x86_64-unknown-linux-musl,aarch64-unknown-linux-musl
+    rustup default "$channel"
+
+    # The machine is aarch64, so the x86_64 musl target needs a cross linker,
+    # and C dependencies (mimalloc) a compiler that sees x86_64 musl headers.
+    # Ubuntu packages musl for the native architecture only, so musl is built
+    # from upstream source for x86_64: the version Rust bundles for the target,
+    # with the checksum Ubuntu publishes for the same tarball. At link time Rust
+    # supplies the libc and start files; gcc only drives the link.
+    # Versioned, so bumping musl_version rebuilds instead of keeping old headers.
+    musl="$HOME/.local/x86_64-linux-musl-$musl_version"
+    if [ ! -x "$musl/bin/musl-gcc" ]; then
+      echo "  building musl $musl_version for x86_64"
+      src=$(mktemp -d)
+      curl -fsSL "https://musl.libc.org/releases/musl-$musl_version.tar.gz" \
+        -o "$src/musl.tar.gz"
+      echo "$musl_sha256  $src/musl.tar.gz" | sha256sum -c --quiet
+      tar -xzf "$src/musl.tar.gz" -C "$src" --strip-components=1
+      (cd "$src" && ./configure --prefix="$musl" --target=x86_64-linux-gnu \
+        --disable-shared --enable-wrapper=gcc >/dev/null && make -s -j"$(nproc)" install >/dev/null)
+      rm -rf "$src"
+    fi
+
+    mkdir -p "$HOME/.cargo"
+    cat >"$HOME/.cargo/config.toml" <<EOF
+[target.x86_64-unknown-linux-musl]
+linker = "x86_64-linux-gnu-gcc"
+
+[env]
+CC_x86_64_unknown_linux_musl = "$musl/bin/musl-gcc"
+AR_x86_64_unknown_linux_musl = "x86_64-linux-gnu-ar"
+EOF
+    echo "  $(rustc --version)"
+    rustup target list --installed | sed "s/^/  target /"
+  ' setup "$channel" "$musl_version" "$musl_sha256"
+
+  say "ready: scripts/lab <command...>"
+}
+
+case "${1:-}" in
+'') usage >&2 && exit 2 ;;
+-h | --help | help) usage ;;
+setup) setup ;;
+*) run "$@" ;;
+esac

scripts/task

@@ -66,8 +66,9 @@ case "${1:-}" in
       exit 1
     fi
     cargo build --release --locked --target "$target" -p liamd -p liam-init
+    # scripts/lab points CARGO_TARGET_DIR outside the checkout.
     for bin in liamd liam-init; do
-      path="target/$target/release/$bin"
+      path="${CARGO_TARGET_DIR:-target}/$target/release/$bin"
       kind=$(file -b "$path")
       case "$kind" in
         *'statically linked'* | *'static-pie linked'*) ;;