Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 40 additions & 14 deletions .github/workflows/cross-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
# License: Apache-2.0
# Copyright: (c) 2025-2026 HYPERI PTY LIMITED
#
# Trigger: - workflow_run after CI succeeds (most common path - fires when
# Trigger: - workflow_run after CI succeeds on main in this repository, never
# for a pull request (most common path - fires when
# hyperi-ci's publish step tags vX.Y.Z; the [skip ci] suffix on
# the chore commit blocks the tag-push trigger so we use the
# ci-completion event instead).
Expand All @@ -26,6 +27,8 @@ name: Cross-build
workflow_run:
workflows: [CI]
types: [completed]
# A fork's branch can also be called main, so resolve-tag checks the repository too.
branches: [main]
workflow_dispatch:
inputs:
tag:
Expand All @@ -39,14 +42,21 @@ permissions:
jobs:
resolve-tag:
# workflow_run gives us no inputs and `github.ref_name` resolves to the
# default branch, so derive the tag explicitly. Skips when CI failed or
# when CI didn't produce a new tag (non-publish run).
# default branch, so derive the tag explicitly.
#
# A workflow_run job holds the publish secrets whatever started the CI run,
# and CI also runs on fork pull requests. Only a successful CI run on main in
# this repository, from a push or a maintainer's dispatch, may start a build.
name: Resolve tag to build
runs-on: ubuntu-latest
if: >-
${{ github.event_name == 'workflow_dispatch'
|| github.event_name == 'push'
|| github.event.workflow_run.conclusion == 'success' }}
|| (github.event.workflow_run.conclusion == 'success'
&& github.event.workflow_run.head_branch == 'main'
&& github.event.workflow_run.head_repository.full_name == github.repository
&& (github.event.workflow_run.event == 'push'
|| github.event.workflow_run.event == 'workflow_dispatch')) }}
outputs:
tag: ${{ steps.pick.outputs.tag }}
steps:
Expand Down Expand Up @@ -75,11 +85,9 @@ jobs:
tag="$REF_NAME"
;;
workflow_run)
# CI completed successfully. Pick the most recent v-tag - if
# CI was a publish run it will have created vX.Y.Z; if it was a
# validate-only run the latest tag is unchanged and rebuilding
# is idempotent (same source -> same bytes overwrite the same R2
# paths).
# Pick the most recent v-tag. A publish run has just created it.
# After any other run it is already published, and the upload
# step finds it there and writes nothing.
git fetch --tags --force origin
tag=$(git tag --sort=-creatordate --list 'v*' | head -1)
if [ -z "$tag" ]; then
Expand All @@ -97,6 +105,11 @@ jobs:
needs: resolve-tag
if: ${{ needs.resolve-tag.outputs.tag != '' }}
runs-on: ${{ matrix.runner }}
# The tag push and the CI completion can both build one tag. Queued rather
# than raced, the second build finds the first one's upload and writes nothing.
concurrency:
group: cross-build-${{ needs.resolve-tag.outputs.tag }}-${{ matrix.os_arch }}
cancel-in-progress: false
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -256,14 +269,27 @@ jobs:
R2_ENDPOINT: https://98d20454e2af7a9397ad9366a1641659.r2.cloudflarestorage.com
R2_BUCKET: bin-repo
TAG: ${{ steps.ver.outputs.tag }}
OS_ARCH: ${{ matrix.os_arch }}
shell: bash
run: |
set -euo pipefail
dest_versioned="s3://${R2_BUCKET}/macbash/${TAG}/"
dest_latest="s3://${R2_BUCKET}/macbash/latest/"
for f in dist/*; do
for dest in "$dest_versioned" "$dest_latest"; do
aws s3 cp --endpoint-url "$R2_ENDPOINT" "$f" "$dest"
# Versioned paths are write-once: the formula and install.sh pin these
# bytes by sha256. The .sha256 goes up last, so it marks a full upload.
sums="dist/macbash-${OS_ARCH}.sha256"
marker="macbash/${TAG}/macbash-${OS_ARCH}.sha256"
published=$(aws s3api list-objects-v2 --endpoint-url "$R2_ENDPOINT" --bucket "$R2_BUCKET" \
--prefix "$marker" --query "length(Contents || \`[]\`)" --output text)
case "$published" in
0) ;;
1) echo "::notice::${marker} already exists and versioned artefacts are write-once, so nothing was uploaded"; exit 0 ;;
*) echo "::error::unexpected R2 listing for ${marker}: ${published}"; exit 1 ;;
esac
for dest in "s3://${R2_BUCKET}/macbash/${TAG}/" "s3://${R2_BUCKET}/macbash/latest/"; do
for f in dist/*; do
if [ "$f" != "$sums" ]; then
aws s3 cp --endpoint-url "$R2_ENDPOINT" "$f" "$dest"
fi
done
aws s3 cp --endpoint-url "$R2_ENDPOINT" "$sums" "$dest"
done
echo "Published to https://downloads.hyperi.io/macbash/${TAG}/"
92 changes: 70 additions & 22 deletions .github/workflows/package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
# License: Apache-2.0
# Copyright: (c) 2025-2026 HYPERI PTY LIMITED
#
# Trigger: - workflow_run after Cross-build succeeds (the standard path)
# Trigger: - workflow_run after Cross-build succeeds on main in this
# repository (the standard path)
# - workflow_dispatch with tag input (manual fallback)
#
# On workflow_run the tag is derived from the most recent v* tag,
Expand All @@ -25,6 +26,8 @@ name: Package
workflow_run:
workflows: [Cross-build]
types: [completed]
# A fork's branch can also be called main, so resolve-tag checks the repository too.
branches: [main]
workflow_dispatch:
inputs:
tag:
Expand All @@ -44,11 +47,18 @@ jobs:
resolve-tag:
# workflow_run gives us no inputs - derive the tag from the latest v* tag,
# same pattern as cross-build.yml's resolve-tag job.
#
# Same gate as cross-build.yml. A Cross-build whose own gate refused a fork's
# CI run concludes "skipped", never "success", so it cannot start this one.
name: Resolve tag to package
runs-on: ubuntu-latest
if: >-
${{ github.event_name == 'workflow_dispatch'
|| github.event.workflow_run.conclusion == 'success' }}
|| (github.event.workflow_run.conclusion == 'success'
&& github.event.workflow_run.head_branch == 'main'
&& github.event.workflow_run.head_repository.full_name == github.repository
&& (github.event.workflow_run.event == 'workflow_run'
|| github.event.workflow_run.event == 'workflow_dispatch')) }}
outputs:
tag: ${{ steps.pick.outputs.tag }}
steps:
Expand Down Expand Up @@ -136,17 +146,28 @@ jobs:
env:
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
ARCH: ${{ matrix.arch }}
shell: bash
run: |
set -euo pipefail
aws configure set aws_access_key_id "$R2_ACCESS_KEY_ID"
aws configure set aws_secret_access_key "$R2_SECRET_ACCESS_KEY"
aws configure set region auto
# Versioned paths are write-once: the formula and install.sh pin these
# bytes by sha256. The .sha256 goes up last, so it marks a full upload.
marker="macbash/${TAG}/macbash-linux-${ARCH}.sha256"
published=$(aws s3api list-objects-v2 --endpoint-url "$R2_ENDPOINT" --bucket "$R2_BUCKET" \
--prefix "$marker" --query "length(Contents || \`[]\`)" --output text)
case "$published" in
0) ;;
1) echo "::notice::${marker} already exists and versioned artefacts are write-once, so nothing was uploaded"; exit 0 ;;
*) echo "::error::unexpected R2 listing for ${marker}: ${published}"; exit 1 ;;
esac
for dest in "s3://${R2_BUCKET}/macbash/${TAG}/" "s3://${R2_BUCKET}/macbash/latest/"; do
aws s3 cp --endpoint-url "$R2_ENDPOINT" \
"dist/macbash-linux-${{ matrix.arch }}.tar.gz" "$dest"
"dist/macbash-linux-${ARCH}.tar.gz" "$dest"
aws s3 cp --endpoint-url "$R2_ENDPOINT" \
"dist/macbash-linux-${{ matrix.arch }}.sha256" "$dest"
"dist/macbash-linux-${ARCH}.sha256" "$dest"
done

deb-rpm:
Expand Down Expand Up @@ -200,23 +221,37 @@ jobs:
nfpm package -f packaging/dist/release/nfpm.yaml -p deb -t out/
nfpm package -f packaging/dist/release/nfpm.yaml -p rpm -t out/
ls -la out
# Per-arch sha for the packages
( cd out && shasum -a 256 ./* ) > "out/macbash-linux-${{ matrix.arch }}.packages.sha256"
# Per-arch sha for the packages. Named, not ./*, which would list the sums file itself.
( cd out && shasum -a 256 ./*.deb ./*.rpm ) > "out/macbash-linux-${{ matrix.arch }}.packages.sha256"

- name: Upload to R2
env:
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
ARCH: ${{ matrix.arch }}
shell: bash
run: |
set -euo pipefail
aws configure set aws_access_key_id "$R2_ACCESS_KEY_ID"
aws configure set aws_secret_access_key "$R2_SECRET_ACCESS_KEY"
aws configure set region auto
# Write-once, as for the tarballs. The packages .sha256 goes up last.
sums="out/macbash-linux-${ARCH}.packages.sha256"
marker="macbash/${TAG}/macbash-linux-${ARCH}.packages.sha256"
published=$(aws s3api list-objects-v2 --endpoint-url "$R2_ENDPOINT" --bucket "$R2_BUCKET" \
--prefix "$marker" --query "length(Contents || \`[]\`)" --output text)
case "$published" in
0) ;;
1) echo "::notice::${marker} already exists and versioned artefacts are write-once, so nothing was uploaded"; exit 0 ;;
*) echo "::error::unexpected R2 listing for ${marker}: ${published}"; exit 1 ;;
esac
for dest in "s3://${R2_BUCKET}/macbash/${TAG}/" "s3://${R2_BUCKET}/macbash/latest/"; do
for f in out/*; do
aws s3 cp --endpoint-url "$R2_ENDPOINT" "$f" "$dest"
if [ "$f" != "$sums" ]; then
aws s3 cp --endpoint-url "$R2_ENDPOINT" "$f" "$dest"
fi
done
aws s3 cp --endpoint-url "$R2_ENDPOINT" "$sums" "$dest"
done

manifests:
Expand Down Expand Up @@ -255,27 +290,40 @@ jobs:
aws configure set aws_secret_access_key "$R2_SECRET_ACCESS_KEY"
aws configure set region auto

# Use text/plain content-type so curl/iex render in browser too.
put_script() {
local ct="text/x-shellscript"
case "$1" in *.ps1) ct="text/plain" ;; esac
aws s3 cp --endpoint-url "$R2_ENDPOINT" --content-type "$ct" \
"packaging/dist/$1" "$2"
}

# Static install scripts at /macbash/install.sh (curl-pipe target),
# always re-rendered from the default branch.
for f in install.sh install.ps1 uninstall.sh uninstall.ps1; do
put_script "$f" "s3://${R2_BUCKET}/macbash/$f"
done

# The per-version copies are write-once, as for the binaries: pinning a
# tag-specific install.sh URL gets the script that shipped with that
# binary. <tag>/install.sh goes up last, so it marks a full upload.
marker="macbash/${TAG}/install.sh"
published=$(aws s3api list-objects-v2 --endpoint-url "$R2_ENDPOINT" --bucket "$R2_BUCKET" \
--prefix "$marker" --query "length(Contents || \`[]\`)" --output text)
case "$published" in
0) ;;
1) echo "::notice::${marker} already exists and versioned artefacts are write-once, so only the root install scripts were uploaded"; exit 0 ;;
*) echo "::error::unexpected R2 listing for ${marker}: ${published}"; exit 1 ;;
esac

# Per-version manifests under /<tag>/manifests/ + /latest/manifests/
for dest in "s3://${R2_BUCKET}/macbash/${TAG}/manifests/" \
"s3://${R2_BUCKET}/macbash/latest/manifests/"; do
aws s3 cp --recursive --endpoint-url "$R2_ENDPOINT" \
packaging/dist/release/ "$dest"
done

# Static install scripts at /macbash/install.sh (curl-pipe target).
# Also mirror to per-version path for reproducibility - pinning a
# tag-specific install.sh URL lets users install a known-good copy
# of the script that downloaded the same-version binary.
for f in install.sh install.ps1 uninstall.sh uninstall.ps1; do
for dest in "s3://${R2_BUCKET}/macbash/$f" \
"s3://${R2_BUCKET}/macbash/${TAG}/$f"; do
# Use text/plain content-type so curl/iex render in browser too.
ct="text/x-shellscript"
case "$f" in *.ps1) ct="text/plain" ;; esac
aws s3 cp --endpoint-url "$R2_ENDPOINT" \
--content-type "$ct" \
"packaging/dist/$f" "$dest"
done
for f in uninstall.ps1 uninstall.sh install.ps1 install.sh; do
put_script "$f" "s3://${R2_BUCKET}/macbash/${TAG}/$f"
done

publish-tap-bucket:
Expand Down
3 changes: 2 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ runs:
env:
MACBASH_VERSION: ${{ inputs.version }}
# Risk accepted: install.sh comes over TLS from the host that serves the
# binary, so fetching it first adds no integrity check; #10 tracks one.
# binary. It refuses any binary without a matching published SHA-256, but
# that sums file comes from the same host and nothing is signed.
# nosemgrep: yaml.github-actions.security.gha-curl-pipe-shell.gha-curl-pipe-shell
run: |
set -euo pipefail
Expand Down
42 changes: 26 additions & 16 deletions build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,31 +7,41 @@
// Copyright: (c) 2025-2026 HYPERI PTY LIMITED

use std::process::Command;
use std::time::{SystemTime, UNIX_EPOCH};

fn main() {
let sha = Command::new("git")
.args(["rev-parse", "HEAD"])
.output()
.ok()
.filter(|o| o.status.success())
.map(|o| String::from_utf8_lossy(&o.stdout).trim().to_string())
.unwrap_or_else(|| "unknown".into());
let sha = git(&["rev-parse", "HEAD"]).unwrap_or_else(|| "unknown".into());
println!("cargo:rustc-env=VERGEN_GIT_SHA={sha}");

let secs = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
println!(
"cargo:rustc-env=VERGEN_BUILD_TIMESTAMP={}",
rfc3339_utc(secs)
);
// Never the clock: the same commit must always build the same bytes.
let built = build_epoch()
.map(rfc3339_utc)
.unwrap_or_else(|| "unknown".into());
println!("cargo:rustc-env=VERGEN_BUILD_TIMESTAMP={built}");

println!("cargo:rerun-if-env-changed=SOURCE_DATE_EPOCH");
println!("cargo:rerun-if-changed=.git/HEAD");
println!("cargo:rerun-if-changed=.git/refs");
}

fn git(args: &[&str]) -> Option<String> {
Command::new("git")
.args(args)
.output()
.ok()
.filter(|o| o.status.success())
.map(|o| String::from_utf8_lossy(&o.stdout).trim().to_string())
}

/// SOURCE_DATE_EPOCH when the builder sets it, else the commit time of HEAD.
fn build_epoch() -> Option<u64> {
match std::env::var("SOURCE_DATE_EPOCH") {
Ok(raw) => Some(raw.trim().parse().unwrap_or_else(|_| {
panic!("SOURCE_DATE_EPOCH must be whole seconds since the Unix epoch, got {raw:?}")
})),
Err(_) => git(&["log", "-1", "--format=%ct", "HEAD"])?.parse().ok(),
}
}

fn rfc3339_utc(secs: u64) -> String {
let days = (secs / 86_400) as i64;
let tod = secs % 86_400;
Expand Down
12 changes: 12 additions & 0 deletions docs/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,18 @@ artefact handoff, its linux tarball jobs POLL R2 for the raw binary before
proceeding. That tolerates hyperi-ci's publish step being slower than the
cross-build.

## What may start a publish

A `workflow_run` job holds the R2 and GitHub App secrets whatever started the upstream run, and CI also runs on pull requests from forks. So both `workflow_run` triggers filter on `branches: [main]`, and each `resolve-tag` job also requires the upstream run to have succeeded in this repository, from a push or a dispatch. A fork can name its branch `main`, which is why the repository check is there as well.

A Cross-build that refuses a run concludes `skipped`, and Package only follows a `success`, so the refusal stops the whole chain.

## Versioned paths are write-once

The Homebrew formula and `install.sh` pin the bytes at `/<tag>/` by SHA-256, so nothing overwrites them. Each upload checks R2 for the file it writes last (its `.sha256`, or `<tag>/install.sh` for the manifests), and when that exists it writes nothing, `latest/` included. The root `install.sh` is the exception: it is re-rendered from the default branch on every Package run. A re-run, or a CI run that published nothing new, therefore leaves the release alone. Replacing a published artefact means deleting it from R2 by hand first.

The binaries carry no build clock either: `build.rs` stamps `SOURCE_DATE_EPOCH` when it is set, else the commit time of HEAD.

## Versioning

semantic-release owns the version, on its default conventional-commit rules:
Expand Down
13 changes: 7 additions & 6 deletions packaging/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,7 @@ sibling repos on the same org:

These are populated per release by copying the rendered manifests in.

Both detect OS/arch, download the matching binary from the configured
R2 download base (or any HTTP host), verify a SHA-256 checksum if
available, and install to a system or user directory.
Both detect OS/arch, download the matching binary from the configured R2 download base (or any HTTP host), and install to a system or user directory. `install.sh` refuses to install unless the binary matches a published SHA-256 for that exact asset: a missing sums file, a sums file without the asset, a mismatch or a download error all exit non-zero.

Adding a new channel: drop a file into `packaging/templates/` or
`packaging/templates-release/`, reference any `${PKG_*}` variable,
Expand Down Expand Up @@ -80,11 +78,14 @@ framework changes required.
The rendered installers expect binaries at this URL layout:

```text
${PKG_DOWNLOAD_BASE}/${VERSION}/${PKG_BINARY}-${os}-${arch} # unix
${PKG_DOWNLOAD_BASE}/${VERSION}/${PKG_BINARY}-${os}-${arch}.exe # windows
${PKG_DOWNLOAD_BASE}/${VERSION}/checksums.sha256 # optional
${PKG_DOWNLOAD_BASE}/${VERSION}/${PKG_BINARY}-${os}-${arch} # unix
${PKG_DOWNLOAD_BASE}/${VERSION}/${PKG_BINARY}-${os}-${arch}.exe # windows
${PKG_DOWNLOAD_BASE}/${VERSION}/${PKG_BINARY}-${os}-${arch}.sha256 # checked first
${PKG_DOWNLOAD_BASE}/${VERSION}/checksums.sha256 # fallback
```

`install.sh` needs one of the two sums files to carry a `<sha256> <asset>` line for the exact asset name. The first file that names the asset decides, so a mismatch there never falls through to the other.

Where `${VERSION}` is either `latest` or a `vX.Y.Z` tag, and `os` is
one of `linux`, `darwin`, `windows`, and `arch` is `amd64` or `arm64`.

Expand Down
Loading
Loading