[calvin-archastro]

Epic: Aster restores real build artifacts from a shared cache, including the GitHub Actions cache
Task: aster-srv brokers the shared artifact cache: manifests in Postgres, blobs in GCS via signed URLs

Local git hooks skipped: pre-push on the submission push and the publish push. CI is the gate for what those hooks check.

Problem and author intent

aster-srv already authenticated GitHub Actions OIDC tokens and IAP viewers, but it did not broker a shared build-artifact cache. Without a gateway, each machine would need bucket credentials or a separate remote cache, and nothing on the server could authorize writes or expire entries.

The intent is for aster-srv to be that gateway. Manifests stay in Postgres so the server can authorize and expire them. Blob bytes stay in GCS under cas/<sha256>. Clients never receive a key file; they upload and download through 15-minute V4 signed URLs that the pod signs with IAM Credentials signBlob as its own Workload Identity service account.

Before this change, a CI token was repository-scoped only. There was no workflow ref on the identity, no writer allowlist, and no cache routes, so a pull-request workflow could not be refused a cache write because no cache write existed. After this change, lookup is available to an authenticated CI token for its own repository and to an allowlisted IAP viewer. Upload and commit require a static development token, or an OIDC token whose ref matches ASTER_SRV_CACHE_WRITERS (default *=refs/heads/<default branch>). A pull-request ref such as refs/pull/N/merge does not match that default, so the server returns 403 and a branch cannot replace a stored key. An existing key is insert-only (ON CONFLICT DO NOTHING → 409). A manifest that names a blob GCS does not have is refused with 422 and is not stored.

What changed

• New cache schema migration: cache.manifests (repo, key, manifest bytes, size, last_used_at) and cache.blobs (sha256, size, last_used_at).
• POST /v1/cache/lookup accepts up to 256 keys. Unknown keys are omitted from hits, not errors. A hit returns the manifest as base64 and a signed GET only for each schema-1 output digest or log whose cas/<sha256> object exists. Lookup refreshes last_used_at on the manifest and on blobs that exist.
• POST /v1/cache/upload (writers only) accepts up to 1024 sha256 digests and returns signed PUT URLs for objects not already in the bucket. If no bucket is configured, upload returns 503.
• POST /v1/cache/commit (writers only) decodes a base64 schema-1 manifest, requires every named blob to exist, then inserts the manifest and blob rows in one transaction. A second commit of the same key is 409. A named blob missing from GCS is 422 with a missing list. A stale cache.blobs row whose object is gone is deleted so a later upload can replace it.
• Signing builds a GCS V4 canonical request (GOOG4-RSA-SHA256, 900-second expiry, UNSIGNED-PAYLOAD) and signs it via signBlob. The URL keeps path slashes; only the string-to-sign percent-encodes segments. Tests inject FixedSigner and an in-memory bucket so no private key is loaded.
• Retention, started from main and repeated every six hours, takes a Postgres advisory lock so one replica runs. It deletes manifests unused for ASTER_SRV_CACHE_DAYS (default 14), then blob rows no remaining manifest names, and deletes those GCS objects before committing the row deletes. A failed object delete rolls the rows back.
• Config: ASTER_SRV_CACHE_BUCKET (unset disables the store), ASTER_SRV_CACHE_DAYS, ASTER_SRV_CACHE_WRITERS (owner/name=ref or *=ref; a pattern ending in * is a prefix match). OIDC ref is stored on Identity; static tokens are marked static_token and may write. IAP viewers may look up an allowlisted repo only, and only if ASTER_SRV_UI_EMAIL_DOMAINS allows the email.
• README documents the env vars, routes, writer policy, and retention.

Scope

ArchAstro/aster 2e006138a093e5b1465689a0e0a80cbaf8e1b3df → 0ea8faaf2aa92e11b5f8247f025c26eb4fd9e657. Seventeen paths, 41 hunks, +1890 / −13. The revision file list is complete (omitted_paths 0, omitted_hunks 0). CI log excerpts in the supplied checks are truncated.

Touched: Cargo.lock, crates/aster-srv/Cargo.toml (base64, sha2), crates/aster-srv/README.md, src/auth.rs, src/cache/{mod,policy,routes,signing,store}.rs, src/config.rs, src/http.rs, src/iap.rs, src/lib.rs, src/main.rs, src/test_results/durations.rs, tests/cache.rs, tests/common/mod.rs.

Not in this revision: the client CacheBackend that would call these routes, and no workflow file change is included.

Risk assessment

Medium. This is a new shared write path. A too-broad ASTER_SRV_CACHE_WRITERS pattern, or treating a missing ref as the default branch, would let a non-default workflow upload and commit. The implementation denies a missing ref, matches the default branch exactly unless the pattern ends in *, and binds a CI bearer token to its own repository. IAP callers cannot write. Keys are not overwritten, and blobs are content-addressed, so a substituted object fails the sha256 name. Residual risks: IamSigner keeps one access token in a OnceCell with no refresh shown, so signBlob can fail after that token expires until process restart; commit records cache.blobs.size_bytes as 0 rather than the object size; lookup omits a blob missing from GCS but does not delete that row (deletion of a stale row happens on upload/commit presence checks and on retention). Retention deletes objects before the transaction commits; a crash after delete and before commit leaves rows that the next pass can retry, because a missing object delete is success.

User impact

Operators must set ASTER_SRV_CACHE_BUCKET or the cache stays disabled. The pod identity needs object access on that bucket and iam.serviceAccounts.signBlob on itself. CI on the default branch (or a configured writer ref) can upload and commit; other refs, including pull requests, can look up but not write. Static development tokens may write. IAP viewers in an allowed email domain can look up allowlisted repos and cannot write. Signed URLs last 15 minutes. Unused manifests and unreferenced blobs age out after 14 days unless ASTER_SRV_CACHE_DAYS is set. Clients must speak the new /v1/cache JSON routes; existing test-results routes are unchanged aside from extra identity fields.

Testing

Canonical integration tests are defined in crates/aster-srv/tests/cache.rs and skip unless ASTER_SRV_TEST_DATABASE_URL is set: unknown_key_is_a_miss, upload_commit_lookup_round_trip, commit_refuses_missing_blob, lookup_omits_a_blob_missing_from_gcs, policy_matrix_pr_oidc_forbidden_iap_reads_only, signed_urls_expire_and_no_key_is_stored, retention_drops_expired_rows_and_objects. They use real Postgres when that URL is set and an in-memory bucket with FixedSigner, not live GCS. crates/aster-srv/src/config.rs tests assert the default bucket is unset, retention is 14 days, and the default writer allowlist accepts refs/heads/main and rejects a pull-request ref and a missing ref. The function name for that config test is not in the excerpt.

Pipeline checks format, lint, and test are reported succeeded with exit code 0. Those excerpts are truncated. The visible test excerpt shows other aster-srv tests and does not name tests/cache.rs, so this evidence does not prove the cache integration tests ran rather than skipped. A green cargo test without the database URL would still skip them. No workflow file in this revision shows that the database URL is set in CI. Patch contents are not proof those tests passed.

Follow-ups and known issues

• The synchronous client that maps cache-backend calls onto these routes is not in this revision.
• Refresh or rebuild the IAM access token used for signBlob; the signer is initialized once.
• Commit inserts blob rows with size_bytes = 0; only the test helper BlobStore::put records the real length.
• Lookup hides a missing GCS object but leaves the cache.blobs row until a presence check or retention.
• Confirm CI actually sets ASTER_SRV_TEST_DATABASE_URL and that tests/cache.rs executed; the supplied test log is truncated and does not show that.
• Production GCS signing and signBlob IAM are not exercised by the in-memory signer.

[calvin-archastro]

ArchDev is watching this pull request. State: settled (cycle 4 of 4 for head e664829).

when	trigger	decision	result
2026-10-18T11:42:45.368Z	ci + 4 review	enqueue job_795a793e07746556da94e00c	published fa48d5e
2026-10-18T12:04:33.934Z	review feedback	enqueue job_58b9679708b59ef7ddb97091	published 2257443
2026-10-18T12:26:11.951Z	verify head	enqueue job_4736edbc57ea48d5e57675b2	no change
2026-10-18T18:48:12.589Z	merge conflict	enqueue job_69e953ff4b9dd22a94e6ad71	published e664829

Paused by the no-autofix label or a draft; never resolves reviewer threads. Jobs: job_795a793e07746556da94e00c, job_58b9679708b59ef7ddb97091, job_4736edbc57ea48d5e57675b2, job_69e953ff4b9dd22a94e6ad71.