Repository navigation
feat: IMAGE_TAG in platform's values, GitHub releases, no registry sync workflow - #10
Merged
Merged
Conversation
…h GitHub releases, drop the registry sync The deploy workflow writes IMAGE_TAG=<tag> into deploy/<slug>/values instead of tags/<slug>.json, and skips a project with no deploy folder. The changelog workflow publishes a GitHub release for every tag it cuts. The registry sync workflow goes; platform's Sync registry runs hourly and on dispatch.
This was referenced Nov 5, 2026
AmrMsCLL
added a commit
to Project-Graphite/platform
that referenced
this pull request
Nov 5, 2026
…he secret store's host side, and the chart, tags and people retired (#43) Phase 1 of the Graphite Hub plan for `platform`: the host side of the console's secret store (step 2), the shared backup service (step 3), plain compose with shared fragments (steps 4 and 5), and the retirement of the chart, tag records and people files (step 7). Goes with Project-Graphite/console#19 and Project-Graphite/actions#10. ## Host and agent - `graphite-up` and the agent use `/etc/graphite/<project>.env` when it exists and `secrets.env` otherwise, so each project moves to the console's store on its first rollout after its secrets are imported there. - The agent runs as root, reads `/etc/graphite/agent.env`, accepts `values` and `files/<name>` in a payload, refuses a missing secret name before it writes anything, and answers `POST /reapply` with `Authorization: Bearer <CONSOLE_TOKEN>` for the console's Redeploy. - `graphite-up` passes `--env-file values` before the secrets file. ## Plain compose - `shared/app.yaml` (`service`: hardening, logging, health timing, limits; `app`: plus the edge alias `${PROJECT}`, `APP_URL`, `APP_PORT`/`APP_MEMORY`/`APP_CPUS`/`APP_RESERVE`), `shared/postgres.yaml`, `shared/redis.yaml`, `shared/redis-password.yaml`. - Console, Strata, Tracker and PixelPocket are each `deploy/<slug>/compose.yaml` plus `deploy/<slug>/values`; the Pages projects keep only `values` with `IMAGE_TAG`. Volume names, container names and the compose project name are unchanged, so data stays where it is. - `deploy.py` flattens the compose (`docker compose config --no-interpolate`), renders it with the secret names stubbed, derives the Traefik route from the service on the edge network, and sends compose, values (with `PROJECT` and `HOST` from the registry) and `files/` to the agent. A removed `deploy/<slug>/` takes a compose project down. - PR checks: `deploy.py check` (every compose renders), `check-memory.py` (reservations against `deploy/host.yaml`), `check-values.py` (no value equal to its default, `ghcr.io` images only on `${IMAGE_TAG:?}` or `sha-`, no `${NAME}` without `:?` or `:-`). ### Rendered both ways Each project rendered through the chart (origin/main) and through the new compose, both interpolated with the same stubbed secrets, differs only in: | Project | Difference | | :--- | :--- | | console | `SECRETS_KEY: ${SECRETS_KEY:-}` added; `graphite.backup` label on `data` | | strata | backup sidecar and its `backups` volume gone; postgres flags reordered and `work_mem=4MB` (the default) dropped; health check reads `$POSTGRES_USER`/`$POSTGRES_DB`; backup labels on postgres and `files-data` | | graphite-tracker | postgres command states the defaults `shared_buffers=128MB`, `max_connections=100`; health check as above; backup label on postgres | | pixelpocket | MinIO scripts as config files instead of inline content; `APP_URL` on the frontend; the unused `MONGO_PASSWORD` gone from mongo | Required secret names and routes are identical for all four. ## Backups `edge/backup.yaml` runs `edge/backup.py` (Python standard library, Docker API over the socket): - Nightly at 03:00 UTC, per compose project: `pg_dump` of every container labelled `graphite.backup.postgres`, run and checked with `pg_restore --list` inside the container, pulled out through the archive API and compared by checksum; a tar of every volume labelled `graphite.backup`, with every SQLite database in it replaced by a consistent copy through SQLite's online backup API (no endpoint in the app). One `/var/lib/graphite/backups/<project>/<stamp>.tar` with a manifest, and a `<stamp>.json` summary beside it. - Every archive is reopened and verified: each member's checksum, every volume tar read to its end, every SQLite copy passes `integrity_check`. A failure marks the run failed, which the console alerts on. - 14 days kept. The console asks for a run, a verification of an uploaded archive, or a restore through request files in `.requests/`; a restore verifies first, stops the project, restores the dumps with `pg_restore --clean --single-transaction` and the volumes, starts it and waits for health. ## Removed `charts/`, `deploy/chart.yaml`, `tags/`, `people/`, `scripts/compile.py`, `scripts/people.py`, `schema/person.schema.json`, the Main workflow, sleep and wake jobs (no project uses them; `edge/wake.yaml` goes), `release-core`, Helm in CI, `sync.py`'s single-manifest check. `validate.py` checks owners against organisation membership. Sync registry runs hourly as well as on dispatch. `deploy/graphite-message-lab/` is removed, which **takes Message Lab down** on merge (volumes kept). `deploy/traceboard/` (disabled) is removed. ## Rollout 1. Merge this, then Project-Graphite/actions#10 straight after. If `main` gained `tags/` commits meanwhile, rebase and copy the newest tags into each `values` `IMAGE_TAG`. 2. Deploy runs on merge: the edge and Message Lab's takedown work with the current agent; the backup service and the compose projects are refused by it before anything is written. 3. Install the new agent over SSH: ```sh sudo install -d -m 700 /etc/graphite sudo mv /etc/graphite-agent.env /etc/graphite/agent.env source=https://raw.githubusercontent.com/project-graphite/platform/main/host sudo curl -fsSL -o /srv/graphite/bin/agent.py "$source/agent.py" sudo curl -fsSL -o /srv/graphite/bin/graphite-up "$source/graphite-up" sudo chmod 755 /srv/graphite/bin/graphite-up sudo curl -fsSL -o /etc/systemd/system/graphite-agent.service "$source/graphite-agent.service" sudo systemctl daemon-reload sudo systemctl restart graphite-agent ``` 4. Run Deploy by hand with project `all`. 5. Back up one project from the console's Backups page, download it, import it and restore it: the phase 1 exit check. The `core-v1.0.0`, `core-v1.0.1` and `core-v2.0.0` tags are left in place; `git push origin --delete core-v1.0.0 core-v1.0.1 core-v2.0.0` removes them when wanted. ## Checks `deploy.py check`, `check-memory.py`, `check-values.py` (plus a negative run catching each of its rules), `validate.py` against live membership, the render diff above, `py_compile` and a local agent answering 401, 409 and a signed `down`. `backup.py` was run locally for packing a volume with a live WAL database, verification, tamper and wrong-project refusal, retention and the schedule. Not run: anything against Docker on the VPS (dumps, archive pulls, restore), `graphite-up` and `bootstrap.sh`. `README.md` is stale throughout on the chart, `core.yaml`/`values.yaml`, `tags/`, `people/`, the Main workflow, `/etc/graphite-agent.env` and the backup sidecar.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1, step 6 of the Graphite Hub plan, with the
actionspart of step 7. Goes with Project-Graphite/platform#43, which replacestags/and the chart withdeploy/<slug>/compose.yamlandvalues.Changes
IMAGE_TAG=<tag>as the first line ofplatform/deploy/<slug>/values, keeping every other line, and commits it as before. A project without adeploy/<slug>/folder (Message Lab after #43) is skipped with a notice instead of failing.sync:releases --cutand publishes a GitHub release, with generated notes, for each tag the cut added. The hub's release records are still written as today until the new hub replaces them.reusable-sync-registry.yml) is removed.platform's Sync registry now runs hourly as well as on dispatch, and the new hub will dispatch it from the organisation webhook. Each project'sregistry.ymlthat called this workflow is removed in its own PR.Merge order
Merge platform#43 first, then this straight after. Between the two, project CI still writes
tags/, which platform no longer reads, so a deploy in that window waits for the next push.Checks
The
valuesrewrite was run locally against a sample file. The workflows are linted by this repository's CI.README.mdis stale onreusable-sync-registry, thetags/<slug>.jsondescription of Deploy, and theregistry.ymlevery project needs.