From 3db7b1c25480e57c2b802806700d0b085840f909 Mon Sep 17 00:00:00 2001 From: edwh Date: Mon, 13 Jul 2026 18:46:05 +0100 Subject: [PATCH 1/7] Per-PR preview deployments on Fly.io Label a PR 'preview' (or dispatch the workflow) to get a test site at restarters-pr-.fly.dev running that PR's code against a copy of the live DB, restored from the latest hourly backup at cold boot. Machines suspend after ~5 min idle (sub-second resume, storage-only cost) and a scheduled sweep destroys apps once the PR closes or loses the label. - fly.pr.toml: per-PR app template (suspend requires no swap; embedded MariaDB; integrations off; cookie gate on; branch banner on) - docker/preview-startup.sh: warming page on :80 within seconds, restore latest backup, truncate restored jobs/failed_jobs (supervisord's worker is hardcoded to queue:work database), run the PR's migrations against real data, report outcome at /_preview_status - Dockerfile.fly: install MariaDB for any non-production startup script - nginx: rate-limit /_login (keyed on Fly-Client-IP); serve /_preview_status gate-exempt - FEATURE__IMAGE_UPLOAD flag gating FixometerFile::upload() and the NetworkController logo path, so previews can't write to the shared production image bucket (their Tigris creds are read-only anyway) - FEATURE__WORDPRESS_INTEGRATION flag enforced centrally in Group::eventsShouldPushToWordpress(); previously WP listeners were only safe because the secret was absent. sync:discourseusernames gains the discourse flag check it was missing - pr-preview.yml: deploys gated by the 'preview' GitHub Environment (required reviewer); workflow_dispatch verifies the PR is not from a fork; cleanup never runs on pull_request events (PR-modified workflow code must not reach the org token) - a 6-hourly sweep via the develop-restricted 'preview-cleanup' environment destroys stale apps - navbar: MAILPIT_URL override so preview banners link the shared Mailpit - Replaces the single-slot preview flow (preview-deploy.yml + fly.preview.toml), which clobbered restarters-dev and never ran migrations One-time setup (environments, org token, read-only Tigris/Drive credentials, FLY_ORG variable, label) is documented in docs/fly-deployment.md. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- .github/workflows/pr-preview.yml | 245 ++++++++++++++++++ .github/workflows/preview-deploy.yml | 35 --- Dockerfile.fly | 10 +- .../Commands/SyncDiscourseUsernames.php | 6 + app/Console/Commands/WordpressCreateEvent.php | 6 + app/Console/Commands/WordpressCreateGroup.php | 6 + app/Group.php | 7 + app/Helpers/FixometerFile.php | 4 + app/Http/Controllers/NetworkController.php | 5 + config/restarters.php | 3 + docker/nginx-fly.conf | 17 ++ docker/preview-startup.sh | 240 +++++++++++++++++ docs/fly-deployment.md | 88 +++++++ fly.pr.toml | 121 +++++++++ fly.preview.toml | 97 ------- resources/views/layouts/navbar.blade.php | 3 +- tests/Feature/UploadsDisabledTest.php | 127 +++++++++ .../Feature/WordpressIntegrationFlagTest.php | 105 ++++++++ 18 files changed, 988 insertions(+), 137 deletions(-) create mode 100644 .github/workflows/pr-preview.yml delete mode 100644 .github/workflows/preview-deploy.yml create mode 100755 docker/preview-startup.sh create mode 100644 fly.pr.toml delete mode 100644 fly.preview.toml create mode 100644 tests/Feature/UploadsDisabledTest.php create mode 100644 tests/Feature/WordpressIntegrationFlagTest.php diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml new file mode 100644 index 0000000000..3f3ed41dc9 --- /dev/null +++ b/.github/workflows/pr-preview.yml @@ -0,0 +1,245 @@ +# Per-PR preview deployments on Fly.io. +# +# Deploy: label a PR with 'preview' (or re-push while labeled, or run this +# workflow manually with a PR number). Every deploy waits for +# approval via the 'preview' GitHub Environment — approvers should +# glance at the PR's diff to .github/workflows/, fly.pr.toml, +# Dockerfile.fly and docker/*.sh before approving, since the deploy +# runs the PR's own code. +# Cleanup: a 6-hourly sweep destroys apps whose PR is closed or no longer +# labeled. It deliberately does NOT run on pull_request events: +# those execute the PR's modified workflow file, which must never +# reach the org token without approval. The sweep runs from the +# default branch via the 'preview-cleanup' environment (restricted +# to the develop branch, no reviewers). +# +# One-time setup is documented in docs/fly-deployment.md (environments, +# FLY_ORG_TOKEN, FLY_PREVIEW_SECRETS, FLY_ORG repo variable, preview label). + +name: PR preview + +on: + pull_request: + types: [labeled, synchronize, reopened] + workflow_dispatch: + inputs: + pr_number: + description: 'PR number to deploy a preview for (leave empty to run the cleanup sweep instead)' + required: false + schedule: + - cron: '43 2,8,14,20 * * *' # cleanup sweep, 4x daily + +permissions: + contents: read + pull-requests: write + +jobs: + deploy: + name: Deploy preview + if: > + (github.event_name == 'workflow_dispatch' && github.event.inputs.pr_number != '') || + (github.event_name == 'pull_request' && + contains(github.event.pull_request.labels.*.name, 'preview') && + github.event.pull_request.head.repo.full_name == github.repository) + runs-on: ubuntu-latest + environment: preview + concurrency: + group: pr-preview-${{ github.event.pull_request.number || github.event.inputs.pr_number }} + cancel-in-progress: true + env: + GH_TOKEN: ${{ github.token }} + steps: + - name: Resolve PR details + id: pr + run: | + if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then + PR="${{ github.event.inputs.pr_number }}" + else + PR="${{ github.event.pull_request.number }}" + fi + INFO=$(gh pr view "$PR" --repo "$GITHUB_REPOSITORY" \ + --json state,headRefOid,headRepository,headRepositoryOwner,labels) + STATE=$(echo "$INFO" | jq -r .state) + HEAD_SHA=$(echo "$INFO" | jq -r .headRefOid) + HEAD_REPO="$(echo "$INFO" | jq -r .headRepositoryOwner.login)/$(echo "$INFO" | jq -r .headRepository.name)" + + # Fork guard. The pull_request trigger already enforces this via the + # job-level `if`, but workflow_dispatch takes a raw PR number, so a + # fork PR could otherwise be deployed with full secrets. + if [ "$HEAD_REPO" != "$GITHUB_REPOSITORY" ]; then + echo "::error::PR #$PR head is in fork '$HEAD_REPO' - previews only deploy same-repo branches." + exit 1 + fi + if [ "$STATE" != "OPEN" ]; then + echo "::error::PR #$PR is $STATE - not deploying." + exit 1 + fi + + echo "number=$PR" >> "$GITHUB_OUTPUT" + echo "sha=$HEAD_SHA" >> "$GITHUB_OUTPUT" + echo "app=restarters-pr-$PR" >> "$GITHUB_OUTPUT" + + - name: Checkout PR head + uses: actions/checkout@v4 + with: + ref: ${{ steps.pr.outputs.sha }} + fetch-depth: 0 + + - name: Overlay preview infra files from develop if the branch predates them + run: | + git fetch origin develop --depth=1 + for f in fly.pr.toml docker/preview-startup.sh; do + if [ ! -f "$f" ]; then + echo "Branch lacks $f - taking it from develop" + git show origin/develop:"$f" > "$f" + fi + done + chmod +x docker/preview-startup.sh + # Older Dockerfile.fly / nginx config don't know about the preview + # startup script or status endpoint - overlay them too. + if ! grep -q 'preview-startup.sh' Dockerfile.fly; then + echo "Branch Dockerfile.fly predates preview support - taking develop's" + git show origin/develop:Dockerfile.fly > Dockerfile.fly + fi + if ! grep -q '_preview_status' docker/nginx-fly.conf; then + echo "Branch nginx-fly.conf predates preview support - taking develop's" + git show origin/develop:docker/nginx-fly.conf > docker/nginx-fly.conf + fi + + - name: Set up flyctl + uses: superfly/flyctl-actions/setup-flyctl@master + + - name: Create app and set secrets (first deploy only) + env: + FLY_API_TOKEN: ${{ secrets.FLY_ORG_TOKEN }} + FLY_PREVIEW_SECRETS: ${{ secrets.FLY_PREVIEW_SECRETS }} + FLY_ORG: ${{ vars.FLY_ORG }} + run: | + APP="${{ steps.pr.outputs.app }}" + if [ -z "$FLY_ORG" ]; then + echo "::error::Repository variable FLY_ORG is not set (Fly organisation slug)." + exit 1 + fi + if flyctl status --app "$APP" > /dev/null 2>&1; then + echo "App $APP already exists." + else + flyctl apps create "$APP" --org "$FLY_ORG" + # Shared read-only credentials (Tigris RO, Drive RO, gate password...) + echo "$FLY_PREVIEW_SECRETS" | flyctl secrets import --app "$APP" --stage + # Fresh per-app key - previews must never hold the production APP_KEY. + flyctl secrets set "APP_KEY=base64:$(openssl rand -base64 32)" \ + --app "$APP" --stage + fi + + - name: Deploy + env: + FLY_API_TOKEN: ${{ secrets.FLY_ORG_TOKEN }} + run: | + APP="${{ steps.pr.outputs.app }}" + PR="${{ steps.pr.outputs.number }}" + sed "s/__PR_NUMBER__/$PR/g" fly.pr.toml > fly.pr.generated.toml + flyctl deploy --config fly.pr.generated.toml --app "$APP" \ + --remote-only --ha=false + # --ha=false has a history of not reliably keeping one machine. + flyctl scale count 1 --app "$APP" --yes + + - name: Wait for restore and migrations + id: status + run: | + APP="${{ steps.pr.outputs.app }}" + URL="https://$APP.fly.dev/_preview_status" + PHASE="" + for i in $(seq 1 80); do + BODY=$(curl -fsS --max-time 10 "$URL" 2>/dev/null) || BODY="" + PHASE=$(echo "$BODY" | jq -r '.phase // empty' 2>/dev/null) || PHASE="" + echo "poll $i: phase='${PHASE:-n/a}'" + case "$PHASE" in + ready|failed) break ;; + esac + sleep 15 + done + echo "phase=${PHASE:-unknown}" >> "$GITHUB_OUTPUT" + echo "detail=$(echo "$BODY" | jq -r '.detail // empty' 2>/dev/null)" >> "$GITHUB_OUTPUT" + + - name: Comment on PR + run: | + PR="${{ steps.pr.outputs.number }}" + APP="${{ steps.pr.outputs.app }}" + PHASE="${{ steps.status.outputs.phase }}" + DETAIL="${{ steps.status.outputs.detail }}" + case "$PHASE" in + ready) STATUS_LINE=":white_check_mark: Ready - restore and migrations succeeded." ;; + failed) STATUS_LINE=":x: Deploy completed but startup reported a failure: ${DETAIL:-see \`flyctl logs --app $APP\`}." ;; + *) STATUS_LINE=":warning: Deployed, but the status endpoint did not report readiness - check \`flyctl logs --app $APP\`." ;; + esac + MARKER="" + BODY=$(cat < /dev/null + else + gh api "repos/$GITHUB_REPOSITORY/issues/$PR/comments" -f body="$BODY" > /dev/null + fi + + - name: Fail the run if startup failed + if: steps.status.outputs.phase != 'ready' + run: | + echo "::error::Preview startup phase: ${{ steps.status.outputs.phase }}" + exit 1 + + cleanup: + name: Destroy previews for closed/unlabeled PRs + if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && github.event.inputs.pr_number == '') + runs-on: ubuntu-latest + environment: preview-cleanup + concurrency: + group: pr-preview-sweep + env: + GH_TOKEN: ${{ github.token }} + FLY_API_TOKEN: ${{ secrets.FLY_ORG_TOKEN }} + steps: + - name: Set up flyctl + uses: superfly/flyctl-actions/setup-flyctl@master + + - name: Sweep + run: | + FAILED=0 + APPS=$(flyctl apps list --json | jq -r '.[] | .Name // .name' | grep -E '^restarters-pr-[0-9]+$' || true) + if [ -z "$APPS" ]; then + echo "No preview apps found." + exit 0 + fi + for APP in $APPS; do + PR="${APP#restarters-pr-}" + INFO=$(gh pr view "$PR" --repo "$GITHUB_REPOSITORY" --json state,labels 2>/dev/null) || INFO="" + if [ -z "$INFO" ]; then + # Never destroy on a failed lookup - it might be transient. + echo "::warning::Could not look up PR #$PR - skipping $APP this sweep." + FAILED=1 + continue + fi + STATE=$(echo "$INFO" | jq -r '.state // "UNKNOWN"') + HAS_LABEL=$(echo "$INFO" | jq -r '[.labels[].name] | contains(["preview"])') + echo "$APP: PR state=$STATE preview-label=$HAS_LABEL" + if [ "$STATE" != "OPEN" ] || [ "$HAS_LABEL" != "true" ]; then + echo "Destroying $APP..." + if ! flyctl apps destroy "$APP" --yes; then + echo "::error::Failed to destroy $APP - it is still incurring storage cost." + FAILED=1 + fi + fi + done + exit $FAILED diff --git a/.github/workflows/preview-deploy.yml b/.github/workflows/preview-deploy.yml deleted file mode 100644 index 3f25581598..0000000000 --- a/.github/workflows/preview-deploy.yml +++ /dev/null @@ -1,35 +0,0 @@ -name: Deploy Preview Branch to restarters.dev - -on: - workflow_dispatch: - inputs: - branch: - description: 'Branch or PR ref to deploy (e.g. develop, master, refs/pull/744/head)' - required: true - default: 'develop' - -jobs: - deploy: - name: Deploy ${{ github.event.inputs.branch }} to restarters.dev - runs-on: ubuntu-latest - - steps: - - name: Checkout branch - uses: actions/checkout@v4 - with: - ref: ${{ github.event.inputs.branch }} - fetch-depth: 0 - - - name: Ensure fly.preview.toml exists - run: | - if [ ! -f fly.preview.toml ]; then - git show origin/master:fly.preview.toml > fly.preview.toml - fi - - - name: Set up flyctl - uses: superfly/flyctl-actions/setup-flyctl@master - - - name: Deploy to restarters-dev with overnight DB restore - run: flyctl deploy --config fly.preview.toml --remote-only - env: - FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} diff --git a/Dockerfile.fly b/Dockerfile.fly index 9ea71c4926..17720324d9 100644 --- a/Dockerfile.fly +++ b/Dockerfile.fly @@ -64,7 +64,8 @@ RUN rm -f .env # ============================================================================= FROM php:8.2-fpm -# yesterday mode adds mariadb-server (co-located DB for restored snapshots). +# Non-production startup scripts (yesterday, PR previews) add mariadb-server +# (co-located DB for restored snapshots). ARG STARTUP_SCRIPT=startup.sh RUN apt-get update && apt-get install -y --no-install-recommends \ nginx \ @@ -77,7 +78,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ default-mysql-client \ rclone \ libpng-dev libjpeg62-turbo-dev libfreetype6-dev libzip-dev libicu-dev libxml2-dev \ - && if [ "$STARTUP_SCRIPT" = "yesterday-startup.sh" ]; then apt-get install -y --no-install-recommends default-mysql-server; fi \ + && if [ "$STARTUP_SCRIPT" != "startup.sh" ]; then apt-get install -y --no-install-recommends default-mysql-server; fi \ && docker-php-ext-configure gd --with-freetype --with-jpeg \ && docker-php-ext-install pdo_mysql bcmath zip intl gd exif pcntl \ && apt-get clean && rm -rf /var/lib/apt/lists/* @@ -155,10 +156,11 @@ RUN mkdir -p storage/framework/{sessions,views,cache/data} \ public/uploads \ && chown -R www-data:www-data storage bootstrap/cache public/uploads -# Copy startup scripts (production + yesterday-restore variant) +# Copy startup scripts (production + yesterday-restore + PR-preview variants) COPY .fly/scripts/startup.sh /usr/local/bin/startup.sh COPY docker/yesterday-startup.sh /usr/local/bin/yesterday-startup.sh -RUN chmod +x /usr/local/bin/startup.sh /usr/local/bin/yesterday-startup.sh && \ +COPY docker/preview-startup.sh /usr/local/bin/preview-startup.sh +RUN chmod +x /usr/local/bin/startup.sh /usr/local/bin/yesterday-startup.sh /usr/local/bin/preview-startup.sh && \ cp /usr/local/bin/${STARTUP_SCRIPT} /usr/local/bin/active-startup.sh EXPOSE 80 diff --git a/app/Console/Commands/SyncDiscourseUsernames.php b/app/Console/Commands/SyncDiscourseUsernames.php index 4d172cc83e..e70ec623f5 100644 --- a/app/Console/Commands/SyncDiscourseUsernames.php +++ b/app/Console/Commands/SyncDiscourseUsernames.php @@ -42,6 +42,12 @@ public function __construct(DiscourseService $discourseService) */ public function handle(): int { + if (! config('restarters.features.discourse_integration')) { + $this->info('Discourse integration is disabled on this site - skipping.'); + + return 0; + } + $discourseApiKey = env('DISCOURSE_APIKEY'); $discourseApiUser = env('DISCOURSE_APIUSER'); diff --git a/app/Console/Commands/WordpressCreateEvent.php b/app/Console/Commands/WordpressCreateEvent.php index 8f966b3fa7..746efcdbfa 100644 --- a/app/Console/Commands/WordpressCreateEvent.php +++ b/app/Console/Commands/WordpressCreateEvent.php @@ -37,6 +37,12 @@ public function __construct() */ public function handle(): void { + if (! config('restarters.features.wordpress_integration')) { + $this->error('WordPress integration is disabled on this site.'); + + return; + } + $id = $this->argument('id'); $event = Party::findOrFail($id); diff --git a/app/Console/Commands/WordpressCreateGroup.php b/app/Console/Commands/WordpressCreateGroup.php index 96850c163c..79234d54f2 100644 --- a/app/Console/Commands/WordpressCreateGroup.php +++ b/app/Console/Commands/WordpressCreateGroup.php @@ -37,6 +37,12 @@ public function __construct() */ public function handle(): void { + if (! config('restarters.features.wordpress_integration')) { + $this->error('WordPress integration is disabled on this site.'); + + return; + } + $id = $this->argument('id'); $group = Group::findOrFail($id); diff --git a/app/Group.php b/app/Group.php index 37a3c7fb62..d61feb3f5c 100644 --- a/app/Group.php +++ b/app/Group.php @@ -475,6 +475,13 @@ public function isMemberOf($network) // If an group is not approved, then we should not push the events to Wordpress. public function eventsShouldPushToWordpress() { + // Central gate for all WordPress publishing: every listener and + // scheduled command routes through here, so preview/staging apps + // can never post to the live site. + if (! config('restarters.features.wordpress_integration')) { + return false; + } + foreach ($this->networks as $network) { if ($network->events_push_to_wordpress) { return true; diff --git a/app/Helpers/FixometerFile.php b/app/Helpers/FixometerFile.php index 1efd5a44ab..d8052e3f3d 100644 --- a/app/Helpers/FixometerFile.php +++ b/app/Helpers/FixometerFile.php @@ -38,6 +38,10 @@ public function copy($from, $to) { * */ public function upload($file, $type, $reference = null, $referenceType = null, $multiple = false, $profile = false, $ajax = false, $crop = true) { + if (! config('restarters.features.image_upload')) { + throw new \Exception('Image uploads are disabled on this site.'); + } + $clear = true; // purge pre-existing images from db - this is the default behaviour if (is_string($file) && isset($_FILES[$file])) { diff --git a/app/Http/Controllers/NetworkController.php b/app/Http/Controllers/NetworkController.php index 7ab7323e6b..ef39106cce 100644 --- a/app/Http/Controllers/NetworkController.php +++ b/app/Http/Controllers/NetworkController.php @@ -131,6 +131,11 @@ public function update(Request $request, Network $network): RedirectResponse $this->authorize('update', $network); if ($request->hasFile('network_logo')) { + if (! config('restarters.features.image_upload')) { + return redirect()->route('networks.edit', [$network]) + ->withWarning('Image uploads are disabled on this site.'); + } + // Determine the correct disk to use (s3 on Fly, public_uploads in dev) $disk = config('filesystems.default') === 's3' ? 's3' : 'public_uploads'; diff --git a/config/restarters.php b/config/restarters.php index 967dab7ee6..1b2ead11e7 100644 --- a/config/restarters.php +++ b/config/restarters.php @@ -4,6 +4,9 @@ 'features' => [ 'discourse_integration' => env('FEATURE__DISCOURSE_INTEGRATION', true) && !empty(env('DISCOURSE_URL')), + // Disabled on preview/staging apps, which share the production image bucket. + 'image_upload' => env('FEATURE__IMAGE_UPLOAD', true), + 'wordpress_integration' => env('FEATURE__WORDPRESS_INTEGRATION', true) && !empty(env('WP_XMLRPC_ENDPOINT')), ], 'wiki' => [ diff --git a/docker/nginx-fly.conf b/docker/nginx-fly.conf index d3076776c0..7600041e74 100644 --- a/docker/nginx-fly.conf +++ b/docker/nginx-fly.conf @@ -21,6 +21,11 @@ http { # Generated at startup by startup.sh: maps $cookie_site_auth → $auth_gate_valid (1=ok, 0=fail) include /etc/nginx/auth-gate-map.conf; + # Rate-limit the site-gate login to slow brute-forcing of the shared + # password. Keyed on Fly-Client-IP (set by the Fly proxy, not spoofable + # through it); requests without the header (local dev) are not limited. + limit_req_zone $http_fly_client_ip zone=auth_login:1m rate=10r/m; + log_format timed '$http_x_forwarded_for ($remote_addr) - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" ' @@ -95,11 +100,23 @@ http { # Login form — exempt from auth_request so it's always reachable. location = /_login { auth_request off; + limit_req zone=auth_login burst=5 nodelay; + limit_req_status 429; fastcgi_pass unix:/var/run/php-fpm.sock; fastcgi_param SCRIPT_FILENAME /var/www/public/_auth_login.php; include fastcgi_params; } + # PR-preview boot status (written by preview-startup.sh; no PII). + # 404s everywhere else — the file only exists on preview apps. + location = /_preview_status { + auth_request off; + access_log off; + default_type application/json; + alias /var/www/storage/framework/preview-status.json; + add_header Cache-Control "no-store"; + } + # Redirect to login, preserving the originally requested URL. location @auth_gate { return 302 /_login?next=$request_uri; diff --git a/docker/preview-startup.sh b/docker/preview-startup.sh new file mode 100755 index 0000000000..0524e1ed95 --- /dev/null +++ b/docker/preview-startup.sh @@ -0,0 +1,240 @@ +#!/usr/bin/env bash + +# Startup script for per-PR preview apps (see fly.pr.toml). +# +# Cold-boot sequence (rootfs is always fresh — Fly resets it on every start): +# 1. Serve a static "warming up" page on port 80 immediately, so the Fly +# proxy has something to route to and the health check passes within +# seconds. Visitors get an auto-refreshing page instead of a proxy error. +# 2. Start a local MariaDB and restore the most recent hourly production +# backup from Google Drive. +# 3. Truncate the restored queue tables (they contain production's +# in-flight jobs, which must never run here). +# 4. Run the PR's migrations against the restored copy, recording the +# outcome in /_preview_status for the deploy workflow to report. +# 5. Swap the warming page for the real app (supervisord). +# +# Suspend/resume never re-runs this script — the whole VM (MariaDB included) +# is snapshotted, so wakes after idle are sub-second. +# +# Do NOT use set -e — we must always reach supervisord at the end. + +LOG=/var/log/preview-restore.log +STATUS_FILE=/var/www/storage/framework/preview-status.json + +log() { echo "$(date -u '+%Y-%m-%d %H:%M:%S UTC'): $*" | tee -a "$LOG"; } + +# Machine-readable status served at /_preview_status (no PII). +# write_status [detail] +write_status() { + printf '{"phase":"%s","restore_ok":%s,"migrate_ok":%s,"detail":"%s","updated_at":"%s"}\n' \ + "$1" "$2" "$3" "${4:-}" "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" > "$STATUS_FILE" + chown www-data:www-data "$STATUS_FILE" 2>/dev/null +} + +# Ensure storage directories exist +mkdir -p /var/www/storage/framework/{sessions,views,cache/data} +mkdir -p /var/www/storage/logs +mkdir -p /var/www/bootstrap/cache +mkdir -p /var/log/nginx +mkdir -p /var/log/sysstat +chown -R www-data:www-data /var/www/storage /var/www/bootstrap/cache + +write_status "starting" false false + +# Previews only restore — they never create backups, and never restart the +# yesterday app. +( crontab -l 2>/dev/null | grep -v 'db-backup' | grep -v 'restart-yesterday' ) | crontab - + +# ── 1. Warming page: bind port 80 within seconds ───────────────────────────── + +mkdir -p /tmp/warming +cat > /tmp/warming/index.html <<'HTML' + + + + Preview warming up + + + + + +

This preview is warming up

+

The database is being restored — usually a few minutes.

+

This page refreshes automatically.

+ + +HTML +echo "User-agent: * +Disallow: /" > /tmp/warming/robots.txt + +cat > /tmp/warming-nginx.conf <<'CONF' +user www-data; +pid /tmp/warming-nginx.pid; +events { worker_connections 128; } +http { + default_type text/html; + server { + listen 80 default_server; + listen [::]:80 default_server; + root /tmp/warming; + + location = /robots.txt { } + location = /_preview_status { + default_type application/json; + alias /var/www/storage/framework/preview-status.json; + add_header Cache-Control "no-store"; + } + location / { + add_header Retry-After 30 always; + add_header Cache-Control "no-store" always; + return 503; + } + error_page 503 /index.html; + location = /index.html { internal; } + } +} +CONF + +nginx -c /tmp/warming-nginx.conf && log "Warming page up on port 80." + +# ── 2. Auth gate + Tigris config for the real nginx (same as production) ───── + +if [ "${BASIC_AUTH_ENABLED:-}" = "true" ]; then + EXPECTED=$(php -r "echo hash_hmac('sha256', getenv('BASIC_AUTH_PASSWORD') ?: 'preview', getenv('APP_KEY') ?: 'fallback');") + printf 'map $cookie_site_auth $auth_gate_valid {\n "%s" 1;\n default 0;\n}\nmap $auth_gate_valid $auth_check_status {\n 1 200;\n default 401;\n}\n' \ + "$EXPECTED" > /etc/nginx/auth-gate-map.conf + printf 'auth_request /_auth_check;\nerror_page 401 = @auth_gate;\n' \ + > /etc/nginx/auth-gate.conf +else + printf 'map $cookie_site_auth $auth_gate_valid { default 1; }\nmap $auth_gate_valid $auth_check_status { default 200; }\n' \ + > /etc/nginx/auth-gate-map.conf + : > /etc/nginx/auth-gate.conf +fi + +if [ -n "$AWS_BUCKET" ]; then + export TIGRIS_BUCKET_URL="https://${AWS_BUCKET}.fly.storage.tigris.dev" + export TIGRIS_BUCKET_HOST="${AWS_BUCKET}.fly.storage.tigris.dev" + envsubst '${TIGRIS_BUCKET_URL} ${TIGRIS_BUCKET_HOST}' < /etc/nginx/nginx.conf > /etc/nginx/nginx.conf.tmp + mv /etc/nginx/nginx.conf.tmp /etc/nginx/nginx.conf +fi + +# ── 3. Start local MariaDB ──────────────────────────────────────────────────── + +log "Starting local MariaDB..." +write_status "restoring" false false +mkdir -p /var/lib/mysql /run/mysqld +chown -R mysql:mysql /var/lib/mysql /run/mysqld + +if [ ! -d /var/lib/mysql/mysql ]; then + mysql_install_db --user=mysql --datadir=/var/lib/mysql > /dev/null 2>&1 +fi + +# 512M buffer pool: the VM has 2GB and no swap (a suspend requirement), so +# leave room for PHP-FPM. +mysqld_safe --user=mysql --skip-networking=0 --bind-address=127.0.0.1 \ + --innodb-buffer-pool-size=512M \ + --innodb-flush-log-at-trx-commit=2 & + +for i in $(seq 1 30); do + if mysqladmin ping --silent 2>/dev/null; then break; fi + sleep 2 +done + +mysql -e "CREATE DATABASE IF NOT EXISTS \`${DB_DATABASE:-restarters}\` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;" 2>/dev/null +mysql -e "CREATE USER IF NOT EXISTS '${DB_USERNAME:-restarters}'@'127.0.0.1' IDENTIFIED BY '${DB_PASSWORD:-restarters}';" 2>/dev/null +mysql -e "GRANT ALL ON \`${DB_DATABASE:-restarters}\`.* TO '${DB_USERNAME:-restarters}'@'127.0.0.1'; FLUSH PRIVILEGES;" 2>/dev/null +# The production backup contains triggers with DEFINER='restarters'@'%'. Create that user locally +# so trigger execution doesn't fail (bind-address=127.0.0.1 so '%' can't connect from outside). +mysql -e "CREATE USER IF NOT EXISTS '${DB_USERNAME:-restarters}'@'%' IDENTIFIED BY '${DB_PASSWORD:-restarters}';" 2>/dev/null +mysql -e "GRANT ALL ON \`${DB_DATABASE:-restarters}\`.* TO '${DB_USERNAME:-restarters}'@'%'; FLUSH PRIVILEGES;" 2>/dev/null + +log "MariaDB ready." + +# ── 4. Restore the most recent backup ───────────────────────────────────────── + +RCLONE_FLAGS="--drive-root-folder-id=$GDRIVE_BACKUP_FOLDER_ID --drive-team-drive=$RCLONE_CONFIG_GDRIVE_TEAM_DRIVE" + +RESTORE_OK=false +BACKUP_FILE=$(rclone lsf "gdrive:" $RCLONE_FLAGS 2>/dev/null | grep 'db-backup-.*\.sql\.gz' | sort | tail -1) + +if [ -z "$BACKUP_FILE" ]; then + log "ERROR: No backup files found in Google Drive. Starting with empty database." +else + log "Restoring most recent backup: $BACKUP_FILE" + + if rclone copy "gdrive:$BACKUP_FILE" /tmp/ $RCLONE_FLAGS 2>>"$LOG"; then + log "Download complete. Restoring database..." + + if gunzip -c "/tmp/$BACKUP_FILE" | mysql -u root \ + --socket=/run/mysqld/mysqld.sock \ + "${DB_DATABASE:-restarters}" 2>>"$LOG"; then + RESTORE_OK=true + log "Database restore complete." + else + log "ERROR: Database restore failed." + fi + + rm -f "/tmp/$BACKUP_FILE" + else + log "ERROR: Download failed." + fi +fi + +# The backup captures production's in-flight queue jobs; the supervisord +# worker runs `queue:work database` regardless of QUEUE_CONNECTION, so these +# must never survive into the preview. +mysql -u root --socket=/run/mysqld/mysqld.sock "${DB_DATABASE:-restarters}" \ + -e "TRUNCATE TABLE jobs; TRUNCATE TABLE failed_jobs;" 2>>"$LOG" \ + && log "Cleared restored queue tables." + +# ── 5. Run the PR's migrations against the restored copy ───────────────────── + +MIGRATE_OK=false +if [ "$RESTORE_OK" = "true" ]; then + write_status "migrating" true false + log "Running migrations..." + php /var/www/artisan migrate --force 2>&1 | tee -a "$LOG" + # tee always succeeds - the pipeline hides migrate's exit code without this. + if [ "${PIPESTATUS[0]}" = "0" ]; then + MIGRATE_OK=true + log "Migrations complete." + else + log "ERROR: migrate --force failed - see above." + fi + timeout 30 php /var/www/artisan translations:import 2>/dev/null || true +fi + +# Warm up Laravel caches +php /var/www/artisan config:cache 2>/dev/null || true +php /var/www/artisan route:cache 2>/dev/null || true +php /var/www/artisan view:cache 2>/dev/null || true +chown -R www-data:www-data /var/www/storage /var/www/bootstrap/cache + +if [ "$RESTORE_OK" = "true" ] && [ "$MIGRATE_OK" = "true" ]; then + write_status "ready" true true +elif [ "$RESTORE_OK" = "true" ]; then + write_status "failed" true false "migrate failed - check machine logs" +else + write_status "failed" false false "restore failed - check machine logs" +fi + +# ── 6. Hand over from the warming page to the real app ─────────────────────── + +if [ -f /tmp/warming-nginx.pid ]; then + kill -QUIT "$(cat /tmp/warming-nginx.pid)" 2>/dev/null + for _ in $(seq 1 10); do + [ ! -f /tmp/warming-nginx.pid ] && break + kill -0 "$(cat /tmp/warming-nginx.pid)" 2>/dev/null || break + sleep 1 + done + # Belt and braces: never let a stuck warming nginx block the real one. + [ -f /tmp/warming-nginx.pid ] && kill -9 "$(cat /tmp/warming-nginx.pid)" 2>/dev/null +fi + +log "Starting supervisord..." +exec /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf diff --git a/docs/fly-deployment.md b/docs/fly-deployment.md index 07f07ecb42..f8d7231547 100644 --- a/docs/fly-deployment.md +++ b/docs/fly-deployment.md @@ -14,6 +14,7 @@ How the application is built, run, and deployed on Fly.io. | `restarters-db` | `fly-mysql.toml` | internal only | — | | `restarters-pma` | `fly-pma.toml` | via `flyctl proxy` only | — | | `restarters-yesterday` | `fly-yesterday.toml` | `yesterday.restarters.net` (sleeps when idle) | — | +| `restarters-pr-` | `fly.pr.toml` (template) | `restarters-pr-.fly.dev` (sleeps when idle) | any PR labeled `preview` | All apps run in the `lhr` (London) region. The DB is on a private 6PN network (`restarters-db.internal`) — only reachable by other apps in the same Fly organisation, not from the public internet. @@ -156,6 +157,93 @@ The `production` branch deploy runs without waiting for tests (it is triggered b --- +## PR Previews + +Any PR can be deployed as a self-contained test site at +`https://restarters-pr-.fly.dev`, running the PR's code against a copy of +the live database. + +### Using it + +1. Add the `preview` label to the PR (or run the **PR preview** workflow + manually with the PR number). Every push to a labeled PR redeploys. +2. Each deploy waits for approval (GitHub Environment `preview`). **Before + approving, glance at the PR's diff to `.github/workflows/`, + `fly.pr.toml`, `Dockerfile.fly` and `docker/*.sh`** — the deploy runs the + PR's own code with the preview credentials. +3. The workflow comments on the PR with the URL and the restore/migration + result. The site is behind the usual cookie gate. + +### How it works + +- Same image build as production (`Dockerfile.fly`), with an embedded local + MariaDB (the `restarters-yesterday` pattern). `docker/preview-startup.sh` + serves a static "warming up" page within seconds of boot, restores the + most recent hourly production backup, truncates the restored `jobs` / + `failed_jobs` tables (production's in-flight jobs must not run here), then + runs `php artisan migrate --force` — so a PR's migrations are exercised + against real data before merge. Progress is reported at + `/_preview_status` (no auth, no PII). +- The machine **suspends** after ~5 min idle (whole-VM snapshot; MariaDB + stays loaded) and resumes sub-second on the next request. If Fly discards + the snapshot, the next visitor gets the warming page while the machine + cold-boots and re-restores. This is why `fly.pr.toml` has no + `swap_size_mb`: swap disables suspend. +- Cost: ~$0.09/hour only while someone is browsing; rootfs storage only + (well under $1/month) while suspended; nothing once destroyed. +- A sweep (4x daily, and on manual dispatch with an empty PR number) + destroys apps whose PR is closed or no longer labeled. Cleanup + deliberately does **not** run on `pull_request` events: those execute the + PR's modified workflow file, which must never reach the org token without + approval. The sweep runs default-branch code via the `preview-cleanup` + environment, which is restricted to the `develop` branch. + +### Environment isolation + +Previews hold **no production-capable credentials**: + +- Fresh `APP_KEY` generated per app (never production's — that key can forge + production session cookies). +- Read-only Tigris credentials; `FEATURE__IMAGE_UPLOAD=false` gives users a + clean error instead of an IAM failure. Image *reads* come from the shared + live bucket. +- Read-only Google Drive service account for the backup restore (the + production backup SA can delete backups; that one never leaves production). +- `FEATURE__WIKI/DISCOURSE/WORDPRESS_INTEGRATION` all `false`; mail goes to + the shared Mailpit (`restarters-dev-mail`). + +### One-time setup + +1. **Fly org token**: `fly tokens create org` → GitHub secret + `FLY_ORG_TOKEN` in **both** the `preview` and `preview-cleanup` + environments (not repo-level). +2. **GitHub environments**: + - `preview` — required reviewers: the maintainers; holds `FLY_ORG_TOKEN` + and `FLY_PREVIEW_SECRETS`. + - `preview-cleanup` — no reviewers; deployment branches restricted to + `develop` only; holds `FLY_ORG_TOKEN`. +3. **Repository variable** `FLY_ORG` = the Fly organisation slug. +4. **`FLY_PREVIEW_SECRETS`** (env-file format, one `KEY=value` per line) in + the `preview` environment: + - `BASIC_AUTH_PASSWORD` — a fresh value (the old committed one is public + in git history; rotate, don't reuse) + - `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_BUCKET` — a + **read-only** Tigris access key for the shared bucket + - `GDRIVE_BACKUP_FOLDER_ID`, `RCLONE_CONFIG_GDRIVE_TYPE=drive`, + `RCLONE_CONFIG_GDRIVE_SCOPE=drive.readonly`, + `RCLONE_CONFIG_GDRIVE_TEAM_DRIVE`, + `RCLONE_CONFIG_GDRIVE_SERVICE_ACCOUNT_CREDENTIALS` — a **new** service + account with only Viewer access to the backup folder + - `MAPBOX_TOKEN`, `GOOGLE_API_CONSOLE_KEY` (optional: `SENTRY_LARAVEL_DSN`) + - Never any production-only key (`MAIL_*`, `DISCOURSE_*`, `WIKI_*`, + `WP_*`, `DRIP_*`, analytics). +5. **Label**: create the `preview` label on the repo. + +Note: previews contain an unscrubbed copy of live data behind the cookie +gate — treat preview URLs and the gate password accordingly. + +--- + ## Database Backups A compressed MySQL dump is uploaded to Google Drive every hour by `docker/db-backup.sh`, scheduled in the production container's crontab (`0 * * * *`). diff --git a/fly.pr.toml b/fly.pr.toml new file mode 100644 index 0000000000..749467bafc --- /dev/null +++ b/fly.pr.toml @@ -0,0 +1,121 @@ +# Template for per-PR preview apps (restarters-pr-.fly.dev). +# +# __PR_NUMBER__ placeholders are substituted by .github/workflows/pr-preview.yml +# before deploying. Do not deploy this file directly. +# +# Each preview runs the PR's code against a local embedded MariaDB restored +# from the most recent hourly production backup at cold boot. The machine +# suspends after ~5 min idle (whole-VM snapshot, so the DB stays loaded and +# resume is sub-second) and is destroyed when the PR closes. +# +# Secrets are set once at app creation by the workflow (see docs/fly-deployment.md): +# a fresh per-app APP_KEY, BASIC_AUTH_PASSWORD, read-only Tigris credentials, +# and read-only Google Drive credentials for the backup restore. + +app = "restarters-pr-__PR_NUMBER__" +primary_region = "lhr" + +[build] + dockerfile = "Dockerfile.fly" + [build.args] + STARTUP_SCRIPT = "preview-startup.sh" + +[env] + APP_NAME = "Restarters PR #__PR_NUMBER__" + APP_ENV = "staging" + APP_DEBUG = "false" + APP_URL = "https://restarters-pr-__PR_NUMBER__.fly.dev" + # Local embedded MariaDB (restored from backup at cold boot; 127.0.0.1 only) + DB_CONNECTION = "mysql" + DB_HOST = "127.0.0.1" + DB_PORT = "3306" + DB_DATABASE = "restarters" + DB_USERNAME = "restarters" + DB_PASSWORD = "restarters_preview" + CACHE_DRIVER = "file" + SESSION_DRIVER = "file" + QUEUE_CONNECTION = "sync" + SESSION_DOMAIN = "" + LOG_CHANNEL = "stderr" + FILESYSTEM_DISK = "s3" + AWS_DEFAULT_REGION = "auto" + AWS_ENDPOINT = "https://fly.storage.tigris.dev" + AWS_USE_PATH_STYLE_ENDPOINT = "true" + # Previews must never reach real external services. + FEATURE__WIKI_INTEGRATION = "false" + FEATURE__DISCOURSE_INTEGRATION = "false" + FEATURE__WORDPRESS_INTEGRATION = "false" + # Reads come from the shared live bucket; writes are disabled (and the + # Tigris credentials are read-only anyway). + FEATURE__IMAGE_UPLOAD = "false" + SENTRY_ENVIRONMENT = "preview" + # Mail goes to the shared Mailpit instance, never to real inboxes. + MAIL_MAILER = "smtp" + MAIL_HOST = "restarters-dev-mail.internal" + MAIL_PORT = "1025" + MAIL_ENCRYPTION = "" + MAIL_FROM_ADDRESS = "preview@restarters.dev" + MAIL_FROM_NAME = "Restarters PR Preview" + MAILPIT_URL = "https://restarters-dev-mail.fly.dev" + # Show the red branch banner (label derived from FLY_APP_NAME → "PR-"). + APP_SHOW_BRANCH = "true" + BASIC_AUTH_ENABLED = "true" + # Table type constants + TBL_USERS = "1" + TBL_GROUPS = "2" + TBL_EVENTS = "3" + TBL_DEVICES = "4" + TBL_IMAGES = "5" + TBL_LINKS = "6" + # Device status constants + DEVICE_FIXED = "1" + DEVICE_REPAIRABLE = "2" + DEVICE_DEAD = "3" + # Category IDs + MISC_CATEGORY_ID = "46" + MISC_CATEGORY_ID_POWERED = "46" + MISC_CATEGORY_ID_UNPOWERED = "50" + # Impact calculation constants + DISPLACEMENT_VALUE = "0.5" + EMISSION_RATIO_UNPOWERED = "17.70147059" + EMISSION_RATIO_POWERED = "32.06679233" + # Session + SESSION_COOKIE = "restarters_pr_session" + SESSION_LIFETIME = "10080" + SESSION_TABLE = "laravel_sessions" + # Misc + L5_SWAGGER_GENERATE_ALWAYS = "false" + VITE_MANIFEST_PATH = "public/build/manifest.json" + +[http_service] + internal_port = 80 + force_https = true + # suspend snapshots the whole VM (needs memory <= 2GB and NO swap - see [[vm]]). + # Resume is sub-second with MariaDB still loaded. If Fly discards the + # snapshot the machine cold-boots and preview-startup.sh re-restores. + auto_stop_machines = "suspend" + auto_start_machines = true + min_machines_running = 0 + +[checks] + [checks.health] + port = 80 + type = "http" + interval = "30s" + timeout = "10s" + # The warming-page nginx binds port 80 within seconds of boot, so the + # check passes long before the DB restore completes. Poll + # /_preview_status to know when the site is actually usable. + grace_period = "120s" + method = "GET" + path = "/robots.txt" + +# No [mounts]: logs go to stderr, the DB lives on the (ephemeral) rootfs and +# is re-restored from backup on every cold boot. Nothing to persist. + +[[vm]] + size = "performance-1x" + # 2048MB is the suspend ceiling. Deliberately NO swap_size_mb: machines + # with swap cannot suspend (Fly silently falls back to stop, which would + # force a full re-restore on every wake). + memory = 2048 diff --git a/fly.preview.toml b/fly.preview.toml deleted file mode 100644 index 9faf481129..0000000000 --- a/fly.preview.toml +++ /dev/null @@ -1,97 +0,0 @@ -app = "restarters-dev" -primary_region = "lhr" - -[build] - dockerfile = "Dockerfile.fly" - [build.args] - # Restore the overnight DB backup at startup (same logic as yesterday container) - STARTUP_SCRIPT = "yesterday-startup.sh" - -# Migrations run in startup.sh instead of release_command because -# release_command machines don't have 6PN network access to the DB. - -[env] - APP_ENV = "staging" - APP_DEBUG = "false" - APP_URL = "https://restarters.dev" - # Use local embedded MariaDB (restored from backup at startup) - DB_CONNECTION = "mysql" - DB_HOST = "127.0.0.1" - DB_PORT = "3306" - DB_DATABASE = "restarters" - DB_USERNAME = "restarters" - DB_PASSWORD = "restarters_preview" - CACHE_DRIVER = "file" - SESSION_DRIVER = "file" - QUEUE_CONNECTION = "sync" - SESSION_DOMAIN = "" - LOG_CHANNEL = "stderr" - FILESYSTEM_DISK = "s3" - AWS_DEFAULT_REGION = "auto" - AWS_ENDPOINT = "https://fly.storage.tigris.dev" - AWS_USE_PATH_STYLE_ENDPOINT = "true" - FEATURE__WIKI_INTEGRATION = "false" - FEATURE__DISCOURSE_INTEGRATION = "false" - SENTRY_ENVIRONMENT = "preview" - MAIL_MAILER = "smtp" - MAIL_HOST = "restarters-dev-mail.internal" - MAIL_PORT = "1025" - MAIL_ENCRYPTION = "" - MAIL_FROM_ADDRESS = "dev@restarters.dev" - MAIL_FROM_NAME = "Restarters Dev (Preview)" - # Table type constants - TBL_USERS = "1" - TBL_GROUPS = "2" - TBL_EVENTS = "3" - TBL_DEVICES = "4" - TBL_IMAGES = "5" - TBL_LINKS = "6" - # Device status constants - DEVICE_FIXED = "1" - DEVICE_REPAIRABLE = "2" - DEVICE_DEAD = "3" - # Category IDs - MISC_CATEGORY_ID = "46" - MISC_CATEGORY_ID_POWERED = "46" - MISC_CATEGORY_ID_UNPOWERED = "50" - # Impact calculation constants - DISPLACEMENT_VALUE = "0.5" - EMISSION_RATIO_UNPOWERED = "17.70147059" - EMISSION_RATIO_POWERED = "32.06679233" - # Session - SESSION_COOKIE = "restarters_session" - SESSION_LIFETIME = "10080" - SESSION_TABLE = "laravel_sessions" - # Misc - L5_SWAGGER_GENERATE_ALWAYS = "false" - VITE_MANIFEST_PATH = "public/build/manifest.json" - BASIC_AUTH_ENABLED = "true" - BASIC_AUTH_PASSWORD = "project" - APP_SHOW_BRANCH = "true" - -[http_service] - internal_port = 80 - force_https = true - auto_stop_machines = "stop" - auto_start_machines = true - min_machines_running = 0 - -[checks] - [checks.health] - port = 80 - type = "http" - interval = "30s" - timeout = "10s" - # Long grace period covers full DB restore (~10 min) - grace_period = "900s" - method = "GET" - path = "/robots.txt" - -[mounts] - source = "app_logs_dev" - destination = "/var/log" - -[[vm]] - size = "performance-1x" - memory = 2048 - swap_size_mb = 2048 diff --git a/resources/views/layouts/navbar.blade.php b/resources/views/layouts/navbar.blade.php index 1a96104de4..4b27c95753 100644 --- a/resources/views/layouts/navbar.blade.php +++ b/resources/views/layouts/navbar.blade.php @@ -18,7 +18,8 @@ if ($flyApp) { // On Fly: derive branch label by stripping the "restarters-" prefix $branch = preg_replace('/^restarters-/', '', $flyApp); - $mailpitUrl = 'https://' . $flyApp . '-mail.fly.dev'; + // PR previews share one Mailpit instance rather than having their own. + $mailpitUrl = env('MAILPIT_URL') ?: 'https://' . $flyApp . '-mail.fly.dev'; } else { // Local dev fallback: read from git $branch = 'unknown'; diff --git a/tests/Feature/UploadsDisabledTest.php b/tests/Feature/UploadsDisabledTest.php new file mode 100644 index 0000000000..29136a9207 --- /dev/null +++ b/tests/Feature/UploadsDisabledTest.php @@ -0,0 +1,127 @@ +assertTrue(config('restarters.features.image_upload')); + } + + /** @test */ + public function group_image_upload_rejected_when_uploads_disabled(): void + { + config(['restarters.features.image_upload' => false]); + + $group = Group::factory()->create(); + $host = User::factory()->host()->create(); + $group->addVolunteer($host); + $group->makeMemberAHost($host); + $this->actingAs($host); + + $_SERVER['DOCUMENT_ROOT'] = getcwd(); + \FixometerFile::$uploadTesting = true; + file_put_contents('/tmp/UT_disabled.jpg', file_get_contents(public_path() . '/images/community.jpg')); + + $_FILES = [ + 'file' => [ + 'error' => '0', + 'name' => 'UT_disabled.jpg', + 'size' => 123, + 'tmp_name' => ['/tmp/UT_disabled.jpg'], + 'type' => 'image/jpg', + ], + ]; + + $imagesBefore = \DB::table('images')->count(); + + $response = $this->json('POST', '/group/image-upload/' . $group->idgroups, []); + $response->assertOk(); + $this->assertStringStartsWith('fail - image could not be uploaded', $response->getContent()); + $this->assertStringContainsString('disabled', $response->getContent()); + + $this->assertEquals($imagesBefore, \DB::table('images')->count()); + } + + /** @test */ + public function group_image_upload_allowed_when_uploads_enabled(): void + { + config(['restarters.features.image_upload' => true]); + + $group = Group::factory()->create(); + $host = User::factory()->host()->create(); + $group->addVolunteer($host); + $group->makeMemberAHost($host); + $this->actingAs($host); + + $_SERVER['DOCUMENT_ROOT'] = getcwd(); + \FixometerFile::$uploadTesting = true; + file_put_contents('/tmp/UT_enabled.jpg', file_get_contents(public_path() . '/images/community.jpg')); + + $_FILES = [ + 'file' => [ + 'error' => '0', + 'name' => 'UT_enabled.jpg', + 'size' => 123, + 'tmp_name' => ['/tmp/UT_enabled.jpg'], + 'type' => 'image/jpg', + ], + ]; + + $response = $this->json('POST', '/group/image-upload/' . $group->idgroups, []); + $response->assertOk(); + $this->assertEquals('success - image uploaded', $response->getContent()); + } + + /** @test */ + public function network_logo_upload_rejected_when_uploads_disabled(): void + { + config(['restarters.features.image_upload' => false]); + Storage::fake('public_uploads'); + + $network = Network::factory()->create(); + $admin = User::factory()->administrator()->create(); + $this->actingAs($admin); + + $response = $this->put(route('networks.update', $network), [ + 'network_logo' => UploadedFile::fake()->image('logo.png'), + ]); + + $response->assertRedirect(route('networks.edit', $network)); + $response->assertSessionHas('warning'); + $this->assertNull($network->fresh()->logo); + } + + /** @test */ + public function network_logo_upload_allowed_when_uploads_enabled(): void + { + config(['restarters.features.image_upload' => true]); + Storage::fake('public_uploads'); + + $network = Network::factory()->create(); + $admin = User::factory()->administrator()->create(); + $this->actingAs($admin); + + $response = $this->put(route('networks.update', $network), [ + 'network_logo' => UploadedFile::fake()->image('logo.png'), + ]); + + $response->assertRedirect(route('networks.edit', $network)); + $logo = $network->fresh()->logo; + $this->assertNotNull($logo); + $this->assertStringStartsWith('network_logos/', $logo); + } +} diff --git a/tests/Feature/WordpressIntegrationFlagTest.php b/tests/Feature/WordpressIntegrationFlagTest.php new file mode 100644 index 0000000000..3bbbfe3568 --- /dev/null +++ b/tests/Feature/WordpressIntegrationFlagTest.php @@ -0,0 +1,105 @@ +app['queue']; + $queueManager->setDefaultDriver('sync'); + } + + private function createEventInPushNetwork(): Party + { + $network = Network::factory()->create([ + 'events_push_to_wordpress' => true, + ]); + $group = Group::factory()->create([ + 'approved' => true, + ]); + $network->addGroup($group); + + return Party::factory()->create([ + 'group' => $group->idgroups, + 'latitude' => 1, + 'longitude' => 1, + 'event_start_utc' => '2100-01-01T10:15:05+05:00', + 'event_end_utc' => '2100-01-01T13:45:05+05:00', + ]); + } + + /** @test */ + public function event_approval_does_not_push_when_integration_disabled(): void + { + config(['restarters.features.wordpress_integration' => false]); + + $this->instance(WordpressClient::class, Mockery::mock(WordpressClient::class, function ($mock) { + $mock->shouldNotReceive('newPost'); + })); + + $event = $this->createEventInPushNetwork(); + + $this->assertFalse($event->shouldPushToWordpress()); + + $event->approve(); + $this->artisan('queue:work --stop-when-empty'); + } + + /** @test */ + public function event_approval_pushes_when_integration_enabled(): void + { + config(['restarters.features.wordpress_integration' => true]); + + $this->instance(WordpressClient::class, Mockery::mock(WordpressClient::class, function ($mock) { + $mock->shouldReceive('newPost')->once(); + })); + + $event = $this->createEventInPushNetwork(); + + $this->assertTrue($event->shouldPushToWordpress()); + + $event->approve(); + $this->artisan('queue:work --stop-when-empty'); + } + + /** @test */ + public function manual_wordpress_commands_refuse_when_integration_disabled(): void + { + config(['restarters.features.wordpress_integration' => false]); + + $event = $this->createEventInPushNetwork(); + + $this->artisan('wordpress:event:create ' . $event->idevents) + ->expectsOutputToContain('disabled') + ->assertExitCode(0); + + $this->artisan('wordpress:group:create ' . $event->theGroup->idgroups) + ->expectsOutputToContain('disabled') + ->assertExitCode(0); + } + + /** @test */ + public function sync_discourseusernames_skips_when_integration_disabled(): void + { + config(['restarters.features.discourse_integration' => false]); + + $this->artisan('sync:discourseusernames') + ->expectsOutputToContain('disabled') + ->assertExitCode(0); + } +} From b79d1cdea48786e16d59cbd980092d9fc7963f02 Mon Sep 17 00:00:00 2001 From: edwh Date: Mon, 13 Jul 2026 19:33:16 +0100 Subject: [PATCH 2/7] Preview deploys build the PR's merge result, not the raw head A stale branch previewed as-is runs old code against a newer live schema. Deploying refs/pull/N/merge (PR + current base, same as CI) means the code is never older than develop, and production schema always lags develop, so the restored backup can't be ahead of the code. Conflicted PRs fail fast with a clear message; the PR comment now notes how many commits (and migration-touching commits) the branch is behind. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- .github/workflows/pr-preview.yml | 45 ++++++++++++++++++++++++++++---- docs/fly-deployment.md | 5 ++++ 2 files changed, 45 insertions(+), 5 deletions(-) diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 3f3ed41dc9..33c49796ee 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -58,9 +58,10 @@ jobs: PR="${{ github.event.pull_request.number }}" fi INFO=$(gh pr view "$PR" --repo "$GITHUB_REPOSITORY" \ - --json state,headRefOid,headRepository,headRepositoryOwner,labels) + --json state,headRefOid,headRepository,headRepositoryOwner,labels,baseRefName) STATE=$(echo "$INFO" | jq -r .state) HEAD_SHA=$(echo "$INFO" | jq -r .headRefOid) + BASE_REF=$(echo "$INFO" | jq -r .baseRefName) HEAD_REPO="$(echo "$INFO" | jq -r .headRepositoryOwner.login)/$(echo "$INFO" | jq -r .headRepository.name)" # Fork guard. The pull_request trigger already enforces this via the @@ -75,17 +76,44 @@ jobs: exit 1 fi + # We deploy the MERGE RESULT (PR + current base), like CI tests, so a + # stale branch is previewed as it would actually land. GitHub only + # provides the merge ref while the PR is conflict-free; mergeable is + # UNKNOWN briefly after a push, so retry before giving up. + MERGEABLE="UNKNOWN" + for _ in $(seq 1 10); do + MERGEABLE=$(gh pr view "$PR" --repo "$GITHUB_REPOSITORY" --json mergeable --jq .mergeable) + [ "$MERGEABLE" != "UNKNOWN" ] && break + sleep 10 + done + if [ "$MERGEABLE" = "CONFLICTING" ]; then + echo "::error::PR #$PR has conflicts with $BASE_REF - resolve them before previewing." + exit 1 + fi + echo "number=$PR" >> "$GITHUB_OUTPUT" echo "sha=$HEAD_SHA" >> "$GITHUB_OUTPUT" + echo "base=$BASE_REF" >> "$GITHUB_OUTPUT" echo "app=restarters-pr-$PR" >> "$GITHUB_OUTPUT" - - name: Checkout PR head + - name: Checkout PR merged with base uses: actions/checkout@v4 with: - ref: ${{ steps.pr.outputs.sha }} + ref: refs/pull/${{ steps.pr.outputs.number }}/merge fetch-depth: 0 - - name: Overlay preview infra files from develop if the branch predates them + - name: Measure staleness against base + id: stale + run: | + git fetch origin "${{ steps.pr.outputs.base }}" + BEHIND=$(git rev-list --count "${{ steps.pr.outputs.sha }}..origin/${{ steps.pr.outputs.base }}" 2>/dev/null || echo 0) + MIG_BEHIND=$(git log --oneline "${{ steps.pr.outputs.sha }}..origin/${{ steps.pr.outputs.base }}" -- database/migrations 2>/dev/null | wc -l) + echo "behind=$BEHIND" >> "$GITHUB_OUTPUT" + echo "mig_behind=$MIG_BEHIND" >> "$GITHUB_OUTPUT" + + # Merging with the base normally brings these files in; this only fires + # for PRs whose base branch predates preview support. + - name: Overlay preview infra files from develop if still missing run: | git fetch origin develop --depth=1 for f in fly.pr.toml docker/preview-startup.sh; do @@ -172,14 +200,21 @@ jobs: failed) STATUS_LINE=":x: Deploy completed but startup reported a failure: ${DETAIL:-see \`flyctl logs --app $APP\`}." ;; *) STATUS_LINE=":warning: Deployed, but the status endpoint did not report readiness - check \`flyctl logs --app $APP\`." ;; esac + BEHIND="${{ steps.stale.outputs.behind }}" + MIG_BEHIND="${{ steps.stale.outputs.mig_behind }}" + STALE_LINE="" + if [ "${BEHIND:-0}" -gt 0 ]; then + STALE_LINE="- Branch is $BEHIND commit(s) behind \`${{ steps.pr.outputs.base }}\` ($MIG_BEHIND touching migrations); the preview is built from the **merge** with current \`${{ steps.pr.outputs.base }}\`." + fi MARKER="" BODY=$(cat < Date: Mon, 13 Jul 2026 20:35:41 +0100 Subject: [PATCH 3/7] Address SonarCloud findings on the preview workflow - Pin superfly/flyctl-actions to a full commit SHA: @master would let a compromised upstream repo ship code straight into the job that holds the org token - Move permissions from workflow level to job level (cleanup only needs read access) - Deduplicate UploadsDisabledTest setup into helpers Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- .github/workflows/pr-preview.yml | 14 ++-- tests/Feature/UploadsDisabledTest.php | 102 +++++++++++++------------- 2 files changed, 57 insertions(+), 59 deletions(-) diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 33c49796ee..08c172008b 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -29,10 +29,6 @@ on: schedule: - cron: '43 2,8,14,20 * * *' # cleanup sweep, 4x daily -permissions: - contents: read - pull-requests: write - jobs: deploy: name: Deploy preview @@ -43,6 +39,9 @@ jobs: github.event.pull_request.head.repo.full_name == github.repository) runs-on: ubuntu-latest environment: preview + permissions: + contents: read + pull-requests: write concurrency: group: pr-preview-${{ github.event.pull_request.number || github.event.inputs.pr_number }} cancel-in-progress: true @@ -135,7 +134,7 @@ jobs: fi - name: Set up flyctl - uses: superfly/flyctl-actions/setup-flyctl@master + uses: superfly/flyctl-actions/setup-flyctl@ed8efb33836e8b2096c7fd3ba1c8afe303ebbff1 # master 2026-04-08 - name: Create app and set secrets (first deploy only) env: @@ -240,6 +239,9 @@ jobs: if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && github.event.inputs.pr_number == '') runs-on: ubuntu-latest environment: preview-cleanup + permissions: + contents: read + pull-requests: read concurrency: group: pr-preview-sweep env: @@ -247,7 +249,7 @@ jobs: FLY_API_TOKEN: ${{ secrets.FLY_ORG_TOKEN }} steps: - name: Set up flyctl - uses: superfly/flyctl-actions/setup-flyctl@master + uses: superfly/flyctl-actions/setup-flyctl@ed8efb33836e8b2096c7fd3ba1c8afe303ebbff1 # master 2026-04-08 - name: Sweep run: | diff --git a/tests/Feature/UploadsDisabledTest.php b/tests/Feature/UploadsDisabledTest.php index 29136a9207..dbfed1dfd4 100644 --- a/tests/Feature/UploadsDisabledTest.php +++ b/tests/Feature/UploadsDisabledTest.php @@ -7,6 +7,7 @@ use App\User; use Illuminate\Http\UploadedFile; use Illuminate\Support\Facades\Storage; +use Illuminate\Testing\TestResponse; use Tests\TestCase; /** @@ -26,29 +27,9 @@ public function group_image_upload_rejected_when_uploads_disabled(): void { config(['restarters.features.image_upload' => false]); - $group = Group::factory()->create(); - $host = User::factory()->host()->create(); - $group->addVolunteer($host); - $group->makeMemberAHost($host); - $this->actingAs($host); - - $_SERVER['DOCUMENT_ROOT'] = getcwd(); - \FixometerFile::$uploadTesting = true; - file_put_contents('/tmp/UT_disabled.jpg', file_get_contents(public_path() . '/images/community.jpg')); - - $_FILES = [ - 'file' => [ - 'error' => '0', - 'name' => 'UT_disabled.jpg', - 'size' => 123, - 'tmp_name' => ['/tmp/UT_disabled.jpg'], - 'type' => 'image/jpg', - ], - ]; - $imagesBefore = \DB::table('images')->count(); - $response = $this->json('POST', '/group/image-upload/' . $group->idgroups, []); + $response = $this->postGroupImage('UT_disabled.jpg'); $response->assertOk(); $this->assertStringStartsWith('fail - image could not be uploaded', $response->getContent()); $this->assertStringContainsString('disabled', $response->getContent()); @@ -61,6 +42,42 @@ public function group_image_upload_allowed_when_uploads_enabled(): void { config(['restarters.features.image_upload' => true]); + $response = $this->postGroupImage('UT_enabled.jpg'); + $response->assertOk(); + $this->assertEquals('success - image uploaded', $response->getContent()); + } + + /** @test */ + public function network_logo_upload_rejected_when_uploads_disabled(): void + { + config(['restarters.features.image_upload' => false]); + + [$network, $response] = $this->putNetworkLogo(); + + $response->assertRedirect(route('networks.edit', $network)); + $response->assertSessionHas('warning'); + $this->assertNull($network->fresh()->logo); + } + + /** @test */ + public function network_logo_upload_allowed_when_uploads_enabled(): void + { + config(['restarters.features.image_upload' => true]); + + [$network, $response] = $this->putNetworkLogo(); + + $response->assertRedirect(route('networks.edit', $network)); + $logo = $network->fresh()->logo; + $this->assertNotNull($logo); + $this->assertStringStartsWith('network_logos/', $logo); + } + + /** + * Post an image to the group image-upload endpoint as a host, using the + * legacy $_FILES mechanism the endpoint expects. + */ + private function postGroupImage(string $filename): TestResponse + { $group = Group::factory()->create(); $host = User::factory()->host()->create(); $group->addVolunteer($host); @@ -69,46 +86,28 @@ public function group_image_upload_allowed_when_uploads_enabled(): void $_SERVER['DOCUMENT_ROOT'] = getcwd(); \FixometerFile::$uploadTesting = true; - file_put_contents('/tmp/UT_enabled.jpg', file_get_contents(public_path() . '/images/community.jpg')); + file_put_contents('/tmp/' . $filename, file_get_contents(public_path() . '/images/community.jpg')); $_FILES = [ 'file' => [ 'error' => '0', - 'name' => 'UT_enabled.jpg', + 'name' => $filename, 'size' => 123, - 'tmp_name' => ['/tmp/UT_enabled.jpg'], + 'tmp_name' => ['/tmp/' . $filename], 'type' => 'image/jpg', ], ]; - $response = $this->json('POST', '/group/image-upload/' . $group->idgroups, []); - $response->assertOk(); - $this->assertEquals('success - image uploaded', $response->getContent()); - } - - /** @test */ - public function network_logo_upload_rejected_when_uploads_disabled(): void - { - config(['restarters.features.image_upload' => false]); - Storage::fake('public_uploads'); - - $network = Network::factory()->create(); - $admin = User::factory()->administrator()->create(); - $this->actingAs($admin); - - $response = $this->put(route('networks.update', $network), [ - 'network_logo' => UploadedFile::fake()->image('logo.png'), - ]); - - $response->assertRedirect(route('networks.edit', $network)); - $response->assertSessionHas('warning'); - $this->assertNull($network->fresh()->logo); + return $this->json('POST', '/group/image-upload/' . $group->idgroups, []); } - /** @test */ - public function network_logo_upload_allowed_when_uploads_enabled(): void + /** + * Upload a network logo as an administrator. + * + * @return array{0: Network, 1: TestResponse} + */ + private function putNetworkLogo(): array { - config(['restarters.features.image_upload' => true]); Storage::fake('public_uploads'); $network = Network::factory()->create(); @@ -119,9 +118,6 @@ public function network_logo_upload_allowed_when_uploads_enabled(): void 'network_logo' => UploadedFile::fake()->image('logo.png'), ]); - $response->assertRedirect(route('networks.edit', $network)); - $logo = $network->fresh()->logo; - $this->assertNotNull($logo); - $this->assertStringStartsWith('network_logos/', $logo); + return [$network, $response]; } } From 8cf328e14d77d1cff716927a73b1290e611e1bdb Mon Sep 17 00:00:00 2001 From: edwh Date: Mon, 13 Jul 2026 21:23:29 +0100 Subject: [PATCH 4/7] Add annotated template for the FLY_PREVIEW_SECRETS blob The workflow now strips comment/blank lines before flyctl secrets import, so the filled-in template can be pasted as-is. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- .github/workflows/pr-preview.yml | 5 +++- docs/fly-deployment.md | 3 +- fly.pr-secrets.example.env | 50 ++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+), 2 deletions(-) create mode 100644 fly.pr-secrets.example.env diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 08c172008b..f2de9b7687 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -152,7 +152,10 @@ jobs: else flyctl apps create "$APP" --org "$FLY_ORG" # Shared read-only credentials (Tigris RO, Drive RO, gate password...) - echo "$FLY_PREVIEW_SECRETS" | flyctl secrets import --app "$APP" --stage + # Strip comments/blank lines so the fly.pr-secrets.example.env + # template can be filled in and pasted as-is. + echo "$FLY_PREVIEW_SECRETS" | grep -vE '^\s*(#|$)' \ + | flyctl secrets import --app "$APP" --stage # Fresh per-app key - previews must never hold the production APP_KEY. flyctl secrets set "APP_KEY=base64:$(openssl rand -base64 32)" \ --app "$APP" --stage diff --git a/docs/fly-deployment.md b/docs/fly-deployment.md index 8afecd5b65..14e4f4363e 100644 --- a/docs/fly-deployment.md +++ b/docs/fly-deployment.md @@ -229,7 +229,8 @@ Previews hold **no production-capable credentials**: `develop` only; holds `FLY_ORG_TOKEN`. 3. **Repository variable** `FLY_ORG` = the Fly organisation slug. 4. **`FLY_PREVIEW_SECRETS`** (env-file format, one `KEY=value` per line) in - the `preview` environment: + the `preview` environment — start from the annotated template + `fly.pr-secrets.example.env` in the repo root: - `BASIC_AUTH_PASSWORD` — a fresh value (the old committed one is public in git history; rotate, don't reuse) - `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_BUCKET` — a diff --git a/fly.pr-secrets.example.env b/fly.pr-secrets.example.env new file mode 100644 index 0000000000..7d6b8aa301 --- /dev/null +++ b/fly.pr-secrets.example.env @@ -0,0 +1,50 @@ +# Template for the FLY_PREVIEW_SECRETS GitHub environment secret used by +# .github/workflows/pr-preview.yml (see docs/fly-deployment.md -> PR Previews). +# +# Usage: +# 1. Copy this file OUTSIDE the repo: cp fly.pr-secrets.example.env ~/preview-secrets.env +# 2. Fill in every <...> value (comments may stay - the workflow strips them). +# 3. gh secret set FLY_PREVIEW_SECRETS --env preview < ~/preview-secrets.env +# 4. shred -u ~/preview-secrets.env +# +# Format: KEY=value, one per line, no quotes, no spaces around '='. +# The service-account JSON must be flattened to a single line (jq -c). +# +# NEVER add production-only keys here (MAIL_*, DISCOURSE_*, WIKI_*, WP_*, +# DRIP_*, analytics) and NEVER the production APP_KEY - previews generate +# their own APP_KEY at first deploy. + +# Site cookie-gate password. Pick a FRESH value: the old preview password is +# permanently visible in this public repo's git history. +BASIC_AUTH_PASSWORD= + +# Tigris (shared image bucket) - create a READ-ONLY access key: +# fly storage dashboard -> Access Keys -> new key, Read-only +# Read-only is the real enforcement behind FEATURE__IMAGE_UPLOAD=false. +# Bucket name: fly ssh console -a restarters -C "printenv AWS_BUCKET" +AWS_ACCESS_KEY_ID= +AWS_SECRET_ACCESS_KEY= +AWS_BUCKET= + +# Google Drive backup restore - a READ-ONLY service account, NOT the +# production backup SA (which can delete backups): +# Google Cloud console, project fixometer-1526501244792 -> Service Accounts +# -> create (e.g. rclone-restarters-restore) -> Keys -> add JSON key. +# Then share the backups folder with the SA's email as VIEWER. +# Folder/team-drive IDs below are the same as production's (already public +# in docs/fly-deployment.md). +GDRIVE_BACKUP_FOLDER_ID=1jk-cibm1W4EewWO1GB_SfNF3hxnaKblf +RCLONE_CONFIG_GDRIVE_TYPE=drive +RCLONE_CONFIG_GDRIVE_SCOPE=drive.readonly +RCLONE_CONFIG_GDRIVE_TEAM_DRIVE=0AGkqkEd84IsuUk9PVA +# Single line! Flatten with: jq -c . < key.json +RCLONE_CONFIG_GDRIVE_SERVICE_ACCOUNT_CREDENTIALS= + +# Copy from production: +# fly ssh console -a restarters -C "printenv MAPBOX_TOKEN" +# fly ssh console -a restarters -C "printenv GOOGLE_API_CONSOLE_KEY" +MAPBOX_TOKEN= +GOOGLE_API_CONSOLE_KEY= + +# Optional: uncomment to report preview errors to Sentry (tagged environment=preview). +# SENTRY_LARAVEL_DSN= From c347c16e6c594d864cd4e6fe625e883e2102655a Mon Sep 17 00:00:00 2001 From: edwh Date: Mon, 13 Jul 2026 21:35:49 +0100 Subject: [PATCH 5/7] Add PREVIEW-ENVIRONMENTS.md operating guide Records the concrete setup (environments, org, restore identity, label) alongside day-to-day usage, expectations, and the security model. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- PREVIEW-ENVIRONMENTS.md | 89 +++++++++++++++++++++++++++++++++++++++++ docs/fly-deployment.md | 3 ++ 2 files changed, 92 insertions(+) create mode 100644 PREVIEW-ENVIRONMENTS.md diff --git a/PREVIEW-ENVIRONMENTS.md b/PREVIEW-ENVIRONMENTS.md new file mode 100644 index 0000000000..6156fd35a1 --- /dev/null +++ b/PREVIEW-ENVIRONMENTS.md @@ -0,0 +1,89 @@ +# PR Preview Environments + +Label a PR with `preview` and approve the workflow run: you get a +self-contained copy of the site at `https://restarters-pr-.fly.dev`, +running that PR's code (merged with current `develop`) against a fresh copy +of the live database. It costs pennies while idle and destroys itself after +the PR closes. + +Technical detail lives in `docs/fly-deployment.md` → *PR Previews*. This file +is the operating guide plus the record of the concrete setup. + +## Using a preview + +1. Add the `preview` label to the PR (or run the **PR preview** workflow via + *Actions → PR preview → Run workflow* with the PR number). +2. The run pauses at **Waiting for review**. Before approving, glance at the + PR's diff to `.github/workflows/`, `fly.pr.toml`, `Dockerfile.fly` and + `docker/*.sh` — approval runs the PR's own code with the preview + credentials. Then *Review deployments → preview → Approve*. +3. ~10 minutes later the workflow comments on the PR with the URL and the + restore/migration result. Enter the gate password (in the + `FLY_PREVIEW_SECRETS` blob as `BASIC_AUTH_PASSWORD`) to browse. +4. Every push to a labeled PR redeploys (each needs an approval click) with a + fresh restore of the latest hourly backup. +5. Remove the label or close the PR and the sweep destroys the app within a + few hours. + +What to expect: + +- **"Warming up" page**: cold boots (first deploy, redeploys, discarded + snapshots) restore the DB for a few minutes behind an auto-refreshing page. + After idle periods the machine *resumes* instead — near-instant. +- **Data is disposable**: it resets to a newer live copy on every deploy and + may reset at any time. Log in with real production credentials (the DB is + an unscrubbed live copy behind the gate). +- **Emails** go to shared Mailpit (https://restarters-dev-mail.fly.dev), + never real recipients. **Image uploads are disabled** (reads come from the + live bucket). Discourse/wiki/WordPress integrations are off. +- A red **PR-\** banner marks the preview, linking to Mailpit. +- ❌ *migrate failed* in the PR comment means the PR's migrations broke + against real data — usually a signal to rebase; check + `flyctl logs --app restarters-pr-`. + +## Handy commands + +```bash +flyctl logs --app restarters-pr- # live logs +flyctl ssh console --app restarters-pr- # shell on the preview +curl https://restarters-pr-.fly.dev/_preview_status # boot status JSON +flyctl apps destroy restarters-pr- --yes # manual teardown +``` + +The cleanup sweep also runs on demand: *Actions → PR preview → Run workflow* +with the PR number left **empty**. + +## Concrete setup (configured 2026-07-13) + +| Piece | Value | +|---|---| +| Fly organisation (`FLY_ORG` repo variable) | `personal` ("Restart Tech") | +| GitHub environment `preview` | required reviewer: edwh; holds `FLY_ORG_TOKEN`, `FLY_PREVIEW_SECRETS` | +| GitHub environment `preview-cleanup` | no reviewers; deployment branches restricted to `develop`; holds `FLY_ORG_TOKEN` | +| Restore identity | `restarters-preview-restore@restarters-previews.iam.gserviceaccount.com` (GCP project `restarters-previews`, owned by edward@therestartproject.org; Drive API enabled). Needs **Viewer** on the backups folder — it must never be able to delete backups. | +| Tigris | previews use a **read-only** access key for `restarters-uploads` (create via `fly storage dashboard restarters-uploads` → Access Keys) | +| Label | `preview` | + +Secrets template: `fly.pr-secrets.example.env`. To rotate or complete the +blob: + +```bash +# edit ~/preview-secrets.env (or start from the template), then: +gh secret set FLY_PREVIEW_SECRETS --env preview < ~/preview-secrets.env +shred -u ~/preview-secrets.env +``` + +## Security model (short version) + +- Previews run arbitrary PR code holding a full live-DB copy, so they get + **no production-capable credentials**: fresh per-app `APP_KEY`, read-only + Tigris, read-only Drive SA, no mail/Discourse/wiki/WordPress/Drip secrets. +- The `preview` environment's required-reviewer gate is what stands between + a pushed commit and the credentials — hence the pre-approval glance at + infra files. +- The sweep's token is unreachable from PR code: it only exists in + `preview-cleanup`, which only `develop`-ref runs (i.e. the schedule) can + use. Never wire cleanup to `pull_request` events. +- The Fly org token can manage every app in the org. If it leaks, revoke with + `fly tokens list` / `fly tokens revoke` and re-set both environment + secrets. diff --git a/docs/fly-deployment.md b/docs/fly-deployment.md index 14e4f4363e..b54082e972 100644 --- a/docs/fly-deployment.md +++ b/docs/fly-deployment.md @@ -159,6 +159,9 @@ The `production` branch deploy runs without waiting for tests (it is triggered b ## PR Previews +Operating guide and concrete setup record: `PREVIEW-ENVIRONMENTS.md` in the +repo root. Technical design below. + Any PR can be deployed as a self-contained test site at `https://restarters-pr-.fly.dev`, running the PR's code against a copy of the live database. From 90955c3ca23e5369979c7459803e4dac3d854373 Mon Sep 17 00:00:00 2001 From: edwh Date: Mon, 13 Jul 2026 21:39:16 +0100 Subject: [PATCH 6/7] Record interim Tigris credential decision in PREVIEW-ENVIRONMENTS.md Previews currently reuse production's read-write Tigris key (owner- accepted 2026-07-13); the image-upload flag is the only upload block until it is swapped for a read-only key. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- PREVIEW-ENVIRONMENTS.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/PREVIEW-ENVIRONMENTS.md b/PREVIEW-ENVIRONMENTS.md index 6156fd35a1..d59437ec95 100644 --- a/PREVIEW-ENVIRONMENTS.md +++ b/PREVIEW-ENVIRONMENTS.md @@ -61,7 +61,7 @@ with the PR number left **empty**. | GitHub environment `preview` | required reviewer: edwh; holds `FLY_ORG_TOKEN`, `FLY_PREVIEW_SECRETS` | | GitHub environment `preview-cleanup` | no reviewers; deployment branches restricted to `develop`; holds `FLY_ORG_TOKEN` | | Restore identity | `restarters-preview-restore@restarters-previews.iam.gserviceaccount.com` (GCP project `restarters-previews`, owned by edward@therestartproject.org; Drive API enabled). Needs **Viewer** on the backups folder — it must never be able to delete backups. | -| Tigris | previews use a **read-only** access key for `restarters-uploads` (create via `fly storage dashboard restarters-uploads` → Access Keys) | +| Tigris | currently reusing production's **read-write** key (accepted interim, 2026-07-13): the `FEATURE__IMAGE_UPLOAD` flag is the only upload block, so the approval gate must catch malicious PRs. TODO: swap to a read-only key (`fly storage dashboard restarters-uploads` → Access Keys) and update `FLY_PREVIEW_SECRETS`. | | Label | `preview` | Secrets template: `fly.pr-secrets.example.env`. To rotate or complete the @@ -76,8 +76,10 @@ shred -u ~/preview-secrets.env ## Security model (short version) - Previews run arbitrary PR code holding a full live-DB copy, so they get - **no production-capable credentials**: fresh per-app `APP_KEY`, read-only - Tigris, read-only Drive SA, no mail/Discourse/wiki/WordPress/Drip secrets. + **no production-capable credentials** where practical: fresh per-app + `APP_KEY`, read-only Drive SA, no mail/Discourse/wiki/WordPress/Drip + secrets. Exception: the Tigris key is currently production's read-write + one (see table above). - The `preview` environment's required-reviewer gate is what stands between a pushed commit and the credentials — hence the pre-approval glance at infra files. From 05f2436756eb8054c3fe9f8216cb11a4f09d3469 Mon Sep 17 00:00:00 2001 From: edwh Date: Mon, 13 Jul 2026 21:45:17 +0100 Subject: [PATCH 7/7] Previews now use a dedicated read-only Tigris key Closes the interim noted in the previous commit. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon --- PREVIEW-ENVIRONMENTS.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/PREVIEW-ENVIRONMENTS.md b/PREVIEW-ENVIRONMENTS.md index d59437ec95..b1f7167d2c 100644 --- a/PREVIEW-ENVIRONMENTS.md +++ b/PREVIEW-ENVIRONMENTS.md @@ -61,7 +61,7 @@ with the PR number left **empty**. | GitHub environment `preview` | required reviewer: edwh; holds `FLY_ORG_TOKEN`, `FLY_PREVIEW_SECRETS` | | GitHub environment `preview-cleanup` | no reviewers; deployment branches restricted to `develop`; holds `FLY_ORG_TOKEN` | | Restore identity | `restarters-preview-restore@restarters-previews.iam.gserviceaccount.com` (GCP project `restarters-previews`, owned by edward@therestartproject.org; Drive API enabled). Needs **Viewer** on the backups folder — it must never be able to delete backups. | -| Tigris | currently reusing production's **read-write** key (accepted interim, 2026-07-13): the `FEATURE__IMAGE_UPLOAD` flag is the only upload block, so the approval gate must catch malicious PRs. TODO: swap to a read-only key (`fly storage dashboard restarters-uploads` → Access Keys) and update `FLY_PREVIEW_SECRETS`. | +| Tigris | dedicated **read-only** access key for `restarters-uploads` (created 2026-07-13 via the Tigris console) — the real enforcement behind the disabled-uploads flag | | Label | `preview` | Secrets template: `fly.pr-secrets.example.env`. To rotate or complete the @@ -76,10 +76,9 @@ shred -u ~/preview-secrets.env ## Security model (short version) - Previews run arbitrary PR code holding a full live-DB copy, so they get - **no production-capable credentials** where practical: fresh per-app - `APP_KEY`, read-only Drive SA, no mail/Discourse/wiki/WordPress/Drip - secrets. Exception: the Tigris key is currently production's read-write - one (see table above). + **no production-capable credentials**: fresh per-app `APP_KEY`, read-only + Tigris key, read-only Drive SA, no mail/Discourse/wiki/WordPress/Drip + secrets. - The `preview` environment's required-reviewer gate is what stands between a pushed commit and the credentials — hence the pre-approval glance at infra files.