Release server 0.0.2

.github/workflows/ci.yml

@@ -2,7 +2,7 @@ name: CI
 
 on:
   push:
-    branches: [master]
+    branches: [development, master]
   pull_request:
 
 jobs:

.github/workflows/release-checks.yml

@@ -4,7 +4,7 @@ on:
   pull_request:
     paths: ['tools/**', '.github/workflows/**', 'pyproject.toml', 'deploy/**']
   push:
-    branches: [master]
+    branches: [development, master]
     paths: ['tools/**', '.github/workflows/**', 'pyproject.toml', 'deploy/**']
 
 permissions:

AGENTS.md

@@ -15,19 +15,23 @@
 
 ## Coding Conventions
 
+When working in the Papyrus workspace, read `../CODING_STYLE.md` for the
+maintainer's reviewed examples. The rules below also apply in standalone checkouts.
+
+- Use meaningful local and callback names; avoid single-letter operands and redundant qualifiers when a clear role name is available.
+- Prefer clear branches or named predicates to nested conditional expressions and substantial inline validation conditions. Preserve predicate polarity and null handling when extracting helpers.
+- Extract meaningful constants and focused helpers for distracting literals or construction details. Preserve types, scope, return values, and behavior.
+- Keep Ruff's 120-character line length. Use trailing commas for deliberate multiline argument layouts, and give multiline model/schema fields breathing room while keeping related simple fields together.
+- Remove documentation that merely repeats a declaration and decorative section banners. Retain substantive contracts and constraints in docstrings.
 - Keep route handlers thin. They should parse input, enforce dependencies/auth, call a service, translate errors, and return schemas.
 - Put business rules, query orchestration, and transaction-aware logic in `papyrus/services`, not in route modules.
 - Reuse or extend existing schema modules before creating new top-level packages.
 - Follow existing async patterns with `AsyncSession`, explicit return types, and Pydantic schemas.
-- Use vertical spacing to separate logical steps inside functions. Keep related statements together, but add a blank line when moving between setup, validation, branching, and side effects.
-- Prefer readable spacing over dense blocks. Short guard clauses, temporary assignments, and context-manager branches should usually be visually separated the way `papyrus/services/email.py` and `papyrus/services/auth/` are structured.
+- Keep consecutive single-line statements together. A change in purpose does not require a blank line, including query execution, result extraction, assignments, persistence calls, and returns.
 - Treat any multiline statement or block as a visual boundary inside a function. If one adjacent statement is multiline and both sides are real code, separate the two statements with a blank line.
 - Apply that rule to multiline conditionals, loops, `with` blocks, `try` blocks, multiline calls, multiline literals, and multiline return values. Treat `if` / `elif` / `else` as one block.
 - Do not add a blank line just because a docstring appears above the first statement in a function. The multiline-block rule applies between code statements, not between a docstring and the first line of code.
-- Add a blank line between query execution and result extraction, especially around `session.execute(...)` and subsequent `scalar*()` reads.
-- Add a blank line before persistence and side-effect boundaries such as `session.add(...)`, `send_email(...)`, `commit()`, redirects, and returned result objects when they start a new phase of the function.
-- Keep tightly coupled short sequences together when they are clearly one step. Do not add spacing mechanically when it makes a two-line operation harder to read.
-- Prefer extra spacing around state transitions and persistence boundaries rather than packing setup, branching, and writes into a single block.
+- Do not pad the start or end of a block. Spacing follows blocks and multiline statements, not inferred logical phases or side effects.
 - When adding a new router module, register it in `papyrus/api/routes/__init__.py`.
 - Avoid adding dependencies unless the user explicitly asks for them.
 - Keep changes scoped. Do not refactor unrelated areas as part of a focused fix.

alembic/versions/b5c6d7e8f901_add_reading_activity_and_goals.py

@@ -0,0 +1,34 @@
+"""Add owned, durable goals and append-only activity without changing book data."""
+
+from collections.abc import Sequence
+
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+from alembic import op
+
+revision: str = "b5c6d7e8f901"
+down_revision: str | Sequence[str] | None = "af0fea8d6317"
+branch_labels: str | Sequence[str] | None = None
+depends_on: str | Sequence[str] | None = None
+
+
+def upgrade() -> None:
+    for table in ("reading_goals", "reading_activities", "goal_periods"):
+        op.create_table(
+            table,
+            sa.Column("id", sa.Uuid(), nullable=False),
+            sa.Column("owner_user_id", sa.Uuid(), nullable=False),
+            sa.Column("payload", postgresql.JSONB(), nullable=False),
+            sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
+            sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
+            sa.ForeignKeyConstraint(["owner_user_id"], ["users.user_id"], ondelete="CASCADE"),
+            sa.PrimaryKeyConstraint("id"),
+        )
+        op.create_index(f"ix_{table}_owner_user_id", table, ["owner_user_id"])
+
+
+def downgrade() -> None:
+    for table in ("goal_periods", "reading_activities", "reading_goals"):
+        op.drop_index(f"ix_{table}_owner_user_id", table_name=table)
+        op.drop_table(table)

deploy/Caddyfile

@@ -1,5 +1,14 @@
 :8080 {
-	root * /srv/web
-	try_files {path} /index.html
-	file_server
+	root * /srv/web/current
+	header Cache-Control "no-cache"
+	@metadata path /release.json
+	handle @metadata {
+		file_server
+	}
+	handle {
+		try_files {path} /index.html
+		file_server {
+			hide .artifact-sha256
+		}
+	}
 }

deploy/README.md

@@ -10,6 +10,16 @@ metadata comes from the installed Python package. GitHub releases record the
 image digest; deploy a recorded digest instead of a mutable tag when stronger
 artifact pinning is needed. The runtime uses UID/GID 10001, not root.
 
+## Branch workflow
+
+Feature/fix PRs target the default `development` branch and leave versions
+unchanged. CI checks integration work without publishing a container. When ready,
+prepare the coordinated version bump on `development`, then promote it to
+`master` with a release PR using **Create a merge commit**. The existing version
+gate publishes only from `master`. Bring `master` back into `development` after
+the release, and deploy additive server changes before a client that needs them.
+See the workspace `RELEASING.md` for coordination and merge order.
+
 ## Host and domains
 
 Use a VM with Docker Engine/Compose v2 and Python 3.12+, enough disk for uploaded
@@ -65,7 +75,9 @@ are assembled from them. Do not rotate the JWT private key on ordinary deploys.
 Configure a real SMTP provider with TLS, verified sender, and its credentials;
 Mailpit is for local development. Add Google OAuth credentials if testing Google
 sign-in. Extract the matching client's `web-release` artifact into `web/`, so
-`web/index.html` exists. The mobile app does not require visiting the web app for
+`web/index.html` exists, then run `python3 migrate_web_layout.py` to initialize
+`web/current/index.html`. This preserves the flat files and snapshots the initial
+release. Subsequent client releases activate versioned directories atomically. The mobile app does not require visiting the web app for
 ordinary reading, but registration verification/reset emails use its routes.
 
 If the GHCR package is private, authenticate Docker on the VM with a restricted
@@ -127,3 +139,20 @@ References:
 - [Hetzner Cloud firewalls](https://docs.hetzner.com/cloud/firewalls/overview/)
 - [Docker Compose deployment](https://docs.docker.com/compose/how-tos/production/)
 - [Caddy automatic HTTPS](https://caddyserver.com/docs/automatic-https)
+
+## Updating the web delivery layout
+
+For an existing deployment, run `python3 migrate_web_layout.py` from `deploy/`
+before installing the updated Caddyfile. It leaves the running flat directory
+intact. Copy the new Caddyfile into the existing bind-mounted file (preserve its
+inode), then validate and reload only the web process:
+
+```sh
+docker compose --env-file production.env -f compose.yml exec -T web caddy validate --config /etc/caddy/Caddyfile
+docker compose --env-file production.env -f compose.yml exec -T web caddy reload --config /etc/caddy/Caddyfile
+```
+
+Keep the parent `web` directory mounted at `/srv/web`. Check `/` and `/login` after
+migration. Client delivery credentials and the restricted receiver are documented
+in the client's `docs/RELEASING.md`. This migration needs no database changes and
+must not run the full server deployment script or restart API/PowerSync services.

deploy/bootstrap-powersync.sql

@@ -5,8 +5,8 @@ WHERE NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'powersync_role')
 \gexec
 ALTER ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD :'source_password';
 GRANT USAGE ON SCHEMA public TO powersync_role;
-GRANT SELECT ON TABLE public.books, public.shelves, public.tags, public.notes, public.annotations, public.bookmarks, public.book_shelves, public.book_tags, public.powersync_demo_items TO powersync_role;
+GRANT SELECT ON TABLE public.books, public.shelves, public.tags, public.notes, public.annotations, public.bookmarks, public.book_shelves, public.book_tags, public.reading_goals, public.reading_activities, public.goal_periods, public.powersync_demo_items TO powersync_role;
 ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
 SELECT 'CREATE PUBLICATION powersync' WHERE NOT EXISTS (SELECT 1 FROM pg_publication WHERE pubname = 'powersync')
 \gexec
-ALTER PUBLICATION powersync SET TABLE public.books, public.shelves, public.tags, public.notes, public.annotations, public.bookmarks, public.book_shelves, public.book_tags, public.powersync_demo_items;
+ALTER PUBLICATION powersync SET TABLE public.books, public.shelves, public.tags, public.notes, public.annotations, public.bookmarks, public.book_shelves, public.book_tags, public.reading_goals, public.reading_activities, public.goal_periods, public.powersync_demo_items;

deploy/migrate_web_layout.py

@@ -0,0 +1,56 @@
+"""Snapshot the existing flat web deployment without deleting or modifying its files."""
+
+import argparse
+import os
+import shutil
+import tempfile
+from pathlib import Path
+
+
+def migrate(root: Path) -> None:
+    current = root / "current"
+
+    if current.is_symlink() and (current / "index.html").is_file():
+        return
+
+    if current.exists() or current.is_symlink():
+        raise ValueError("An invalid current entry already exists; inspect it before migrating")
+
+    if not (root / "index.html").is_file():
+        raise ValueError("Extract an initial web release into this directory before migrating")
+
+    sources = [path for path in root.iterdir() if path.name != "releases" and not path.name.startswith(".")]
+
+    if any(path.is_symlink() or any(child.is_symlink() for child in path.rglob("*")) for path in sources):
+        raise ValueError("The initial web deployment must not contain symlinks")
+
+    releases = root / "releases"
+    releases.mkdir(exist_ok=True)
+    snapshot = Path(tempfile.mkdtemp(prefix="initial-", dir=releases))
+
+    try:
+        for source in sources:
+            if source.is_dir():
+                shutil.copytree(source, snapshot / source.name)
+            else:
+                shutil.copy2(source, snapshot / source.name)
+
+        snapshot.chmod(0o755)
+        link = root / ".current-next"
+        link.symlink_to(f"releases/{snapshot.name}")
+        os.replace(link, current)
+    except BaseException:
+        shutil.rmtree(snapshot)
+        raise
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--root", type=Path, default=Path(__file__).resolve().parent / "web")
+    args = parser.parse_args()
+    migrate(args.root)
+    print("Versioned web layout ready; original flat files remain intact")
+
+
+if __name__ == "__main__":
+    main()

deploy/validate_config.py

@@ -48,8 +48,10 @@ def main() -> None:
     for name in ("powersync-private.pem", "powersync-public.pem"):
         if not (directory / "secrets" / name).is_file():
             raise ValueError(f"Missing secrets/{name}; see the deployment runbook")
-    if not (directory / "web/index.html").is_file():
-        raise ValueError("Missing web/index.html; extract the client web-release artifact into deploy/web")
+    if not (directory / "web/current/index.html").is_file():
+        raise ValueError(
+            "Missing web/current/index.html; initialize the versioned web layout in the deployment runbook"
+        )
     print("Deployment configuration validated")
 
 

docs-tracking.md

@@ -0,0 +1,52 @@
+# Reading tracking contract and rollout
+
+Tracking schema version 2 is advertised additively by `GET /v1/sync/settings`.
+Goals/session/statistics REST endpoints now return owned persisted records and
+ledger-derived progress. Existing metric strings remain valid; `reading_days` is
+additive. Goal definition, activity, and period payloads are validated in the same
+transaction/owner lock used by ordinary PowerSync uploads.
+
+`reading_goals`, `reading_activities`, and `goal_periods` store owner-scoped JSONB
+payloads with UUID identities. Activity is immutable: correction/undo appends a
+reversal referencing an owned original. Retries are idempotent. Shelf membership
+and book-title snapshots survive book deletion. Activity/period deletion through
+sync is rejected; deleting a goal retains rule/period history. Account deletion
+still removes owned data.
+
+Progress unions overlapping time intervals and per-book/day content coverage,
+counts distinct confirmed books, and applies creation cutoffs, pauses, captured
+IANA timezone, Monday weeks, and historical rule revisions. Period history holds
+rules rather than mutable counters, allowing delayed corrections to reproject it.
+Shared fixtures in `tests/fixtures/tracking_projections.json` match Dart tests.
+
+## Deployment order
+
+1. Apply Alembic revision `b5c6d7e8f901` (`uv run alembic upgrade head`). This is
+   additive and preserves existing library tables/data.
+2. Add the three tracking tables to the PostgreSQL PowerSync publication and grants.
+   Updated `deploy/bootstrap-powersync.sql` and `scripts/setup_local_powersync.sh`
+   include them. Existing deployments must run the updated bootstrap; creating
+   database tables alone does not update an existing publication.
+3. Deploy the updated PowerSync stream configuration and server capability.
+4. Release the integrated client after verifying the capability and transport.
+
+Older servers are supported by client local staging; unsupported tracking never
+blocks ordinary library uploads. Versions remain unchanged in these development
+PRs. Run server tests only against a separate test database; the fixtures recreate
+tables. The tracking migration test verifies existing books survive and Alembic
+metadata matches the upgraded schema.
+
+## Selected books (schema v2)
+
+A goal with `scope: "book"` may include `book_ids`, a canonical list of selected
+book UUIDs. `scope_id` remains a selected book for legacy payload compatibility.
+Omitting `book_ids` retains the original single-book meaning. Every selected book
+is ownership-checked; changing the selection requires a replacement goal.
+Completion targets cannot exceed the number of distinct selected books.
+
+The client omits the additive field for single-book goals and stages multiple-book
+goals and their period snapshots separately on v1 servers. Ordinary tracking and
+library uploads continue. Staged records promote when v2 is discovered. Deploy
+the v2 server before releasing this client; use the updated client on all devices
+to calculate multiple-book progress. Existing JSONB and SQLite payload columns
+carry the selection; no additional database migration or stream changes are needed.

papyrus/api/deps.py

@@ -114,7 +114,14 @@ class PaginationParams:
     def __init__(
         self,
         page: Annotated[int, Query(ge=1, description="Page number")] = 1,
-        limit: Annotated[int, Query(ge=1, le=100, description="Items per page")] = 20,
+        limit: Annotated[
+            int,
+            Query(
+                ge=1,
+                le=100,
+                description="Items per page",
+            ),
+        ] = 20,
         sort: Annotated[
             str | None,
             Query(description="Sort field with optional - prefix for descending"),