Skip to content

factory r3: round-3 schema and vocabulary (one migration, entities, stores, Actor/refs, views, web types, fixture) - #137

Merged
LittleChimera merged 3 commits into
stagingfrom
round3/r3-03
Oct 27, 2026
Merged

LittleChimera merged 3 commits into
stagingfrom
round3/r3-03

Conversation

@LittleChimera

@LittleChimera LittleChimera commented Oct 27, 2026 •

Copy link
Copy Markdown

Related issue

Closes #

Round-3 work package R3-03 (schema and vocabulary). Spec: ROUND3-SPEC §16, plus the data shapes of §8–§15.

Summary

All round-3 persistence lands in one forward-only migration, so the later round-3 packages add no migrations and never fight over alembic heads. This is the data layer only: nothing calls the new tables yet, so runtime behaviour does not change.

  • Migration fz1a2b3c4d5e (chains after fy1a2b3c4d5e, the single head on staging):
    • 12 new tables: factory_people, factory_chat_messages, factory_chat_reads, factory_proposals, factory_tasks, factory_loops, factory_issues, factory_spans, factory_cost_daily, factory_versions, factory_definitions, factory_feedback. They follow the cockpit conventions: workspace_id-led keys, Uuid16 ids, SMALLINT enums with named CHECKs, compressed JSON, and no foreign keys.
    • New columns:
      • factory_projects: status (default 1), parent_slug, closed_at, def_version, version_id.
      • factory_prs: opened_at, first_ready_at, task_id, version_id.
      • factory_decisions: assignee, claimed_by, claimed_at, thread, subject.
    • ck_factory_decisions_source is dropped and recreated as IN (1..9) through batch_alter_table, so it also works on SQLite. downgrade() reverses everything.
  • Models, codecs, entities: ORM models, enum codecs for every new enum (the decision sources gain proposal=6, loop=7, access_request=8, done_check=9) and frozen entity dataclasses.
  • Store (FactoryStore, with in-memory and SQL implementations that behave the same). Each method's docstring names the package that will use it.
    • CRUD and queries:
      • people by factory and by user;
      • chat pages by thread and seq, thread stats and next seq;
      • read marks that only move forward;
      • proposals: compare-and-set on status, dedupe key, filters;
      • tasks: upsert by (project, key), an atomic attempt counter;
      • loops: upsert by (kind, subject);
      • issues: re-raise by dedupe_key (bumps the count and reopens resolved issues), numbered per class;
      • spans: insert-or-ignore by dedupe_key, open-span and range queries;
      • cost ledger: atomic upsert-increment;
      • versions: deduped by content hash;
      • definitions: immutable versions;
      • feedback.
    • Per-factory number sequences (C-n, T-n, INC-n/F-n, D-n, chat seq, definition versions) are serialized where the backend can: a transaction advisory lock on PostgreSQL, a GET_LOCK named lock held across the write on MySQL, and BEGIN IMMEDIATE on SQLite. Every write replays UNIQUE collisions and lock conflicts (MySQL 1213/1205, PostgreSQL 40001/40P01) after a jittered exponential pause; sequence writes get a larger budget (12), because CockroachDB serializes nothing and relies on the key.
    • Free-text values (proposal/task titles, issue rule/category/target/summary, span name, loop verdict/reason, ledger agent/model, version tag/note) are cut to their VARCHAR bounds in both stores, so an over-long value never raises DataError on PostgreSQL.
    • append_chat with a message_id that already exists returns the stored message in both stores (a replayed append adds nothing).
  • Vocabulary (omnigent/factory.py):
    • labels def_version, version, bundle, mode;
    • ROLE_PLANNER, which is in ROLES but not TOP_LEVEL_ROLES. The planner registers the roster tool only, matching spec §18. The role guard now imports it from here (R3-04 defined it locally with a TODO), and R3-04's planner rules apply;
    • the 8 new decision actions;
    • state, kind and taxonomy tuples;
    • Room thread keys.
    • LABEL_DEFAULT_REPO, LABEL_DEFAULT_BASE, PROJECT_KINDS and the ProjectKind view type come from R3-01 (merged first); this PR reuses them.
  • server/factory/actor.py: Actor (§10.1) plus ref parse/format/find for T-n, C-n, D-n, INC-n, F-n, !n, #slug, p:<slug> and task:<slug>/<key>. A project slugged factory or improvements formats as p:<slug>, so it never reads back as a Room thread.
  • class alias views (ProposalView, IssueView) use populate_by_name and dump by alias by default, so they construct by attribute name and emit class on any pydantic 2.x (validate_by_name / serialize_by_alias alone need 2.11). Raising the floor instead would touch uv.lock, which trips the security gate's OSV scan.
  • Views:
    • schemas.py adds PersonView, ViewerBlock, ChatMessageView, ThreadView, ProposalView, TaskView, LoopView, IssueView, SpanView, NowState and DefinitionView.
    • The snapshot additions are all optional fields: viewer, people_count, presence, counts, def_version, engine_mode, project/PR/member/decision extras and totals.usd_per_h.
    • factoryApi.ts mirrors them. TeamStrip gets the planner label.
  • Fixture snapshot.round3.json, derived from the live run:
    • 7 projects, including base-env (integration);
    • 41 agents and 3 humans (creator, operator, viewer);
    • a pending projects proposal C-4 mirrored as D-33;
    • the db-version task graph, with the implement loop on T-4 exhausted at 3/3 (D-31);
    • INC-13 (agent.repeat) and C-14 (tuning) on trial;
    • definition v7 pinned.
fy1a2b3c4d5e (cockpit) ──► fz1a2b3c4d5e (round 3: +12 tables, +14 columns, source CHECK 1..9)
                              ▲ every later round-3 package reads/writes through FactoryStore

Test Plan

  • uv run --frozen pytest -q tests/db/test_migration_factory_round3_tables.py tests/stores/test_factory_store_round3.py tests/factory/test_factory_vocabulary.py tests/server/factory/test_contract_fixture.py
    • Migration up/down/up. Checks keys, indexes and UNIQUEs, that the ORM matches the migrated schema, that cockpit-era rows survive with status=1, that each CHECK accepts its last code and rejects the next, and that the downgrade drops round-3 decision sources.
    • Store round trips for every table, run against both the SQL store and the in-memory store. Covers dedupe and upsert semantics, plus 8 threads racing on each numbered sequence and on the span, issue and proposal dedupe keys.
    • Codec coverage, the vocabulary matching the codec tables, ref parser table tests, and Actor.
    • The contract test: both fixtures round-trip through the pydantic models, and the round-3 shape and cross-references hold.
  • The same migration and store files also pass on PostgreSQL 16: a local container with OMNIGENT_TEST_DB_URI=postgresql+psycopg://…. The whole tests/db + tests/stores set passes there too (1335 passed), except the Databricks volume store, which needs the databricks extra.
  • pytest tests/server/factory tests/runner tests/examples tests/policies tests/test_factory_gates.py tests/factory tests/integration/test_factory_tree.py: everything passes except 7 runner tests that fail locally for environment reasons. 4 need the databricks module, 1 needs linux_bwrap (Linux only), and 2 compare against the local Claude subscription model catalog. None of these touch factory code.
  • alembic heads returns fz1a2b3c4d5e only.
  • Review fixes, PostgreSQL 16 container: the round-3 and cockpit store suites plus both factory migration tests pass (143). test_definition_numbering_replays_unserialized_racers races 8 auto-numbered put_definition calls with the sequence lock off (the CockroachDB situation); with the old DefinitionVersionExists check it failed 1 run in 3 on PostgreSQL, with the fix it passes every run.
  • test_writes_replay_lock_conflicts pins the replay classification: MySQL 1213/1205 and PostgreSQL 40P01 replay, other operational errors raise at once.
  • Web: npm run type-check, npm run lint and npm run format:check pass; vitest on src/components/factory, src/lib/factory* and src/pages/factory passes (634 tests); storybook build passes.
  • pre-commit run --files <changed>: ruff, the hardcoded-model check, prettier, oxlint and web-tsc all pass. pyrefly shows no errors in the touched files (the existing 24 are missing opentelemetry imports and one launcher override).

Demo

N/A. No visible UI change; the only web edits are types and the planner role label.

Type of change

  • Bug fix
  • Feature
  • UI / frontend change
  • Refactor / chore
  • Docs
  • Test / CI
  • Breaking change

Test coverage

  • Unit tests added / updated
  • Integration tests added / updated
  • E2E tests added / updated
  • Manual verification completed
  • Existing tests cover this change
  • Not applicable

Coverage notes

Manual verification: I ran the migration and store suites against a throwaway local PostgreSQL 16 container, which is the same image as CI's stores-postgres lane.

Not verified locally:

  • CockroachDB and MySQL (no local images); CI's stores-crdb and stores-mysql lanes run the same store tests.
  • alembic upgrade head on a copy of the sandbox database. The Postgres test seeds cockpit-era rows and upgrades them instead.
  • The deployed server starting with this migration, which needs the newTag bump.

Updated existing tests:

  • the two head pins: fy1a2b3c4d5e → fz1a2b3c4d5e;
  • the cockpit CHECK test, whose rejected decision-source code moves from 6 to 10;
  • the stable decision-source codes in test_enum_codecs.py;
  • the factory tool registration matrix, which gains the planner.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the size/XL Pull request size: XL label Oct 27, 2026
LittleChimera and others added 2 commits October 27, 2026 22:00
…tores, Actor/refs, views, web types, fixture)

All round-3 persistence lands in one forward-only migration (fz1a2b3c4d5e) so
later packages add no migrations:
- 12 tables: factory_people, factory_chat_messages, factory_chat_reads,
  factory_proposals, factory_tasks, factory_loops, factory_issues,
  factory_spans, factory_cost_daily, factory_versions, factory_definitions,
  factory_feedback (workspace-led keys, Uuid16 ids, SMALLINT enums with named
  CHECKs, compressed JSON, no foreign keys).
- New columns on factory_projects (status, parent_slug, closed_at,
  def_version, version_id), factory_prs (opened_at, first_ready_at, task_id,
  version_id) and factory_decisions (assignee, claimed_by, claimed_at, thread,
  subject); ck_factory_decisions_source widened to 1..9 via batch_alter_table.

Data layer only, no behaviour change:
- enum codecs for every new enum; decision sources proposal/loop/
  access_request/done_check.
- entities and FactoryStore CRUD + queries (in-memory and SQL) for every
  table, each naming its consumer package; per-factory numbering (C-n, T-n,
  INC-n/F-n, chat seq, definition versions) is serialized with advisory locks
  on PostgreSQL plus UNIQUE-retry; spans insert-or-ignore by dedupe_key; cost
  ledger atomic upsert-increment; versions deduped by content hash.
- omnigent/factory.py vocabulary (labels, ROLE_PLANNER, decision actions,
  states/kinds/taxonomies, thread keys); the planner registers the roster
  tool only.
- server/factory/actor.py: Actor plus ref parsing/formatting for T-n, C-n,
  D-n, INC-n, F-n, !n, #slug, p:<slug>, task:<slug>/<key>.
- schemas.py round-3 views (PersonView, ViewerBlock, ChatMessageView,
  ThreadView, ProposalView, TaskView, LoopView, IssueView, SpanView,
  NowState, DefinitionView) and optional snapshot additions; matching
  factoryApi.ts types.
- snapshot.round3.json fixture (7 projects incl. base-env, 41 agents, 3
  humans, C-4 pending, db-version task graph, implement 3/3 exhausted,
  INC-13, C-14 on trial, def_version 7) validated by the contract test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- put_definition raises DefinitionVersionExists only for an explicit
  version; an auto-numbered put that loses its number replays.
- _write replays UNIQUE collisions and lock conflicts (MySQL 1213/1205,
  PostgreSQL 40001/40P01) with jittered exponential backoff; numbered
  sequences get a larger budget and are serialized on MySQL with a
  GET_LOCK named lock held across the write (advisory lock on Postgres).
- Both stores clip free-text values to their VARCHAR bounds.
- append_chat with an existing message_id returns the stored message in
  both stores.
- The role guard imports ROLE_PLANNER from the vocabulary (R3-04 TODO).
- project_ref formats projects named like a Room thread as p:<slug>.
- pydantic floor raised to 2.11 (validate_by_name / serialize_by_alias).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raising the floor touched uv.lock, which trips the OSV scan in the
security gate. _ClassView now uses populate_by_name and dumps by alias
by default, which hold on any pydantic 2.x.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LittleChimera
LittleChimera merged commit ed1c611 into staging Oct 27, 2026
54 of 67 checks passed
@LittleChimera
LittleChimera deleted the round3/r3-03 branch October 27, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Pull request size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant