diff --git a/docs/EXTERNAL_PATTERN_SCOUT.de.md b/docs/EXTERNAL_PATTERN_SCOUT.de.md index f592195..f507b90 100644 --- a/docs/EXTERNAL_PATTERN_SCOUT.de.md +++ b/docs/EXTERNAL_PATTERN_SCOUT.de.md @@ -23,6 +23,104 @@ Ein begrenzter Scout ist sinnvoll, wenn mindestens eines davon gilt: Normale Produktarbeit darf nicht blockieren, nur weil kein aktueller Scout existiert. +## Suchbreite + +Gesucht wird nach dem **Problem**, nicht nur nach Projekten, die wie Loop42 aussehen. + +Mit den wörtlichen Begriffen aus Nutzerziel oder Projekt starten und dann bewusst +auf Synonyme, Mechaniken und benachbarte Fachgebiete erweitern. Ein brauchbarer +Kandidat muss sich nicht selbst als KI-Agent, Loop oder Orchestrator bezeichnen, +wenn er denselben Fehlermodus löst. + +Sinnvolle Suchfamilien sind unter anderem: + +- KI / Agenten / Prompts / Context Engineering / Memory / Evals / Grader; +- Durable Workflows / State Machines / Checkpoints / Replay / Reconciliation; +- Distributed Systems / Idempotency / Optimistic Concurrency / Leases / Fencing / + Event Sourcing / Saga / Outbox; +- Queues / Worker / Supervision / Retry / Circuit Breaker / Dead-Letter Handling / + Backpressure / begrenzte Parallelität; +- inkrementelle Buildsysteme / Dependency Graphs / Invalidierung / + content-addressed State / Reproduzierbarkeit / Caching; +- Policy / Provenance / Attestations / Audit / Autorisierung; +- Fault Injection / Chaos Engineering / Model Checking / Property-based Testing; +- relevante technische Regelsysteme, wenn ihr Problem zum Consumer-Fall passt. + +Ein Scout soll relevante Familien stichprobenartig mischen, statt nur kleine +Varianten derselben Produktkategorie zu suchen. Keyword-Erweiterung ist eine +Discovery-Heuristik und keine Erlaubnis, einen größeren Backlog zu erzeugen. + +Ein ähnliches Produkt ist weder Voraussetzung noch automatisch ein guter Fund. +Bevorzugt wird die kleinste übertragbare Mechanik, die das aktuelle Problem löst. + +## Perspektiv-Probe + +Wenn Unsicherheit, wiederholtes Scheitern oder eine wesentliche Designentscheidung +es rechtfertigt, DARF der Scout das Problem in einem begrenzten Durchlauf neu +betrachten, bevor Kandidaten ausgewählt werden. Nur Linsen verwenden, die Suche +oder Pre-Mortem tatsächlich verändern können: + +- **Danach-Linse** — wenn der aktuelle Plan funktioniert, was wird zum nächsten + Engpass, zur ungültigen Annahme oder zum Bedienaufwand? +- **Umkehr-Linse** — welche zentrale Annahme würde den aktuellen Problemrahmen + falsch machen, und zeigt Entfernen oder Umkehren einer Abhängigkeit einen + einfacheren Weg? +- **Außen-Linse** — wie würde ein Operator, Maintainer, Safety-Reviewer oder ein + benachbartes Fachgebiet dasselbe Problem beschreiben? + +Die Probe liefert höchstens ein oder zwei konkrete alternative Hypothesen oder +Suchrichtungen. Sie ist keine Brainstorming-Pflicht und MUSS entfallen, wenn sie +keine eigene Entscheidung, kein neues Evidence-Ziel und keine neue +Fehlerhypothese erzeugt. + + +## Lückenerkennung und Wiederverwendungsprüfung + +Vor größerer neuer Infrastruktur, einer wichtigen Workflow-Änderung oder +wiederholtem Scheitern **eine begrenzte Lücken- und Reuse-Probe in RECONCILE / +TARGET** durchführen. Sie ist Teil des vorhandenen Matrixloop und der +/truth-/gaps-Prüfungen, **keine zusätzliche Loop-Stufe**, kein Agent, Scheduler, +Zustandsspeicher, Recht oder neuer Wahrheitsanspruch. + +1. **Zuerst abgleichen.** Vereinbartes Ziel, Abnahmekriterien, aktuelle Quelle + und Revision, vorhandene Implementierung, offene Issues/PRs/Branches, + frühere Recherche und Fähigkeiten bereits vorhandener Werkzeuge festhalten. + Designziele, Drafts und Tests auf alten Revisionen sind keine integrierten + aktuellen Funktionen. +2. **Kleinen Umfang abstecken.** Den betroffenen durchgängigen Arbeitsweg und + Fehlermodi stichprobenartig prüfen statt nur die erstgenannte Bibliothek/API. + Bei Bedarf Bedienweg, Sicherheit, Recovery sowie Dependency-/License-Folgen + einbeziehen; nicht bei jedem Aufruf das gesamte Produkt auditieren. +3. **Suchabdeckung ehrlich kennzeichnen.** Je relevantem Ausschnitt zwischen + `CHECKED_NO_DELTA` (auf benannter Grundlage geprüft, kein nützlicher Gewinn), + `USEFUL_CANDIDATE` (belegte prüfbare Alternative), `NOT_SEARCHED` + (Recherche-Lücke) und `BLOCKED_UNKNOWN` (Quelle, Fähigkeit oder Beleg + fehlt) unterscheiden. **NOT_SEARCHED ist nicht NO_USEFUL_DELTA.** Fehlende + Quellen sind kein Negativbefund; Grenze nennen und abhängige Behauptung stoppen. +4. **Wiederverwenden vor Neuerfinden.** Vorhandene eigene Implementierung, + offene Arbeit, unterstützte Funktionen installierter Werkzeuge und neue + Plugin-/SDK-Möglichkeiten mit kleinen externen Mustern vergleichen. Auch + benachbarte Fachgebiete und reale Bedienerprobleme prüfen, wenn sie eine + Entscheidung verändern können. `REUSE_EXISTING` oder `SMALL_ADAPTER` + einem neuen Framework vorziehen. +5. **Priorisieren und stoppen.** Wenige wesentliche blinde Flecken nach + Auswirkung, Safety/Korrektheit, gespartem Aufwand, Kosten und Unsicherheit + ordnen. Kleinsten reproduzierbaren Gegenfall oder Kompatibilitätstest + auswählen oder einen begrenzten `NEEDS_RESEARCH`-Folgeschritt nennen. + Kandidaten- und Zeit-/Run-Budget beachten; keine Aufgaben nur für eine + vollständige Matrix erzeugen. + +Ergebnis ist ein kleines **Abdeckungs-Delta**, kein Vollständigkeitsversprechen: +geprüfter Umfang und Revision, wichtigste bislang übersehene Suchrichtung, +Quellen/Aktualität, bereits vorhandene eigene Stärke, kleinste sinnvolle +Fremdlösung, Einführungs-/Prüfaufwand, offene Unsicherheiten und nächster Test +oder `NO_USEFUL_DELTA`. Gültige frühere Prüfergebnisse wiederverwenden, solange +ihre Abhängigkeiten unverändert sind. Design, offene PR, Benchmark-Behauptung +oder ungetestete Idee nie stillschweigend zu integrierter Funktion erklären. +Produktbezogene Ergebnisse bleiben in der autoritativen Consumer-Quelle; +der Scout schlägt nur Muster vor. Umsetzung und geschützte Entscheidungen +behalten ihre bisherigen Freigabegrenzen. + ## Scout-Contract Ein Scout braucht: @@ -83,3 +181,7 @@ Nicht: Loop42 besitzt die generische Scouting-Methode. Consumer-Projekte entscheiden selbst, ob ein gefundenes Muster für ihre Domäne relevant ist. CORA kann zum Beispiel ein generisches Checkpoint- oder Authority-Muster übernehmen, ohne Produktsemantik, Druckerannahmen oder Safety-Claims eines Fremdprojekts zu importieren. + +## Ausführbarer, optionaler Preflight + +Der lesende Check `python -m tools.checks.gap_reuse_preflight` ist in [Gap & Reuse Preflight](GAP_REUSE_PREFLIGHT.de.md) beschrieben. Ein Host muss ihn innerhalb von RECONCILE / TARGET ausdrücklich aufrufen; er führt keine eigene Websuche aus und läuft nicht bei jeder Codeänderung. Fehlende PR-, Tool- und externe Recherchedaten bleiben `NOT_SEARCHED`, bis ein autorisierter Adapter sie liefert. Die CLI ist noch kein autonomer vollständiger Scout und keine Freigabe. diff --git a/docs/EXTERNAL_PATTERN_SCOUT.md b/docs/EXTERNAL_PATTERN_SCOUT.md index 8c8f9e3..9eb9535 100644 --- a/docs/EXTERNAL_PATTERN_SCOUT.md +++ b/docs/EXTERNAL_PATTERN_SCOUT.md @@ -23,6 +23,98 @@ Run a bounded scout when at least one of these is true: Do not block ordinary product work merely because no recent scout exists. +## Search breadth + +Search for the **problem**, not only for projects that look like Loop42. + +Start with the user's or project's literal terms, then deliberately expand into +synonyms, mechanisms and adjacent disciplines. A useful candidate does not need +to describe itself as an AI agent, loop or orchestrator if it solves the same +failure mode. + +Useful search families include: + +- AI / agents / prompts / context engineering / memory / evals / graders; +- durable workflows / state machines / checkpoints / replay / reconciliation; +- distributed systems / idempotency / optimistic concurrency / leases / fencing / + event sourcing / saga / outbox; +- queues / workers / supervision / retry / circuit breaker / dead-letter handling / + backpressure / bounded concurrency; +- incremental build systems / dependency graphs / invalidation / content-addressed + state / reproducibility / caching; +- policy / provenance / attestations / audit / authorization; +- fault injection / chaos engineering / model checking / property-based testing; +- relevant domain control systems when the consumer problem maps to them. + +A scout should sample across relevant families instead of repeatedly searching +minor variations of one product category. Keyword expansion is a discovery +heuristic, not permission to create a larger backlog. + +A similar product is neither required nor sufficient. Prefer the smallest +transferable mechanism that addresses the current problem. + +## Perspective probe + +When uncertainty, repeated failure, or a substantial design choice justifies it, +the scout MAY spend one bounded pass reframing the problem before selecting +candidates. Use only the lenses that can change the search or pre-mortem: + +- **downstream lens** — if the current plan succeeds, what becomes the next + bottleneck, invalidated assumption, or operator burden? +- **inversion lens** — which central assumption would make the current framing + wrong, and can removing or reversing a dependency expose a simpler route? +- **outside lens** — how would an operator, maintainer, safety reviewer, or + adjacent discipline describe the same problem? + +The probe produces at most one or two concrete alternate hypotheses or search +directions. It is not a brainstorming quota and MUST be skipped when it adds no +distinct decision, evidence target, or failure hypothesis. + + +## Gap detection and reuse gate + +Before substantial new infrastructure, an important workflow change, or a repeated +failure, run **one bounded gap-and-reuse probe inside RECONCILE / TARGET**. +This is part of the existing Matrixloop and /truth-/gaps workflow, **not a new +loop stage**, agent, scheduler, database, permission or source of truth. + +1. **Reconcile first.** State the agreed target, acceptance criteria, live + source/revision, current implementation, existing work (issues, PRs, branches), + previous research and relevant capabilities already provided by installed tools. + Design intentions, drafts and passing tests on older revisions are not current + integrated behavior. +2. **Map a small scope.** Sample the affected end-to-end workflow and failure + modes rather than searching only the first named library or API. Include the + operator-facing path, security, recovery and dependency/license impact where + relevant; do not audit every product area on every invocation. +3. **Label search coverage honestly.** For each relevant slice distinguish + `CHECKED_NO_DELTA` (searched on the stated basis; no useful improvement), + `USEFUL_CANDIDATE` (grounded alternative worth evaluation), + `NOT_SEARCHED` (coverage gap), and `BLOCKED_UNKNOWN` (required source, + capability or evidence unavailable). **NOT_SEARCHED is not NO_USEFUL_DELTA.** + A missing source is not a negative finding; record the limit and stop that + dependent claim. +4. **Check reuse before invention.** Compare the current owned implementation, + still-open work, supported existing-tool features and newly released plugin/ + SDK capabilities with small external patterns. Also sample an adjacent + discipline or real user/operator failure if it may change the decision. + Prefer `REUSE_EXISTING` or `SMALL_ADAPTER` over a replacement framework. +5. **Prioritize and stop.** Rank at most a few material blind spots by impact, + safety/correctness, likely saved effort, cost and uncertainty. Select the + smallest reproducible counterexample or compatibility test, or record a + bounded `NEEDS_RESEARCH` follow-up. Respect the scout's candidate and + time/run budgets; do not create backlog merely to fill a coverage matrix. + +The output is a small **coverage delta**, not a global completeness claim: +checked scope and revision; highest-impact missed search angle; evidence/source +and freshness; what already works internally; best reusable alternative; +adoption/verification cost; explicit unknowns; recommended next test or +`NO_USEFUL_DELTA`. Reuse previous valid coverage when its dependencies have +not changed. Do not silently promote a design, open PR, benchmark claim or +unverified idea to integrated capability. Consumer-specific findings stay in +the consumer's authoritative records. The scout only proposes patterns; +implementation and any operator-owned action retain their normal gates. + ## Scout contract A scout must be bounded by: @@ -83,3 +175,7 @@ Do not: Loop42 owns the generic scouting method. Consumer projects decide whether a discovered pattern is relevant to their own domain. CORA, for example, may borrow a generic checkpoint or authority pattern without importing another project's product semantics, printer assumptions, or safety claims. + +## Executable opt-in preflight + +A bounded, read-only local check is now available as `python -m tools.checks.gap_reuse_preflight` (see [Gap & Reuse Preflight](GAP_REUSE_PREFLIGHT.md)). A host must explicitly invoke it inside RECONCILE / TARGET; it does not search the web or auto-run for every change. Unavailable GitHub/open-work, installed-tool and external-research evidence stays `NOT_SEARCHED` until an authorized adapter supplies it. This CLI is not a complete autonomous scout or an adoption approval. diff --git a/docs/GAP_REUSE_PREFLIGHT.de.md b/docs/GAP_REUSE_PREFLIGHT.de.md new file mode 100644 index 0000000..84ecd32 --- /dev/null +++ b/docs/GAP_REUSE_PREFLIGHT.de.md @@ -0,0 +1,35 @@ +# Gap & Reuse Preflight + +Status: **optionales, rein lesendes CLI-Werkzeug**, kein autonomer Scout, Slash-Befehl, Hintergrunddienst, zusätzliche Matrixloop-Stufe oder Freigabesystem. + +Aufruf im vorhandenen Git-Checkout während **RECONCILE / TARGET**: + +```bash +python -m tools.checks.gap_reuse_preflight --repo . --target "checkpoint recovery" --trigger architecture --expected-head "$(git rev-parse HEAD)" +``` + +Weitere Auslöser: `new_infrastructure`, `repeated_failure`, `dependency_decision`, `scheduled`. `routine` überspringt den Check bewusst. Ein Host muss Auslöser und Zeitpunkt selbst bestimmen; **kein** automatischer Chat-Dispatcher ist vorhanden. + +Die CLI nutzt Loop42s vorhandene Repository-Map und liefert höchstens drei mögliche Wiederverwendungskandidaten. Dateinamen- und Symboltreffer beweisen **keine** funktionierende Lösung. Eigener Code wird vor offenen Arbeiten, installierten Tools und externen Mustern betrachtet. + +## Optionaler Live-Adapter für offene Arbeit + +Bei einer größeren Abhängigkeits- oder Capability-Entscheidung kann derselbe RECONCILE-/TARGET-Aufruf offene PRs und Issues direkt lesen: + +```bash +python -m tools.checks.gap_reuse_preflight --repo . --target "checkpoint recovery" --trigger dependency_decision --expected-head "$(git rev-parse HEAD)" --github-repository example/project +``` + +`--github-repository owner/name` wählt die Quelle auf github.com ausdrücklich aus. Sie wird nicht aus Remotes abgeleitet; der Host muss die passende Quelle wählen. Nach der anfänglichen HEAD-/Root-Prüfung sammelt der Adapter nur bei Nicht-`routine`-Auslösern Belege. Anschließend wird HEAD erneut geprüft. Es wird kein Chat-/Host-Hook registriert und keine Hintergrundausführung aktiviert. + +Genau **ein GET** auf GitHubs [Repository-Issue-Endpunkt](https://docs.github.com/en/rest/issues/issues#list-repository-issues) liest auch PRs: höchstens 100 offene Einträge und 512 KiB, mit 10 Sekunden Socket-Timeout. Keine Wiederholung, Pagination oder Weiterleitung. Pagination-Header, fehlender Zugriff, Rate Limit, Transportfehler oder ungültige Antworten ergeben `BLOCKED_UNKNOWN`; die eigene Codeprüfung bleibt nutzbar. `routine` erzeugt auch mit gesetzter Option keinen Abruf. + +Abgeglichen werden ausschließlich **Titel und Zielbegriffe**. Höchstens acht Metadatenhinweise gehen in die bestehende Auswahl mit maximal drei Kandidaten ein. Die URLs müssen zum gewählten Repository passen. Inhalte, Kommentare, Patches oder Anweisungen werden nicht interpretiert oder ausgeführt. `CHECKED_NO_DELTA` bedeutet nur: kein Begriffstreffer in der vollständig gelieferten Titelliste. Titel bleiben untrusted Daten; PR-Revision, Implementierung und Eignung müssen separat geprüft werden. Der Zeitstempel dokumentiert die Beobachtung; die lokale Revision bindet sie an den Aufruf und friert den entfernten Arbeitsstand nicht ein. + +Öffentliche Quellen funktionieren anonym. Ein vom Host bereits bereitgestelltes `GITHUB_TOKEN` wird nur für den ausdrücklich gewählten Abruf verwendet; Zugangsdaten werden weder beschafft noch Berechtigungen geändert. Tokenwerte, rohe API-Antworten und Exception-Texte erscheinen nicht im Ergebnis. Keine Geheimnisse im Zieltext und keine privaten Berichte committen. Vorhandene Repository-Connectoren können weiterhin `--evidence` liefern. Gleichzeitige Live-Abfrage und `open_work`-Belege werden zurückgewiesen, damit kein Beleg still überschrieben wird. + +Autorisierte Adapter können mit `--evidence datei.json` Belege zu `open_work`, `installed_capabilities` und `external_patterns` liefern. Die Felder und ein JSON-Beispiel stehen in der [englischen Referenz](GAP_REUSE_PREFLIGHT.md). Ohne Adapterbeleg ist der Status **NOT_SEARCHED**, nicht „keine Verbesserung gefunden“. Weitere Werte: `CHECKED_NO_DELTA`, `USEFUL_CANDIDATE`, `BLOCKED_UNKNOWN`. Geprüfte Angaben brauchen Quelle, Zeitpunkt und bei projektgebundenen Daten eine passende HEAD-Revision. Veraltete Angaben werden blockiert. + +Das Ergebnis ist revisionsgebundenes JSON; `adoption_authorized` bleibt immer `false`. Keine Codeänderung, Installation, Freigabe oder Consumer-Pin-Aktualisierung. Vorschläge müssen gegen die aktuelle Quelle und die bestehenden Sicherheits- und Prüfgates getestet werden. + +**Für einen vollständig autonom arbeitenden Scout fehlen noch:** Host-Trigger, Tool-Inventar, externe Muster-Recherche und echte Consumer-Akzeptanztests. Der optionale GitHub-Adapter ist eine bedarfsgesteuerte Titelprüfung, keine Forschungsautomatik oder Übernahme-/Mergefreigabe. diff --git a/docs/GAP_REUSE_PREFLIGHT.md b/docs/GAP_REUSE_PREFLIGHT.md new file mode 100644 index 0000000..38872db --- /dev/null +++ b/docs/GAP_REUSE_PREFLIGHT.md @@ -0,0 +1,53 @@ +# Gap & Reuse Preflight + +Status: **opt-in read-only CLI**, not an autonomous scout, slash command, scheduler, new Matrixloop stage, or approval authority. + +Invoke from an existing git checkout during the host's **RECONCILE / TARGET** step for meaningful new architecture, infrastructure, repeated failures, dependency decisions, or scheduled maintenance: + +```bash +python -m tools.checks.gap_reuse_preflight --repo . --target "checkpoint recovery" --trigger architecture --expected-head "$(git rev-parse HEAD)" +``` + +Routine work uses `--trigger routine` and **skips** the scan. Other triggers: `new_infrastructure`, `repeated_failure`, `dependency_decision`, `scheduled`. + +The tool reuses `tools.context.repository_map` to inspect tracked working-tree paths/symbols, bind the report to HEAD and a map fingerprint, and produce at most three *possible* reuse leads. A matching symbol is not proof the implementation works or is suitable. Without an explicitly selected adapter it stays offline. It does not inspect installed tools, search external patterns, or execute suggested changes. + +## Optional live open-work adapter + +For a substantial dependency/capability decision, the host can use the same RECONCILE / TARGET call to collect open PR/issue leads instead of preparing an evidence file: + +```bash +python -m tools.checks.gap_reuse_preflight --repo . --target "checkpoint recovery" --trigger dependency_decision --expected-head "$(git rev-parse HEAD)" --github-repository example/project +``` + +`--github-repository owner/name` explicitly selects the source on github.com; it is not inferred from remotes and is not proof that the selected source is the checkout's origin. The host owns that choice. The adapter runs only for non-`routine` triggers, after the initial HEAD/root check, and the preflight rechecks HEAD afterward. There is no registered host/chat hook or automatic background activation. + +The standard-library adapter makes **one GET** to GitHub's [repository issues endpoint](https://docs.github.com/en/rest/issues/issues#list-repository-issues), which includes PRs. It reads at most 100 open records and 512 KiB with a 10-second socket timeout. No retries, pagination or redirects are followed. A pagination header, denied/rate-limited access, transport failure or invalid response produces `BLOCKED_UNKNOWN` and leaves the owned-code scan usable. Routine calls make no request even if the option is present. + +It matches target keywords against **titles only**, returns at most eight metadata leads to the existing three-candidate selection, and checks that their URLs identify the selected repository. Bodies, comments, patches and retrieved instructions are not interpreted or executed. `CHECKED_NO_DELTA` means no keyword match in the complete returned title list, not absence of useful code or work. Titles are untrusted data; current PR head, implementation and suitability still need independent inspection. A selected repository can change after the timestamp; the local revision binds the observation to this preflight, not the remote work's immutable state. + +Public sources work anonymously. If already supplied by the host, `GITHUB_TOKEN` is used only for this explicitly selected call; the tool neither obtains credentials nor changes permissions. It never returns token values, raw API bodies or exception text. Do not pass secrets in target text or commit private reports. A host with an existing connected repository tool can continue supplying `--evidence` instead; supplying both a live adapter and `open_work` evidence is rejected rather than silently overwriting either observation. + +An authorized host adapter may supply researched evidence with `--evidence evidence.json`: + +```json +{ + "open_work": { + "status": "USEFUL_CANDIDATE", + "source": "authenticated PR inventory", + "checked_at": "2026-10-28T08:20:00+01:00", + "revision": "", + "items": [ + {"name": "existing PR", "reference": "https://github.com/example/project/pull/12", "reason": "possible reuse"} + ] + }, + "installed_capabilities": {"status": "NOT_SEARCHED", "reason": "host inventory unavailable"}, + "external_patterns": {"status": "NOT_SEARCHED", "reason": "no research performed"} +} +``` + +Coverage labels: `CHECKED_NO_DELTA`, `USEFUL_CANDIDATE`, `NOT_SEARCHED`, `BLOCKED_UNKNOWN`. Missing evidence is **NOT_SEARCHED**, never a negative research result. Checked evidence needs a source and offset-aware timestamp; open work/tool evidence must match the exact HEAD or is downgraded to `BLOCKED_UNKNOWN`. Adapter-provided claims are not independently verified by this CLI. + +Priority: existing code → open work → installed capabilities → external patterns. Output is structured JSON with scope, coverage, next action, fingerprint, and `adoption_authorized: false`. No edits, installs, approvals, consumer pin changes or merges are performed. The host must independently inspect and verify the recommendation through existing action-policy/verification gates. + +**Still missing for autonomous operation:** host-side trigger dispatch, installed-tool inventory, bounded external pattern search adapters, actual consumer acceptance tests. The optional live GitHub adapter is an on-demand title probe, not autonomous scouting or a reuse/merge approval. diff --git a/docs/MATRIXLOOP.md b/docs/MATRIXLOOP.md index 6a3d5be..5279d34 100644 --- a/docs/MATRIXLOOP.md +++ b/docs/MATRIXLOOP.md @@ -81,6 +81,16 @@ Before inventing substantial new infrastructure or orchestration, Matrixloop may Scouting is an input to TARGET / PRE-MORTEM, not another loop stage. Discovery does not authorize adoption: relevance, project fit, expected value, provenance/license status and reproduction against the current source of truth still apply. +Before substantial reinvention or when a prior search missed an important +product area, TARGET can use the scout's bounded **gap detection and reuse gate** +after RECONCILE has established live truth. Compare the current problem, +already implemented/ongoing work and supported existing-tool capabilities +before seeking alternatives. Mark areas that were not searched as +`NOT_SEARCHED`, not as `NO_USEFUL_DELTA`; keep unavailable sources +`BLOCKED_UNKNOWN`. Select one useful next test or stop under the existing +budgets. This is **not an extra Matrixloop stage**, a new scheduler, or a +second source of truth. + See `docs/EXTERNAL_PATTERN_SCOUT.md`. @@ -113,3 +123,5 @@ Loop42 owns only the generic mechanics. Candidate meaning, domain requirements, engineering/safety margins, objective definitions, recommendation semantics and all execution authority remain with the consumer. No consumer pin advances by adding this contract. + +The optional executable Gap & Reuse preflight is documented in [Gap & Reuse Preflight](GAP_REUSE_PREFLIGHT.md). It is a read-only RECONCILE/TARGET input, not a new required stage or automatic chat/runtime hook. For substantial dependency/capability decisions, the host can select `--github-repository owner/name` in the same call to collect bounded open PR/issue title leads. Routine work skips collection; incomplete or unavailable coverage remains `BLOCKED_UNKNOWN`. Existing tool inventories can still enter through supplied evidence. Recommendations require independent verification. diff --git a/docs/TRUTH_AND_GAPS.de.md b/docs/TRUTH_AND_GAPS.de.md index b1b0b0c..17aee7c 100644 --- a/docs/TRUTH_AND_GAPS.de.md +++ b/docs/TRUTH_AND_GAPS.de.md @@ -35,6 +35,18 @@ Je wesentlicher Lücke nennen: unerfülltes Kriterium, Beleg oder Unsicherheit, Zuerst Sicherheits-/Korrektheitsblocker, dann Voraussetzungen und den kleinsten nützlichen Fortschritt priorisieren. Verbesserungen außerhalb des vereinbarten Ziels sind optionale Vorschläge. Mit vorhandenen Issues, Branches und nachweislich abgeschlossener Arbeit abgleichen. Erledigte Arbeit nur bei geänderten Abhängigkeiten oder konkretem Gegenbeispiel wieder öffnen. +Bei wesentlicher neuer Infrastruktur oder wiederholtem Scheitern darf `/gaps` +auch eine **Recherche-Abdeckungslücke** benennen: einen wichtigen Produkt-/ +Workflow-Teil, eine vorhandene Werkzeugfähigkeit, offene Projektarbeit oder ein +benachbartes Muster, das bislang nicht untersucht wurde. Den bestehenden +begrenzten [External Pattern Scout](EXTERNAL_PATTERN_SCOUT.de.md) +innerhalb RECONCILE / TARGET verwenden. Geprüfte Bereiche von +`NOT_SEARCHED` und nicht zugänglichen `BLOCKED_UNKNOWN`-Quellen +unterscheiden; beides beweist nicht, dass keine sinnvolle Alternative existiert. +Vor neuer Mechanik `REUSE_EXISTING` und `SMALL_ADAPTER` prüfen. Diese +optionale Suche macht Verbesserungen außerhalb des vereinbarten Umfangs nicht +zum Abnahmeblocker und erteilt keine Übernahmefreigabe. + Standardausgabe: kurzes Belegurteil, höchstens drei wichtigste Lücken, eine empfohlene nächste Aktion und klare Prüfgrenzen. „Keine Lücke im geprüften Umfang gefunden“ bedeutet keine globale Vollständigkeit. Fehlt notwendige Evidenz, abhängige Arbeit mit benanntem Blocker stoppen; andere bereits erlaubte, unabhängige Arbeit kann weitergehen. ## Einbindung in den Matrixloop @@ -69,3 +81,5 @@ Vertragsbeispiele für die Abnahme, keine ausgeführten Softwaretests. ## Übernahmegrenze Ein Merge dieses Vertrags in Loop42 verändert weder Consumer-Bindungen noch Host-Fähigkeiten oder Automatisierungen. Neue Revisionen werden über den bestehenden Prüf-/Bindungsprozess übernommen. Bis dahin kann eine direkt angeforderte Prüfung im Gespräch diese Methode verwenden, ohne Implementierung in der gebundenen Laufzeit zu behaupten. + +Für einen reproduzierbaren, begrenzten *lokalen* Wiederverwendungsbericht kann ein autorisierter Host die [Gap & Reuse Preflight CLI](GAP_REUSE_PREFLIGHT.de.md) aufrufen. Es entsteht kein nativer Slash-Dispatcher und keine automatische externe Recherche. Nicht geprüfte Bereiche bleiben `NOT_SEARCHED`. diff --git a/docs/TRUTH_AND_GAPS.md b/docs/TRUTH_AND_GAPS.md index 8a949db..7ba62fb 100644 --- a/docs/TRUTH_AND_GAPS.md +++ b/docs/TRUTH_AND_GAPS.md @@ -35,6 +35,16 @@ For each material gap report: unmet criterion, supporting evidence or uncertaint Prioritize safety/correctness blockers, then prerequisites and the smallest useful deliverable. Improvements outside the accepted goal are optional proposals, not blockers. Deduplicate against existing issues, branches and verified completed work. Do not reopen completed work without changed dependencies or a concrete counterexample. +For material new infrastructure or a repeated failure, `/gaps` MAY also +identify a **research-coverage gap**: a relevant product/workflow slice, existing +tool capability, open project work or adjacent pattern not yet investigated. +Use the existing bounded [external pattern scout](EXTERNAL_PATTERN_SCOUT.md) +inside RECONCILE / TARGET. Distinguish checked areas from `NOT_SEARCHED` +and unavailable `BLOCKED_UNKNOWN` sources; neither proves that no useful +alternative exists. Check `REUSE_EXISTING` and `SMALL_ADAPTER` before +proposing new machinery. Such optional discovery does not make an +out-of-scope enhancement an acceptance blocker or authorize adoption. + Default output: a short evidence verdict, up to three highest-impact gaps, one recommended next action, and explicit coverage limits. "No gaps found in the checked scope" is not a claim of global completeness. If required evidence is unavailable, record the blocker and stop that dependent work; other already-authorized independent work may continue. ## Integration with Matrixloop @@ -69,3 +79,5 @@ These are contract acceptance examples, not executed software tests. ## Adoption boundary Merging this contract into Loop42 does not change consumer pins, host capabilities or automations. Consumers adopt a new revision only through their existing verification/binding process. Until then, a directly requested conversational assessment may use this method without claiming that the pinned runtime implements it. + +For a reproducible bounded *local* reuse lead report, an authorized harness may opt into the [Gap & Reuse Preflight CLI](GAP_REUSE_PREFLIGHT.md). It does not install a native slash-command router or perform automatic external research; unobserved areas remain `NOT_SEARCHED`. diff --git a/tests/frozen_scenarios/external_pattern_scout_v1.json b/tests/frozen_scenarios/external_pattern_scout_v1.json index 51e2b8a..f09f421 100644 --- a/tests/frozen_scenarios/external_pattern_scout_v1.json +++ b/tests/frozen_scenarios/external_pattern_scout_v1.json @@ -19,7 +19,19 @@ "license_review": "not_required_for_independent_reimplementation", "adoption_cost": "low" } - ] + ], + "coverage": { + "checked": [ + "existing_checkpoint_and_recovery_contract" + ], + "not_searched": [ + "new_capability_of_an_existing_worker_tool" + ], + "blocked_unknown": [ + "local_worker_environment_not_accessible" + ], + "invalid_conclusion": "nothing useful exists anywhere" + } }, "expected": { "must_not_auto_adopt": true, @@ -28,6 +40,16 @@ "must_preserve_consumer_truth_boundary": true, "direct_code_reuse_requires_license_review": true, "preferred_outcome": "ADOPT_CANDIDATE", - "preferred_candidate": "SmallCheckpointPattern" + "preferred_candidate": "SmallCheckpointPattern", + "must_search_adjacent_domains": true, + "must_not_require_similar_product": true, + "must_probe_downstream_effects": true, + "must_probe_alternate_frame": true, + "must_bound_perspective_probe": true, + "must_check_existing_capabilities_first": true, + "must_distinguish_unsearched_from_no_delta": true, + "must_keep_inaccessible_sources_unknown": true, + "must_prioritize_bounded_blind_spots": true, + "must_preserve_existing_loop_and_consumer_truth": true } } diff --git a/tests/test_external_pattern_scout.py b/tests/test_external_pattern_scout.py index 335b38d..1c75158 100644 --- a/tests/test_external_pattern_scout.py +++ b/tests/test_external_pattern_scout.py @@ -14,6 +14,14 @@ def test_contract_is_bounded_and_non_authoritative(self): self.assertIn("time/run budget", text) self.assertIn("Discovery is not adoption", text) self.assertIn("Direct code reuse requires", text) + self.assertIn("Search for the **problem**", text) + self.assertIn("adjacent disciplines", text) + self.assertIn("A similar product is neither required nor sufficient", text) + self.assertIn("## Perspective probe", text) + self.assertIn("downstream lens", text) + self.assertIn("inversion lens", text) + self.assertIn("outside lens", text) + self.assertIn("not a brainstorming quota", text) def test_consumer_truth_boundary_is_explicit(self): text = (ROOT / "docs/EXTERNAL_PATTERN_SCOUT.md").read_text(encoding="utf-8") @@ -29,8 +37,63 @@ def test_frozen_scenario_guards_adoption(self): self.assertTrue(expected["must_compare_value_to_cost"]) self.assertTrue(expected["must_preserve_consumer_truth_boundary"]) self.assertTrue(expected["direct_code_reuse_requires_license_review"]) + self.assertTrue(expected["must_search_adjacent_domains"]) + self.assertTrue(expected["must_not_require_similar_product"]) + self.assertTrue(expected["must_probe_downstream_effects"]) + self.assertTrue(expected["must_probe_alternate_frame"]) + self.assertTrue(expected["must_bound_perspective_probe"]) self.assertEqual(expected["preferred_candidate"], "SmallCheckpointPattern") + + def test_gap_detection_is_bounded_and_reuse_first(self): + text = (ROOT / "docs/EXTERNAL_PATTERN_SCOUT.md").read_text(encoding="utf-8") + german = (ROOT / "docs/EXTERNAL_PATTERN_SCOUT.de.md").read_text(encoding="utf-8") + self.assertIn("## Gap detection and reuse gate", text) + self.assertIn("RECONCILE / TARGET", text) + self.assertIn("not a new", text) + self.assertIn("CHECKED_NO_DELTA", text) + self.assertIn("USEFUL_CANDIDATE", text) + self.assertIn("NOT_SEARCHED", text) + self.assertIn("BLOCKED_UNKNOWN", text) + self.assertIn("NOT_SEARCHED is not NO_USEFUL_DELTA", text) + self.assertIn("REUSE_EXISTING", text) + self.assertIn("SMALL_ADAPTER", text) + self.assertIn("previous valid coverage", text) + self.assertIn("Consumer-specific findings", text) + self.assertIn("## Lückenerkennung und Wiederverwendungsprüfung", german) + for status in ("CHECKED_NO_DELTA", "USEFUL_CANDIDATE", "NOT_SEARCHED", "BLOCKED_UNKNOWN"): + self.assertIn(status, german) + + def test_unsearched_and_inaccessible_areas_cannot_count_as_no_delta(self): + path = ROOT / "tests/frozen_scenarios/external_pattern_scout_v1.json" + data = json.loads(path.read_text(encoding="utf-8")) + self.assertTrue(data["fixture"]["coverage"]["not_searched"]) + self.assertTrue(data["fixture"]["coverage"]["blocked_unknown"]) + expected = data["expected"] + for key in ( + "must_check_existing_capabilities_first", + "must_distinguish_unsearched_from_no_delta", + "must_keep_inaccessible_sources_unknown", + "must_prioritize_bounded_blind_spots", + "must_preserve_existing_loop_and_consumer_truth", + ): + self.assertTrue(expected[key], key) + + + def test_matrixloop_and_gaps_reuse_the_same_probe(self): + matrixloop = (ROOT / "docs/MATRIXLOOP.md").read_text(encoding="utf-8") + gaps = (ROOT / "docs/TRUTH_AND_GAPS.md").read_text(encoding="utf-8") + gaps_de = (ROOT / "docs/TRUTH_AND_GAPS.de.md").read_text(encoding="utf-8") + self.assertIn("gap detection and reuse gate", matrixloop) + self.assertIn("not an extra Matrixloop stage", matrixloop) + self.assertIn("NOT_SEARCHED", matrixloop) + self.assertIn("research-coverage gap", gaps) + self.assertIn("external pattern scout", gaps) + self.assertIn("REUSE_EXISTING", gaps) + self.assertIn("NOT_SEARCHED", gaps_de) + self.assertIn("BLOCKED_UNKNOWN", gaps_de) + self.assertIn("REUSE_EXISTING", gaps_de) + if __name__ == "__main__": unittest.main() diff --git a/tests/test_gap_reuse_github.py b/tests/test_gap_reuse_github.py new file mode 100644 index 0000000..82c37f4 --- /dev/null +++ b/tests/test_gap_reuse_github.py @@ -0,0 +1,127 @@ +import io +from http.client import IncompleteRead +import json +import unittest +from unittest.mock import patch +from urllib.error import HTTPError, URLError +from urllib.request import Request + +from tools.checks import gap_reuse_github as github + + +class Response(io.BytesIO): + def __init__(self, payload, link=""): + super().__init__(payload) + self.headers = {"Link": link} + + +def item(number=12, title="Checkpoint recovery", pr=False): + kind = "pull" if pr else "issues" + result = {"number": number, "title": title, "state": "open", + "html_url": f"https://github.com/example/project/{kind}/{number}"} + if pr: + result["pull_request"] = {"url": f"https://api.github.com/repos/example/project/pulls/{number}"} + return result + + +class GitHubOpenWorkTests(unittest.TestCase): + def collect(self, values, link="", target="checkpoint", token=None): + response = Response(json.dumps(values).encode(), link) + with patch.object(github, "build_opener") as opener: + opener.return_value.open.return_value = response + result = github.collect_open_work("example/project", target=target, + revision="a" * 40, token=token) + request = opener.return_value.open.call_args.args[0] + self.assertEqual(request.full_url, "https://api.github.com/repos/example/project/issues?state=open&sort=updated&direction=desc&per_page=100&page=1") + self.assertEqual(request.get_method(), "GET") + self.assertEqual(opener.return_value.open.call_args.kwargs["timeout"], 10) + opener.return_value.open.assert_called_once() + return result, request + + def test_live_shape_issues_and_prs_become_untrusted_leads(self): + result, request = self.collect([item(pr=True), item(13), item(14, "unrelated")]) + self.assertEqual(result["status"], "USEFUL_CANDIDATE") + self.assertEqual([i["name"] for i in result["items"]], + ["PR #12 Checkpoint recovery", "Issue #13 Checkpoint recovery"]) + self.assertEqual(result["revision"], "a" * 40) + self.assertIn("title", result["source"]) + self.assertIsNone(request.get_header("Authorization")) + + def test_negative_result_is_only_scoped_title_probe(self): + result, _ = self.collect([item(title="other")]) + self.assertEqual(result["status"], "CHECKED_NO_DELTA") + self.assertEqual(result["items"], []) + self.assertIn("title", result["source"]) + + def test_token_is_sent_only_in_header_and_not_returned(self): + result, request = self.collect([], token="synthetic-token") + self.assertEqual(request.get_header("Authorization"), "Bearer synthetic-token") + self.assertNotIn("synthetic-token", json.dumps(result)) + + def test_pagination_never_claims_complete_negative_or_follows_link(self): + for values in ([], [item()]): + with self.subTest(values=values): + result, _ = self.collect(values, '; rel="next"') + self.assertEqual(result["status"], "BLOCKED_UNKNOWN") + self.assertIn("page", result["reason"]) + self.assertFalse(result.get("items")) + + def test_candidate_limit_does_not_claim_unsearched_items_are_absent(self): + result, _ = self.collect([item(n) for n in range(1, 21)]) + self.assertEqual(result["status"], "USEFUL_CANDIDATE") + self.assertEqual(len(result["items"]), 8) + + def test_http_auth_rate_limit_timeout_and_redirect_are_unknown_without_secret(self): + for error in (HTTPError("https://api.github.com/", 401, "SECRET", {}, None), + HTTPError("https://api.github.com/", 429, "SECRET", {}, None), + HTTPError("https://api.github.com/", 301, "SECRET", {}, None), + URLError("SECRET"), TimeoutError("SECRET"), IncompleteRead(b"SECRET")): + with self.subTest(error=type(error)): + with patch.object(github, "build_opener") as opener: + opener.return_value.open.side_effect = error + result = github.collect_open_work("example/project", target="checkpoint", + revision="a" * 40, token="SECRET") + self.assertEqual(result["status"], "BLOCKED_UNKNOWN") + self.assertNotIn("SECRET", json.dumps(result)) + opener.return_value.open.assert_called_once() + + def test_invalid_oversized_cross_repo_or_closed_payloads_are_unknown(self): + bad = [b'{"message":"denied"}', b'[{"number":1,"number":2}]', + b'[{"number":NaN}]', b'\xff', b'x' * (github.MAX_RESPONSE_BYTES + 1), + b'[' * 20000 + b']' * 20000] + for changes in ({"state": "closed"}, {"html_url": "https://github.com/other/repo/issues/12"}, + {"number": True}, {"title": ""}, {"number": 1.5}): + bad.append(json.dumps([{**item(), **changes}]).encode()) + for payload in bad: + with self.subTest(payload=payload[:60]): + with patch.object(github, "build_opener") as opener: + opener.return_value.open.return_value = Response(payload) + result = github.collect_open_work("example/project", target="checkpoint", revision="a" * 40) + self.assertEqual(result["status"], "BLOCKED_UNKNOWN") + self.assertFalse(result.get("items")) + + def test_invalid_repository_or_unsearchable_target_makes_no_request(self): + for repository, target in (("../secret", "checkpoint"), ("example/project?x", "checkpoint"), + ("example/project", "und the"), ("example/project", "a")): + with self.subTest(repository=repository, target=target): + with patch.object(github, "build_opener") as opener: + with self.assertRaises(ValueError): + github.collect_open_work(repository, target=target, revision="a" * 40) + opener.assert_not_called() + + def test_header_injection_and_invalid_revision_rejected_before_request(self): + for revision, token in (("short", None), ("a" * 40, "token\nInjected: yes")): + with patch.object(github, "build_opener") as opener: + with self.assertRaises(ValueError): + github.collect_open_work("example/project", target="checkpoint", + revision=revision, token=token) + opener.assert_not_called() + + def test_redirect_handler_refuses_credential_forwarding(self): + handler = github.NoRedirect() + self.assertIsNone(handler.redirect_request(Request("https://api.github.com/"), None, 302, + "redirect", {}, "https://untrusted.invalid/")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_gap_reuse_preflight.py b/tests/test_gap_reuse_preflight.py new file mode 100644 index 0000000..75174a2 --- /dev/null +++ b/tests/test_gap_reuse_preflight.py @@ -0,0 +1,157 @@ +from pathlib import Path +import json +import subprocess +import sys +import tempfile +import unittest +from unittest.mock import patch + +from tools.checks.gap_reuse_preflight import preflight, _load_evidence + + +def git(repo, *args): + return subprocess.run(["git", "-C", str(repo), *args], check=True, capture_output=True, text=True).stdout.strip() + + +class GapReusePreflightTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.repo = Path(self.temp.name) + git(self.repo, "init") + git(self.repo, "config", "user.email", "loop42@example.invalid") + git(self.repo, "config", "user.name", "Loop42 Test") + (self.repo / "checkpoint.py").write_text("def recover_checkpoint():\n return True\n", encoding="utf-8") + (self.repo / "README.md").write_text("# Test repository\n", encoding="utf-8") + git(self.repo, "add", ".") + git(self.repo, "commit", "-m", "fixture") + self.head = git(self.repo, "rev-parse", "HEAD") + + def tearDown(self): + self.temp.cleanup() + + def check(self, target="checkpoint recovery", trigger="architecture", evidence=None): + return preflight(self.repo, target=target, trigger=trigger, expected_head=self.head, evidence=evidence) + + def test_real_local_scan_finds_existing_candidate_and_honest_unknowns(self): + result = self.check() + self.assertEqual(result["candidates"][0]["name"], "checkpoint.py") + self.assertEqual(result["next_action"], "VERIFY_REUSE_FIT_WITH_SMALLEST_TEST") + self.assertEqual([x["status"] for x in result["coverage"]][1:], ["NOT_SEARCHED"] * 3) + self.assertFalse(result["adoption_authorized"]) + + def test_no_match_does_not_mean_no_external_solution(self): + result = self.check("unrelated kubernetes") + self.assertEqual(result["next_action"], "RESEARCH_ONLY_HIGHEST_IMPACT_UNKNOWN_IF_NEEDED") + self.assertEqual(result["coverage"][-1]["status"], "NOT_SEARCHED") + + def test_open_work_and_tools_precede_external_candidates(self): + stamp = "2026-10-28T08:20:00+01:00" + def area(name, revision=None): + result = {"status":"USEFUL_CANDIDATE", "source":"fixture://" + name, "checked_at":stamp, + "items":[{"name":name, "reference":"fixture://" + name, "reason":"reusable pattern"}]} + if revision is not None: + result["revision"] = revision + return result + result = self.check("unrelated target", evidence={ + "external_patterns":area("framework"), + "open_work":area("existing_pr", self.head), + "installed_capabilities":area("built_in_tool", self.head), + }) + self.assertEqual([x["name"] for x in result["candidates"]], ["existing_pr", "built_in_tool", "framework"]) + + def test_stale_open_work_never_counted_as_checked(self): + result = self.check("unrelated target", evidence={"open_work": { + "status":"USEFUL_CANDIDATE", "source":"fixture", "checked_at":"2026-10-28T08:20:00Z", + "revision":"0" * 40, "items":[{"name":"old", "reference":"x", "reason":"old"}]}}) + self.assertEqual(result["coverage"][1]["status"], "BLOCKED_UNKNOWN") + self.assertFalse(result["candidates"]) + + def test_checked_no_delta_needs_evidence_and_cannot_have_candidates(self): + with self.assertRaisesRegex(ValueError, "requires source"): + self.check(evidence={"external_patterns": {"status":"CHECKED_NO_DELTA"}}) + with self.assertRaisesRegex(ValueError, "no-delta"): + self.check(evidence={"external_patterns": {"status":"CHECKED_NO_DELTA", "source":"x", "checked_at":"2026-10-28T08:20:00Z", "items":[{"name":"a", "reference":"x", "reason":"r"}]}}) + + def test_stale_head_rejected_and_routine_not_scanned(self): + with self.assertRaisesRegex(ValueError, "live HEAD differs"): + preflight(self.repo, target="x", trigger="architecture", expected_head="0" * 40) + result = self.check(trigger="routine") + self.assertEqual(result["status"], "SKIPPED") + self.assertEqual(result["coverage"], []) + + def test_cli_read_only_and_valid_json(self): + cli = Path(__file__).resolve().parents[1] / "tools/checks/gap_reuse_preflight.py" + invocation = [sys.executable, "-m", "tools.checks.gap_reuse_preflight", + "--repo", str(self.repo), "--target", "checkpoint recovery", + "--trigger", "repeated_failure", "--expected-head", self.head] + run = subprocess.run(invocation, capture_output=True, text=True, cwd=cli.parents[2]) + self.assertEqual(run.returncode, 0, run.stderr) + self.assertEqual(json.loads(run.stdout)["schema"], "loop42.gap-reuse-preflight.v1") + self.assertEqual(git(self.repo, "status", "--porcelain"), "") + + def test_evidence_duplicate_keys_rejected(self): + evidence = self.repo / "evidence.json" + evidence.write_text('{"open_work":{},"open_work":{}}', encoding="utf-8") + with self.assertRaisesRegex(ValueError, "duplicate JSON key"): + _load_evidence(evidence) + + def test_github_cli_routine_skips_even_unavailable_repository(self): + run = subprocess.run([ + sys.executable, "-m", "tools.checks.gap_reuse_preflight", + "--repo", str(self.repo), "--target", "checkpoint recovery", + "--trigger", "routine", "--expected-head", self.head, + "--github-repository", "example/project", + ], capture_output=True, text=True) + self.assertEqual(run.returncode, 0, run.stderr) + self.assertEqual(json.loads(run.stdout)["status"], "SKIPPED") + + def test_github_collection_is_revision_guarded_and_preserves_other_evidence(self): + with patch("tools.checks.gap_reuse_preflight.collect_open_work") as collect: + collect.return_value = { + "status": "USEFUL_CANDIDATE", "source": "GitHub title probe", + "checked_at": "2026-10-28T08:20:00Z", "revision": self.head, + "items": [{"name": "PR #12 checkpoint", "reference": "https://github.com/example/project/pull/12", + "reason": "title match; inspect fit"}], + } + result = preflight(self.repo, target="unrelated target", trigger="architecture", + expected_head=self.head, github_repository="example/project") + self.assertEqual(result["coverage"][1]["status"], "USEFUL_CANDIDATE") + self.assertEqual(result["candidates"][0]["name"], "PR #12 checkpoint") + self.assertEqual(result["coverage"][2]["status"], "NOT_SEARCHED") + self.assertFalse(result["adoption_authorized"]) + collect.reset_mock() + with self.assertRaisesRegex(ValueError, "live HEAD differs"): + preflight(self.repo, target="checkpoint", trigger="architecture", + expected_head="0" * 40, github_repository="example/project") + collect.assert_not_called() + + def test_github_collection_does_not_silently_replace_supplied_open_work(self): + with self.assertRaisesRegex(ValueError, "open_work"): + preflight(self.repo, target="checkpoint", trigger="architecture", + expected_head=self.head, github_repository="example/project", + evidence={"open_work": {"status": "NOT_SEARCHED"}}) + + def test_github_unknown_does_not_block_owned_scan_and_routine_never_collects(self): + with patch("tools.checks.gap_reuse_preflight.collect_open_work") as collect: + collect.return_value = {"status": "BLOCKED_UNKNOWN", "reason": "access unavailable"} + result = preflight(self.repo, target="checkpoint", trigger="architecture", + expected_head=self.head, github_repository="example/project") + self.assertEqual(result["coverage"][1]["status"], "BLOCKED_UNKNOWN") + self.assertEqual(result["candidates"][0]["name"], "checkpoint.py") + collect.reset_mock() + preflight(self.repo, target="checkpoint", trigger="routine", + expected_head=self.head, github_repository="example/project") + collect.assert_not_called() + + def test_head_move_during_github_collection_rejected(self): + def moved(*args, **kwargs): + git(self.repo, "commit", "--allow-empty", "-m", "concurrent change") + return {"status": "NOT_SEARCHED"} + with patch("tools.checks.gap_reuse_preflight.collect_open_work", side_effect=moved): + with self.assertRaisesRegex(ValueError, "HEAD changed"): + preflight(self.repo, target="checkpoint", trigger="architecture", + expected_head=self.head, github_repository="example/project") + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/checks/README.md b/tools/checks/README.md index b8b8f96..03b6201 100644 --- a/tools/checks/README.md +++ b/tools/checks/README.md @@ -4,6 +4,10 @@ Home for generic Loop42 verification, provenance, freshness and policy checks. Product-specific safety/test logic remains in the consumer project. +## Gap & Reuse preflight + +[`gap_reuse_preflight.py`](../../docs/GAP_REUSE_PREFLIGHT.md) supplies a bounded read-only RECONCILE / TARGET input. It reuses the repository map and can collect live open PR/issue title leads with explicit `--github-repository owner/name`. Routine work skips collection. Missing sources stay unknown; recommendations grant no adoption authority. No host hook or scheduler is installed. + ## Consumer profile validation `consumer_profile.py` validates the project-agnostic `loop42.consumer-profile.v1` contract and can fail on a stale exact-revision pin. diff --git a/tools/checks/gap_reuse_github.py b/tools/checks/gap_reuse_github.py new file mode 100644 index 0000000..cbd9b15 --- /dev/null +++ b/tools/checks/gap_reuse_github.py @@ -0,0 +1,101 @@ +"""Bounded GitHub title probe. Returned metadata is untrusted discovery data.""" +from __future__ import annotations + +from datetime import datetime, timezone +from http.client import HTTPException +import json +import re +from typing import Any +from urllib.error import HTTPError +from urllib.request import HTTPRedirectHandler, Request, build_opener + +MAX_RESPONSE_BYTES = 512 * 1024 +MAX_ITEMS = 8 +_REPOSITORY = re.compile(r"[A-Za-z0-9][A-Za-z0-9-]{0,38}/[A-Za-z0-9_][A-Za-z0-9_.-]{0,99}") + + +class NoRedirect(HTTPRedirectHandler): + """Do not follow repository moves or forward credentials to another URL.""" + + def redirect_request(self, req, fp, code, msg, headers, newurl): + return None + + +def _unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate JSON key") + result[key] = value + return result + + +def _reject_constant(value): + raise ValueError("non-standard JSON constant") + + +def collect_open_work(repository: str, *, target: str, revision: str, + token: str | None = None) -> dict[str, Any]: + """One GET, no retry/pagination. Scope: open PR/issue titles, not code fit.""" + if not isinstance(repository, str) or not _REPOSITORY.fullmatch(repository): + raise ValueError("github repository must be owner/name on github.com") + if not isinstance(target, str) or not target.strip() or len(target) > 240: + raise ValueError("target must be 1-240 characters") + if not isinstance(revision, str) or not re.fullmatch(r"[0-9a-f]{40}", revision): + raise ValueError("revision must be a full 40-hex commit") + terms = {term.lower() for term in re.findall(r"[A-Za-z0-9_.-]{3,}", target) + if term.lower() not in {"with", "from", "this", "that", "eine", "einen", "und", "the", "for"}} + if not terms: + raise ValueError("target needs a searchable term of at least three characters") + if token is not None and (not isinstance(token, str) or not token or not token.isascii() + or any(ord(char) < 33 or ord(char) > 126 for char in token)): + raise ValueError("invalid GitHub token format") + + url = (f"https://api.github.com/repos/{repository}/issues" + "?state=open&sort=updated&direction=desc&per_page=100&page=1") + headers = {"Accept": "application/vnd.github+json", "User-Agent": "Loop42-gap-reuse-preflight", + "X-GitHub-Api-Version": "2026-03-10"} + if token: + headers["Authorization"] = f"Bearer {token}" + blocked = {"status": "BLOCKED_UNKNOWN", "reason": "GitHub open-work probe unavailable"} + try: + with build_opener(NoRedirect()).open(Request(url, headers=headers), timeout=10) as response: + raw = response.read(MAX_RESPONSE_BYTES + 1) + if len(raw) > MAX_RESPONSE_BYTES: + return {**blocked, "reason": "GitHub response exceeds 512 KiB limit"} + if response.headers.get("Link", ""): + # No pagination is followed, even when a supplied link points elsewhere. + return {**blocked, "reason": "GitHub page coverage incomplete; pagination not followed"} + values = json.loads(raw.decode("utf-8"), object_pairs_hook=_unique, + parse_constant=_reject_constant) + if not isinstance(values, list) or len(values) > 100: + raise ValueError("invalid issue list") + candidates = [] + seen = set() + for value in values: + if not isinstance(value, dict): + raise ValueError("invalid issue") + number, title = value.get("number"), value.get("title") + if (type(number) is not int or number <= 0 or number in seen + or not isinstance(title, str) or not 0 < len(title.strip()) <= 256 + or value.get("state") != "open"): + raise ValueError("invalid open issue metadata") + seen.add(number) + kind = "pull" if "pull_request" in value else "issues" + reference = f"https://github.com/{repository}/{kind}/{number}" + if not isinstance(value.get("html_url"), str) or value["html_url"].lower() != reference.lower(): + raise ValueError("issue reference differs from selected repository") + if any(term in title.lower() for term in terms): + label = "PR" if kind == "pull" else "Issue" + candidates.append({"name": f"{label} #{number} {title}"[:300], + "reference": reference, + "reason": "title keyword match only; inspect live state, revision and reuse fit"}) + return {"status": "USEFUL_CANDIDATE" if candidates else "CHECKED_NO_DELTA", + "source": f"GitHub {repository}: open PR/issue title probe, one page <=100, <=8 leads", + "checked_at": datetime.now(timezone.utc).isoformat(), "revision": revision, + "items": candidates[:MAX_ITEMS]} + except HTTPError as error: + return {**blocked, "reason": f"GitHub HTTP {error.code}; access/rate/redirect failure, no retry"} + except (OSError, ValueError, HTTPException, RecursionError): + # Exception text and response bodies can contain credentials or private data. + return {**blocked, "reason": "GitHub transport or response invalid; no retry"} diff --git a/tools/checks/gap_reuse_preflight.py b/tools/checks/gap_reuse_preflight.py new file mode 100644 index 0000000..d0a4b32 --- /dev/null +++ b/tools/checks/gap_reuse_preflight.py @@ -0,0 +1,208 @@ +#!/usr/bin/env python3 +"""Bounded read-only gap/reuse preflight for existing Matrixloop RECONCILE/TARGET.""" +from __future__ import annotations + +import argparse +from datetime import datetime +import hashlib +import json +import os +from pathlib import Path +import re +import subprocess +import sys +from typing import Any + +from tools.context.repository_map import repository_map +from tools.checks.gap_reuse_github import collect_open_work + +SCHEMA = "loop42.gap-reuse-preflight.v1" +TRIGGERS = frozenset({"architecture", "new_infrastructure", "repeated_failure", "dependency_decision", "scheduled", "routine"}) +AREAS = ("open_work", "installed_capabilities", "external_patterns") +STATES = frozenset({"CHECKED_NO_DELTA", "USEFUL_CANDIDATE", "NOT_SEARCHED", "BLOCKED_UNKNOWN"}) +MAX_EVIDENCE_BYTES = 64 * 1024 +MAX_ITEMS_PER_AREA = 8 +MAX_CANDIDATES = 3 +_TERMS = re.compile(r"[A-Za-z0-9_.-]{3,}") + + +def _git(repo: Path, *args: str) -> str: + return subprocess.run(["git", "-C", str(repo), *args], check=True, + capture_output=True, encoding="utf-8", timeout=20).stdout.strip() + + +def _strict_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + out: dict[str, Any] = {} + for key, value in pairs: + if key in out: + raise ValueError(f"duplicate JSON key: {key}") + out[key] = value + return out + + +def _reject_constant(value: str) -> None: + raise ValueError(f"invalid JSON constant: {value}") + + +def _load_evidence(path: Path | None) -> dict[str, Any]: + if path is None: + return {} + if path.is_symlink() or not path.is_file(): + raise ValueError("evidence must be a regular file") + if path.stat().st_size > MAX_EVIDENCE_BYTES: + raise ValueError("evidence exceeds 64 KiB") + try: + data = json.loads(path.read_text(encoding="utf-8"), + object_pairs_hook=_strict_object, parse_constant=_reject_constant) + except (UnicodeError, json.JSONDecodeError) as exc: + raise ValueError("invalid UTF-8 JSON evidence") from exc + if not isinstance(data, dict) or set(data) - set(AREAS): + raise ValueError("evidence must be an object with only known area keys") + return data + + +def _checked_at(value: str) -> bool: + if not isinstance(value, str): + return False + try: + ts = datetime.fromisoformat(value.replace("Z", "+00:00")) + return ts.tzinfo is not None and ts.utcoffset() is not None + except ValueError: + return False + + +def _evidence_area(name: str, raw: Any, head: str) -> tuple[dict[str, Any], list[dict[str, str]]]: + if raw is None: + return {"area": name, "status": "NOT_SEARCHED", "basis": "no adapter evidence supplied"}, [] + if not isinstance(raw, dict) or set(raw) - {"status", "source", "checked_at", "revision", "items", "reason"}: + raise ValueError(f"invalid {name} evidence") + status = raw.get("status") + if status not in STATES: + raise ValueError(f"invalid {name} coverage status") + if status in {"NOT_SEARCHED", "BLOCKED_UNKNOWN"}: + if raw.get("items"): + raise ValueError(f"{name}: unchecked area cannot claim candidates") + return {"area": name, "status": status, + "basis": str(raw.get("reason", "adapter did not verify this area"))[:200]}, [] + if not isinstance(raw.get("source"), str) or not raw["source"].strip() or not _checked_at(raw.get("checked_at")): + raise ValueError(f"{name}: checked coverage requires source and offset-aware checked_at") + if name != "external_patterns" and raw.get("revision") != head: + return {"area": name, "status": "BLOCKED_UNKNOWN", + "basis": "supplied evidence revision differs from current HEAD"}, [] + items = raw.get("items", []) + if not isinstance(items, list) or len(items) > MAX_ITEMS_PER_AREA: + raise ValueError(f"{name}: too many or invalid items") + if status == "CHECKED_NO_DELTA" and items: + raise ValueError(f"{name}: no-delta coverage cannot contain candidates") + if status == "USEFUL_CANDIDATE" and not items: + raise ValueError(f"{name}: candidate status requires items") + candidates: list[dict[str, str]] = [] + for item in items: + if not isinstance(item, dict) or set(item) != {"name", "reference", "reason"}: + raise ValueError(f"{name}: item requires name, reference and reason") + if not all(isinstance(value, str) and 0 < len(value) <= 300 for value in item.values()): + raise ValueError(f"{name}: invalid candidate fields") + candidates.append({"area": name, **item, "evidence": "adapter-supplied; fit not independently verified"}) + return {"area": name, "status": status, "basis": raw["source"][:240], + "checked_at": raw["checked_at"]}, candidates + + +def preflight(repo: Path, *, target: str, trigger: str, expected_head: str, + evidence: dict[str, Any] | None = None, + github_repository: str | None = None) -> dict[str, Any]: + if not isinstance(target, str) or not target.strip() or len(target) > 240: + raise ValueError("target must be 1-240 characters") + if trigger not in TRIGGERS: + raise ValueError("unsupported trigger") + if not isinstance(expected_head, str) or not re.fullmatch(r"[0-9a-f]{40}", expected_head): + raise ValueError("expected_head must be a full 40-hex commit") + repo = repo.resolve() + head = _git(repo, "rev-parse", "HEAD") + root = Path(_git(repo, "rev-parse", "--show-toplevel")).resolve() + if root != repo or head != expected_head: + raise ValueError("repo root or live HEAD differs; reconcile first") + if evidence is None: + evidence = {} + if not isinstance(evidence, dict) or set(evidence) - set(AREAS): + raise ValueError("unknown evidence areas") + + if trigger == "routine": + result: dict[str, Any] = { + "schema": SCHEMA, "head": head, "target": target, "trigger": trigger, + "status": "SKIPPED", "coverage": [], "candidates": [], + "next_action": "PROCEED_WITH_NORMAL_MATRIXLOOP", "adoption_authorized": False, + "limitations": ["Routine work is not blocked by a scout."], + } + else: + if github_repository is not None: + if "open_work" in evidence: + raise ValueError("choose supplied open_work evidence or GitHub collection, not both") + evidence = {**evidence, "open_work": collect_open_work( + github_repository, target=target, revision=head, + token=os.environ.get("GITHUB_TOKEN"))} + result = {"schema": SCHEMA, "head": head, "target": target, "trigger": trigger, + "status": "INSPECTED", "coverage": [], "candidates": [], "adoption_authorized": False, + "limitations": ["Name/symbol matching is a lead, not proof of implementation or fit.", + "GitHub is queried only when explicitly selected; title leads are not verified reuse.", + "No tool inventory or external pattern research is performed."]} + mapped = repository_map(repo, query=target, budget_bytes=32768) + terms = {term.lower() for term in _TERMS.findall(target) + if term.lower() not in {"with", "from", "this", "that", "eine", "einen", "und", "the", "for"}} + matched = [] + for entry in mapped["entries"]: + names = " ".join([entry["path"], *[str(s.get("name", "")) for s in entry.get("symbols", [])]]).lower() + if any(term in names for term in terms): + matched.append({"area": "owned_repository", "name": entry["path"], + "reference": f"{head}:{entry['path']}", + "reason": "filename/symbol keyword match only; inspect behavior and reuse fit", + "evidence": "tracked working-tree symbol map; not verified reuse"}) + status = "USEFUL_CANDIDATE" if matched else ("NOT_SEARCHED" if mapped["truncated"] else "CHECKED_NO_DELTA") + result["coverage"].append({"area": "owned_repository", "status": status, + "basis": "tracked working-tree path/symbol probe; not full implementation audit", + "fingerprint": mapped["fingerprint"], "truncated": mapped["truncated"]}) + result["candidates"].extend(matched[:MAX_CANDIDATES]) + for name in AREAS: + coverage, candidates = _evidence_area(name, evidence.get(name), head) + result["coverage"].append(coverage) + result["candidates"].extend(candidates) + priority = {"owned_repository": 0, "open_work": 1, "installed_capabilities": 2, "external_patterns": 3} + result["candidates"].sort(key=lambda value: (priority[value["area"]], value["name"])) + result["candidates"] = result["candidates"][:MAX_CANDIDATES] + if result["candidates"]: + result["next_action"] = "VERIFY_REUSE_FIT_WITH_SMALLEST_TEST" + elif any(area["status"] in {"NOT_SEARCHED", "BLOCKED_UNKNOWN"} for area in result["coverage"]): + result["next_action"] = "RESEARCH_ONLY_HIGHEST_IMPACT_UNKNOWN_IF_NEEDED" + else: + result["next_action"] = "NO_USEFUL_DELTA_IN_CHECKED_SCOPE" + if _git(repo, "rev-parse", "HEAD") != head: + raise ValueError("live HEAD changed during preflight") + result["fingerprint"] = hashlib.sha256( + json.dumps(result, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8") + ).hexdigest() + return result + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repo", type=Path, required=True) + parser.add_argument("--target", required=True) + parser.add_argument("--trigger", choices=sorted(TRIGGERS), required=True) + parser.add_argument("--expected-head", required=True) + parser.add_argument("--evidence", type=Path) + parser.add_argument("--github-repository", help="opt-in one-page open PR/issue title probe for owner/name") + args = parser.parse_args(argv) + try: + report = preflight(args.repo, target=args.target, trigger=args.trigger, + expected_head=args.expected_head, evidence=_load_evidence(args.evidence), + github_repository=args.github_repository) + except (ValueError, OSError, subprocess.SubprocessError) as exc: + print(f"loop42 gap/reuse preflight: {exc}", file=sys.stderr) + return 2 + print(json.dumps(report, ensure_ascii=False, indent=2)) + return 0 + + +if __name__ == "__main__": + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8") + raise SystemExit(main())