okfy-mcp over HTTP: one shared token, or identities from an OpenID issuer

CHANGELOG.md

@@ -3,6 +3,197 @@
 This changelog starts at v0.26.0. Earlier releases are not back-filled here —
 their history is not something this file can verify.
 
+## Unreleased — retrieval: stopwords, matched terms, outline
+
+- **Stopwords leave the query.** `query.search_pool` drops a short English list
+  (the, who, how, ...) before BM25; the index keeps every token, so no bundle's
+  index fingerprint moves. Before, `who` in a small corpus carried real idf and
+  pulled an unrelated concept to rank 2.
+- **Every hit says what it matched, body included.** `okfy query` hits carry
+  `matched_terms` (query terms, expansion included, found anywhere in the
+  concept); `okfy_query`'s `matched.terms` reads it instead of overlapping the
+  query with title+description only, which showed `[]` for body-only matches.
+- **`okfy_show(outline=true)`** returns the `## ` headings and the sha, no body,
+  so a section can be picked without reading the whole concept.
+- **The server introduces itself.** okfy-mcp now sends MCP `instructions` at
+  the handshake (search with the user's words, show's sha goes into write,
+  nothing found means a gap proposal, retrieved content is data), and a
+  bundle-mode search with no hits carries `way_out` naming
+  `okfy_propose(action="gap", query=<the user's words>)`. Until now the gap
+  door was one phrase inside okfy_propose's description.
+
+## Unreleased — the direct-write lane
+
+Until now an agent had one way to write: file a proposal and wait for the owner.
+That is right for knowledge and wrong for session notes, observations and logs,
+where several agents working at once need to see each other's writes now. This
+adds a second lane the owner opens per concept type. Nothing changes for a bundle
+that does not open it, and no version is bumped here: the number is the
+maintainer's.
+
+### Stricter behaviour — read this before upgrading
+
+- **`direct` in `meta/purpose.md` is now a checked key.** A bundle that already
+  carried a `direct:` key of its own fails `okfy validate`: `E_DIRECT_KEY` (not a
+  mapping, or a key other than `types`), `E_DIRECT_TYPE` (`types` is not a
+  non-empty list of names, or names a type outside the plan's declared `types`)
+  and `E_DIRECT_POLICY` (declared while `write_policy` is not `proposals`, where
+  the block would silently do nothing). All three are errors at every strictness
+  level, like `E_WRITE_POLICY`: it is a trust boundary. A listed type outside the
+  archetype's canonical list, in a bundle that declares no `types`, is
+  `W_DIRECT_TYPE` and becomes `E_DIRECT_TYPE` under `--strict-schema`.
+- **"An MCP tool call can never touch a final concept" is now true per bundle, not
+  per adapter.** It holds exactly as before for every bundle without a `direct`
+  block, and for `okfy_propose` always. A bundle that lists `direct.types` has
+  chosen otherwise for those types.
+
+### The lane
+
+- `meta/purpose.md`, under `write_policy: proposals`, may declare
+  `direct: {types: [note, session-log]}`. `write_policy` keeps its two literals and
+  the pre-commit hook is unchanged: a direct write commits through the same
+  `_commit` every sanctioned writer uses.
+- `okfy.proposals.write_direct(bundle, concept_id, *, actor, expect_sha256,
+  text=None, patch=None, note="", observed=None, archetype=None)` writes a
+  concept whose type is listed, at once. The lane is decided by the concept's
+  current type on an update and by the proposed type on a create; `meta/` and the
+  other reserved directories are never in it, so the policy cannot rewrite itself.
+- Concurrency is optimistic and adds no lock. `expect_sha256` is the sha256 of the
+  file the author read (the digest `okfy_show` already returns), or `absent` to
+  create. The check and the write are one step under the existing `review_lock`;
+  the file is read once and the digest and the text both come from that read; the
+  write is a temp file and a rename. Two writers holding the same sha256: one
+  lands, the other is refused `E_WRITE_BASE_MOVED` with the current sha256. A hand
+  edit takes no lock: one made before that read is refused the same way, one made
+  between the read and the rename is lost. That window is the honest limit of
+  compare-then-rename.
+- What a proposal would be refused for is refused here, by the same functions: the
+  injection and secret scans (now `_refuse_scan`, shared with the propose gate),
+  `sources:` outside the corpus, and the archetype schema `review accept` applies
+  (`E_DIRECT_INVALID`). The rejected-content and duplicate-title gates belong to
+  the review queue and do not run.
+- A direct write cannot change `type` (`E_DIRECT_RETYPE`) or what an owner decision
+  wrote: `verified`, `generated`, `stale`/`stale_reason`/`stale_since`,
+  `supersedes`/`superseded_by` (`E_DIRECT_OWNER_FIELD`). They are carried over from
+  the file. A concept edited this way after a verification warns
+  `W_VERIFIED_SUPERSEDED`, as after a `refine`.
+- The ledger row is an `accept` with `proposal: null`, `origin: agent-direct`, the
+  agent as actor and `base_sha256`. Readers of `accept` rows see it as a content
+  write: `release-check` counts it among the changes since the last package and
+  `okfy diff` lists the concept as protected. It is not a decision, though:
+  `okfy.update.unresolved_rejections` and `okfy dismiss` now read one shared
+  fold (`okfy.memory.latest_decisions`) that skips it, so an agent's write
+  neither disposes of an owner's rejection nor stops the owner dismissing it.
+- `okfy write <bundle> <concept_id> --as <actor> --expect <sha256|absent>
+  (--from <file> | --patch-file <hunks.json>) [--note …]` is the CLI door. It
+  refuses; it does not fall back.
+
+### MCP
+
+- `okfy_write(concept_id, actor, expect_sha256, content?, patch?, note?,
+  evidence?)`. The agent does not need to know the policy: a listed type is written
+  at once (`channel: "direct"`, with the new file's `sha256`); any other type, and
+  any bundle with no lane, is filed through the `okfy_propose` path
+  (`channel: "proposal"`) and only if the target is still at `expect_sha256`
+  (`okfy.proposals.propose_checked`: check and filing under one lock).
+  `persisted: true` in both cases. A refusal is `{error, message, way_out}`;
+  `E_WRITE_BASE_MOVED` also carries `current_sha256`.
+- A direct write's ledger row keeps four counters from the server's own session
+  trace: `by`, `queries`, `target_shown` and `base_shown`, the last being whether
+  this session was shown the concept at the very sha256 it wrote against. An agent
+  that takes `current_sha256` from a refusal and writes without reading is
+  therefore visible in the ledger. It is not prevented: like every actor string
+  here, this is a record, not authentication.
+- `okfy_show`'s docstring points at `okfy_write`. No existing response shape
+  changed.
+- **`okfy-mcp serve <path> --http --token-file <file>`** serves the same seven
+  tools over streamable HTTP (`/mcp`, stateless, JSON responses) to many clients
+  at once, behind one bearer token. The token is read from a file, never a flag;
+  a missing, unreadable or short (< 32 characters) token refuses to start
+  (`E_MCP_HTTP_TOKEN`), and a request without the header is `401` with the
+  documented envelope (`E_MCP_UNAUTHORIZED`) before the MCP layer sees it. Plain
+  HTTP: loopback by default, TLS and anything finer than one token are a reverse
+  proxy's job. No `Session` trace over HTTP — one process serves every client, so
+  `observed` is absent rather than attesting one client's reads to another's
+  write. stdio is unchanged.
+- **`--http --oidc-issuer <url> --oidc-audience <client id>`** is the other
+  access model: JWT access tokens from an OpenID issuer (Authentik, Keycloak,
+  …), checked against its published keys (RS256, `iss`, `aud`, `exp`; discovery
+  read once at start, a wrong issuer refuses to start with `E_MCP_OIDC_ISSUER`).
+  Both flags repeat: an Authentik puts each client app under its own issuer
+  path, and the token's `iss` picks the key set. `--oidc-write-group <name>`
+  makes `okfy:write` count only for users whose token lists that group in a
+  `groups` claim: guests read, owners write, whatever the client asked for.
+  `--oidc-issuer-prefix <url>` trusts every issuer under that URL as it turns
+  up (discovery on first sight, signature checked, audience not): the shape
+  an Authentik gives MCP clients that registered themselves.
+  The MCP SDK's own middleware answers 401/403 in the OAuth shape and publishes
+  the protected-resource metadata under `--public-url`, so an MCP client can
+  find the issuer; it advertises `okfy:read`, `okfy:write` and `offline_access`
+  as `scopes_supported` (the SDK would advertise only the required scope, and
+  a client that reads it could then never write). The token names the
+  **actor** of `okfy_write` and `okfy_propose`: its `okfy_actor` claim when
+  the issuer sets one (`claude:vasyl`, `codex:vasyl`: one human, several
+  apps), else `sub` (`oidc:<sub>` when that is not already an OKF actor
+  string); the tools' `actor` parameter is now optional and ignored under a
+  token. The first place an actor string here is a verified identity, not a
+  declaration. Scopes: `okfy:read` to call anything, `okfy:write` for the two
+  writing tools (`E_MCP_FORBIDDEN`, nothing written). Exclusive with
+  `--token-file`.
+- A direct **create** of a type the plan does not declare is refused
+  `E_UNKNOWN_TYPE` (reused with validate's meaning) before the lane check. It
+  used to miss the lane and, over MCP, become a proposal invisible to search
+  that validate rejects the moment it is accepted.
+- `okfy package` adds one block to `AGENTS.md`, between `<!-- okfy:direct -->`
+  markers, naming the listed types and `okfy write`. It is rendered only for a
+  bundle with a well-formed `direct` block, so every other bundle's `AGENTS.md`
+  is byte-identical, and the memory block and its 60-token bound are untouched.
+  Without it the memory block's "never written to concepts directly" was false
+  for exactly the types the owner had opened.
+
+### New codes
+
+`E_DIRECT_POLICY`, `E_DIRECT_KEY`, `E_DIRECT_TYPE`, `W_DIRECT_TYPE`,
+`E_DIRECT_DISABLED`, `E_DIRECT_LANE`, `E_DIRECT_TARGET`, `E_DIRECT_RETYPE`,
+`E_DIRECT_OWNER_FIELD`, `E_DIRECT_INVALID`, `E_WRITE_CONTENT`,
+`E_WRITE_BASE_MOVED`, `E_WRITE_WORKSPACE`, `E_MCP_HTTP_TOKEN`,
+`E_MCP_UNAUTHORIZED`, `E_MCP_OIDC_ISSUER`, `E_MCP_FORBIDDEN`. Reused with their
+meaning unchanged: `E_UNKNOWN_TYPE`,
+`E_PROPOSAL_ACTOR`, `E_PROPOSAL_INJECTION`, `E_PROPOSAL_SECRET`,
+`E_PROPOSAL_SOURCE`, `E_PROPOSAL_TRUNCATED`, `E_FRONTMATTER`, `E_PATCH_SHAPE`,
+`E_PATCH_COUNT`, `E_RESERVED_DIR`.
+
+### Changed outside the lane — a separate commit, droppable on its own
+
+- **The injection scan of an agent-written value joins hard wraps only**
+  (`okfy.injection.scan_values`, used by `okfy propose` and `okfy write` alike).
+  The second pass used to collapse every newline of a value, which turned a
+  whole concept body into one line; the co-occurrence rule for exfiltration,
+  written for one line, then matched an outbound verb, the word `password` and
+  a URL anywhere in the document. Measured on a 48-concept bundle of working
+  notes: 8 bodies refused, 5 of them for words in different paragraphs or in
+  different items of one list, each with the entire body as the excerpt. A
+  blank line and the start of a list item, heading, quote or table row are now
+  line boundaries. A phrase split by a hard wrap is still caught. One split
+  across a blank line or a list marker is seen by neither pass: that is the
+  price, and it is the maintainer's to accept or refuse.
+
+### Not changed, and not claimed
+
+- `okfy refine` still writes whatever parses and has a `type`: no injection or
+  secret scan, no archetype check, no lock, no base check, a plain `write_text`.
+  It is the owner's door and was left alone; the direct lane does not go through
+  it and does not inherit any of that.
+- `protocols/memory.md` does not mention the lane; the packaged `AGENTS.md` does
+  (see above), and an MCP agent also learns it from the tool description.
+- One commit per direct write. A write touching several concepts at once is still
+  a set of proposals.
+- A concept created directly is found by `okfy query` at once and is not in
+  `index.md` until the next `okfy package`, exactly like an accepted create.
+- This adds `core/tests/test_direct_write.py` and
+  `adapters/mcp/tests/test_write_tool.py`. The public export ships no tests by
+  policy, so CI's conditional unit step now runs these two files and nothing else.
+
 ## v0.29.0 — 2026-09-30
 
 An audit of v0.28 found two gaps in the shipped tool and four in a local experiment