Skip to content

Save the graph schema, and serve it only against its own database - #45

Merged
adamjohnwright merged 6 commits into
mainfrom
feat/schema-extract-and-file
Oct 14, 2026
Merged

adamjohnwright merged 6 commits into
mainfrom
feat/schema-extract-and-file

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

What

  • reactome-mcp extract-schema — runs the server's own fetchGraphSchema() once against a database named with --uri (--database, --user, --password-env VAR, --out file; stdout without --out) and writes the schema JSON. One extractor; no second implementation.
  • schemaFormat + source fingerprint on every schema: Neo4j version/edition, Reactome release from DBInfo (omitted when there is no DBInfo node), total node and relationship counts (count store), generatedAt.
  • NEO4J_SCHEMA_FILE — with graph access on, the first schema request reads the file, takes the live fingerprint (4 cheap queries, not the extraction), and serves the file only if it matches. Different release / counts / Neo4j version, another schemaFormat, an incomplete snapshot, a missing or non-JSON/malformed file → a warning naming the reason and live extraction. A database that cannot be fingerprinted is not a match. With graph access off, the file is never read.
  • README: saving the schema, the fingerprint guard, producing reactome-<release>-schema.json from the release pipeline, and Pointing at your own database for curators. CHANGELOG. Digest header now names the release.

Decisions worth a look

  1. How the CLI gets past MCP_ALLOW_CYPHER. It sets NEO4J_URI + MCP_ALLOW_CYPHER=1 for its own process only, after its arguments validate, and then uses the same gated getDriver() — rather than adding a second, ungated way to build a driver. So it reads in the same READ sessions under the same timeouts. The gate test asserts the CLI imports no server/tool/resource/transport module, and that reactome-mcp extract-schema dispatches to it without building a server. Recorded as constitution 2.0.1 (PATCH) under Principle IV.
  2. --uri is required; user/database/password fall back to NEO4J_USER/NEO4J_DATABASE/NEO4J_PASSWORD like the server. --password, --password=… and -p are refused (exit 2) without echoing the value; --password-env naming an unset variable is refused rather than silently connecting without credentials.
  3. Incomplete schemas are never snapshotted: the CLI refuses to write one (exit 1), and the server rejects a file with unavailable sections — a snapshot is served for as long as the DB matches, so a gap would never be retried.
  4. The CLI ignores an inherited NEO4J_SCHEMA_FILE so it always extracts rather than copying a matching snapshot.
  5. Fingerprint compares version, edition, release, node and relationship counts; generatedAt is recorded but not compared. Release compares as a string (97 ≡ "97"). Snapshot JSON is validated field by field with zod, not cast (Principle I).
  6. fetchGraphSchema() now asks the gate itself before touching the file or the DB (mutation-checked: removing it fails 5 gate tests).
  7. Writes go to <out>.<pid>.partial then rename, so a failed run never leaves a truncated file.

Also fixed

build() in src/clients/neo4j.ts hooked SIGINT/SIGTERM/beforeExit on every driver build and never unhooked; a process that reconnected exceeded Node's listener warning. Now hooked once per process (test: 12 CLI runs add at most one listener).

Tests (fake driver; no real DB)

Written first and shown failing before the code: tests/extract-schema-cli.test.ts (structure equals the server's extraction + fingerprint, stdout/--out, no MCP_ALLOW_CYPHER needed, READ + timeouts, password refusal / env / --password-env, usage errors, incomplete schema, DB down, listener leak), tests/graph-schema-file.test.ts (matching file skips every expensive schema query; 11 mismatch cases + missing file fall back with a warning; no-DBInfo file vs no-DBInfo DB matches; unfingerprintable DB never served), additions to tests/graph-schema.test.ts (fingerprint, no-DBInfo), and the gate matrix (snapshot read 0 times closed / once open, per entrypoint and for fetchGraphSchema directly). The scripted 4.4 responder moved to tests/helpers/community44.ts, shared by three files.

npm run check green: lint, format, both typechecks, build, 268 tests.

Not done: no live smoke run against a real graph database — credentials for the local one were not available to this change, so the CLI and NEO4J_SCHEMA_FILE are verified against the fake driver only. Worth one manual run of extract-schema against the Release 97 container, then the server with NEO4J_SCHEMA_FILE, before merging.

🤖 Generated with Claude Code

adamjohnwright and others added 6 commits October 14, 2026 14:33
One extractor, two uses. `reactome-mcp extract-schema --uri ...` runs
fetchGraphSchema() once and writes the schema JSON; NEO4J_SCHEMA_FILE has
the server serve such a file instead of extracting at startup.

Every schema now carries `schemaFormat` and a `source` fingerprint: Neo4j
version and edition, the release from DBInfo (absent where there is no
DBInfo node), total node and relationship counts, and when it was read.
On the first schema request the server reads the live fingerprint (four
cheap queries, not the extraction) and serves the file only if it agrees;
another release, a changed count, another format, an incomplete snapshot
or a missing or malformed file is warned about by name and the schema is
extracted live. A database that cannot be fingerprinted is not a match.
fetchGraphSchema() asks the gate first, so with graph access off the file
is never read; the gate matrix now asserts that in both directions, and
that the CLI imports no server, tool, resource or transport.

The CLI is an operator command, not an instance, so it opens the gate for
its own process only. It reads through the same gated driver in READ
sessions under the schema timeouts, refuses a password argument (`ps`
shows it) and takes it from NEO4J_PASSWORD or --password-env, ignores an
inherited NEO4J_SCHEMA_FILE, and refuses to write a schema with
unavailable sections.

Also: the driver hooked SIGINT/SIGTERM/beforeExit on every build and
never unhooked, so a process that reconnected piled listeners past Node's
leak warning. Hooked once per process now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README: saving the schema, the fingerprint guard, producing
reactome-<release>-schema.json from the release pipeline, and pointing an
internal instance at your own database. CHANGELOG entries. Constitution
2.0.1 (PATCH): records why the extract-schema command connects without
MCP_ALLOW_CYPHER and is not an instance with graph access.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #45: a fingerprint of version, release and two totals passed a
snapshot through after SET n.newProp, CREATE INDEX, or one node deleted
and one added. The live fingerprint now also reads the connected
database's name (db.info()) and digests -- sha256 of each list in
canonical order -- of db.labels(), db.relationshipTypes(),
db.propertyKeys(), the indexes (name, labels, properties, state) and the
constraints, with the existing SHOW/db.* fallbacks. Metadata only; no
property scan. schemaFormat is 2, so format 1 files fall back on format.

A saved schema whose body disagrees with its own fingerprint (counts,
label and type lists, indexes, constraints, Neo4j version) is refused;
the CLI refuses to write one, which is what a database written to during
the run produces.

The CLI never echoes an argument value: an unknown option is named up to
its '=', a stray value is "not shown", and --password-env names a
variable only when it is a valid variable name.

The "served from file" log line carries the database name as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI on Node 20 failed "falls back ... on a constraint list that disagrees":
the test proved fallback by fetchedAt differing from the file's, and the
live extraction finished in the same millisecond the snapshot was taken.

Every snapshot in the file tests now carries a node property row the
scripted database never reports (node properties are neither
fingerprinted nor checked against the fingerprint, so the file stays
valid and matching). A served schema has it; a live one does not. That,
with the expensive-query check already there, is what each fallback test
asserts; the matching test asserts the reverse.

The clock is frozen for the whole suite so the timestamps always
collide: under it the old assertion failed every fallback case, and the
new one passes. With the file wrongly served on a fingerprint mismatch
and the query check blinded, the marker alone fails 14 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI on Node 24: "the stdio entrypoint warms the schema over one driver"
saw no driver and the next test saw two. The tests waited a fixed 25 ms
for the fire-and-forget warm-up; under load the driver build landed in
the following test.

warmGraphSchema() now records that it was asked and keeps the warm-up
it started (`_graphSchemaWarmup()`, for tests; the promise never
rejects). The entrypoint tests wait, bounded, for the entrypoint to ask
and then await that warm-up. After every test that starts one, any
warm-up still running is awaited, the driver closed, and the test that
left it fails, so nothing can land in the next test. The extract-schema
dispatch test waits for process.exitCode rather than a time.

The fake neo4j-driver module now loads after 60 ms, as on a busy runner,
so a fixed wait fails every run here instead of sometimes in CI: with
it, the old waits failed the stdio test and leaked a driver into a later
one. A test that skips the wait now fails itself with "warm-up still
running", and nothing else. Passes 30/30 alongside two parallel
full-suite loops, and with a 300 ms load.

No other test starts an entrypoint or a warm-up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@adamjohnwright
adamjohnwright merged commit 3a77797 into main Oct 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant