[stiebitzhofer]

Closes #135

What

• Worker:
  • /spec/<act>/<version>: JSON-LD by default, HTML on Accept: text/html, immutable cache. Its sub-resources context.jsonld, compressed.schema.json and full.schema.json are immutable too.
  • /spec/<act> (the version series), /catalog (DCAT) and /ns (the mi: vocabulary): short cache with Vary: Accept.
• Build:
  • add-only spec/<act>/<version>/, copied byte-identical;
  • a PoC-bannered HTML page per spec;
  • scripts/lib/dcat.ts builds the catalogue (dictionary releases + specs, with checksums).
• Checks:
  • spec/ is add-only;
  • JSON-LD parses (oxigraph);
  • adms:status mi:ProofOfConcept present;
  • CC0 licence present;
  • every pin's checksum equals the served entry bytes.
    The two-approval gate and CODEOWNERS cover spec/.
• Vocabulary: new mi: terms for content specifications (elementId, granularity levels, condition, ContentSpecification, ProofOfConcept, act, dppSchemaVersion, passportContext, …).
• Deploy: /spec/ and /catalog are added to the origin probes.
• Hook fix: .githooks/pre-push unsets git's repository variables. Pushed from a worktree, git exported GIT_DIR/GIT_INDEX_FILE to the hook, and the tests' throwaway repos (git -C <tmp>) inherited them. The tests then committed, tagged and set core.bare=true on the real repository.

Everything served under /spec/ is a PoC, not production.

Follow-up

The first real spec (spec/steel/v0.0.3/) arrives in a separate hand-over PR after this is released (schema#478).

Verification

npm run validate and npm test pass (120 tests, 97.4 % coverage).

🤖 Generated with Claude Code

[stiebitzhofer]

Security review

Scope: git diff origin/main...HEAD on feat/spec-hosting-135 (3 commits). I confirmed two of the findings by running scripts/lib/spec.ts locally against scratch trees. I did not run anything against deployed infrastructure.

Findings

1. Medium: symlinks under spec/ get around the add-only guarantee. scripts/lib/spec.ts:88-89,128
loadSpecs uses readdirSync/statSync/readFileSync, and all three follow symlinks. Nothing calls lstat, and nothing in scripts/ or the workflows checks for git mode 120000. Check 1 (checks.ts:297-313) sees an added symlink as an ordinary A and lets it through.

• Scenario A (whole version): a hand-over PR adds spec/steel/v0.0.2 as a symlink to a directory outside spec/, for example under test/fixtures/ or examples/. From then on, editing that directory in a normal PR changes what https://material-identity.eu/spec/steel/v0.0.2 serves, with immutable, max-age=31536000. Check 1 never fires because no path under spec/ changed. checkSpecs passes as long as the target stays self-consistent, because its DCAT checksums sit inside the same symlinked directory.
• Scenario B (single file): full.schema.json -> ../../../elsewhere/x.json. I confirmed that loadSpecs reports no issue and loads the target's bytes. The DCAT checksum check catches a later change only if the record lists that file as a distribution. checkSpecs requires a distribution only for the spec itself (checks.ts "lists no distribution of the spec itself"), so a record that leaves out context/compressed/full distributions leaves those files freely mutable.
• An absolute symlink makes the deploy runner publish one of its own files to the public site. The file has to parse as JSON, for example $GITHUB_EVENT_PATH.
• Fix: reject any entry under spec/ that is not a regular file or directory (use lstatSync, and/or check git ls-files -s for mode 120000). Also require a checksummed distribution for every served file, not only the spec.

2. Medium/Low: javascript: href injection in the spec page. scripts/lib/render.ts renderSpecPage (licence link), iriLink/externalLink (render.ts:229-231)
Text content is escaped correctly everywhere. The problem is URLs: IRIs from the spec's RDF go into href without a scheme check. The CC0 gate uses ask(), which only proves that a CC0 triple exists. summarize() then shows one(), which returns the lowest value in UTF-16 sort order. I confirmed this locally: with dcterms:license [<…/zero/1.0/>, <JAVASCRIPT:alert(document.domain)>], oxigraph accepts the uppercase-scheme IRI, ask returns true, and summary.license is JAVASCRIPT:alert(document.domain). The page then renders <a href="JAVASCRIPT:alert(document.domain)" rel="external license">, which runs script on material-identity.eu when clicked. The same path exists through status (gated by ask the same way), series, membership sh:path/mi:itemType and mi:accessCategory. None of those are scheme-checked, and this repo deliberately does not re-validate spec content. Impact is limited because the site is static with no session. Still, script on the canonical origin can rewrite the displayed digests and the PoC banner on a page that is cached as immutable. Fix: allow only https?: (and urn:, rendered as text, not a link) before emitting href. Also render the specific value the gate checked (CC0, mi:ProofOfConcept) rather than one()'s first value.

3. Low: renaming files into spec/ skips the two-yes gate (check 7). .github/workflows/pr-checks.yml:65
The gate selects f.status === 'added'. GitHub's PR files API reports renamed (and occasionally copied) when a file moves with high similarity. Check 1 runs --no-renames and sees A, so it passes, but check 7 sees renamed and returns "not applicable". Concrete case: this PR adds a complete green hand-over at test/fixtures/green/spec/demo/v0.0.1/. A later PR that git mvs it to spec/demo/v0.0.1/ gets hosted as immutable without an accepted issue. The same filter already applies to published/, but past publish PRs (#89, #127, #129) all showed added, so in practice only the new tree is exposed. Fix: ['added', 'renamed', 'copied'].includes(f.status), or for renamed files check whether the destination is under a gated prefix.

4. Low: the generated spec HTML is cached as immutable for a year. worker/index.ts (SPEC branch, negotiated(pathname, accept, IMMUTABLE))
The JSON-LD representation is the byte-identical hand-over, so immutable fits it. The HTML representation is renderSpecPage output, and that depends on the site template: the PoC disclaimer, the footer (this PR itself changes the footer on every page) and dictionary labels. Any later change to the PoC/legal wording, or a takedown, cannot reach caches that hold /spec/<act>/<version> as HTML for up to a year. Suggest IMMUTABLE only for the JSON-LD branch and SHORT (or max-age=86400) for HTML; Vary: Accept is already set.

5. Info: a spec's DCAT record can make statements about other subjects in /catalog. scripts/lib/dcat.ts findDatasetNode/embeddedDataset/buildCatalog
The record's dataset node is merged "as written", including nested nodes. A nested {"@id": "https://material-identity.eu/def/<uuid>", ...}, @reverse, or @included puts triples about dictionary entries, releases or other specs into the catalogue graph. Spec bytes cannot overwrite entry files (output paths are fixed and constrained by ACT_PATTERN/VERSION_PATTERN, so the RDF-level effect is the whole issue). Remote @context is already refused because loading has no document loader. Consider rejecting records whose node tree contains an @id outside spec.iri and its #fragment distributions.

Checked and clean

• decide() route confusion: the Worker matches on new URL().pathname, which normalises .., %2e%2e and .%2E before matching. %2F and every other %-escape fail the [a-z0-9-] / v\d+.\d+.\d+ classes and fall through to the existing passthrough (origin 404), so no traversal. Routes are anchored, SPEC_FILE names are a closed list, and /spec/<act>/<version>/ with a trailing slash falls to the 300 s passthrough. No /spec/ pattern can reach /def/, /.well-known/ or /examples/.
• Cache on mutable routes: /spec/<act>, /catalog and /ns (including Turtle) all use max-age=300. Raw /catalog/index.*, /ns/index.* and /spec/<act>/index.* pass through at 300 s. Origin non-200 responses become 404 with max-age=60, so a not-yet-hosted spec IRI is never cached as immutable. Vary: Accept is present on every negotiated route.
• Content types: set explicitly on all new routes (application/ld+json, application/schema+json, text/turtle; charset=utf-8, text/html; charset=utf-8). There is no nosniff header site-wide, but that predates this PR and none of the new JSON types get sniffed as HTML by browsers.
• HTML text injection: every title, description, condition, element id, language tag, version and checksum goes through esc(), which covers & < > ", and all attributes are double-quoted. The only gap is the URL scheme (finding 2).
• SPARQL construction in spec.ts/checks.ts: the interpolated values are spec.iri, act and version, and all of them are constrained by the directory regexes.
• Workflows: pr-checks stays on pull_request (no pull_request_target) with read-only permissions, and the only expression in run: is base.sha. In deploy.yml, spec-paths is built from file names that loadSpecs limits to a fixed set, newlines are stripped before $GITHUB_OUTPUT, and the unquoted $SPEC_PATHS expands names with no glob characters. Actions stay SHA-pinned.
• Add-only gate, beyond symlinks: --no-renames turns a rename into D+A, and the D is flagged. Mode-only changes show as M and type changes as T, and both are flagged. Extra or missing files in a version directory are refused by loadSpecs, and the build refuses to run on any load issue.
• Override of dictionary entries or other routes: output paths are site/spec/<act>/<version>/{index.jsonld,index.html,context.jsonld,*.schema.json} and site/spec/<act>/index.*, nothing else.
• .githooks/pre-push: unsetting the git repository variables only narrows what the tests can touch. CODEOWNERS for /spec/ is fine.

[stiebitzhofer]

Security review: findings addressed in 4b88154

1. Symlinks under spec/ (Medium). loadSpecs now lstats every entry and refuses a symlink or special file instead of following it, whether it is spec/ itself, an act, a version directory or a file. Check 1 now reads git diff --raw and refuses an added 120000 (symlink) or 160000 (submodule) under spec/ or published/. The DCAT record must also list every served file (spec JSON-LD, context, compressed and full schemas) as a distribution, and every checksum it states for a URL must match the hosted bytes, including duplicates. Tests cover a symlinked file, version dir, act dir and spec/, plus a real temp repo with an added symlink.
2. javascript: href (Medium/Low). Only http(s) IRIs (case-insensitive) become <a href>. Everything else renders as escaped <code>. This covers the licence, the status, the series, membership entries and access categories. The licence and status checks now require exactly one value, CC0 and mi:ProofOfConcept, in both the spec and its DCAT record, so the page shows the value the check verified. Tests cover javascript:, JAVASCRIPT:, mixed case and data: rendering as text, and a multi-valued licence and status failing the check.
3. Two-yes gate and renames (Low). pr-checks.yml now counts added, renamed and copied files whose new path is under published/ or spec/. This step is inline github-script and is not unit-tested; actionlint passes apart from the existing ubuntu-26.04 runner-label warning.
4. Immutable spec HTML (Low). The spec's JSON-LD stays immutable. Its HTML now gets max-age=300. The cache header follows the representation actually served, and Vary: Accept is unchanged. A worker test covers this.
5. DCAT merge (Info). The embedded dataset node is parsed. If it states anything about a named subject other than the spec or its #fragments, the spec check fails: that covers nested nodes, @reverse, @included and named graphs. Plain references such as the release and the series are still allowed.

npm run validate passes. npm test passes 127/127 with 97.85% line coverage.

[stiebitzhofer]

Code-review findings handled in 8e3c220. I checked each finding against the code before fixing it, and all eight that were in scope turned out to be real.

1. Two-yes gate label filter for spec/ hand-overs. Not changed. This is pending an owner decision (a dedicated hand-over label/issue type, or a gate keyed per directory).
2. Membership references vs pins. checkSpecs now applies the schema repo's rule. Every membership's sh:path (IRI), mi:itemType and mi:accessCategory must be pinned, and every pin must be used by a membership (membershipReferences in spec.ts). Pins were already checked against published/.
3. DCAT checked ≠ DCAT served. Status, licence, dcat:Dataset type, isVersionOf, checksums and foreign subjects are now all checked on catalogStatements(spec). That is the set of statements the record adds to the built /catalog: buildCatalog([], [spec]) minus the same catalogue with the record reduced to a bare reference. This covers the bytes actually served. It also covers the case the finding missed, where the catalogue's own context changes how a term in the embedded node reads. There is a test with a property-scoped context the embedding drops: the record still reads as PoC on its own, but now fails.
4. Conflicting pin checksums. distributionPins returns one pin per distinct (entry, checksum) pair, sorted, and every pair is checked. There is a test with the wrong digest placed first and placed last.
5. Pinned release. A new spec — pinned dictionary release check. The spec must pin exactly one dcterms:requires dcat:Dataset, its dcat:version must be a vYYYY.MM.DD tag at /release/<tag> that exists in getReleaseTags(), and every pinned entry must have first shipped (getReleases()) at or before that release. The check is skipped with a notice when there is no git context or no release tag in the clone (shallow checkouts such as expiry-watch.yml fetch none). It is tested with real temp-repo tags. One related fix: getReleases() now counts only vYYYY.MM.DD tags, so a tag like draft-base can no longer be taken for a release.
6. dcat:isVersionOf. It must be exactly seriesIri(act), in the spec and in the record as served.
7. GIT_ scrub.* gitEnv() in lib/git.ts removes every variable git rev-parse --local-env-vars lists (asked of git once). The lib's git() and all test git helpers use it. The pre-push hook now runs unset $(git rev-parse --local-env-vars) as defence in depth (shellcheck clean). A test sets GIT_DIR/GIT_WORK_TREE to a bogus path, shows that an unscrubbed child fails, and shows that the helpers and getReleaseTags use the temp repo.
8. summarize once. emitSpecs summarizes each spec once and returns {spec, summary} pairs. buildSeries, the series page, the spec page and the catalogue page all take the passed summary.
9. rel=alternate. The JSON-LD alternates are now /spec/<act>/<version>, /spec/<act>, /catalog and /ns. The in-body "Machine-readable" download links still point at the files, because a browser click sends Accept: text/html and the canonical IRI would return the same page.

Also checked: the real steel/v0.0.3 hand-over from material-identity/schema passes the new spec and release checks, pinning v2026.10.01.

npm test: 136/136 pass, 98.06% line coverage. npm run validate and npm run build are OK.

[stiebitzhofer]

c0210f9 makes the release IRIs that specs pin resolve on this site, as the owner decided: https://material-identity.eu/release/<tag>.

• /release/<tag> serves each vYYYY.MM.DD tag as a dcat:Dataset, with:
  • dcat:version and dcterms:issued (the tag date)
  • PoC status and CC0
  • the GitHub release page, plus a distribution that links dictionary-<tag>.tar.gz with no checksum (the offline build doesn't know the asset's digest)
  • dcterms:hasPart: every entry the release contains, each with the SHA-256 of its /def/<uuid> JSON
  • mi:firstShipped: the entries this release adds. This is a new /ns term.
• Caching and negotiation: JSON-LD by default with an immutable cache; HTML with the PoC banner gets the short cache. cite-as works like it does for a spec. An unknown tag returns 404.
• /release is a dcat:Catalog of all releases, short cache. /catalog lists each release with the same node, and the catalogue and spec pages now link to /release/<tag>.
• Agreement with the pinned-release check: the check and the release pages use the same two rules. shippedBy decides what a release contains, and entrySha256 is the digest a spec pins.
• Deploy probes now cover /release/index.* and every release page the build emits.
• Tests: worker routes, cache headers and the 404, the build output in a tagged throwaway repo (including agreement with the check), and determinism. 147/147 pass, 98.14% line coverage. npm run validate passes and actionlint is clean.

The two-yes gate label filter and the dcat:DatasetSeries modelling are unchanged. /release is a dcat:Catalog, so neither pending decision is pre-empted.