[assassin808]

A published protocol.json (hsenv 0.8.1 release projection) holds placeholders where wording from a non-redistributable source goes. This PR restores that wording on the researcher's machine with the bundled hsenv.rehydrate. A study runs only once every screen is filled in and checked.

Design

• Helper pack/runtime/lib/hs_original_text.py, run by main from the app's own copy and isolated like the checks, with network access only for downloads:
  • downloads link-tier sources:
    • http(s) only, never a local or private address, redirects included; a host that does not resolve here is never fetched
    • at most 90 s per download and 150 s per check; each socket read waits at most the time left and takes what has arrived, so a slow server stops at the deadline
    • cached; a failure is retried after 10 minutes
    • a web page returned for a non-HTML source counts as a failed download (a sign-in or paywall page)
  • tries the researcher's files in sources/private/, paper.pdf and sources/
  • checks each item by its own text hash
  • caches the restored protocol per protocol.json version in <userData>/original-text/, never in the project
• Readiness (main/original-text.ts, new text lane in package-readiness.ts):
  • runs when the study opens, after a file is dropped in, and in readiness checks; a result is kept at most 10 minutes
  • missing wording blocks a protocol run, and the study still previews
  • writes a status for the agent to .humanstudy/original-text.json (counts, titles, needs; no wording)
  • the held list (distribution) is pinned in app storage the first time the study is seen. An edited or removed list blocks the run, and the message says to put back protocol.json as received.
  • this build has no import path for a published protocol.json: the library forks only program.json, package/, artifacts/ and sources/. When one lands, it must call repinHeld after writing a new published version.
• Engine checks:
  • no app-side rule decides which texts are excused. Each held source gets a source text of placeholders, so the engine's own verbatim rule finds the held placeholders and still blocks any other text cited verbatim from that source (regression test for the review's probe q3).
  • after the wording is restored, hsenv.checks runs again on the restored copy in app storage, and any finding blocks
    • the result is cached by the restored copy plus the sourcetext/ contents, and only for completed runs
    • the message is plain en/zh, with no check names and no blame on the publisher
• Run:
  • the restored copy travels with the trusted run code as restored/protocol.json
  • app_run reads it only from beside its own code and plays it only if every held text hashes to its text_sha256, no placeholder is left and nothing else differs
  • it refuses any protocol with placeholder runs and no restored copy
  • records keep no screens; the run-records test checks every file the run leaves in the project
• Private copies:
  • a drop into a published study goes to sources/private/, and an app note in the prompt (not in the researcher's bubble) tells the agent the file was received to fill in the wording and must not be opened
  • the run skill and AGENTS.md forbid opening files in sources/private/ or the app's original-text/ storage
  • research capture has one boundary for private material, redactCapture:
    • every research-v1 event (agent adapter, selection, program.json and artifact versions) and every telemetry metadata object passes it before being stored, put in a blob or sent
    • an event that names a private path, or quotes a run of 40 letters or digits from indexed private text, keeps only its structural fields
    • paths count also when built in pieces, written with ./ segments or URL-encoded; globs and searches under sources/ count in published or private-folder projects
    • the agent adapter also blanks the end event of a tool call that read a private path
    • the skeleton is letters only (CJK included; index format v2, older files are rebuilt), and each text is also checked with line enumerators removed (1., Q1:, (a), Item 3:, bullets), so re-numbered items and a 1-5 column still match. Against an 8-item questionnaire index, 0 of 8 ordinary texts were flagged.
    • an event target that names a withheld file (sources/private/…) is not logged
    • every string value is checked raw, never the event's JSON: also with escape sequences replaced by spaces (strings holding JSON or a heredoc) and with a read tool's line-number prefixes removed, so quotes across lines, tabs or escapes are caught
    • every blob passes the same check: putBlob refuses private text, fit checks before anything goes inline or to a blob, and the workspace context is redacted before it is fitted
  • project files go through mayCollectFile:
    • a published study's run outputs (artifacts/simulations/, runs/) and any file quoting private text are never collected
    • in published or private-folder projects, files that can't be read as text and texts over 300k characters are not collected either; ordinary projects keep collecting their PDFs and figures
  • the per-project index stores hashes only, never text:
    • only what verifiably ships is subtracted: the strings of the protocol.json as received (kept with the pin at first sight) and files whose sha256 a hosted source of that copy records
    • nothing written on this computer counts as public (sources/ outside private/, sources/researcher/, effects.json, the live protocol.json)
    • each file is cached by its sha256 and extracted once
    • the index is written before any download; a drop indexes the copy before importFiles returns, and the chip shows "Adding …"
    • the app merges the sorted per-project files once per session and after each drop, and prunes the index of a folder that no longer exists
  • the agent note matches exact path tokens of the [Attached: …] line
• UI (no new buttons):
  • the Simulation tab words each need in en and zh, mentioning the publisher only for link sources
  • "held list changed" and "checks failed with the wording filled in" have their own heading and button text
  • the preview shows "[text not available]"
• PDF: pypdfium2 is already in the runtime lock (via pdfplumber). Without it, PDF sources say so and the other formats still work.

Follow-up for hsenv 0.8.2 / sandbox skills/run (vendored, not edited here)

• run_study.load_protocol (or hsenv.engine.Protocol.load) should refuse a protocol with a non-empty distribution.held or runs of U+2591, so placeholders cannot reach AI participants by any route. The app already refuses them in app_run.
• checks.check_verbatim should report the paths it counts as unchecked, so the app can excuse exactly the held paths instead of passing a placeholder source text.

Tests

• invented fixture (CC0); the Python suite covers: link source over a local HTTP server, user files, hash mismatch, sign-in page, changed file online, slow server cut off, dead link then the researcher's copy, no PDF reader, other extractor version, held picture, redirects to local addresses, unresolvable host
• node tests:
  • real-engine q3 regression; held-list pin; checks on the restored copy (findings never quote it)
  • published-run pack test: forged, placeholder, stripped-list and environment-variable copies are refused, and no file the run left holds wording
  • preview; telemetry test showing that a dropped private file's text never reaches capture
• npm run typecheck, the full node suite (320 pass, 2 skipped), pack/runtime/tests (574 OK) and scripts/tests: green, with HUMANSTUDY_TEST_PYTHON set to the app runtime's Python

Follow-ups

• Accepted limit of the beta (telemetry is opt-in research data): a quote whose items, each shorter than 40 letters, are reordered, or separated by inserted HTML tags or Likert labels, is not recognised by content. The path rule still covers reading the file itself.
• The index covers the first million letters of each private copy or download; a quote from text past that point is not recognised.
• "Export complete conversation" and "Export my research data" are explicit local saves and are not redacted.

Open questions

• Model calls send the restored wording to the provider through the hosted relay. Is that acceptable under "never uploaded"?
• Held pictures and rendered regions are not supported yet: such studies stay blocked.
• Any pypdfium2 upgrade invalidates published PDF locators.
• An agent that reads a private file anyway: capture is redacted, but the text still reaches the model.

🤖 Generated with Claude Code