Document copy-db's blob companion directory and restore steps

[kriszyp]

Companion to HarperFast/harper#2098 (fixes HarperFast/harper#2048).

copy-db previously left the database's file-backed blobs behind, so a copy moved to another host, or restored under a different database name, silently lost every blob. It now copies each blob root to <target>-blobs/<rootIndex>/ alongside the database file, and refuses a target — or a companion directory — that already exists rather than merging the copy into it.

Documents:

• the blob companion directory, its <rootIndex> layout, and the restore steps (including restoring under a different database name, which is where the blob roots have to be placed by hand), plus that a multi-root database needs every <rootIndex> tree restored into the slot of the same index
• that neither the target path nor its -blobs companion directory may already exist — stricter than earlier v5 releases, so a script re-copying to a fixed path has to remove both first (CLI parameter list) — and that the cleanup a failed copy performs is best effort, since copyDb's catch removes each artifact with .catch(() => {})
• that only Blob values are file-backed (Bytes values live inside the record), and that the roots are <storage.blobPaths[n]>/<database>
• that an unmounted blob root yields a database-file-only copy that still exits 0, and points the reader at the warning copyDatabaseBlobs emits at copy time naming each missing root — the exit status carries none of it, so a scheduled copy checking only the exit code keeps rotating incomplete copies
• that copy compaction is LMDB-only (copy-db fails on a RocksDB source and compact-on-start skips RocksDB), and that copy-db also fails on a database whose tables span more than one storage environment rather than copying one environment and exiting 0 — the question the page raised by being LMDB-only but silent on per-table path configs
• that the backup compactOnStartKeepBackup retains is the pre-compaction database file only, so it is a compaction rollback rather than a point-in-time backup (LMDB get_backup has the same limitation)

Engine scope is carried by <EngineBadge engines="LMDB" /> on Copy Compaction, Compact on Start, and the copy-db CLI entry instead of prose, and Compact on Start now carries a <VersionBadge type="changed" version="v5.3.0" /> for its two new skips.

For the human reviewer

• Do not merge before Fix copy-db producing a silently corrupt, non-restorable database copy harper#2098. These pages publish live — docusaurus.config.ts sets lastVersion: 'current', so reference/ is the published v5 documentation and the version badge labels the change without gating it. Published against 5.2.x, an operator follows these restore steps, gets no blob directory, and may still treat the copy as a working backup.
• The v5.3.0 badge is an inference. #2098 is still open and carries no milestone; the newest core tag is v5.2.6, so the next minor is v5.3.0. If #2098 lands elsewhere, the four <VersionBadge type="changed" version="v5.3.0" /> tags and the three "As of v5.3.0" sentences need updating.
• No release-notes/v5-lincoln/5.3.md exists, so the 5.3 copy-db change has no release-note entry — reviewers flagged this against AGENTS.md in three rounds. Left as-is deliberately: that file covers a whole release, and authoring it from this one change is the release-notes pass's job, not this PR's.
• Two reviewer findings are core behaviors, not docs changes, and are left for Fix copy-db producing a silently corrupt, non-restorable database copy harper#2098: copy-db exits 0 when every blob root is missing on disk (an unmounted blobPaths volume yields a database-file-only copy that a backup script records as success), and the target/companion cleanup after a failed copy is best effort, so a copy whose own cleanup fails still blocks the next run. Both are now documented as behavior; neither is fixed here.
• reference/backups/overview.md contradicts this page on LMDB get_backup and blobs. :8 tells every reader a backup is "all tables, the audit/transaction log, and any file-backed blobs", carving an LMDB exception only for table subsets and audit, and :13 describes get_backup as streaming "a full snapshot" — while compaction.md:88 now says an LMDB get_backup streams the .mdb file only. The page an operator lands on when choosing a backup strategy is the one that is wrong, so the correction belongs in its own issue against backups/overview.md rather than here.

Verification

• npm run build passes. The two broken anchors it reports are pre-existing and in files this PR does not touch (backups/overview, release-notes/v5-lincoln/5.1).
• npm run format:check clean.
• Every claim checked against Fix copy-db producing a silently corrupt, non-restorable database copy harper#2098's bin/copyDb.ts (target and companion existence guards, the rootStores.size > 1 throw, the RocksDB rejection, the best-effort cleanup in the catch, the compact-on-start skips) and dataLayer/blobBackup.ts (copyBlobRootsByIndex, copyTree's missing-root no-op, the README.md mapping listing every configured root), plus resources/blob.ts getBlobPathsForDatabaseName. main does not carry this code.
• The mkdir -p added to the restore example is load-bearing: cp -r src/. dest/ creates one missing directory level but fails with two, which is the case for a database whose blob root does not exist yet.

Refs HarperFast/harper#2048

Generated by Claude Opus 5.

_{Review-Coverage: authored=claude; ran=gemini,codex; declined=cursor-grok,cursor-composer,domain; rounds=8 @ 04536bd}

_{Human-Review-Need: 4 @ 04536bd}

[gemini-code-assist]

Code Review

This pull request updates the documentation for the harper copy-db command and database compaction, adding details about file-backed blobs, target path constraints, and database engine applicability (LMDB vs. RocksDB). The review feedback suggests adding <VersionBadge> elements to properly document these behavior changes on existing surfaces, following repository guidelines.

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-620

This preview will update automatically when you push new commits.

[DavidCockerill]

Approving. No findings.

"The copy is not restorable without this directory" in bold is the sentence that matters, and it is not hedged. Anyone who took a copy-db backup before this was written could reasonably believe they hold a restorable copy and not.

The <rootIndex> explanation earns its length: preserving the source root's position is what lets a multi-root database restore each root to its original slot, and writing a README.md into the directory recording the mapping means the copy carries its own restore instructions rather than depending on someone finding this page later. Showing the restore under a different database name is also the case people actually hit.

The LMDB-only scope appears in both the CLI reference and the compaction page, so a reader arriving from either gets it.

_{— DAIvid (Claude Opus 5)}

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-620

This preview will update automatically when you push new commits.

[Ethan-Arrowood]

@kriszyp fyi you have some ghost commits in here! Not a big deal cause I'll squash merge anyways but just calling out incase.

[Ethan-Arrowood]

I'm going to review this myself shortly, but rerequesting reviews since additional commits were pushed since the previous approvals. I thought GH was supposed to do this automatically; maybe thats buried in a setting somewhere.

[DavidCockerill]

Approving the content — but I'd hold the merge until HarperFast/harper#2098 lands.

🧊 In plain terms: this page documents a copy-db that copies your blob files alongside the database, and gives restore steps that depend on that. That version of copy-db isn't released yet. Published as-is, someone on 5.2.x follows the steps, gets no blob directory, and may still treat the copy as a working backup — losing every blob at restore time, which is the exact bug the unmerged fix exists to stop.

Why I think the sequencing matters more than usual here rather than being the normal docs-ahead-of-release case:

• harper#2098 is still open, the newest core tag is v5.2.6, and no -blobs logic exists on main in harper or harper-pro.
• These pages publish live: docusaurus.config.ts sets lastVersion: 'current' with current: { label: 'v5' }, so reference/ is the published v5 documentation. The <VersionBadge version="v5.3.0" /> labels the change but doesn't gate it.
• I checked the merge base: this is the first mention of blobs on either page, so it introduces the claim rather than refining one.

The content itself is accurate, and I verified it against #2098's implementation rather than against main, since main doesn't have it: the target-and-companion-directory refusal, compact-on-start skipping RocksDB and skipping multi-environment databases, and the existsSync filtering of blob roots all check out. The cp -r → cp correction on the .mdb file is right too.

One substantive addition. On the unmounted-blob-root case, the page tells readers to confirm <target-database-path>-blobs exists afterwards. #2098 actually warns at copy time, naming the missing roots and stating plainly that the copy is missing those blobs. Pointing readers at that warning is more actionable than post-hoc inspection, and it turns a silent-ish trap into something the command tells you about.

Two threads inline: a gap on multi-environment databases, and a cross-page contradiction that wants its own issue.

Three nits, take or leave: Compact on Start gains two v5.3.0 behaviour changes with no <VersionBadge type="changed"> while the other two sections this diff touches carry one; engine scope is prose where the repo has <EngineBadge engines="LMDB" /> and the backups pages use it; and the unmounted-root case is the highest-stakes point on the page but sits in plain prose while lower-stakes material gets > **Note:** callouts.

_{— DAIvid (Claude Opus 5) · cross-model: Codex (graded) + Gemini + Harper domain adjudication · core-behaviour claims traced against harper#2098, not main}

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-620

This preview will update automatically when you push new commits.