From 81ceb345ce4ca0e48e537a239951f9110fa78751 Mon Sep 17 00:00:00 2001 From: Zixuan Chen Date: Thu, 22 Oct 2026 12:46:24 +0800 Subject: [PATCH 1/2] docs: explain pending import persistence Model: gpt-6.1-sol --- pages/docs/api/js.mdx | 34 +++++++++- pages/docs/concepts/import_status.mdx | 49 ++++++++++---- pages/docs/tutorial/encoding.mdx | 11 ++-- pages/docs/tutorial/get_started.mdx | 6 ++ pages/docs/tutorial/persistence.mdx | 95 ++++++++++++++++++++++++++- pages/docs/tutorial/sync.mdx | 64 ++++++++++-------- 6 files changed, 209 insertions(+), 50 deletions(-) diff --git a/pages/docs/api/js.mdx b/pages/docs/api/js.mdx index 120d125..627a018 100644 --- a/pages/docs/api/js.mdx +++ b/pages/docs/api/js.mdx @@ -596,7 +596,8 @@ type ExportMode = **⚠️ Important Notes:** - **Shallow snapshots**: Cannot import updates from before the shallow start point. Peers can only sync if they have versions after this point. -- **Auto-commit**: The document automatically commits pending operations before export. +- **Auto-commit**: The document automatically commits local uncommitted operations before export. Remote pending imports still require their missing causal dependencies. +- **Remote pending updates**: Snapshot and update exports exclude imported operations waiting for missing causal dependencies. See [`exportPendingUpdates()`](#LoroDoc.exportPendingUpdates) to persist them separately. - **Performance**: Export new snapshots periodically to reduce import times for new peers. **Examples:** @@ -627,6 +628,35 @@ const shallowSnapshot = doc.export({ + +```typescript no_run +exportPendingUpdates(): Uint8Array | undefined +``` + + +**Availability: unreleased.** This method is available in the development WASM and pure TypeScript implementations. It is not available in `loro-crdt@1.13.3`; use a build containing this API. + +Exports imported remote operations that are waiting for missing causal predecessors. These are [pending imports](/docs/concepts/import_status#pending-operations), distinct from uncommitted local edits. Ordinary snapshot and update exports do not include them. + +**Returns:** `undefined` when no remote operations remain pending; otherwise, binary updates in the existing FastUpdates format. Pass these bytes to ordinary [`import()`](#LoroDoc.import) to restore them. The bytes preserve the pending operations, but need not match the original received blobs byte for byte. + +**Behavior and limits:** + +- Read-only: does not consume the pending queue, commit a local transaction, emit events, or advance the applied version vector (`appliedVV`) or frontiers. +- Persist a normal snapshot and pending bytes as one consistent checkpoint. Prevent document mutations/imports between the two exports; the calls are not atomic. When the result is `undefined`, clear any older stored pending record. See the [persistence example](/docs/tutorial/persistence#persisting-pending-imports-unreleased). +- Restore with `doc.import(snapshot)`, then `doc.import(pending)` if pending bytes exist. Operations still wait for their actual missing predecessors; the API cannot fetch or reconstruct predecessors never received from a remote peer. +- Exporting pending bytes, or receiving `undefined`, does not prove the document is synced. Transport cursors, durable storage, and removal of original updates remain the application's responsibility. + +The corresponding Rust method is: + +```rust +LoroDoc::export_pending_updates(&self) -> Result>, LoroEncodeError> +``` + +`Ok(None)` corresponds to `undefined`; an export error must not be treated as an empty queue. See the [Rust API entry point](https://docs.rs/loro/latest/loro/struct.LoroDoc.html) (the published reference may not list this unreleased method yet) and the separate [FFI bindings](https://github.com/loro-dev/loro-ffi). Check availability in the binding you use; this development API does not imply an FFI release. + + + ```typescript no_run import(data: Uint8Array): ImportStatus @@ -639,7 +669,7 @@ Imports updates or snapshots into the document. Returns an `ImportStatus` descri - `data` - Binary data or another LoroDoc to import from -**⚠️ Important:** LoroDoc will automatically commits pending operations before import. If the doc is in detached mode, the imported operations are recorded into OpLog but not applied to DocState until you call `attach()`, see [Attached vs Detached States](/docs/concepts/attached_detached) adn [OpLog and DocState](/docs/concepts/oplog_docstate). +**⚠️ Important:** LoroDoc automatically commits local uncommitted operations before import. Remote pending operations can apply only when their missing dependencies arrive. If the doc is in detached mode, the imported operations are recorded into OpLog but not applied to DocState until you call `attach()`, see [Attached vs Detached States](/docs/concepts/attached_detached) and [OpLog and DocState](/docs/concepts/oplog_docstate). **Example:** diff --git a/pages/docs/concepts/import_status.mdx b/pages/docs/concepts/import_status.mdx index ddc38b9..4ddbb6c 100644 --- a/pages/docs/concepts/import_status.mdx +++ b/pages/docs/concepts/import_status.mdx @@ -8,27 +8,44 @@ ```ts twoslash interface ImportStatus { - success: PeerVersionRange; // What was applied - pending?: PeerVersionRange; // What needs dependencies + success: Map; // What was integrated into history + pending: Map | null; // What still needs dependencies } -interface PeerVersionRange { - [peerId: `${number}`]: { - start: number; // Inclusive - end: number; // Exclusive (e.g., 0-50 = ops 0-49) - }; -} +type PeerID = `${number}`; +type CounterSpan = { + start: number; // Inclusive + end: number; // Exclusive (e.g., 0-50 = ops 0-49) +}; ``` +Both fields are present in the WASM and pure TypeScript implementations. +`pending` is `null` when the import reports no pending operations. These ranges +are JavaScript `Map` values: use `.size`, `.get(peerId)`, or iteration; +`Object.keys()` does not enumerate Map entries. + +`ImportStatus` reports an import result, not a complete query of the document's +pending queue; `pending === null` does not guarantee that no earlier imports +remain pending. Use the **unreleased** [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) +to export all currently received but unintegrated operations; `undefined` means +that queue is empty, not that the document is synced. + ## Pending Operations Operations become pending when they depend on missing operations (causal dependencies). +These are imported remote operations, not local edits waiting for `commit()`. +Normal snapshot and update exports exclude them. The unreleased +[`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) API exports +them separately for persistence; see the [checkpoint and recovery example](/docs/tutorial/persistence#persisting-pending-imports-unreleased). +Restoring those bytes with ordinary `import()` preserves the pending operations, +but they apply only after their actual missing dependencies arrive. ### Common Scenario: Out-of-Order Delivery ```ts twoslash import { LoroDoc } from "loro-crdt"; +import { expect } from "expect"; // ---cut--- const doc = new LoroDoc(); @@ -36,16 +53,22 @@ const doc = new LoroDoc(); const peerA = new LoroDoc(); peerA.getText("text").insert(0, "Hello"); const update1 = peerA.export({ mode: "update" }); +const afterUpdate1 = peerA.oplogVersion(); peerA.getText("text").insert(5, " World"); -const update2 = peerA.export({ mode: "update", from: peerA.version() }); +const update2 = peerA.export({ mode: "update", from: afterUpdate1 }); // If update2 arrives first: const status = doc.import(update2); -console.log(status.pending); // Ops 10-19 pending, need 0-9 +expect(status.success).toBeInstanceOf(Map); +expect(status.pending).toBeInstanceOf(Map); +expect(status.pending?.size).toBeGreaterThan(0); // Ops 5-10 need ops 0-4. +expect(doc.getText("text").toString()).toBe(""); // Import missing dependencies: -doc.import(update1); // Both updates now applied +const completed = doc.import(update1); // Both updates now applied +expect(completed.pending).toBeNull(); +expect(doc.getText("text").toString()).toBe("Hello World"); ``` ## Handling Pending Operations @@ -60,7 +83,7 @@ async function handleImport( ) { const status = doc.import(update); - if (status.pending) { + if (status.pending !== null && status.pending.size > 0) { const missing = await fetchMissing(doc.version()); doc.import(missing); } @@ -78,7 +101,7 @@ const update = new Uint8Array(); // ---cut--- const doc = new LoroDoc(); const status = doc.import(update); -if (status.pending) { +if (status.pending !== null && status.pending.size > 0) { console.warn("Operations pending:", status.pending); // Fetch missing updates } diff --git a/pages/docs/tutorial/encoding.mdx b/pages/docs/tutorial/encoding.mdx index 7526e7e..88487e3 100644 --- a/pages/docs/tutorial/encoding.mdx +++ b/pages/docs/tutorial/encoding.mdx @@ -12,8 +12,9 @@ Loro introduces three encoding modes to meet the needs for different use cases: - **Updates Encoding**: Encodes all or a specified range of operations, mainly used for delta updates of documents. -- **Snapshot Encoding**: Encodes the entire document, including both all the - operations and the current document state. +- **Snapshot Encoding**: Encodes the document's integrated operation history and + current state. Imported operations waiting for missing dependencies are excluded; + see [pending-import persistence](./persistence#persisting-pending-imports-unreleased). - **Shallow Snapshot Encoding**: Loro 1.0 provides a new encoding format for discarding useless historical operations. It is a special snapshot that encodes the most recent historical operations, the starting state of the @@ -87,8 +88,10 @@ doc1.import(bytes); console.log(doc1.toJSON()); // { text: "hello" } ``` -Snapshot Encoding comprehensively captures a document's current state and its -historical operations. This mode can quickly obtain the latest document state. +Snapshot Encoding captures a document's current state and integrated operation +history for fast loading. Like update exports, it excludes imported operations +waiting for missing causal dependencies. See [pending-import persistence](./persistence#persisting-pending-imports-unreleased) +to preserve those operations across restarts. ## Shallow Snapshot Encoding diff --git a/pages/docs/tutorial/get_started.mdx b/pages/docs/tutorial/get_started.mdx index 6bace6b..a60eb7b 100644 --- a/pages/docs/tutorial/get_started.mdx +++ b/pages/docs/tutorial/get_started.mdx @@ -275,6 +275,9 @@ Loro is a pure library and does not handle network protocols or storage mechanis To save the document, use `doc.export({mode: "snapshot"})` to get its binary form. To open it again, use `doc.import(data)` to load this binary data. +Snapshots exclude imported operations waiting for missing dependencies. See +[pending-import persistence](./persistence#persisting-pending-imports-unreleased) to preserve them across restarts. + ```ts twoslash import { LoroDoc, LoroList, LoroMap, LoroText } from "loro-crdt"; import { expect } from "expect"; @@ -348,6 +351,9 @@ const update1 = doc.export({ mode: "update", from: lastSavedVersion }); If updates accumulate, exporting a new snapshot can quicken import times and decrease the overall size of the exported data. +Before discarding the original updates, durably store everything needed for +recovery, including pending imports; see [persistence](./persistence#persisting-pending-imports-unreleased). + You can store the binary data exported from Loro wherever you prefer. ## Sync diff --git a/pages/docs/tutorial/persistence.mdx b/pages/docs/tutorial/persistence.mdx index 0fcf198..f5d01bc 100644 --- a/pages/docs/tutorial/persistence.mdx +++ b/pages/docs/tutorial/persistence.mdx @@ -8,6 +8,11 @@ operations. Here's how it works: 3. Implement a debounce or throttle mechanism to trigger snapshot saves after a certain number of edits. +Snapshots contain integrated operation history and state, but exclude imported operations +waiting for missing causal dependencies. If your application receives such +updates, retain their original bytes or use the +[pending-import checkpoint](#persisting-pending-imports-unreleased) described below. + This method simplifies initial application development but has a drawback: user edits are not immediately saved. Let's explore how to quickly save each user edit while minimizing resource consumption. @@ -24,12 +29,98 @@ To achieve both quick saves and resource efficiency: - Loro validates the checksum embedded in every snapshot/update header during import, so corrupted binary payloads (for example, from bit flips in queues or object storage) are rejected before they touch your document state. - When loading a document, import the snapshot and all related updates to get the latest version. -- After importing, export a new snapshot to replace the old one and remove - imported updates for faster future loading. +- After importing, export a new snapshot for faster future loading. Remove + original updates only after durable storage contains everything needed to + recover them. A snapshot alone does not preserve pending imports: retain their + original bytes, or persist the snapshot together with pending bytes as below. - If your `LoroDoc` has grown large and older history can be safely recycled, use `Shallow Snapshot Encoding` to reduce snapshot size. You can archive the history before the shallow snapshot's start version in cold storage. +### Persisting Pending Imports (Unreleased) + +[`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) is an +**unreleased** API in the development WASM and pure TypeScript implementations. +The example requires a build containing it; `loro-crdt@1.13.3` does not have it. +Here, pending means remote operations already imported but waiting for causal +predecessors, not local edits waiting for `commit()`. + +Capture a normal snapshot and the pending bytes without allowing document +mutations or imports between the two calls. Save both as one consistent +checkpoint, using a storage transaction or a versioned generation. Snapshot +export commits local edits as usual; pending export itself is read-only and does +not consume the queue, commit edits, or advance the applied version vector. +Replace the previous checkpoint only after both exports succeed. If the pending +result is `undefined`, delete the previous pending record or explicitly store its +absence; skipping the write would leave stale bytes behind. An export error is +not an empty queue. + +This self-contained example delivers B before its dependency A, simulates a +restart, then imports the real A. The `saved` object represents one checkpoint; +an application must write it to durable storage. + +```ts no_run +import { LoroDoc } from "loro-crdt"; + +function assert(condition: unknown, message: string): asserts condition { + if (!condition) throw new Error(message); +} + +const source = new LoroDoc(); +source.getText("text").insert(0, "base|"); +const base = source.export({ mode: "snapshot" }); +const beforeA = source.oplogVersion(); +source.getText("text").insert(5, "A"); +const updateA = source.export({ mode: "update", from: beforeA }); +const beforeB = source.oplogVersion(); +source.getText("text").insert(6, "B"); +const updateB = source.export({ mode: "update", from: beforeB }); + +const receiver = new LoroDoc(); +receiver.import(base); +const status = receiver.import(updateB); // A has not arrived. +assert(status.pending instanceof Map && status.pending.size > 0, "B is pending"); +assert(receiver.getText("text").toString() === "base|", "B cannot apply yet"); + +// Do not interleave document mutations/imports between these exports. +let saved = { + snapshot: receiver.export({ mode: "snapshot" }), + pending: receiver.exportPendingUpdates(), +}; +assert(saved.pending !== undefined, "Checkpoint must preserve B"); +assert(receiver.exportPendingUpdates() !== undefined, "Export does not consume B"); +// Durably replace BOTH fields in one storage transaction/generation here. + +const snapshotOnly = new LoroDoc(); +snapshotOnly.import(saved.snapshot); +assert(snapshotOnly.exportPendingUpdates() === undefined, "Snapshot excludes B"); + +const restored = new LoroDoc(); +restored.import(saved.snapshot); +restored.import(saved.pending); // Ordinary import; no special restore API. +assert(restored.getText("text").toString() === "base|", "B still needs A"); +assert(restored.exportPendingUpdates() !== undefined, "B survives restart"); + +restored.import(updateA); // The actual missing predecessor finally arrives. +assert(restored.getText("text").toString() === "base|AB", "A unblocks B"); +assert(restored.exportPendingUpdates() === undefined, "No pending operations remain"); + +saved = { + snapshot: restored.export({ mode: "snapshot" }), + pending: restored.exportPendingUpdates(), +}; +assert(saved.pending === undefined, "Clear the older pending record when saving"); +// Durably replace the checkpoint, including removing the old pending field. +``` + +The snapshot and pending bytes restore what was received; they cannot supply +predecessors that the remote peer never sent. Even an empty pending queue does +not prove synchronization is complete. Advancing a durable receive cursor or +discarding original update bytes requires that everything needed for recovery is +durably stored first, whether as original updates or a snapshot plus pending bytes. +This API neither defines nor advances a transport cursor; your storage and +transport protocol must define that boundary. + For collaboration, the binary data generated by snapshot/updates can be transmitted through any medium, such as WebRTC, WebSocket, or HTTP. diff --git a/pages/docs/tutorial/sync.mdx b/pages/docs/tutorial/sync.mdx index 05bc174..dde4071 100644 --- a/pages/docs/tutorial/sync.mdx +++ b/pages/docs/tutorial/sync.mdx @@ -112,27 +112,26 @@ doc1.subscribeLocalUpdates((update) => { ## Understanding the `import()` Return Value -The `import()` method in Loro's JavaScript/WASM binding returns an object that provides feedback on the import operation. This object, let's call it `ImportStatusJS`, has the following structure: +The `import()` method in both Loro's WASM binding and pure TypeScript implementation returns an `ImportStatus` with the following structure: ```ts no_run twoslash -interface ImportStatusJS { - success: PeerVersionRange; - pending?: PeerVersionRange; // Optional: only present if there are pending operations +interface ImportStatus { + success: Map; + pending: Map | null; } -interface PeerVersionRange { - [peerId: string]: { - start: number; // Start counter (inclusive) - end: number; // End counter (exclusive) - }; -} +type PeerID = `${number}`; +type CounterSpan = { + start: number; // Start counter (inclusive) + end: number; // End counter (exclusive) +}; ``` ### Fields Explained: -1. **`success`** (Object, `PeerVersionRange`) - - **Description**: This field is always present and details the ranges of operations (changes) that were successfully imported and applied to the Loro document. - - **Structure**: It's an object where: +1. **`success`** (`Map`) + - **Description**: This field is always present and details the ranges of operations integrated into history by this import, including earlier pending operations whose dependencies just arrived. + - **Structure**: It's a JavaScript `Map` where: - Each **key** is a `string` representing a `PeerID` (the unique identifier of a collaborator or a source of changes). - Each **value** is an object `{ start: number, end: number }` defining a continuous range of operation counters for that specific peer. - `start`: The starting counter of the successfully imported range (inclusive). @@ -143,34 +142,41 @@ interface PeerVersionRange { // Assuming importResult is the return value of doc.import(bytes) console.log(importResult.success); // Example output: - // { - // "clientA_peerId": { "start": 0, "end": 50 }, - // "server_peerId": { "start": 120, "end": 150 } + // Map(2) { + // "1" => { start: 0, end: 50 }, + // "2" => { start: 120, end: 150 } // } - // This means operations from clientA (counters 0-49) and - // operations from server (counters 120-149) were successfully imported. + // This means operations from peer "1" (counters 0-49) and + // peer "2" (counters 120-149) were successfully imported. ``` -2. **`pending`** (Object, `PeerVersionRange`, optional) - - **Description**: This field is only present if some operations from the imported data could not be applied because they depend on other operations that Loro has not seen yet (i.e., their causal dependencies are missing). It details these "pending" operation ranges. - - **Structure**: Identical to the `success` field. An object mapping `PeerID` strings to `{ start: number, end: number }` counter ranges. +2. **`pending`** (`Map | null`) + - **Description**: This field is always present. It is a `Map` of operation ranges waiting for missing causal dependencies, or `null` when the import reports no pending operations. + - **Structure**: When non-null, a JavaScript `Map` from `PeerID` strings to `{ start: number, end: number }` counter ranges, like `success`. - **Purpose**: Informs the application that certain changes are known but are "on hold" awaiting their prerequisites. To apply these pending changes, the missing prerequisite operations must be imported first. This is crucial for maintaining data consistency in collaborative scenarios. - **Example**: ```javascript // Assuming importResult is the return value of doc.import(bytes) - if (importResult.pending) { + if (importResult.pending !== null && importResult.pending.size > 0) { console.log(importResult.pending); // Example output: - // { - // "clientA_peerId": { "start": 50, "end": 60 }, - // "clientB_peerId": { "start": 10, "end": 25 } + // Map(2) { + // "1" => { start: 50, end: 60 }, + // "2" => { start: 10, end: 25 } // } - // This means operations from clientA (counters 50-59) and - // operations from clientB (counters 10-24) are pending due to missing dependencies. + // This means operations from peer "1" (counters 50-59) and + // peer "2" (counters 10-24) are pending due to missing dependencies. } ``` +`ImportStatus` reports an import result, not a complete query of the document's +pending queue; `pending === null` does not guarantee that no earlier imports +remain pending. Use the **unreleased** [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) +to export all currently received but unintegrated operations; `undefined` means +that queue is empty, not that the document is synced. + ### How to Use This Information: -- Check the `success` field to confirm which updates were applied. -- If the `pending` field exists and is not empty, it signals that further updates (dependencies) are required to fully integrate all known changes. Your application might need to fetch or request these missing updates from other peers or a central server. +- Use `success.get(peerId)` or iterate over `success` to inspect integrated ranges; use `.size` to check whether a Map is empty. `Object.keys()` does not enumerate Map entries. +- If `pending !== null && pending.size > 0`, further updates (dependencies) are required to integrate those pending operations. Request these missing updates from other peers or a central server. +- To preserve pending imports across restarts, see [persisting pending imports](/docs/tutorial/persistence#persisting-pending-imports-unreleased) with the unreleased [`exportPendingUpdates()` API](/docs/api/js#LoroDoc.exportPendingUpdates). A normal snapshot alone excludes these operations; saving them does not establish that the document is synced. From d5adccad15c7daa20b5e87f4def908fda3cf67ac Mon Sep 17 00:00:00 2001 From: Zixuan Chen Date: Thu, 22 Oct 2026 20:56:14 +0800 Subject: [PATCH 2/2] docs: mark pending export API as released Document WASM 1.17.0, pure TypeScript 0.4.0 and Rust 1.17.0 availability; update persistence anchors and keep language binding releases separate. Model: gpt-6.1-sol --- pages/docs/api/js.mdx | 6 +++--- pages/docs/concepts/import_status.mdx | 6 +++--- pages/docs/tutorial/encoding.mdx | 4 ++-- pages/docs/tutorial/get_started.mdx | 4 ++-- pages/docs/tutorial/persistence.mdx | 10 +++++----- pages/docs/tutorial/sync.mdx | 4 ++-- 6 files changed, 17 insertions(+), 17 deletions(-) diff --git a/pages/docs/api/js.mdx b/pages/docs/api/js.mdx index 627a018..57ff228 100644 --- a/pages/docs/api/js.mdx +++ b/pages/docs/api/js.mdx @@ -634,7 +634,7 @@ exportPendingUpdates(): Uint8Array | undefined ``` -**Availability: unreleased.** This method is available in the development WASM and pure TypeScript implementations. It is not available in `loro-crdt@1.13.3`; use a build containing this API. +**Availability:** [`loro-crdt@1.17.0`](https://www.npmjs.com/package/loro-crdt/v/1.17.0) (WASM) and [`loro.js@0.4.0`](https://www.npmjs.com/package/loro.js/v/0.4.0) (pure TypeScript), and later versions. Exports imported remote operations that are waiting for missing causal predecessors. These are [pending imports](/docs/concepts/import_status#pending-operations), distinct from uncommitted local edits. Ordinary snapshot and update exports do not include them. @@ -643,7 +643,7 @@ Exports imported remote operations that are waiting for missing causal predecess **Behavior and limits:** - Read-only: does not consume the pending queue, commit a local transaction, emit events, or advance the applied version vector (`appliedVV`) or frontiers. -- Persist a normal snapshot and pending bytes as one consistent checkpoint. Prevent document mutations/imports between the two exports; the calls are not atomic. When the result is `undefined`, clear any older stored pending record. See the [persistence example](/docs/tutorial/persistence#persisting-pending-imports-unreleased). +- Persist a normal snapshot and pending bytes as one consistent checkpoint. Prevent document mutations/imports between the two exports; the calls are not atomic. When the result is `undefined`, clear any older stored pending record. See the [persistence example](/docs/tutorial/persistence#persisting-pending-imports). - Restore with `doc.import(snapshot)`, then `doc.import(pending)` if pending bytes exist. Operations still wait for their actual missing predecessors; the API cannot fetch or reconstruct predecessors never received from a remote peer. - Exporting pending bytes, or receiving `undefined`, does not prove the document is synced. Transport cursors, durable storage, and removal of original updates remain the application's responsibility. @@ -653,7 +653,7 @@ The corresponding Rust method is: LoroDoc::export_pending_updates(&self) -> Result>, LoroEncodeError> ``` -`Ok(None)` corresponds to `undefined`; an export error must not be treated as an empty queue. See the [Rust API entry point](https://docs.rs/loro/latest/loro/struct.LoroDoc.html) (the published reference may not list this unreleased method yet) and the separate [FFI bindings](https://github.com/loro-dev/loro-ffi). Check availability in the binding you use; this development API does not imply an FFI release. +`Ok(None)` corresponds to `undefined`; an export error must not be treated as an empty queue. See [`LoroDoc::export_pending_updates` in Rust 1.17.0](https://docs.rs/loro/1.17.0/loro/struct.LoroDoc.html#method.export_pending_updates). The separate [FFI binding source](https://github.com/loro-dev/loro-ffi/pull/17) exposes the corresponding method; check the release of the language binding you use. A core release does not publish downstream binding packages. diff --git a/pages/docs/concepts/import_status.mdx b/pages/docs/concepts/import_status.mdx index 4ddbb6c..810a217 100644 --- a/pages/docs/concepts/import_status.mdx +++ b/pages/docs/concepts/import_status.mdx @@ -26,7 +26,7 @@ are JavaScript `Map` values: use `.size`, `.get(peerId)`, or iteration; `ImportStatus` reports an import result, not a complete query of the document's pending queue; `pending === null` does not guarantee that no earlier imports -remain pending. Use the **unreleased** [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) +remain pending. Use [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) to export all currently received but unintegrated operations; `undefined` means that queue is empty, not that the document is synced. @@ -35,9 +35,9 @@ that queue is empty, not that the document is synced. Operations become pending when they depend on missing operations (causal dependencies). These are imported remote operations, not local edits waiting for `commit()`. -Normal snapshot and update exports exclude them. The unreleased +Normal snapshot and update exports exclude them. The [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) API exports -them separately for persistence; see the [checkpoint and recovery example](/docs/tutorial/persistence#persisting-pending-imports-unreleased). +them separately for persistence; see the [checkpoint and recovery example](/docs/tutorial/persistence#persisting-pending-imports). Restoring those bytes with ordinary `import()` preserves the pending operations, but they apply only after their actual missing dependencies arrive. diff --git a/pages/docs/tutorial/encoding.mdx b/pages/docs/tutorial/encoding.mdx index 88487e3..5fc3bd1 100644 --- a/pages/docs/tutorial/encoding.mdx +++ b/pages/docs/tutorial/encoding.mdx @@ -14,7 +14,7 @@ Loro introduces three encoding modes to meet the needs for different use cases: used for delta updates of documents. - **Snapshot Encoding**: Encodes the document's integrated operation history and current state. Imported operations waiting for missing dependencies are excluded; - see [pending-import persistence](./persistence#persisting-pending-imports-unreleased). + see [pending-import persistence](./persistence#persisting-pending-imports). - **Shallow Snapshot Encoding**: Loro 1.0 provides a new encoding format for discarding useless historical operations. It is a special snapshot that encodes the most recent historical operations, the starting state of the @@ -90,7 +90,7 @@ console.log(doc1.toJSON()); // { text: "hello" } Snapshot Encoding captures a document's current state and integrated operation history for fast loading. Like update exports, it excludes imported operations -waiting for missing causal dependencies. See [pending-import persistence](./persistence#persisting-pending-imports-unreleased) +waiting for missing causal dependencies. See [pending-import persistence](./persistence#persisting-pending-imports) to preserve those operations across restarts. ## Shallow Snapshot Encoding diff --git a/pages/docs/tutorial/get_started.mdx b/pages/docs/tutorial/get_started.mdx index a60eb7b..0d10504 100644 --- a/pages/docs/tutorial/get_started.mdx +++ b/pages/docs/tutorial/get_started.mdx @@ -276,7 +276,7 @@ To save the document, use `doc.export({mode: "snapshot"})` to get its binary form. To open it again, use `doc.import(data)` to load this binary data. Snapshots exclude imported operations waiting for missing dependencies. See -[pending-import persistence](./persistence#persisting-pending-imports-unreleased) to preserve them across restarts. +[pending-import persistence](./persistence#persisting-pending-imports) to preserve them across restarts. ```ts twoslash import { LoroDoc, LoroList, LoroMap, LoroText } from "loro-crdt"; @@ -352,7 +352,7 @@ If updates accumulate, exporting a new snapshot can quicken import times and decrease the overall size of the exported data. Before discarding the original updates, durably store everything needed for -recovery, including pending imports; see [persistence](./persistence#persisting-pending-imports-unreleased). +recovery, including pending imports; see [persistence](./persistence#persisting-pending-imports). You can store the binary data exported from Loro wherever you prefer. diff --git a/pages/docs/tutorial/persistence.mdx b/pages/docs/tutorial/persistence.mdx index f5d01bc..aabe71d 100644 --- a/pages/docs/tutorial/persistence.mdx +++ b/pages/docs/tutorial/persistence.mdx @@ -11,7 +11,7 @@ operations. Here's how it works: Snapshots contain integrated operation history and state, but exclude imported operations waiting for missing causal dependencies. If your application receives such updates, retain their original bytes or use the -[pending-import checkpoint](#persisting-pending-imports-unreleased) described below. +[pending-import checkpoint](#persisting-pending-imports) described below. This method simplifies initial application development but has a drawback: user edits are not immediately saved. Let's explore how to quickly save each user @@ -37,11 +37,11 @@ To achieve both quick saves and resource efficiency: use `Shallow Snapshot Encoding` to reduce snapshot size. You can archive the history before the shallow snapshot's start version in cold storage. -### Persisting Pending Imports (Unreleased) +### Persisting Pending Imports -[`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) is an -**unreleased** API in the development WASM and pure TypeScript implementations. -The example requires a build containing it; `loro-crdt@1.13.3` does not have it. +[`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) is available +since `loro-crdt@1.17.0` (WASM) and `loro.js@0.4.0` (pure TypeScript). The example +below imports the WASM package; use `"loro.js"` for the pure TypeScript implementation. Here, pending means remote operations already imported but waiting for causal predecessors, not local edits waiting for `commit()`. diff --git a/pages/docs/tutorial/sync.mdx b/pages/docs/tutorial/sync.mdx index dde4071..73b6d76 100644 --- a/pages/docs/tutorial/sync.mdx +++ b/pages/docs/tutorial/sync.mdx @@ -171,7 +171,7 @@ type CounterSpan = { `ImportStatus` reports an import result, not a complete query of the document's pending queue; `pending === null` does not guarantee that no earlier imports -remain pending. Use the **unreleased** [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) +remain pending. Use [`exportPendingUpdates()`](/docs/api/js#LoroDoc.exportPendingUpdates) to export all currently received but unintegrated operations; `undefined` means that queue is empty, not that the document is synced. @@ -179,4 +179,4 @@ that queue is empty, not that the document is synced. - Use `success.get(peerId)` or iterate over `success` to inspect integrated ranges; use `.size` to check whether a Map is empty. `Object.keys()` does not enumerate Map entries. - If `pending !== null && pending.size > 0`, further updates (dependencies) are required to integrate those pending operations. Request these missing updates from other peers or a central server. -- To preserve pending imports across restarts, see [persisting pending imports](/docs/tutorial/persistence#persisting-pending-imports-unreleased) with the unreleased [`exportPendingUpdates()` API](/docs/api/js#LoroDoc.exportPendingUpdates). A normal snapshot alone excludes these operations; saving them does not establish that the document is synced. +- To preserve pending imports across restarts, see [persisting pending imports](/docs/tutorial/persistence#persisting-pending-imports) with the [`exportPendingUpdates()` API](/docs/api/js#LoroDoc.exportPendingUpdates). A normal snapshot alone excludes these operations; saving them does not establish that the document is synced.