diff --git a/AGENTS.md b/AGENTS.md index 91f574a..d371703 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,7 +48,7 @@ pnpm clean ### Protocol Overview - **CRDT magic bytes**: `%LOR` (Loro), `%EPH` (Ephemeral), `%YJS`, `%YAW`, `%ELO` (E2EE Loro). -- **Messages**: JoinRequest/JoinResponseOk/JoinError, DocUpdate, DocUpdateFragmentHeader/Fragment, UpdateError, Leave. +- **Messages**: JoinRequest/JoinResponseOk/JoinError, DocUpdate (with batchId), DocUpdateFragmentHeader/Fragment, Ack, RoomError, Leave. - **Limits**: 256 KiB max per message; large payloads are fragmented and reassembled. - **Keepalive**: Text frames `"ping"`/`"pong"` are connection‑scoped and bypass the envelope. - **%ELO**: DocUpdate payload is a container of encrypted records (DeltaSpan/Snapshot). Each record has a plaintext header (peer/version metadata, `keyId`, 12‑byte IV) and AES‑GCM ciphertext (`ct||tag`). Servers route/broadcast without decrypting. diff --git a/CLAUDE.md b/CLAUDE.md index ad54979..7773d3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -32,7 +32,7 @@ pnpm -r clean Source of truth: `/protocol.md`. - Envelope: 4‑byte CRDT magic, varBytes roomId (≤128B), 1‑byte type, payload -- Types: JoinRequest/JoinResponseOk/JoinError, DocUpdate, DocUpdateFragmentHeader/Fragment, UpdateError, Leave +- Types: JoinRequest/JoinResponseOk/JoinError, DocUpdate (with batchId), DocUpdateFragmentHeader/Fragment, Ack, RoomError, Leave - Limits: 256 KiB per message; fragment large updates - Keepalive: connection‑scoped text frames "ping"/"pong" @@ -45,7 +45,7 @@ Key TS files: `packages/loro-protocol/src/{bytes,encoding,protocol}.ts`. ## Notes for Implementers -- Fragmentation: reassemble fragments by batch header + index; timeout triggers UpdateError.FragmentTimeout +- Fragmentation: reassemble fragments by batch header + index; timeout triggers Ack.FragmentTimeout - Rooms: a WS connection can join multiple CRDT+room pairs - Auth/persistence hooks exposed by `SimpleServer` and Rust server diff --git a/README.md b/README.md index aeeff86..5b94e15 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ loro-protocol is a small, transport-agnostic syncing protocol for collaborative CRDT documents. This repo hosts the protocol implementation, a WebSocket client, and minimal servers for local testing or self‑hosting. -- Protocol: multiplex multiple rooms on one connection, 256 KiB max per message, large update fragmentation supported +- Protocol: multiplex multiple rooms on one connection, 256 KiB max per message, large update fragmentation supported, positive/negative delivery via `Ack`, room eviction via `RoomError` - CRDTs: Loro document, Loro ephemeral store; extensible (e.g., Yjs, Yjs Awareness) - Transports: WebSocket or any integrity-preserving transport (e.g., WebRTC) @@ -158,7 +158,7 @@ The Rust workspace contains a minimal async WebSocket server (`loro-websocket-se ## Protocol Highlights - Magic bytes per CRDT: "%LOR" (Loro doc), "%EPH" (Loro ephemeral), "%EPS" (persisted Loro ephemeral – tells the server to keep the latest state so new peers can load it immediately), "%YJS", "%YAW", … -- Messages: JoinRequest/JoinResponseOk/JoinError, DocUpdate, DocUpdateFragmentHeader/Fragment, UpdateError, Leave +- Messages: JoinRequest/JoinResponseOk/JoinError, DocUpdate (with batchId), DocUpdateFragmentHeader/Fragment, Ack, RoomError, Leave - Limits: 256 KiB per message; large updates must be fragmented; default reassembly timeout 10s - Multi‑room: room ID is part of every message; one connection can join multiple rooms diff --git a/conductor.json b/conductor.json new file mode 100644 index 0000000..0287ba9 --- /dev/null +++ b/conductor.json @@ -0,0 +1,6 @@ +{ + "scripts": { + "setup": "pnpm install && pnpm build", + "run": "pnpm test" + } +} diff --git a/llms.md b/llms.md index ad3207d..e6d4539 100644 --- a/llms.md +++ b/llms.md @@ -35,11 +35,12 @@ end-to-end encrypted extension, and the full surface of the | `0x00` | `JoinRequest` | `varBytes joinPayload` (app-defined metadata, e.g., auth/session info), `varBytes version`. | | `0x01` | `JoinResponseOk` | `varString permission ("read"/"write")`, `varBytes version`, `varBytes extraMetadata`. | | `0x02` | `JoinError` | `u8 code`, `varString message`, optional `varBytes receiverVersion` when `code=version_unknown`. | -| `0x03` | `DocUpdate` | `varUint N` updates followed by `N` `varBytes` chunks. | +| `0x03` | `DocUpdate` | `varUint N` updates followed by `N` `varBytes` chunks, then 8-byte `batchId`. | | `0x04` | `DocUpdateFragmentHeader` | `8-byte batchId`, `varUint fragmentCount`, `varUint totalSizeBytes`. | | `0x05` | `DocUpdateFragment` | `8-byte batchId`, `varUint index`, `varBytes fragment`. | -| `0x06` | `UpdateError` | `u8 code`, `varString message`, optional batch ID when `code=fragment_timeout`. | +| `0x06` | `RoomError` | `u8 code`, `varString message`; receipt means the peer is evicted from the room until rejoin. | | `0x07` | `Leave` | No additional payload. | +| `0x08` | `Ack` | `8-byte refId` (batch or fragment ID) + `u8 status` (see §1.4). | ### 1.2 Sync Lifecycle @@ -50,12 +51,14 @@ end-to-end encrypted extension, and the full surface of the - or `JoinError` when the join payload is rejected (e.g., auth failure) or for unknown version. On `version_unknown`, the server includes its version for reseeding. -3. Clients broadcast local edits using `DocUpdate`. Payloads exceeding the - size limit are sliced into fragments: send header first, then numbered - fragments. Recipients reassemble by `batchId`. -4. Clients send `Leave` when unsubscribing from a room. -5. Servers may emit `UpdateError` when denying updates; clients SHOULD surface - these via adaptor callbacks. +3. Clients broadcast local edits using `DocUpdate` (with an 8-byte `batchId`). + Oversize payloads are sliced into fragments: send header first, then + numbered fragments. Recipients reassemble by `batchId`. +4. Server replies with `Ack` for every `DocUpdate`/fragment batch: `status=0` + on success, non-zero mirrors legacy `UpdateError` codes. +5. Clients send `Leave` when unsubscribing from a room. +6. Servers send `RoomError` to evict a peer from a room; the peer must rejoin + to resume traffic. ### 1.3 Fragmentation Rules @@ -64,24 +67,27 @@ end-to-end encrypted extension, and the full surface of the overhead (~240 KiB per fragment in the current implementation). - Receivers store fragments per `batchId`, expect all `fragmentCount` entries, and enforce a default 10-second reassembly timeout. -- On timeout (`UpdateError.fragment_timeout`), senders SHOULD resend the full - batch (header + fragments). +- On timeout, receivers send `Ack` with `status=fragment_timeout`; senders + SHOULD resend the full batch (header + fragments). -### 1.4 Error Codes +### 1.4 Status / Error Codes - **`JoinError` codes:** - `0x00 unknown` - `0x01 version_unknown` (`receiverVersion` included) - `0x02 auth_failed` - `0x7F app_error` (`varString app_code`) -- **`UpdateError` codes:** - - `0x00 unknown` +- **`Ack.status` codes (mirrors legacy `UpdateError`):** + - `0x00 ok` + - `0x01 unknown` - `0x03 permission_denied` - `0x04 invalid_update` - `0x05 payload_too_large` - `0x06 rate_limited` - - `0x07 fragment_timeout` (`8-byte batchId`) - - `0x7F app_error` (`varString app_code`) + - `0x07 fragment_timeout` + - `0x7F app_error` +- **`RoomError` codes:** + - `0x01 unknown` (eviction; peer must rejoin) - Protocol violations MAY be raised via host callbacks; implementations often close the connection on unrecoverable errors. @@ -156,7 +162,7 @@ The server parses headers for routing/deduplication but never decrypts `ct`. - Message size remains bounded by the base protocol (use fragments as needed). - Receivers SHOULD deduplicate spans via `peerId`/`start`/`end` metadata. - Unknown `keyId` SHOULD trigger key resolution and a retry; persistent - failure is surfaced locally instead of emitting `UpdateError`. + failure is surfaced locally; no Ack is emitted because encryption/auth is end-to-end. --- @@ -272,7 +278,7 @@ The resolved room implements: to stay within limits. - On receiving `DocUpdate`/fragments, the client reassembles updates and passes them to `crdtAdaptor.applyUpdate`. -- `UpdateError` messages invoke `crdtAdaptor.handleUpdateError` if provided. +- `Ack` messages with non‑zero status trigger `crdtAdaptor.onUpdateError(updates, status)` using the original sent batch; missing batches are still logged. ### 3.8 Ping/Pong Integration diff --git a/packages/loro-adaptors/README.md b/packages/loro-adaptors/README.md index 59743e3..385129a 100644 --- a/packages/loro-adaptors/README.md +++ b/packages/loro-adaptors/README.md @@ -126,4 +126,4 @@ pnpm typecheck ## License -MIT \ No newline at end of file +MIT diff --git a/packages/loro-adaptors/src/elo-adaptor.ts b/packages/loro-adaptors/src/elo-adaptor.ts index 087f9e9..0b05287 100644 --- a/packages/loro-adaptors/src/elo-adaptor.ts +++ b/packages/loro-adaptors/src/elo-adaptor.ts @@ -2,9 +2,6 @@ import { LoroDoc, VersionVector, decodeImportBlobMeta } from "loro-crdt"; import { CrdtType, JoinResponseOk, - UpdateError, - MessageType, - UpdateErrorCode, } from "loro-protocol"; import type { CrdtAdaptorContext, CrdtDocAdaptor } from "./types"; import { @@ -27,7 +24,11 @@ export interface EloAdaptorConfig { err: Error, meta: { kind: "delta" | "snapshot"; keyId: string } ) => void; - onUpdateError?: (error: UpdateError) => void; + onUpdateError?: ( + updates: Uint8Array[], + errorCode: number, + reason?: string + ) => void; } export class EloAdaptor implements CrdtDocAdaptor { @@ -146,13 +147,8 @@ export class EloAdaptor implements CrdtDocAdaptor { await this.sendSnapshot(); } } catch (err) { - this.config.onUpdateError?.({ - type: MessageType.UpdateError, - crdt: this.crdtType, - roomId: "", - code: UpdateErrorCode.Unknown, - message: err instanceof Error ? err.message : String(err), - }); + // Surface failure to host. + console.error("ELO adaptor failed to package/send update", err); } })(); }); @@ -166,6 +162,14 @@ export class EloAdaptor implements CrdtDocAdaptor { return undefined; } + onUpdateError( + updates: Uint8Array[], + errorCode: number, + reason?: string + ): void { + this.config.onUpdateError?.(updates, errorCode, reason); + } + async handleJoinOk(res: JoinResponseOk): Promise { if (this.destroyed) return; try { @@ -235,10 +239,6 @@ export class EloAdaptor implements CrdtDocAdaptor { } } - handleUpdateError(error: UpdateError): void { - this.config.onUpdateError?.(error); - } - destroy(): void { if (this.destroyed) return; this.destroyed = true; diff --git a/packages/loro-adaptors/src/flock-adaptor.ts b/packages/loro-adaptors/src/flock-adaptor.ts index 40c0198..61fd2e7 100644 --- a/packages/loro-adaptors/src/flock-adaptor.ts +++ b/packages/loro-adaptors/src/flock-adaptor.ts @@ -1,9 +1,7 @@ import { Flock } from "@loro-dev/flock"; import { CrdtType, - JoinError, JoinResponseOk, - UpdateError, } from "loro-protocol"; import type { CrdtAdaptorContext, CrdtDocAdaptor } from "./types"; @@ -97,7 +95,11 @@ function deserializeBundle(bytes: Uint8Array): FlockExportBundle { export interface FlockAdaptorConfig { onImportError?: (error: Error, data: Uint8Array[]) => void; - onUpdateError?: (error: UpdateError) => void; + onUpdateError?: ( + updates: Uint8Array[], + errorCode: number, + reason?: string + ) => void; } /** @@ -147,8 +149,6 @@ export class FlockAdaptor implements CrdtDocAdaptor { return compareVersions(this.flock.version(), remote); } - handleJoinErr?: (err: JoinError) => Promise; - setCtx(ctx: CrdtAdaptorContext): void { this.ctx = ctx; if (this.unsubscribe) { @@ -174,6 +174,14 @@ export class FlockAdaptor implements CrdtDocAdaptor { return undefined; } + onUpdateError( + updates: Uint8Array[], + errorCode: number, + reason?: string + ): void { + this.config.onUpdateError?.(updates, errorCode, reason); + } + async handleJoinOk(res: JoinResponseOk): Promise { if (this.destroyed) return; try { @@ -228,10 +236,6 @@ export class FlockAdaptor implements CrdtDocAdaptor { } } - handleUpdateError(error: UpdateError): void { - this.config.onUpdateError?.(error); - } - destroy(): void { if (this.destroyed) return; this.destroyed = true; diff --git a/packages/loro-adaptors/src/loro-adaptor.ts b/packages/loro-adaptors/src/loro-adaptor.ts index 76a816e..52fe206 100644 --- a/packages/loro-adaptors/src/loro-adaptor.ts +++ b/packages/loro-adaptors/src/loro-adaptor.ts @@ -1,15 +1,17 @@ import { LoroDoc, VersionVector } from "loro-crdt"; import { CrdtType, - JoinError, JoinResponseOk, - UpdateError, } from "loro-protocol"; import type { CrdtAdaptorContext, CrdtDocAdaptor } from "./types"; export interface LoroAdaptorConfig { onImportError?: (error: Error, data: Uint8Array[]) => void; - onUpdateError?: (error: UpdateError) => void; + onUpdateError?: ( + updates: Uint8Array[], + errorCode: number, + reason?: string + ) => void; } export class LoroAdaptor implements CrdtDocAdaptor { @@ -55,8 +57,6 @@ export class LoroAdaptor implements CrdtDocAdaptor { return this.doc.version().compare(vv) as 0 | 1 | -1 | undefined; } - handleJoinErr?: ((err: JoinError) => Promise) | undefined; - getDoc(): LoroDoc { return this.doc; } @@ -79,6 +79,14 @@ export class LoroAdaptor implements CrdtDocAdaptor { return undefined; } + onUpdateError( + updates: Uint8Array[], + errorCode: number, + reason?: string + ): void { + this.config.onUpdateError?.(updates, errorCode, reason); + } + async handleJoinOk(res: JoinResponseOk): Promise { if (this.destroyed) return; @@ -110,7 +118,7 @@ export class LoroAdaptor implements CrdtDocAdaptor { this.ctx?.send([updates]); } } catch (error) { - this.ctx!.onJoinFailed( + this.ctx?.onJoinFailed( error instanceof Error ? error.message : String(error) ); throw error; @@ -126,7 +134,11 @@ export class LoroAdaptor implements CrdtDocAdaptor { // Pending updates may occur when concurrent changes happen } } catch (error) { - this.ctx!.onImportError( + this.config.onImportError?.( + error instanceof Error ? error : new Error(String(error)), + updates + ); + this.ctx?.onImportError( error instanceof Error ? error : new Error(String(error)), updates ); @@ -142,12 +154,6 @@ export class LoroAdaptor implements CrdtDocAdaptor { } } - handleUpdateError(error: UpdateError): void { - if (this.config.onUpdateError) { - this.config.onUpdateError(error); - } - } - destroy(): void { if (this.destroyed) return; this.destroyed = true; diff --git a/packages/loro-adaptors/src/loro-ephemeral-adaptor.ts b/packages/loro-adaptors/src/loro-ephemeral-adaptor.ts index cf3b381..825f47e 100644 --- a/packages/loro-adaptors/src/loro-ephemeral-adaptor.ts +++ b/packages/loro-adaptors/src/loro-ephemeral-adaptor.ts @@ -22,8 +22,6 @@ export class LoroEphemeralAdaptor implements CrdtDocAdaptor { return 0 as const; } - handleJoinErr?: undefined; - getStore(): EphemeralStore { return this.store; } diff --git a/packages/loro-adaptors/src/loro-persistent-store-adaptor.ts b/packages/loro-adaptors/src/loro-persistent-store-adaptor.ts index 2c0d92b..2d61461 100644 --- a/packages/loro-adaptors/src/loro-persistent-store-adaptor.ts +++ b/packages/loro-adaptors/src/loro-persistent-store-adaptor.ts @@ -22,8 +22,6 @@ export class LoroPersistentStoreAdaptor implements CrdtDocAdaptor { return 0 as const; } - handleJoinErr?: undefined; - getStore(): EphemeralStore { return this.store; } diff --git a/packages/loro-adaptors/src/types.ts b/packages/loro-adaptors/src/types.ts index 318b9bb..b6bce44 100644 --- a/packages/loro-adaptors/src/types.ts +++ b/packages/loro-adaptors/src/types.ts @@ -1,9 +1,4 @@ -import { - CrdtType, - JoinResponseOk, - JoinError, - UpdateError, -} from "loro-protocol"; +import { CrdtType, JoinResponseOk } from "loro-protocol"; export interface CrdtDocAdaptor { crdtType: CrdtType; @@ -45,11 +40,16 @@ export interface CrdtDocAdaptor { currentVersion: Uint8Array ) => Uint8Array | undefined; /** - * Handle update errors with context-specific logic - * @param error The update error details + * Called when the server rejects a previously sent update batch. + * @param updates The original updates that were sent + * @param errorCode The numeric update status code from the Ack + * @param reason Optional human-readable reason if available */ - handleUpdateError?: (error: UpdateError) => void; - handleJoinErr?: (err: JoinError) => Promise; + onUpdateError?: ( + updates: Uint8Array[], + errorCode: number, + reason?: string + ) => void; destroy: () => void; } diff --git a/packages/loro-protocol/src/encoding.ts b/packages/loro-protocol/src/encoding.ts index d09d324..46763b9 100644 --- a/packages/loro-protocol/src/encoding.ts +++ b/packages/loro-protocol/src/encoding.ts @@ -2,7 +2,7 @@ import { BytesReader, BytesWriter } from "./bytes"; import { CrdtType, JoinErrorCode, - UpdateErrorCode, + UpdateStatusCode, MessageType, ProtocolMessage, JoinRequest, @@ -11,11 +11,13 @@ import { DocUpdate, DocUpdateFragmentHeader, DocUpdateFragment, - UpdateError, + RoomError, Leave, MAX_MESSAGE_SIZE, hexToBytes, bytesToHex, + Ack, + RoomErrorCode, HexString, } from "./protocol"; @@ -52,8 +54,9 @@ function isMessageType(x: number): x is MessageType { x === MessageType.DocUpdate || x === MessageType.DocUpdateFragmentHeader || x === MessageType.DocUpdateFragment || - x === MessageType.UpdateError || - x === MessageType.Leave + x === MessageType.RoomError || + x === MessageType.Leave || + x === MessageType.Ack ); } @@ -66,18 +69,44 @@ function isJoinErrorCode(x: number): x is JoinErrorCode { ); } -function isUpdateErrorCode(x: number): x is UpdateErrorCode { +function isUpdateStatusCode(x: number): x is UpdateStatusCode { return ( - x === UpdateErrorCode.Unknown || - x === UpdateErrorCode.PermissionDenied || - x === UpdateErrorCode.InvalidUpdate || - x === UpdateErrorCode.PayloadTooLarge || - x === UpdateErrorCode.RateLimited || - x === UpdateErrorCode.FragmentTimeout || - x === UpdateErrorCode.AppError + x === UpdateStatusCode.Ok || + x === UpdateStatusCode.Unknown || + x === UpdateStatusCode.PermissionDenied || + x === UpdateStatusCode.InvalidUpdate || + x === UpdateStatusCode.PayloadTooLarge || + x === UpdateStatusCode.RateLimited || + x === UpdateStatusCode.FragmentTimeout || + x === UpdateStatusCode.AppError ); } +function isRoomErrorCode(x: number): x is RoomErrorCode { + return ( + x === RoomErrorCode.RejoinSuggested || + x === RoomErrorCode.Evicted || + x === RoomErrorCode.Unknown + ); +} + +function readBatchId(reader: BytesReader, context: string): HexString { + if (reader.remaining < 8) { + throw new Error(`Invalid ${context}: missing batch ID`); + } + const batchIdBytes = reader.readBytes(8); + if (batchIdBytes.byteLength !== 8) { + throw new Error(`Invalid ${context}: batch ID must be exactly 8 bytes`); + } + return bytesToHex(batchIdBytes); +} + +function ensureFullyConsumed(reader: BytesReader, context: string) { + if (reader.remaining !== 0) { + throw new Error(`Invalid ${context}: unexpected trailing bytes`); + } +} + export function encode(message: ProtocolMessage): Uint8Array { const writer = new BytesWriter(); if (!validCrdtTypes.includes(message.crdt)) { @@ -132,6 +161,8 @@ export function encode(message: ProtocolMessage): Uint8Array { for (const update of message.updates) { writer.pushVarBytes(update); } + // Append 8-byte Update Batch ID + writer.pushBytes(hexToBytes(message.batchId)); break; } case MessageType.DocUpdateFragmentHeader: { @@ -150,19 +181,14 @@ export function encode(message: ProtocolMessage): Uint8Array { writer.pushVarBytes(message.fragment); break; } - case MessageType.UpdateError: { + case MessageType.RoomError: { writer.pushByte(message.code); writer.pushVarString(message.message); - if ( - message.code === UpdateErrorCode.FragmentTimeout && - message.batchId !== undefined - ) { - const batchIdBytes = hexToBytes(message.batchId); - writer.pushBytes(batchIdBytes); - } - if (message.code === UpdateErrorCode.AppError && message.appCode) { - writer.pushVarString(message.appCode); - } + break; + } + case MessageType.Ack: { + writer.pushBytes(hexToBytes(message.refId)); + writer.pushByte(message.status); break; } case MessageType.Leave: { @@ -274,21 +300,21 @@ export function decode(data: Uint8Array): ProtocolMessage { for (let i = 0; i < count; i++) { updates.push(reader.readVarBytes()); } + const batchId = readBatchId(reader, "DocUpdate"); + ensureFullyConsumed(reader, "DocUpdate"); return { type, crdt, roomId, updates, + batchId, } as DocUpdate; } case MessageType.DocUpdateFragmentHeader: { - if (reader.remaining < 8) { - throw new Error("Invalid DocUpdateFragmentHeader: missing batch ID"); - } - const batchIdBytes = reader.readBytes(8); - const batchId = bytesToHex(batchIdBytes); + const batchId = readBatchId(reader, "DocUpdateFragmentHeader"); const fragmentCount = reader.readUleb128(); const totalSizeBytes = reader.readUleb128(); + ensureFullyConsumed(reader, "DocUpdateFragmentHeader"); return { type, crdt, @@ -299,13 +325,10 @@ export function decode(data: Uint8Array): ProtocolMessage { } as DocUpdateFragmentHeader; } case MessageType.DocUpdateFragment: { - if (reader.remaining < 8) { - throw new Error("Invalid DocUpdateFragment: missing batch ID"); - } - const batchIdBytes = reader.readBytes(8); - const batchId = bytesToHex(batchIdBytes); + const batchId = readBatchId(reader, "DocUpdateFragment"); const index = reader.readUleb128(); const fragment = reader.readVarBytes(); + ensureFullyConsumed(reader, "DocUpdateFragment"); return { type, crdt, @@ -315,32 +338,37 @@ export function decode(data: Uint8Array): ProtocolMessage { fragment, } as DocUpdateFragment; } - case MessageType.UpdateError: { + case MessageType.RoomError: { const codeByte2 = reader.readByte(); - const code = isUpdateErrorCode(codeByte2) + const code = isRoomErrorCode(codeByte2) ? codeByte2 - : UpdateErrorCode.Unknown; + : RoomErrorCode.Unknown; const message = reader.readVarString(); - let batchId: HexString | undefined; - let appCode: string | undefined; - - if (code === UpdateErrorCode.FragmentTimeout && reader.remaining >= 8) { - const batchIdBytes = reader.readBytes(8); - batchId = bytesToHex(batchIdBytes); - } - if (code === UpdateErrorCode.AppError && reader.remaining > 0) { - appCode = reader.readVarString(); - } - return { type, crdt, roomId, code, message, - batchId, - appCode, - } as UpdateError; + } as RoomError; + } + case MessageType.Ack: { + if (reader.remaining < 9) { + throw new Error("Invalid Ack: missing fields"); + } + const refId = readBatchId(reader, "Ack"); + const statusByte = reader.readByte(); + const status = isUpdateStatusCode(statusByte) + ? statusByte + : UpdateStatusCode.Unknown; + ensureFullyConsumed(reader, "Ack"); + return { + type, + crdt, + roomId, + refId, + status, + } as Ack; } case MessageType.Leave: { return { diff --git a/packages/loro-protocol/src/protocol.ts b/packages/loro-protocol/src/protocol.ts index 2446340..696d45e 100644 --- a/packages/loro-protocol/src/protocol.ts +++ b/packages/loro-protocol/src/protocol.ts @@ -41,8 +41,9 @@ export const MessageType = { DocUpdate: 0x03, DocUpdateFragmentHeader: 0x04, DocUpdateFragment: 0x05, - UpdateError: 0x06, + RoomError: 0x06, Leave: 0x07, + Ack: 0x08, } as const; export type MessageType = (typeof MessageType)[keyof typeof MessageType]; @@ -54,8 +55,16 @@ export const JoinErrorCode = { } as const; export type JoinErrorCode = (typeof JoinErrorCode)[keyof typeof JoinErrorCode]; -export const UpdateErrorCode = { - Unknown: 0x00, +export const RoomErrorCode = { + RejoinSuggested: 0x01, + Evicted: 0x02, + Unknown: 0x7f, +} as const; +export type RoomErrorCode = (typeof RoomErrorCode)[keyof typeof RoomErrorCode]; + +export const UpdateStatusCode = { + Ok: 0x00, + Unknown: 0x01, PermissionDenied: 0x03, InvalidUpdate: 0x04, PayloadTooLarge: 0x05, @@ -63,8 +72,8 @@ export const UpdateErrorCode = { FragmentTimeout: 0x07, AppError: 0x7f, } as const; -export type UpdateErrorCode = - (typeof UpdateErrorCode)[keyof typeof UpdateErrorCode]; +export type UpdateStatusCode = + (typeof UpdateStatusCode)[keyof typeof UpdateStatusCode]; export type RoomId = string; @@ -98,6 +107,7 @@ export interface JoinError extends MessageBase { export interface DocUpdate extends MessageBase { type: typeof MessageType.DocUpdate; updates: Uint8Array[]; + batchId: HexString; } export type HexString = `0x${string}`; @@ -115,12 +125,16 @@ export interface DocUpdateFragment extends MessageBase { fragment: Uint8Array; } -export interface UpdateError extends MessageBase { - type: typeof MessageType.UpdateError; - code: UpdateErrorCode; +export interface RoomError extends MessageBase { + type: typeof MessageType.RoomError; + code: RoomErrorCode; message: string; - batchId?: HexString; - appCode?: string; +} + +export interface Ack extends MessageBase { + type: typeof MessageType.Ack; + refId: HexString; // Update Batch ID or Fragment batch ID + status: UpdateStatusCode; } export function hexToBytes(hex: HexString): Uint8Array { @@ -150,5 +164,6 @@ export type ProtocolMessage = | DocUpdate | DocUpdateFragmentHeader | DocUpdateFragment - | UpdateError - | Leave; + | RoomError + | Leave + | Ack; diff --git a/packages/loro-protocol/tests/__snapshots__/encoding.snap.test.ts.snap b/packages/loro-protocol/tests/__snapshots__/encoding.snap.test.ts.snap index ab3cff9..23ae5c3 100644 --- a/packages/loro-protocol/tests/__snapshots__/encoding.snap.test.ts.snap +++ b/packages/loro-protocol/tests/__snapshots__/encoding.snap.test.ts.snap @@ -1,6 +1,10 @@ // Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html -exports[`encoding snapshots > DocUpdate multiple updates 1`] = `"0x25594a5309726f6f6d2d3132333403030301020304040506070108"`; +exports[`encoding snapshots > Ack fragment timeout 1`] = `"0x2559415709726f6f6d2d3132333408010000000000000007"`; + +exports[`encoding snapshots > Ack ok 1`] = `"0x254c4f5209726f6f6d2d3132333408010203040506070800"`; + +exports[`encoding snapshots > DocUpdate multiple updates 1`] = `"0x25594a5309726f6f6d2d31323334030303010203040405060701080102030405060708"`; exports[`encoding snapshots > DocUpdateFragment 1`] = `"0x254c4f5209726f6f6d2d313233340500000000000000000320000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"`; @@ -20,8 +24,4 @@ exports[`encoding snapshots > JoinResponseOk write + extra 1`] = `"0x254c4f52097 exports[`encoding snapshots > Leave 1`] = `"0x254c4f5209726f6f6d2d3132333407"`; -exports[`encoding snapshots > UpdateError app error with appCode 1`] = `"0x254c4f5209726f6f6d2d31323334067f10437573746f6d20617070206572726f720f637573746f6d5f636f64655f313233"`; - -exports[`encoding snapshots > UpdateError fragment timeout with batchId 1`] = `"0x2559415709726f6f6d2d31323334060710467261676d656e742074696d656f75740100000000000000"`; - -exports[`encoding snapshots > UpdateError permission denied 1`] = `"0x254c4f5209726f6f6d2d313233340603134e6f207772697465207065726d697373696f6e"`; +exports[`encoding snapshots > RoomError eviction 1`] = `"0x254c4f5209726f6f6d2d3132333406020765766963746564"`; diff --git a/packages/loro-protocol/tests/encoding.snap.test.ts b/packages/loro-protocol/tests/encoding.snap.test.ts index 1cf16f7..43dee27 100644 --- a/packages/loro-protocol/tests/encoding.snap.test.ts +++ b/packages/loro-protocol/tests/encoding.snap.test.ts @@ -4,7 +4,8 @@ import { CrdtType, MessageType, JoinErrorCode, - UpdateErrorCode, + UpdateStatusCode, + RoomErrorCode, bytesToHex, type JoinRequest, type JoinResponseOk, @@ -12,7 +13,8 @@ import { type DocUpdate, type DocUpdateFragmentHeader, type DocUpdateFragment, - type UpdateError, + type RoomError, + type Ack, type Leave, } from "../src/protocol"; @@ -20,6 +22,7 @@ import { describe("encoding snapshots", () => { const roomId = "room-1234"; + const batchId = "0x0102030405060708"; it("JoinRequest", () => { const msg: JoinRequest = { @@ -100,6 +103,7 @@ describe("encoding snapshots", () => { new Uint8Array([4, 5, 6, 7]), new Uint8Array([8]), ], + batchId, }; expect(bytesToHex(encode(msg))).toMatchSnapshot(); }); @@ -130,37 +134,35 @@ describe("encoding snapshots", () => { expect(bytesToHex(encode(msg))).toMatchSnapshot(); }); - it("UpdateError permission denied", () => { - const msg: UpdateError = { - type: MessageType.UpdateError, + it("RoomError eviction", () => { + const msg: RoomError = { + type: MessageType.RoomError, crdt: CrdtType.Loro, roomId, - code: UpdateErrorCode.PermissionDenied, - message: "No write permission", + code: RoomErrorCode.Evicted, + message: "evicted", }; expect(bytesToHex(encode(msg))).toMatchSnapshot(); }); - it("UpdateError fragment timeout with batchId", () => { - const msg: UpdateError = { - type: MessageType.UpdateError, - crdt: CrdtType.YjsAwareness, + it("Ack ok", () => { + const msg: Ack = { + type: MessageType.Ack, + crdt: CrdtType.Loro, roomId, - code: UpdateErrorCode.FragmentTimeout, - message: "Fragment timeout", - batchId: "0x0100000000000000", + refId: batchId, + status: UpdateStatusCode.Ok, }; expect(bytesToHex(encode(msg))).toMatchSnapshot(); }); - it("UpdateError app error with appCode", () => { - const msg: UpdateError = { - type: MessageType.UpdateError, - crdt: CrdtType.Loro, + it("Ack fragment timeout", () => { + const msg: Ack = { + type: MessageType.Ack, + crdt: CrdtType.YjsAwareness, roomId, - code: UpdateErrorCode.AppError, - message: "Custom app error", - appCode: "custom_code_123", + refId: "0x0100000000000000", + status: UpdateStatusCode.FragmentTimeout, }; expect(bytesToHex(encode(msg))).toMatchSnapshot(); }); diff --git a/packages/loro-protocol/tests/encoding.test.ts b/packages/loro-protocol/tests/encoding.test.ts index abcf715..b57dd6d 100644 --- a/packages/loro-protocol/tests/encoding.test.ts +++ b/packages/loro-protocol/tests/encoding.test.ts @@ -4,19 +4,22 @@ import { CrdtType, MessageType, JoinErrorCode, - UpdateErrorCode, + UpdateStatusCode, + RoomErrorCode, type JoinRequest, type JoinResponseOk, type JoinError, type DocUpdate, type DocUpdateFragmentHeader, type DocUpdateFragment, - type UpdateError, + type RoomError, + type Ack, type Leave, } from "../src/protocol"; describe("Message Encoding and Decoding", () => { const roomId = "room-123"; + const batchId = "0x0102030405060708"; describe("JoinRequest", () => { it("encodes and decodes JoinRequest correctly", () => { @@ -177,6 +180,7 @@ describe("Message Encoding and Decoding", () => { crdt: CrdtType.Loro, roomId, updates: [new Uint8Array([111, 222])], + batchId, }; const encoded = encode(message); @@ -186,6 +190,7 @@ describe("Message Encoding and Decoding", () => { if (decoded.type !== MessageType.DocUpdate) throw new Error("bad type"); expect(decoded.updates.length).toBe(1); expect(Array.from(decoded.updates[0])).toEqual([111, 222]); + expect(decoded.batchId).toBe(batchId); }); it("encodes and decodes DocUpdate with multiple updates", () => { @@ -198,6 +203,7 @@ describe("Message Encoding and Decoding", () => { new Uint8Array([4, 5, 6, 7]), new Uint8Array([8]), ], + batchId, }; const encoded = encode(message); @@ -209,6 +215,7 @@ describe("Message Encoding and Decoding", () => { expect(Array.from(decoded.updates[0])).toEqual([1, 2, 3]); expect(Array.from(decoded.updates[1])).toEqual([4, 5, 6, 7]); expect(Array.from(decoded.updates[2])).toEqual([8]); + expect(decoded.batchId).toBe(batchId); }); it("encodes and decodes DocUpdate with empty updates array", () => { @@ -217,6 +224,7 @@ describe("Message Encoding and Decoding", () => { crdt: CrdtType.Loro, roomId, updates: [], + batchId, }; const encoded = encode(message); @@ -225,6 +233,39 @@ describe("Message Encoding and Decoding", () => { expect(decoded.type).toBe(MessageType.DocUpdate); if (decoded.type !== MessageType.DocUpdate) throw new Error("bad type"); expect(decoded.updates.length).toBe(0); + expect(decoded.batchId).toBe(batchId); + }); + + it("rejects DocUpdate when batchId is shorter than 8 bytes", () => { + const message: DocUpdate = { + type: MessageType.DocUpdate, + crdt: CrdtType.Loro, + roomId, + updates: [new Uint8Array([1])], + batchId, + }; + + const encoded = encode(message); + const truncated = encoded.slice(0, encoded.length - 1); // drop one batchId byte + + expect(() => decode(truncated)).toThrow(/batch ID/); + }); + + it("rejects DocUpdate when extra bytes follow the batchId", () => { + const message: DocUpdate = { + type: MessageType.DocUpdate, + crdt: CrdtType.Loro, + roomId, + updates: [new Uint8Array([1])], + batchId, + }; + + const encoded = encode(message); + const withExtra = new Uint8Array(encoded.length + 1); + withExtra.set(encoded); + withExtra[withExtra.length - 1] = 0xff; + + expect(() => decode(withExtra)).toThrow(/trailing bytes/); }); }); @@ -300,92 +341,61 @@ describe("Message Encoding and Decoding", () => { }); }); - describe("UpdateError", () => { - it("encodes and decodes UpdateError with permission denied", () => { - const message: UpdateError = { - type: MessageType.UpdateError, + describe("RoomError", () => { + it("encodes and decodes RoomError eviction", () => { + const message: RoomError = { + type: MessageType.RoomError, crdt: CrdtType.Loro, roomId, - code: UpdateErrorCode.PermissionDenied, - message: "No write permission", + code: RoomErrorCode.Evicted, + message: "evicted", }; const encoded = encode(message); const decoded = decode(encoded); - expect(decoded.type).toBe(MessageType.UpdateError); - if (decoded.type !== MessageType.UpdateError) throw new Error("bad type"); - expect(decoded.code).toBe(UpdateErrorCode.PermissionDenied); - expect(decoded.message).toBe("No write permission"); - expect(decoded.batchId).toBeUndefined(); - expect(decoded.appCode).toBeUndefined(); + expect(decoded.type).toBe(MessageType.RoomError); + if (decoded.type !== MessageType.RoomError) throw new Error("bad type"); + expect(decoded.code).toBe(RoomErrorCode.Evicted); + expect(decoded.message).toBe("evicted"); }); + }); - it("encodes and decodes UpdateError with fragment timeout", () => { - const message: UpdateError = { - type: MessageType.UpdateError, - crdt: CrdtType.YjsAwareness, + describe("Ack", () => { + it("encodes and decodes Ack ok", () => { + const message: Ack = { + type: MessageType.Ack, + crdt: CrdtType.Loro, roomId, - code: UpdateErrorCode.FragmentTimeout, - message: "Fragment timeout", - batchId: "0x0100000000000000", + refId: batchId, + status: UpdateStatusCode.Ok, }; const encoded = encode(message); const decoded = decode(encoded); - expect(decoded.type).toBe(MessageType.UpdateError); - if (decoded.type !== MessageType.UpdateError) throw new Error("bad type"); - expect(decoded.code).toBe(UpdateErrorCode.FragmentTimeout); - expect(decoded.message).toBe("Fragment timeout"); - expect(decoded.batchId).toBe("0x0100000000000000"); + expect(decoded.type).toBe(MessageType.Ack); + if (decoded.type !== MessageType.Ack) throw new Error("bad type"); + expect(decoded.refId).toBe(batchId); + expect(decoded.status).toBe(UpdateStatusCode.Ok); }); - it("encodes and decodes UpdateError with app error", () => { - const message: UpdateError = { - type: MessageType.UpdateError, - crdt: CrdtType.Loro, + it("encodes and decodes Ack with fragment timeout", () => { + const message: Ack = { + type: MessageType.Ack, + crdt: CrdtType.LoroEphemeralStore, roomId, - code: UpdateErrorCode.AppError, - message: "Custom app error", - appCode: "custom_code_123", + refId: "0x0000000000000001", + status: UpdateStatusCode.FragmentTimeout, }; const encoded = encode(message); const decoded = decode(encoded); - expect(decoded.type).toBe(MessageType.UpdateError); - if (decoded.type !== MessageType.UpdateError) throw new Error("bad type"); - expect(decoded.code).toBe(UpdateErrorCode.AppError); - expect(decoded.message).toBe("Custom app error"); - expect(decoded.appCode).toBe("custom_code_123"); - }); - - it("encodes and decodes UpdateError with various error codes", () => { - const errorCodes = [ - UpdateErrorCode.Unknown, - UpdateErrorCode.InvalidUpdate, - UpdateErrorCode.PayloadTooLarge, - UpdateErrorCode.RateLimited, - ]; - - for (const code of errorCodes) { - const message: UpdateError = { - type: MessageType.UpdateError, - crdt: CrdtType.Loro, - roomId, - code, - message: `Error with code ${code}`, - }; - - const encoded = encode(message); - const decoded = decode(encoded); - expect(decoded.type).toBe(MessageType.UpdateError); - if (decoded.type !== MessageType.UpdateError) - throw new Error("bad type"); - expect(decoded.code).toBe(code); - expect(decoded.message).toBe(`Error with code ${code}`); - } + expect(decoded.type).toBe(MessageType.Ack); + if (decoded.type !== MessageType.Ack) throw new Error("bad type"); + expect(decoded.refId).toBe("0x0000000000000001"); + expect(decoded.status).toBe(UpdateStatusCode.FragmentTimeout); }); }); @@ -458,6 +468,7 @@ describe("Message Encoding and Decoding", () => { crdt: CrdtType.Loro, roomId, updates: [largeUpdate], + batchId, }; expect(() => encode(message)).toThrow(/Message size .* exceeds maximum/); diff --git a/packages/loro-websocket/src/client/index.ts b/packages/loro-websocket/src/client/index.ts index 9de839f..bdb79f3 100644 --- a/packages/loro-websocket/src/client/index.ts +++ b/packages/loro-websocket/src/client/index.ts @@ -10,12 +10,15 @@ import { JoinError, DocUpdateFragmentHeader, DocUpdateFragment, - UpdateError, - UpdateErrorCode, + Ack, + RoomError, + RoomErrorCode, + UpdateStatusCode, Leave, JoinErrorCode, MAX_MESSAGE_SIZE, bytesToHex, + HexString, } from "loro-protocol"; import type { CrdtDocAdaptor } from "loro-adaptors"; @@ -38,8 +41,9 @@ interface PendingRoom { } interface InternalRoomHandler { - handleDocUpdate(updates: Uint8Array[]): void; - handleUpdateError(error: UpdateError): void; + handleDocUpdate(updates: Uint8Array[], refId?: HexString): void; + handleAck(ack: Ack): void; + handleRoomError(error: RoomError): void; } interface ActiveRoom { @@ -60,6 +64,12 @@ type NodeProcessLike = { removeListener?: (event: string, listener: () => void) => unknown; }; +function randomBatchId(): HexString { + const bytes = new Uint8Array(8); + crypto.getRandomValues(bytes); + return bytesToHex(bytes); +} + /** * The websocket client's high-level connection status. * - `Connecting`: initial connect or a manual `connect()` in progress. @@ -156,7 +166,9 @@ export class LoroWebsocketClient { private pendingRooms: Map = new Map(); private activeRooms: Map = new Map(); // Buffer for %ELO only: backfills can arrive immediately after JoinResponseOk - private preJoinUpdates: Map = new Map(); + private preJoinUpdates: Map> = new Map(); + // Track outbound update batches so we can surface errors with payload context + private sentUpdateBatches: Map = new Map(); private fragmentBatches: Map = new Map(); private roomAdaptors: Map = new Map(); // Track roomId for each active id so we can rejoin on reconnect @@ -439,6 +451,10 @@ export class LoroWebsocketClient { } this.fragmentBatches.clear(); } + // Drop any unacked outbound batches to avoid leaking memory across reconnects + if (this.sentUpdateBatches.size) { + this.sentUpdateBatches.clear(); + } // Reset any in-flight RTT probe to allow future pings after reconnect this.awaitingPongSince = undefined; this.ops.onWsClose?.(); @@ -546,49 +562,63 @@ export class LoroWebsocketClient { if (!roomId) continue; const active = this.activeRooms.get(id); if (!active) continue; - // Prepare a lightweight pending entry so JoinError handling can retry version formats - const roomPromise = Promise.resolve(active.room); - const pending: PendingRoom = { - room: roomPromise, - resolve: (res: JoinResponseOk) => { - // On successful rejoin, let adaptor reconcile to server - adaptor - .handleJoinOk(res) - .catch(e => { - console.error(e); - }) - .finally(() => { - this.pendingRooms.delete(id); - this.emitRoomStatus(id, RoomJoinStatus.Joined); - }); - }, - reject: (error: Error) => { - console.error("Rejoin failed:", error); - // Remove stale room entry so callers can decide to rejoin manually - this.cleanupRoom(roomId, adaptor.crdtType); - this.emitRoomStatus(id, RoomJoinStatus.Error); - }, - adaptor, - roomId, - auth: this.roomAuth.get(id), - isRejoin: true, - }; - this.pendingRooms.set(id, pending); + this.sendRejoinRequest(id, roomId, adaptor, active.room, this.roomAuth.get(id)); + } + } - try { - this.ws.send( - encode({ - type: MessageType.JoinRequest, - crdt: adaptor.crdtType, - roomId, - auth: pending.auth ?? new Uint8Array(), - version: adaptor.getVersion(), - } as JoinRequest) - ); - this.emitRoomStatus(id, RoomJoinStatus.Reconnecting); - } catch (e) { - console.error("Failed to send rejoin request:", e); + private sendRejoinRequest( + id: string, + roomId: string, + adaptor: CrdtDocAdaptor, + room: LoroWebsocketClientRoom, + auth?: Uint8Array + ) { + // Prepare a lightweight pending entry so JoinError handling can retry version formats + const pending: PendingRoom = { + room: Promise.resolve(room), + resolve: (res: JoinResponseOk) => { + adaptor + .handleJoinOk(res) + .catch(e => { + console.error(e); + }) + .finally(() => { + this.pendingRooms.delete(id); + this.emitRoomStatus(id, RoomJoinStatus.Joined); + }); + }, + reject: (error: Error) => { + console.error("Rejoin failed:", error); + this.cleanupRoom(roomId, adaptor.crdtType); + this.emitRoomStatus(id, RoomJoinStatus.Error); + }, + adaptor, + roomId, + auth, + isRejoin: true, + }; + this.pendingRooms.set(id, pending); + + const payload = encode({ + type: MessageType.JoinRequest, + crdt: adaptor.crdtType, + roomId, + auth: auth ?? new Uint8Array(), + version: adaptor.getVersion(), + } as JoinRequest); + + try { + if (this.ws && this.ws.readyState === WebSocket.OPEN) { + this.ws.send(payload); + } else { + this.enqueueJoin(payload); + void this.connect(); } + this.emitRoomStatus(id, RoomJoinStatus.Reconnecting); + } catch (e) { + console.error("Failed to send rejoin request:", e); + this.cleanupRoom(roomId, adaptor.crdtType); + this.emitRoomStatus(id, RoomJoinStatus.Error); } } @@ -617,12 +647,12 @@ export class LoroWebsocketClient { case MessageType.DocUpdate: { const active = this.activeRooms.get(roomId); if (active) { - active.handler.handleDocUpdate(msg.updates); + active.handler.handleDocUpdate(msg.updates, msg.batchId); } else { const pending = this.pendingRooms.get(roomId); if (pending) { const buf = this.preJoinUpdates.get(roomId) ?? []; - buf.push(...msg.updates); + buf.push({ updates: msg.updates, refId: msg.batchId }); this.preJoinUpdates.set(roomId, buf); } } @@ -636,15 +666,29 @@ export class LoroWebsocketClient { this.handleFragment(msg); break; } - case MessageType.UpdateError: { + case MessageType.RoomError: { const active = this.activeRooms.get(roomId); + const adaptor = this.roomAdaptors.get(roomId); + const auth = this.roomAuth.get(roomId); + const shouldRejoin = msg.code === RoomErrorCode.RejoinSuggested; if (active) { - active.handler.handleUpdateError(msg); + active.handler.handleRoomError(msg); + } + // Drop any in-flight join since the server explicitly removed us + this.pendingRooms.delete(roomId); + if (shouldRejoin && active && adaptor) { + this.sendRejoinRequest(roomId, msg.roomId, adaptor, active.room, auth); } else { - const roomIdStr = msg.roomId; - console.error( - `Update error for room ${roomIdStr}: ${msg.code} - ${msg.message}` - ); + // Remove local room state so client does not auto-retry unless requested + this.cleanupRoom(msg.roomId, msg.crdt); + this.emitRoomStatus(roomId, RoomJoinStatus.Error); + } + break; + } + case MessageType.Ack: { + const active = this.activeRooms.get(roomId); + if (active) { + active.handler.handleAck(msg); } break; } @@ -664,17 +708,17 @@ export class LoroWebsocketClient { // Set up timeout (10 seconds default) const timeoutId = setTimeout(() => { this.fragmentBatches.delete(batchKey); - // Send timeout error + // Notify server to prompt resend try { if (this.ws && this.ws.readyState === WebSocket.OPEN) { this.ws.send( encode({ - type: MessageType.UpdateError, + type: MessageType.Ack, crdt: msg.crdt, roomId: msg.roomId, - code: UpdateErrorCode.FragmentTimeout, - message: `Fragment reassembly timeout for batch ${msg.batchId}`, - } as UpdateError) + refId: msg.batchId, + status: UpdateStatusCode.FragmentTimeout, + } as Ack) ); } } catch { } @@ -725,12 +769,12 @@ export class LoroWebsocketClient { const active = this.activeRooms.get(id); if (active) { // Treat reassembled data as a single update - active.handler.handleDocUpdate([reassembledData]); + active.handler.handleDocUpdate([reassembledData], batch.header.batchId); } else { const pending = this.pendingRooms.get(id); if (pending) { const buf = this.preJoinUpdates.get(id) ?? []; - buf.push(reassembledData); + buf.push({ updates: [reassembledData], refId: batch.header.batchId }); this.preJoinUpdates.set(id, buf); } } @@ -753,7 +797,9 @@ export class LoroWebsocketClient { const buf = this.preJoinUpdates.get(id); if (buf && buf.length) { try { - handler.handleDocUpdate(buf); + for (const entry of buf) { + handler.handleDocUpdate(entry.updates, entry.refId); + } } finally { this.preJoinUpdates.delete(id); } @@ -768,9 +814,6 @@ export class LoroWebsocketClient { pending: PendingRoom, roomId: string ) { - // First, let adaptor handle the error for custom logic - await pending.adaptor.handleJoinErr?.(msg); - if (msg.code === JoinErrorCode.VersionUnknown) { // Try alternative version format const currentVersion = pending.adaptor.getVersion(); @@ -819,6 +862,7 @@ export class LoroWebsocketClient { cleanupRoom(roomId: string, crdtType: CrdtType) { const id = crdtType + roomId; + this.purgeSentBatchesForRoom(id); this.activeRooms.delete(id); this.pendingRooms.delete(id); this.roomAdaptors.delete(id); @@ -938,15 +982,6 @@ export class LoroWebsocketClient { }, onImportError: (error: Error, data: Uint8Array[]) => { console.error(`Import error: ${error.message}`, data); - this.ws.send( - encode({ - type: MessageType.UpdateError, - crdt: crdtAdaptor.crdtType, - roomId, - code: UpdateErrorCode.AppError, - message: error.message, - } as UpdateError) - ); }, }); // Create room and register before invoking adaptor.handleJoinOk to ensure @@ -1050,6 +1085,14 @@ export class LoroWebsocketClient { Math.min(240 * 1024, MAX_MESSAGE_SIZE - 4096) ); + const batchId = randomBatchId(); + const roomKey = `${crdt}${roomId}`; + // Store the original payload so we can surface detailed errors on Ack + this.sentUpdateBatches.set(batchId, { + roomKey, + updates: [update.slice()], + }); + if (update.length <= FRAG_LIMIT) { // Send as a single DocUpdate with one update entry ws.send( @@ -1058,6 +1101,7 @@ export class LoroWebsocketClient { crdt, roomId, updates: [update], + batchId, } as DocUpdate) ); return; @@ -1065,9 +1109,6 @@ export class LoroWebsocketClient { // Fragment the update into multiple DocUpdateFragment messages const fragmentCount = Math.ceil(update.length / FRAG_LIMIT); - const batchBytes = new Uint8Array(8); // 8-byte batch ID per protocol - crypto.getRandomValues(batchBytes); - const batchId = bytesToHex(batchBytes); const header: DocUpdateFragmentHeader = { type: MessageType.DocUpdateFragmentHeader, @@ -1107,6 +1148,22 @@ export class LoroWebsocketClient { ); } + consumeSentBatch(refId: HexString): { roomKey: string; updates: Uint8Array[] } | undefined { + const entry = this.sentUpdateBatches.get(refId); + if (entry) { + this.sentUpdateBatches.delete(refId); + } + return entry; + } + + private purgeSentBatchesForRoom(roomKey: string): void { + for (const [refId, entry] of Array.from(this.sentUpdateBatches.entries())) { + if (entry.roomKey === roomKey) { + this.sentUpdateBatches.delete(refId); + } + } + } + /** * Destroy the client, removing listeners and stopping timers. * After destroy, the instance should not be used. @@ -1134,6 +1191,7 @@ export class LoroWebsocketClient { this.ops.onWsClose?.(); } this.queuedJoins = []; + this.sentUpdateBatches.clear(); this.detachSocketListeners(ws); try { this.removeNetworkListeners?.(); @@ -1428,12 +1486,48 @@ class LoroWebsocketClientRoomImpl return this.crdtAdaptor.waitForReachingServerVersion(); } - handleDocUpdate(updates: Uint8Array[]) { - this.crdtAdaptor.applyUpdate(updates); + handleDocUpdate(updates: Uint8Array[], refId?: HexString) { + try { + this.crdtAdaptor.applyUpdate(updates); + } catch (error) { + // Surface failure for visibility and inform server about invalid batch + console.error("Failed to apply remote update", error); + // Inform server that the update failed if we can reference the batch ID + if (refId && this.client.socket?.readyState === WebSocket.OPEN) { + try { + this.client.socket.send( + encode({ + type: MessageType.Ack, + crdt: this.crdtType, + roomId: this.roomId, + refId, + status: UpdateStatusCode.InvalidUpdate, + } as Ack) + ); + } catch (err) { + console.error("Failed to send failure Ack", err); + } + } + } } - handleUpdateError(error: UpdateError) { - this.crdtAdaptor.handleUpdateError?.(error); + handleAck(ack: Ack) { + const sent = this.client.consumeSentBatch(ack.refId); + if (ack.status !== UpdateStatusCode.Ok) { + const updates = sent?.updates ?? []; + const reason = updateStatusToReason(ack.status); + this.crdtAdaptor.onUpdateError?.(updates, ack.status, reason); + if (!sent) { + console.warn(`Ack status ${ack.status} for ${this.crdtType}:${this.roomId} (ref ${ack.refId}) with no matching batch`); + } + } + } + + handleRoomError(error: RoomError) { + // Only mark destroyed for terminal errors; rejoin-suggested errors keep the room alive for retry. + if (error.code !== RoomErrorCode.RejoinSuggested) { + this.destroyed = true; + } } async leave() { @@ -1466,6 +1560,27 @@ class LoroWebsocketClientRoomImpl // --- Keepalive helpers (ping/pong) --- +function updateStatusToReason(status: UpdateStatusCode): string | undefined { + switch (status) { + case UpdateStatusCode.Unknown: + return "unknown"; + case UpdateStatusCode.PermissionDenied: + return "permission_denied"; + case UpdateStatusCode.InvalidUpdate: + return "invalid_update"; + case UpdateStatusCode.PayloadTooLarge: + return "payload_too_large"; + case UpdateStatusCode.RateLimited: + return "rate_limited"; + case UpdateStatusCode.FragmentTimeout: + return "fragment_timeout"; + case UpdateStatusCode.AppError: + return "app_error"; + default: + return undefined; + } +} + // --- Internal ping helpers --- function isPositive(v: unknown): v is number { return typeof v === "number" && isFinite(v) && v > 0; diff --git a/packages/loro-websocket/src/server/simple-server.ts b/packages/loro-websocket/src/server/simple-server.ts index 67053f0..ebdf7dd 100644 --- a/packages/loro-websocket/src/server/simple-server.ts +++ b/packages/loro-websocket/src/server/simple-server.ts @@ -1,4 +1,5 @@ import { WebSocketServer, WebSocket } from "ws"; +import { randomBytes } from "node:crypto"; import type { RawData } from "ws"; // no direct CRDT imports here; handled by CrdtDoc implementations import { @@ -11,8 +12,8 @@ import { JoinError, JoinErrorCode, DocUpdate, - UpdateError, - UpdateErrorCode, + Ack, + UpdateStatusCode, Leave, ProtocolMessage, Permission, @@ -29,7 +30,7 @@ import { export interface SimpleServerConfig { port: number; - host?: string; + host?: string; // default 127.0.0.1 to avoid sandbox binding errors saveInterval?: number; onLoadDocument?: ( roomId: string, @@ -65,6 +66,7 @@ interface ClientConnection { totalSize: number; received: number; header: DocUpdateFragmentHeader; + timeoutId?: NodeJS.Timeout; } >; permissions: Map; // roomKey -> permission @@ -262,6 +264,17 @@ export class SimpleServer { case MessageType.Leave: this.handleLeave(client, message); break; + case MessageType.Ack: + // Clients may report failures when they cannot apply a server update. + if (message.status !== UpdateStatusCode.Ok) { + console.warn( + `Client reported update failure for ${message.crdt}:${message.roomId} ref ${message.refId} status ${message.status}` + ); + } + break; + case MessageType.RoomError: + // Server does not expect these from clients; ignore. + break; default: throw new Error(`Unsupported message type: ${message.type}`); } @@ -334,7 +347,8 @@ export class SimpleServer { type: MessageType.DocUpdate, crdt: message.crdt, roomId: message.roomId, - updates: joinResult.updates + updates: joinResult.updates, + batchId: this.newBatchId(), }); } } catch (error) { @@ -356,14 +370,13 @@ export class SimpleServer { // (Clients fragment large updates; this is a safety net.) const oversized = message.updates.some(u => u.length > MAX_MESSAGE_SIZE); if (oversized) { - const updateError: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.PayloadTooLarge, - message: `Update payload exceeds ${MAX_MESSAGE_SIZE} bytes`, - }; - this.sendMessage(client.ws, updateError); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.PayloadTooLarge, + message.crdt, + message.roomId, + ); return; } @@ -371,14 +384,14 @@ export class SimpleServer { // Check if client has joined this room if (!client.rooms.has(roomKey)) { - const updateError: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.PermissionDenied, - message: "Must join room before sending updates", - }; - this.sendMessage(client.ws, updateError); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.PermissionDenied, + message.crdt, + message.roomId, + ); + client.fragments.delete(message.batchId); return; } @@ -386,14 +399,13 @@ export class SimpleServer { // Check if client has write permission if (permission !== "write") { - const updateError: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.PermissionDenied, - message: "Write permission required to update document", - }; - this.sendMessage(client.ws, updateError); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.PermissionDenied, + message.crdt, + message.roomId, + ); return; } @@ -409,14 +421,14 @@ export class SimpleServer { ); roomDoc.data = newDocumentData; } catch (error) { - const updateError: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.InvalidUpdate, - message: error instanceof Error ? error.message : "Invalid update", - }; - this.sendMessage(client.ws, updateError); + console.warn("applyUpdates failed for DocUpdate", error); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.InvalidUpdate, + message.crdt, + message.roomId, + ); return; } @@ -426,12 +438,22 @@ export class SimpleServer { } const updatesForBroadcast = message.updates; + // Notify sender + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.Ok, + message.crdt, + message.roomId, + ); + if (updatesForBroadcast.length > 0) { const outgoing: DocUpdate = { type: MessageType.DocUpdate, crdt: message.crdt, roomId: message.roomId, updates: updatesForBroadcast, + batchId: message.batchId, }; this.broadcastToRoom( message.roomId, @@ -442,14 +464,13 @@ export class SimpleServer { } } catch (error) { console.error(error); - const updateError: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.InvalidUpdate, - message: error instanceof Error ? error.message : "Invalid update", - }; - this.sendMessage(client.ws, updateError); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.Unknown, + message.crdt, + message.roomId, + ); } } @@ -461,23 +482,36 @@ export class SimpleServer { // Check if client has joined this room if (!client.rooms.has(roomKey)) { - const error: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.PermissionDenied, - message: "Must join room before sending fragments", - }; - this.sendMessage(client.ws, error); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.PermissionDenied, + message.crdt, + message.roomId, + ); return; } - client.fragments.set(message.batchId, { - data: Array.from({ length: message.fragmentCount }), + const batch = { + data: Array.from({ length: message.fragmentCount }, () => new Uint8Array()), totalSize: message.totalSizeBytes, received: 0, header: message, - }); + timeoutId: undefined as NodeJS.Timeout | undefined, + }; + + batch.timeoutId = setTimeout(() => { + client.fragments.delete(message.batchId); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.FragmentTimeout, + message.crdt, + message.roomId, + ); + }, 10000); + + client.fragments.set(message.batchId, batch); } private async handleFragment( @@ -486,15 +520,13 @@ export class SimpleServer { ): Promise { const batch = client.fragments.get(message.batchId); if (!batch) { - const error: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.FragmentTimeout, - message: "Fragment batch not found", - batchId: message.batchId, - }; - this.sendMessage(client.ws, error); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.FragmentTimeout, + message.crdt, + message.roomId, + ); return; } @@ -503,6 +535,7 @@ export class SimpleServer { // Check if all fragments received if (batch.received === batch.data.length) { + if (batch.timeoutId) clearTimeout(batch.timeoutId); // Reconstruct the complete update const totalData = new Uint8Array(batch.totalSize); let offset = 0; @@ -517,29 +550,25 @@ export class SimpleServer { // DocUpdate messages. const roomKey = this.getRoomKey(message.roomId, message.crdt); if (!client.rooms.has(roomKey)) { - const error: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.PermissionDenied, - message: "Must join room before sending fragments", - batchId: message.batchId, - }; - this.sendMessage(client.ws, error); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.PermissionDenied, + message.crdt, + message.roomId, + ); return; } const permission = client.permissions.get(roomKey); if (permission !== "write") { - const error: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.PermissionDenied, - message: "Write permission required to update document", - batchId: message.batchId, - }; - this.sendMessage(client.ws, error); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.PermissionDenied, + message.crdt, + message.roomId, + ); client.fragments.delete(message.batchId); return; } @@ -556,14 +585,14 @@ export class SimpleServer { ); roomDoc.data = newDocumentData; } catch (error) { - const updateError: UpdateError = { - type: MessageType.UpdateError, - crdt: message.crdt, - roomId: message.roomId, - code: UpdateErrorCode.InvalidUpdate, - message: error instanceof Error ? error.message : "Invalid update", - }; - this.sendMessage(client.ws, updateError); + console.warn("applyUpdates failed for fragmented DocUpdate", error); + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.InvalidUpdate, + message.crdt, + message.roomId, + ); client.fragments.delete(message.batchId); return; } @@ -571,6 +600,15 @@ export class SimpleServer { roomDoc.dirty = true; } + // Notify sender + this.sendAck( + client.ws, + message.batchId, + UpdateStatusCode.Ok, + message.crdt, + message.roomId, + ); + // Broadcast original fragments to other clients in the room const header = client.fragments.get(message.batchId)!.header; this.wss?.clients.forEach(ws => { @@ -603,6 +641,9 @@ export class SimpleServer { private handleDisconnect(client: ClientConnection): void { client.rooms.clear(); + for (const [, frag] of client.fragments) { + if (frag.timeoutId) clearTimeout(frag.timeoutId); + } client.fragments.clear(); client.permissions.clear(); } @@ -669,6 +710,27 @@ export class SimpleServer { } } + private newBatchId(): HexString { + return `0x${randomBytes(8).toString("hex")}`; + } + + private sendAck( + ws: WebSocket, + refId: HexString, + status: UpdateStatusCode, + crdt: CrdtType, + roomId: RoomId + ): void { + const ack: Ack = { + type: MessageType.Ack, + crdt, + roomId, + refId, + status, + }; + this.sendMessage(ws, ack); + } + private broadcastToRoom( roomId: RoomId, crdtType: CrdtType, diff --git a/packages/loro-websocket/tests/e2e-elo.test.ts b/packages/loro-websocket/tests/e2e-elo.test.ts index 807a5c7..fc57975 100644 --- a/packages/loro-websocket/tests/e2e-elo.test.ts +++ b/packages/loro-websocket/tests/e2e-elo.test.ts @@ -8,9 +8,9 @@ import { encodeEloContainer, tryDecode, MessageType, - UpdateErrorCode, + UpdateStatusCode, type ProtocolMessage, - type UpdateError, + type Ack, } from "loro-protocol"; import { EloAdaptor } from "loro-adaptors/loro"; @@ -48,7 +48,9 @@ describe("E2E: %ELO join/backfill and live updates", () => { }); afterAll(async () => { - await server.stop(); + if (server) { + await server.stop(); + } }, 15000); it("joins with backfill and syncs live", async () => { @@ -154,7 +156,9 @@ describe("E2E: %ELO server validation and errors", () => { }); afterAll(async () => { - await server.stop(); + if (server) { + await server.stop(); + } }, 15000); it("rejects delta with invalid IV length", async () => { @@ -184,17 +188,20 @@ describe("E2E: %ELO server validation and errors", () => { const container = encodeEloContainer([rec]); // Send DocUpdate + const batchId = "0x0102030405060708"; ws.send( encode({ type: MessageType.DocUpdate, crdt: "%ELO", roomId: "elo-bad", updates: [container], + batchId, } as any) ); - const err = await waitForUpdateError(ws); - expect(err.code).toBe(UpdateErrorCode.InvalidUpdate); + const ack = await waitForAck(ws); + expect(ack.status).toBe(UpdateStatusCode.InvalidUpdate); + expect(ack.refId).toBe(batchId); }, 15000); it("rejects delta with too-long peerId and keyId", async () => { @@ -222,17 +229,20 @@ describe("E2E: %ELO server validation and errors", () => { ct: new Uint8Array([0]), }); const container = encodeEloContainer([rec]); + const batchId = "0x0100000000000000"; ws.send( encode({ type: MessageType.DocUpdate, crdt: "%ELO", roomId: "elo-bad2", updates: [container], + batchId, } as any) ); - const err = await waitForUpdateError(ws); - expect(err.code).toBe(UpdateErrorCode.InvalidUpdate); + const ack = await waitForAck(ws); + expect(ack.status).toBe(UpdateStatusCode.InvalidUpdate); + expect(ack.refId).toBe(batchId); }, 15000); it("client encode rejects payloads larger than MAX_MESSAGE_SIZE", async () => { @@ -258,6 +268,7 @@ describe("E2E: %ELO server validation and errors", () => { crdt: "%ELO", roomId: "elo-big", updates: [big], + batchId: "0x00ff00ff00ff00ff", } as any); } catch (e) { threw = true; @@ -360,7 +371,7 @@ async function waitForJoinOk(ws: WebSocket): Promise { }); } -async function waitForUpdateError(ws: WebSocket): Promise { +async function waitForAck(ws: WebSocket): Promise { return await new Promise((resolve, reject) => { const t = setTimeout(() => { reject(new Error("timeout")); @@ -368,7 +379,7 @@ async function waitForUpdateError(ws: WebSocket): Promise { ws.on("message", (data: unknown) => { try { const msg = tryDecodeMsg(data); - if (msg?.type === MessageType.UpdateError) { + if (msg?.type === MessageType.Ack) { clearTimeout(t); resolve(msg); } diff --git a/packages/loro-websocket/tests/e2e-update-error.test.ts b/packages/loro-websocket/tests/e2e-update-error.test.ts new file mode 100644 index 0000000..15ad2a2 --- /dev/null +++ b/packages/loro-websocket/tests/e2e-update-error.test.ts @@ -0,0 +1,151 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { WebSocketServer, WebSocket } from "ws"; +import getPort from "get-port"; +import { LoroAdaptor } from "loro-adaptors/loro"; +import { + Ack, + MessageType, + UpdateStatusCode, + encode, + decode, + type JoinRequest, + type ProtocolMessage, +} from "loro-protocol"; +import { LoroWebsocketClient } from "../src/client"; + +// Make WebSocket available globally for the client +Object.defineProperty(globalThis, "WebSocket", { + value: WebSocket, + configurable: true, + writable: true, +}); + +describe("E2E: onUpdateError", () => { + let server: WebSocketServer; + let port: number; + let lastUpdates: Uint8Array[] = []; + + beforeAll(async () => { + try { + port = await getPort(); + server = new WebSocketServer({ port }); + + server.on("connection", ws => { + ws.on("message", data => { + const msg = decode(toUint8Array(data)); + if (!msg) return; + handleMessage(ws, msg); + }); + }); + } catch (e) { + console.warn("Skipping onUpdateError e2e: cannot start ws server", e); + } + }); + + afterAll(async () => { + for (const client of server.clients) { + try { + client.terminate(); + } catch { } + } + await new Promise(resolve => { + server.close(() => { + resolve(); + }); + }); + }); + + it("invokes adaptor onUpdateError with original batch", async () => { + const errors: Array<{ updates: Uint8Array[]; code: number; reason?: string }> = []; + const adaptor = new LoroAdaptor(undefined, { + onUpdateError: (updates, code, reason) => { + errors.push({ updates: updates.map(u => u.slice()), code, reason }); + }, + }); + + const client = new LoroWebsocketClient({ url: `ws://localhost:${port}` }); + await client.waitConnected(); + + const room = await client.join({ roomId: "err-room", crdtAdaptor: adaptor }); + + // Emit one local update + const text = adaptor.getDoc().getText("body"); + text.insert(0, "boom"); + adaptor.getDoc().commit(); + + await waitUntil(() => errors.length > 0, 2000, 25); + + const err = errors[0]; + expect(err.code).toBe(UpdateStatusCode.InvalidUpdate); + expect(err.reason).toBe("invalid_update"); + expect(err.updates.length).toBe(1); + expect(lastUpdates.length).toBe(1); + expect((err.updates[0]?.length ?? 0)).toBeGreaterThan(0); + + await room.destroy(); + client.destroy(); + }, 5000); + + function handleMessage(ws: WebSocket, msg: ProtocolMessage) { + switch (msg.type) { + case MessageType.JoinRequest: { + const jr: JoinRequest = msg; + ws.send( + encode({ + type: MessageType.JoinResponseOk, + crdt: jr.crdt, + roomId: jr.roomId, + permission: "write", + version: new Uint8Array(), + extraMetadata: new Uint8Array(), + }) + ); + break; + } + case MessageType.DocUpdate: { + lastUpdates = msg.updates.map(u => new Uint8Array(u)); + const ack: Ack = { + type: MessageType.Ack, + crdt: msg.crdt, + roomId: msg.roomId, + refId: msg.batchId, + status: UpdateStatusCode.InvalidUpdate, + }; + ws.send(encode(ack)); + break; + } + default: + break; + } + } +}); + +function toUint8Array(data: unknown): Uint8Array { + if (typeof data === "string") return new TextEncoder().encode(data); + if (data instanceof ArrayBuffer) + return new Uint8Array(data); + // @ts-expect-error Buffer from ws + if (typeof Buffer !== "undefined" && Buffer.isBuffer(data)) { + // @ts-expect-error Buffer type + return new Uint8Array(data); + } + if (Array.isArray(data)) { + // eslint-disable-next-line @typescript-eslint/no-unsafe-argument + // @ts-expect-error Buffer[] from ws aggregate + return toUint8Array(Buffer.concat(data)); + } + return new Uint8Array(); +} + +async function waitUntil( + cond: () => boolean, + timeoutMs: number, + interval = 25 +): Promise { + const start = Date.now(); + while (Date.now() - start < timeoutMs) { + if (cond()) return; + await new Promise(r => setTimeout(r, interval)); + } + throw new Error("Condition not met within timeout"); +} diff --git a/packages/loro-websocket/tests/e2e.test.ts b/packages/loro-websocket/tests/e2e.test.ts index ba9c931..67f6fcb 100644 --- a/packages/loro-websocket/tests/e2e.test.ts +++ b/packages/loro-websocket/tests/e2e.test.ts @@ -1,5 +1,6 @@ import { describe, it, expect, beforeAll, afterAll } from "vitest"; import { WebSocket, WebSocketServer } from "ws"; +import type { RawData } from "ws"; import getPort from "get-port"; import { SimpleServer } from "../src/server/simple-server"; import { LoroWebsocketClient, ClientStatus } from "../src/client"; @@ -7,6 +8,16 @@ import type { LoroWebsocketClientRoom } from "../src/client"; import { LoroAdaptor } from "loro-adaptors/loro"; import { FlockAdaptor } from "loro-adaptors/flock"; import { Flock } from "@loro-dev/flock"; +import { + encode, + decode, + MessageType, + RoomErrorCode, + type JoinResponseOk, + type RoomError, +} from "loro-protocol"; + + // Make WebSocket available globally for the client Object.defineProperty(globalThis, "WebSocket", { @@ -30,7 +41,6 @@ describe("E2E: Client-Server Sync", () => { }, 15000); it("should sync two clients through server", async () => { - // Create two clients const client1 = new LoroWebsocketClient({ url: `ws://localhost:${port}` }); const client2 = new LoroWebsocketClient({ url: `ws://localhost:${port}` }); @@ -420,9 +430,9 @@ describe("E2E: Client-Server Sync", () => { await waitUntil( () => statuses1.filter(s => s === ClientStatus.Connected).length > - initialConnected1 && + initialConnected1 && statuses2.filter(s => s === ClientStatus.Connected).length > - initialConnected2, + initialConnected2, 5000, 25 ); @@ -623,7 +633,7 @@ describe("E2E: Client-Server Sync", () => { ? data : Buffer.isBuffer(data) ? data.toString() - : new TextDecoder().decode(new Uint8Array(data as ArrayBuffer)); + : new TextDecoder().decode(new Uint8Array(data)); if (text === "ping") { // Only respond for non-fatal connections if (connId >= 2) { @@ -635,7 +645,7 @@ describe("E2E: Client-Server Sync", () => { setTimeout(() => { try { ws.close(1008, "policy"); - } catch {} + } catch { } }, 30); } }); @@ -830,7 +840,7 @@ describe("E2E: Client-Server Sync", () => { ? data : Buffer.isBuffer(data) ? data.toString() - : new TextDecoder().decode(new Uint8Array(data as ArrayBuffer)); + : new TextDecoder().decode(new Uint8Array(data)); if (text === "ping") { if (id >= 2) ws.send("pong"); // first connection intentionally ignores to trigger timeout @@ -870,6 +880,116 @@ describe("E2E: Client-Server Sync", () => { }, 12000); }); +describe("E2E: RoomError rejoin policy", () => { + it("does not auto rejoin after eviction (0x02)", async () => { + const port = await getPort(); + const wss = new WebSocketServer({ port }); + let joinCount = 0; + + wss.on("connection", ws => { + ws.on("message", data => { + let msg; + try { + msg = decode(toUint8Array(data)); + } catch { + return; + } + if (msg.type === MessageType.JoinRequest) { + joinCount += 1; + const res: JoinResponseOk = { + type: MessageType.JoinResponseOk, + crdt: msg.crdt, + roomId: msg.roomId, + permission: "write", + version: new Uint8Array(), + }; + ws.send(encode(res)); + setTimeout(() => { + const err: RoomError = { + type: MessageType.RoomError, + crdt: msg.crdt, + roomId: msg.roomId, + code: RoomErrorCode.Evicted, + message: "evicted", + }; + ws.send(encode(err)); + }, 50); + } + }); + }); + + const client = new LoroWebsocketClient({ url: `ws://localhost:${port}` }); + await client.waitConnected(); + const adaptor = new LoroAdaptor(); + await client.join({ roomId: "room-evict", crdtAdaptor: adaptor }); + + // Wait to allow potential auto-rejoin attempts + await new Promise(r => setTimeout(r, 400)); + + expect(joinCount).toBe(1); + + client.destroy(); + wss.close(); + }, 5000); + + it("auto rejoins once when server suggests rejoin (0x01)", async () => { + const port = await getPort(); + const wss = new WebSocketServer({ port }); + let joinCount = 0; + + wss.on("connection", ws => { + ws.on("message", data => { + let msg; + try { + msg = decode(toUint8Array(data)); + } catch { + return; + } + if (msg.type === MessageType.JoinRequest) { + joinCount += 1; + const res: JoinResponseOk = { + type: MessageType.JoinResponseOk, + crdt: msg.crdt, + roomId: msg.roomId, + permission: "write", + version: new Uint8Array(), + }; + ws.send(encode(res)); + // Only prompt rejoin on first join + if (joinCount === 1) { + setTimeout(() => { + const err: RoomError = { + type: MessageType.RoomError, + crdt: msg.crdt, + roomId: msg.roomId, + code: RoomErrorCode.RejoinSuggested, + message: "please rejoin", + }; + ws.send(encode(err)); + }, 50); + } + } + }); + }); + + const client = new LoroWebsocketClient({ + url: `ws://localhost:${port}`, + reconnect: { initialDelayMs: 50, maxDelayMs: 150, jitter: 0 }, + }); + await client.waitConnected(); + const adaptor = new LoroAdaptor(); + await client.join({ roomId: "room-rejoin", crdtAdaptor: adaptor }); + + await waitUntil(() => joinCount >= 2, 5000, 20).catch(() => { }); + // Ensure no extra rejoins + await new Promise(r => setTimeout(r, 200)); + expect(joinCount).toBe(2); + + client.destroy(); + wss.close(); + }, 8000); +}); + function installMockWindow(initialOnline = true) { const originalWindowDescriptor = Object.getOwnPropertyDescriptor( globalThis, @@ -994,3 +1114,19 @@ async function waitUntil( } throw new Error("Condition not met within timeout"); } + +function toUint8Array(data: RawData | string): Uint8Array { + if (typeof data === "string") return new TextEncoder().encode(data); + if (Array.isArray(data)) { + // RawData can be Buffer[] when `ws` aggregates chunks + // eslint-disable-next-line @typescript-eslint/no-unsafe-argument + return toUint8Array(Buffer.concat(data as Buffer[])); + } + if (data instanceof ArrayBuffer) return new Uint8Array(data); + if (ArrayBuffer.isView(data)) return new Uint8Array(data.buffer, data.byteOffset, data.byteLength); + // Buffer from ws in Node + // @ts-ignore + if (typeof Buffer !== "undefined" && Buffer.isBuffer(data)) return new Uint8Array(data); + return new Uint8Array(); +} +// (duplicate WebSocketServer import removed) diff --git a/packages/loro-websocket/tests/permission.test.ts b/packages/loro-websocket/tests/permission.test.ts index 8430ecf..77e86b2 100644 --- a/packages/loro-websocket/tests/permission.test.ts +++ b/packages/loro-websocket/tests/permission.test.ts @@ -5,6 +5,8 @@ import { SimpleServer } from "../src/server/simple-server"; import { LoroWebsocketClient } from "../src/client"; import { LoroAdaptor } from "loro-adaptors/loro"; + + // Make WebSocket available globally for the client Object.defineProperty(globalThis, "WebSocket", { value: WebSocket, @@ -20,7 +22,7 @@ describe("Permission Enforcement", () => { port = await getPort(); server = new SimpleServer({ port, - authenticate: async (roomId, crdtType, auth) => { + authenticate: async (_roomId, _crdtType, auth) => { const authStr = new TextDecoder().decode(auth); return authStr === "readonly" ? "read" : "write"; }, diff --git a/protocol-e2ee.md b/protocol-e2ee.md index b13fb26..6a154b3 100644 --- a/protocol-e2ee.md +++ b/protocol-e2ee.md @@ -83,7 +83,7 @@ Notes: - The server MUST be able to parse `kind`, version metadata (`peerId/start/end` for delta or `vv` for snapshot), `iv`, and `keyId` without decrypting `ct`. - The CRDT bytes (Loro updates or snapshot encoding) are entirely inside `ct` and only readable by clients with the correct key. -- Field bounds: Implementations SHOULD reject records with `peerId` > 64 bytes or `keyId` > 64 UTF‑8 bytes with `UpdateError.invalid_update`. Oversized messages remain subject to the 256 KB limit (`payload_too_large`). +- Field bounds: Implementations SHOULD reject records with `peerId` > 64 bytes or `keyId` > 64 UTF‑8 bytes with `Ack.status=invalid_update`. Oversized messages remain subject to the 256 KB limit (`payload_too_large`). ### AEAD AAD and IV @@ -116,7 +116,7 @@ Although the server cannot decrypt, it SHOULD index the plaintext headers for ef - Storage model: `Map>` where each Span is `[start,end)` with associated `keyId` and raw `record` bytes. Spans are kept sorted by `start`. - Dedup/merge (Span Override): On receiving a new DeltaSpan for `peerId = P` with span `[S,E)`, replace only if the new span fully covers existing spans for the same `peerId` (i.e., for each covered span `[s,e)`, `S ≤ s` and `E ≥ e`). This replacement is based solely on version metadata and applies regardless of `keyId`. - Querying: Given a requester version vector `VV`, send all delta spans where `end > VV[peerId]` (i.e., spans that extend beyond the requester’s counter for that `peerId`). -- Validation: Servers SHOULD validate header encodings (e.g., `end > start`, for DeltaSpan plaintext, `iv` length == 12, recommended `peerId`/`keyId` bounds) and reject malformed updates with `UpdateError.invalid_update`. Over‑limit payloads SHOULD be rejected with `UpdateError.payload_too_large`. +- Validation: Servers SHOULD validate header encodings (e.g., `end > start`, for DeltaSpan plaintext, `iv` length == 12, recommended `peerId`/`keyId` bounds) and reject malformed updates with `Ack.status=invalid_update`. Over‑limit payloads SHOULD be rejected with `Ack.status=payload_too_large`. - Snapshots: On receiving a snapshot record, applications MAY choose to replace prior stored deltas according to their retention policy (e.g., keep a window of recent deltas). Authorization for snapshot override is an application concern. The server MUST treat `keyId` as opaque metadata and MUST NOT rely on or require any specific format. @@ -131,7 +131,7 @@ type GetPrivateKey = (keyId?: string) => Promise<{ keyId: string; key: Uint8Arra - Export: when sending, export CRDT updates as contiguous spans per peer, encrypt each span into a DeltaSpan record, then encode into the container. - Import: when receiving, parse the container, for each record read the explicit `iv`, build AAD from the exact encoded header, decrypt `ct`, and batch‑import the plaintext updates/snapshot into the CRDT. -- Unknown key handling: If a record’s `keyId` is unknown, clients SHOULD fetch/resolve the key and retry decryption. If resolution fails or decryption fails with a known key, surface a local error (do not emit `UpdateError`). +- Unknown key handling: If a record’s `keyId` is unknown, clients SHOULD fetch/resolve the key and retry decryption. If resolution fails or decryption fails with a known key, surface a local error (no Ack is emitted because the server is unaware). Key rotation is handled by publishing new records with a new `keyId`. Receivers that encounter an unknown `keyId` must obtain the key via application logic and then retry decryption. @@ -143,7 +143,7 @@ Key rotation is handled by publishing new records with a new `keyId`. Receivers ## Errors and Telemetry - Client‑local decrypt failures: Report via a local callback (e.g., `onError({ kind: 'decrypt_failed' | 'unknown_key', peerId, start, end, keyId })`). Servers cannot detect these. -- Server rejections: Map header validation issues to `UpdateError.invalid_update`; size over limits to `UpdateError.payload_too_large`; fragment timeout behavior is unchanged from the base protocol. +- Server rejections: Map header validation issues to `Ack.status=invalid_update`; size over limits to `Ack.status=payload_too_large`; fragment timeout behavior is unchanged from the base protocol. - Observability: For debugging, servers SHOULD log anonymized header fields (e.g., `peerId`, `start`, `end`, `keyId`) and high‑level outcomes. Servers MUST NOT log `ct` contents. ## Normative Test Vector (DeltaSpan) diff --git a/protocol.md b/protocol.md index a1f85ea..3d85007 100644 --- a/protocol.md +++ b/protocol.md @@ -1,6 +1,6 @@ # Loro Syncing Protocol -Protocol version: 0. +Protocol version: 1. It is the application's responsibility to ensure Req and Recv use the same protocol version. @@ -62,6 +62,7 @@ Note: Keepalive frames are special and bypass this envelope entirely. When the e - 0x03: DocUpdate. - `varUint` N. - N `varBytes`. + - 8-byte Update Batch ID appended at the end of the message, used to correlate ACKs. - 0x04: DocUpdateFragmentHeader. - 8-byte ID for this batch of fragments. - `varUint` fragment count @@ -70,11 +71,14 @@ Note: Keepalive frames are special and bypass this envelope entirely. When the e - 8-byte ID for this batch of fragments. - `varUint` nth fragment. - `varBytes` update fragment. -- 0x06: UpdateError. - - 1-byte error code. See Errors. +- 0x06: RoomError. +- 1-byte error code. See Errors and status. - `varString` message (human-friendly). - - Optional: for `fragment_timeout`, append 8-byte batch ID. + - Receiving this means the peer has been removed from the room and will not receive further messages until it rejoins. - 0x07: Leave. Unsubscribe from the room. +- 0x08: Ack. + - 8-byte reference ID pointing to an Update Batch ID or a Fragment ID. + - 1-byte status. `0x00` means accepted; non-zero values follow Update Status Codes. ## Syncing Process @@ -86,9 +90,11 @@ Req sends a `JoinRequest` to Recv. When Recv receives updates in the same room from other peers, it broadcasts them to all the other peers through `DocUpdate` or `DocUpdateFragment` messages. -When Req makes local edits on the document, it sends `DocUpdate` or `DocUpdateFragment` messages to Recv. +When Req makes local edits on the document, it sends `DocUpdate` (with its Update Batch ID) or `DocUpdateFragment` messages to Recv. Recv MUST reply with an `Ack` referencing that Update Batch ID (or the fragment batch ID when fragments are used). A status of `0x00` confirms acceptance; non-zero statuses follow Update Status Codes. For example, if Req lacks write permission, Recv sends `Ack(status=0x03 permission_denied)` for that batch. + +**WebSocket (client–server) directionality:** when Recv (the server) pushes updates to Req (the client), Req SHOULD NOT send `Ack(status=0x00)` back. The server already assumes delivery over the WebSocket. The client MAY send a non‑zero `Ack` (referencing the server’s batch ID) to report that it failed to apply the update (for example `invalid_update` or `fragment_timeout`). -- If Req doesn't have permission to edit the document, an `UpdateError(code=0x03 permission_denied)` is sent to Req. +If Recv forces the peer out of the room (permission change, quota enforcement, malicious behavior, etc.), it sends `RoomError`. After receiving `RoomError`, the peer MUST treat the room as closed and will not receive further messages until it rejoins. Req sends `Leave` if it is no longer interested in updates for the target document. @@ -98,16 +104,20 @@ It is usually not efficient to send a large document without splitting it into f This protocol limits message size to 256 KB. Large updates must be split into fragments of up to 256 KB each. The default reassembly timeout is 10 seconds. +`Ack` messages for fragmented updates reference the 8-byte batch ID defined in `DocUpdateFragmentHeader`. + If Recv times out waiting for remaining fragments of a batch, it MUST: - Discard all partial fragments for that batch ID. -- Send `UpdateError(code=0x07 fragment_timeout)` with the 8-byte batch ID so Req can resend. +- Send `Ack(status=0x07 fragment_timeout)` with the 8-byte batch ID so Req can resend. -Upon receiving `fragment_timeout`, Req SHOULD resend the whole batch (header + all fragments) with the same or a new batch ID. +Upon receiving `Ack(status=0x07 fragment_timeout)`, Req SHOULD resend the whole batch (header + all fragments) with the same or a new batch ID. -## Errors +If Req (client) times out while reassembling fragments sent by Recv (server), it MAY send `Ack(status=0x07 fragment_timeout)` with that batch ID to signal the failure. Servers MAY choose to resend the batch or fall back to a snapshot/delta strategy. -Two error message types with small numeric codes for clarity and easy parsing. +## Errors and status + +Two message types carry error semantics; update-level results travel in `Ack` status bytes. ### JoinError (0x02) @@ -121,19 +131,28 @@ Codes: - 0x02 auth_failed: authentication/authorization failed or the join payload was rejected. - 0x7F app_error: Extra `varString app_code` (free-form, e.g., `quota_exceeded`). -### UpdateError (0x06) +### RoomError (0x06) - Fields: 1-byte `code`, `varString message`. +- Semantics: the peer has been forcibly removed from the room (permission change, quota enforcement, malicious behavior, etc.). No further messages for the room will be delivered until it rejoins. +- Client behavior: do **not** auto-rejoin after a `RoomError` unless the code explicitly asks for it (0x01). A code of 0x01 is a hint that another peer needs you to rejoin to recover consistency (e.g., missing updates while the sender lacks your version vector). When receiving 0x01, clients SHOULD issue a single immediate `JoinRequest`; for any other code, clients MUST stay disconnected until the application chooses to rejoin. Codes: -- 0x00 unknown: unspecified error. +- 0x01 rejoin_suggested: peer requests a fresh join so both sides can reconcile state. Clients MAY auto-rejoin once. +- 0x02 evicted: explicit eviction; clients MUST NOT auto-rejoin. +- 0x7F unknown/app_error: unspecified room-level failure; treat as fatal unless the application decides otherwise. + +### Update Status Codes (Ack status byte) + +- 0x00 ok: update accepted. +- 0x01 unknown: unspecified failure. - 0x03 permission_denied: requester has no write permission. - 0x04 invalid_update: update payload is malformed or rejected. - 0x05 payload_too_large: a single message or reassembled update exceeded limits. - 0x06 rate_limited: sender is rate limited. -- 0x07 fragment_timeout: timed out waiting for remaining fragments. Extra: 8-byte `batch_id`. -- 0x7F app_error: Extra `varString app_code` (free-form, e.g., `quota_exceeded`). +- 0x07 fragment_timeout: timed out waiting for remaining fragments (use the referenced batch ID). +- 0x7F app_error: application-defined error code; details are application-specific because `Ack` carries only the status byte. ### Local (non-message) issues @@ -141,7 +160,7 @@ Codes: ### Library hook (optional) -Implementations may expose `onError({ roomId, kind, code, message, app_code? })` where `kind` is `join` or `update` based on the message type. +Implementations may expose `onError({ roomId, kind, code, message, app_code? })` for `join` or `room` messages and `onUpdateStatus({ roomId, refId, status })` to surface non-zero `Ack` statuses. ## Keepalive: Ping/Pong (Out‑of‑Band) @@ -154,4 +173,19 @@ Some environments (e.g., browsers) do not expose transport‑level WebSocket pin - Rate limiting: Implementations MAY rate‑limit excessive keepalive traffic. - Timeouts: Applications MAY use `ping`/`pong` round‑trip time to detect dead connections and reconnect. -Implementation note: On platforms that support automatic responses (e.g., Cloudflare Durable Objects), servers MAY configure an auto‑response mapping `ping -> pong` to avoid waking application logic for keepalive traffic. +Implementation note: On platforms that support automatic responses (e.g., Cloudflare Durable Objects), servers MAY configure an auto-response mapping `ping -> pong` to avoid waking application logic for keepalive traffic. + +## Version Differences + +- v1 appends an 8-byte Update Batch ID to every `DocUpdate` so peers can correlate acknowledgments. +- v1 introduces `Ack` (0x08) to positively confirm or reject updates; v0 relied on `UpdateError` messages and had no explicit success signal. +- v1 repurposes 0x06 from `UpdateError` to `RoomError`, which evicts the peer from the room until it rejoins. +- v1 shifts update status codes: `ok` is now `0x00`, `unknown` moves to `0x01`, and the remaining codes keep their numeric values from v0. +- v1 clarifies that a `RoomError` means the peer will stop receiving room traffic unless it performs a new join handshake. + +## Why these changes (v1) + +- Reliable delivery semantics: explicit `Ack` for each client‑originated update batch (or fragment batch) lets applications know whether a change was accepted instead of inferring success from silence; clients only emit `Ack` when a server‑originated update fails. +- Clear eviction signal: `RoomError` distinguishes "your update failed" from "you are no longer in the room", enabling clients to stop syncing and prompt rejoin or escalation. +- Better error mapping: moving `ok` to `0x00` and `unknown` to `0x01` aligns status bytes with common success/failure conventions and leaves room for app-specific errors. +- Debuggability: 64-bit batch IDs make ACK correlation collision-resistant even on long-lived, multiplexed connections while adding negligible overhead versus the 256 KiB frame limit. diff --git a/rust/loro-protocol/README.md b/rust/loro-protocol/README.md index 0c47ae5..cb7547d 100644 --- a/rust/loro-protocol/README.md +++ b/rust/loro-protocol/README.md @@ -3,7 +3,7 @@ Rust implementation of the Loro syncing protocol encoder/decoder. Mirrors the TypeScript package in `packages/loro-protocol` and follows the wire format described in `protocol.md` and the end-to-end encrypted flow in `protocol-e2ee.md`. ## Features -- Encode/decode Join, DocUpdate, FragmentHeader/Fragment, UpdateError, and Leave messages +- Encode/decode Join, DocUpdate (with batch IDs), FragmentHeader/Fragment, Ack, RoomError, and Leave messages - 256 KiB message size guard to match the wire spec - Bytes utilities (`BytesWriter`, `BytesReader`) for varint/varbytes/varstring - `%ELO` container parsing; Rust-side encryption helpers are WIP and may evolve diff --git a/rust/loro-protocol/src/bytes.rs b/rust/loro-protocol/src/bytes.rs index 0b6827a..552e3c7 100644 --- a/rust/loro-protocol/src/bytes.rs +++ b/rust/loro-protocol/src/bytes.rs @@ -9,7 +9,9 @@ pub struct BytesWriter { impl BytesWriter { #[must_use] pub fn new() -> Self { - Self { buf: Vec::with_capacity(32) } + Self { + buf: Vec::with_capacity(32), + } } #[inline] @@ -54,15 +56,21 @@ impl BytesWriter { #[inline] #[must_use] - pub fn len(&self) -> usize { self.buf.len() } + pub fn len(&self) -> usize { + self.buf.len() + } #[inline] #[must_use] - pub fn is_empty(&self) -> bool { self.buf.is_empty() } + pub fn is_empty(&self) -> bool { + self.buf.is_empty() + } } impl Default for BytesWriter { - fn default() -> Self { Self::new() } + fn default() -> Self { + Self::new() + } } pub struct BytesReader<'a> { @@ -72,11 +80,15 @@ pub struct BytesReader<'a> { impl<'a> BytesReader<'a> { #[must_use] - pub fn new(buf: &'a [u8]) -> Self { Self { buf, off: 0 } } + pub fn new(buf: &'a [u8]) -> Self { + Self { buf, off: 0 } + } #[inline] #[must_use] - pub fn remaining(&self) -> usize { self.buf.len().saturating_sub(self.off) } + pub fn remaining(&self) -> usize { + self.buf.len().saturating_sub(self.off) + } pub fn read_byte(&mut self) -> Result { if self.off >= self.buf.len() { @@ -88,10 +100,14 @@ impl<'a> BytesReader<'a> { } pub fn read_bytes(&mut self, len: usize) -> Result<&'a [u8], String> { - if self.off.checked_add(len).is_some_and(|e| e <= self.buf.len()) { + if self + .off + .checked_add(len) + .is_some_and(|e| e <= self.buf.len()) + { let start = self.off; self.off += len; - Ok(&self.buf[start..start+len]) + Ok(&self.buf[start..start + len]) } else { Err("readBytes out of bounds".into()) } @@ -104,16 +120,20 @@ impl<'a> BytesReader<'a> { loop { let byte = self.read_byte()?; result |= u64::from(byte & 0x7f) << shift; - if (byte & 0x80) == 0 { break; } + if (byte & 0x80) == 0 { + break; + } shift += 7; - if shift > 63 { return Err("uleb128 too large".into()); } + if shift > 63 { + return Err("uleb128 too large".into()); + } } Ok(result) } pub fn read_var_bytes(&mut self) -> Result<&'a [u8], String> { - let len = usize::try_from(self.read_uleb128()?) - .map_err(|_| "length too large".to_string())?; + let len = + usize::try_from(self.read_uleb128()?).map_err(|_| "length too large".to_string())?; self.read_bytes(len) } @@ -126,7 +146,9 @@ impl<'a> BytesReader<'a> { /// Current cursor position (number of bytes consumed so far). #[inline] - pub fn position(&self) -> usize { self.off } + pub fn position(&self) -> usize { + self.off + } } #[cfg(test)] @@ -136,14 +158,31 @@ mod tests { #[test] fn round_trip_uleb128() { let values: [u64; 14] = [ - 0, 1, 2, 10, 127, 128, 129, 255, 256, 16383, 16384, 0xffff, 0x1f_ffff, 0x0fff_ffff, + 0, + 1, + 2, + 10, + 127, + 128, + 129, + 255, + 256, + 16383, + 16384, + 0xffff, + 0x1f_ffff, + 0x0fff_ffff, ]; let mut w = BytesWriter::new(); - for &n in &values { w.push_uleb128(n); } + for &n in &values { + w.push_uleb128(n); + } let buf = w.finalize(); let mut r = BytesReader::new(&buf); let mut out = Vec::new(); - for _ in 0..values.len() { out.push(r.read_uleb128().unwrap()); } + for _ in 0..values.len() { + out.push(r.read_uleb128().unwrap()); + } assert_eq!(out, values); assert_eq!(r.remaining(), 0); } @@ -151,9 +190,11 @@ mod tests { #[test] fn round_trip_varbytes_and_varstring() { let empty: Vec = vec![]; - let small: Vec = vec![1,2,3,4,5]; + let small: Vec = vec![1, 2, 3, 4, 5]; let mut large: Vec = vec![0; 5000]; - for (i, b) in large.iter_mut().enumerate() { *b = (i & 0xff) as u8; } + for (i, b) in large.iter_mut().enumerate() { + *b = (i & 0xff) as u8; + } let mut w = BytesWriter::new(); w.push_var_bytes(&empty); diff --git a/rust/loro-protocol/src/elo.rs b/rust/loro-protocol/src/elo.rs index 1f92a12..1ace443 100644 --- a/rust/loro-protocol/src/elo.rs +++ b/rust/loro-protocol/src/elo.rs @@ -56,7 +56,9 @@ pub fn decode_elo_container<'a>(data: &'a [u8]) -> Result, String> for _ in 0..n { out.push(r.read_var_bytes()?); } - if r.remaining() != 0 { return Err("ELO container has trailing bytes".into()); } + if r.remaining() != 0 { + return Err("ELO container has trailing bytes".into()); + } Ok(out) } @@ -72,7 +74,10 @@ pub fn parse_elo_record_header<'a>(record: &'a [u8]) -> Result(r: &mut BytesReader<'a>, record: &'a [u8]) -> Result, String> { +fn parse_delta<'a>( + r: &mut BytesReader<'a>, + record: &'a [u8], +) -> Result, String> { let peer_id = r.read_var_bytes()?.to_vec(); let start = r.read_uleb128()?; let end = r.read_uleb128()?; @@ -80,27 +85,47 @@ fn parse_delta<'a>(r: &mut BytesReader<'a>, record: &'a [u8]) -> Result start".into()); } - if iv_bytes.len() != 12 { return Err("Invalid ELO delta span: IV must be 12 bytes".into()); } - if peer_id.len() > 64 { return Err("Invalid ELO delta span: peerId too long".into()); } - if key_id.as_bytes().len() > 64 { return Err("Invalid ELO delta span: keyId too long".into()); } + if end <= start { + return Err("Invalid ELO delta span: end must be > start".into()); + } + if iv_bytes.len() != 12 { + return Err("Invalid ELO delta span: IV must be 12 bytes".into()); + } + if peer_id.len() > 64 { + return Err("Invalid ELO delta span: peerId too long".into()); + } + if key_id.as_bytes().len() > 64 { + return Err("Invalid ELO delta span: keyId too long".into()); + } let mut iv = [0u8; 12]; iv.copy_from_slice(iv_bytes); Ok(ParsedEloRecord { kind: EloRecordKind::DeltaSpan, - header: EloHeader::Delta(EloDeltaHeader { peer_id, start, end, key_id, iv }), + header: EloHeader::Delta(EloDeltaHeader { + peer_id, + start, + end, + key_id, + iv, + }), aad: &record[..aad_len], ct, }) } -fn parse_snapshot<'a>(r: &mut BytesReader<'a>, record: &'a [u8]) -> Result, String> { - let count = usize::try_from(r.read_uleb128()?).map_err(|_| "vv length too large".to_string())?; +fn parse_snapshot<'a>( + r: &mut BytesReader<'a>, + record: &'a [u8], +) -> Result, String> { + let count = + usize::try_from(r.read_uleb128()?).map_err(|_| "vv length too large".to_string())?; let mut vv: Vec<(Vec, u64)> = Vec::with_capacity(count); for _ in 0..count { let pid = r.read_var_bytes()?.to_vec(); @@ -111,12 +136,20 @@ fn parse_snapshot<'a>(r: &mut BytesReader<'a>, record: &'a [u8]) -> Result 64 { return Err("Invalid ELO snapshot: keyId too long".into()); } + if iv_bytes.len() != 12 { + return Err("Invalid ELO snapshot: IV must be 12 bytes".into()); + } + if key_id.as_bytes().len() > 64 { + return Err("Invalid ELO snapshot: keyId too long".into()); + } // vv should be sorted by peer_id bytes ascending - if !is_sorted_by_bytes(&vv) { return Err("Invalid ELO snapshot: vv not sorted by peer id bytes".into()); } + if !is_sorted_by_bytes(&vv) { + return Err("Invalid ELO snapshot: vv not sorted by peer id bytes".into()); + } let mut iv = [0u8; 12]; iv.copy_from_slice(iv_bytes); @@ -142,7 +175,9 @@ fn compare_bytes(a: &[u8], b: &[u8]) -> i32 { for i in 0..n { let da = a[i]; let db = b[i]; - if da != db { return (da as i32) - (db as i32); } + if da != db { + return (da as i32) - (db as i32); + } } (a.len() as i32) - (b.len() as i32) } diff --git a/rust/loro-protocol/src/encoding.rs b/rust/loro-protocol/src/encoding.rs index e083c7a..a23c022 100644 --- a/rust/loro-protocol/src/encoding.rs +++ b/rust/loro-protocol/src/encoding.rs @@ -12,12 +12,18 @@ fn encode_crdt(w: &mut BytesWriter, crdt: CrdtType) { } fn decode_crdt(r: &mut BytesReader) -> Result { - if r.remaining() < 4 { return Err("Invalid message: too short for CRDT type".into()); } + if r.remaining() < 4 { + return Err("Invalid message: too short for CRDT type".into()); + } let bytes = r.read_bytes(4)?; let mut arr = [0u8; 4]; arr.copy_from_slice(bytes); - CrdtType::from_magic_bytes(arr).ok_or_else(|| format!("Invalid CRDT type: {}{}{}{}", - arr[0] as char, arr[1] as char, arr[2] as char, arr[3] as char)) + CrdtType::from_magic_bytes(arr).ok_or_else(|| { + format!( + "Invalid CRDT type: {}{}{}{}", + arr[0] as char, arr[1] as char, arr[2] as char, arr[3] as char + ) + }) } fn encode_permission(w: &mut BytesWriter, p: Permission) { @@ -46,16 +52,59 @@ pub fn encode(message: &ProtocolMessage) -> Result, String> { // Common: crdt, room_id, type match message { - ProtocolMessage::JoinRequest { crdt, room_id, auth: _, version: _ } - | ProtocolMessage::JoinResponseOk { crdt, room_id, permission: _, version: _, extra: _ } - | ProtocolMessage::JoinError { crdt, room_id, code: _, message: _, receiver_version: _, app_code: _ } - | ProtocolMessage::DocUpdate { crdt, room_id, updates: _ } - | ProtocolMessage::DocUpdateFragmentHeader { crdt, room_id, batch_id: _, fragment_count: _, total_size_bytes: _ } - | ProtocolMessage::DocUpdateFragment { crdt, room_id, batch_id: _, index: _, fragment: _ } - | ProtocolMessage::UpdateError { crdt, room_id, code: _, message: _, batch_id: _, app_code: _ } + ProtocolMessage::JoinRequest { + crdt, + room_id, + auth: _, + version: _, + } + | ProtocolMessage::JoinResponseOk { + crdt, + room_id, + permission: _, + version: _, + extra: _, + } + | ProtocolMessage::JoinError { + crdt, + room_id, + code: _, + message: _, + receiver_version: _, + app_code: _, + } + | ProtocolMessage::DocUpdate { crdt, room_id, .. } + | ProtocolMessage::DocUpdateFragmentHeader { + crdt, + room_id, + batch_id: _, + fragment_count: _, + total_size_bytes: _, + } + | ProtocolMessage::DocUpdateFragment { + crdt, + room_id, + batch_id: _, + index: _, + fragment: _, + } + | ProtocolMessage::RoomError { + crdt, + room_id, + code: _, + message: _, + } + | ProtocolMessage::Ack { + crdt, + room_id, + ref_id: _, + status: _, + } | ProtocolMessage::Leave { crdt, room_id } => { let room_id_bytes = room_id.as_bytes(); - if room_id_bytes.len() > MAX_ROOM_ID_LENGTH { return Err("Room ID too long".into()); } + if room_id_bytes.len() > MAX_ROOM_ID_LENGTH { + return Err("Room ID too long".into()); + } encode_crdt(&mut w, *crdt); w.push_var_bytes(room_id_bytes); } @@ -67,9 +116,12 @@ pub fn encode(message: &ProtocolMessage) -> Result, String> { ProtocolMessage::JoinResponseOk { .. } => MessageType::JoinResponseOk as u8, ProtocolMessage::JoinError { .. } => MessageType::JoinError as u8, ProtocolMessage::DocUpdate { .. } => MessageType::DocUpdate as u8, - ProtocolMessage::DocUpdateFragmentHeader { .. } => MessageType::DocUpdateFragmentHeader as u8, + ProtocolMessage::DocUpdateFragmentHeader { .. } => { + MessageType::DocUpdateFragmentHeader as u8 + } ProtocolMessage::DocUpdateFragment { .. } => MessageType::DocUpdateFragment as u8, - ProtocolMessage::UpdateError { .. } => MessageType::UpdateError as u8, + ProtocolMessage::RoomError { .. } => MessageType::RoomError as u8, + ProtocolMessage::Ack { .. } => MessageType::Ack as u8, ProtocolMessage::Leave { .. } => MessageType::Leave as u8, }; w.push_byte(ty); @@ -80,44 +132,77 @@ pub fn encode(message: &ProtocolMessage) -> Result, String> { w.push_var_bytes(auth); w.push_var_bytes(version); } - ProtocolMessage::JoinResponseOk { permission, version, extra, .. } => { + ProtocolMessage::JoinResponseOk { + permission, + version, + extra, + .. + } => { encode_permission(&mut w, *permission); w.push_var_bytes(version); - if let Some(e) = extra { w.push_var_bytes(e); } else { w.push_var_bytes(&[]); } + if let Some(e) = extra { + w.push_var_bytes(e); + } else { + w.push_var_bytes(&[]); + } } - ProtocolMessage::JoinError { code, message, receiver_version, app_code, .. } => { + ProtocolMessage::JoinError { + code, + message, + receiver_version, + app_code, + .. + } => { w.push_byte(*code as u8); w.push_var_string(message); if matches!(code, JoinErrorCode::VersionUnknown) { - if let Some(v) = receiver_version { w.push_var_bytes(v); } + if let Some(v) = receiver_version { + w.push_var_bytes(v); + } } if matches!(code, JoinErrorCode::AppError) { - if let Some(app) = app_code { w.push_var_string(app); } + if let Some(app) = app_code { + w.push_var_string(app); + } } } - ProtocolMessage::DocUpdate { updates, .. } => { + ProtocolMessage::DocUpdate { + updates, batch_id, .. + } => { w.push_uleb128(updates.len() as u64); - for u in updates { w.push_var_bytes(u); } + for u in updates { + w.push_var_bytes(u); + } + // Append 8-byte Update Batch ID (fixed width for hot-path parsing) + w.push_bytes(&batch_id.0); } - ProtocolMessage::DocUpdateFragmentHeader { batch_id, fragment_count, total_size_bytes, .. } => { + ProtocolMessage::DocUpdateFragmentHeader { + batch_id, + fragment_count, + total_size_bytes, + .. + } => { w.push_bytes(&batch_id.0); w.push_uleb128(*fragment_count); w.push_uleb128(*total_size_bytes); } - ProtocolMessage::DocUpdateFragment { batch_id, index, fragment, .. } => { + ProtocolMessage::DocUpdateFragment { + batch_id, + index, + fragment, + .. + } => { w.push_bytes(&batch_id.0); w.push_uleb128(*index); w.push_var_bytes(fragment); } - ProtocolMessage::UpdateError { code, message, batch_id, app_code, .. } => { + ProtocolMessage::RoomError { code, message, .. } => { w.push_byte(*code as u8); w.push_var_string(message); - if matches!(code, UpdateErrorCode::FragmentTimeout) { - if let Some(id) = batch_id { w.push_bytes(&id.0); } - } - if matches!(code, UpdateErrorCode::AppError) { - if let Some(app) = app_code { w.push_var_string(app); } - } + } + ProtocolMessage::Ack { ref_id, status, .. } => { + w.push_bytes(&ref_id.0); + w.push_byte(*status as u8); } ProtocolMessage::Leave { .. } => {} } @@ -126,7 +211,8 @@ pub fn encode(message: &ProtocolMessage) -> Result, String> { if out.len() > MAX_MESSAGE_SIZE { return Err(format!( "Message size {} exceeds maximum {}", - out.len(), MAX_MESSAGE_SIZE + out.len(), + MAX_MESSAGE_SIZE )); } Ok(out) @@ -152,68 +238,142 @@ pub fn decode(buf: &[u8]) -> Result { MessageType::JoinRequest => { let auth = r.read_var_bytes()?.to_vec(); let version = r.read_var_bytes()?.to_vec(); - PM::JoinRequest { crdt, room_id, auth, version } + PM::JoinRequest { + crdt, + room_id, + auth, + version, + } } MessageType::JoinResponseOk => { let permission = decode_permission(&mut r)?; let version = r.read_var_bytes()?.to_vec(); let extra = r.read_var_bytes()?.to_vec(); - PM::JoinResponseOk { crdt, room_id, permission, version, extra: Some(extra) } + PM::JoinResponseOk { + crdt, + room_id, + permission, + version, + extra: Some(extra), + } } MessageType::JoinError => { - let code = r.read_byte().ok().and_then(JoinErrorCode::from_u8).unwrap_or(JoinErrorCode::Unknown); + let code = r + .read_byte() + .ok() + .and_then(JoinErrorCode::from_u8) + .unwrap_or(JoinErrorCode::Unknown); let message = r.read_var_string()?; let mut receiver_version = None; let mut app_code = None; if matches!(code, JoinErrorCode::VersionUnknown) && r.remaining() > 0 { - if let Ok(bytes) = r.read_var_bytes() { receiver_version = Some(bytes.to_vec()); } + if let Ok(bytes) = r.read_var_bytes() { + receiver_version = Some(bytes.to_vec()); + } } if matches!(code, JoinErrorCode::AppError) && r.remaining() > 0 { - if let Ok(app) = r.read_var_string() { app_code = Some(app); } + if let Ok(app) = r.read_var_string() { + app_code = Some(app); + } + } + PM::JoinError { + crdt, + room_id, + code, + message, + receiver_version, + app_code, } - PM::JoinError { crdt, room_id, code, message, receiver_version, app_code } } MessageType::DocUpdate => { let count = r.read_uleb128()? as usize; let mut updates = Vec::with_capacity(count); - for _ in 0..count { updates.push(r.read_var_bytes()?.to_vec()); } - PM::DocUpdate { crdt, room_id, updates } + for _ in 0..count { + updates.push(r.read_var_bytes()?.to_vec()); + } + if r.remaining() < 8 { + return Err("Invalid DocUpdate: missing batch ID".into()); + } + let id = r.read_bytes(8)?; + let mut arr = [0u8; 8]; + arr.copy_from_slice(id); + let batch_id = BatchId(arr); + PM::DocUpdate { + crdt, + room_id, + updates, + batch_id, + } } MessageType::DocUpdateFragmentHeader => { - if r.remaining() < 8 { return Err("Invalid DocUpdateFragmentHeader: missing batch ID".into()); } + if r.remaining() < 8 { + return Err("Invalid DocUpdateFragmentHeader: missing batch ID".into()); + } let id = r.read_bytes(8)?; let mut arr = [0u8; 8]; arr.copy_from_slice(id); let batch_id = BatchId(arr); let fragment_count = r.read_uleb128()?; let total_size_bytes = r.read_uleb128()?; - PM::DocUpdateFragmentHeader { crdt, room_id, batch_id, fragment_count, total_size_bytes } + PM::DocUpdateFragmentHeader { + crdt, + room_id, + batch_id, + fragment_count, + total_size_bytes, + } } MessageType::DocUpdateFragment => { - if r.remaining() < 8 { return Err("Invalid DocUpdateFragment: missing batch ID".into()); } + if r.remaining() < 8 { + return Err("Invalid DocUpdateFragment: missing batch ID".into()); + } let id = r.read_bytes(8)?; let mut arr = [0u8; 8]; arr.copy_from_slice(id); let batch_id = BatchId(arr); let index = r.read_uleb128()?; let fragment = r.read_var_bytes()?.to_vec(); - PM::DocUpdateFragment { crdt, room_id, batch_id, index, fragment } + PM::DocUpdateFragment { + crdt, + room_id, + batch_id, + index, + fragment, + } } - MessageType::UpdateError => { - let code = r.read_byte().ok().and_then(UpdateErrorCode::from_u8).unwrap_or(UpdateErrorCode::Unknown); + MessageType::RoomError => { + let code = r + .read_byte() + .ok() + .and_then(RoomErrorCode::from_u8) + .unwrap_or(RoomErrorCode::Unknown); let message = r.read_var_string()?; - let mut batch_id = None; - let mut app_code = None; - if matches!(code, UpdateErrorCode::FragmentTimeout) && r.remaining() >= 8 { - let id = r.read_bytes(8)?; - let mut arr = [0u8; 8]; - arr.copy_from_slice(id); - batch_id = Some(BatchId(arr)); + PM::RoomError { + crdt, + room_id, + code, + message, } - if matches!(code, UpdateErrorCode::AppError) && r.remaining() > 0 { - if let Ok(app) = r.read_var_string() { app_code = Some(app); } + } + MessageType::Ack => { + if r.remaining() < 9 { + return Err("Invalid Ack: missing fields".into()); + } + let id = r.read_bytes(8)?; + let mut arr = [0u8; 8]; + arr.copy_from_slice(id); + let ref_id = BatchId(arr); + let status = r + .read_byte() + .ok() + .and_then(UpdateStatusCode::from_u8) + .unwrap_or(UpdateStatusCode::Unknown); + PM::Ack { + crdt, + room_id, + ref_id, + status, } - PM::UpdateError { crdt, room_id, code, message, batch_id, app_code } } MessageType::Leave => PM::Leave { crdt, room_id }, }; @@ -247,15 +407,20 @@ mod tests { crdt: CrdtType::Yjs, room_id: "room-123".to_string(), permission: Permission::Read, - version: vec![10,20], + version: vec![10, 20], extra: None, }; let enc = encode(&msg).unwrap(); let dec = decode(&enc).unwrap(); match dec { - ProtocolMessage::JoinResponseOk { permission, version, extra, .. } => { + ProtocolMessage::JoinResponseOk { + permission, + version, + extra, + .. + } => { assert!(matches!(permission, Permission::Read)); - assert_eq!(version, vec![10,20]); + assert_eq!(version, vec![10, 20]); // decoder reads varBytes even if empty is encoded assert_eq!(extra.unwrap_or_default(), Vec::::new()); } @@ -284,6 +449,7 @@ mod tests { crdt: CrdtType::Loro, room_id: "room-oversized".into(), updates: vec![big_update], + batch_id: BatchId([0; 8]), }; let err = encode(&msg).unwrap_err(); assert!(err.contains("exceeds maximum")); diff --git a/rust/loro-protocol/src/protocol.rs b/rust/loro-protocol/src/protocol.rs index 643fd6c..93ef04f 100644 --- a/rust/loro-protocol/src/protocol.rs +++ b/rust/loro-protocol/src/protocol.rs @@ -66,8 +66,9 @@ pub enum MessageType { DocUpdate = 0x03, DocUpdateFragmentHeader = 0x04, DocUpdateFragment = 0x05, - UpdateError = 0x06, + RoomError = 0x06, Leave = 0x07, + Ack = 0x08, } impl MessageType { @@ -79,8 +80,9 @@ impl MessageType { 0x03 => MessageType::DocUpdate, 0x04 => MessageType::DocUpdateFragmentHeader, 0x05 => MessageType::DocUpdateFragment, - 0x06 => MessageType::UpdateError, + 0x06 => MessageType::RoomError, 0x07 => MessageType::Leave, + 0x08 => MessageType::Ack, _ => return None, }) } @@ -108,11 +110,12 @@ impl JoinErrorCode { } } -/// Error codes for UpdateError (0x06). +/// Status codes carried by Ack (0x08) and mapped from legacy UpdateError codes. #[derive(Debug, Clone, Copy, PartialEq, Eq)] #[repr(u8)] -pub enum UpdateErrorCode { - Unknown = 0x00, +pub enum UpdateStatusCode { + Ok = 0x00, + Unknown = 0x01, PermissionDenied = 0x03, InvalidUpdate = 0x04, PayloadTooLarge = 0x05, @@ -121,16 +124,39 @@ pub enum UpdateErrorCode { AppError = 0x7f, } -impl UpdateErrorCode { +impl UpdateStatusCode { pub fn from_u8(v: u8) -> Option { Some(match v { - 0x00 => UpdateErrorCode::Unknown, - 0x03 => UpdateErrorCode::PermissionDenied, - 0x04 => UpdateErrorCode::InvalidUpdate, - 0x05 => UpdateErrorCode::PayloadTooLarge, - 0x06 => UpdateErrorCode::RateLimited, - 0x07 => UpdateErrorCode::FragmentTimeout, - 0x7f => UpdateErrorCode::AppError, + 0x00 => UpdateStatusCode::Ok, + 0x01 => UpdateStatusCode::Unknown, + 0x03 => UpdateStatusCode::PermissionDenied, + 0x04 => UpdateStatusCode::InvalidUpdate, + 0x05 => UpdateStatusCode::PayloadTooLarge, + 0x06 => UpdateStatusCode::RateLimited, + 0x07 => UpdateStatusCode::FragmentTimeout, + 0x7f => UpdateStatusCode::AppError, + _ => return None, + }) + } +} + +/// Error codes for RoomError (0x06). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[repr(u8)] +pub enum RoomErrorCode { + /// Peer requests that we rejoin to recover consistency (e.g., missing updates). + RejoinSuggested = 0x01, + /// Peer/server evicted us; do not auto-rejoin. + Evicted = 0x02, + Unknown = 0x7f, +} + +impl RoomErrorCode { + pub fn from_u8(v: u8) -> Option { + Some(match v { + 0x01 => RoomErrorCode::RejoinSuggested, + 0x02 => RoomErrorCode::Evicted, + 0x7f => RoomErrorCode::Unknown, _ => return None, }) } @@ -195,6 +221,7 @@ pub enum ProtocolMessage { crdt: CrdtType, room_id: String, updates: Vec>, + batch_id: BatchId, }, DocUpdateFragmentHeader { crdt: CrdtType, @@ -210,13 +237,17 @@ pub enum ProtocolMessage { index: u64, fragment: Vec, }, - UpdateError { + RoomError { crdt: CrdtType, room_id: String, - code: UpdateErrorCode, + code: RoomErrorCode, message: String, - batch_id: Option, - app_code: Option, + }, + Ack { + crdt: CrdtType, + room_id: String, + ref_id: BatchId, + status: UpdateStatusCode, }, Leave { crdt: CrdtType, diff --git a/rust/loro-protocol/tests/elo.rs b/rust/loro-protocol/tests/elo.rs index 8f557d7..c2237d7 100644 --- a/rust/loro-protocol/tests/elo.rs +++ b/rust/loro-protocol/tests/elo.rs @@ -1,6 +1,13 @@ -use loro_protocol::{bytes::BytesWriter, protocol::CrdtType, elo::*}; +use loro_protocol::{bytes::BytesWriter, elo::*, protocol::CrdtType}; -fn make_delta_record(peer: &[u8], start: u64, end: u64, key: &str, iv: &[u8;12], ct: &[u8]) -> Vec { +fn make_delta_record( + peer: &[u8], + start: u64, + end: u64, + key: &str, + iv: &[u8; 12], + ct: &[u8], +) -> Vec { let mut w = BytesWriter::new(); w.push_byte(EloRecordKind::DeltaSpan as u8); w.push_var_bytes(peer); @@ -15,7 +22,7 @@ fn make_delta_record(peer: &[u8], start: u64, end: u64, key: &str, iv: &[u8;12], w2.finalize() } -fn make_snapshot_record(vv: &[(&[u8], u64)], key: &str, iv: &[u8;12], ct: &[u8]) -> Vec { +fn make_snapshot_record(vv: &[(&[u8], u64)], key: &str, iv: &[u8; 12], ct: &[u8]) -> Vec { let mut w = BytesWriter::new(); w.push_byte(EloRecordKind::Snapshot as u8); w.push_uleb128(vv.len() as u64); @@ -41,7 +48,7 @@ fn crdt_magic_includes_elo() { #[test] fn decode_container_and_parse_delta() { let peer = b"peer-a"; - let iv: [u8;12] = [1,2,3,4,5,6,7,8,9,10,11,12]; + let iv: [u8; 12] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]; let ct: Vec = vec![0xaa, 0xbb, 0xcc, 0xdd]; let rec = make_delta_record(peer, 5, 8, "k1", &iv, &ct); @@ -73,7 +80,7 @@ fn decode_container_and_parse_delta() { fn parse_snapshot_sorted_vv_and_ct() { // Already sorted by peer bytes let vv: Vec<(&[u8], u64)> = vec![(b"a", 1), (b"b", 2), (b"zz", 9)]; - let iv: [u8;12] = [9,9,9,9,9,9,9,9,9,9,9,9]; + let iv: [u8; 12] = [9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9]; let ct: Vec = vec![0x01, 0x02]; let rec = make_snapshot_record(&vv, "kid", &iv, &ct); let parsed = parse_elo_record_header(&rec).expect("parse snapshot"); @@ -97,9 +104,9 @@ fn parse_snapshot_sorted_vv_and_ct() { fn invalid_cases() { // bad iv length let peer = b"p"; - let ct: Vec = vec![1,2,3]; + let ct: Vec = vec![1, 2, 3]; let mut iv_bad = [0u8; 12]; - iv_bad[..8].copy_from_slice(&[1,2,3,4,5,6,7,8]); + iv_bad[..8].copy_from_slice(&[1, 2, 3, 4, 5, 6, 7, 8]); // Manually construct with 8-byte iv let mut w = BytesWriter::new(); w.push_byte(EloRecordKind::DeltaSpan as u8); @@ -113,24 +120,50 @@ fn invalid_cases() { w2.push_bytes(&hdr); w2.push_var_bytes(&ct); let rec = w2.finalize(); - assert!(parse_elo_record_header(&rec).unwrap_err().contains("IV must be 12")); + assert!(parse_elo_record_header(&rec) + .unwrap_err() + .contains("IV must be 12")); // peer id too long (65 bytes) let long_peer = vec![0u8; 65]; - let rec = make_delta_record(&long_peer, 1, 2, "k", &[1,2,3,4,5,6,7,8,9,10,11,12], &[1]); - assert!(parse_elo_record_header(&rec).unwrap_err().contains("peerId too long")); + let rec = make_delta_record( + &long_peer, + 1, + 2, + "k", + &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], + &[1], + ); + assert!(parse_elo_record_header(&rec) + .unwrap_err() + .contains("peerId too long")); // key id too long (65 ascii) let long_key = "A".repeat(65); - let rec = make_delta_record(b"p", 1, 2, &long_key, &[1,2,3,4,5,6,7,8,9,10,11,12], &[1]); - assert!(parse_elo_record_header(&rec).unwrap_err().contains("keyId too long")); + let rec = make_delta_record( + b"p", + 1, + 2, + &long_key, + &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], + &[1], + ); + assert!(parse_elo_record_header(&rec) + .unwrap_err() + .contains("keyId too long")); // container trailing bytes let mut w = BytesWriter::new(); w.push_uleb128(1); - w.push_var_bytes(&make_delta_record(b"p", 1, 2, "k", &[1,2,3,4,5,6,7,8,9,10,11,12], &[1])); + w.push_var_bytes(&make_delta_record( + b"p", + 1, + 2, + "k", + &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], + &[1], + )); w.push_byte(0xff); // trailing let container = w.finalize(); assert!(decode_elo_container(&container).is_err()); } - diff --git a/rust/loro-protocol/tests/elo_normative_vector.rs b/rust/loro-protocol/tests/elo_normative_vector.rs index 1ea3429..5a4b854 100644 --- a/rust/loro-protocol/tests/elo_normative_vector.rs +++ b/rust/loro-protocol/tests/elo_normative_vector.rs @@ -14,7 +14,10 @@ fn hex_to_bytes(s: &str) -> Vec { fn bytes_to_hex(bytes: &[u8]) -> String { let mut s = String::from("0x"); - for b in bytes { use std::fmt::Write; let _ = write!(s, "{:02x}", b); } + for b in bytes { + use std::fmt::Write; + let _ = write!(s, "{:02x}", b); + } s } diff --git a/rust/loro-protocol/tests/js_snapshots.rs b/rust/loro-protocol/tests/js_snapshots.rs index fc9be77..2929809 100644 --- a/rust/loro-protocol/tests/js_snapshots.rs +++ b/rust/loro-protocol/tests/js_snapshots.rs @@ -2,7 +2,10 @@ use loro_protocol::*; fn to_hex(data: &[u8]) -> String { let mut s = String::from("0x"); - for b in data { use std::fmt::Write as _; let _ = write!(s, "{:02x}", b); } + for b in data { + use std::fmt::Write as _; + let _ = write!(s, "{:02x}", b); + } s } @@ -13,8 +16,8 @@ fn snapshot_join_request() { let msg = ProtocolMessage::JoinRequest { crdt: CrdtType::Loro, room_id: "room-1234".to_string(), - auth: vec![10,20,30], - version: vec![40,50,60], + auth: vec![10, 20, 30], + version: vec![40, 50, 60], }; let hex = to_hex(&encode(&msg).unwrap()); assert_eq!(hex, "0x254c4f5209726f6f6d2d3132333400030a141e0328323c"); @@ -27,7 +30,7 @@ fn snapshot_join_response_read() { crdt: CrdtType::Yjs, room_id: "room-1234".to_string(), permission: Permission::Read, - version: vec![11,22,33], + version: vec![11, 22, 33], extra: None, }; let hex = to_hex(&encode(&msg).unwrap()); @@ -41,11 +44,14 @@ fn snapshot_join_response_write_extra() { crdt: CrdtType::Loro, room_id: "room-1234".to_string(), permission: Permission::Write, - version: vec![44,55], - extra: Some(vec![66,77,88]), + version: vec![44, 55], + extra: Some(vec![66, 77, 88]), }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x254c4f5209726f6f6d2d3132333401057772697465022c3703424d58"); + assert_eq!( + hex, + "0x254c4f5209726f6f6d2d3132333401057772697465022c3703424d58" + ); } #[test] @@ -60,7 +66,10 @@ fn snapshot_join_error_auth_failed() { app_code: None, }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x2545504809726f6f6d2d31323334020213496e76616c69642063726564656e7469616c73"); + assert_eq!( + hex, + "0x2545504809726f6f6d2d31323334020213496e76616c69642063726564656e7469616c73" + ); } #[test] @@ -75,7 +84,10 @@ fn snapshot_join_error_version_unknown() { app_code: None, }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x254c4f5209726f6f6d2d3132333402011056657273696f6e206d69736d61746368026364"); + assert_eq!( + hex, + "0x254c4f5209726f6f6d2d3132333402011056657273696f6e206d69736d61746368026364" + ); } #[test] @@ -99,10 +111,14 @@ fn snapshot_doc_update_multiple() { let msg = ProtocolMessage::DocUpdate { crdt: CrdtType::Yjs, room_id: "room-1234".to_string(), - updates: vec![vec![1,2,3], vec![4,5,6,7], vec![8]], + updates: vec![vec![1, 2, 3], vec![4, 5, 6, 7], vec![8]], + batch_id: BatchId::from_hex("0x0102030405060708").unwrap(), }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x25594a5309726f6f6d2d3132333403030301020304040506070108"); + assert_eq!( + hex, + "0x25594a5309726f6f6d2d31323334030303010203040405060701080102030405060708" + ); } #[test] @@ -116,14 +132,19 @@ fn snapshot_fragment_header() { total_size_bytes: 1_024_000, }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x254c4f5209726f6f6d2d3132333404ff112233445566770a80c03e"); + assert_eq!( + hex, + "0x254c4f5209726f6f6d2d3132333404ff112233445566770a80c03e" + ); } #[test] fn snapshot_fragment() { // exports[`encoding snapshots > DocUpdateFragment 1`] let mut frag = vec![0u8; 32]; - for (i, b) in frag.iter_mut().enumerate() { *b = (i & 0xff) as u8; } + for (i, b) in frag.iter_mut().enumerate() { + *b = (i & 0xff) as u8; + } let msg = ProtocolMessage::DocUpdateFragment { crdt: CrdtType::Loro, room_id: "room-1234".to_string(), @@ -136,48 +157,42 @@ fn snapshot_fragment() { } #[test] -fn snapshot_update_error_permission_denied() { - // exports[`encoding snapshots > UpdateError permission denied 1`] - let msg = ProtocolMessage::UpdateError { +fn snapshot_room_error_eviction() { + // exports[`encoding snapshots > RoomError eviction 1`] + let msg = ProtocolMessage::RoomError { crdt: CrdtType::Loro, room_id: "room-1234".to_string(), - code: UpdateErrorCode::PermissionDenied, - message: "No write permission".into(), - batch_id: None, - app_code: None, + code: RoomErrorCode::Evicted, + message: "evicted".into(), }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x254c4f5209726f6f6d2d313233340603134e6f207772697465207065726d697373696f6e"); + assert_eq!(hex, "0x254c4f5209726f6f6d2d3132333406020765766963746564"); } #[test] -fn snapshot_update_error_fragment_timeout() { - // exports[`encoding snapshots > UpdateError fragment timeout with batchId 1`] - let msg = ProtocolMessage::UpdateError { - crdt: CrdtType::YjsAwareness, +fn snapshot_ack_ok() { + // exports[`encoding snapshots > Ack ok 1`] + let msg = ProtocolMessage::Ack { + crdt: CrdtType::Loro, room_id: "room-1234".to_string(), - code: UpdateErrorCode::FragmentTimeout, - message: "Fragment timeout".into(), - batch_id: Some(BatchId([1,0,0,0,0,0,0,0])), - app_code: None, + ref_id: BatchId::from_hex("0x0102030405060708").unwrap(), + status: UpdateStatusCode::Ok, }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x2559415709726f6f6d2d31323334060710467261676d656e742074696d656f75740100000000000000"); + assert_eq!(hex, "0x254c4f5209726f6f6d2d3132333408010203040506070800"); } #[test] -fn snapshot_update_error_app_error() { - // exports[`encoding snapshots > UpdateError app error with appCode 1`] - let msg = ProtocolMessage::UpdateError { - crdt: CrdtType::Loro, +fn snapshot_ack_fragment_timeout() { + // exports[`encoding snapshots > Ack fragment timeout 1`] + let msg = ProtocolMessage::Ack { + crdt: CrdtType::YjsAwareness, room_id: "room-1234".to_string(), - code: UpdateErrorCode::AppError, - message: "Custom app error".into(), - batch_id: None, - app_code: Some("custom_code_123".into()), + ref_id: BatchId([1, 0, 0, 0, 0, 0, 0, 0]), + status: UpdateStatusCode::FragmentTimeout, }; let hex = to_hex(&encode(&msg).unwrap()); - assert_eq!(hex, "0x254c4f5209726f6f6d2d31323334067f10437573746f6d20617070206572726f720f637573746f6d5f636f64655f313233"); + assert_eq!(hex, "0x2559415709726f6f6d2d3132333408010000000000000007"); } #[test] diff --git a/rust/loro-websocket-client/src/lib.rs b/rust/loro-websocket-client/src/lib.rs index d9ebe09..a40bdcb 100644 --- a/rust/loro-websocket-client/src/lib.rs +++ b/rust/loro-websocket-client/src/lib.rs @@ -47,6 +47,7 @@ use std::{ sync::{ atomic::{AtomicU64, Ordering}, Arc, + Mutex as StdMutex, }, }; use tokio::{ @@ -58,7 +59,7 @@ use tokio_tungstenite::{connect_async, tungstenite::Message, MaybeTlsStream, Web use aes_gcm::aead::{Aead, KeyInit}; use loro::LoroDoc; pub use loro_protocol as protocol; -use protocol::{encode, try_decode, CrdtType, ProtocolMessage}; +use protocol::{encode, try_decode, CrdtType, ProtocolMessage, RoomErrorCode, UpdateStatusCode}; /// Errors that may occur in the client. #[derive(Debug)] @@ -217,12 +218,7 @@ mod tests { async fn set_ctx(&mut self, _ctx: CrdtAdaptorContext) {} - async fn handle_join_ok( - &mut self, - _permission: protocol::Permission, - _version: Vec, - ) { - } + async fn handle_join_ok(&mut self, _permission: protocol::Permission, _version: Vec) {} async fn apply_update(&mut self, updates: Vec>) { self.updates.lock().await.extend(updates); @@ -237,6 +233,8 @@ mod tests { let adaptors = Arc::new(Mutex::new(HashMap::new())); let pre_join_buf = Arc::new(Mutex::new(HashMap::new())); let frag_batches = Arc::new(Mutex::new(HashMap::new())); + let batch_counter = Arc::new(AtomicU64::new(1)); + let sent_batches = Arc::new(StdMutex::new(HashMap::new())); let config = Arc::new(ClientConfig::default()); let worker = ConnectionWorker::new( @@ -246,6 +244,8 @@ mod tests { adaptors.clone(), pre_join_buf, frag_batches, + batch_counter, + sent_batches, config, ); @@ -343,6 +343,8 @@ struct ConnectionWorker { adaptors: Arc>>>, pre_join_buf: Arc>>>>, frag_batches: Arc>>, + next_batch_id: Arc, + sent_batches: Arc>>>>, config: Arc, } @@ -354,6 +356,8 @@ impl ConnectionWorker { adaptors: Arc>>>, pre_join_buf: Arc>>>>, frag_batches: Arc>>, + next_batch_id: Arc, + sent_batches: Arc>>>>, config: Arc, ) -> Self { Self { @@ -363,6 +367,8 @@ impl ConnectionWorker { adaptors, pre_join_buf, frag_batches, + next_batch_id, + sent_batches, config, } } @@ -480,18 +486,13 @@ impl ConnectionWorker { sleep(timeout).await; let mut m = batches.lock().await; if m.remove(&(key_clone.clone(), batch_id)).is_some() { - let err = ProtocolMessage::UpdateError { + let ack = ProtocolMessage::Ack { crdt: key_clone.crdt, room_id: key_clone.room.clone(), - code: protocol::UpdateErrorCode::FragmentTimeout, - message: format!( - "Fragment reassembly timeout for batch {}", - batch_id.to_hex() - ), - batch_id: Some(batch_id), - app_code: None, + ref_id: batch_id, + status: UpdateStatusCode::FragmentTimeout, }; - if let Ok(data) = encode(&err) { + if let Ok(data) = encode(&ack) { let _ = tx_timeout.send(Message::Binary(data.into())); } } @@ -534,16 +535,85 @@ impl ConnectionWorker { ); } } - ProtocolMessage::UpdateError { code, message, .. } => { + ProtocolMessage::RoomError { code, message, .. } => { + let mut adaptor = self.adaptors.lock().await.remove(&key); + if let Some(adaptor_ref) = adaptor.as_mut() { + adaptor_ref.handle_room_error(code, &message).await; + } + + // Always clear local state for this room; rejoin (if any) will register anew. + self.cleanup_room(&key).await; + eprintln!("room error {:?}: {}", code, message); + + if matches!(code, RoomErrorCode::RejoinSuggested) { + if let Some(adaptor_box) = adaptor { + let room_name = key.room.clone(); + let client = LoroWebsocketClient { + tx: self.tx.clone(), + rooms: self.rooms.clone(), + pending: self.pending.clone(), + adaptors: self.adaptors.clone(), + pre_join_buf: self.pre_join_buf.clone(), + next_batch_id: self.next_batch_id.clone(), + sent_batches: self.sent_batches.clone(), + config: self.config.clone(), + }; + tokio::spawn(async move { + if let Err(err) = client.join_with_adaptor(&room_name, adaptor_box).await { + eprintln!("rejoin after RoomError failed: {}", err); + } + }); + } + } + } + ProtocolMessage::Ack { ref_id, status, .. } => { + let sent_payloads = { + let mut map = self.sent_batches.lock().unwrap(); + map.remove(&(key.clone(), ref_id)) + }; + if let Some(adaptor) = self.adaptors.lock().await.get_mut(&key) { - adaptor.handle_update_error(code, &message).await; - } else { - eprintln!("update error (no adaptor): {:?} - {}", code, message); + if status != UpdateStatusCode::Ok { + adaptor + .handle_update_error( + sent_payloads.clone().unwrap_or_default(), + status, + Some(format!("{:?}", status)), + ) + .await; + } + adaptor.handle_ack(ref_id, status).await; + } else if status != UpdateStatusCode::Ok { + eprintln!( + "ack status {:?} for {:?} ref {:?}", + status, + key.room, + ref_id.to_hex() + ); } } ProtocolMessage::Leave { .. } | ProtocolMessage::JoinRequest { .. } => {} } } + + async fn cleanup_room(&self, key: &RoomKey) { + if let Some(state) = self.rooms.lock().await.remove(key) { + if let Some(sub) = state.sub { + sub.unsubscribe(); + } + } + self.adaptors.lock().await.remove(key); + self.pre_join_buf.lock().await.remove(key); + self.pending.lock().await.remove(key); + self.frag_batches + .lock() + .await + .retain(|(room, _), _| room != key); + self.sent_batches + .lock() + .unwrap() + .retain(|(room, _), _| room != key); + } } #[derive(Clone, Debug, PartialEq, Eq)] @@ -598,6 +668,9 @@ pub struct LoroWebsocketClient { pre_join_buf: Arc>>>>, // For generating unique fragment batch ids next_batch_id: Arc, + // Track outbound batches to surface update errors with original payloads + sent_batches: + Arc>>>>, // Configurable knobs config: Arc, } @@ -647,6 +720,9 @@ impl LoroWebsocketClient { Arc::new(Mutex::new(HashMap::new())); let frag_batches_reader: Arc>> = Arc::new(Mutex::new(HashMap::new())); + let batch_counter = Arc::new(AtomicU64::new(1)); + let sent_batches: Arc>>>> = + Arc::new(StdMutex::new(HashMap::new())); // Reader let cfg = Arc::new(config); @@ -657,6 +733,8 @@ impl LoroWebsocketClient { adaptors_reader.clone(), pre_join_buf_reader.clone(), frag_batches_reader, + batch_counter.clone(), + sent_batches.clone(), cfg.clone(), ) .spawn(stream); @@ -667,7 +745,8 @@ impl LoroWebsocketClient { pending, adaptors: adaptors_reader, pre_join_buf: pre_join_buf_reader, - next_batch_id: Arc::new(AtomicU64::new(1)), + next_batch_id: batch_counter, + sent_batches, config: cfg, }) } @@ -714,13 +793,22 @@ impl LoroWebsocketClient { if matches!(permission, protocol::Permission::Write) { let tx2 = self.tx.clone(); let key2 = key.clone(); + let batch_counter = self.next_batch_id.clone(); + let sent_batches = self.sent_batches.clone(); let sub = { let guard = doc.lock().await; guard.subscribe_local_update(Box::new(move |bytes| { + let batch_id = protocol::BatchId( + batch_counter.fetch_add(1, Ordering::Relaxed).to_be_bytes(), + ); + if let Ok(mut map) = sent_batches.lock() { + map.insert((key2.clone(), batch_id), vec![bytes.clone()]); + } let msg = ProtocolMessage::DocUpdate { crdt: key2.crdt, room_id: key2.room.clone(), updates: vec![bytes.clone()], + batch_id, }; if let Ok(data) = encode(&msg) { let _ = tx2.send(Message::Binary(data.into())); @@ -792,6 +880,8 @@ impl LoroWebsocketClient { let crdt = key.crdt; let batch_counter = self.next_batch_id.clone(); let cfg = self.config.clone(); + let sent_batches = self.sent_batches.clone(); + let room_key = key.clone(); let send_update = move |upd: Vec| { // Leave headroom for envelope overhead using configured limits. let frag_limit = std::cmp::max( @@ -802,11 +892,19 @@ impl LoroWebsocketClient { ), ); + let batch_id = + protocol::BatchId(batch_counter.fetch_add(1, Ordering::Relaxed).to_be_bytes()); + + if let Ok(mut map) = sent_batches.lock() { + map.insert((room_key.clone(), batch_id), vec![upd.clone()]); + } + if upd.len() <= frag_limit { let msg = ProtocolMessage::DocUpdate { crdt, room_id: room_vec.clone(), updates: vec![upd], + batch_id, }; if let Ok(data) = encode(&msg) { let _ = tx2.send(Message::Binary(data.into())); @@ -814,8 +912,6 @@ impl LoroWebsocketClient { } else { let total = upd.len(); let n = (total + frag_limit - 1) / frag_limit; - let batch_id = - protocol::BatchId(batch_counter.fetch_add(1, Ordering::Relaxed).to_be_bytes()); // header let header = ProtocolMessage::DocUpdateFragmentHeader { crdt, @@ -862,21 +958,8 @@ impl LoroWebsocketClient { let _ = tx_err.send(Message::Binary(data.into())); } }; - let tx_imp = self.tx.clone(); - let room_vec3 = key.room.clone(); - let crdt3 = key.crdt; let on_import_error = move |err: String, _data: Vec>| { - let msg = ProtocolMessage::UpdateError { - crdt: crdt3, - room_id: room_vec3.clone(), - code: protocol::UpdateErrorCode::AppError, - message: err, - batch_id: None, - app_code: None, - }; - if let Ok(data) = encode(&msg) { - let _ = tx_imp.send(Message::Binary(data.into())); - } + eprintln!("import error in adaptor: {}", err); }; adaptor @@ -1017,6 +1100,9 @@ impl LoroWebsocketClientRoom { } // Drop adaptor self.inner.adaptors.lock().await.remove(&self.key); + if let Ok(mut map) = self.inner.sent_batches.lock() { + map.retain(|(room, _), _| room != &self.key); + } Ok(()) } } @@ -1029,7 +1115,8 @@ fn msg_crdt(msg: &ProtocolMessage) -> CrdtType { | ProtocolMessage::DocUpdate { crdt, .. } | ProtocolMessage::DocUpdateFragmentHeader { crdt, .. } | ProtocolMessage::DocUpdateFragment { crdt, .. } - | ProtocolMessage::UpdateError { crdt, .. } + | ProtocolMessage::RoomError { crdt, .. } + | ProtocolMessage::Ack { crdt, .. } | ProtocolMessage::Leave { crdt, .. } => *crdt, } } @@ -1042,7 +1129,8 @@ fn msg_room_id(msg: &ProtocolMessage) -> String { | ProtocolMessage::DocUpdate { room_id, .. } | ProtocolMessage::DocUpdateFragmentHeader { room_id, .. } | ProtocolMessage::DocUpdateFragment { room_id, .. } - | ProtocolMessage::UpdateError { room_id, .. } + | ProtocolMessage::RoomError { room_id, .. } + | ProtocolMessage::Ack { room_id, .. } | ProtocolMessage::Leave { room_id, .. } => room_id.clone(), } } @@ -1063,7 +1151,15 @@ pub trait CrdtDocAdaptor { async fn set_ctx(&mut self, ctx: CrdtAdaptorContext); async fn handle_join_ok(&mut self, permission: protocol::Permission, version: Vec); async fn apply_update(&mut self, updates: Vec>); - async fn handle_update_error(&mut self, _code: protocol::UpdateErrorCode, _message: &str) {} + async fn handle_ack(&mut self, _ref_id: protocol::BatchId, _status: UpdateStatusCode) {} + async fn handle_update_error( + &mut self, + _updates: Vec>, + _status: UpdateStatusCode, + _reason: Option, + ) { + } + async fn handle_room_error(&mut self, _code: RoomErrorCode, _message: &str) {} async fn handle_join_err(&mut self, _code: protocol::JoinErrorCode, _message: &str) {} async fn get_alternative_version(&mut self, _current: &[u8]) -> Option> { None @@ -1139,8 +1235,6 @@ impl CrdtDocAdaptor for LoroDocAdaptor { let _ = guard.import(&u); } } - - async fn handle_update_error(&mut self, _code: protocol::UpdateErrorCode, _message: &str) {} } impl Drop for LoroDocAdaptor { @@ -1322,8 +1416,6 @@ impl CrdtDocAdaptor for EloDocAdaptor { } } } - - async fn handle_update_error(&mut self, _code: protocol::UpdateErrorCode, _message: &str) {} } impl Drop for EloDocAdaptor { diff --git a/rust/loro-websocket-server/src/lib.rs b/rust/loro-websocket-server/src/lib.rs index dacd62b..a56aff7 100644 --- a/rust/loro-websocket-server/src/lib.rs +++ b/rust/loro-websocket-server/src/lib.rs @@ -43,7 +43,9 @@ use tokio_tungstenite::tungstenite::{self, Message}; use loro::awareness::EphemeralStore; use loro::{ExportMode, LoroDoc}; pub use loro_protocol as protocol; -use protocol::{try_decode, BytesReader, CrdtType, JoinErrorCode, Permission, ProtocolMessage}; +use protocol::{ + try_decode, CrdtType, JoinErrorCode, Permission, ProtocolMessage, UpdateStatusCode, +}; use tracing::{debug, error, info, warn}; // Limits to protect server memory from abusive fragment headers @@ -637,13 +639,20 @@ where } } - fn apply_updates(&mut self, room: &RoomKey, updates: &[Vec]) { - if let Some(state) = self.docs.get_mut(room) { - if let Err(e) = state.doc.apply_updates(updates) { - warn!(room=?room.room, %e, "apply_updates failed"); - } else if state.doc.should_persist() { - state.dirty = true; + fn apply_updates(&mut self, room: &RoomKey, updates: &[Vec]) -> Result<(), String> { + match self.docs.get_mut(room) { + Some(state) => { + if let Err(e) = state.doc.apply_updates(updates) { + warn!(room=?room.room, %e, "apply_updates failed"); + Err(e) + } else { + if state.doc.should_persist() { + state.dirty = true; + } + Ok(()) + } } + None => Err("room not found".into()), } } @@ -728,6 +737,29 @@ where } static NEXT_ID: AtomicU64 = AtomicU64::new(1); +static NEXT_BATCH_ID: AtomicU64 = AtomicU64::new(1); + +fn next_batch_id() -> protocol::BatchId { + protocol::BatchId(NEXT_BATCH_ID.fetch_add(1, Ordering::Relaxed).to_be_bytes()) +} + +fn send_ack( + tx: &Sender, + crdt: CrdtType, + room: &str, + ref_id: protocol::BatchId, + status: UpdateStatusCode, +) { + let ack = ProtocolMessage::Ack { + crdt, + room_id: room.to_string(), + ref_id, + status, + }; + if let Ok(bytes) = loro_protocol::encode(&ack) { + let _ = tx.send(Message::Binary(bytes.into())); + } +} struct HubRegistry { config: ServerConfig, @@ -1029,6 +1061,7 @@ where crdt, room_id: room.room.clone(), updates: vec![snap], + batch_id: next_batch_id(), }; if let Ok(bytes) = loro_protocol::encode(&du) { let _ = tx.send(Message::Binary(bytes.into())); @@ -1055,6 +1088,7 @@ where crdt, room_id: room.room.clone(), updates: vec![u], + batch_id: next_batch_id(), }; if let Ok(bytes) = loro_protocol::encode(&du) { let _ = tx.send(Message::Binary(bytes.into())); @@ -1078,17 +1112,13 @@ where room: room_id.clone(), }; if !joined_rooms.contains(&room) { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::PermissionDenied, - message: "Must join room before sending updates".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::PermissionDenied, + ); continue; } // Permission check @@ -1099,17 +1129,13 @@ where .get(&(conn_id, room.clone())) .copied(); if !matches!(perm, Some(Permission::Write)) { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::PermissionDenied, - message: "Write permission required to update document".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::PermissionDenied, + ); continue; } // Bounds checks @@ -1117,17 +1143,13 @@ where || fragment_count > MAX_FRAGMENTS || total_size_bytes > MAX_BATCH_BYTES { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::PayloadTooLarge, - message: "Fragmented batch exceeds server limits".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::PayloadTooLarge, + ); continue; } // Initialize batch (guard against hijack by another sender) @@ -1135,17 +1157,13 @@ where let key = (room.clone(), batch_id); if let Some(existing) = h.fragments.get(&key) { if existing.from_conn != conn_id { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::InvalidUpdate, - message: "Batch ID already in use by another sender".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::InvalidUpdate, + ); continue; } // else: duplicate header from same sender -> accept and broadcast as-is @@ -1173,17 +1191,13 @@ where room: room_id.clone(), }; if !joined_rooms.contains(&room) { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::PermissionDenied, - message: "Must join room before sending updates".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::PermissionDenied, + ); continue; } // Validate batch existence and sender binding; also index bounds @@ -1191,17 +1205,13 @@ where let key = (room.clone(), batch_id); if let Some(b) = h.fragments.get(&key) { if b.from_conn != conn_id { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::InvalidUpdate, - message: "Fragment from wrong sender for batch".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::InvalidUpdate, + ); // do not broadcast continue; } @@ -1210,31 +1220,23 @@ where .map(|i| i < b.chunks.len()) .unwrap_or(false) { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::InvalidUpdate, - message: "Fragment index out of range".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::InvalidUpdate, + ); continue; } } else { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::InvalidUpdate, - message: "Unknown fragment batch".into(), - batch_id: Some(batch_id), - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::FragmentTimeout, + ); continue; } // Broadcast this fragment as-is to others (only after validation) @@ -1244,22 +1246,29 @@ where h.add_fragment_and_maybe_finish(&room, batch_id, index, fragment) { // On completion: parse and apply to stored doc state if applicable - match crdt { + let apply_result = match crdt { CrdtType::Loro | CrdtType::LoroEphemeralStore | CrdtType::LoroEphemeralStorePersisted => { - if let Ok(updates) = parse_docupdate_payload(&buf) { - let start = std::time::Instant::now(); - h.apply_updates(&room, &updates); - let elapsed_ms = start.elapsed().as_millis(); - debug!(room=?room.room, updates=%updates.len(), ms=%elapsed_ms, "applied reassembled updates"); + let start = std::time::Instant::now(); + let res = h.apply_updates(&room, &[buf.clone()]); + let elapsed_ms = start.elapsed().as_millis(); + if res.is_ok() { + debug!(room=?room.room, updates=1, ms=%elapsed_ms, "applied reassembled updates"); } + res } CrdtType::Elo => { // Apply as indexing-only - h.apply_updates(&room, &[buf]); + h.apply_updates(&room, &[buf.clone()]) } - _ => {} + _ => Ok(()), + }; + + if apply_result.is_ok() { + send_ack(&tx, crdt, &room.room, batch_id, UpdateStatusCode::Ok); + } else { + send_ack(&tx, crdt, &room.room, batch_id, UpdateStatusCode::InvalidUpdate); } } } @@ -1267,24 +1276,32 @@ where crdt, room_id, updates, + batch_id, } => { let room = RoomKey { crdt, room: room_id.clone(), }; + let oversized = + updates.iter().any(|u| u.len() > protocol::MAX_MESSAGE_SIZE); + if oversized { + send_ack( + &tx, + crdt, + &room.room, + batch_id, + UpdateStatusCode::PayloadTooLarge, + ); + continue; + } if !joined_rooms.contains(&room) { - // Not joined: reject with PermissionDenied - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::PermissionDenied, - message: "Must join room before sending updates".into(), - batch_id: None, - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::PermissionDenied, + ); warn!(room=?room.room, "update rejected: not joined"); } else { // Check permission @@ -1295,40 +1312,52 @@ where .get(&(conn_id, room.clone())) .copied(); if !matches!(perm, Some(Permission::Write)) { - let err = ProtocolMessage::UpdateError { + send_ack( + &tx, crdt, - room_id: room.room.clone(), - code: protocol::UpdateErrorCode::PermissionDenied, - message: "Write permission required to update document" - .into(), - batch_id: None, - app_code: None, - }; - if let Ok(bytes) = loro_protocol::encode(&err) { - let _ = tx.send(Message::Binary(bytes.into())); - } + &room.room, + batch_id, + UpdateStatusCode::PermissionDenied, + ); continue; } let mut h = hub.lock().await; - match crdt { + let apply_result = match crdt { CrdtType::Loro | CrdtType::LoroEphemeralStore | CrdtType::LoroEphemeralStorePersisted => { let start = std::time::Instant::now(); - h.apply_updates(&room, &updates); + let res = h.apply_updates(&room, &updates); let elapsed_ms = start.elapsed().as_millis(); - h.broadcast(&room, conn_id, Message::Binary(data)); - debug!(room=?room.room, updates=%updates.len(), ms=%elapsed_ms, "applied and broadcast updates"); + if res.is_ok() { + debug!(room=?room.room, updates=%updates.len(), ms=%elapsed_ms, "applied and broadcast updates"); + } + res } CrdtType::Elo => { // Index headers only; payload remains opaque to server. - h.apply_updates(&room, &updates); - h.broadcast(&room, conn_id, Message::Binary(data)); - debug!(room=?room.room, updates=%updates.len(), "indexed and broadcast ELO updates"); - } - _ => { - h.broadcast(&room, conn_id, Message::Binary(data)); + h.apply_updates(&room, &updates) } + _ => Ok(()), + }; + + if apply_result.is_ok() { + h.broadcast(&room, conn_id, Message::Binary(data)); + send_ack( + &tx, + crdt, + &room.room, + batch_id, + UpdateStatusCode::Ok, + ); + } else { + send_ack( + &tx, + crdt, + &room.room, + batch_id, + UpdateStatusCode::InvalidUpdate, + ); } } } @@ -1369,17 +1398,3 @@ where debug!(conn_id, "connection closed and cleaned up"); Ok(()) } - -fn parse_docupdate_payload(buf: &[u8]) -> Result>, String> { - let mut r = BytesReader::new(buf); - let n = usize::try_from(r.read_uleb128()?).map_err(|_| "length too large".to_string())?; - let mut out: Vec> = Vec::with_capacity(n); - for _ in 0..n { - let b = r.read_var_bytes()?.to_vec(); - out.push(b); - } - if r.remaining() != 0 { - return Err("trailing bytes".into()); - } - Ok(out) -} diff --git a/rust/loro-websocket-server/tests/elo_accept_broadcast.rs b/rust/loro-websocket-server/tests/elo_accept_broadcast.rs index cb936f6..7e2d3b8 100644 --- a/rust/loro-websocket-server/tests/elo_accept_broadcast.rs +++ b/rust/loro-websocket-server/tests/elo_accept_broadcast.rs @@ -1,6 +1,7 @@ use loro_websocket_client::Client; use loro_websocket_server as server; -use loro_websocket_server::protocol::{self as proto, CrdtType}; +use loro_websocket_server::protocol::{self as proto, BatchId, CrdtType}; +use loro_websocket_server::protocol::bytes::BytesWriter; use std::sync::Arc; #[tokio::test(flavor = "current_thread")] @@ -76,12 +77,13 @@ async fn elo_accepts_join_and_broadcasts_updates() { "both clients should receive JoinResponseOk for %ELO" ); - // Client 1 sends an opaque %ELO DocUpdate payload (container bytes are irrelevant to the server) - let opaque_update: Vec = vec![0xde, 0xad, 0xbe, 0xef]; + // Client 1 sends a minimal valid %ELO container (1 delta record) + let opaque_update: Vec = build_minimal_elo_container(); let du = proto::ProtocolMessage::DocUpdate { crdt: CrdtType::Elo, room_id: room_id.clone(), updates: vec![opaque_update.clone()], + batch_id: BatchId([3, 3, 3, 3, 3, 3, 3, 3]), }; c1.send(&du).await.unwrap(); @@ -92,6 +94,7 @@ async fn elo_accepts_join_and_broadcasts_updates() { crdt, room_id: rid, updates, + batch_id: _, }) = c2.next().await.unwrap() { if matches!(crdt, CrdtType::Elo) && rid == room_id && updates.len() == 1 { @@ -108,3 +111,22 @@ async fn elo_accepts_join_and_broadcasts_updates() { server_task.abort(); } + +fn build_minimal_elo_container() -> Vec { + // Record: kind=DeltaSpan, peerId=[1], start=1, end=2, keyId="k1", iv=12 zero bytes, ct=[0] + let mut rec = BytesWriter::new(); + rec.push_byte(0x00); + rec.push_var_bytes(&[1]); + rec.push_uleb128(1); + rec.push_uleb128(2); + rec.push_var_string("k1"); + rec.push_var_bytes(&[0u8; 12]); + rec.push_var_bytes(&[0]); + let rec_bytes = rec.finalize(); + + // Container with 1 record + let mut container = BytesWriter::new(); + container.push_uleb128(1); + container.push_var_bytes(&rec_bytes); + container.finalize() +} diff --git a/rust/loro-websocket-server/tests/readonly_receive.rs b/rust/loro-websocket-server/tests/readonly_receive.rs index 35e5f04..62106a2 100644 --- a/rust/loro-websocket-server/tests/readonly_receive.rs +++ b/rust/loro-websocket-server/tests/readonly_receive.rs @@ -1,7 +1,9 @@ use loro as loro_crdt; use loro_websocket_client::Client; use loro_websocket_server as server; -use loro_websocket_server::protocol::{self as proto, CrdtType, Permission}; +use loro_websocket_server::protocol::{ + self as proto, BatchId, CrdtType, Permission, UpdateStatusCode, +}; use std::sync::Arc; #[tokio::test(flavor = "current_thread")] @@ -77,6 +79,7 @@ async fn readonly_receives_updates_writer_sends() { crdt: CrdtType::Loro, room_id: room.clone(), updates: vec![snap], + batch_id: BatchId([1, 1, 1, 1, 1, 1, 1, 1]), }; writer.send(&upd).await.unwrap(); @@ -98,18 +101,18 @@ async fn readonly_receives_updates_writer_sends() { } assert!(got_update, "reader did not receive update"); - // Reader attempts to send an update -> expect UpdateError.PermissionDenied + // Reader attempts to send an update -> expect Ack.PermissionDenied let upd2 = proto::ProtocolMessage::DocUpdate { crdt: CrdtType::Loro, room_id: room.clone(), updates: vec![vec![1, 2, 3]], + batch_id: BatchId([9, 9, 9, 9, 9, 9, 9, 9]), }; reader.send(&upd2).await.unwrap(); let mut got_err = false; for _ in 0..3 { - if let Some(proto::ProtocolMessage::UpdateError { code, .. }) = reader.next().await.unwrap() - { - assert!(matches!(code, proto::UpdateErrorCode::PermissionDenied)); + if let Some(proto::ProtocolMessage::Ack { status, .. }) = reader.next().await.unwrap() { + assert!(matches!(status, UpdateStatusCode::PermissionDenied)); got_err = true; break; } diff --git a/rust/loro-websocket-server/tests/reject_update_without_join.rs b/rust/loro-websocket-server/tests/reject_update_without_join.rs index 7c804be..852322f 100644 --- a/rust/loro-websocket-server/tests/reject_update_without_join.rs +++ b/rust/loro-websocket-server/tests/reject_update_without_join.rs @@ -1,6 +1,6 @@ use loro_websocket_client::Client; use loro_websocket_server as server; -use loro_websocket_server::protocol::{self as proto, CrdtType}; +use loro_websocket_server::protocol::{self as proto, BatchId, CrdtType, UpdateStatusCode}; use std::sync::Arc; #[tokio::test(flavor = "current_thread")] @@ -25,19 +25,20 @@ async fn reject_update_without_join() { crdt: CrdtType::Loro, room_id: "room-no-join".to_string(), updates: vec![vec![1, 2, 3]], + batch_id: BatchId([2, 2, 2, 2, 2, 2, 2, 2]), }; c.send(&msg).await.unwrap(); - // Expect UpdateError.PermissionDenied + // Expect Ack.PermissionDenied let mut got_err = false; for _ in 0..3 { - if let Some(proto::ProtocolMessage::UpdateError { code, .. }) = c.next().await.unwrap() { - assert!(matches!(code, proto::UpdateErrorCode::PermissionDenied)); + if let Some(proto::ProtocolMessage::Ack { status, .. }) = c.next().await.unwrap() { + assert!(matches!(status, UpdateStatusCode::PermissionDenied)); got_err = true; break; } } - assert!(got_err, "did not receive UpdateError.PermissionDenied"); + assert!(got_err, "did not receive Ack.PermissionDenied"); server_task.abort(); }