P2P Sync

Document Store syncs between peers using a pull-based protocol. Your app declares which document trees to sync; the system handles discovery, incremental transfer, and progress tracking.

Setup

import { RpcApp, SyncProtocol } from '@wishcore/wish-sdk';
import { DocumentStore, SurrealdbPersistence, SurrealSyncProgress } from 'document-store';

const store = new DocumentStore({
    storage: new SurrealdbPersistence('surrealkv://./data')
});
await store.storage.ready();

store.registerType('note');

const sync = new SyncProtocol({
    store,

    // Your app decides what to sync — return root hashes per identity
    getRoots: async (uid) => {
        return myApp.getRootsForUser(uid);
    },

    // Optional: persist sync progress across restarts
    syncProgress: new SurrealSyncProgress(store.storage.db),

    // Called after documents arrive from a peer
    onSync: ({ injected }) => {
        if (injected) console.log('Got new documents');
    }
});

const app = new RpcApp({
    name: 'MyApp',
    protocols: { myapp: sync }
});

How sync works

Sync is pull-based — the receiver decides what to fetch.

The pull flow

When a peer connects, the local side pulls from them:

1. Discover    — fetch documents directly shared with me (invites, grants)
2. Get roots   — ask the app what document trees to sync (getRoots callback)
3. Pull trees  — for each root, fetch changes since last sync
4. Inject      — verify and store each document locally

Both sides pull independently — A pulls from B, B pulls from A.

Incremental sync

The protocol tracks progress per peer per root. After the initial sync, only new changes are transferred. If a peer disconnects and reconnects, sync resumes from where it left off.

Progress is tracked using receivedAt timestamps — when documents were received locally, not when they were created. This ensures children added after a parent are never missed.

The sync cursor only advances for successfully injected items. If a document fails validation or signature verification, the cursor stays put — the document will be retried on the next sync cycle.

On local change

When you add or edit a document, notify peers so they pull the change:

sync.notifyPeers();

This emits a document.changed signal. Connected peers then run the pull flow to fetch what's new.

App owns the roots

Your app decides what to sync by returning root hashes from the getRoots callback. This is the key design decision — the sync engine doesn't scan all documents or maintain a global sync list.

Common patterns:

getRoots: async (uid) => {
    // Sync all discussions the user is in
    const discussions = await db.query(
        'SELECT hash FROM discussion WHERE uid = $uid',
        { uid }
    );
    return discussions.map(d => d.hash);
}

getRoots: async (uid) => {
    // Read from a pullRoots document (syncs between own devices)
    const pullRoots = await getPullRootsForUser(uid);
    return pullRoots?.roots || [];
}

When roots change (user joins a discussion, accepts an invite), call triggerPull to sync immediately:

sync.controller.triggerPull(uid);

Notifying a frontend

If your app has a WebSocket frontend, combine peer and frontend notifications:

import { createWebRpc } from '@wishcore/wish-sdk';

const webRpc = createWebRpc({ port: 8090 });

const sync = new SyncProtocol({
    store,
    getRoots: async (uid) => getMyRoots(uid),
    onSync: ({ injected }) => {
        if (injected) webRpc.emit('signals', ['document.changed']);
    }
});

const notifyChange = () => {
    webRpc.emit('signals', ['document.changed']);
    sync.notifyPeers();
};

// Use after every write
const [errors, hash] = await store.add('note', { uid, title: 'Hello' });
if (!errors) notifyChange();

What syncs

Everything in the change log:

• Document creates (the original CBOR + signatures)
• Document edits (each edit as a separate CBOR record)
• Deletes (a $delete edit)

The rendered view is not synced — it's rebuilt locally from the change log. This means peers with different storage backends still sync correctly.

Access control

The server side checks share policies before sending documents:

• share: { self: true } — syncs only to the owner's other devices
• share: { users: { ... } } — syncs only to listed users (used by invites)
• share: { ref: parentHash } — inherits access from parent (membership tokens)
• share: { public: ... } — syncs to everyone

A custom shareResolver callback can extend these policies — for example, sharing identity documents with all contacts.

See Share Policies for configuration.

Sync progress

By default, sync progress is in-memory (lost on restart → full re-sync). For persistent progress:

import { SurrealSyncProgress } from 'document-store';

const sync = new SyncProtocol({
    store,
    getRoots: async (uid) => getMyRoots(uid),
    syncProgress: new SurrealSyncProgress(store.storage.db),
});

Available implementations:

Implementation	Backend	Use case
MemorySyncProgress	In-memory	Testing, short-lived processes
SurrealSyncProgress	SurrealDB	Desktop apps using SurrealDB
SqliteSyncProgress	SQLite	Desktop apps using SQLite
MongoSyncProgress	MongoDB	Server deployments

All persistent implementations use a write-back cache — progress is batched to avoid per-document database writes.

Resetting progress

Force a full re-sync when needed:

// Re-pull a specific tree from all peers
await sync.controller.resetForRoot(rootHash);

// Re-pull everything
await sync.controller.resetAll();

Conflict resolution

Document Store uses operation-based sync, not state-based. Each edit is a discrete operation (like $set: { title: 'New' }) applied in timestamp order.

When two peers edit the same document offline:

Peer A: $set: { title: 'A title' }   at T=100
Peer B: $set: { title: 'B title' }   at T=101

After sync, both peers apply edits in order: A's edit first, then B's. Result: title = 'B title' on both sides. Last write wins per field.

For array operations, edits are also applied in order. See Editing for details on operator semantics.

Protocol endpoints

The sync protocol registers three RPC endpoints that peers call:

Endpoint	Purpose
document.discover	Returns documents directly shared with the requesting peer (via share.users)
document.pull	Returns tree changes for a root hash since a timestamp
document.find	Returns a specific document by type + uid (point lookup)

Plus one signal:

Signal	Purpose
document.changed	Notifies peers that new data is available — triggers them to pull