Custom Backend

Implement the PersistenceLayer interface to add support for a new database.

What you're implementing

The persistence layer handles three storage concerns:

Concern	Methods	Purpose
Change log	log, getLog, getChangeLog, getChangeLogHashes	Raw CBOR for P2P sync
Document store	save, update, findOne, versions, editCount	Internal state for computing edits
Rendered documents	renderInsert, renderUpdate, renderDelete	Latest state for app reads
Trash	saveDeleted, listDeleted, removeDeleted	Soft-deleted documents

Your app reads directly from the rendered documents. The other tables are internal to document-store.

The interface

interface PersistenceLayer {
    // --- Change log (sync) ---

    /** Store CBOR records for sync */
    log(items: { hash: Buffer, cbor: Buffer, signed?: any[] }[]): Promise<void>;

    /** Get a single change record by hash */
    getLog(hash: Buffer): Promise<(Document & { cbor: Buffer }) | null>;

    /** Get all changes since timestamp, sorted by time ascending */
    getChangeLog(since: number): Promise<ChangeRecord[]>;

    /** Get change hashes, optionally since timestamp */
    getChangeLogHashes(since?: number): Promise<Buffer[]>;

    // --- Document store (internal) ---

    /** Store documents (initial add) */
    save(docs: DocumentStoreItem[]): Promise<void>;

    /** Update a document (after edit) */
    update(hash: Buffer, doc: DocumentStoreItem): Promise<void>;

    /** Find document by hash */
    findOne(hash: Buffer): Promise<DocumentStoreItem | null>;

    /** Get version history for a document */
    versions(hash: Buffer): Promise<{ original: any, edits: any[] } | null>;

    /** Count edits for a document */
    editCount(hash: Buffer): Promise<number>;

    /** Full-text search — return matching hashes */
    search(query: string, section?: string): Promise<Buffer[]>;

    /** Delete from document store (preserves change log) */
    deleteFromDocDb(hashes: Buffer[]): Promise<void>;

    /** Delete from change log by root hashes */
    deleteFromChangeDb(rootHashes: Buffer[]): Promise<void>;

    // --- Rendered documents (app reads) ---

    /** Insert rendered document */
    renderInsert(type: string, doc: any): Promise<void>;

    /** Update rendered document */
    renderUpdate(type: string, match: { hash: Buffer }, doc: any): Promise<void>;

    /** Delete rendered documents */
    renderDelete(type: string, hashes: Buffer[]): Promise<void>;

    // --- Trash ---

    /** Store deleted document for restore */
    saveDeleted(item: {
        hash: Buffer;
        type: string;
        deletedAt: number;
        deletedBy: Buffer;
        content: Record<string, unknown>;
    }): Promise<void>;

    /** List deleted documents */
    listDeleted(options?: {
        type?: string;
        limit?: number;
        offset?: number;
        since?: number;
        before?: number;
        deletedBy?: Buffer;
        deletedAt?: number;
    }): Promise<Array<{
        hash: Buffer;
        type: string;
        deletedAt: number;
        deletedBy: Buffer;
        content: Record<string, unknown>;
    }>>;

    /** Remove from trash */
    removeDeleted(hash: Buffer): Promise<void>;
}

Implementation guide

Change log

The change log stores raw CBOR. Each record has:

interface ChangeRecord {
    hash: Buffer;     // SHA-256 of CBOR data
    prev?: Buffer;    // Previous version (for edits)
    root?: Buffer;    // Original document (for edits)
    _type?: string;   // Document type
    time: number;     // Timestamp
    cbor: Buffer;     // Raw CBOR data
    signed?: any[];   // Signatures (stored separately)
}

Index on time (for getChangeLog), prev, and root.

Document store

DocumentStoreItem wraps a rendered document with search keywords:

interface DocumentStoreItem {
    hash?: string;           // Hex string
    doc: Document;           // Full document
    keywords: KeywordMap;    // Search index data
}

Index on hash. The doc object contains all fields including system fields.

Rendered documents

This is where your app reads. Each type gets its own table/collection. The store calls:

• renderInsert(type, doc) — new document
• renderUpdate(type, { hash }, doc) — document edited
• renderDelete(type, hashes) — documents deleted

The doc object is the final rendered state — all edits applied, system fields included based on registerType({ render }) options.

This is the simplest part. It's just insert, update by hash, and delete by hash.

Trash

Soft-deleted documents are stored separately for restore. Simple CRUD on a separate table, keyed by hash.

Example skeleton

import { PersistenceLayer, DocumentStoreItem, ChangeRecord } from 'document-store';

class MyDatabasePersistence implements PersistenceLayer {
    constructor(private db: MyDatabase) {}

    async log(items) {
        for (const item of items) {
            // Decode CBOR to extract indexed fields
            const doc = decodeCbor(item.cbor);
            await this.db.insert('changes', {
                hash: item.hash,
                prev: doc.prev,
                root: doc.root,
                _type: doc._type,
                time: doc.time,
                cbor: item.cbor,
                signed: item.signed ? JSON.stringify(item.signed) : null,
            });
        }
    }

    async getLog(hash) {
        const row = await this.db.findByKey('changes', hash);
        if (!row) return null;
        const doc = decodeCbor(row.cbor);
        doc.hash = row.hash;
        doc.cbor = row.cbor;
        if (row.signed) doc.signed = JSON.parse(row.signed);
        return doc;
    }

    async getChangeLog(since) {
        return this.db.query('changes', { time: { $gt: since } }, { sort: { time: 1 } });
    }

    async renderInsert(type, doc) {
        await this.db.insert(type, { hash: doc.hash, ...doc });
    }

    async renderUpdate(type, match, doc) {
        await this.db.update(type, { hash: match.hash }, doc);
    }

    async renderDelete(type, hashes) {
        await this.db.deleteMany(type, { hash: { $in: hashes } });
    }

    // ... implement remaining methods
}

Testing your backend

The document-store test suite can run against any backend. Set the TEST_DB environment variable:

# Run tests against your backend
TEST_DB=mydb npm test

Add your backend to the test setup in test/setup/mocha-setup-and-teardown.ts.