Reference

The shared database: its data model, its files, and every operation it performs.

The system in one page

A database holds a tree of nodes. A node holds a value, or holds child nodes. Child names come from the schema for a struct or a union, and from the entry itself for a collection. A node's identifier is the hash of its parent's identifier and its name, so every machine works out the same identifier for the same path, and no machine can craft an identifier that collides with another path.

Each machine owns one directory and appends 64-byte records to daily log files inside it. Values longer than 16 bytes go into data files beside the log. Nothing else writes those files, so a syncing folder never produces a conflicted copy.

A reader merges the records from every machine. For each node it keeps the record with the highest version, and compares hashes when two versions are equal. Reading a node means finding that winner. Writing means appending one record with a version one above the highest known.

Once a day ends, the machine seals it: it writes a day index holding a hash of each log file and a set of sorted lookup tables, then updates a small tree of month, year and root index files. A reader that holds a verified index knows exactly what that machine published, which is what makes safe cleanup possible.

Deleting an entry writes a tombstone, which hides the entry and everything under it straight away. The machine that created the entry watches for that tombstone and drops the entry's container record at its next compaction. The entry is then gone for good, and every other machine can drop the records it held under it.

Rules that hold everywhere

Rule	Reason
A machine writes only its own directory.	A syncing folder never has to merge a file, so it never makes a conflicted copy.
The current day is append-only.	A reader always holds a prefix of it, and a partly synced file is still usable.
Absence only hides a node. Presence removes it.	A missing record may be a deletion or a file in transit. Acting on it would delete live data.
Deletion applies to collection entries only.	Struct fields and union options keep fixed names, so a permanent tombstone would block that path for good.
Only the creator writes an entry's container record.	One machine decides when the entry stops existing, which makes the deletion protocol exact.
A deleted entry never comes back.	Every cleanup rule depends on death being permanent.

Terms

Term	Meaning
Machine	One server process writing one directory. A person with two computers runs two machines.
Record	64 bytes describing one version of one node.
Entry	A member of a collection. The only kind of node that can be deleted.
Container record	The record that says an entry exists, written once by its creator.
Tombstone	A record with the highest possible version, marking an entry as deleted.
Day index	The file written when a day is sealed, holding hashes and lookup tables for that day.
Gone	An entry whose container record is absent from its creator's verified day index. Cleanup starts here.
Witness	A machine holding records under an entry it did not create.

Reference

The system in one page

Pages

Data model

Records

Storage layout

Indexes

Reading

Writing

Deleting

Compaction

Syncing

Rules that hold everywhere

Terms