Hexes, tessellations, lattices
A Lattice That Elects Its Own Leader and Heals Itself.
Start one hex and you have a database. Start more with the same lattice name and secret, on this machine or on hundreds of others, and they discover each other, elect an Overseer, replicate, and fail over on their own. No coordinator, no sidecar, no operator in the loop.
Listen to the podcast - 5:24 min
Resilience: Recovery, Replication and Multi-Cloud
What happens when a hex crashes, loses a vertex or disappears altogether: the write-ahead log, Reed-Solomon repair, replication between hexes, elections and failover, and the latency trade-offs of a lattice spread across clouds.
Four hexes, one Overseer. Replicas follow the change feed and serve reads.
Dashed hexes are ones you could add: on any cloud, in any region or on your own hardware, as many as you like.
Every running hex is also an MCP server: Claude and other AI agents can connect to any of them (with an Oracle, it does the answering). AI and MCP
An Oracle (optional, in the next release) answers the lattice's AI questions, from MCP, the chat and A2A, and keeps no data of its own: it reads what each question needs from a replica, as the user who asked, so AI work stays off the hexes that hold your data. The Oracle
HexDB is one self-contained binary for Windows, macOS and Linux on x86_64 or arm64, or a Docker image, with no cloud service behind it. A cloud VM, a server in your own building and a small arm64 box on a shelf can all be hexes in the same lattice, as long as they can reach each other.
Try it for real: node scripts/lattice-demo.mjs, then type stop 1.
The Overseer keeps every slice; the 3 replicas share them, each slice on one replica. Lose any one hex and every document still exists.
Each square is one of the lattice's 64 slices; a filled one is kept on that hex. Placement is computed the same way on every hex (rendezvous hashing), so a hex that joins or leaves moves only its own slices.
Four Roles
- Overseer. Takes writes, publishes the change feed, runs plugins, stream sources and schedules.
- Harvester. A read replica that follows the Overseer and can be elected if it fails.
- Replicant. A standby copy that follows but is never elected. For backups and rebuilding Harvesters.
- Oracle. Optional: answers the lattice's AI questions and keeps no data, so AI work stays off the hexes that hold it.
The Vocabulary
- Hex
- One running HexDB server. Whatever its role, every hex is also an MCP server: an AI agent can connect to any of them and read the lattice with its API key's permissions.
- Tessellation
- A named collection, like a table. A user tessellation holds JSON documents; a files tessellation holds binary files, each with a metadata document you query like any other. Roles are granted per tessellation.
- Lattice
- A group of hexes sharing a name and a secret that can reach each other. A single hex is a lattice of one.
- Slice
- One of 64 parts of a lattice's documents. With slicing (the default), the Overseer keeps every slice and each replica keeps a share, so replicas need less disk and memory.
- Vertex
- One of six separate regions of a hex's memory, each at its own addresses. Four data vertices each hold a quarter of every document; two parity vertices hold Reed-Solomon check data. A corrupt page in RAM only touches the one vertex that lives there: the document is rebuilt from the other five, the bad fragment is rewritten, and a whole vertex can be rebuilt and reassigned to fresh memory.
As Many Hexes as You Have Machines
The local demo runs three and the playground four, but nothing in HexDB caps the count. Put a hex on every server you own, in one rack or across regions: each needs only the lattice name, the secret and a few peers to start from, and joins as a replica with a full copy, or with its share of the slices. More hexes means more machines you can lose, and with slicing off, more read capacity. Writes still go through the one Overseer, so add hexes for reads and resilience rather than write throughput.
More Hexes, Less to Keep on Each
Slicing is on by default: the replicas don't keep full copies. The data falls into 64 slices: the Overseer keeps them all, and each slice is also kept by one replica (or more, if you ask), so every document survives the loss of any one hex. Add a replica and the others hand it a share; lose the Overseer and the most up-to-date replica collects the rest from its peers before it takes over. With only two hexes, the replica keeps everything. Replicas holding slices pass queries to the Overseer, which still keeps every slice, so slices save space on the replicas, not on the Overseer.
[replication]
slicing = true # replicas share 64 slices instead of keeping full copies
slice_copies = 1 # replicas per slice (the Overseer keeps every slice)Built In, Not Bolted On
AI, with your own key
Tags and describes uploads, picks video thumbnails and answers questions about your data, with your provider and your budget.
ClaudeOpenAIOllamaMalware scanning
ClamAV and YARA scan every upload before it's stored; a match is quarantined and never replicates. Free, and nothing leaves your servers.
ClamAVYARA-XQuarantineFile import
Like SSIS for files: pull buckets and folders in on a schedule, folders kept. New files and versions arrive; deletes at the source don't.
S3Azure BlobGoogle DriveDropboxHexDB files
Durability Is a Dial
Replication is asynchronous by default. Two settings let you require that a write exists on replicas before it's acknowledged, and that an Overseer refuses writes when it can't see a majority.
# hexdb.toml: trade a little latency for a lot of safety
[replication]
min_acks = 1 # replicas that must hold a write before it's acknowledged
ack_timeout_ms = 5000 # on timeout the write stays committed; the client gets 503
quorum = 2 # hexes the Overseer must see to accept writes (no split brain)
forward_writes = true # a write sent to a replica is forwarded to the OverseerWhatever the dial says, a write is in the Overseer's write-ahead log, synced to disk, before it's acknowledged. A replica that falls behind or restarts catches up from the change history (24 hours of it by default) instead of copying everything again.
node scripts/lattice-demo.mjs # three hexes on 7800/7810/7820 with sample data
> stop 1 # stop the Overseer and watch another hex take over
> start 1 # it comes back as a replicaAnatomy of a hex
What One Hex Does With Your Data.
A hex is one running HexDB server. Step through with the arrows, or press play and it advances every few seconds, to follow a document from the moment a request arrives, through the write-ahead log, into six error-correcting vertices, out to the change feed, and down to disk.
A Request Arrives
REST, GraphQL, the admin UI and the drivers all come through the same door.
- One process serves the HTTP API, GraphQL and the admin UI, on one port.
- Every request needs credentials: a session cookie or an hxk_ API key. There is no unauthenticated mode, and unknown paths get 401 too.
- A write sent to a replica is forwarded to the Overseer, signed with the lattice key, and the response says where it went.
Step 1 of 7. Use the arrows to move at your own pace.
The Parts, in One Glance
- The server
One Rust binary,
hexdb_api, serving REST, GraphQL and the admin UI on a single port. ThehexdbCLI starts, stops, inspects and backs it up. - Tessellations
Named collections, like tables: of JSON documents, or of files, each file with a metadata document you query like any other. Each can have indexes, a text index with its own analyzer, and a versioned schema with parent links. Roles are granted per tessellation.
- Querying
One filter language for equality, ranges, sets, prefixes and nested paths, with sorting, paging, counts and aggregations. Full-text search with stemming, n-grams and autocomplete. Spoken three ways: REST, GraphQL and read-only SQL. Projections pick fields in GraphQL (field paths) and SQL (select columns).
- Percolation
Queries that wait for documents instead of the other way round. A stream source, a trigger or a plugin registers a filter, and every write that matches it is delivered as it commits: to a webhook, a script, Kafka, or a consumer group.
- Memory
The newest unflushed documents plus an LRU cache of recent reads, within
memory.ram_mb. Every document in memory is spread over six vertices, each a separate region of RAM at its own addresses: four data, two parity. A corrupt page damages one fragment, not the document. - Disk
A write-ahead log for durability, immutable SSTables for bulk storage, encrypted index snapshots, a sealed catalog, and a change-history archive. All AES-256-GCM.
- The catalog
Tessellations, index definitions, schemas, the change-history ID and the lattice name, in one sealed file. Replicas copy it from the Overseer and re-read it within a poll when its fingerprint changes.
- Background work
TTL sweeps, vertex integrity checks, flushes, compaction, metrics every 15 seconds, discovery probes, and, on the Overseer, plugins, stream deliveries, after triggers and schedules.
Storage engine
Built so You Don't Lose Data. Or Sleep.
A log-structured engine written from scratch in Rust: a write-ahead log with group commit, SSTables with compaction, encryption on every byte that touches disk, and error-correcting memory that repairs itself.
The Write Path
- Validate
Permissions, size limits, the schema, and expected versions for read-modify-write operations.
- Sequence and Apply
Under one state lock the batch gets consecutive sequence numbers, is queued to the WAL and applied to memory and indexes. A query always sees its own writes.
- Group Commit
A dedicated thread batches whatever is waiting into one write and one fsync. Zstandard first, then AES-256-GCM.
- Acknowledge and Publish
Once durable, the write is acknowledged, published to the change feed, and, with
min_acks, held until enough replicas confirm.
On Disk and in Memory
- Memory
- Newest unflushed documents plus an LRU cache, each spread over four data vertices and two parity vertices with Reed-Solomon coding, about 1.5 times the JSON size. Each vertex is a separate region of RAM, so a corrupt page costs one fragment, which is rebuilt from the others. On disk, authenticated encryption means a bad sector is detected rather than served, and other hexes hold copies. See the hex above.
- SSTables
- Immutable, per tessellation, with Bloom filters and a chunked encrypted index. Documents are stored as plain JSON, compressed with Zstandard, then encrypted: large structured documents shrink to about 16% of their size. About 2 bytes of RAM per document.
- Compaction
- Merges SSTables, drops tombstones and expired documents, and rewrites files on old formats or old keys.
- Recovery
- Replays the WAL in order. A torn final record is tolerated; anything that can't be decrypted stops startup rather than discarding data.
Encrypted at Rest, Rotated Without Downtime
WAL records, SSTable bodies and indexes, the catalog, settings, metrics and index snapshots are all AES-256-GCM with random 96-bit nonces, authenticated with their IDs so files and entries can't be swapped. Only SSTable headers are plaintext. A server whose data needs a key it doesn't have refuses to start rather than discard anything.
# Every developer and every deployment has its own key. Keep it.
printf '[storage]\nencryption_key = "%s"\n' "$(hexdb secret)" > hexdb_api/hexdb.local.toml
# Rotate without downtime: add the new key, move the old one to previous_encryption_keys,
# restart, and let compaction rewrite the files. Then drop the old key.
[storage]
encryption_key = "base64:NEW..."
previous_encryption_keys = ["base64:OLD..."]What's in the Data Directory
Everything a hex knows lives in one folder. Back it up online with one command, and restore by pointing a hex at the copy.
<storage.path>/
├── catalog.hxe tessellations, indexes, schemas, history ID, lattice name (sealed)
├── settings.hxe runtime settings changed from the UI or API (sealed)
├── metrics-history.hxe 6 hours of metrics samples (sealed)
├── query-stats.hxe the query advisor's statistics (sealed)
├── hexdb.pid PID, endpoint and shutdown token (owner-only)
├── wal/
│ ├── <first-seq>.wal active write-ahead log segments
│ └── archive/<first-seq>.wal the change history
├── indexes/<tessellation>/<index>.hxi index snapshots (sealed)
├── blobs/vertex-<n>/ file content: one pack series per vertex
│ ├── <number>.hxp a pack
│ └── remote.json which packs are on the storage backend
└── <tessellation>/<ulid>.hxs SSTables, including system tessellationsPerformance
Measured on a Laptop, With Durability On.
Numbers from the benchmark that ships in the repository, run on an Intel Core i9-12900H laptop with a Samsung 980 PRO NVMe drive under Windows 11, server and client on the same machine. Every write is in the write-ahead log and fsynced before the client hears back. Run it on your own hardware and compare.
- 2.3msSingle write, median
One client, one document per request. 1,940 writes per second from 8 clients, thanks to group commit.
- 0.12msRead by ID, median
17,600 reads per second from 8 concurrent clients.
- 0.65msIndexed lookup, median
An equality query over 100,000 documents using a field index. SQL
SELECT COUNT(*)answers in 0.24 ms from the maintained count. - 20MBTo download
The Windows installer is 18 MB. Installed it is 65 MB: a 50 MB server with storage engine, REST, GraphQL, SQL and replication; 9 MB CLI; 3 MB ODBC driver; 3 MB admin UI. 58 MB on Linux.
- 1.5xJSON sizeMemory per document
Four data fragments plus two parity fragments across the six vertices, each with its hash.
- 16%of JSON on diskLarge structured documents
A 10 MB report compresses to 1.6 MB before encryption. Small documents average 81% after encryption and per-document overhead.
- 1,100MB/sAES-256-GCM on one core
Encrypting at 1,100 MB/s and decrypting at 1,500 MB/s, so encryption at rest is never the bottleneck. On Linux, 1,930 and 2,170 MB/s.
- 0.36sRebuild a lost vertex
Repairing the fragments of 100,000 documents after a vertex is lost. Documents stay readable throughout, even with two of six vertices gone.
- 2.5sFailover to a new Overseer
With discovery every second. With the default 5-second interval it is 15 to 20 s, because a hex is declared lost after three missed rounds.
- 550msWrite a 10 MB document
Structured JSON; reading it back takes 163 ms. Incompressible data writes in 84 ms and reads in 24 ms.
Not everything is fast. Aggregations group in memory after the filter, so a GROUP BY with SUM and AVG over 100,000 orders takes about 1.1 seconds. Documents over 1 MB need limits.max_document_kb raised, up to 64 MB.
The full report lists every measurement with how it was taken, including compression rates per document shape, Reed-Solomon encode and repair throughput, crash recovery with and without unflushed writes, and failover timing. The benchmark is part of the repository, so the same numbers can be produced on any machine:
cargo run --release -p hexdb_benchRecommended Hardware
Starting points for each hex, from how HexDB uses memory, CPU, disk and network, and from what we learned running the playground. They aren't cloud benchmarks: run hexdb_bench on a candidate instance before you commit, and watch /status as real traffic arrives.
- Memory: Budget a Quarter of RAM
memory.ram_mbcounts JSON bytes, but a document held in memory costs about 3.7 times that once it's parsed and spread over six vertices. Setram_mbto roughly a quarter of the machine's RAM; whatever doesn't fit is read from the SSTables through the cache. On small Linux hosts,MALLOC_ARENA_MAX=2keeps the allocator from growing past it. - CPU: Dedicated, Never Burstable
Burstable instances (AWS T, Azure B, GCP shared-core) are fine for one hex on a laptop budget, not for a lattice. When the credits run out, heartbeats miss, replicas think the Overseer is gone and elect another. Two vCPUs to start, four for production; aggregations and index builds use what you give them.
- Disk: Low-Latency SSD, Room to Compact
Every write is fsynced before it's acknowledged, so disk latency is write latency. Use SSD with at least 3,000 IOPS. Size each disk for all the data, since any hex elected Overseer collects every slice after compression, the WAL and 24 hours of change history, with free space equal to your largest tessellation for compaction. Files take about 1.5 times their compressed content on disk, plus about 80 MB of RAM per TB for the chunk index.
- Network: Keep the Write Path Close
The Overseer streams every write to every hex, so replication traffic is your write volume times the number of replicas. Hexes in other regions or clouds are fine for reads; with
replication.min_acksset, each acknowledged write waits for a round trip, so keep at least one replica near the Overseer. Open 7700 (API) and 7702 (discovery) between hexes. - Overseer: Give It the Biggest Machine
Without slicing, elections rank hexes by RAM, then disk, so the largest hex leads, and it does the most work: it takes every write and feeds every replica. Size it first, or pin the role with
identity.role. Lowernetwork.discovery_interval_secondsfor faster failover: 1 second takes about 2.6 s, the default 5 about 15 to 20 s.
- StartDevelopment, staging, a small app2 vCPU · 8 GB · 50 GB SSDram_mb = 2048
- AWS
- m7i.large x86 · m7g.large arm64gp3
- Azure
- D2s v5 x86 · D2ps v5 arm64Premium SSD v2
- GCP
- n2-standard-2 x86 · t2a-standard-2 arm64Hyperdisk Balanced
- ProductionEach hex of a production lattice4 vCPU · 16 GB · 200 GB SSDram_mb = 4096
- AWS
- m7i.xlarge x86 · m7g.xlarge arm64gp3, 3,000+ IOPS
- Azure
- D4s v5 x86 · D4ps v5 arm64Premium SSD v2
- GCP
- n2-standard-4 x86 · t2a-standard-4 arm64Hyperdisk Balanced
- LargeBig working sets, heavy aggregation, the Overseer8 vCPU · 64 GB · 500 GB+ SSDram_mb = 16384
- AWS
- r7i.2xlarge x86 · r7g.2xlarge arm64gp3, 6,000+ IOPS
- Azure
- E8s v5 x86 · E8ps v5 arm64Premium SSD v2
- GCP
- n2-highmem-8 x86 · c4a-highmem-8 arm64Hyperdisk Balanced
| Per hex | AWS | Azure | GCP |
|---|---|---|---|
Start Development, staging, a small app 2 vCPU · 8 GB · 50 GB SSD ram_mb = 2048 | m7i.large x86 m7g.large arm64 gp3 | D2s v5 x86 D2ps v5 arm64 Premium SSD v2 | n2-standard-2 x86 t2a-standard-2 arm64 Hyperdisk Balanced |
Production Each hex of a production lattice 4 vCPU · 16 GB · 200 GB SSD ram_mb = 4096 | m7i.xlarge x86 m7g.xlarge arm64 gp3, 3,000+ IOPS | D4s v5 x86 D4ps v5 arm64 Premium SSD v2 | n2-standard-4 x86 t2a-standard-4 arm64 Hyperdisk Balanced |
Large Big working sets, heavy aggregation, the Overseer 8 vCPU · 64 GB · 500 GB+ SSD ram_mb = 16384 | r7i.2xlarge x86 r7g.2xlarge arm64 gp3, 6,000+ IOPS | E8s v5 x86 E8ps v5 arm64 Premium SSD v2 | n2-highmem-8 x86 c4a-highmem-8 arm64 Hyperdisk Balanced |
The arm64 machines (AWS Graviton, Azure Ampere, Google Axion and Ampere) run the Linux arm64 build and usually cost less for the same shape. Any current general-purpose or memory-optimized family with the same shape works just as well.
