Open source · Written in RustThe document database that ships as one binary.
JSON documents over REST, GraphQL and read-only SQL. Full-text search, ACID transactions, streams, functions and plugins. Encrypted at rest, replicated across a lattice of hexes that elects its own leader and survives losing one.
- REST + GraphQL + SQL
- one query engine
- AES-256-GCM
- every byte on disk
- Auto failover
- no coordinator
# Insert a document
curl -X POST http://localhost:7700/orders \
-H "Authorization: Bearer $HEXDB_TOKEN" \
-H "Content-Type: application/json" \
-d '{ "customer": "ada", "total": 42, "status": "paid" }'
# Query it back, largest first
curl -G http://localhost:7700/orders \
-H "Authorization: Bearer $HEXDB_TOKEN" \
--data-urlencode 'filter={"status":"paid","total":{"$gte":10}}' \
--data-urlencode 'sort=-total'{
"documents": [
{ "id": "01JTY87RVJ9B5863KMB2YD896B",
"customer": "ada", "total": 42, "status": "paid" }
],
"total": 1,
"plan": { "indexes": ["status"], "scanned": 1 }
}The one-slide version
HexDB, Explained at a High Level
No jargon. What it does for the business, what it replaces, and why it costs so little. Print it or put it on a screen.
In one sentence: HexDB is a database that keeps your data encrypted, copied and available, in a single program that runs anywhere, with no licence fees.
- 01
Stolen data is unreadable
Everything HexDB writes to disk is encrypted with AES-256, the standard banks and governments rely on. Without your key, a stolen disk or backup is noise.
Keys can be changed without stopping the service.
- 02
It keeps running when hardware fails
Run it on two or more servers and they keep full copies of each other. If the one in charge dies, another takes over in seconds, by itself.
In memory, each record is kept in six pieces with two spare. Any two can be corrupted and it is rebuilt.
- 03
It grows by adding servers
Need to serve more readers or survive more failures? Start another server with the same name and secret. It finds the others, copies the data and starts working.
No clustering software, no coordinator, no consultant.
- 04
It runs wherever your teams work
Linux, macOS, Windows and Docker. Official libraries for JavaScript, TypeScript, Python and .NET (including Entity Framework), and plain web requests for everything else.
Any language, any framework, any cloud or none.
- 05
It costs almost nothing to own
Open source, with no licence fees. One small program does the work of a database, a search engine, a message queue, a job scheduler and an admin console, so there is less to buy, run and secure.
Runs comfortably on modest hardware.
- 06
It stores less and finds things faster
Records are compressed before they are encrypted, so disks and backups stay small. Indexes and full-text search find what you need without reading everything.
Reports and real-time alerts are built in, and BI tools can ask it questions in SQL.
What it replaces
- Database
- Search engine
- Message queue
- Job scheduler
- Admin console
- Encryption layer
Fewer moving parts means fewer things to patch, monitor, licence and explain to auditors.
Everything in the box
A whole data platform, not just a key-value store with ambitions.
Most databases make you bolt on search, a message bus, a scheduler, encryption and an admin console. HexDB ships them in the same process, behind the same permissions, replicated the same way.
Anatomy of a hex
What one hex does with your data.
A hex is one running HexDB server. Step through with the arrows, or press play and it advances every few seconds, to follow a document from the moment a request arrives, through the write-ahead log, into six error-correcting vertices, out to the change feed, and down to disk.
A request arrives
REST, GraphQL, the admin UI and the drivers all come through the same door.
- One process serves the HTTP API, GraphQL and the admin UI, on one port.
- Every request needs credentials: a session cookie or an hxk_ API key. There is no unauthenticated mode, and unknown paths get 401 too.
- A write sent to a replica is forwarded to the Overseer, signed with the lattice key, and the response says where it went.
Step 1 of 7. Use the arrows to move at your own pace.
The parts, in one glance
- The server
- One Rust binary, hexdb_api, serving REST, GraphQL and the admin UI on a single port. The hexdb CLI starts, stops, inspects and backs it up.
- Tessellations
- Named collections of JSON documents, like tables. Each can have indexes, a text index with its own analyzer, and a versioned schema. Roles are granted per tessellation.
- Querying
- One filter language for equality, ranges, sets, prefixes and nested paths, with sorting, paging, counts and aggregations. Full-text search with stemming, n-grams and autocomplete. Spoken three ways: REST, GraphQL and read-only SQL. Projections pick fields in GraphQL (field paths) and SQL (select columns).
- Percolation
- Queries that wait for documents instead of the other way round. A stream source, a trigger or a plugin registers a filter, and every write that matches it is delivered as it commits: to a webhook, a script, Kafka, or a consumer group.
- Memory
- The newest unflushed documents plus an LRU cache of recent reads, within memory.ram_mb. Every document in memory is spread over six vertices: four data, two parity.
- Disk
- A write-ahead log for durability, immutable SSTables for bulk storage, encrypted index snapshots, a sealed catalog, and a change-history archive. All AES-256-GCM.
- The catalog
- Tessellations, index definitions, schemas, the change-history ID and the lattice name, in one sealed file. Replicas copy it from the Overseer and re-read it within a poll when its fingerprint changes.
- Background work
- TTL sweeps, vertex integrity checks, flushes, compaction, metrics every 15 seconds, discovery probes, and, on the Overseer, plugins, stream deliveries, after triggers and schedules.
Hexes, tessellations, lattices
A lattice that elects its own leader and heals itself.
Start one hex and you have a database. Start three with the same lattice name and secret, and they discover each other, elect an Overseer, replicate, and fail over on their own. No coordinator, no sidecar, no operator in the loop.
Three hexes, one Overseer. Replicas follow the change feed and serve reads.
Try it for real: node scripts/lattice-demo.mjs, then type stop 1.
Three roles
- Overseer. Takes writes, publishes the change feed, runs plugins, stream sources and schedules.
- Harvester. A read replica that follows the Overseer and can be elected if it fails.
- Replicant. A standby copy that follows but is never elected. For backups and rebuilding Harvesters.
The vocabulary
- Hex
- One running HexDB server.
- Tessellation
- A collection of documents, like a table. Roles are granted per tessellation.
- Lattice
- A group of hexes sharing a name and a secret that can reach each other. A single hex is a lattice of one.
- Vertex
- One of the six regions of a hex's memory. Four data vertices each hold a quarter of every document; two parity vertices hold Reed-Solomon check data that can rebuild any two missing quarters.
Durability is a dial
Replication is asynchronous by default. Two settings let you require that a write exists on replicas before it's acknowledged, and that an Overseer refuses writes when it can't see a majority.
# hexdb.toml: trade a little latency for a lot of safety
[replication]
min_acks = 1 # replicas that must hold a write before it's acknowledged
ack_timeout_ms = 5000 # on timeout the write stays committed; the client gets 503
quorum = 2 # hexes the Overseer must see to accept writes (no split brain)
forward_writes = true # a write sent to a replica is forwarded to the Overseernode scripts/lattice-demo.mjs # three hexes on 7800/7810/7820 with sample data
> stop 1 # stop the Overseer and watch another hex take over
> start 1 # it comes back as a replicaAdmin UI
A control room in every hex.
Every HexDB server hosts its own admin UI. Watch the lattice, browse and edit documents, run GraphQL, manage indexes, schemas, streams, functions, users and roles, read the audit trail and tail the log, all from one place and all through the same permissions as the API.
What you're looking at
- 1Only the pages your role allows
The sidebar shows what the signed-in user may use. Everything else is hidden here and refused by the server anyway.
- 2Live figures
Documents in memory and on disk, tessellations, memory and disk against their budgets, writes and reads per minute, uptime and version.
- 3Activity, 15 minutes to 6 hours
Documents per tessellation, operations per minute, or storage over time. The history survives restarts.
- 4Vertex health
The six memory vertices of this hex, each with its size and repairs. Any two can fail and documents are rebuilt from the rest.
- 5The lattice
Every hex with its role, address, replication state and lag. “Add a hex” shows the settings a new server needs to join.
- 6Flush and back up
Write unflushed data to SSTables or take a consistent online backup, right from the dashboard, if your role has the maintenance permission.
- Served at /ui/ by every hex, replicas included. No separate service to deploy or secure.
- It calls the same API as any client, with the signed-in user's permissions, so what you see is what the API allows.
- Light, dark and system themes. Keyboard shortcuts in the query console (Ctrl+Enter runs).
- Sessions use HttpOnly, SameSite cookies with CSRF protection; MFA and API keys are managed on the Account page.
Every page, and what it does
- Dashboard
- Live document, memory, disk and operation figures, activity charts, vertex health, the lattice and 'Add a hex'
- Queries
- A GraphQL console with schema-aware completion, examples and history
- Tessellations
- Create and delete; manage indexes (with analyzer choice and suggestions) and schemas, with version history and rollback
- Documents
- Browse with filters and sorting; create, edit and delete documents
- Streams
- Create streams, publish, read messages, watch consumer groups and deliveries
- Functions
- Create and run functions; manage schedules and triggers
- Users, Roles
- Accounts, role grants, user attributes, custom roles with row filters and hidden fields, MFA resets
- Audit Trail
- Security events, filterable by user, action and time
- Plugins
- Loaded plugins, their state and deliveries
- Logs
- The server log, tailed live
- Settings
- Runtime settings and the effective configuration
- Account
- Your password, MFA and API keys
Users, Roles, Audit Trail, Plugins, Logs and Settings appear only for roles that hold the matching permission.
The API
Small surface. Deep features.
If you can send JSON over HTTP you already know how to use HexDB. Here's what that buys you.
Plain HTTP. Every write is atomic and durable before it returns.
- PUT replaces, PATCH is a JSON merge patch, null removes a field.
- Bulk writes, update-by-filter and upserts are each a single atomic write.
- Every write takes an Idempotency-Key: a retry returns the original response, even after a crash.
- A write to a tessellation that doesn't exist creates it.
# Create: 201 with the document and a Location header. ?ttl=<seconds> sets an expiry.
curl -X POST http://localhost:7700/articles -H "Content-Type: application/json" \
-d '{ "title": "Quantum Tessellation", "tags": ["hexdb", "rust"], "published": true, "views": 445 }'
# Read, replace, merge-patch (null removes a field), delete
curl http://localhost:7700/articles/01JTY87RVJ9B5863KMB2YD896B # ETag = version
curl -X PUT http://localhost:7700/articles/01JTY... -d '{ "title": "New" }'
curl -X PATCH http://localhost:7700/articles/01JTY... -d '{ "views": 446, "tags": null }'
curl -X DELETE http://localhost:7700/articles/01JTY... # 204
# Bulk: one atomic write, up to 10,000 documents
curl -X POST http://localhost:7700/articles/_bulk -d '[{ "title": "One" }, { "title": "Two" }]'
# Patch everything a filter matches
curl -X POST http://localhost:7700/articles/_update \
-d '{ "filter": { "status": "draft" }, "update": { "status": "published" } }'
# => { "matched": 10, "modified": 10 }
# Upsert by key fields
curl -X POST http://localhost:7700/customers/_upsert \
-d '{ "key": ["source_id"], "documents": [{ "source_id": 17, "name": "Ada" }] }'Drivers
Or skip HTTP and use a client.
Node.js, Python and .NET drivers cover documents, queries, aggregations, transactions, idempotency keys, the change feed, streams, functions and GraphQL. They retry 429 and 503 where it's safe to. On .NET there is also an Entity Framework Core provider: LINQ filters, sorting and paging run on the server, and each SaveChanges is one atomic transaction.
import { HexDB } from "hexdb"
const db = new HexDB({ url: "http://127.0.0.1:7700", apiKey: process.env.HEXDB_API_KEY })
const orders = db.tessellation("orders")
await orders.insert({ customer: "ada", total: 12 })
const page = await orders.query({ filter: { total: { $gt: 10 } }, sort: "-total" })Storage engine
Built so you don't lose data. Or sleep.
A log-structured engine written from scratch in Rust: a write-ahead log with group commit, SSTables with compaction, encryption on every byte that touches disk, and error-correcting memory that repairs itself.
The write path
- 01Validate
Permissions, size limits, the schema, and expected versions for read-modify-write operations.
- 02Sequence and apply
Under one state lock the batch gets consecutive sequence numbers, is queued to the WAL and applied to memory and indexes. A query always sees its own writes.
- 03Group commit
A dedicated thread batches whatever is waiting into one write and one fsync. Zstandard first, then AES-256-GCM.
- 04Acknowledge and publish
Once durable, the write is acknowledged, published to the change feed, and, with min_acks, held until enough replicas confirm.
On disk and in memory
- Memory
- Newest unflushed documents plus an LRU cache, each spread over four data vertices and two parity vertices with Reed-Solomon coding. See the hex above.
- SSTables
- Immutable, per tessellation, with Bloom filters and a chunked encrypted index. About 2 bytes of RAM per document.
- Compaction
- Merges SSTables, drops tombstones and expired documents, and rewrites files on old formats or old keys.
- Recovery
- Replays the WAL in order. A torn final record is tolerated; anything that can't be decrypted stops startup rather than discarding data.
Encrypted at rest, rotated without downtime
WAL records, SSTable bodies and indexes, the catalog, settings, metrics and index snapshots are all AES-256-GCM with random 96-bit nonces, authenticated with their IDs so files and entries can't be swapped. Only SSTable headers are plaintext. A server whose data needs a key it doesn't have refuses to start rather than discard anything.
# Every developer and every deployment has its own key. Keep it.
printf '[storage]\nencryption_key = "%s"\n' "$(hexdb secret)" > hexdb_api/hexdb.local.toml
# Rotate without downtime: add the new key, move the old one to previous_encryption_keys,
# restart, and let compaction rewrite the files. Then drop the old key.
[storage]
encryption_key = "base64:NEW..."
previous_encryption_keys = ["base64:OLD..."]What's in the data directory
Everything a hex knows lives in one folder. Back it up online with one command, and restore by pointing a hex at the copy.
<storage.path>/
├── catalog.hxe tessellations, indexes, schemas, history ID, lattice name (sealed)
├── settings.hxe runtime settings changed from the UI or API (sealed)
├── metrics-history.hxe 6 hours of metrics samples (sealed)
├── query-stats.hxe the query advisor's statistics (sealed)
├── hexdb.pid PID, endpoint and shutdown token (owner-only)
├── wal/
│ ├── <first-seq>.wal active write-ahead log segments
│ └── archive/<first-seq>.wal the change history
├── indexes/<tessellation>/<index>.hxi index snapshots (sealed)
└── <tessellation>/<ulid>.hxs SSTables, including system tessellationsSecurity
Secure by default, not by checklist.
There is no unauthenticated mode. Every request outside a short public list needs credentials, every refusal is audited, and everything on disk is encrypted with a key only you hold.
Nine permissions, any combination
Built-in roles cover the usual cases (reader, writer, owner, operator, auditor, admin). Custom roles pick from the same nine permissions.
| Permission | Allows | Scope |
|---|---|---|
| read | Get, list, query, count, aggregate, the change feed; consume streams | per tessellation |
| write | Insert, replace, patch, delete; publish to streams | per tessellation |
| manage | Delete tessellations; manage indexes, schemas and advice; configure streams | per tessellation |
| status | Server status, metrics, storage and lattice details | global |
| logs | The server log | global |
| audit | The audit trail | global |
| plugins | Plugins and their delivery state | global |
| maintenance | Flush, compact and back up | global |
| admin | Everything: users, roles, settings, functions, schedules, joining hexes | global |
Grants, in one request
A grant names a role and the tessellations it applies to. Streams use the resource name stream:<name>. A grant on * covers everything.
# A grant names a role and the tessellations it applies to
curl -X POST http://localhost:7700/users -H "Content-Type: application/json" -d '{
"login": "ada", "password": "correct horse battery staple",
"roles": [{ "name": "writer", "tessellations": ["orders", "customers"] },
{ "name": "reader", "tessellations": ["*"] }] }'
# Custom roles from any set of permissions
curl -X POST http://localhost:7700/roles \
-d '{ "name": "analyst", "permissions": ["read", "status"] }'Row filters and hidden fields
A role can be restricted per tessellation: a filter that limits which documents it sees and writes, using the caller's login, email or attributes, and a list of fields it can never see or change.
# A role that only sees and writes its own region, and never sees cost or margin
curl -X POST http://localhost:7700/roles -H "Content-Type: application/json" -d '{
"name": "regional", "permissions": ["read", "write"],
"restrictions": { "orders": { "filter": { "region": { "$user": "attributes.region" } },
"hide": ["cost", "margin"] } } }'
curl -X PATCH http://localhost:7700/users/ada -d '{ "attributes": { "region": "EU" } }'
# Queries, counts, aggregations, GraphQL and the change feed all see only matching documents.
# Filtering or sorting on a hidden field is refused, so values can't be probed.- • Responses carry nosniff, DENY framing, a no-referrer policy and a Content Security Policy.
- • No CORS headers are sent, so other sites can't read responses. Cookie-authenticated changes must be same-origin (CSRF protection).
- • TLS is two settings away, with HSTS, Secure cookies and HTTPS replication between hexes.
Operations
A CLI and an API for everything the UI does.
Nothing in HexDB is only reachable by clicking. The hexdb command starts, stops, inspects, backs up and queries a server, and every dashboard action is a documented REST call you can script.
hexdb start [-s] start the server (-s: in the background)
hexdb stop [--force] stop it gracefully
hexdb health is it up?
hexdb status status and metrics
hexdb secret print a new random key
hexdb backup [--name N] [--list] back up the running server
hexdb sql "SELECT ..." [-p V] run a SQL query and print a table
hexdb plugins add|remove|list manage the plugin registry
hexdb lattice spawn --count 2 start more hexes that join this lattice
hexdb lattice list|stop list or stop themcurl http://localhost:7700/status # documents, memory, disk, vertices, lattice, lag
curl "http://localhost:7700/status/history?minutes=60" # a sample every 15 s, kept for 6 hours
curl "http://localhost:7700/logs?level=warn&limit=100" # tail, search and filter the log
curl -X POST http://localhost:7700/backup -d '{"name": "nightly-1"}' # consistent, while serving writes
curl http://localhost:7700/openapi.json # the REST API as OpenAPI 3.1- Backups
- Consistent and online. SSTables are hard-linked, the WAL is rotated, and writes keep flowing. Restore by pointing a hex at the copy with the same keys.
- Metrics and logs
- A sample every 15 seconds, kept for six hours across restarts. The last 5,000 log records are searchable live. Export both with the OpenTelemetry plugin.
- Runtime settings
- Limits, retention, session length and replication acknowledgements change live from the UI or API. The rest apply at the next restart and are listed as pending until then.
- Upgrades
- Stop, replace the binaries and the UI, start. Older file formats are read and rewritten by compaction. In a lattice, upgrade replicas first and let failover move the Overseer.
- Packaging
- Archives for Linux, macOS and Windows, a Debian package, a Docker image and compose file, a systemd unit, and Homebrew, Chocolatey and winget manifests.
- Health checks
- /health is public and cheap. /status adds documents, memory, disk, vertices, lattice membership and replication lag for anything that wants detail.
Ecosystem
Plugs into what you already run.
Plugins extend HexDB without changing it. Each is a folder with a manifest: a process in any language reading JSON lines on stdin, a webhook, or a built-in exporter. They run on the Overseer, so a lattice delivers each payload once, and they resume from a saved position after a restart.
| Plugin | Type | What it does |
|---|---|---|
| @streams/kafka | stream | Produces changes to Kafka, keyed by document ID |
| @streams/kinesis | stream | Puts changes on an Amazon Kinesis stream |
| @sinks/otlp | metrics | Metrics to an OpenTelemetry endpoint (built in) |
| @sinks/log-file | logs | Rotating JSON-lines log files |
| @sinks/http-logs | logs | Log records to Splunk HEC, Logstash and friends |
| @sources/postgres | source | Copies a PostgreSQL table and keeps it in sync |
| @sources/mssql | source | The same for SQL Server |
| @sources/s3 | source | Imports JSON and JSON-lines objects from S3 |
| @enrichers/ai-keywords | stream | Adds keywords to new documents with Claude |
| @examples/change-logger | stream | Logs each change; the minimal example |
Process plugins run in a clean environment with only the variables they're given. The server's own keys are never passed.
import { payloads, api, log } from "../../sdk/hexdb-plugin.mjs"
for await (const change of payloads()) {
if (change.op === "put") log("order", change.id, change.document.total)
}
await api().post("/orders", { total: 12 }) // plugins with [access]Get started
From a clean machine to a running lattice in minutes.
Five steps from source, or two commands with Docker. The quick start in the README has the Windows PowerShell versions and the details.
- 1
Install the prerequisites
Git, Rust (stable), Node.js 22 or later and a C toolchain. On Windows use PowerShell.
# Linux (Debian/Ubuntu) sudo apt update && sudo apt install -y build-essential git pkg-config libssl-dev curl curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y # macOS xcode-select --install brew install node - 2
Build it
The first Rust build takes a few minutes and produces the server and the CLI.
git clone https://github.com/dreaminhex/hexdb.git && cd hexdb cd hexdb_admin && npm ci && npm run build && cd .. cargo build --release -p hexdb_api -p hexdb_cli - 3
Put hexdb on your PATH
Installing into Cargo's bin folder works the same on every OS.
cargo install --path hexdb_cli cargo install --path hexdb_api hexdb --help - 4
Create your encryption key
HexDB encrypts everything it stores. The key lives in a git-ignored file.
printf '[storage]\nencryption_key = "%s"\n' "$(hexdb secret)" > hexdb_api/hexdb.local.toml - 5
Start it and sign in
The first start prints a generated administrator password. Open the admin UI, change it, and create an API key.
cd hexdb_api hexdb start # open http://localhost:7700/ui/
node scripts/seed.mjs.FAQ
Questions people ask first.
Contact
Talk to the person who built it.
Questions about a deployment, a feature you need, a bug you hit, or a project you'd like help with. Messages go straight to the maintainer.
HexDB is open source and built in the open. Stars, issues and pull requests are all welcome.