Waratah

Waratah is a distributed RDF quadstore engine designed from the ground up for horizontal scaling and sharded storage, to overcome the limitations of traditional vertically scaled RDF storage engines.

It speaks SPARQL 1.1 — SELECT, ASK, CONSTRUCT, and DESCRIBE — over an RDF4J-compatible REST protocol, and extends it with a GeoSPARQL v1 plugin (geof:* functions, S2 spatial indexing) and a temporal plugin (tf: interval functions, dyadic time indexing).

Waratah is pre-1.0 software (workspace version 0.3.x). The storage-compute split, the hexastore partitioning, and the vectorized query engine exist and run today as a single-binary developer cluster; the consensus (Multi-Raft) and multi-node shuffle layers are planned phases, not current behavior. See internals for what the engine actually does today.

How it is put together

Storage–compute disaggregation: Stateless compute nodes parse SPARQL, plan, and execute over columnar frames; storage shards own the data. A single waratah binary runs the whole developer topology — see Getting Started.
Three-way hexastore partitioning: Every quad is written to three partitions holding complementary index orders: the S node keeps SPOG/SOPG, the P node PSOG/POSG, the O node OPSG plus a TSG index dedicated to rdf:type. Any single triple pattern routes to one partition and resolves against a long key prefix.
Vectorized, late-materialized execution: Scans return encoded 128-bit term columns, not strings. Joins, filters, and aggregations run on those columns in batches (SCAN_BATCH = 10_000); strings are resolved from the registry only for final result bindings. There is no tuple-at-a-time iteration across node boundaries.
Mergeable planner statistics: Per-predicate Count-Min Sketches and HyperLogLog distinct counts merge algebraically from storage into the planner. They pick join order and join strategy only — they are advisory and never change answers.
MVCC versioned storage: Quad records are versioned keys (encoded terms ‖ descending commit_ts) in RocksDB; reads run at compute-assigned snapshot timestamps with no cluster-wide read locks.

Query and HTTP surface

RDF4J / SPARQL endpoint: SPARQL Protocol queries with content negotiation (JSON / XML / CSV / TSV results; Turtle / N-Triples / TriG / N-Quads / RDF-XML / JSON-LD graphs).
Ingest API: Stage RDF files (upload or paste), then ingest them into a repository, optionally into a named graph.
Cluster API: Process health, live cluster snapshot with per-process version strings, and notebook-service discovery.

The same listener serves the pre-built grevillea console (a static SPA) at / and an optional Jupyter notebook sidecar at /jupyter.

Conformance and internals

Deviations from spec documents every known departure from the W3C RDF/SPARQL standards, with the reason it was kept. Planner and executor optimizations is the full inventory of how queries are planned and executed.

Where to start

Getting Started — build, run the demo cluster, load data, query it.
CLI Reference — every waratah subcommand and flag.