Getting Started

How to build Waratah and get a queryable cluster running on one machine.

Prerequisites

  • Rust — stable toolchain.

  • C++ toolchain — g++ (or another C++ compiler) for the bundled RocksDB + LZ4 sources, plus clang and libclang for bindgen’s generated FFI.

  • Node.js + npm — only for building this documentation site (scripts/docs/build.sh), not for the engine.

The storage brick links RocksDB through waratah-rocksdb-sys (waratah_rocksdb_sys/ at the repo root) — a Waratah-owned fork of Oxigraph’s oxrocksdb-sys that builds a trimmed RocksDB tree plus bundled LZ4 through the cc crate and generates the C FFI with bindgen at compile time. On Linux the build probes for liburing and links it dynamically when found (set ROCKSDB_USE_IO_URING=0 to disable; a binary built with it needs liburing at runtime). Both the RocksDB and in-memory backends are always compiled; tests can use WARATAH_STORAGE_BACKEND=memory to skip disk entirely.

Build

# Whole workspace. Also refreshes the waratah-partition / waratah-registry
# binaries that `waratah start` spawns as child processes.
cargo build --workspace

# Faster check without producing artifacts:
cargo check --workspace
If your environment sets CARGO_TARGET_DIR to a shared cache (some editors do), override it back to the checkout’s own target/ directory — otherwise every build recompiles the workspace.

Run the demo cluster

waratah start runs a five-process local cluster: three S/P/O hexastore partitions and one string registry as child processes, plus compute and the RDF4J HTTP API in the CLI process.

./target/debug/waratah start --load-demo

The command daemonizes by default: it returns once the server is up, and stdout/stderr are appended to log files instead of the terminal. With a data dir they live under {data_dir}/logs/ (output.txt, error.txt, and debug.txt when --debug is given); for the memory backend they land in the working directory. Watch them with tail -f:

# memory backend (no --data-dir): logs are in the working directory
tail -f output.txt

The RDF4J base URL is http://127.0.0.1:8080/rdf4j/, and the grevillea console is served at http://127.0.0.1:8080/. Cluster state is at GET /waratah/health and GET /waratah/cluster.

Add --debug to log SPARQL query text, per-query phase timings, and client-facing errors — stderr with --foreground; when daemonized, {data_dir}/logs/debug.txt with a data dir, ./debug.txt for the memory backend.

Keep the cluster attached to the terminal instead (the historical behavior) with --foreground; note that --query and --no-http imply it, since a one-shot query’s results and the REPL need the calling terminal:

./target/debug/waratah start --load-demo \
  --query 'SELECT ?person ?weather WHERE {
    ?person <http://ex/livesIn> ?city . ?city <http://ex/hasWeather> ?weather . }'

Stop a running cluster with waratah stop (memory backend; finds ./waratah-dev.pid in the working directory) or waratah stop --data-dir <dir> (SIGTERM via its pidfile; storage children and the notebook sidecar go with it).

In-process storage: waratah standalone

waratah standalone opens the S/P/O partitions and the registry inside the CLI process — no child processes, no gRPC, always in the foreground. Same data-dir layout as waratah import --offline, same SPARQL and HTTP session:

./target/debug/waratah standalone --load-demo

Bulk-load RDF

waratah import streams Turtle / TriG / N-Triples / N-Quads / RDF/XML (optionally .gz) into a RocksDB data directory:

# Default: spawns S/P/O + registry children, streams quads in fixed-size
# chunks, flushes MemTables, optionally force-compacts once at the end.
./target/debug/waratah import data.ttl.gz --data-dir ./waratah-data --compact

# Write in-process (no gRPC), skip the write-ahead log:
./target/debug/waratah import data.ttl.gz --data-dir ./waratah-data --offline --disable-wal

# Attach to a running cluster instead of writing a data dir:
./target/debug/waratah import data.ttl --connect --compact

# Then query that data dir without spawning children:
./target/debug/waratah standalone --data-dir ./waratah-data \
  --query 'SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 10'

Persistence and spill directories

  • --data-dir DIR persists partitions to RocksDB under DIR/main/{s,p,o,registry} (created if missing). Without it, storage runs in memory.

  • --spill-dir DIR is the root for query spill files: external ORDER BY runs, Grace join partitions, parked pages, and DISTINCT seen-sets. It defaults to {data_dir}/spill. Without a data dir, spill falls back to the system temp directory — which is RAM-backed (tmpfs) on many Linux systems; a warning is logged when that happens. Point it at real disk for large queries.

Querying

Anything that speaks the SPARQL Protocol works against the RDF4J endpoint — curl, RDF4J clients, or the console. A minimal example:

curl -G 'http://127.0.0.1:8080/rdf4j/repositories/main' \
  --data-urlencode 'query=SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 5' \
  -H 'Accept: text/csv'

Serialization is content-negotiated; see the RDF4J endpoint page for the full format matrix. Known W3C-spec departures are listed in Deviations from spec.

What to look at next