Waratah ingest API (server contract)

This document is the server-side HTTP contract for the Import page. The WebUI is a different project in a different git repo and is not implemented here. Hand this page (and ref/waratah_openapi.yaml) to the frontend agent.

RDF4J SPARQL stays at /rdf4j (see the RDF4J endpoint page). Staging + ingest is Waratah-specific under /waratah. There is no SPARQL INSERT DATA path, no RDF4J /statements upload in v1, and no CORS (proxy /waratah and /rdf4j onto the same origin as the UI).

Base URL

waratah start default (foreground or daemonized):

http://127.0.0.1:8080
API Prefix

RDF4J (SPARQL, size, …)

http://127.0.0.1:8080/rdf4j/

Ingest (this page)

http://127.0.0.1:8080/waratah/

Repository id is main. Staging files live in {--ingest-dir}/{repository_id} (default {data_dir}/ingest/main, or ./ingest/main when there is no --data-dir). Operator-copied files and UI uploads share that one tree. The repository id in the URL selects the staging folder; today only main is registered.

Typical Import-page flow

  1. GET /waratah/repositories/main/ingest — flat list of RDF files. Render a directory tree from name prefixes (nested/file1.ttl → folder nested / file file1.ttl).

  2. Upload: one complete multipart/form-data POST per file. Track progress with XHR/fetch upload loaded / total on that request. The server does not assemble chunks.

  3. Or paste a snippet: JSON POST, then it appears in the list like any other file.

  4. User clicks a row and Ingest: JSON POST with that name (and optional graph). The request is synchronous and can take a long time. No job id, no cancel in v1.

  5. Files stay on disk after ingest (re-ingest is allowed). There is no delete-from-staging endpoint in v1.

Paths

Always send relative POSIX paths: nested/file1.ttl. Never send ~, ../, ./, or absolute paths. The server strips those prefixes and rejects a remaining .. component (so nested/../../../etc/passwd is 400).

Endpoints

List staged files

GET /waratah/repositories/{repository_id}/ingest

{
  "files": [
    {
      "name": "nested/file1.ttl",
      "bytes": 12840,
      "mtime_ms": 1756350000000,
      "format": "turtle",
      "gzip": false
    }
  ]
}

Only known RDF extensions are listed (.ttl, .turtle, .nt, .nq, .trig, .rdf, .xml, .owl, plus .gz). Dotfiles and non-RDF files are omitted. Nested directories appear only as prefixes in name.

Upload a file

POST /waratah/repositories/{repository_id}/ingest/files Content-Type: multipart/form-data

Part Required Meaning

file

yes

File bytes. Use the original filename if path is omitted (basename only).

path

no

Destination relative to the ingest root, e.g. nested/file1.ttl.

Response 201:

{ "name": "nested/file1.ttl", "bytes": 12840 }

Unknown RDF extensions → 400. Overwrites an existing file. Creates parent directories. Body limit 1 GiB (413 if exceeded).

Upload a snippet

POST /waratah/repositories/{repository_id}/ingest/snippets Content-Type: application/json

{
  "path": "nested/paste",
  "format": "turtle",
  "data": "@prefix ex: <http://ex/> . ex:s ex:p ex:o .",
  "base_iri": null
}

format is one of: turtle, ntriples, nquads, trig, rdfxml. If path has no RDF extension, the server appends one (.ttl, .nt, .nq, .trig, .rdf). Nested path is allowed.

Optional base_iri is stored beside the file and used when that file is ingested (unless ingest supplies its own base_iri). There is no graph field here — choose the named graph on the ingest POST.

Response 201: { "name": "nested/paste.ttl", "bytes": … }.

Ingest a staged file

POST /waratah/repositories/{repository_id}/ingest Content-Type: application/json

{ "name": "nested/file1.ttl", "base_iri": null, "graph": null }
Field Required Meaning

name

yes

Relative POSIX path from the list / upload response.

base_iri

no

Override the file-URL base IRI (and any snippet sidecar).

graph

no

Absolute IRI of the named graph for default-graph triples. Omit, null, or "" to keep RDF DefaultGraph.

graph uses oxrdfio with_default_graph: Turtle / N-Triples / N3 / RDF/XML triples all land in that named graph. In TriG / N-Quads, only default-graph triples are remapped; triples already written in an explicit named graph stay there. Invalid IRI → 400.

If the Import page has a GraphDB-style "target graph" field, send it here, not on snippet write. The same staged file can be ingested again with a different graph.

Parses the file, sends quads to each storage partition, flushes memtables, and refreshes planner statistics. The file is not deleted.

Response 200:

{ "name": "nested/file1.ttl", "quads": 42, "registered": 10 }

Missing file → 404. RDF parse errors and invalid graph IRI → 400.

Errors (plain text body)

Status When

400

Bad path, unknown format, malformed RDF, invalid graph IRI, missing multipart file

404

Unknown repository, or ingest name not in the staging dir

413

Body larger than 1 GiB

501

Ingest directory not configured (tests / no HTTP ingest dir)

Curl examples

BASE=http://127.0.0.1:8080

curl "$BASE/waratah/repositories/main/ingest"

curl -X POST "$BASE/waratah/repositories/main/ingest/files" \
  -F "path=nested/file1.ttl" \
  -F "file=@./file1.ttl"

curl -X POST "$BASE/waratah/repositories/main/ingest/snippets" \
  -H "Content-Type: application/json" \
  -d '{"path":"paste.ttl","format":"turtle","data":"<http://ex/s> <http://ex/p> <http://ex/o> ."}'

curl -X POST "$BASE/waratah/repositories/main/ingest" \
  -H "Content-Type: application/json" \
  -d '{"name":"nested/file1.ttl"}'

curl -X POST "$BASE/waratah/repositories/main/ingest" \
  -H "Content-Type: application/json" \
  -d '{"name":"nested/file1.ttl","graph":"http://ex/g1"}'

Out of scope for v1 (do not call)

  • SPARQL Update / INSERT DATA

  • RDF4J POST /rdf4j/repositories/main/statements

  • Chunked or resumable uploads

  • Delete a staged file

  • Async jobs, progress polling, or cancel