1.1 The eight assumptions

Most research-supporting systems silently assume one or more of the following:

1. There is one correct value to store for a given proposition.
2. There is one canonical schema to map data into.
3. There is one entity-resolution answer per referent.
4. A user either has access to a record or they do not.
5. Provenance is metadata attached to records rather than the central object of the system.
6. Machine confidence can stand in for scholarly review.
7. Exports are static files rather than reproducible views.
8. Contradictions are quality failures, not data.

donto rejects all eight. Its operating model is open-world, evidence-first, contradiction-preserving, governance-native, bitemporal, multimodal, and schema-plural (PRD §0). The product question the system was designed to answer is:

Given a contested question, can the system return the relevant claims, the evidence behind them, the schema mappings that make them comparable, the identity hypotheses they depend on, the disagreements between them, the access policies governing them, and a reproducible release artefact?

If it can, the substrate works.

1.2 The proving domain

The first proving domain is language documentation, because it exhibits every constraint of contested knowledge simultaneously: incompatible analytical schemas (Universal Dependencies vs WALS vs Grambank), disputed identities (dialect/language boundaries; ISO codes vs Glottolog vs internal community ontologies), multimodal evidence (text, audio, interlinear glosses, phonetic transcription), restricted cultural material (community-governed sacred or sensitive records), diachronic change (reconstructed vs attested forms), formal validation (paradigms have shape constraints), and corpus-scale annotation.

The exercise domain is genealogy — specifically, the North-Queensland family-history corpus that exercises donto in production at genes.apexpots.com. Genealogy is, in our experience, the hardest realistic instance of the same problem: name spelling drifts across records, the same person appears under maiden, married, and clan names, colonial-era records contain misclassifications and falsifications as data, identity is contested across native-title determinations, and oral-history claims contradict written archival ones in irreducibly load-bearing ways.

The point of the genealogy exercise is not the genealogy. It is that every contradiction in genealogy is a tripwire for donto's invariants. Each friction point we hit becomes either a new tripwire test in packages/donto-client/tests/ or an amendment to the PRD; we are deliberately running the substrate at its limits because that is how we learn what the substrate must be.

1.3 Contribution

This paper describes:

• A 14-family object model (claims, frames, contexts, sources, revisions, anchors, predicates, entities, alignment edges, argument edges, identity hypotheses, policy capsules, attestations, release manifests) implemented as 91 PostgreSQL relations across 131 idempotent migrations (§5).
• DontoQL, a 21-clause query language that compiles to a unified algebra alongside a SPARQL 1.1 subset; first-class clauses for bitemporal time-travel, identity lenses, predicate-closure expansion, modality and extraction-level filtering, policy enforcement, and contradiction-pressure ordering (§6).
• A six-aperture exhaustive-by-default extraction pipeline (surface, linguistic, presupposition, inferential, conceivable, recursive) with content-hash deduplication, vocabulary-aware prompting that reuses existing predicates rather than minting new ones, and confidence-to-maturity mapping that caps machine-extracted claims below human-reviewed maturity (§7).
• A Trust Kernel layering policy capsules, attestation credentials, and append-only event logs over a fail-closed default policy, with one historical write-side enforcement gap (F-1) closed by migration 0123 (§8).
• A typed predicate-alignment layer with eleven relations and three per-relation safety flags (safe_for_query_expansion, safe_for_export, safe_for_logical_inference), plus a materialised closure that the evaluator rides at query time (§9).
• A three-tier source-provenance trace (exact line equality → substring within line → full-body fallback) with cross-shard surface-text caching, idempotent resumability, and content- addressed blob storage (§9).
• A Lean 4 formal overlay (donto_engine) that certifies shape and rule reports asynchronously over a line-delimited JSON protocol, with the invariant that Lean certifies and does not gate: ingest is never blocked on the Lean side (§10).
• A release builder producing JSONL + RO-Crate + optional CLDF artefacts, signed with Ed25519 over a manifest SHA-256 in the did:key format, and a federation analysis comparing five candidate stacks (Verifiable Credentials + DID, Solid Pods, SPARQL federation, DataCite, RO-Crate) with reasoning for the chosen architecture (§11).
• An empirical characterisation of the live system: insert throughput, point-query latency, batch-query scaling, policy gate cost, concurrent-writer behaviour, and observations about predicate proliferation, evidence-anchor sparsity, and retraction rate that motivate concrete next steps (§12, §13).

We are not claiming any one component is novel in isolation: typed schema mapping is older than the semantic web, bitemporal databases have been formalised since the 1990s, paraconsistent logics go back to da Costa and Belnap, and the FAIR/CARE principles articulate the ethics we operate under. The contribution of this paper is the composition: a single substrate where these properties are not optional library features bolted onto a triplestore but invariants enforced from the schema upward, and a working system of nontrivial scale that we can characterise empirically rather than hypothetically.

2.1 Triple stores, quad stores, and named graphs

RDF [Cyganiak et al., 2014] and its 1.2 extension (RDF-star) describe data as subject–predicate–object triples. Quads extend this with a fourth term, typically called a graph name or named graph, which the W3C SPARQL specification [Harris and Seaborne, 2013] uses to partition triples. donto's donto_statement schema is a quad — (subject, predicate, object, context) — but the context column plays a distinct role from a SPARQL named graph: it is not merely a partition key but the carrier of policy, scope inheritance, kind (source, snapshot, hypothesis, user, pipeline, trust, derivation, quarantine, custom, system), and mode (permissive, curated). A statement's context determines who can read it, what alignments expand over it, and how it inherits policy from parent contexts. This is closer to the attribution role of Wikibase qualifiers and references [Vrandečić and Krötzsch, 2014] than to a SPARQL graph.

2.2 Bitemporal databases

donto stores valid_time (a daterange in the calendar sense — when the fact was true in the world) and tx_time (a tstzrange in real-time — when the fact was believed in the database). Retraction closes tx_time; the row is never deleted. This is the model formalised by Snodgrass [1999] and operationalised in Datomic [Hickey, 2012] and XTDB [Pratt et al., 2019]. Where donto differs is in committing to bitemporality across every mutating object, not just statements: alignment edges, identity hypotheses, policy capsules, attestations, and review decisions are all written through an append-only donto_event_log (migration 0090). The invariant no destructive overwrite (I3) is enforced uniformly.

2.3 Paraconsistent logic

Classical logic explodes on contradiction: from P ∧ ¬P any proposition follows. Paraconsistent logics, beginning with Jaśkowski [1948] and da Costa [1974] and continuing through Belnap's four-valued logic [Belnap, 1977] and Priest's LP [Priest, 1979], are calculi that tolerate contradictions without exploding. donto is not a logic; it is a substrate. But it is paraconsistent in the storage-semantic sense: two statements with the same subject and predicate but conflicting objects, polarities, or valid-time intervals can coexist as currently-believed rows, and the system exposes a contradiction frontier view enumerating such pairs. The algebraic structure is a directed argument graph (the donto_argument table) with nine typed relations (supports, rebuts, undercuts, qualifies, explains, alternative_analysis_of, same_evidence_different_analysis, same_claim_different_schema, supersedes) and a strength on [0, 1]; the truth model per claim, when needed, maps to a Belnap-style four-valued state (T, F, B for "both", N for "neither").

2.4 Provenance

PROV-O [Lebo et al., 2013] defines a vocabulary for who produced what when from what: entities, activities, agents, and a small relation set (wasGeneratedBy, used, wasDerivedFrom, wasAttributedTo). donto's donto_extraction_run, donto_document_revision, donto_evidence_link, and donto_agent tables expose roughly PROV-O-compatible structure (the agent crate even speaks PROV-O at the export boundary), but the central commitment is stronger: provenance is not metadata attached to a record, but the organising primary key of the data model. A claim without an evidence anchor is either an explicit hypothesis_only row or a violation of invariant I1.

2.5 RO-Crate, CARE, and FAIR

RO-Crate [Soiland-Reyes et al., 2022] is a metadata format for packaging research artefacts in a self-describing JSON-LD envelope. donto's release builder emits RO-Crate alongside its native JSONL, treating the crate as the citable-and-portable export format. We inherit the FAIR principles [Wilkinson et al., 2016] (findable, accessible, interoperable, reusable) and operationalise the CARE principles [Carroll et al., 2020] (collective benefit, authority to control, responsibility, ethics) through the Trust Kernel: source policy classification is required before ingest, the default for an unclassified policy is restricted_pending_review (fail-closed), and exports check derived-claim policy inheritance before emitting.

2.6 Knowledge graph extraction with language models

Recent surveys of LLM-driven knowledge graph construction [arXiv:2510.20345 and related] describe pipelines that map text through a single extractor producing typed triples. donto's extraction maximalism (§7) is a deliberate departure: rather than one pass with a single "tier" label, six independent analytical apertures decompose the same source through different lenses (surface, linguistic, presupposition, inferential, conceivable, recursive). The maturity ladder caps machine extraction at E1 (candidate) regardless of model confidence; human review is required to advance to E3 (reviewed).

2.7 Entity resolution at scale

Probabilistic record linkage [Fellegi and Sunter, 1969] and its contemporary descendants (Splink [Linacre, 2022], Dedupe [Gregg and Eder, 2015], Magellan [Konda et al., 2016], Ditto [Li et al., 2020]) formulate entity matching as a likelihood-ratio computation over features. donto stores identity as an explicit hypothesis layer (invariant I8: identity is a hypothesis, not a foreign key). Symbols (the IRIs the LLM mints) are kept verbatim; identity edges (donto_identity_edge) are weighted, bitemporal, method-attributed assertions about coreference; identity hypotheses (donto_identity_hypothesis) name clustering solutions over those edges. Queries select an "identity lens" at query time rather than collapsing entities at storage time. This means a single accepted merge does not destroy the original symbols; the system can answer "who was Mary Watson, under the strict lens?" and "under the exploratory lens?" with different answers.

2.8 Temporal modelling

Allen's interval algebra [Allen, 1983] defines thirteen interval relations (before, meets, overlaps, starts, during, finishes, equals, and their six inverses). donto preserves Allen-style relations as a bitset on temporal claims, allowing queries such as "birth before marriage" and "residence overlaps with employment". Historical dates use EDTF [LoC, 2019] semantics: "1860" is a year-grain expression covering [1860-01-01, 1861-01-01), not 1860-01-01; uncertainty markers (~, ?), intervals, and unbounded endpoints round-trip through the donto_time_expression overlay (proposed in ARCHITECTURE-REPORT §temporal and partially landed). Reducing historical dates to false day-grain precision is treated as a category error.

2.9 Formal verification overlays

Cyc and SUMO embedded inference engines directly. donto's choice is to keep the database in execution authority and let a Lean 4 binary (donto_engine) own meaning. Lean certifies shapes (functional, typed-literal, parent–child age-gap) and derivation rules (transitive closure, inverse, symmetric) over scoped statement lists; it never gates ingest. If the Lean engine is unreachable, dontosrv degrades the shape/rule/cert endpoints to "unavailable" and continues serving everything else (apps/dontosrv/src/lean.rs:117-150). The Lean overlay is more in the spirit of Coq's proof-by-extraction or Liquid Haskell's refinement types — formal where useful, optional where not.

3.1 Deployment

The production deployment runs on a single GCE e2-standard-4 VM (4 vCPU, 16 GB RAM) with PostgreSQL 16 inside a Docker container (donto-pg), volume-bound to an attached SSD at /mnt/donto-data/pgdata. systemd manages five long-running services (dontosrv, donto-api, donto-api-worker, donto-debug, dontopedia-web) plus Caddy as a TLS terminator and reverse proxy. Temporal workflows (extraction, alignment, entity resolution) run in donto-api-worker against the same Postgres instance.

3.2 Process boundaries

The choice to keep pg_donto as a pgrx extension rather than a sidecar microservice was deliberate. The SQL substrate is the contract; the pgrx layer exists to make Rust the lingua franca for plan-quality hot paths (e.g., immutable polarity / maturity decoding) and to package all 131 migrations as extension_sql_file! declarations so a CREATE EXTENSION pg_donto on a fresh database produces a working substrate. The HTTP sidecar dontosrv does not own substrate state; it is a typed gateway over donto-client's SQL surface.

3.3 Codebase footprint

                  packages/   apps/    total
Rust                ~14.0 K   ~9.3 K   52,943 LOC
SQL                              —     13,234 LOC (131 migrations)
Python (donto-api)               —      8,349 LOC
Lean 4                ~1.7 K     —      1,656 LOC
TypeScript (client-ts) ~0.9 K     —        899 LOC

The two largest crates are dontosrv (5,088 LOC of HTTP routing and sidecar protocol) and donto-cli (4,226 LOC across ~40 subcommands). donto-client (the typed Rust wrapper over the SQL surface) is 2,665 LOC; donto-query (DontoQL + SPARQL parser, algebra, evaluator) is 2,739 LOC. The five linguistic-pilot importer crates (donto-ling-{cldf,ud,unimorph,lift,eaf}) total ~2.3 K LOC.

The test surface is unusually large: 592 #[tokio::test] annotations, 91 #[test], and 511 invocations of the pg_or_skip! macro that lets database-touching tests skip cleanly when Postgres is unreachable. The 77-file tripwire suite in packages/donto-client/tests/ totals 19,968 LOC — more than the combined LOC of every package except dontosrv. The invariants_*.rs files (governance, paraconsistency, bitemporal, predicate, modality, hypothesis, evidence, releases) encode the PRD invariants as executable assertions.

4.1 The ten non-negotiable invariants

The PRD specifies ten invariants the substrate must enforce (donto/docs/DONTO-PRD.md §2). They are non-negotiable: amendments go to the PRD first, never to code without spec. They are listed here in shortened form; the precise text is in the PRD.

I1. No claim without evidence or explicit hypothesis status. A statement must reference an donto_evidence_link row, or carry a hypothesis_only=true flag, before it can advance past maturity E1.

I2. No restricted source without policy. A document cannot be registered without a policy_id. Migration 0123 promotes donto_document.policy_id to NOT NULL with a fail-closed default (policy:default/restricted_pending_review, zero allowed actions); this closes the historical F-1 gap where legacy ingest paths could produce unpoliced rows.

I3. No destructive overwrite. Corrections, retractions, merges, splits, alignments, and policy changes are append-only events. The system supports transaction-time reconstruction of what was believed and visible at any prior system time.

I4. Contradictions are preserved. Two sources disagreeing about Annie Davis's birth year both live in the database forever. The system creates a donto_argument row with relation rebuts or alternative_analysis_of and a donto_proof_obligation with kind needs_contradiction_review.

I5. Machine confidence is not maturity. A model may report confidence on [0, 1]. Maturity is earned by evidence quality, review, cross-source corroboration, or formal validation. Auto-promotion is capped at E2 for any extraction-produced claim; E3+ requires a human review decision. The helpers.py:54-59 confidence-to-maturity mapping (0.95 → 4, 0.8 → 3, 0.6 → 2, 0.4 → 1, else 0) sets the ceiling, not the floor.

I6. Governance propagates to derivatives. A claim derived from a restricted source inherits the most restrictive applicable policy of its source anchors. Embeddings, translations, summaries, and exports all inherit. Overrides require a qualified authority's attestation.

I7. Schema mappings are typed and scoped. No two predicates are "the same" by default. Alignment edges declare one of eleven relations (exact_equivalent, close_match, broad_match, narrow_match, inverse_of, decomposes_to, has_value_mapping, incompatible_with, derived_from, local_specialization, not_equivalent) and three per-edge safety booleans: safe_for_query_expansion, safe_for_export, safe_for_logical_inference. Closure expansion respects safety flags.

I8. Identity is a hypothesis, not a foreign key. Person, place, language, lexeme, morpheme, source, specimen, case, and concept identity may all be contested. The system stores identity hypotheses (eight kinds: same_as, different_from, broader_than, narrower_than, split_candidate, merge_candidate, successor_of, alias_of) and lets users query under selected identity lenses (strict, likely, exploratory, custom).

I9. Adapters must report information loss. Every import and export adapter produces a LossReport that explicitly names what the source format cannot represent (governance, contradiction, time, n-ary frames, anchors, review state).

I10. A release is a reproducible view. A release is a named query plus a policy report, source manifest, transformation manifest, checksum manifest, and reproducibility contract.

4.2 The maturity ladder

donto tracks a six-level epistemic ladder per claim:

Promotion is monotonic per claim event. A claim may be superseded or retracted; the maturity history remains queryable. The flags smallint on donto_statement packs polarity into bits 0–1 and maturity into bits 2–4 (migration 0002), with the stored 4 → "E5 Certified" / stored 5 → "E4 Corroborated" non-monotone detail explicitly documented in migration 0102 so the ordering helpers know about it.

4.3 Polarity, modality, and confidence

Three orthogonal axes describe the epistemic shape of a claim.

Polarity is one of asserted (default), negated (explicit rejection — "X is not Y"), absent (the source explicitly does not mention this), or unknown (the source mentions but is not clear). It is packed into the flags smallint and queryable via DontoQL's POLARITY clause.

Modality is a sparse overlay (donto_statement_modality, migration 0099) with fifteen values: descriptive, prescriptive, reconstructed, inferred, elicited, corpus_observed, typological_summary, experimental_result, clinical_observation, legal_holding, archival_metadata, oral_history, community_protocol, model_output, other. Statements without a modality row are present in the system but filtered out of modality-restricted queries.

Confidence is stored as up to four parallel values (donto_confidence, migration 0101): machine_confidence (model-reported), calibrated_confidence (empirically calibrated against reviewer decisions), human_confidence (reviewer-reported), and source_reliability_weight (source/method-level). Queries may select a confidence lens; the system does not collapse to a scalar by default.

4.4 Contexts

Every statement is filed under exactly one context (the donto_statement.context column references donto_context.iri). Contexts form a forest (migration 0001) with parent links; multiple parents are supported for secondary attachments via donto_statement_context (migration 0103). A context has a kind (source, snapshot, hypothesis, user, pipeline, trust, derivation, quarantine, custom, system) and a mode (permissive or curated). Curated contexts route shape violations to a quarantine context rather than rejecting them outright; permissive contexts accept and emit a proof obligation.

The default context for any assert call that does not specify one is donto:anonymous. There is no nullable context column anywhere in the schema.

4.5 Bitemporality

donto_statement.valid_time is a daterange capturing world-time applicability — when the claim was true in the world. tx_time is a tstzrange capturing system-time belief — when the row was asserted. Both lower-inclusive ('[)'); lower_inc(tx_time) is a table-level CHECK constraint.

Retraction (donto_retract) closes the upper bound of tx_time without creating a new row; the statement transitions from "currently believed" to "was once believed". Correction (donto_correct) retracts the old row and inserts a new one referencing the prior via lineage (donto_stmt_lineage). Both operations emit donto_audit log rows.

The discipline extends through every mutating object via donto_event_log (migration 0090) — alignments, identity edges, policies, attestations, reviews, releases. The single rule the substrate enforces: never DELETE FROM donto_statement. The CLAUDE.md non-negotiable list states this in capitals.

5.1 The core: donto_statement

-- packages/sql/migrations/0001_core.sql:43-96
create table if not exists donto_statement (
    statement_id  uuid primary key default gen_random_uuid(),
    subject       text not null,
    predicate     text not null,
    object_iri    text,
    object_lit    jsonb,    -- {"v": <value>, "dt": <datatype-iri>, "lang": <tag-or-null>}
    context       text not null references donto_context(iri),
    tx_time       tstzrange not null default tstzrange(now(), null, '[)'),
    valid_time    daterange not null default daterange(null, null, '[)'),
    flags         smallint not null default 0,
    content_hash  bytea generated always as (digest(...)) stored,
    constraint donto_statement_object_one_of
        check ((object_iri is not null) <> (object_lit is not null)),
    constraint donto_statement_tx_lower_inc check (lower_inc(tx_time))
);

create unique index if not exists donto_statement_open_content_uniq
    on donto_statement (content_hash) where upper(tx_time) is null;

create index if not exists donto_statement_spo_idx
    on donto_statement (subject, predicate, object_iri);
create index if not exists donto_statement_pos_idx
    on donto_statement (predicate, object_iri, subject);
create index if not exists donto_statement_osp_idx
    on donto_statement (object_iri, subject, predicate)
    where object_iri is not null;
create index if not exists donto_statement_valid_time_idx
    on donto_statement using gist (valid_time);
create index if not exists donto_statement_tx_time_idx
    on donto_statement using gist (tx_time);
create index if not exists donto_statement_object_lit_gin
    on donto_statement using gin (object_lit jsonb_path_ops)
    where object_lit is not null;

Three indexes cover the standard SPO/POS/OSP join orders. GiST indexes handle bitemporal range queries. Literal-object substring search rides the trigram index added in migration 0131 (object_iri_trgm). Idempotence on assert is enforced by a partial unique index on (content_hash) where upper(tx_time) is null: only currently-believed rows must be unique by content. A retracted row with the same content can later be re-asserted as a separate row.

5.2 Flag packing

-- packages/sql/migrations/0002_flags.sql
create or replace function donto_pack_flags(polarity text, maturity int)
returns smallint language sql immutable as $$
    select (
        (case lower(polarity)
            when 'asserted' then 0 when 'negated'  then 1
            when 'absent'   then 2 when 'unknown'  then 3 else null end)
        | ((maturity & 7) << 2)
    )::smallint
$$;

Bits 0–1 are polarity, bits 2–4 are maturity, bits 5–15 are reserved. The function is IMMUTABLE and PARALLEL SAFE; a Rust mirror exists in pg_donto/src/lib.rs:211-263 for plan-quality hot paths.

5.3 Contexts

-- packages/sql/migrations/0001_core.sql:17-38
create table if not exists donto_context (
    iri          text primary key,
    kind         text not null check (kind in (
        'source','snapshot','hypothesis','user','pipeline',
        'trust','derivation','quarantine','custom','system')),
    parent       text references donto_context(iri),
    label        text,
    metadata     jsonb not null default '{}'::jsonb,
    mode         text not null default 'permissive'
                 check (mode in ('permissive','curated')),
    created_at   timestamptz not null default now(),
    closed_at    timestamptz,
    constraint donto_context_no_self_parent
        check (parent is distinct from iri)
);

donto_resolve_scope (migration 0003) walks the context forest either downward (default: include descendants) or upward (optional: include ancestors), with set-based include/exclude lists.

5.4 Evidence and arguments

donto_document (migration 0023) registers a source artefact with a required policy_id (post-0123). donto_document_revision holds content-addressed revision bodies (the blob backend stores the actual bytes; the revision holds the metadata and optionally inlines short bodies). donto_span carries char-offset spans over revisions.

-- packages/sql/migrations/0029_evidence_links.sql:11-59
create table if not exists donto_evidence_link (
    link_id              uuid primary key default gen_random_uuid(),
    statement_id         uuid not null references donto_statement(statement_id),
    link_type            text not null check (link_type in (
        'extracted_from', 'supported_by', 'contradicted_by',
        'derived_from', 'cited_in', 'anchored_at', 'produced_by'
    )),
    target_document_id   uuid references donto_document(document_id),
    target_revision_id   uuid references donto_document_revision(revision_id),
    target_span_id       uuid references donto_span(span_id),
    target_annotation_id uuid references donto_annotation(annotation_id),
    target_run_id        uuid references donto_extraction_run(run_id),
    target_statement_id  uuid references donto_statement(statement_id),
    confidence           double precision,
    context              text references donto_context(iri),
    tx_time              tstzrange not null default tstzrange(now(), null, '[)'),
    metadata             jsonb not null default '{}'::jsonb,
    created_at           timestamptz not null default now(),
    constraint donto_evidence_link_has_target check (
        (target_document_id is not null)::int +
        (target_revision_id is not null)::int +
        (target_span_id is not null)::int +
        (target_annotation_id is not null)::int +
        (target_run_id is not null)::int +
        (target_statement_id is not null)::int = 1
    ),
    constraint donto_evidence_link_tx_lower_inc check (lower_inc(tx_time))
);

donto_argument (migration 0031) holds typed support / attack relations between statements (nine relations including supports, rebuts, undercuts, qualifies, endorses, supersedes, potentially_same, same_referent, same_event) with a strength on [0, 1] and an open-edge unique index that lets the same pair of statements be related differently in different contexts.

5.5 Identity layer

donto_entity_symbol (migration 0057) records every freely-minted IRI a model or import produces, with a trigram index on the normalised label for blocking. donto_identity_edge (migration 0060) records weighted, bitemporal coreference assertions with one of four relations: same_referent, possibly_same_referent, distinct_referent, not_enough_information. The edge table constraints left_symbol_id < right_symbol_id to avoid double- representation.

donto_identity_hypothesis names a clustering solution over the edges (e.g., the strict_identity_v1 hypothesis only takes edges with confidence ≥ 0.98 and no cannot-link).

5.6 Predicate alignment

-- packages/sql/migrations/0048_predicate_alignment.sql
create table if not exists donto_predicate_alignment (
    alignment_id     uuid primary key default gen_random_uuid(),
    source_iri       text not null,
    target_iri       text not null,
    relation         text not null check (relation in (
        'exact_equivalent', 'inverse_equivalent',
        'sub_property_of', 'close_match',
        'decomposition', 'not_equivalent'
    )),
    confidence       double precision not null default 1.0
                     check (confidence >= 0 and confidence <= 1),
    valid_time       daterange not null default daterange(null, null, '[)'),
    tx_time          tstzrange not null default tstzrange(now(), null, '[)'),
    run_id           uuid,
    provenance       jsonb not null default '{}'::jsonb,
    registered_by    text,
    registered_at    timestamptz not null default now(),
    constraint donto_pa_distinct check (source_iri <> target_iri),
    constraint donto_pa_tx_lower_inc check (lower_inc(tx_time))
);

A materialised closure table (donto_predicate_closure) pre-computes transitive chains so the evaluator can ride alignments at query time without a recursive CTE per row. The closure is rebuilt by donto_rebuild_predicate_closure().

5.7 Trust kernel

donto_policy_capsule (migration 0111) holds a policy with fifteen boolean allowed-actions (read_metadata, read_content, quote, view_anchor_location, derive_claims, derive_embeddings, translate, summarize, export_claims, export_sources, export_anchors, train_model, publish_release, share_with_third_party, federated_query).

donto_access_assignment maps targets (document, revision, span, context, statement, frame, release, entity, predicate) to policies.

donto_attestation (migration 0112) records credentials: a holder agent, an issuer agent, a policy IRI, a subset of allowed actions, a purpose (review, community_curation, private_research, publication, model_training, audit, extraction, federation, inspection), a required rationale, and lifecycle fields (issued_at, expires_at, revoked_at).

The top-level access check donto_authorise(holder, target_kind, target_id, action) combines policy effective-action AND with attestation OR semantics. The effective-action helper donto_effective_actions does a bool_and over all assigned policies for the target, defaulting to the fail-closed restricted policy when none is assigned.

5.8 Substrate scale and storage

The live database holds:

The retraction rate is 281 / 39,293,802 ≈ 7.1 × 10⁻⁶ — see the empirical discussion in §13.

The polarity distribution is heavily skewed toward asserted:

The maturity distribution shows the cap-at-E1-or-below pattern of machine extraction:

*The flags-bit encoding stores 4 = E5 Certified and 5 = E4 Corroborated (migration 0102), reflecting a historical naming ambiguity preserved for backward compatibility.

6.1 Grammar

query           := keyword_clause+
keyword_clause  :=
    'SCOPE'           scope_descriptor
  | 'PRESET'          IDENT_or_PREFIXED_or_STRING
  | 'MATCH'           triple (',' triple)*
  | 'FILTER'          filter_expr (',' filter_expr)*
  | 'POLARITY'        ident_in_set
  | 'MATURITY'        '>='? INT
  | 'IDENTITY'        ident
  | 'IDENTITY_LENS'   ident
  | 'PREDICATES'      ('EXPAND' | 'STRICT' | 'EXPAND_ABOVE' INT)
  | 'MODALITY'        ident (',' ident)*
  | 'EXTRACTION_LEVEL' ident (',' ident)*
  | 'TRANSACTION_TIME' 'AS_OF' STRING_or_PREFIXED
  | 'AS_OF'           STRING_or_PREFIXED
  | 'POLICY' 'ALLOWS' ident
  | 'SCHEMA_LENS'     (iri | ident)
  | 'EXPANDS_FROM'    'concept' '(' iri ')' 'USING' 'schema_lens' '(' iri ')'
  | 'ORDER_BY'        ident ('DESC'|'ASC')?
  | 'WITH'            'evidence' '=' ident
  | 'PROJECT'         var (',' var)*
  | 'LIMIT'           INT
  | 'OFFSET'          INT

triple   := term term term ('IN' term)?
term     := var | iri | string-lit | int-lit
var      := '?' IDENT
iri      := '<' chars '>' | PREFIXED
filter_expr := term op term         -- op ∈ { = != < <= > >= }

Clauses may appear in any order. Whitespace is insignificant; # introduces an end-of-line comment.

6.2 Defaults

Without any clauses, MATCH ?s ?p ?o returns asserted, currently- believed rows across every context, at any maturity, with the identity lens default (no expansion across sameAs clusters), predicate expansion expand (rides the alignment closure), and no ordering. The choice to default to asserted-only prevents contradictions from leaking into queries without explicit request.

6.3 Worked examples

Contested birth claims as of a system time. Find every claim about Annie Davis's birth that disagrees with another claim, in research contexts where the maturity is at least E2 (evidence- supported), as of the state of the store on 2026-04-01:

SCOPE include ctx:genes/annie-davis ancestors
PRESET curated
MATCH ?stmt ex:about ex:annie-davis,
      ?stmt ex:predicate ex:born_in,
      ?stmt ex:object    ?place
FILTER ?place != "unknown"
POLARITY asserted
TRANSACTION_TIME AS_OF "2026-04-01T00:00:00Z"
PREDICATES EXPAND_ABOVE 75
PROJECT ?stmt, ?place
LIMIT 50

Release-safe claims. Show only claims under a hypothesis context that the policy permits for publication, with evidence redacted where required:

SCOPE include ctx:project:language-pilot ancestors
MATCH ?stmt ?p ?o
MATURITY >= 3
POLICY ALLOWS publish_release
WITH evidence = redacted_if_required

Cluster-expansion search. Same query, broaden across identity clusters and alignment expansion:

MATCH ?s ex:bornInPlace ?city
IDENTITY_LENS expand_clusters
PREDICATES EXPAND_ABOVE 70
LIMIT 100

Contradiction frontier. Order results by how contested each binding's leading statement is:

MATCH ?stmt ex:about ?subject,
      ?stmt ex:predicate ?pred
ORDER BY contradiction_pressure DESC
PROJECT ?stmt, ?subject, ?pred
LIMIT 50

contradiction_pressure is attack_count − support_count from donto_contradiction_frontier, joined against the binding's most recent matched statement_id. Rows without any argument edges sort to pressure = 0.

As-of historical view. Reconstruct what was known last week:

MATCH ?p ex:about ex:somebody, ?p ?pred ?val
AS_OF "2026-05-21T00:00:00Z"
LIMIT 100

6.4 SPARQL subset

The SPARQL parser (packages/donto-query/src/sparql.rs) accepts PREFIX, SELECT (including SELECT *), WHERE, GRAPH, basic FILTER (numeric and string comparison), LIMIT, and OFFSET. Property paths, OPTIONAL, UNION, aggregates (COUNT, SUM, GROUP BY), and mutation (INSERT DATA, DELETE DATA) are deliberately out of scope. donto's mutating operations are expressed through donto_assert / donto_retract / donto_correct at the SQL layer; the query language is read-only.

6.5 Evaluator

The Phase-4 evaluator is a nested-loop join with variable unification:

// packages/donto-query/src/evaluator.rs

for pattern in &query.patterns {
    for binding in &env {
        let substituted = substitute(pattern, binding);
        let rows = client.match_pattern(
            substituted.subject_iri,
            substituted.predicate_iri,
            substituted.object_iri,
            scope, polarity, min_maturity,
            as_of_tx, as_of_valid,
        ).await?;
        for row in rows {
            if let Some(extended) = unify(binding, pattern, &row) {
                next_env.push(extended);
            }
        }
    }
    env = next_env;
}

apply_filters(&mut env, &query.filters);
apply_overlays(&mut env, query.modality, query.extraction_level);
apply_policy_gate(&mut env, query.policy_allows);
apply_order_by(&mut env, query.order_by);
apply_offset_limit(&mut env, query.offset, query.limit);
attach_evidence(&mut env, query.evidence_shape);

This is correct but unoptimised: the query planner is a deliberately- deferred Phase-10 work item. The evaluator's HTTP entry point at POST /dontoql handles ~50 ms p50 for simple patterns at the current scale.

7.1 The six apertures

7.2 Yield curve

A 1,376-character biographical text run through both pipelines gave:

1-pass tier        :   95 facts at $0.0042 in   65 s
6-pass aperture    :  341 facts at $0.0252 in  449 s    (3.6× yield, 6× cost)
  surface          :   87
  linguistic       :  127
  presupposition   :   34
  inferential      :   12
  conceivable      :   54
  recursive        :   27
distinct predicates:  171
distinct subjects  :   70
anchor coverage    :    0.842
hypothesis density :    0.258
dedup collisions   :    6

The yield is a floor, not a target. The Maximalism doc characterises ambitions to push toward ~20,000–30,000 facts per source at fuller aperture coverage, gated by cost (target ≈ 0.02–0.05 per 1 kB of source for the full 15-aperture pass, $0.001/kB on cache hits).

7.3 The conceivable aperture and quarantine

The conceivable aperture is the most controversial. It deliberately floods the candidate space with unanchored, hypothesis-only claims (persons have hair, organisations have employees, projects have contributors). The position is explicit:

Maximal extraction is a design stance, not a yield target. Mine everything. Quarantine the malformed. Flag the hypothetical. Let the curation gate, not the extractor, decide what counts. (EXTRACTION-MAXIMALISM L311-314)

Quarantine is implemented as a sink that routes invalid candidates to a ctx:quarantine/<source> context with policy restricted_pending_review. Conceivable-aperture output lands in the candidate space at maturity E1 with hypothesis_only=true; downstream curation (Trust Kernel policy gate, maturity ladder, reviewer acceptance) decides what survives into a release.

7.4 Orchestration

The Python implementation in apps/donto-api/extraction/ is laid out as:

apertures.py     -- six aperture prompts (Surface, Linguistic, ...)
exhaustive.py    -- multi-pass orchestrator (asyncio.gather + dedup)
dispatch.py      -- single-pass M5 path (still useful for cheap runs)
validation.py    -- hard-gate validator (anchor + hypothesis_only invariant)
quarantine.py    -- quarantine sink
policy_gate.py   -- Trust Kernel probe before any external model call
main.py::extract_exhaustive -- POST /extract/exhaustive

The exhaustive orchestrator (exhaustive.py:138-244) gathers the non-recursive apertures in parallel via asyncio.gather, dedups in-flight using a content-key hash, then seeds Recursive from the top 12 most-frequent subjects and object IRIs in the union. Vocabulary-aware prompting (vocab.py) injects the current top-80 predicates and entity candidates into the system prompt so the model reuses existing IRIs rather than minting fresh ones. We discuss the empirical fallout of this prompt in §13.

7.5 Confidence and maturity

helpers.py:54-59 maps model confidence to maturity ceiling:

def confidence_to_maturity(c: float) -> int:
    if c >= 0.95: return 4
    if c >= 0.80: return 3
    if c >= 0.60: return 2
    if c >= 0.40: return 1
    return 0

This is a ceiling, not a floor: the post-PRD policy caps extraction- produced claims at E1 regardless of model confidence. The PRD I5 invariant is enforced by an ingest validator that drops high-confidence model claims back to E1 unless a reviewer attestation is present in the same transaction.

7.6 Temporal workflows

Long-running extraction jobs are submitted as Temporal workflows (apps/donto-api/workflows.py). The four-stage pipeline extracting → ingesting → aligning → resolving → completed is implemented as activities with explicit retry policies (30-minute exhaustive extract activity timeout, 3 retries; 5-minute ingest timeout, 5 retries; alignment and resolution similarly bounded). The workflow is queryable mid-flight via the status method.

8.1 The capsule model

A donto_policy_capsule (migration 0111) is one of nine policy kinds (public, open_metadata_restricted_content, community_restricted, embargoed, licensed, private, regulated, sealed, unknown_restricted) with a JSONB allowed-actions object covering fifteen actions. An inheritance rule (max_restriction is the default) defines how a derived row combines the policies of its source anchors:

allowed_actions = jsonb_build_object(
    'read_metadata',       false,
    'read_content',        false,
    'quote',               false,
    'view_anchor_location',false,
    'derive_claims',       false,
    'derive_embeddings',   false,
    'translate',           false,
    'summarize',           false,
    'export_claims',       false,
    'export_sources',      false,
    'export_anchors',      false,
    'train_model',         false,
    'publish_release',     false,
    'share_with_third_party', false,
    'federated_query',     false
)

The boilerplate default is fail-closed; a public-by-default classification must be explicit.

8.2 Effective action computation

The donto_effective_actions(target_kind, target_id) function returns the AND across all policies assigned to the target via donto_access_assignment. With no assignment, it returns the fail-closed default policy's allowed-actions:

foreach v_key in array v_keys loop
    select bool_and(coalesce((p.allowed_actions->>v_key)::boolean, false))
    into v_allowed
    from donto_access_assignment a
    join donto_policy_capsule   p on p.policy_iri = a.policy_iri
    where a.target_kind = p_target_kind
      and a.target_id   = p_target_id
      and p.revocation_status = 'active'
      and (p.expiry is null or p.expiry > now());
    v_result := v_result || jsonb_build_object(v_key, coalesce(v_allowed, false));
end loop;
return v_result;

Revoked, expired, and superseded policies do not contribute. The choice of bool_and over bool_or operationalises the max-restriction doctrine: every assigned policy must permit the action for it to be permitted.

8.3 Attestations

A donto_attestation is a holder credential: agent X is entitled to perform action Y under policy Z for purpose P with rationale R until expiry T. The check donto_authorise(holder, target, action) combines the effective-action AND with an attestation OR — if any attestation grants the holder the action under any policy assigned to the target, the call returns true.

Revocation is immediate for new checks (set revoked_at = now()); in- flight reads in the same transaction may still proceed but cannot read newly-revoked data.

8.4 The F-1 closure

docs/REVIEW-FINDINGS.md records eighteen findings F-1 through F-18 from an adversarial review of the Trust Kernel. Sixteen are DOC-severity (intentional, documented, no work required) and two are HARD-severity, both now resolved. The historical F-1 finding is worth recording in full because it illustrates how the substrate preserves invariants in the face of legacy code paths:

Pre-migration 0123, the legacy donto_ensure_document SQL function did not require policy_id, leaving the I2 invariant ("no source without policy") enforced only by fail-closed read defaults rather than write-time refusal.

Resolution: migration 0123 backfilled NULL policy_id with the seeded fail-closed policy:default/restricted_pending_review (allowed_actions all false), set that capsule as the column default, promoted policy_id to NOT NULL, and validated the previously-NOT-VALID foreign key. The legacy path now succeeds but the produced row has the fail-closed policy; explicit-policy callers (via donto_register_source) continue to pass policy explicitly.

Tripwire (inverted): invariants_adversarial:: legacy_register_document_lands_on_default_restricted_policy — asserts the legacy path succeeds and that the produced document has the fail-closed policy. If a future migration removes the default or the constraint, this test fails and prompts a deliberate decision.

Production state at resolution: 8,211 legacy donto_document rows backfilled to policy:default/restricted_pending_review. Read access remained fail-closed both before and after.

The pattern — backfill, then NOT NULL, then validate FK, all in one idempotent migration — is the standard donto fix for an invariant gap that has been silently tolerated.

8.5 The HTTP-middleware follow-on

The substrate side of the F-1 closure is complete. The remaining work, recorded in ROADMAP-AFTER-MAY18.md, is the HTTP middleware that enforces policy presence on write paths at the sidecar layer rather than at the SQL layer. The genes corpus has hundreds of unpoliced legacy sources (now bound to fail-closed defaults at the substrate but still ingested via the legacy path); the middleware work is the visible end-to-end testbed for the Trust Kernel beyond the database layer.

9.1 Symbols, mentions, signatures, edges, hypotheses

donto's open-world entity-resolution architecture decomposes the "who is this person?" problem into five distinct objects:

• donto_entity_symbol — a string identifier produced by extraction, legacy import, user action, or external KB import. Examples: ex:mary-watson, ex:mrs-watson, ctx:legacy/31448699f0e5.
• donto_entity_mention — an occurrence of a symbol in a document / span / extraction run.
• donto_entity_signature — the current feature vector or profile for a symbol, derived from statements and evidence.
• donto_identity_edge — a weighted, bitemporal assertion about whether two symbols co-refer.
• donto_identity_hypothesis — a named clustering solution over identity edges.

Three default hypotheses ship:

strict_identity_v1       : same_referent edges with confidence >= 0.98 only
likely_identity_v1       : >= 0.85, no strong cannot-link
exploratory_identity_v1  : >= 0.60, useful for search and discovery

Production queries can be asked under any lens. A merge accepted under likely does not destroy the original symbols; under strict, the same two symbols may still resolve to distinct entities.

9.2 Candidate generation: blocking

At 19,230 contexts and 938,918 distinct predicates, naive O(N²) pairwise comparison of subjects is infeasible. The ARCHITECTURE-REPORT proposes (and we are landing in phases) nine independent blocking channels for persons:

B1 : normalised full-name trigram
B2 : surname + given initial
B3 : maiden-name / married-name variants
B4 : birth-year bucket ±N years
B5 : birth/death/residence place
B6 : spouse/parent/child neighbourhood overlap
B7 : document/source-local co-occurrence
B8 : embedding nearest neighbours over compact entity profiles
B9 : legacy imported identifier / external ID / source record ID

Trigram (pg_trgm) indexes provide cheap B1–B3 lookups; the embedding channel (B8) is staged for the pgvector-backed predicate-similarity work but the entity dimension is later.

9.3 Pairwise scoring

The scoring formula, generalising Fellegi–Sunter:

LLR(i,j) = log prior_block(i,j)
         + Σ_k log P(feature_k | same) / P(feature_k | different)
         + λ_rel    * relational_overlap(i,j)
         + λ_time   * temporal_compatibility(i,j)
         + λ_place  * spatial_compatibility(i,j)
         + λ_neural * neural_pair_score(i,j)
         + λ_source * source_dependence_adjustment(i,j)

p_same   = sigmoid(a * LLR + b)

Ditto-style transformer matchers [Li et al., 2020] inform the neural-pair component for hard pairs only; running a deep matcher over all candidate pairs would be economically prohibitive.

9.4 Clustering: the reversibility principle

Naive transitive closure on positive identity edges produces catastrophic merges in genealogy: Mary Watson ≈ Mrs Watson, Mrs Watson ≈ Mary Watson née Oxley, Mary Watson née Oxley ≈ Mary Oxley, Mary Oxley ≈ another Mary Oxley, and suddenly the nineteenth-century Cooktown beche-de-mer fisherman's widow is the same person as a twentieth-century Sydney accountant.

The constrained clustering algorithm we are landing:

Input
  positive edges with weights
  negative / cannot-link edges with weights
  soft ontology and temporal constraints
  source-dependence penalties

Output
  clusters maximising same-edge agreement
  while minimising distinct-edge violations

Implemented in phases: union only p_same ≥ 0.98 and no strong cannot-link; local correlation-clustering-style optimisation inside candidate blocks; produce ambiguous bridge edges as proof obligations, not automatic merges; human-or-agent review only on high-impact ambiguous bridges.

The non-negotiable principle is reversibility. A merge accepted at time t and reversed at time t' > t must leave the database queryable as-of any t″ ∈ [t, t'] with the merged view, and as-of any t″ ∉ [t, t'] with the unmerged view. The substrate enforces this by leaving original symbols in donto_statement untouched; query projection maps symbol → referent under the requested hypothesis at evaluation time, never at write time.

9.5 Predicate alignment

The 938,918 distinct predicates in the live database are not a sign of expressive richness; they are a sign of LLM-driven predicate proliferation. Each freely-minted predicate (ex:fatherOf, ex:isFatherOf, ex:hadFather, ex:was-father-of, ...) is a row in donto_predicate and a leaf in the alignment graph.

The alignment layer's job is to make queries portable across this proliferation. An alignment edge declares one of eleven relations (PRD §6.10) and three per-edge safety booleans:

safe_for_query_expansion : evaluator may ride this edge in EXPAND mode
safe_for_export          : downstream exporters may collapse via this edge
safe_for_logical_inference : OWL-style entailment may use this edge

A user can register an alignment between ex:fatherOf and ex:hadFather with relation inverse_equivalent, safety flag safe_for_query_expansion=true, scope = ctx:genes/registry. The materialised closure (donto_predicate_closure) pre-computes transitive chains the evaluator can ride at query time. donto align auto proposes alignments via embedding-similarity over predicate descriptors; auto-proposals land at confidence < 1.0 and are reviewer-promotable.

9.6 Source provenance: the three-tier trace

A claim with surface text "Mary Watson, born Cornwall 1860" should resolve to a donto_span row at byte offsets [1342, 1376) of revision rev_92f1... of document doc:genes/trove-cooktown/watkins-1881. The packages/donto-trace/ crate implements this resolution as a three-tier search.

For each statement carrying a donto:textSpan or ex:normalized_claims/text_span predicate, the trace worker:

1. Cache check (cross-shard). Hash the surface text; look up donto_trace_log for any prior match (across all runs, not just this run). If found, reuse the prior result. Production data shows ~4.3× dedup: 1.5 M textSpan statements → 349 K distinct surface texts.

2. Tier 1 — exact line equality. Query donto_revision_line where line_text = needle and length(line_text) ≤ 400. A partial B-tree index on short lines makes this O(log n).

3. Tier 2 — substring within a line. Query donto_revision_line with line_text LIKE pattern ESCAPE '\'. Uses a trigram GIN index on line_text for substring search. Cheaper than Tier 3 because the per-row haystack is small.

4. Tier 3 — full-body fallback. Query donto_document_revision.body_inline with the same LIKE pattern. Slow but catches multi-line quotes. Only runs if the surface text contains a newline.

The four match types:

Exact       : surface text matches verbatim       confidence = 1.0
Normalized  : whitespace-collapsed match           confidence = 0.9
Ambiguous   : same text in >1 revision             confidence = 0.5
NotFound    : no match found                       confidence = 0.0

Writes are strictly additive. A successful match emits four rows:

• a donto_span (via donto_create_char_span),
• a donto:hasSpan claim linking the original statement to the span,
• a donto_evidence_link with type anchored_at and the match confidence,
• a donto_trace_log row for resumability.

The legacy donto:textSpan literal is never retracted. The new donto:hasSpan claim coexists with it.

Resumability: the worklist query (packages/donto-trace/src/lib.rs: 336-369) skips statements already in donto_trace_log for the given run_name. SIGINT sets an interrupted flag the main loop checks at batch boundaries; the current batch completes cleanly and the process exits. A re-run with the same run_name continues from where it left off.

9.7 The content-addressed blob store

The blob store (packages/donto-blob/, migration 0125) is content- addressed by SHA-256. Three backends are provided (LocalFsBlobStore, GcsBlobStore, MockBlobStore) behind a common trait:

pub trait BlobStore: Send + Sync {
    fn backend(&self) -> &'static str;
    async fn put_bytes(&self, bytes: &[u8], mime: Option<&str>)
        -> Result<BlobSummary>;
    async fn put_file(&self, path: &Path, mime: Option<&str>)
        -> Result<BlobSummary>;
    async fn exists(&self, sha256: &[u8; 32]) -> Result<bool>;
    async fn fetch(&self, sha256: &[u8; 32]) -> Result<Vec<u8>>;
    fn uri_for(&self, sha256: &[u8; 32]) -> String;
}

put_file two-passes: hash first, then upload only if not already present. donto_register_blob(sha256, byte_size, mime_type, bucket_uri) records the blob in donto_blob; the revision table references it. The same revision body uploaded by ten different documents lands as one blob with ten document-revision references.

10.1 Type system

-- packages/lean/Donto/Core.lean
inductive Polarity where
  | asserted | negated | absent | unknown
  deriving Repr, BEq, DecidableEq

inductive Modality where
  | observed | derived | hypothesized | retracted
  deriving Repr, BEq, DecidableEq

inductive Confidence where
  | uncertified | speculative | moderate | strong
  deriving Repr, BEq, DecidableEq

structure Maturity where
  level : Nat
  hLE   : level ≤ 4

structure Statement where
  id         : Option String
  subject    : String
  predicate  : String
  object     : Object
  context    : String
  polarity   : Polarity
  modality   : Modality
  confidence : Confidence
  maturity   : Maturity
  valid_from : Option String
  valid_to   : Option String

The Lean types mirror the Postgres schema. The structural isomorphism is what makes the DIR encoding straightforward: a Lean Statement decodes from the same JSON shape that donto-client serialises.

10.2 Shape combinators

-- packages/lean/Donto/Shapes.lean
structure Shape where
  iri      : String
  label    : String
  severity : Severity
  evaluate : List Statement → ShapeReport

Built-in shapes ship with the library:

• builtin:functional/* — at-most-one object per (subject, predicate) pair.
• builtin:datatype/* — typed-literal datatype enforcement.
• builtin:parent-child-age-gap — genealogy domain shape: ex:parentOf edges require parents 12–80 years older than children, compared via ex:birthYear. Missing or unreasonable gaps produce violation rows.

Each shape is a pure predicate over a scoped statement list: it does not touch the database directly. dontosrv ships a focus selector — a scoped pattern that materialises the statement list — along with the shape IRI; Lean returns a ShapeReport with violation rows.

10.3 Rules

-- packages/lean/Donto/Rules.lean
structure Rule where
  iri    : String
  label  : String
  output : Context
  mode   : RuleMode  -- eager | batch | onDemand
  apply  : List Statement → List Statement

transitiveClosure p emits all (a, p+, c) from (a, p, b) and (b, p, c), in the output context, at modality derived and maturity 3 (E3 by the storage encoding, which maps to "reviewed" in the maturity-ladder doc; rules are formal certifications and thus considered as-if-reviewed). Inverse and symmetric rules emit bidirectional pairs.

10.4 The protocol

donto_engine reads lines from stdin; for each line it parses a JSON envelope, dispatches via Donto.Engine.dispatch, and emits a JSON response. Failures (parse errors, unknown shape IRIs) produce explicit error envelopes:

request:
{
  "version"   : "0.1.0-json",
  "kind"      : "validate_request",
  "shape_iri" : "lean:builtin/parent-child-age-gap",
  "scope"     : {...},
  "statements": [...]
}

response:
{
  "version"     : "0.1.0-json",
  "kind"        : "validate_response",
  "shape_iri"   : "...",
  "focus_count" : N,
  "violations"  : [...]
}

10.5 The certifies-not-gates invariant

dontosrv spawns the Lean engine with a 10-second startup banner timeout and enforces a 30-second per-request timeout. On engine death or unresponsiveness the parent closes the child and returns sidecar_unavailable for subsequent shape/rule calls; the rest of dontosrv continues to serve ingest, query, and policy traffic without interruption.

This is the principle: Lean certifies, doesn't gate. Ingest never waits on the formal overlay. The sidecar's absence degrades shape/rule/cert calls only.

11.1 The release contract

A release (PRD §17.1) is eleven things:

1. Release query.
2. Claim set.
3. Evidence manifest.
4. Source manifest.
5. Policy report.
6. Transformation report.
7. Adapter loss report.
8. Checksum manifest.
9. Citation metadata.
10. Reproduction instructions.
11. Optional export packages.

packages/donto-release/ builds these from a ReleaseSpec JSON. Release blockers include any included claim with a policy disallowing publication, any source with unresolved policy, any restricted-anchor reference without redaction, any claim below the release-maturity threshold, any adapter loss report with unaccepted critical loss, and any required review that has not occurred.

11.2 Native and optional exports

The native format is donto-release.jsonl (one statement per line, with checksums) plus manifest.json. Optional exports include RO-Crate (a BagIt-compatible JSON-LD metadata file), CLDF (for language datasets — lossy, requires --max-cldf-loss gate), CoNLL-U (for corpus releases), CSV/TSV for tabular subsets, and RDF / JSON-LD for linked-data consumers.

11.3 Ed25519 signing

// packages/donto-release/src/envelope.rs
pub struct Keypair { signing: ed25519_dalek::SigningKey, /* ... */ }

impl Keypair {
    pub fn generate() -> Self { /* OsRng */ }
    pub fn from_seed(seed: [u8; 32]) -> Self { /* deterministic */ }
    pub fn did_key(&self) -> String {
        // multicodec [0xed, 0x01] + base32(verifying_key)
    }
}

pub struct ReleaseEnvelope {
    pub manifest_id     : String,
    pub manifest_sha256 : String,
    pub issuer_did      : String,
    pub signature_suite : String,   // "Ed25519Signature2020"
    pub signature       : String,
    pub created_at      : String,
}

pub fn sign(manifest: &ReleaseManifest, kp: &Keypair) -> ReleaseEnvelope { /* … */ }
pub fn verify(env: &ReleaseEnvelope) -> Result<(), VerifyError> { /* … */ }

The CLI subcommand donto release pipeline orchestrates the full five-stage emission: build manifest from spec → write native JSONL → sign → write RO-Crate metadata → optional CLDF export. Verification requires no network: did:key is self-contained.

11.4 Federation: an evaluation

docs/M9-FEDERATION-MEMO.md records an evaluation of five candidate federation stacks. The explicit non-goal is "any researcher querying any other researcher's tree". The actual question is whether instance B can verify a release manifest from instance A without re-ingesting A's source content.

W3C Verifiable Credentials + DID — Verdict: strongest fit. Each manifest is a signed VC; the signer is identified by a DID. Selective disclosure (BBS+, SD-JWT) hides claim payloads when policy demands. Cost: BBS+ requires pairing-friendly curves not in standard Postgres/Rust crypto stacks (~weeks of implementation).

Solid Pods — Verdict: defer. The pod model assumes one principal per pod. donto already has multi-context, multi-authority semantics that the pod model doesn't represent natively.

SPARQL federation (SERVICE) — Verdict: reject as primary layer. Information leakage through query shape and count is a known risk; donto's PRD invariant "cross-instance restricted content cannot leak through counts or errors" rules out the naive implementation.

DataCite-style citation metadata — Verdict: proceed. The ReleaseManifest is essentially what DataCite expects. The federation piece is publishing the manifest to a registry (DataCite, Zenodo, OpenAIRE) and adopting their identifier scheme. Compatible with VC/DID for trust: the manifest is a VC; publication registers its reference.

RO-Crate — Verdict: proceed independently. RO-Crate is a format, not a federation protocol. Pairs naturally with VC/DID (sign the crate's metadata) and DataCite (publish the signed crate's identifier).

The synthesis (M9-MEMO §4):

Manifest format          : RO-Crate (M7 work, landed)
Signing layer            : VC over the manifest (M9 spike, did:key)
Publishing               : DataCite-style citation metadata
Live cross-instance query: explicit non-goal for v1

Acceptance: instance A builds release R; the output is an RO-Crate signed by a VC issued under A's DID; instance B fetches the crate, verifies the VC, reads the release manifest, and can answer "does this crate contain claims about entity X" without ever storing A's raw rows.

12.1 Benchmark hardware

GCE e2-standard-4 (4 vCPU, 16 GB RAM), PostgreSQL 16 in the donto-pg Docker container with volume bind to /mnt/donto-data/pgdata (SSD). Workload is the synthetic fixture in donto-cli bench: write N rows under a throwaway context, time one point query and one batch query.

12.2 Insert throughput and point-query latency (H1)

Insert throughput is essentially flat at ~2.5–3.0 K row/s through three decades of scale. Linear extrapolation gives ~70 min for a 10 M-row cold ingest (the H10 hard target); the genes prod corpus (39 M statements) would be ~4.2 hours cold, though production ingest is faster through batched pipelines than the single-row CLI path.

Point queries stay sub-100 ms through 1 M rows on the SPO / POS / OSP indexes. PRD §25 H1 hard target is 100 ms at 10 M; the trend is on track.

12.3 Extended benchmark suite (H2–H9)

Observations.

• Batch (full-context-scan) queries (H4) grow linearly with N as expected; the planner does not optimise unbound predicate-and- subject patterns.
• The aligned-query path (H2) is faster than the unaligned path at 100 K and 1 M because the closure-rebuild work moved the alignment join to a hot in-memory cache.
• The AS_OF bitemporal path (H3) is essentially constant: GiST index on tx_time plus partial-index on currently-open rows.
• Contradiction frontier (H5) is bound by index lookup, not by row count: the frontier view is materialised.
• Modality filter setup (H7 setup at 1 M) takes 15 s; the filter itself runs in ~1.7 s. Adding a composite (modality, statement_id) index is a deferred tuning candidate.
• Policy allows (H8) scales modestly with N: ~107 ms at 10 K → ~812 ms at 100 K, extrapolating to ~8 s at 1 M. Fine for curated read workloads, a tuning candidate for hot paths.
• 4× concurrent writers (H9) complete in ~400 ms regardless of bench scale. Advisory-lock + unique-content-hash path doesn't contend.

12.4 Live corpus characterisation

The genes corpus, as of 2026-05-28:

Statements:                39,294,083
Distinct predicates:          938,918
Distinct contexts:             19,230
Evidence links:             1,837,151
Database size:                  48 GB
donto_statement table:          32 GB
Currently-believed:        39,293,802
Retracted:                        281

Polarity:

asserted : 39,292,908  (99.997%)
negated  :        813  ( 0.002%)
unknown  :        331
absent   :         31

Maturity (storage-bit encoding):

0  (raw)             : 22,082,198   (56.2%)
1  (candidate)       : 14,870,096   (37.8%)
2  (evidence-supp.)  :     47,905   ( 0.1%)
3  (reviewed)        :    244,806   ( 0.6%)
4  (E5 certified*)   :  2,049,078   ( 5.2%)

Top context (by statement count):

ctx:genealogy/research-db                          21,842,452
ctx:genealogy/smoketest                             4,114,991
ctx:genealogy/analysis-db                           3,852,396
ctx:genes/yeatman-knoll-coleman-banjo-gibson        1,159,340
ctx:genes/edward-herbert-chinese-stack/qld          1,011,967
ctx:genealogy/research-db/source/unknown              497,658
ctx:genealogy/resources                               328,863
ctx:genes/naa-32841845                                271,739
ctx:genes/edward-herbert-father                       192,952
ctx:genes/trove-cooktown/reynolds                     142,192
ctx:genes/trove-cooktown/beche-de-mer                 114,401

Top predicates (by statement count):

rdf:type                  3,751,377
donto:status              1,629,900
donto:aboutPredicate      1,629,842
donto:confidenceLabel     1,226,023
donto:predicate           1,224,518
donto:textSpan            1,220,903
donto:extractionModel     1,206,699
donto:objectValue         1,113,534
donto:hasSpan             1,095,921
donto:claimB              1,086,531
donto:aboutSubject        1,086,531
donto:claimA              1,086,531
ex:knownAs                1,081,364
donto:createdAt             857,137
donto:inSource              764,847

The top fifteen predicates are dominated by reified meta-statements (donto:status, donto:aboutPredicate, donto:textSpan, donto:extractionModel, donto:claimA/B) rather than domain predicates. This is a direct consequence of how the M5 extractor emitted facts: each claim was reified into ~7–10 quad rows. The exhaustive (M6) extractor does not reify in the same way; the older half of the database carries a heavy reification tail.

12.5 Real workloads

Three substantive research workloads have exercised the system in the last twelve months:

Annie Davis. Multiple genealogical sources disagree on Annie Davis's parents, birth year, and place. donto stores claims from the Davis family oral history, the Brackenridge family records, the Mareeba marriage register, three colonial-era obituaries, and the 2007 EKY native-title determination, all of which contradict each other on at least one field. The contradiction frontier view exposes the disagreements; the maturity ladder caps the reviewable claims at E3 pending family-elder review; the policy capsule for oral-history sources defaults to community_restricted.

Caroline Brown / Kaitchi. A second-generation EKY apical ancestor whose parents and grandparents are subject to active litigation in the Federal Court of Australia. donto holds 820 lines of dossier and ~80 kinship triples, of which only three carry evidence links to primary sources — a sparsity flagged for the Trust Kernel HTTP-middleware testbed (see §13).

Blucher and the Maryborough boiling-down works. A nineteenth- century Aboriginal figure mis-identified across colonial archives; the corpus distinguishes the Maryborough "King Blucher" (Maryborough boss-name, NMP worker) from the authentic apical Bujilkabu claim. The PDFs at genes.apexpots.com/pdfs/ and the verbatim primary sources at genes.apexpots.com/blucher/s/ are produced from this data.

These workloads have driven changes to the substrate: the contradiction frontier view (H5), the multi-context attachment (migration 0103, for claims that belong to both a hypothesis lens and an oral-history source), and the source-provenance trace (donto-trace, §9.6).

12.6 Long-running queries

Two queries the agent submission for this paper did not return in its time budget:

SELECT COUNT(DISTINCT subject) FROM donto_statement;
-- > 30 min, did not complete

SELECT COUNT(*) FROM (
    SELECT subject, predicate
    FROM donto_statement
    WHERE upper_inf(tx_time)
    GROUP BY subject, predicate
    HAVING COUNT(DISTINCT donto_polarity(flags)) > 1
) t;
-- > 5 min, did not complete

Subject cardinality and contradiction-by-polarity are both load-bearing characterisation queries we would like to report empirically. That they do not complete in routine time is itself a finding: a donto_subject_stats matview (companion to the existing donto_subject_count matview that powers the /subjects/all directory page) is the natural fix and is the top item on the scheduled-refresh roadmap.

13.1 What worked

Append-only discipline. The decision to never DELETE FROM donto_statement and to extend the same discipline to alignments, identity, policies, and attestations through donto_event_log has not produced an unmanageable storage explosion. At 39.3 M rows and only 281 retractions, the bitemporal overhead is essentially nil. The discipline pays for itself the first time you need to answer "what did we believe last Tuesday?"

Idempotent migrations. All 131 migrations are idempotent (if not exists, create or replace, advisory-locked migrator, SHA-256 ledger). New migrations are added by sequential number; prior migrations are never edited. The model means a fresh database can be brought up to head with one donto migrate call, and a production database can absorb new migrations without service interruption (the F-1 closure ran in production with zero downtime).

The pgrx packaging. The pgrx extension pg_donto embeds every migration via extension_sql_file! and provides Rust mirrors for plan-quality immutable helpers. The cost was non-trivial (managing pgrx version skew, learning the Rust-Postgres ABI) but the result is a single CREATE EXTENSION pg_donto on a fresh database produces a working substrate. This is what packaging looks like when you take Postgres seriously as the boundary.

The tripwire test suite. 77 files, ~20 K LOC, ~592 #[tokio::test] and ~91 #[test] annotations, with the pg_or_skip! macro letting database-touching tests skip cleanly when Postgres is unreachable. The suite encodes PRD invariants as executable assertions. Every PRD §I-clause has at least one tripwire; every new invariant lands with at least one new test. The convention has caught more regressions than any single review.

13.2 What surprised

Predicate proliferation. 938,918 distinct predicates is far beyond what we expected. The freely-minted-predicate problem is the direct downstream of giving LLMs the latitude to mint IRIs without a registry lookup, which we did for M5 because the alternative — "the model must use one of these 12,000 existing predicates" — was producing systematic under-coverage (the model would refuse to extract a claim it had no good predicate for). The vocab-aware extraction (commit 31c519b, "vocab-aware extraction — stop minting fresh predicates") is the partial fix; the alignment-closure backfill (~922 K predicates → nearest canonical, cosine ≥ 0.9) is the larger one.

Evidence-anchor sparsity. 1,837,151 evidence links / 39,294,083 statements ≈ 4.7 % anchor coverage. This is the gap between the intended invariant (I1: no claim without evidence or hypothesis_only) and the lived reality of a corpus accumulated across two extractor generations. The anchor-aware ingest (commit 5928bff) plus the donto trace provenance backward-fill (Stage D) is closing this gap; the roadmap target is ≥ 50 % coverage.

Retraction rarity. 281 retractions across 39.3 M statements (7 × 10⁻⁶). This is a striking finding. It tells us that we are operating donto as an append-mostly archive rather than as a constantly-revised research notebook. The substrate supports retraction (donto_retract, donto_correct) and the discipline is encoded in the CLAUDE.md non-negotiable list. But the actual usage shows that researchers are accumulating contradictory claims rather than retracting older ones — exactly as the paraconsistency invariant says they should. The two oldest sources disagreeing about Annie Davis's birth year both still live.

The reification tail. The top fifteen predicates by row count are dominated by reified meta-statements (donto:status, donto:aboutPredicate, donto:textSpan, donto:claimA, donto:claimB). The M5 extractor reified each claim into ~7–10 rows; the M6 exhaustive extractor does not. The older half of the database carries the reification tail; the newer half is substantially less dense. The plan for unwinding the tail is quarantine (move ex:normalized_claims/* to a retract-or-quarantine context, ~2.37 M rows) plus a re-ingest under the new vocabulary.

Long-running characterisation queries. Subject cardinality and polarity-mixed contradictions do not return in routine time at 39 M rows. We thought we knew the index story for these — index on subject, index on (subject, predicate) — but DISTINCT and GROUP BY HAVING over the whole table both require parallel hash-aggregate plans that take longer than the agent budget. The fix is matviews (donto_subject_stats, donto_contradiction_pressure) refreshed on a daily schedule.

13.3 The Conceivable Aperture: an honest accounting

Of the six apertures, Conceivable is the one we are least sure about. It produces unanchored, hypothesis-only claims by design. The position recorded in the Maximalism doc — "mine everything; let curation decide" — is principled, but we have not yet tested whether downstream curation actually filters Conceivable output at useful rates, or whether it floods the candidate space in ways that make E2 promotion costly. The provisional answer is to keep it on by default but in a dedicated ctx:.../conceivable sub-context, so that release builders can exclude the entire context with one clause.

13.4 Lean parity

The Lean overlay is operationally working — donto_engine spawns under dontosrv, certifies the three built-in shapes (functional, typed-literal, parent–child age-gap), and the autoresearch-genealogy/lean/Genealogy/ library has a substantially more developed catalogue (one-birth-per-person, sameAs symmetry/transitivity, parent-date plausibility). The gap is that the two libraries are not yet converged: shapes that exist in Genealogy do not yet exist in packages/lean/. The convergence work is straightforward (port the shape combinator with its proof of soundness) but unspectacular; landing it is the natural milestone that completes the Lean side of the substrate.

14.1 Substrate

• HTTP-middleware Trust Kernel enforcement. The substrate-side policy is fail-closed. The HTTP-side enforcement (block writes that don't carry policy) is the visible end-to-end Trust Kernel test, with hundreds of unpoliced legacy sources in the genes corpus as the natural testbed.
• Subject cardinality matview. The donto_subject_stats matview is the route to answering basic characterisation queries at corpus scale without ad-hoc full table scans.
• Predicate alignment backfill. Across 938 K predicates, the nearest-canonical alignment proposal (cosine ≥ 0.9) needs to be run, reviewed at the high-confidence band, and applied. The closure-rebuild work happens daily on a systemd timer; the proposal stage does not.
• H10 scale lock. PRD §25 hard target is 100 ms at 10 M rows. The extrapolation says yes; the actual run gives a number we can cite. The scheduling cost is ~70 minutes for the insert pass plus benchmark; it is worth committing the calendar slot.

14.2 Substrate completeness vs application

The substrate (M0–M4) is complete. Open work clusters around applications:

• M5 (Extraction): scheduled runs as a systemd timer (~20 LOC shell). Currently runs are submitted ad-hoc via donto extract.
• M6 (Language pilot): all five importers (CLDF, CoNLL-U, UniMorph, LIFT, EAF) shipped with tests; the work is running them against real datasets (Glottolog, Universal Dependencies, UniMorph, LIFT, ELAN-attested corpora) and dispatching from the CLI.
• M7 (Release builder): wrap the existing JSONL + RO-Crate + Ed25519 pipeline as a donto release CLI verb.
• M8 (Scale): see H10 above.
• M9 (Federation): publish a release end-to-end via DataCite (or Zenodo) and demonstrate the two-instance smoke-test (instance A signs, instance B verifies).

14.3 Domain

• Lean parity with the autoresearch-genealogy/lean/Genealogy/ library.
• Orphan-research-notes cross-linker: the genes workspace has several hundred .md files that are not yet linked from any context's source registration.
• DontoQL WITH evidence result shape (current evaluator records the directive but does not change the row shape).
• CLI manpage and completions install: the CLI emits these on donto man / donto completions; package the install as part of a donto install-completions subcommand.

14.4 What we are deliberately not doing

• Live cross-instance federated query (§11.4 non-goal).
• Closed-world entity reconciliation. Identity is a hypothesis, not a foreign key.
• Aggressive normalisation at write time. The substrate writes what the extractor emits; alignment and identity are evaluated at query time under the user's chosen lens.
• OWL-style entailment as primary semantics. Lean certifies; Postgres executes; OWL is one possible alignment safety flag among three.