20cc52c3a5
- BLAKE3 keyed proof-of-work on object storage and auth challenges, with frozen vectors cross-checked against an independent Python reference implementing the single-block hash it needs. - Sparse Merkle trie over object IDs: order-independent roots, inclusion and absence proofs (internal/smt). - Signed checkpoint chain per relay: transport key amendment to INV-1, /v1/checkpoint/* and inclusion/absence proof endpoints, restart-safe epoch continuity (internal/checkpoint). - Head gossip with TOFU pinning and equivocation detection; light node (cmd/lightnode) that stores no history: quorum of pinned relays, every served object proven against the agreed root, LRU disk cache. - WebSocket streaming on relay and light node (coder/websocket): scoped channels mirroring REST, raw envelopes verified client-side; light node marks streamed objects unproven until checkpoint coverage. - Protocol v1 additions: DelegationClaim tag 0x07 with deterministic chain resolution in verify.Graph, KeyRotationRequest/Confirm tags 0x08/0x09 with hash-bound two-sided consent and Policy.RotationMaxAge; spec sections, frozen vectors appended byte-identically, Python reference extended. - Optional permissioned BFT finality over gossip (internal/bft): prevote/precommit with quorum certificates verifiable offline. - Quick wins: Policy.TrustedIssuers, per-type stored metrics, batch fetch, lexicographic lists with stable cursor pagination. - Security review of the network layer (docs/SECURITY-REVIEW.md) with findings F-01..F-09; hub send/close race and unstable pagination fixed under review. 12 packages green, vet/gofmt clean, protocol fuzzing stable.
4.7 KiB
Checkpoints, proofs and light nodes
This document specifies the relay's signed commitment to its own object set, the proof endpoints built on it, and the light node that consumes both. It is the mechanism that turns the INV-1 caveat ("a relay can withhold anything") from an act of faith into a detectable event.
Companion documents: TRUST-MODEL.md, API.md, PROTOCOL.md.
1. The idea
A relay stores a set of content-addressed objects. The sparse Merkle root of
that set (internal/smt) is a function of the set alone: two relays that
store the same objects compute the same root regardless of insertion order or
restart history. A relay signs {epoch, size, root, prev, created_at} every
interval or every N new objects; the prev field chains each head to the
hash of its predecessor, making per-relay history tamper-evident without any
global consensus.
Decentralization falls out of gossiping heads: mirrors that disagree about what a relay stores are detectable by comparing what independent peers claim under the same relay key. No token, no mining, no block cadence — approval flows keep their 60-second lifetime, and phones can participate as light nodes instead of miners.
2. What the relay key may and may not do
INV-1 amendment: the relay holds exactly one key. Its entire power is to
describe its own storage — signing checkpoints and nothing else. Statement
signing still lives exclusively in internal/identity/signer, which server
code cannot import (enforced by TestServerDoesNotImportSigner), and the
checkpoint package never touches protocol objects. Both properties are
source-scanned by TestNoSigningOutsideSigner.
The seed persists in <data-dir>/relay_key.seed (0600). In-memory mode
generates an ephemeral key per process; heads are then valid only while the
process lives.
3. Trie and proofs
Hash domain (SHA-256):
leaf = H(0x00 || object_id)
branch = H(0x01 || uvarint(prefixLen) || prefixBits || left || right)
empty = H(0x02)
The branch prefix is inside the hash so trie shape is committed. Branches split exactly where their keys first differ, so shape is canonical.
- Inclusion proof (
GET /v1/proof/object/{id}): sibling path from the leaf to the root. Verifies only for the exact object id. - Absence proof (
GET /v1/proof/absent/{id}): witnesses the spot where the id would live — either the neighbouring leaf reached by walking the id's bits, or the branch whose prefix the id leaves mid-way. This is what lets a consumer prove "nothing else about this subject exists" rather than trusting a relay's silence.
Proofs verify against a root the consumer obtained independently (from a quorum of peers), never against a root the serving relay just made up.
4. Endpoints
| Method | Path | Purpose |
|---|---|---|
| GET | /v1/checkpoint/latest |
newest signed head {bytes, signature, id, public_key, checkpoint} |
| GET | /v1/checkpoint/{epoch} |
a specific historical head |
| GET | /v1/proof/object/{id} |
inclusion proof against the current root |
| GET | /v1/proof/absent/{id} |
absence proof against the current root |
| POST | /v1/gossip/checkpoint |
announce a peer's signed head |
| GET | /v1/peers/heads |
observed peer heads |
bytes is authoritative (base64); the decoded checkpoint view is a
convenience, mirroring how envelopes work. Gossip accepts a head only if the
signature verifies under the announcing key and the id matches the bytes;
same-key/same-epoch/different-bytes is recorded as divergence
(trust_gossip_divergence) and exposed, never silently resolved.
Configuration flags: -ckpt-interval (default 60s), -ckpt-every
(default 128 new objects).
5. Light nodes
cmd/lightnode follows heads from -peers, requires -quorum distinct
relay keys to agree on one root (TOFU pinning unless -pin-keys given), then
serves the read API locally:
- every served object was proven present in the agreed set before caching;
- every absence claim was proven against the agreed root;
- storage is the working set plus heads — no history;
- cached objects keep serving after all peers disappear.
Run it against two or more independent relays holding the same data:
lightnode -peers https://a.example,https://b.example \
-quorum 2 -cache-dir /var/lib/lightnode -addr :8090
The node makes no authorization decisions; consumers verify statements locally as before (INV-5). Trust your own verification stack, not the mirror.