niko_trust/REPORT_STAGE2B.md

Stage 2b Report — TCE wire codec (internal/tce) and protocol objects (internal/protocol)

Status: complete. All tests pass; the Go codec is cross-checked against the Python reference. Stop before Stage 3 (server policy evaluation) — nothing here touches §9–§11 semantics; those are out of scope for this stage.

Objective

Implement the exact binary TCE codec (§2–§6 of docs/PROTOCOL.md) and the six protocol objects (§7–§8, §12) in Go 1.25: EncodeX/DecodeX/VerifyX, with rule-level, golden, mutation, fuzz and invariant coverage, and a frozen testdata/vectors/tce_vectors.json regression set.

Files changed (this stage)

• internal/protocol/objects.go — 6 structs, unexported tce/sig retention, TCE()/Signature() (copy) accessors, ErrNil sentinel.
• internal/protocol/encode.go — EncodeIdentity/Claim/Revocation/ApprovalRequest/ApprovalResponse/AuthAssertion.
• internal/protocol/decode.go — strict DecodeX (whole-object limit first, tag check, address.ValidatePubKey on every pubkey, End() trailing guard, byte-copies so decoded buffers never alias input).
• internal/protocol/verify.go — VerifyX, VerifyApprovalResponse (binds request via request_hash, responder==recipient, timing window), VerifyRevocationOf, ValidateCurrent, ClaimStatusAt.
• internal/protocol/helpers_test.go — vector loader, signerFixtures, rejectEntry.TCEHex changed to *string (distinguishes empty from absent).
• internal/protocol/{vectors,rules,verify,rejects,mutation,fuzz,invariants}_test.go.
• internal/tce/{primitives,vectors,fuzz,invariants}_test.go (existing codec, new fuzz + invariant coverage).

Canonical format (reaffirmed, per §8)

Field order fixed: identity(identity,alias,created_at) · claim(issuer,subject,claims,created_at,expires_at,serial,nonce) · revocation(issuer,claim_id,reason,created_at,nonce) · request(sender,recipient,action,payload,message,created_at,expires_at,nonce) · response(request_hash,responder,decision,created_at,nonce) · auth(identity,challenge,scope,audience,created_at). DecodeX retains the received bytes in o.tce; VerifyX verifies the signature over o.tce (property #2: "the bytes that arrived are the bytes that are verified"). Encoder ignores any tce/sig set by the caller.

Test inventory & results

Suite	Covers	Result
tce primitives/fuzz	§2,§4.1,§4.3,§6.2,§4.6,§5,§6.1, fixed/trailing/ID/injective	PASS
tce vectors	28 accept + 18 reject number vectors from frozen file + idempotence	PASS
tce invariants	no encoding/json, no signing, allowed deps	PASS
protocol vectors	9 golden (pubkey/address match seed, object_id, len, signature verify, encode(decode)==, revocation→claim & response→request bindings)	PASS
protocol rules	per-object §8 limits & reject paths (empty/expiry/nonce/33-entry/whole-limit/degenerate-key/alias/message/action/reason/scope/audience/lifetime)	PASS
protocol verify	claim sig+foreign/short/empty/tampered/wrong-type; response request_hash/responder/early/late/window; auth exact/empty/affix; revocation binding; ValidateCurrent/ClaimStatusAt	PASS
protocol rejects	8 exact tce_hex rejects + unsorted/dup/value-tag (hand-patched bytes)	PASS
protocol mutation	TestMutationSweep (flip every byte of TCE ×{0x01,0x80} + every sig bit → verify fails; baseline verifies) + TestDenyAndAllowDifferInOneByte	PASS
protocol fuzz	FuzzDecodeIsTotalAndNonMalleable (decode totality + encode(decode(b))==b) and FuzzClaimBuildRoundTrip (decode(encode(x))==x, ID-stable)	PASS (≈545k / ≈23k execs, 8s each)
protocol invariants	no encoding/json, no signing, allowed deps	PASS

go test ./... -count=1: all packages OK. go vet ./...: clean.

Coverage

• internal/tce: 85.3% of statements (was 74.7% — added number_test.go covering §5.1 CanonicalNumber/IsCanonicalNumber, id_test.go for ID accessors/ParseID/IDFromBytes/MarshalText/UnmarshalText, and value_test.go for Value constructors/accessors/Equal/GoString).
• internal/protocol: 91.4% of statements (was 89.5% — added accessors_test.go for Decision.String, every Signature() accessor, ValidateCurrent/ClaimStatusAt clock-skew boundaries, and VerifyRevocationOf issuer/claim-ID binding errors).

Hardening pass (fuzz depth, OSS-Fuzz, regression corpus)

• Fuzz caught a real test bug. FuzzStringValidation found a seed where a long (≥128-byte) valid string failed the round-trip assertion. The assertion wrongly assumed a single-byte length prefix; the encoder uses a uvarint prefix, which is multi-byte for long strings. The codec was correct — the test was fixed to decode the uvarint prefix (internal/tce/fuzz_test.go). The failing input is committed as internal/tce/testdata/fuzz/FuzzStringValidation/f921751fe02821d6.
• All six fuzz targets run stable under extended fuzzing (≥20s each, millions of execs, no crashes/hangs): tce FuzzUvarint, FuzzDecodePrimitives, FuzzStringValidation; protocol FuzzDecodeIsTotalAndNonMalleable (decode totality + encode(decode(b))==b), FuzzClaimBuildRoundTrip (decode(encode(x))==x, ID-stable). The live corpus is persisted in the Go fuzz cache ($GOCACHE/fuzz) and replayed on every go test -fuzz; reviewable seed inputs are committed via f.Add (golden vectors + edge cases: multi-byte string prefix, UTF-8, multi-entry maps, large numbers).
• OSS-Fuzz / go-fuzz harness. Added internal/tce/fuzz.go and internal/protocol/fuzz.go under //go:build gofuzz, each exposing the standard func Fuzz(data []byte) int entry point asserting the §12.4 totality/injection properties. Both compile cleanly with go build -tags gofuzz ./internal/tce ./internal/protocol (verified). These are excluded from normal go test builds, so they do not affect the unit suite. To run under OSS-Fuzz, build with go-fuzz-build/go-fuzz (the gofuzz tag), which instruments the same code paths the unit fuzz targets exercise.

Cross-check vs Python reference

python3 tools/reference/tce_reference.py regenerated and the output was byte-for-byte identical to testdata/vectors/tce_vectors.json (canonical bytes, object_id_hex, signature_hex). The reference signs over the raw TCE bytes and derives object_id as SHA256(tce) — exactly what the Go VerifyX and ComputeID assume — so the golden vectors are authoritative and the Go tests exercise them directly. No drift between implementations.

Deviations / interpretation notes (must survive into Stage 3)

1. Response timing window. §8.5/§13.1 delimit an approval's validity only by request.created_at/request.expires_at. Because responder and requester clocks are independent, the implemented window is request.created_at - MaxClockSkew <= response.created_at <= request.expires_at + MaxClockSkew with a single MaxClockSkew = 120. This is a tolerance interpretation, not a protocol change. Surfaced here so Stage 3 clock handling stays consistent.
2. Public-key validation. crypto/ed25519.Verify does not reject small-order points; decode.go/encode.go validate every pubkey via address.ValidatePubKey (filippo.io/edwards25519) before any signature step (non-negotiable #3). crypto/ed25519 is imported by protocol for ed25519.Verify only; signing (Sign/NewKeyFromSeed/GenerateKey) lives solely in internal/identity/signer.
3. No JSON in the codec. internal/tce and internal/protocol do not import encoding/json (enforced by TestNoJSONImport). The json field in vectors is reference-only and never parsed by the codec.
4. Request/response coupling. VerifyResponse does not exist standalone (non-negotiable #3): a response is only meaningful bound to its request via VerifyApprovalResponse(requestTCE, requestSig, responseTCE, responseSig).
5. Audience is exact, constant-time, and an empty expected audience is rejected (ErrEmptyAudience).

Not done (future stages)

• Stage 3 server policy engine (§9 proof satisfaction, §10 proof search, §11 boolean/threshold/rate-limit evaluation). This stage ships only the codec + cryptographic/structural verification + frozen regression vectors.
• Higher fuzz durations / OSS-Fuzz harness wiring (current runs are 8s smoke fuzzes; corpus is seeded from the 9 golden objects).
• End-to-end "deny reason" / get_claims-style query APIs.