release(v4.0.0): Shade GA — V3.x consolidation + audit prep

V3.1 → V3.12 consolidated and tagged for the first GA release. Wire format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers byte-for-byte. The version bump is semantic: audit-cycle complete, opt-in surface fully exposed, threat model refreshed for every new surface. Highlights: - All 24 @shade/* packages bumped to 4.0.0 in lockstep. - CHANGELOG 4.0.0 section is the canonical manifest of what landed. - THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12 Web-Worker boundary) + residual-risks table refreshed. - OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox, bridge, observer, /metrics, /healthz, /ready. - MIGRATION 0.3.x → 4.0 documented + smoke-tested against shade migrate-storage on a real SQLite DB. - docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer. - scripts/soak.ts harness for the GA-stable 2-week soak window. - All V*.md plans archived under docs/archive/ with Status: Done. - Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen non-realtime stack. Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green. Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports version 4.0.0 on /health. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
parent 8b055912b7
commit e6fdf31b49
298 changed files with 37909 additions and 256 deletions
--- a/docs/audit/REVIEW-BUNDLE.md
+++ b/docs/audit/REVIEW-BUNDLE.md
@@ -0,0 +1,143 @@
+# Shade 4.0 — External Crypto Review Bundle
+
+This document is the entrypoint for an external cryptographic review of
+Shade 4.0. It collects, in one place, every artifact a reviewer needs to
+audit the protocol implementation **without** rooting around the
+codebase first.
+
+## Tag under review
+
+- **Version:** `4.0.0`
+- **Tag:** `v4.0.0`
+- **Date:** 2026-05-03
+- **Repo:** `https://gt.zyon.no/Stian/Shade` (mirror at the
+  consumer-app repos that vendor this code)
+- **Out-of-scope:** Voice / Video / Broadcast — moved to V5.0 and
+  reviewed separately.
+
+## What's in scope
+
+Reviewers focus on the protocol-cryptographic core. Each scope cell maps
+to one or more packages plus the spec / threat-model section that
+describes its design.
+
+### A — Protocol core (highest priority)
+
+| Surface | Spec | Code |
+|---------|------|------|
+| X3DH initial key agreement | [`docs/archive/V3.1.md`](../archive/V3.1.md), [`THREAT-MODEL.md` §1, §2](../../THREAT-MODEL.md) | [`packages/shade-core/src/x3dh.ts`](../../packages/shade-core/src/x3dh.ts) |
+| Double Ratchet | [`docs/archive/V3.1.md`](../archive/V3.1.md), [`THREAT-MODEL.md` §3](../../THREAT-MODEL.md) | [`packages/shade-core/src/ratchet.ts`](../../packages/shade-core/src/ratchet.ts) |
+| Sender keys (group ratchet) | [`docs/archive/V3.10.md` § Group send](../archive/V3.10.md) | [`packages/shade-core/src/sender-keys.ts`](../../packages/shade-core/src/sender-keys.ts) |
+| Wire envelopes `0x01`, `0x02`, `0x11` | [`packages/shade-proto/README.md`](../../packages/shade-proto/README.md) | [`packages/shade-proto/src/`](../../packages/shade-proto/src/) |
+| At-rest storage encryption | [`docs/storage-encryption.md`](../storage-encryption.md), [`THREAT-MODEL.md` §4](../../THREAT-MODEL.md) | [`packages/shade-storage-encrypted/src/`](../../packages/shade-storage-encrypted/src/) |
+| Social recovery (Shamir + AEAD-gated reconstruction) | [`docs/recovery.md`](../recovery.md), [`THREAT-MODEL.md` §8](../../THREAT-MODEL.md) | [`packages/shade-recovery/src/`](../../packages/shade-recovery/src/) |
+
+### B — Trust + transport
+
+| Surface | Spec | Code |
+|---------|------|------|
+| WebRTC P2P transport binding | [`docs/webrtc.md`](../webrtc.md), [`THREAT-MODEL.md` §11](../../THREAT-MODEL.md) | [`packages/shade-transport-webrtc/src/`](../../packages/shade-transport-webrtc/src/) |
+| Key Transparency log + verifier | [`docs/key-transparency.md`](../key-transparency.md), [`docs/archive/V3.12-DESIGN.md`](../archive/V3.12-DESIGN.md), [`THREAT-MODEL.md` §2 (mitigated-by-V3.12)](../../THREAT-MODEL.md) | [`packages/shade-key-transparency/src/`](../../packages/shade-key-transparency/src/) |
+| Fingerprint gates | [`docs/trust-ux.md`](../trust-ux.md), [`THREAT-MODEL.md` §10](../../THREAT-MODEL.md) | [`packages/shade-sdk/src/fingerprint-gates.ts`](../../packages/shade-sdk/src/fingerprint-gates.ts) |
+
+### C — Lower-priority surfaces
+
+| Surface | Spec | Code |
+|---------|------|------|
+| Inbox store-and-forward | [`docs/inbox.md`](../inbox.md), [`THREAT-MODEL.md` §6](../../THREAT-MODEL.md) | [`packages/shade-inbox-server/src/`](../../packages/shade-inbox-server/src/), [`packages/shade-inbox/src/`](../../packages/shade-inbox/src/) |
+| Bridge transports (SSE / long-poll / WS) | [`docs/transport.md`](../transport.md) | [`packages/shade-transport-bridge/src/`](../../packages/shade-transport-bridge/src/) |
+| Web Workers crypto | [`docs/web-workers.md`](../web-workers.md), [`THREAT-MODEL.md` §12](../../THREAT-MODEL.md) | [`packages/shade-crypto-web/src/worker*`](../../packages/shade-crypto-web/src/) |
+| Files RPC | [`docs/files.md`](../files.md) | [`packages/shade-files/src/`](../../packages/shade-files/src/) |
+| Streams (chunked AEAD over ratchet) | [`docs/streams.md`](../streams.md) | [`packages/shade-streams/src/`](../../packages/shade-streams/src/), [`packages/shade-transfer/src/`](../../packages/shade-transfer/src/) |
+| Observability | [`docs/observability.md`](../observability.md) | [`packages/shade-observability/src/`](../../packages/shade-observability/src/) |
+
+## Threat model
+
+The full threat model is at [`THREAT-MODEL.md`](../../THREAT-MODEL.md).
+Every numbered "Mitigations" entry ends with a `[tests:]` footnote
+linking to the file(s) that holds the mitigation in place. Reviewers
+can re-run any individual test in isolation:
+
+```bash
+bun test packages/shade-core/tests/ratchet.test.ts
+bun test packages/shade-streams/tests/aead.test.ts
+bun test packages/shade-key-transparency/tests/manager.test.ts
+```
+
+## Cross-platform parity
+
+The wire format and KDF-label corpus are byte-identical between TS
+(bun) and Kotlin (gradle). The CI gate that enforces this lives at
+[`.gitea/workflows/cross-vectors.yml`](../../.gitea/workflows/cross-vectors.yml).
+Vectors are generated by [`scripts/generate-vectors.ts`](../../scripts/generate-vectors.ts);
+hand-edits to [`test-vectors/`](../../test-vectors/) are rejected by CI.
+
+```bash
+# Re-run the cross-platform vector suite locally:
+bun run test:vectors
+cd android && ./gradlew :shade-android:test
+```
+
+## Build instructions (reproducible)
+
+```bash
+git clone https://gt.zyon.no/Stian/Shade
+cd Shade
+git checkout v4.0.0
+bun install --frozen-lockfile
+
+# TS suite
+bun test
+
+# Kotlin / vector suite
+cd android && ./gradlew :shade-android:test
+```
+
+Container image (prekey + transfer + bridge + KT):
+
+```bash
+docker pull gt.zyon.no/stian/shade-prekey:4.0.0
+docker run --rm -p 3900:3900 \
+  -e SHADE_PREKEY_PG_URL=postgres://… \
+  gt.zyon.no/stian/shade-prekey:4.0.0
+```
+
+The `Dockerfile` is at [`packages/shade-server/Dockerfile`](../../packages/shade-server/Dockerfile).
+Multi-stage; the runtime stage uses a non-root user.
+
+## Assumptions and known limitations
+
+1. The runtime is honest. A malicious Bun / browser engine can defeat
+   any JS library; we ride the platform's `SubtleCrypto` / `@noble/curves`
+   for primitives and trust them.
+2. `THREAT-MODEL.md` section "Assumptions" is the canonical list; review
+   the residual-risks table at the bottom of the same file for
+   intentional gaps.
+3. We do **not** claim resistance to power-analysis or fault-injection
+   side channels.
+4. Memory zeroization is best-effort. V8 / JSC may retain freed buffers;
+   we zero what we can synchronously reach.
+
+## How to report findings
+
+- **Severity-prioritized** (CVSS 3.1 if you can, otherwise plain
+  language).
+- **Reproducer in repo style** — a failing `bun test` is preferred over
+  prose.
+- **Email** the maintainer (`Sterister@live.no`); see
+  [`SECURITY.md`](../../SECURITY.md) for PGP / age key arrangement.
+
+## Timeline
+
+The 4.0 audit window is open immediately after tag. We aim for a
+4–8-week review cycle (see V4.0 plan). Any **critical** or **high**
+severity finding pauses the GA-stable announcement until the fix
+ships. Findings ship as `4.0.x` patch releases — wire-format unchanged.
+
+## Out-of-scope (deferred to V5.0)
+
+- Voice (`@shade/voice`) — SFrame-style frame keys, key-rotation policies.
+- Video (`@shade/video`) — codec edges (AV1/VP9/H.264).
+- Broadcast (`@shade/broadcast`) — relay-helper threat model.
+
+These will get their own review window when V5.0 is ready.
--- a/docs/audit/SCOPE.md
+++ b/docs/audit/SCOPE.md
@@ -0,0 +1,75 @@
+# Shade 4.0 — Audit Scope
+
+A short, structural list a reviewer can scan before opening a single
+file. Everything here is a pointer to the deeper material in
+[`REVIEW-BUNDLE.md`](./REVIEW-BUNDLE.md) and the package READMEs.
+
+## In scope
+
+- **Protocol primitives**: X3DH, Double Ratchet, sender keys.
+- **Wire format**: `0x01` PreKeyMessage, `0x02` RatchetMessage, `0x11`
+  StreamChunk. Length prefixes (u32) and AAD bindings.
+- **Storage encryption** (`@shade/storage-encrypted`): KDF chain,
+  per-(table,column) DEKs, AEAD AAD layout, online re-key.
+- **Recovery** (`@shade/recovery`): Shamir over GF(2^8),
+  AEAD-authenticated reconstruction, fingerprint gate on guardian
+  release, share-grant / share-decline envelope schema.
+- **WebRTC P2P** (`@shade/transport-webrtc`): SDP/ICE signaling rides
+  the ratchet; chunk frames AEAD-bound to streamId/laneId/seq; glare
+  resolution determinism.
+- **Key Transparency** (`@shade/key-transparency`): Merkle log over
+  pre-hashed leaves, address-sorted index, signed STH, witness
+  cross-check, split-view detection.
+- **Inbox** (`@shade/inbox-server`): TOFU registration, per-PUT signed
+  blobs, idempotent on `(address, msgId)`, replay window.
+- **Bridge** (`@shade/transport-bridge`): SSE / long-poll / WS
+  carriers; signed-query auth (no headers on `EventSource`).
+- **Crypto in workers** (`@shade/crypto-web/worker`): key-isolation
+  boundary, postMessage protocol, idle terminate.
+- **Trust UX gates** (`@shade/sdk` `Shade.beforeFirstLargeFile`,
+  `beforeBackupImport`, `beforeNewDeviceTrust`).
+
+## Out of scope
+
+- **Voice / Video / Broadcast** (`@shade/voice` etc.) — V5.0; reviewed
+  when the package ships.
+- **Build system** (Vite, Rollup, Gradle wiring) — out of crypto scope.
+- **App-level UI** (`@shade/widgets`) — re-renders the primitives
+  above; the cryptographic decisions are in the SDK / core packages
+  the widgets consume.
+- **Browser / native WebRTC stacks** — we ride the platform's
+  `RTCPeerConnection` and `SubtleCrypto`.
+- **Operating system / hardware threat model** — filesystem
+  encryption, secure-enclave key storage, swap-encryption, coredump
+  handling. Operator responsibility.
+
+## Methodology suggestions
+
+1. Start with [`THREAT-MODEL.md`](../../THREAT-MODEL.md) — every entry
+   has a `[tests:]` footnote. Toggle each test off, confirm it fails;
+   toggle the corresponding mitigation off, confirm it fails.
+2. Re-derive every KDF label from the spec; check
+   [`scripts/generate-vectors.ts`](../../scripts/generate-vectors.ts) and
+   the recorded vectors in [`test-vectors/`](../../test-vectors/) match.
+3. Run the cross-platform suite on **both** TS (bun) and Kotlin
+   (gradle) — divergence is a vector-format bug.
+4. Audit the AEAD AAD construction at every layer:
+   - Ratchet: header bytes (counter + DH pub) → AES-GCM AAD.
+   - Streams: `streamId || laneId || seq || isLast` → AES-GCM AAD.
+   - Storage: `(table, column, pk)` → AES-GCM AAD.
+5. Trace the boundary between the worker-side crypto thread and the
+   main thread — confirm that no handle to a wrapped DEK or a
+   ratcheted chain key crosses over.
+
+## Open questions for reviewer commentary
+
+- The witness gossip channel for V3.12 is currently in-band over the
+  ratchet; should we cross-pin against an out-of-band log mirror in
+  4.x, or wait for a federated relay tier?
+- WebRTC peer-glare is resolved by lexicographic address compare — a
+  reviewer could confirm the equivalent constructions in libsignal or
+  Matrix and flag if our edge cases match.
+- Storage encryption uses AES-GCM with a per-row IV. The IV is
+  random, not deterministic; reviewers should confirm the
+  combinatorial-collision threshold matches the per-column row count
+  bounds.