docs(readme): full 4.0 GA refresh — status table, all V3.x surfaces, audit pointers

- Status table (V3.2 → V3.12 all green; audit + soak harness ready) - "What you get" expanded: storage encryption, fingerprint gates, inbox, bridge, web workers, recovery, KT, observability, soak - Quick start: opt-in surface examples (gates, workers, WebRTC, KT) + at-rest encryption snippet + migrate-storage CLI invocation - Architecture diagram refreshed: container now exposes /v1/inbox/*, /v1/bridge/*, /v1/kt/*, ops endpoints; WebRTC P2P pipe documented - Packages table: all 24 entries at 4.0.0 (added observability, keychain, storage-encrypted, transport-bridge, etc.) - Publishing section: 4.0.x examples; soak harness section added - Security properties: V3.2 / V3.3 / V3.6 / V3.8 / V3.10 / V3.11 rows added alongside the existing forward-secrecy / PCS list - Documentation: grouped into Operator+integrator, Per-surface, Threat-model+audit, Migration+history, Examples - Container includes: full V4.0 surface (inbox, bridge, KT, ops probes); image pinned at gt.zyon.no/stian/shade-prekey:4.0.0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:58:38 +02:00
parent de25b19033
commit 40766c60f4
1 changed files with 257 additions and 65 deletions
--- a/README.md
+++ b/README.md
@@ -3,37 +3,82 @@
 End-to-end encryption library implementing the Signal Protocol (X3DH + Double Ratchet) for TypeScript/Bun. Drop into any project — frontend, backend, mobile — to get forward secrecy, post-compromise recovery, and self-healing security.

 > **4.0.0 — General Availability.** All V3.1 → V3.12 work is merged,
-> the cross-platform vector suite is green on TS + Kotlin, the threat
-> model has been refreshed for every new surface, and the core stack
-> (X3DH, ratchet, storage encryption, recovery, WebRTC P2P, Key
-> Transparency) has been packaged for external review. The wire
-> format is **unchanged from 0.4.x** — 4.0 peers interoperate with
-> 0.4.x peers byte-for-byte. See
+> the cross-platform vector suite is green on TS + Kotlin (1000 / 1000
+> + 11 / 11), the threat model has been refreshed for every new
+> surface, and the core stack (X3DH, ratchet, storage encryption,
+> recovery, WebRTC P2P, Key Transparency) has been packaged for
+> external review. The wire format is **unchanged from 0.4.x** — 4.0
+> peers interoperate with 0.4.x peers byte-for-byte. See
 > [MIGRATION.md § 0.3.x → 4.0](./MIGRATION.md#migrating-from-03x-to-40-ga)
 > for the upgrade path and [CHANGELOG.md § 4.0.0](./CHANGELOG.md) for
 > the consolidated release notes. Voice / Video have been moved to
 > [V5.0](./docs/V5.0.md), to be built on top of the frozen 4.0
 > baseline.

+## Status
+
+| Area | 4.0 status | Pointers |
+|------|-----------|----------|
+| Protocol core (X3DH + ratchet + sender keys) | ✅ Done — frozen | [`packages/shade-core`](./packages/shade-core) |
+| Storage encryption (V3.2) | ✅ Done — opt-in `EncryptedSQLiteStorage` / `EncryptedPostgresStorage`, key sources: passphrase / OS keychain / app-injected | [`docs/storage-encryption.md`](./docs/storage-encryption.md) |
+| Fingerprint gates & trust UX (V3.3) | ✅ Done — `Shade.beforeFirstLargeFile / beforeBackupImport / beforeNewDeviceTrust` | [`docs/trust-ux.md`](./docs/trust-ux.md) |
+| Observability v2 (V3.4) | ✅ Done — OpenTelemetry-shaped events, `/metrics`, observer dashboard | [`docs/observability.md`](./docs/observability.md) |
+| Android parity & cross-platform CI (V3.5) | ✅ Done — TS + Kotlin vector-gate live; Android `KeystoreStorage` is post-GA | [`android/shade-android/README.md`](./android/shade-android/README.md), [`docs/cross-platform.md`](./docs/cross-platform.md) |
+| Async store-and-forward (V3.6) | ✅ Done — `@shade/inbox` + `@shade/inbox-server` | [`docs/inbox.md`](./docs/inbox.md) |
+| Transport bridge (V3.7) | ✅ Done — SSE / long-poll / WS adapters | [`docs/transport.md`](./docs/transport.md) |
+| Web Workers crypto (V3.8) | ✅ Done — lane keys never cross the thread boundary | [`docs/web-workers.md`](./docs/web-workers.md) |
+| Rich file metadata + thumbnails (V3.9) | ✅ Done — in `@shade/files` | [`docs/files.md`](./docs/files.md) |
+| Social key recovery (V3.10) | ✅ Done — Shamir + AEAD-gated reconstruction + guardian widgets | [`docs/recovery.md`](./docs/recovery.md) |
+| WebRTC P2P transport (V3.11) | ✅ Done — `RTCDataChannel` with `MultiTransportFallback([webrtc, http])` | [`docs/webrtc.md`](./docs/webrtc.md) |
+| Key Transparency (V3.12) | ✅ Done — opt-in Merkle log, signed STH, witness gossip | [`docs/key-transparency.md`](./docs/key-transparency.md) |
+| External crypto review | 🟡 Bundle ready — review window open after tag | [`docs/audit/REVIEW-BUNDLE.md`](./docs/audit/REVIEW-BUNDLE.md) |
+| Soak (≥ 2 weeks under load) | 🟡 Harness shipped — operator runs it | [`scripts/soak.ts`](./scripts/soak.ts) (`bun run soak --hours 336`) |
+| Voice / Video / Broadcast | 🔜 V5.0 — built on top of frozen 4.0 stack | [`docs/V5.0.md`](./docs/V5.0.md) |
+
 ## What you get

+**Protocol core**
 - **X3DH** initial key agreement (works asynchronously via prekey bundles)
 - **Double Ratchet** for per-message forward secrecy and post-compromise security
- **Self-authenticated prekey server** (Hono, Docker-ready) with rate limiting, metrics, health checks
- **Persistent storage backends**: SQLite (zero-config) and PostgreSQL (Drizzle)
+- **Sender keys** for group ratchet (1:N broadcast key derivation)
 - **Identity rotation** with grace period for old sessions
 - **Safety numbers** (Signal-style fingerprints) for out-of-band verification
 - **Constant-time comparisons** and **memory zeroization** for hardened operation
- **Binary wire format** that's significantly smaller than JSON
+- **Binary wire format** (`@shade/proto`) — significantly smaller than JSON
+
+**Storage**
+- **Persistent backends**: SQLite (zero-config, `bun:sqlite`) and PostgreSQL (Drizzle, FOR UPDATE SKIP LOCKED)
 - **Crash-safe** — sessions survive container restarts, power outages, SIGKILL
- **Live observability** — bundled dashboard SPA + embeddable React widgets to see what's happening between every step
- **E2EE file transfers** — multi-lane chunked uploads/downloads with resume, integrity checks, and HTTP/WS fallback (`@shade/streams` + `@shade/transfer`)
- **WebRTC P2P transport (V3.11)** — opt-in `RTCDataChannel` upload path with public-STUN defaults, TURN-relay support, glare-safe peer pool, and automatic `MultiTransportFallback` back to HTTP when NAT traversal fails (`@shade/transport-webrtc`, [docs/webrtc.md](./docs/webrtc.md))
- **Web Workers crypto** — AEAD, HKDF, HMAC, X25519, Ed25519 and per-lane stream state run in a dedicated worker. 100 MB+ uploads stay smooth without frame drops, lane keys never cross the thread boundary (`@shade/crypto-web/worker`, [docs/web-workers.md](./docs/web-workers.md))
- **E2EE filesystem RPC** — typed `list/stat/mkdir/delete/move/read/write/getThumbnail` + custom ops between peers, with rate-limit, retention, and fingerprint-gate hooks (`@shade/files`)
- **Async store-and-forward** — deliver to offline recipients via a relay that holds ciphertext-only blobs with TTL, idempotent PUT, signed fetch/ack, and an `onMessageQueued` push-trigger hook (`@shade/inbox` + `@shade/inbox-server`)
- **Social key recovery** — Shamir-split your identity to `n` guardians; any threshold-many `k` together restore it on a new device. No centralized recovery agent; OOB-fingerprint gate on every guardian release; AES-GCM authenticates the reconstruction (`@shade/recovery` + `<RecoverySetup />` / `<RecoveryRequest />` / `<RecoveryApprove />` widgets, [docs/recovery.md](./docs/recovery.md))
- **Key Transparency (V3.12)** — opt-in append-only Merkle log over the prekey server. Every `register` / `delete` becomes a signed leaf; every bundle-fetch carries an inclusion proof; an Ed25519-signed Tree Head ties roots to a fixed `log_id`. A `LightWitness` cross-checks STHs across clients so a malicious server that splits its view or rewrites history is caught (`@shade/key-transparency`, [docs/key-transparency.md](./docs/key-transparency.md))
+- **At-rest encryption (V3.2, opt-in)** — AES-256-GCM under per-(table,column) DEKs; key sources: passphrase (scrypt), OS keychain (`@shade/keychain`), or app-injected. Online re-key, no downtime.
+
+**Servers**
+- **Self-authenticated prekey server** (`@shade/server`, Hono, Docker-ready) with rate limiting, metrics, health checks
+- **Async store-and-forward relay** (`@shade/inbox-server`) — TTL-bound ciphertext blobs, signed PUT/FETCH/ACK, idempotent on `(address, msgId)`, per-recipient quota
+- **Bridge transports** (`@shade/transport-bridge`) — WS → SSE → long-poll fallback chain for clients that can't keep a WebSocket open. Same `IncomingMessage` shape across all three.
+- **Standalone container** — one image bundles prekey + inbox + bridge + transfer + KT + observer
+
+**Trust UX**
+- **Fingerprint gates (V3.3)** — `Shade.beforeFirstLargeFile(threshold, handler)`, `beforeBackupImport`, `beforeNewDeviceTrust`. Gates raise `FingerprintNotVerifiedError` on the operations that matter, default-warn TOFU otherwise.
+- **`<FingerprintCompare />` / `<FingerprintGate />`** widgets for the matching UI side.
+
+**File transfer & filesystem**
+- **E2EE file transfers** (`@shade/streams` + `@shade/transfer`) — multi-lane chunked uploads/downloads with resume, integrity checks (per-lane sha256 + overall sha256), HTTP/WS fallback, `MultiTransportFallback` for N-ary demotion
+- **WebRTC P2P transport (V3.11, opt-in)** — `RTCDataChannel` chunk path with public-STUN defaults, TURN-relay support, glare-safe peer pool, automatic fallback to HTTP when NAT traversal fails (`@shade/transport-webrtc`)
+- **Web Workers crypto (V3.8, opt-in)** — AEAD, HKDF, HMAC, X25519, Ed25519 and per-lane stream state run in a dedicated worker. 100 MB+ uploads stay smooth without frame drops; lane keys never cross the thread boundary (`@shade/crypto-web/worker`)
+- **E2EE filesystem RPC** (`@shade/files`) — typed `list/stat/mkdir/delete/move/read/write/getThumbnail` + custom ops, with rate-limit, retention, fingerprint-gate, and metrics hooks. React hooks under `@shade/files/react`.
+
+**Recovery**
+- **Social key recovery (V3.10)** — Shamir-split your identity to `n` guardians; any threshold-many `k` together restore it on a new device. No centralized recovery agent; OOB-fingerprint gate per guardian release; AES-GCM-authenticated reconstruction (`@shade/recovery` + `<RecoverySetup />` / `<RecoveryRequest />` / `<RecoveryApprove />`)
+
+**Verifiable distribution**
+- **Key Transparency (V3.12, opt-in)** — append-only Merkle log over the prekey server. Every `register` / `delete` becomes a signed leaf; every bundle-fetch carries an inclusion proof; an Ed25519-signed Tree Head ties roots to a fixed `log_id`. A `LightWitness` cross-checks STHs across clients so a malicious server that splits its view or rewrites history is caught (`@shade/key-transparency`).
+
+**Observability**
+- **Live observability** — OpenTelemetry-shaped events, bundled dashboard SPA + embeddable React widgets to see what's happening between every step (`@shade/observability` + `@shade/observer` + `@shade/dashboard`)
+
+**Tooling**
+- **CLI** — `shade init` scaffolder, `shade migrate-storage` (V3.2), `shade rotate-storage-key`, `shade fingerprint`, `shade rotate`, `shade peer`, `shade dashboard`, `shade doctor`, `shade backup` (`@shade/cli`)
+- **Soak harness** — `bun run soak --hours 336` for the 2-week GA-stable window (`scripts/soak.ts`)

 ## Quick start

@@ -84,7 +129,38 @@ const plaintext = await shade.receive('alice@example.com', incomingEnvelope);
 console.log(await shade.fingerprint);
 ```

-Need to ship a file or expose a filesystem to a peer? `Shade.files` is the high-level entrypoint:
+### Opt-in surfaces (V3.x → 4.0 GA)
+
+All of these are off by default. Wire them only where you need them.
+
+```ts
+// V3.3 — Fingerprint gates: enforce verification on the operations that matter
+shade.beforeFirstLargeFile(10 * 1024 * 1024, async ({ peer, fingerprint }) => {
+  return await ui.confirmSafetyNumberMatches(peer, fingerprint);
+});
+shade.beforeBackupImport(async ({ embeddedFingerprint }) => { /* ... */ });
+shade.beforeNewDeviceTrust(async ({ peer, oldFp, newFp }) => { /* ... */ });
+
+// V3.8 — Web Workers crypto: opt-in, lane keys stay off the main thread
+shade.configureWorkerCrypto({
+  workerUrl: new URL('@shade/crypto-web/worker', import.meta.url),
+});
+
+// V3.11 — WebRTC P2P transport: file transfers ride RTCDataChannel where NAT allows
+import { nativeRtcFactory } from '@shade/transport-webrtc';
+shade.configureWebRTC({
+  factory: nativeRtcFactory(),
+  iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
+});
+
+// V3.12 — Key Transparency: detect server-side bundle swaps
+const shade = await createShade({
+  prekeyServer: 'https://shade.example.com',
+  keyTransparency: { mode: 'observe-strict', logPublicKey: PINNED_KEY_BYTES_32 },
+});
+```
+
+### Files RPC (`@shade/files`)

 ```ts
 // Server side — Bob exposes a virtual filesystem
@@ -97,13 +173,13 @@ const stop = await shade.files.serve({

 // Client side — Alice consumes Bob's filesystem
 const fs = await shade.files.client('bob');
-await fs.write('/photos/cover.png', new Uint8Array(...));   // auto inline/streams
+await fs.write('/photos/cover.png', new Uint8Array([/* ... */]));   // auto inline/streams
 const result = await fs.read('/photos/cover.png');
 ```

 Files ≤ 256 KiB ride inline in the RPC envelope; larger files automatically promote to multi-lane `@shade/transfer` streams with sha256 integrity. See [`docs/files.md`](./docs/files.md) for the full API.

-Or use the lower-level packages directly if you need full control:
+### Lower-level access

 ```ts
 import { ShadeSessionManager } from '@shade/core';
@@ -117,20 +193,43 @@ const manager = new ShadeSessionManager(
 await manager.initialize();
 ```

+### At-rest encryption (V3.2)
+
+```ts
+import { KeyManager, EncryptedSQLiteStorage } from '@shade/storage-encrypted';
+
+const km = await KeyManager.open({
+  kind: 'passphrase',
+  passphrase: process.env.SHADE_STORAGE_PASSPHRASE!,
+  salt: loadSaltFromDisk(),
+});
+const storage = await EncryptedSQLiteStorage.open({
+  dbPath: '/data/shade-client.db',
+  keyManager: km,
+});
+```
+
+To migrate an existing 0.3.x SQLite DB in place:
+
+```bash
+shade migrate-storage \
+  --key-source passphrase \
+  --passphrase "$SHADE_STORAGE_PASSPHRASE" \
+  --salt-file /data/shade-client.db.salt
+```
+
 ## Architecture — keys vs. payloads

-Shade splits the network into two planes. The **prekey server** only
-sees public keys; **everything else** rides the encrypted Double
-Ratchet between peers. If you remember nothing else from this README,
-remember this picture:
+Shade splits the network into a **public-key plane** (the prekey
+server) and an **encrypted plane** (everything else). The prekey
+server only sees public key material. If you remember nothing else
+from this README, remember this picture:

 ```
-              Shade Prekey Server (Hono, public keys only)
+              Shade Prekey Container (Hono — public keys only)
                           │
-                  POST /v1/keys/register   (signed)
-                  GET  /v1/keys/bundle/:address
-                  POST /v1/keys/replenish  (signed)
-                  DELETE /v1/keys/:address (signed)
+        /v1/keys/*     /v1/inbox/*     /v1/bridge/*     /v1/transfer/*
+        /v1/kt/*       /metrics       /healthz   /ready
                           │
        ┌──────────────────┴──────────────────┐
        │                                     │
@@ -142,11 +241,20 @@ remember this picture:
        │◄── Double Ratchet messages ────────►│   ← end-to-end,
        │   (ratchet 0x02 / chunks 0x11)      │     never on the
        │                                     │     prekey server
-        │◄── @shade/transfer chunks ─────────►│
-        │   POST /v1/transfer/:id/chunk        │   ← peer-to-peer
-        │   GET  /v1/transfer/:id/state        │     HTTP, opaque
-        │                                     │     ciphertext
+        │
+        │◄── @shade/transfer chunks ─────────►│   ← peer-to-peer
+        │   POST /v1/transfer/:id/chunk        │     HTTP, opaque
+        │   GET  /v1/transfer/:id/state        │     ciphertext
+        │
+        │◄── @shade/inbox blobs (offline) ───►│   ← TTL-bound
+        │   POST /v1/inbox/:address            │     ciphertext-only
+        │   POST /v1/inbox/:address/fetch      │     relay
+        │
+        │◄── @shade/transport-webrtc ────────►│   ← optional P2P
+        │   RTCDataChannel `shade-transfer/v1` │     `MultiTransportFallback`
+        │                                     │     auto-demotes to HTTP
    SQLiteStorage / PostgresStorage       SQLiteStorage / PostgresStorage
+    + EncryptedSQLiteStorage (V3.2)       + EncryptedSQLiteStorage (V3.2)
        (private keys + sessions)             (private keys + sessions)
 ```

@@ -155,17 +263,23 @@ remember this picture:
 - Identity public keys (Ed25519 + X25519)
 - Signed prekeys + one-time prekey bundles
 - Registration / replenish / delete writes, all Ed25519-signed
- Operator-only metrics, `/health`, and the optional observer
-  dashboard
+- (V3.6) Inbox ciphertext blobs with TTL — same container, separate
+  routes; the relay only sees `address || msgId || ciphertext-bytes`
+- (V3.7) Bridge transports (SSE / long-poll / WS) — also delivered by
+  the same Hono app for clients that can't hold a WebSocket
+- (V3.12, opt-in) KT inclusion proofs + signed tree heads on
+  `/v1/kt/*` — verifiable distribution
+- Operator-only metrics and the optional observer dashboard

 ### What does **not** go via the prekey server

 - **Message plaintext, ever.** Encrypted ratchet envelopes flow peer-
  to-peer over whatever transport you choose (HTTP, WebSocket, your
-  own broker).
+  own broker, or the inbox relay above — which carries ciphertext only).
 - **File chunks.** `@shade/transfer` POSTs ciphertext directly to the
  receiver's `/v1/transfer/:streamId/chunk` route — the prekey server
-  is not involved.
+  is not involved. With V3.11 + `configureWebRTC()`, chunks ride
+  `RTCDataChannel` peer-to-peer; the relay is bypassed entirely.
 - **Identity private keys.** They never leave the device's storage.
 - **Filesystem RPC.** `@shade/files` rides the Double Ratchet for
  control + small payloads, then promotes to direct `@shade/transfer`
@@ -176,6 +290,8 @@ remember this picture:
 The prekey server is metadata-bearing (see `THREAT-MODEL.md § 2`):
 it sees who registers, who fetches whose bundle, and when. It does
 **not** see message contents, transfer contents, or session state.
+**V3.12 Key Transparency** (opt-in) makes its bundle distribution
+*verifiable* so a malicious server that swaps a bundle is caught.

 For the full threat model and mitigations, read
 [THREAT-MODEL.md](./THREAT-MODEL.md). For deployment-time guarantees,
@@ -183,53 +299,84 @@ read [docs/PRODUCTION-CHECKLIST.md](./docs/PRODUCTION-CHECKLIST.md).

 ## Packages

+All packages publish in lockstep at `4.0.0`.
+
 | Package | Purpose |
 |---------|---------|
-| `@shade/core` | Protocol logic (X3DH, Double Ratchet, session manager, errors, events) |
+| `@shade/core` | Protocol logic (X3DH, Double Ratchet, sender keys, session manager, errors, events) |
+| `@shade/proto` | Compact binary wire format (`0x01` PreKeyMessage, `0x02` RatchetMessage, `0x11` StreamChunk) |
 | `@shade/crypto-web` | SubtleCrypto + @noble/curves provider, in-memory storage. Includes the V3.8 Web Workers entrypoint (`@shade/crypto-web/worker`) — drop-in `WorkerCryptoProvider` plus `createEncryptStream` / `createDecryptStream` TransformStream factories |
-| `@shade/storage-sqlite` | Persistent SQLite storage (zero-config, bun:sqlite) |
-| `@shade/storage-postgres` | PostgreSQL storage with Drizzle for shared databases |
-| `@shade/server` | Prekey server (Hono routes, auth, rate limit, health, metrics) |
-| `@shade/transport` | HTTP + WebSocket transport wrappers with auto-encryption |
+| `@shade/observability` | OpenTelemetry-shaped event bus consumed by `@shade/observer`, server hooks, and the dashboard |
+| `@shade/keychain` | OS keychain bindings (libsecret / Keychain / Credential Manager) used by `@shade/storage-encrypted` and the CLI |
+| `@shade/key-transparency` | Key Transparency (V3.12) — RFC 6962-style append-only Merkle log, address-index commitment, signed tree heads, and a `LightWitness` for split-view detection. Opt-in on both server and client. |
+| `@shade/storage-sqlite` | Persistent SQLite storage (zero-config, `bun:sqlite`); also ships `SqliteInboxStore` |
+| `@shade/storage-postgres` | PostgreSQL storage with Drizzle for shared databases; also ships `PostgresInboxStore` + `PostgresKTLogStore` |
+| `@shade/storage-encrypted` | At-rest encryption (V3.2) — `EncryptedSQLiteStorage` / `EncryptedPostgresStorage`, `KeyManager`, online re-key |
+| `@shade/streams` | Multi-lane chunk encryption — HKDF-derived per-lane keys, deterministic AES-GCM nonces, streaming SHA-256, file metadata + thumbnails (V3.9) |
+| `@shade/transport` | HTTP + WebSocket transport wrappers with auto-encryption; KT-verifying `fetchBundleVerified` |
 | `@shade/transport-bridge` | WS → SSE → long-poll fallback chain (V3.7) — single `IncomingMessage` shape across transports for clients that can't keep a WebSocket open |
 | `@shade/transport-webrtc` | V3.11 P2P chunk transport via `RTCDataChannel`. Plugs into `@shade/transfer` as an `ITransferTransport`; signaling rides Shade's own ratchet. Memory factory + native (`globalThis.RTCPeerConnection`) factory included; `MultiTransportFallback([webrtc, http])` wired automatically when `shade.configureWebRTC()` is called. |
-| `@shade/proto` | Compact binary wire format (smaller than JSON) |
-| `@shade/streams` | Multi-lane chunk encryption — HKDF-derived per-lane keys, deterministic AES-GCM nonces, streaming SHA-256 |
-| `@shade/transfer` | Transfer engine on top of streams: parallel lanes, resume, HTTP + WS transport with auto-fallback, integrity verification |
-| `@shade/files` | Typed E2EE filesystem RPC — list/stat/mkdir/delete/move/read/write/getThumbnail + custom ops, auto inline/streams routing, production hooks (rate limit, retention, fingerprint gate, metrics), React hooks |
-| `@shade/recovery` | Social key recovery (V3.10) — Shamir-split identity to `n` guardians; threshold-many `k` reconstruct on a new device. AES-GCM-authenticated reconstruction; OOB-fingerprint gate per guardian release |
-| `@shade/key-transparency` | Key Transparency (V3.12) — RFC 6962-style append-only Merkle log, address-index commitment, signed tree heads, and a `LightWitness` for split-view detection. Opt-in on both server and client. See [docs/key-transparency.md](./docs/key-transparency.md) |
+| `@shade/server` | Prekey server (Hono routes, auth, rate limit, health, metrics). `createPrekeyServerWithKT(...)` opts into V3.12 KT mode |
 | `@shade/inbox-server` | Async store-and-forward relay (V3.6) — Hono routes, signed PUT/FETCH/DELETE, per-recipient TTL + quota, idempotent on `(address, msgId)`. Bundles into the same standalone container as the prekey server |
 | `@shade/inbox` | Inbox client + durable outgoing queue + receive cursor + push-trigger hook (`onMessageQueued`); composes on top of `Shade.send`/`Shade.receive` for offline-recipient delivery |
+| `@shade/transfer` | Transfer engine on top of streams: parallel lanes, resume, HTTP + WS transport with auto-fallback, `MultiTransportFallback` (N-ary demotion), integrity verification |
+| `@shade/files` | Typed E2EE filesystem RPC — list/stat/mkdir/delete/move/read/write/getThumbnail + custom ops, auto inline/streams routing, production hooks (rate limit, retention, fingerprint gate, metrics), React hooks under `@shade/files/react` |
+| `@shade/recovery` | Social key recovery (V3.10) — Shamir-split identity to `n` guardians; threshold-many `k` reconstruct on a new device. AES-GCM-authenticated reconstruction; OOB-fingerprint gate per guardian release |
 | `@shade/observer` | Live debugger backend (snapshot, SSE, dashboard) — see [README](./packages/shade-observer/README.md) |
-| `@shade/widgets` | Embeddable React widgets including transfer uploader/downloader — see [README](./packages/shade-widgets/README.md) |
 | `@shade/dashboard` | Standalone dashboard SPA bundled into the observer |
-| `@shade/sdk` | High-level wrapper with `createShade()` one-liner, auto-publish, auto-establish, auto-replenish, `Shade.files` namespace |
-| `@shade/cli` | `shade init` scaffolder + utilities (fingerprint, rotate, peer, dashboard, doctor) |
+| `@shade/sdk` | High-level wrapper with `createShade()` one-liner, auto-publish, auto-establish, auto-replenish, `Shade.files` namespace, fingerprint gates, KT integration, WebRTC opt-in |
+| `@shade/widgets` | Embeddable React widgets — fingerprint compare/gate, recovery setup/request/approve, transfer uploader/downloader, observer panels |
+| `@shade/cli` | `shade init` scaffolder + utilities (fingerprint, rotate, peer, dashboard, doctor, backup, migrate-storage, rotate-storage-key) |

 ## Shade as a modular toolkit

-Shade is split into packages so each project can depend on **only what it needs**—encrypted messaging, file transfer, prekey hosting, or lower-level building blocks. You do not need one giant stack for every use case.
+Shade is split into packages so each project can depend on **only what it needs**—encrypted messaging, file transfer, prekey hosting, social recovery, KT verification, or lower-level building blocks. You do not need one giant stack for every use case.

 For a **plain-language map** (which packages to add, what the prekey server does vs your own wiring, and where to start in code), see **[docs/SHADE-BY-SCENARIO.md](./docs/SHADE-BY-SCENARIO.md)**.

 ## Publishing

-All packages publish to a self-hosted Gitea npm registry on `gt.zyon.no`.
+All packages publish to a self-hosted Gitea npm registry on `gt.zyon.no`. The Docker image of the standalone container ships at `gt.zyon.no/stian/shade-prekey:<tag>`.

 ```bash
 # Bump all packages in lockstep
-bun run version 1.1.0
+bun run version 4.0.1

-# Dry-run (pack all tarballs without publishing)
+# Dry-run (pack all tarballs without publishing) — no token required
 bun run publish:dry

-# Real publish (requires GITEA_TOKEN env var)
+# Real publish — interactive (prompts for GITEA_TOKEN, checks
+# registry for conflicts, publishes via scripts/publish-all.ts)
 bun run publish:all

-# Or via CI: push a git tag v1.1.0 and .gitea/workflows/publish.yml runs
+# Build + push the standalone Docker image
+bun run scripts/build-docker.ts -- --tag 4.0.1 --push
 ```

+The interactive `scripts/publish-shade.sh` is the human entrypoint;
+`scripts/publish-all.ts` is the headless variant used by CI and
+`publish:dry`. They share a single `PACKAGES` list (24 entries at
+4.0.0) so the two flows can never drift.
+
+## Soak / GA-stable
+
+Before tagging `4.0.0` as `latest` and recommending production
+upgrades, run the combined soak harness for ≥ 2 weeks:
+
+```bash
+# Full GA window (V4.0 §Soak): 14 days × 24 hours
+bun run soak --hours 336
+
+# Smoke (~3 minutes — ratchet ping-pong, integrity check)
+bun run soak:smoke
+```
+
+The harness fans out N concurrent ratchet pairs, ping-pongs at
+~400 ops/sec/pair, and reports cumulative counters every minute.
+Any exception in any pair is captured and re-raised at shutdown so
+silent failures cannot hide. Wrap it in `systemd-run --user`,
+`nohup`, or a Gitea scheduled job for the actual 2-week window.
+
 ## Security properties

 | Property | Description |
@@ -242,27 +389,57 @@ bun run publish:all
 | **Memory zeroization** | Key material is zeroed after use (best-effort in JS) |
 | **Identity verification** | Safety numbers (60 digits) for out-of-band comparison |
 | **Identity rotation** | 7-day grace period for old sessions during rotation |
+| **At-rest encryption** *(V3.2, opt-in)* | AES-256-GCM under per-(table, column) DEKs; AAD binds `(table, column, pk)`; passphrase / OS keychain / app-injected master key; online re-key |
+| **Fingerprint gates** *(V3.3)* | `Shade.beforeFirstLargeFile / beforeBackupImport / beforeNewDeviceTrust` raise `FingerprintNotVerifiedError` on the operations that matter; defaults to TOFU + warning when no gate is registered |
+| **Async store-and-forward** *(V3.6)* | Relay only sees `address || msgId || ciphertext`; idempotent PUT; signed FETCH/ACK; TTL-bounded |
+| **Web-Worker key isolation** *(V3.8)* | Lane keys, identity keys, and ratchet chain keys live inside a dedicated worker; main thread only ferries plaintext via transferable buffers; idle terminate releases worker memory |
+| **Social key recovery** *(V3.10)* | Shamir over GF(2^8); AEAD-authenticated reconstruction (forged shares fail); guardian-side fingerprint gate before share release |
+| **WebRTC P2P transport** *(V3.11)* | Same Double Ratchet authenticates SDP/ICE signaling; chunk frames AEAD-bound to `streamId/laneId/seq`; deterministic glare resolution; `MultiTransportFallback` auto-demotes to HTTP |
 | **Key Transparency** *(V3.12, opt-in)* | Append-only Merkle log + signed tree heads + witness gossip — split-view and history-rewrite are detected by clients |

 ## Documentation

+**Operator + integrator**
+
 - [docs/SHADE-BY-SCENARIO.md](./docs/SHADE-BY-SCENARIO.md) — **Modular toolkit**: pick packages by scenario (messages, files, browser, ops)
 - [docs/PRODUCTION-CHECKLIST.md](./docs/PRODUCTION-CHECKLIST.md) — Pre-flight gates for going to production
+- [docs/DEPLOYMENT.md](./docs/DEPLOYMENT.md) — Full deployment guide (Docker, env vars, PostgreSQL, backup, Dokploy)
+- [docs/ROADMAP.md](./docs/ROADMAP.md) — V3.x → 4.0 GA → V5.0 trajectory
+
+**Per-surface deep-dives**
+
 - [docs/files.md](./docs/files.md) — `@shade/files` API + design (filesystem RPC, custom ops, hooks, React)
- [docs/recovery.md](./docs/recovery.md) — `@shade/recovery` social key recovery (V3.10): Shamir setup, guardian-side gates, threshold tuning
 - [docs/streams.md](./docs/streams.md) — `@shade/streams` + `@shade/transfer` deep dive (incl. hardening + retention)
 - [docs/inbox.md](./docs/inbox.md) — `@shade/inbox` + `@shade/inbox-server` async store-and-forward relay (V3.6)
 - [docs/transport.md](./docs/transport.md) — `@shade/transport-bridge` SSE / long-poll / WS bridge layer (V3.7)
+- [docs/web-workers.md](./docs/web-workers.md) — V3.8 Web Workers crypto: setup, bundler recipes (Vite/Webpack/Rollup), Safari notes, lifecycle, threat-model
+- [docs/recovery.md](./docs/recovery.md) — `@shade/recovery` social key recovery (V3.10): Shamir setup, guardian-side gates, threshold tuning
 - [docs/webrtc.md](./docs/webrtc.md) — `@shade/transport-webrtc` P2P transport (V3.11): NAT-traversal, TURN config, glare resolution, wire format, multi-fallback wiring
 - [docs/key-transparency.md](./docs/key-transparency.md) — `@shade/key-transparency` (V3.12): operator + client onboarding, witness role, recovery procedures
- [docs/V3.12-DESIGN.md](./docs/V3.12-DESIGN.md) — V3.12 design notat (threat model, RFC 6962 vs CONIKS choice, freshness model, open-questions resolution)
- [docs/web-workers.md](./docs/web-workers.md) — V3.8 Web Workers crypto: setup, bundler recipes (Vite/Webpack/Rollup), Safari notes, lifecycle, threat-model
+- [docs/storage-encryption.md](./docs/storage-encryption.md) — V3.2 at-rest encryption: design, key sources, rotation
+- [docs/trust-ux.md](./docs/trust-ux.md) — V3.3 fingerprint gates: when each fires, handler patterns, widget integration
+- [docs/observability.md](./docs/observability.md) — V3.4 event bus + dashboard
+- [docs/cross-platform.md](./docs/cross-platform.md) — V3.5 Android parity + cross-platform vector regime
+
+**Threat model + audit**
+
 - [SECURITY.md](./SECURITY.md) — Reporting vulnerabilities, security policy, threat-/test-matrix
- [THREAT-MODEL.md](./THREAT-MODEL.md) — Honest threat model and assumptions
+- [THREAT-MODEL.md](./THREAT-MODEL.md) — Honest threat model and assumptions (12 numbered sections + residual-risks table)
+- [docs/audit/REVIEW-BUNDLE.md](./docs/audit/REVIEW-BUNDLE.md) — External crypto-review entrypoint (scope, build instructions, reporting)
+- [docs/audit/SCOPE.md](./docs/audit/SCOPE.md) — One-page audit-scope summary
+
+**Migration + history**
+
+- [MIGRATION.md](./MIGRATION.md) — How to replace existing crypto with Shade + the [0.3.x → 4.0 upgrade path](./MIGRATION.md#migrating-from-03x-to-40-ga)
+- [CHANGELOG.md](./CHANGELOG.md) — `4.0.0` GA section + every prior release
+- [docs/archive/](./docs/archive/) — V2.1 / V2.2 / V2.3 backlog and V3.1 → V3.12 implementation plans (all `Status: Done`)
+- [docs/V5.0.md](./docs/V5.0.md) — Voice / Video / Broadcast (post-GA, built on the frozen 4.0 stack)
+
+**Examples**
+
 - [examples/](./examples/) — Runnable example applications, including
  [`07-streams-upload`](./examples/07-streams-upload) (multi-lane file transfer)
  and [`08-files-browser`](./examples/08-files-browser) (filesystem RPC)
- [MIGRATION.md](./MIGRATION.md) — How to replace existing crypto with Shade

 ## Deployment — one container per project

@@ -274,15 +451,30 @@ docker run -d \
  -v my-project-shade:/data \
  -p 3900:3900 \
  -e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
-  gt.zyon.no/stian/shade-prekey:latest
+  gt.zyon.no/stian/shade-prekey:4.0.0
 ```

 The container includes:
+
 - **Prekey server** — `/v1/keys/*` REST API
- **Observer dashboard** — `/shade-observer/dashboard/` (off unless token is set)
+- **Inbox relay (V3.6)** — `/v1/inbox/*` async store-and-forward; enable
+  with `SHADE_INBOX_DB_PATH=/data/inbox.db` (or `SHADE_INBOX_PG_URL`).
+  `SHADE_INBOX_PRUNE_INTERVAL_MINUTES` controls TTL prune cadence.
+- **Bridge transports (V3.7)** — `/v1/bridge/{stream,poll,ws}` SSE +
+  long-poll + WS adapters for clients that can't keep a WebSocket open.
+- **Transfer routes** — `/v1/transfer/*` chunk + state + control routes
+  for `@shade/transfer`.
+- **Key Transparency (V3.12)** — `/v1/kt/*` exposes `log_id`, latest +
+  historical STH, and consistency proofs. Enable with
+  `SHADE_KT_*` env vars; off by default.
+- **Observer dashboard** — `/shade-observer/dashboard/` (off unless
+  `SHADE_OBSERVER_TOKEN` is set)
 - **OpenAPI spec** — `/openapi.yaml` and interactive `/docs` viewer
+  (covers all 27 routes — prekey, inbox, bridge, transfer, KT,
+  observer, `/metrics`, `/healthz`, `/ready`)
 - **Prometheus metrics** — `/metrics`
- **Health check** — `/health`
+- **Health probes** — `/health` (full), `/healthz` (liveness),
+  `/ready` (readiness)
 - **Stale cleanup** — purges inactive identities automatically

 See [docs/DEPLOYMENT.md](./docs/DEPLOYMENT.md) for the full deployment guide, environment variables, PostgreSQL config, backup strategy, and Dokploy instructions.