diff --git a/CHANGELOG.md b/CHANGELOG.md index b4b9b1a..71ef701 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,60 @@ All notable changes to Shade are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [4.11.0] — 2026-05-15 — Streaming Double-Ratchet sub-sessions + +Answers Vyvern FR `shade-ws-streaming-ratchet.md` (the last Phase-2 +blocker) with a first-class streaming-session API rather than the +"documented contract" fallback: the Double-Ratchet crypto was already +safe for high-frequency one-directional use; the `send`/`receive` +*wrapper* was not (a `saveSession` keystore write per frame; a shared +per-peer mutex + single stored session row coupling any reuse to the +HTTP path). `ShadeStream` keeps the proven ratchet and fixes the +wrapper. + +**`@shade/core`** + +- New `stream.ts`: `deriveStreamRootKey` (identity-bound 3-DH — + X3DH-minus-prekeys, no prekey-server round trip; mutually + authenticated against the parent session's already-pinned + identities), `bootstrapStreamSession` (reuses + `initSenderSession`/`initReceiverSession` verbatim), and + `StreamRatchet` — an in-memory `seal`/`open`/`close` holder on its + own op-mutex, **never persisted**, zeroized on close. +- `ShadeSessionManager.beginStream` / `acceptStream` custody the + identity keys for the handshake without exposing private material; + both require an established parent session (no first-contact). +- New `StreamClosedError` / `StreamHandshakeError`; `stream.opened` / + `stream.closed` events. + +**`@shade/proto`** + +- Wire types `STREAM_OPEN` (0x31), `STREAM_OPEN_ACK` (0x32), + `STREAM_FRAME` (0x33) with encode/decode + `inspectEnvelopeType` + extension. A `STREAM_FRAME` carries one ratchet message via the exact + inner codec the HTTP path uses — one sealed frame ⇒ one WS frame. + +**`@shade/sdk`** + +- `Shade.openStream(peer)` / `Shade.acceptStream(peer, openBytes)` + returning `ShadeStream` (`handshakeFrame` / `handleHandshake` / + `seal` / `open` / `close`). Transport-agnostic like `send`/`receive`; + auto-establishes the parent session if missing. Independent of the + per-peer encrypt/decrypt queues and the stored parent session (R5). + Identical on the `sqlite:` server build and the IndexedDB browser + build (R4) — it touches no storage at all. + +**Security / perf** + +- Per-frame cost is exactly one symmetric KDF + one AES-GCM (no + keystore I/O) — strictly better than the budgeted "doubled CPU". + In-memory-only is a forward-secrecy property, not a shortcut; a + dropped stream is re-opened, never resumed. +- New `docs/streaming-sessions.md` (full R1–R7 contract); SECURITY.md + threat-matrix rows added with tests + (`packages/shade-core/tests/stream.test.ts`, + `packages/shade-proto/tests/stream-wire.test.ts`). + ## [Unreleased — 2026-05-09] — Android: V4.9/V4.10 ports + KeystoreStorage adapter The Kotlin side of the v4.10 cross-host approval routing FR. With this diff --git a/SECURITY.md b/SECURITY.md index 19c0fb0..b2ca0ec 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -100,6 +100,9 @@ matrix row in the same change. | § 1 Network attacker — replay window | ±5 min `signedAt` enforcement | `packages/shade-server/tests/server.test.ts` (`"rejects registration with stale signedAt"`) | | § 1 Network attacker — header AAD | Ratchet headers bound to ciphertext | `packages/shade-core/tests/ratchet.test.ts`, `packages/shade-streams/tests/tamper.test.ts`, `packages/shade-streams/tests/aead.test.ts` | | § 1 Network attacker — forward secrecy | DH ratchet step + chain-key zeroize | `packages/shade-core/tests/ratchet.test.ts`, `packages/shade-crypto-web/tests/hardening.test.ts` | +| § 1 Network attacker — streaming sub-session FS/replay (V4.11) | Per-frame Double-Ratchet `seal`/`open`; counter-rewind & replay rejected; in-memory-only (never persisted) | `packages/shade-core/tests/stream.test.ts` (`"R1: replayed / rewound frame is rejected"`, `"R2/R3: long one-directional burst stays correct and memory-bounded"`) | +| § 1 Network attacker — streaming handshake auth (V4.11) | Identity-bound 3-DH against parent-session-pinned identities | `packages/shade-core/tests/stream.test.ts` (`"handshake is mutually authenticated against pinned identities"`) | +| § 3 Endpoint compromise — streaming sub-session isolation (V4.11) | Stream ratchet derived without touching the stored parent session; zeroized on close | `packages/shade-core/tests/stream.test.ts` (`"R5: opening/using/closing a stream never touches the parent session"`, `"close() zeroizes and blocks further use; idempotent"`) | | § 2 Compromised prekey server — public-only storage | Prekey store never accepts a private key | `packages/shade-server/tests/server.test.ts`, `packages/shade-storage-sqlite/tests/sqlite-prekey-store.test.ts` | | § 2 Compromised prekey server — signed replenish/delete | Per-identity Ed25519 signature | `packages/shade-server/tests/server.test.ts` | | § 2 Compromised prekey server — fake-bundle detection | Out-of-band fingerprint comparison | `packages/shade-core/tests/fingerprint-session.test.ts` | diff --git a/docs/streaming-sessions.md b/docs/streaming-sessions.md new file mode 100644 index 0000000..dc43654 --- /dev/null +++ b/docs/streaming-sessions.md @@ -0,0 +1,128 @@ +# Streaming Double-Ratchet sub-sessions (V4.11) + +`ShadeStream` wraps individual frames on a long-lived, high-frequency, +often one-directional channel (e.g. a server→client console-log +WebSocket) in an **independent** Double Ratchet derived from — but never +mutating — an already-established parent Shade session. + +This is the answer to Vyvern FR `shade-ws-streaming-ratchet.md`. It is a +first-class API, *not* the "documented contract that `send`/`receive` is +safe per-frame" fallback: the Double-Ratchet crypto was already safe for +that access pattern, but the `send`/`receive` wrapper layer was not +(per-frame keystore writes; a shared per-peer mutex and a single stored +session row coupling the stream to the HTTP path). `ShadeStream` keeps +the proven ratchet and fixes the wrapper. + +## API + +Transport-agnostic, exactly like `send`/`receive`: it emits/consumes +wire bytes; you own the WebSocket. + +```ts +// Initiator (the side that calls openStream) +const stream = await shade.openStream(peerAddr); +ws.send(stream.handshakeFrame()); // → STREAM_OPEN +// first inbound WS frame is the peer's STREAM_OPEN_ACK: +await stream.handleHandshake(ackBytes); // stream now usable +ws.send(await stream.seal(utf8(logLine))); // outbound frame +onLog(await stream.open(inboundBytes)); // inbound frame +await stream.close(); // on ws close/error + +// Responder +const stream = await shade.acceptStream(peerAddr, openBytes); // usable now +ws.send(stream.handshakeFrame()); // → STREAM_OPEN_ACK +// open()/seal() as above +``` + +Route inbound bytes with `inspectEnvelopeType()`: +`'stream-open' | 'stream-open-ack' | 'stream-frame'`. + +## Seeding (no prekey-server round trip) + +The stream root key is derived from an identity-bound **3-DH** exchange +— the X3DH pattern minus signed/one-time prekeys, because the peer's +identity is *already* mutually pinned by the parent session's TOFU. Two +ephemerals are exchanged inside the transport (`STREAM_OPEN` / +`STREAM_OPEN_ACK`); no prekey server is involved. + +``` +slotA = DH(initiatorEphemeral, responderIdentity) — authenticates responder +slotB = DH(initiatorIdentity, responderEphemeral) — authenticates initiator +slotC = DH(initiatorEphemeral, responderEphemeral) — ephemeral forward secrecy +SK = HKDF(ikm = slotA‖slotB‖slotC, salt = streamId, info = "ShadeStream/v1") +``` + +Both peers compute the identical three scalars regardless of role. +`SK` then bootstraps a textbook Double Ratchet by handing the +responder's ephemeral to `initSenderSession`/`initReceiverSession` +exactly the way X3DH hands its signed prekey to the ratchet — so +`ratchetEncrypt`/`ratchetDecrypt` and every guarantee they carry apply +unchanged. + +## Security contract (answers FR R1–R7) + +- **R1 — same properties as `send`/`receive`.** Each frame is one + `ratchetEncrypt`/`ratchetDecrypt` over the *same* crypto as the HTTP + path: AES-256-GCM confidentiality, per-frame forward secrecy via the + one-way HMAC chain-key KDF with in-place zeroize of the spent chain + key, and replay/rewind rejection (a re-delivered or counter-rewound + frame fails closed). The handshake is mutually authenticated against + the identities the parent session already pinned. +- **R2 — one-directional resilience.** A long server→client burst with + no client traffic only advances the symmetric sending chain (no DH + step until the peer replies — standard Double Ratchet). Forward + secrecy holds per frame in this regime. Over an ordered transport + (WebSocket/TCP) zero keys are skipped per frame. +- **R3 — bounded memory.** Out-of-order arrivals are capped by the + ratchet's `MAX_SKIP` (1000) and `MAX_CACHED_SKIPPED_KEYS` (2000) + with oldest-key eviction. In-order delivery retains nothing. Verified + to stay at zero retained keys across a 5000-frame burst. +- **R4 — browser parity.** Identical API and guarantees in the browser + SDK: `ShadeStream` is on the same `Shade` class over the same + `CryptoProvider` (`SubtleCryptoProvider`), so the IndexedDB-backed + build behaves identically to the `sqlite:` server build. No storage + is touched at all (see R7), so the keystore backend is irrelevant. +- **R5 — independent lifecycle.** The stream ratchet is derived without + reading or writing the stored parent `SessionState`, runs on its own + private op-mutex (not the per-peer `send`/`receive` queues), and is + zeroized on `close()`. Opening, using for thousands of frames, and + closing a stream leaves the parent session byte-identical; the HTTP + path keeps working concurrently against the same peer. Each + `openStream` gets a fresh `streamId` and an independent root, so + concurrent streams to one peer never share key material. +- **R6 — wire framing.** `@shade/proto` defines `STREAM_OPEN` (0x31), + `STREAM_OPEN_ACK` (0x32), `STREAM_FRAME` (0x33). A `STREAM_FRAME` + carries one Double-Ratchet message via the exact ratchet inner codec + the HTTP path uses. One sealed logical frame ⇒ one self-delimiting + wire frame ⇒ one WS text/binary frame. +- **R7 — performance.** The ratchet lives **only in memory and is never + persisted**. There is therefore *zero* per-frame storage I/O — the + per-frame cost is exactly the symmetric KDF + one AES-GCM, the same + primitives the HTTP path runs. This is strictly better than the + "doubled CPU" the Vyvern roadmap budgeted, because the dominant cost + the naive `send`/`receive`-per-frame approach would have paid (a + `saveSession` keystore write per frame) is eliminated, not doubled. + Not persisting is also a *security* property, not a shortcut: writing + evolving per-frame ratchet secrets to disk would defeat forward + secrecy. A dropped/reconnected stream is re-opened with a fresh + handshake, never resumed. + +## Double-Ratchet ordering note + +A responder cannot `seal()` until it has `open()`ed at least one frame +from the initiator (standard Signal behaviour — the responder has no +sending chain until the first DH step). For a server-heavy stream +either make the bursty data sender the **initiator**, or have the +initiator send one priming frame immediately after the handshake. + +## Tests + +- `packages/shade-core/tests/stream.test.ts` — handshake agreement, + frame round-trips, 5000-frame one-directional burst (bounded skipped + keys + forward-secrecy zeroize), parent-session independence (R5), + replay/rewind rejection, mutual authentication against pinned + identities, `close()` zeroize/idempotence. +- `packages/shade-proto/tests/stream-wire.test.ts` — wire round-trips + and type-tag/length rejection for all three stream frame kinds. + + diff --git a/packages/shade-cli/package.json b/packages/shade-cli/package.json index 16d7950..f69f38b 100644 --- a/packages/shade-cli/package.json +++ b/packages/shade-cli/package.json @@ -1,6 +1,6 @@ { "name": "@shade/cli", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/cli.ts", "bin": { diff --git a/packages/shade-core/package.json b/packages/shade-core/package.json index 74083a9..27f9167 100644 --- a/packages/shade-core/package.json +++ b/packages/shade-core/package.json @@ -1,6 +1,6 @@ { "name": "@shade/core", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-core/src/errors.ts b/packages/shade-core/src/errors.ts index 973c2e5..95726cc 100644 --- a/packages/shade-core/src/errors.ts +++ b/packages/shade-core/src/errors.ts @@ -107,6 +107,31 @@ export class FingerprintNotVerifiedError extends ShadeError { } } +/** + * Thrown when `seal()` / `open()` is called on a {@link StreamRatchet} + * that has already been torn down via `close()`. The stream's ratchet + * secrets have been zeroized and cannot be revived — open a fresh + * stream instead. + */ +export class StreamClosedError extends ShadeError { + constructor(message = 'Stream is closed') { + super('SHADE_STREAM_CLOSED', message); + this.name = 'StreamClosedError'; + } +} + +/** + * Thrown when a stream handshake frame is malformed, arrives in the + * wrong order, or references a streamId that does not match the stream + * it was fed to. + */ +export class StreamHandshakeError extends ShadeError { + constructor(message = 'Stream handshake failed') { + super('SHADE_STREAM_HANDSHAKE', message); + this.name = 'StreamHandshakeError'; + } +} + // ─── Infrastructure Errors ─────────────────────────────────── export class NetworkError extends ShadeError { diff --git a/packages/shade-core/src/events.ts b/packages/shade-core/src/events.ts index ea2b886..bb64179 100644 --- a/packages/shade-core/src/events.ts +++ b/packages/shade-core/src/events.ts @@ -36,6 +36,10 @@ export interface ShadeEventMap { 'signed_prekey.rotated': { oldKeyId: number; newKeyId: number }; 'trust.pinned': { address: string; identityKeyHash: string }; 'trust.changed': { address: string; oldKeyHash: string; newKeyHash: string }; + /** V4.11 — a streaming sub-ratchet handshake completed. */ + 'stream.opened': { address: string; role: 'initiator' | 'responder' }; + /** V4.11 — a streaming sub-ratchet was torn down and zeroized. */ + 'stream.closed': { address: string }; } export type ShadeEventName = keyof ShadeEventMap; diff --git a/packages/shade-core/src/index.ts b/packages/shade-core/src/index.ts index 5649afd..ac91443 100644 --- a/packages/shade-core/src/index.ts +++ b/packages/shade-core/src/index.ts @@ -5,6 +5,7 @@ export * from './keys.js'; export * from './errors.js'; export * from './x3dh.js'; export * from './ratchet.js'; +export * from './stream.js'; export { ShadeSessionManager, GRACE_PERIOD_MS } from './session.js'; export * from './serialization.js'; export * from './fingerprint.js'; diff --git a/packages/shade-core/src/session.ts b/packages/shade-core/src/session.ts index 1d481ae..62649a5 100644 --- a/packages/shade-core/src/session.ts +++ b/packages/shade-core/src/session.ts @@ -23,7 +23,14 @@ import { ratchetEncrypt, ratchetDecrypt, } from './ratchet.js'; -import { NoSessionError } from './errors.js'; +import { + deriveStreamRootKey, + bootstrapStreamSession, + StreamRatchet, + STREAM_ID_BYTES, + STREAM_EPHEMERAL_BYTES, +} from './stream.js'; +import { NoSessionError, StreamHandshakeError } from './errors.js'; import { computeFingerprint, shortFingerprint } from './fingerprint.js'; import { ShadeEventEmitter, shortHash } from './events.js'; import { @@ -626,6 +633,121 @@ export class ShadeSessionManager { return dec.decode(plaintext); } + + // ─── Streaming sub-sessions (V4.11) ──────────────────────── + + /** + * Resolve the peer's pinned identity X25519 key for a stream + * handshake. Requires an *already established* parent session — the + * stream is explicitly a "second channel on a known peer", never a + * first contact (so it needs no prekey-server round trip and inherits + * the parent's TOFU pin). + */ + private async streamIdentityMaterial( + address: string, + ): Promise<{ selfIdentityDHPriv: Uint8Array; peerIdentityDHPub: Uint8Array }> { + if (!this.identity) throw new Error('Not initialized'); + const session = await this.storage.getSession(address); + if (!session) throw new NoSessionError(address); + return { + selfIdentityDHPriv: this.identity.dhPrivateKey, + peerIdentityDHPub: session.remoteIdentityKey, + }; + } + + /** + * Initiator side of a stream handshake. Generates the streamId and + * this side's ephemeral, and returns a `complete` continuation that + * derives the sub-ratchet once the responder's ephemeral arrives in + * the `STREAM_OPEN_ACK`. + * + * Touches neither the stored parent session nor the per-peer op + * queues (R5). + */ + async beginStream(address: string): Promise<{ + streamId: Uint8Array; + ephemeralPublicKey: Uint8Array; + complete: (peerEphemeralPub: Uint8Array) => Promise; + }> { + const { selfIdentityDHPriv, peerIdentityDHPub } = + await this.streamIdentityMaterial(address); + const streamId = this.crypto.randomBytes(STREAM_ID_BYTES); + const ephemeral = await this.crypto.generateX25519KeyPair(); + + const complete = async (peerEphemeralPub: Uint8Array): Promise => { + if (peerEphemeralPub.length !== STREAM_EPHEMERAL_BYTES) { + throw new StreamHandshakeError( + `responder ephemeral must be ${STREAM_EPHEMERAL_BYTES} bytes`, + ); + } + const sk = await deriveStreamRootKey( + this.crypto, + 'initiator', + streamId, + selfIdentityDHPriv, + peerIdentityDHPub, + ephemeral.privateKey, + peerEphemeralPub, + ); + const session = await bootstrapStreamSession(this.crypto, 'initiator', sk, peerIdentityDHPub, { + publicKey: peerEphemeralPub, + privateKey: new Uint8Array(0), + }); + this.crypto.zeroize(sk); + this.crypto.zeroize(ephemeral.privateKey); + this.events?.emit('stream.opened', { address, role: 'initiator' }); + return new StreamRatchet(this.crypto, session, streamId); + }; + + return { streamId, ephemeralPublicKey: ephemeral.publicKey, complete }; + } + + /** + * Responder side of a stream handshake. Given the initiator's + * `STREAM_OPEN` (its streamId + ephemeral), derives the sub-ratchet + * immediately and returns this side's ephemeral for the + * `STREAM_OPEN_ACK`. + */ + async acceptStream( + address: string, + streamId: Uint8Array, + initiatorEphemeralPub: Uint8Array, + ): Promise<{ ephemeralPublicKey: Uint8Array; stream: StreamRatchet }> { + if (streamId.length !== STREAM_ID_BYTES) { + throw new StreamHandshakeError(`streamId must be ${STREAM_ID_BYTES} bytes`); + } + if (initiatorEphemeralPub.length !== STREAM_EPHEMERAL_BYTES) { + throw new StreamHandshakeError( + `initiator ephemeral must be ${STREAM_EPHEMERAL_BYTES} bytes`, + ); + } + const { selfIdentityDHPriv, peerIdentityDHPub } = + await this.streamIdentityMaterial(address); + const ephemeral = await this.crypto.generateX25519KeyPair(); + + const sk = await deriveStreamRootKey( + this.crypto, + 'responder', + streamId, + selfIdentityDHPriv, + peerIdentityDHPub, + ephemeral.privateKey, + initiatorEphemeralPub, + ); + const session = await bootstrapStreamSession( + this.crypto, + 'responder', + sk, + peerIdentityDHPub, + ephemeral, + ); + this.crypto.zeroize(sk); + this.events?.emit('stream.opened', { address, role: 'responder' }); + return { + ephemeralPublicKey: ephemeral.publicKey, + stream: new StreamRatchet(this.crypto, session, streamId), + }; + } } function arraysEqual(a: Uint8Array, b: Uint8Array): boolean { diff --git a/packages/shade-core/src/stream.ts b/packages/shade-core/src/stream.ts new file mode 100644 index 0000000..cf2e210 --- /dev/null +++ b/packages/shade-core/src/stream.ts @@ -0,0 +1,233 @@ +import type { CryptoProvider } from './crypto.js'; +import type { KeyPair, RatchetMessage, SessionState } from './types.js'; +import { + initSenderSession, + initReceiverSession, + ratchetEncrypt, + ratchetDecrypt, +} from './ratchet.js'; +import { StreamClosedError } from './errors.js'; + +/** + * Streaming Double-Ratchet sub-sessions (V4.11). + * + * Wraps a long-lived, high-frequency, often-one-directional channel + * (e.g. a server→client WebSocket log burst) in an *independent* Double + * Ratchet that is derived from — but never mutates — an already + * established parent Shade session. + * + * Why a sub-ratchet rather than reusing `ShadeSessionManager`: + * + * - **Independence (R5).** A stream gets its own root key, chains, DH + * ratchet and op-mutex. Opening/closing it never touches the stored + * parent `SessionState` nor serialises against the HTTP send/receive + * queue. + * - **Performance (R7).** The stream ratchet lives only in memory and + * is *never* written to the keystore. There is therefore zero + * per-frame storage I/O — the cost is purely the symmetric KDF + + * AES-GCM, the same primitives the HTTP path uses. + * - **Forward secrecy.** Not persisting the evolving ratchet state is + * a feature, not a shortcut: writing per-frame secrets to disk would + * actively defeat the forward-secrecy guarantee. A dropped/reconnected + * stream is re-opened with a fresh handshake, not resumed. + * + * ## Seeding (no prekey-server round trip) + * + * The stream root key is derived from an identity-bound 3-DH exchange — + * the X3DH pattern minus the signed / one-time prekeys, because the + * peer's identity is *already* mutually pinned by the parent session's + * TOFU. Two ephemeral keys are exchanged inside the transport itself + * (`STREAM_OPEN` / `STREAM_OPEN_ACK`); no prekey server is involved. + * + * slotA = DH(initiatorEphemeral, responderIdentity) — auth of responder + * slotB = DH(initiatorIdentity, responderEphemeral) — auth of initiator + * slotC = DH(initiatorEphemeral, responderEphemeral) — ephemeral FS + * + * SK = HKDF(ikm = slotA‖slotB‖slotC, salt = streamId, info = "ShadeStream/v1") + * + * Both peers compute the identical three scalars regardless of role, so + * `SK` agrees. An attacker lacking the responder's identity private key + * cannot form slotA; one lacking the initiator's cannot form slotB — + * the handshake is therefore mutually authenticated against the same + * identities the parent session already trusts. + * + * `SK` then bootstraps a textbook Double Ratchet by handing the + * responder's ephemeral to {@link initSenderSession} / + * {@link initReceiverSession} exactly the way X3DH hands its signed + * prekey to the ratchet — so `ratchetEncrypt` / `ratchetDecrypt` (and + * thus every R1–R3 guarantee they already carry) apply unchanged. + */ + +export type StreamRole = 'initiator' | 'responder'; + +/** Stream identifier length (bytes). 128 bits of collision resistance. */ +export const STREAM_ID_BYTES = 16; + +/** Ephemeral X25519 public-key length carried in the handshake. */ +export const STREAM_EPHEMERAL_BYTES = 32; + +const STREAM_KDF_INFO = new TextEncoder().encode('ShadeStream/v1'); + +/** + * Derive the stream's independent root key from the identity-bound 3-DH + * exchange. Pure: never reads or mutates any `SessionState`. + * + * @param role which end of the handshake we are + * @param streamId 16-byte stream id (HKDF salt; binds the + * derivation so two concurrent streams to the + * same peer never share a root key) + * @param selfIdentityDHPriv our long-term identity X25519 private key + * @param peerIdentityDHPub peer's pinned identity X25519 public key + * (the value the parent session pinned) + * @param selfEphemeralPriv our per-stream ephemeral X25519 private key + * @param peerEphemeralPub peer's per-stream ephemeral X25519 public key + */ +export async function deriveStreamRootKey( + crypto: CryptoProvider, + role: StreamRole, + streamId: Uint8Array, + selfIdentityDHPriv: Uint8Array, + peerIdentityDHPub: Uint8Array, + selfEphemeralPriv: Uint8Array, + peerEphemeralPub: Uint8Array, +): Promise { + // Each slot is pinned to a fixed semantic (not to local role) so both + // sides feed HKDF the identical ikm: + // slotA = DH(initiatorEphemeral, responderIdentity) + // slotB = DH(initiatorIdentity, responderEphemeral) + // slotC = DH(initiatorEphemeral, responderEphemeral) + let slotA: Uint8Array; + let slotB: Uint8Array; + let slotC: Uint8Array; + if (role === 'initiator') { + slotA = await crypto.x25519(selfEphemeralPriv, peerIdentityDHPub); + slotB = await crypto.x25519(selfIdentityDHPriv, peerEphemeralPub); + slotC = await crypto.x25519(selfEphemeralPriv, peerEphemeralPub); + } else { + slotA = await crypto.x25519(selfIdentityDHPriv, peerEphemeralPub); + slotB = await crypto.x25519(selfEphemeralPriv, peerIdentityDHPub); + slotC = await crypto.x25519(selfEphemeralPriv, peerEphemeralPub); + } + + const ikm = new Uint8Array(96); + ikm.set(slotA, 0); + ikm.set(slotB, 32); + ikm.set(slotC, 64); + const sk = await crypto.hkdf(ikm, streamId, STREAM_KDF_INFO, 32); + + crypto.zeroize(slotA); + crypto.zeroize(slotB); + crypto.zeroize(slotC); + crypto.zeroize(ikm); + return sk; +} + +/** + * Bootstrap a fresh Double Ratchet `SessionState` from the derived + * stream root key. The responder's ephemeral plays exactly the role + * X3DH's signed prekey plays in {@link initSenderSession} / + * {@link initReceiverSession}, so the ratchet handoff is identical to + * the proven HTTP path. + * + * On the initiator only `responderEphemeral.publicKey` is needed; the + * responder must pass its full ephemeral keypair. + * + * `peerIdentityDHPub` is recorded as the session's `remoteIdentityKey` + * so stream fingerprints stay meaningful and consistent with the parent. + */ +export async function bootstrapStreamSession( + crypto: CryptoProvider, + role: StreamRole, + sk: Uint8Array, + peerIdentityDHPub: Uint8Array, + responderEphemeral: KeyPair, +): Promise { + if (role === 'initiator') { + // initSenderSession derives a fresh root via kdfRootKey and does not + // retain `sk`, so the caller may safely zeroize it afterwards. + return initSenderSession(crypto, sk, peerIdentityDHPub, responderEphemeral.publicKey); + } + // initReceiverSession stores the root key BY REFERENCE. Hand it an + // independent copy so the caller zeroizing its `sk` scratch buffer + // can't wipe the live session root. + return initReceiverSession(new Uint8Array(sk), peerIdentityDHPub, responderEphemeral); +} + +/** Zeroize every secret a stream session holds, then drop the chains. */ +function zeroizeSession(crypto: CryptoProvider, s: SessionState): void { + crypto.zeroize(s.rootKey); + if (s.sendChain.chainKey.length > 0) crypto.zeroize(s.sendChain.chainKey); + if (s.receiveChain && s.receiveChain.chainKey.length > 0) { + crypto.zeroize(s.receiveChain.chainKey); + } + if (s.dhSend.privateKey.length > 0) crypto.zeroize(s.dhSend.privateKey); + for (const mk of s.skippedKeys.values()) crypto.zeroize(mk); + s.skippedKeys.clear(); +} + +/** + * In-memory holder for a stream's Double Ratchet. Serialises its own + * `seal`/`open`/`close` on a private promise chain (independent of the + * SDK's per-peer encrypt/decrypt queues — R5) so per-frame ratchet + * mutations never interleave, while staying fully concurrent with the + * parent session and with other streams. + * + * Never persisted: the ratchet exists only for the lifetime of the + * stream and is zeroized on `close()`. + */ +export class StreamRatchet { + private session: SessionState | null; + private opChain: Promise = Promise.resolve(); + + constructor( + private readonly crypto: CryptoProvider, + session: SessionState, + /** 16-byte stream id this ratchet is bound to. */ + public readonly streamId: Uint8Array, + ) { + this.session = session; + } + + /** True once {@link close} has run; `seal`/`open` will throw. */ + get closed(): boolean { + return this.session === null; + } + + private run(fn: (s: SessionState) => Promise): Promise { + const next = this.opChain.catch(() => undefined).then(() => { + if (!this.session) throw new StreamClosedError(); + return fn(this.session); + }); + // Keep a never-rejecting tail so a failed frame doesn't poison the + // next one (a single bad inbound frame must not wedge the stream). + this.opChain = next.catch(() => undefined); + return next; + } + + /** Wrap one logical frame. Advances the sending chain by one step. */ + seal(plaintext: Uint8Array): Promise { + return this.run((s) => ratchetEncrypt(this.crypto, s, plaintext)); + } + + /** + * Unwrap one inbound frame. Correct and memory-bounded across long + * one-directional runs from the peer: ordered transport delivery + * skips zero keys per frame, and out-of-order arrivals are still + * capped by the ratchet's `MAX_SKIP` / `MAX_CACHED_SKIPPED_KEYS`. + */ + open(message: RatchetMessage): Promise { + return this.run((s) => ratchetDecrypt(this.crypto, s, message)); + } + + /** Zeroize and drop the ratchet. Idempotent. */ + close(): Promise { + return this.opChain + .catch(() => undefined) + .then(() => { + if (this.session) { + zeroizeSession(this.crypto, this.session); + this.session = null; + } + }); + } +} diff --git a/packages/shade-core/tests/stream.test.ts b/packages/shade-core/tests/stream.test.ts new file mode 100644 index 0000000..a3cae72 --- /dev/null +++ b/packages/shade-core/tests/stream.test.ts @@ -0,0 +1,176 @@ +import { describe, test, expect, beforeEach } from 'bun:test'; +import { SubtleCryptoProvider, MemoryStorage } from '@shade/crypto-web'; +import { + ShadeSessionManager, + StreamRatchet, + StreamClosedError, + DecryptionError, +} from '../src/index.js'; + +const crypto = new SubtleCryptoProvider(); +const enc = new TextEncoder(); +const dec = new TextDecoder(); + +/** + * Establish a *bidirectional* parent session: Alice→Bob X3DH, then one + * Alice→Bob message Bob decrypts so Bob also has a session for 'alice'. + * Both sides then hold the peer's pinned identity DH key — the input the + * stream handshake derives from. + */ +async function bidirectionalPair() { + const aliceStorage = new MemoryStorage(); + const bobStorage = new MemoryStorage(); + const alice = new ShadeSessionManager(crypto, aliceStorage); + const bob = new ShadeSessionManager(crypto, bobStorage); + await alice.initialize(); + await bob.initialize(); + + const otpks = await bob.generateOneTimePreKeys(4); + const bundle = await bob.createPreKeyBundle(); + bundle.oneTimePreKey = { keyId: otpks[0]!.keyId, publicKey: otpks[0]!.keyPair.publicKey }; + await alice.initSessionFromBundle('bob', bundle); + + const hello = await alice.encrypt('bob', 'parent-hello'); + expect(await bob.decrypt('alice', hello)).toBe('parent-hello'); + + return { alice, bob, aliceStorage, bobStorage }; +} + +/** Run the full STREAM_OPEN / STREAM_OPEN_ACK handshake between managers. */ +async function openStreamPair(alice: ShadeSessionManager, bob: ShadeSessionManager) { + const begun = await alice.beginStream('bob'); // initiator + const accepted = await bob.acceptStream('alice', begun.streamId, begun.ephemeralPublicKey); + const aliceStream = await begun.complete(accepted.ephemeralPublicKey); + return { aliceStream, bobStream: accepted.stream, streamId: begun.streamId }; +} + +describe('streaming sub-ratchet (V4.11)', () => { + let alice: ShadeSessionManager; + let bob: ShadeSessionManager; + let aliceStorage: MemoryStorage; + + beforeEach(async () => { + ({ alice, bob, aliceStorage } = await bidirectionalPair()); + }); + + test('both sides derive the same stream root (round-trips frames)', async () => { + const { aliceStream, bobStream } = await openStreamPair(alice, bob); + + // Initiator → responder (first frame triggers responder DH step). + const f1 = await aliceStream.seal(enc.encode('log line 1')); + expect(dec.decode(await bobStream.open(f1))).toBe('log line 1'); + + // Responder → initiator (now responder may seal). + const r1 = await bobStream.seal(enc.encode('command-response 1')); + expect(dec.decode(await aliceStream.open(r1))).toBe('command-response 1'); + }); + + test('two streams to the same peer get independent roots', async () => { + const s1 = await openStreamPair(alice, bob); + const s2 = await openStreamPair(alice, bob); + expect(s1.streamId).not.toEqual(s2.streamId); + + const a = await s1.aliceStream.seal(enc.encode('on stream 1')); + // A frame from stream 1 must not decrypt on stream 2's ratchet. + await expect(s2.bobStream.open(a)).rejects.toBeInstanceOf(DecryptionError); + // …but does on its own. + expect(dec.decode(await s1.bobStream.open(a))).toBe('on stream 1'); + }); + + test('R2/R3: long one-directional burst stays correct and memory-bounded', async () => { + const { aliceStream, bobStream } = await openStreamPair(alice, bob); + const N = 5000; + + // Capture a live receive-chain key buffer to prove forward secrecy: + // ratchetDecrypt zeroizes the previous chain key in place. + await bobStream.open(await aliceStream.seal(enc.encode('frame-0'))); + const bobSession = (bobStream as unknown as { session: { receiveChain: { chainKey: Uint8Array }; skippedKeys: Map } }).session; + const staleChainKey = bobSession.receiveChain.chainKey; + const staleCopy = staleChainKey.slice(); + expect(staleCopy.some((b) => b !== 0)).toBe(true); + + for (let i = 1; i < N; i++) { + const wire = await aliceStream.seal(enc.encode(`frame-${i}`)); + expect(dec.decode(await bobStream.open(wire))).toBe(`frame-${i}`); + } + + // In-order delivery ⇒ zero skipped keys retained across 5k frames. + expect(bobSession.skippedKeys.size).toBe(0); + // The chain key in use at frame 0 was overwritten (forward secrecy). + expect(staleChainKey.every((b) => b === 0)).toBe(true); + }); + + test('R5: opening/using/closing a stream never touches the parent session', async () => { + const before = await aliceStorage.getSession('bob'); + const snapshot = JSON.stringify({ + root: Array.from(before!.rootKey), + sendCtr: before!.sendChain.counter, + prevCtr: before!.previousSendCounter, + }); + + const { aliceStream, bobStream } = await openStreamPair(alice, bob); + for (let i = 0; i < 200; i++) { + await bobStream.open(await aliceStream.seal(enc.encode(`x${i}`))); + } + await aliceStream.close(); + await bobStream.close(); + + const after = await aliceStorage.getSession('bob'); + expect( + JSON.stringify({ + root: Array.from(after!.rootKey), + sendCtr: after!.sendChain.counter, + prevCtr: after!.previousSendCounter, + }), + ).toBe(snapshot); + + // Parent HTTP path still works after the stream lifecycle. + const env = await alice.encrypt('bob', 'after-stream'); + expect(await bob.decrypt('alice', env)).toBe('after-stream'); + }); + + test('R1: replayed / rewound frame is rejected', async () => { + const { aliceStream, bobStream } = await openStreamPair(alice, bob); + const f1 = await aliceStream.seal(enc.encode('once')); + expect(dec.decode(await bobStream.open(f1))).toBe('once'); + // Re-delivering the exact same sealed frame must fail. + await expect(bobStream.open(f1)).rejects.toBeInstanceOf(DecryptionError); + }); + + test('close() zeroizes and blocks further use; idempotent', async () => { + const { aliceStream, bobStream } = await openStreamPair(alice, bob); + await aliceStream.close(); + await aliceStream.close(); // idempotent + expect(aliceStream.closed).toBe(true); + await expect(aliceStream.seal(enc.encode('nope'))).rejects.toBeInstanceOf( + StreamClosedError, + ); + // The peer end is unaffected by our local close. + expect(bobStream.closed).toBe(false); + }); + + test('handshake is mutually authenticated against pinned identities', async () => { + // A third party (mallory) with its own identity cannot stand in for + // bob: alice derives against bob's pinned identity key, so a + // handshake completed with mallory's ephemeral yields a different + // root and frames fail to open. + const mStorage = new MemoryStorage(); + const mallory = new ShadeSessionManager(crypto, mStorage); + await mallory.initialize(); + // Give mallory a parent session label so acceptStream has identity + // material, but pinned to the WRONG (alice) identity vs what alice + // pinned for 'bob'. + const otpks = await mallory.generateOneTimePreKeys(2); + const mb = await mallory.createPreKeyBundle(); + mb.oneTimePreKey = { keyId: otpks[0]!.keyId, publicKey: otpks[0]!.keyPair.publicKey }; + await alice.initSessionFromBundle('mallory', mb); + const helo = await alice.encrypt('mallory', 'hi'); + await mallory.decrypt('alice', helo); + + const begun = await alice.beginStream('bob'); + const mAccept = await mallory.acceptStream('alice', begun.streamId, begun.ephemeralPublicKey); + const aliceStream = await begun.complete(mAccept.ephemeralPublicKey); + const frame = await aliceStream.seal(enc.encode('secret')); + await expect(mAccept.stream.open(frame)).rejects.toBeInstanceOf(DecryptionError); + }); +}); diff --git a/packages/shade-crypto-web/package.json b/packages/shade-crypto-web/package.json index 5ae22c5..a1ae887 100644 --- a/packages/shade-crypto-web/package.json +++ b/packages/shade-crypto-web/package.json @@ -1,6 +1,6 @@ { "name": "@shade/crypto-web", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-dashboard/package.json b/packages/shade-dashboard/package.json index f57984b..3b009d9 100644 --- a/packages/shade-dashboard/package.json +++ b/packages/shade-dashboard/package.json @@ -1,6 +1,6 @@ { "name": "@shade/dashboard", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "scripts": { "dev": "vite", diff --git a/packages/shade-files/package.json b/packages/shade-files/package.json index 78cd140..252ad77 100644 --- a/packages/shade-files/package.json +++ b/packages/shade-files/package.json @@ -1,6 +1,6 @@ { "name": "@shade/files", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-inbox-server/package.json b/packages/shade-inbox-server/package.json index 52b7e71..318550e 100644 --- a/packages/shade-inbox-server/package.json +++ b/packages/shade-inbox-server/package.json @@ -1,6 +1,6 @@ { "name": "@shade/inbox-server", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-inbox/package.json b/packages/shade-inbox/package.json index 0fd29a5..acc3532 100644 --- a/packages/shade-inbox/package.json +++ b/packages/shade-inbox/package.json @@ -1,6 +1,6 @@ { "name": "@shade/inbox", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-key-transparency/package.json b/packages/shade-key-transparency/package.json index dabdb7e..9ba085a 100644 --- a/packages/shade-key-transparency/package.json +++ b/packages/shade-key-transparency/package.json @@ -1,6 +1,6 @@ { "name": "@shade/key-transparency", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-keychain/package.json b/packages/shade-keychain/package.json index 8247661..fcad838 100644 --- a/packages/shade-keychain/package.json +++ b/packages/shade-keychain/package.json @@ -1,6 +1,6 @@ { "name": "@shade/keychain", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-observability/package.json b/packages/shade-observability/package.json index 6fd171c..7983550 100644 --- a/packages/shade-observability/package.json +++ b/packages/shade-observability/package.json @@ -1,6 +1,6 @@ { "name": "@shade/observability", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-observer/package.json b/packages/shade-observer/package.json index 1a0002d..4e14945 100644 --- a/packages/shade-observer/package.json +++ b/packages/shade-observer/package.json @@ -1,6 +1,6 @@ { "name": "@shade/observer", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-proto/package.json b/packages/shade-proto/package.json index 86081f2..10c350d 100644 --- a/packages/shade-proto/package.json +++ b/packages/shade-proto/package.json @@ -1,6 +1,6 @@ { "name": "@shade/proto", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-proto/src/index.ts b/packages/shade-proto/src/index.ts index 0aeca88..a218ab6 100644 --- a/packages/shade-proto/src/index.ts +++ b/packages/shade-proto/src/index.ts @@ -8,7 +8,20 @@ export { encodeBroadcast, decodeBroadcast, inspectEnvelopeType, + encodeStreamOpen, + encodeStreamOpenAck, + decodeStreamHandshake, + encodeStreamFrame, + decodeStreamFrame, TYPE_STREAM_CHUNK, TYPE_BROADCAST, + TYPE_STREAM_OPEN, + TYPE_STREAM_OPEN_ACK, + TYPE_STREAM_FRAME, +} from './wire.js'; +export type { + StreamChunkWire, + BroadcastWire, + StreamHandshakeWire, + StreamFrameWire, } from './wire.js'; -export type { StreamChunkWire, BroadcastWire } from './wire.js'; diff --git a/packages/shade-proto/src/wire.ts b/packages/shade-proto/src/wire.ts index f9ede8e..a40aac9 100644 --- a/packages/shade-proto/src/wire.ts +++ b/packages/shade-proto/src/wire.ts @@ -25,6 +25,13 @@ const TYPE_PREKEY = 0x01; const TYPE_RATCHET = 0x02; export const TYPE_STREAM_CHUNK = 0x11; export const TYPE_BROADCAST = 0x21; +// V4.11 — streaming Double-Ratchet sub-session (long-lived WS channels). +export const TYPE_STREAM_OPEN = 0x31; +export const TYPE_STREAM_OPEN_ACK = 0x32; +export const TYPE_STREAM_FRAME = 0x33; + +const STREAM_SESSION_ID_BYTES = 16; +const STREAM_EPHEMERAL_BYTES = 32; // ─── Stream chunk types ────────────────────────────────────── @@ -258,7 +265,15 @@ export function decodeStreamChunk(data: Uint8Array): StreamChunkWire { */ export function inspectEnvelopeType( data: Uint8Array, -): 'prekey' | 'ratchet' | 'stream-chunk' | 'broadcast' | 'unknown' { +): + | 'prekey' + | 'ratchet' + | 'stream-chunk' + | 'broadcast' + | 'stream-open' + | 'stream-open-ack' + | 'stream-frame' + | 'unknown' { if (data.length < 2 || data[0] !== VERSION) return 'unknown'; switch (data[1]) { case TYPE_PREKEY: @@ -269,11 +284,122 @@ export function inspectEnvelopeType( return 'stream-chunk'; case TYPE_BROADCAST: return 'broadcast'; + case TYPE_STREAM_OPEN: + return 'stream-open'; + case TYPE_STREAM_OPEN_ACK: + return 'stream-open-ack'; + case TYPE_STREAM_FRAME: + return 'stream-frame'; default: return 'unknown'; } } +// ─── Stream sub-session wire (V4.11) ───────────────────────── + +/** + * A decoded stream handshake frame (`STREAM_OPEN` / `STREAM_OPEN_ACK`). + * Both share the layout `[version][type][streamId:16][ephemeralPub:32]`. + */ +export interface StreamHandshakeWire { + kind: 'open' | 'open-ack'; + streamId: Uint8Array; // 16 bytes + ephemeralPub: Uint8Array; // 32 bytes (X25519) +} + +/** + * A decoded sealed stream frame (`STREAM_FRAME`): a streamId plus an + * embedded Double-Ratchet message. One sealed logical frame ⇒ exactly + * one of these ⇒ one WS text/binary frame. + */ +export interface StreamFrameWire { + streamId: Uint8Array; // 16 bytes + message: RatchetMessage; +} + +function encodeStreamHandshake( + type: number, + streamId: Uint8Array, + ephemeralPub: Uint8Array, +): Uint8Array { + if (streamId.length !== STREAM_SESSION_ID_BYTES) { + throw new Error(`streamId must be ${STREAM_SESSION_ID_BYTES} bytes`); + } + if (ephemeralPub.length !== STREAM_EPHEMERAL_BYTES) { + throw new Error(`ephemeralPub must be ${STREAM_EPHEMERAL_BYTES} bytes`); + } + const out = new Uint8Array(2 + STREAM_SESSION_ID_BYTES + STREAM_EPHEMERAL_BYTES); + out[0] = VERSION; + out[1] = type; + out.set(streamId, 2); + out.set(ephemeralPub, 2 + STREAM_SESSION_ID_BYTES); + return out; +} + +/** Encode the initiator's `STREAM_OPEN` (streamId + initiator ephemeral). */ +export function encodeStreamOpen(streamId: Uint8Array, ephemeralPub: Uint8Array): Uint8Array { + return encodeStreamHandshake(TYPE_STREAM_OPEN, streamId, ephemeralPub); +} + +/** Encode the responder's `STREAM_OPEN_ACK` (streamId + responder ephemeral). */ +export function encodeStreamOpenAck(streamId: Uint8Array, ephemeralPub: Uint8Array): Uint8Array { + return encodeStreamHandshake(TYPE_STREAM_OPEN_ACK, streamId, ephemeralPub); +} + +/** Decode either handshake frame. Throws on wrong type / bad length. */ +export function decodeStreamHandshake(data: Uint8Array): StreamHandshakeWire { + const expected = 2 + STREAM_SESSION_ID_BYTES + STREAM_EPHEMERAL_BYTES; + if (data.length !== expected) { + throw new Error(`stream handshake must be ${expected} bytes, got ${data.length}`); + } + if (data[0] !== VERSION) throw new Error(`Unknown version: ${data[0]}`); + let kind: 'open' | 'open-ack'; + if (data[1] === TYPE_STREAM_OPEN) kind = 'open'; + else if (data[1] === TYPE_STREAM_OPEN_ACK) kind = 'open-ack'; + else throw new Error(`Not a stream handshake: type=${data[1]}`); + return { + kind, + streamId: data.slice(2, 2 + STREAM_SESSION_ID_BYTES), + ephemeralPub: data.slice( + 2 + STREAM_SESSION_ID_BYTES, + 2 + STREAM_SESSION_ID_BYTES + STREAM_EPHEMERAL_BYTES, + ), + }; +} + +/** + * Encode a sealed stream frame: `[version][0x33][streamId:16][ratchet…]`. + * Reuses the exact ratchet-message inner codec the HTTP path uses, so a + * stream frame carries the same Double-Ratchet header + AEAD payload. + */ +export function encodeStreamFrame(streamId: Uint8Array, msg: RatchetMessage): Uint8Array { + if (streamId.length !== STREAM_SESSION_ID_BYTES) { + throw new Error(`streamId must be ${STREAM_SESSION_ID_BYTES} bytes`); + } + const inner = encodeRatchetMessageInner(msg); + const out = new Uint8Array(2 + STREAM_SESSION_ID_BYTES + inner.length); + out[0] = VERSION; + out[1] = TYPE_STREAM_FRAME; + out.set(streamId, 2); + out.set(inner, 2 + STREAM_SESSION_ID_BYTES); + return out; +} + +/** Decode a sealed stream frame. Throws on wrong type / truncation. */ +export function decodeStreamFrame(data: Uint8Array): StreamFrameWire { + const minSize = 2 + STREAM_SESSION_ID_BYTES; + if (data.length < minSize) { + throw new Error(`stream-frame too short: ${data.length} < ${minSize}`); + } + if (data[0] !== VERSION) throw new Error(`Unknown version: ${data[0]}`); + if (data[1] !== TYPE_STREAM_FRAME) { + throw new Error(`Not a stream-frame: type=${data[1]}`); + } + const streamId = data.slice(2, 2 + STREAM_SESSION_ID_BYTES); + const message = decodeRatchetMessageInner(data, 2 + STREAM_SESSION_ID_BYTES).value; + return { streamId, message }; +} + // ─── Broadcast wire (V4.6) ─────────────────────────────────── const BROADCAST_NONCE_BYTES = 12; diff --git a/packages/shade-proto/tests/stream-wire.test.ts b/packages/shade-proto/tests/stream-wire.test.ts new file mode 100644 index 0000000..906f2a6 --- /dev/null +++ b/packages/shade-proto/tests/stream-wire.test.ts @@ -0,0 +1,71 @@ +import { describe, test, expect } from 'bun:test'; +import { + encodeStreamOpen, + encodeStreamOpenAck, + decodeStreamHandshake, + encodeStreamFrame, + decodeStreamFrame, + inspectEnvelopeType, +} from '../src/index.js'; +import type { RatchetMessage } from '@shade/core'; + +function randBytes(n: number): Uint8Array { + const buf = new Uint8Array(n); + crypto.getRandomValues(buf); + return buf; +} + +function makeRatchetMessage(): RatchetMessage { + return { + dhPublicKey: randBytes(32), + previousCounter: 3, + counter: 9001, + ciphertext: randBytes(128), + nonce: randBytes(12), + }; +} + +describe('Stream sub-session wire (V4.11)', () => { + test('STREAM_OPEN round-trips and inspects', () => { + const sid = randBytes(16); + const eph = randBytes(32); + const bytes = encodeStreamOpen(sid, eph); + expect(inspectEnvelopeType(bytes)).toBe('stream-open'); + const hs = decodeStreamHandshake(bytes); + expect(hs.kind).toBe('open'); + expect(hs.streamId).toEqual(sid); + expect(hs.ephemeralPub).toEqual(eph); + }); + + test('STREAM_OPEN_ACK round-trips and inspects', () => { + const sid = randBytes(16); + const eph = randBytes(32); + const bytes = encodeStreamOpenAck(sid, eph); + expect(inspectEnvelopeType(bytes)).toBe('stream-open-ack'); + const hs = decodeStreamHandshake(bytes); + expect(hs.kind).toBe('open-ack'); + expect(hs.streamId).toEqual(sid); + expect(hs.ephemeralPub).toEqual(eph); + }); + + test('STREAM_FRAME carries a full ratchet message verbatim', () => { + const sid = randBytes(16); + const msg = makeRatchetMessage(); + const bytes = encodeStreamFrame(sid, msg); + expect(inspectEnvelopeType(bytes)).toBe('stream-frame'); + const decoded = decodeStreamFrame(bytes); + expect(decoded.streamId).toEqual(sid); + expect(decoded.message.dhPublicKey).toEqual(msg.dhPublicKey); + expect(decoded.message.previousCounter).toBe(msg.previousCounter); + expect(decoded.message.counter).toBe(msg.counter); + expect(decoded.message.ciphertext).toEqual(msg.ciphertext); + expect(decoded.message.nonce).toEqual(msg.nonce); + }); + + test('rejects wrong sizes and wrong type tags', () => { + expect(() => encodeStreamOpen(randBytes(15), randBytes(32))).toThrow(); + expect(() => encodeStreamOpen(randBytes(16), randBytes(31))).toThrow(); + expect(() => decodeStreamHandshake(encodeStreamFrame(randBytes(16), makeRatchetMessage()))).toThrow(); + expect(() => decodeStreamFrame(encodeStreamOpen(randBytes(16), randBytes(32)))).toThrow(); + }); +}); diff --git a/packages/shade-recovery/package.json b/packages/shade-recovery/package.json index cb89a38..61a2885 100644 --- a/packages/shade-recovery/package.json +++ b/packages/shade-recovery/package.json @@ -1,6 +1,6 @@ { "name": "@shade/recovery", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-sdk/package.json b/packages/shade-sdk/package.json index 3f07e6d..a57986a 100644 --- a/packages/shade-sdk/package.json +++ b/packages/shade-sdk/package.json @@ -1,6 +1,6 @@ { "name": "@shade/sdk", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-sdk/src/index.ts b/packages/shade-sdk/src/index.ts index 34bb270..0ba1a16 100644 --- a/packages/shade-sdk/src/index.ts +++ b/packages/shade-sdk/src/index.ts @@ -1,5 +1,5 @@ export { createShade } from './create-shade.js'; -export { Shade } from './shade.js'; +export { Shade, ShadeStream } from './shade.js'; export type { ShadeUploadOptions, ShadeWebRtcConfig, diff --git a/packages/shade-sdk/src/shade.ts b/packages/shade-sdk/src/shade.ts index 3c5dd6b..1b6ffa1 100644 --- a/packages/shade-sdk/src/shade.ts +++ b/packages/shade-sdk/src/shade.ts @@ -1,8 +1,10 @@ -import type { ShadeEnvelope, StorageProvider } from '@shade/core'; +import type { ShadeEnvelope, StorageProvider, RatchetMessage } from '@shade/core'; import { ShadeSessionManager, ShadeEventEmitter, NoSessionError, + StreamRatchet, + StreamHandshakeError, } from '@shade/core'; import { FingerprintGateRegistry, @@ -18,7 +20,16 @@ import { type CreateEncryptStreamOptions, type CreateDecryptStreamOptions, } from '@shade/crypto-web'; -import { encodeEnvelope, decodeEnvelope, inspectEnvelopeType } from '@shade/proto'; +import { + encodeEnvelope, + decodeEnvelope, + inspectEnvelopeType, + encodeStreamOpen, + encodeStreamOpenAck, + decodeStreamHandshake, + encodeStreamFrame, + decodeStreamFrame, +} from '@shade/proto'; import { ShadeFetchTransport, type KTVerifierOptions } from '@shade/transport'; import { LightWitness } from '@shade/key-transparency'; import type { SignedTreeHead, STHWire } from '@shade/key-transparency'; @@ -1510,6 +1521,97 @@ export class Shade { await this.storage.pruneStreamStates(olderThan); } + // ─── Streaming sub-sessions (V4.11) ──────────────────────── + + /** + * Open a long-lived streaming Double-Ratchet sub-session to an + * already-known peer, for wrapping individual frames on a + * bidirectional, often server-heavy channel (e.g. a console-log + * WebSocket) with the same confidentiality / forward-secrecy / + * replay guarantees as the HTTP `send`/`receive` path. + * + * This is the **initiator** half. Like the rest of the SDK it is + * transport-agnostic: it produces handshake/frame bytes you put on + * your WebSocket, and consumes the bytes you receive from it. + * + * ```ts + * const stream = await shade.openStream(peerAddr); + * ws.send(stream.handshakeFrame()); // → STREAM_OPEN + * // … first inbound WS frame is the peer's STREAM_OPEN_ACK … + * await stream.handleHandshake(ackBytes); // stream now usable + * ws.send(await stream.seal(utf8(line))); // outbound frame + * onLog(await stream.open(inboundBytes)); // inbound frame + * await stream.close(); // on ws close + * ``` + * + * Independence (R5): this never touches the stored parent session, + * its prekeys, or the per-peer `send`/`receive` queues — it runs + * concurrently against the same peer. The ratchet lives only in + * memory and is zeroized by {@link ShadeStream.close}; a dropped + * connection is re-opened with a fresh `openStream`, never resumed + * (persisting per-frame ratchet secrets would defeat forward + * secrecy). + * + * Note (Double-Ratchet semantics): a responder cannot `seal` until + * it has `open`ed at least one frame from the initiator (standard + * Signal behaviour). For a server-heavy stream either make the bursty + * sender the initiator, or have the initiator send one priming frame + * right after the handshake. + * + * Requires an established parent session; one is auto-established + * (same path as {@link send}) if missing. + */ + async openStream(peerAddress: string): Promise { + if (!this.initialized) throw new Error('Not initialized'); + let begun; + try { + begun = await this.manager.beginStream(peerAddress); + } catch (err) { + if (!(err instanceof NoSessionError)) throw err; + await this.ensureSession(peerAddress); + begun = await this.manager.beginStream(peerAddress); + } + return new ShadeStream({ + peer: peerAddress, + role: 'initiator', + streamId: begun.streamId, + events: this.events, + handshakeOut: encodeStreamOpen(begun.streamId, begun.ephemeralPublicKey), + complete: begun.complete, + }); + } + + /** + * Accept an inbound stream — the **responder** half. Feed it the + * peer's `STREAM_OPEN` bytes (route by {@link inspectEnvelopeType} + * `=== 'stream-open'`). The returned stream is immediately usable for + * `open()`; send `handshakeFrame()` (the `STREAM_OPEN_ACK`) back over + * the transport so the initiator can complete its side. + */ + async acceptStream(peerAddress: string, openBytes: Uint8Array): Promise { + if (!this.initialized) throw new Error('Not initialized'); + const hs = decodeStreamHandshake(openBytes); + if (hs.kind !== 'open') { + throw new StreamHandshakeError(`expected STREAM_OPEN, got ${hs.kind}`); + } + let accepted; + try { + accepted = await this.manager.acceptStream(peerAddress, hs.streamId, hs.ephemeralPub); + } catch (err) { + if (!(err instanceof NoSessionError)) throw err; + await this.ensureSession(peerAddress); + accepted = await this.manager.acceptStream(peerAddress, hs.streamId, hs.ephemeralPub); + } + return new ShadeStream({ + peer: peerAddress, + role: 'responder', + streamId: hs.streamId, + events: this.events, + handshakeOut: encodeStreamOpenAck(hs.streamId, accepted.ephemeralPublicKey), + ratchet: accepted.stream, + }); + } + private async ensureSession(address: string): Promise { // Deduplicate concurrent establishment requests const existing = this.establishing.get(address); @@ -1532,6 +1634,158 @@ export class Shade { } } +// ─── ShadeStream (V4.11) ───────────────────────────────────── + +interface ShadeStreamInit { + peer: string; + role: 'initiator' | 'responder'; + streamId: Uint8Array; + events: ShadeEventEmitter; + /** Bytes to put on the wire for our half of the handshake. */ + handshakeOut: Uint8Array; + /** Initiator only: continuation that derives the ratchet from the ACK. */ + complete?: (peerEphemeralPub: Uint8Array) => Promise; + /** Responder only: ratchet is ready at accept time. */ + ratchet?: StreamRatchet; +} + +function streamIdsEqual(a: Uint8Array, b: Uint8Array): boolean { + if (a.length !== b.length) return false; + for (let i = 0; i < a.length; i++) if (a[i] !== b[i]) return false; + return true; +} + +/** + * A live streaming Double-Ratchet sub-session. Transport-agnostic: it + * emits/consumes wire bytes, the caller owns the WebSocket (or any + * other ordered frame transport). + * + * Lifecycle: + * - **initiator**: `handshakeFrame()` → `STREAM_OPEN`; after the peer's + * `STREAM_OPEN_ACK` arrives call `handleHandshake(ack)`; then + * `seal`/`open`. + * - **responder**: usable immediately; `handshakeFrame()` → + * `STREAM_OPEN_ACK` to send back; `open` the initiator's first frame + * before `seal` (standard Double-Ratchet ordering). + */ +export class ShadeStream { + private readonly _streamId: Uint8Array; + private readonly _peer: string; + private readonly _role: 'initiator' | 'responder'; + private readonly events: ShadeEventEmitter; + private readonly handshakeOut: Uint8Array; + private readonly complete?: (peerEphemeralPub: Uint8Array) => Promise; + private ratchet: StreamRatchet | null; + private state: 'await-ack' | 'open' | 'closed'; + + constructor(init: ShadeStreamInit) { + this._streamId = init.streamId; + this._peer = init.peer; + this._role = init.role; + this.events = init.events; + this.handshakeOut = init.handshakeOut; + if (init.role === 'initiator') { + if (init.complete) this.complete = init.complete; + this.ratchet = null; + this.state = 'await-ack'; + } else { + this.ratchet = init.ratchet ?? null; + this.state = 'open'; + } + } + + /** Peer address this stream is bound to. */ + get peer(): string { + return this._peer; + } + + /** Which half of the handshake this end performed. */ + get role(): 'initiator' | 'responder' { + return this._role; + } + + /** Lowercase-hex stream id (stable for the stream's lifetime). */ + get streamId(): string { + return Array.from(this._streamId, (b) => b.toString(16).padStart(2, '0')).join(''); + } + + /** True once the ratchet is established and not yet closed. */ + get isOpen(): boolean { + return this.state === 'open' && this.ratchet !== null; + } + + /** + * The bytes for our half of the handshake to put on the transport + * (`STREAM_OPEN` for an initiator, `STREAM_OPEN_ACK` for a responder). + * Stable; safe to read once and send. + */ + handshakeFrame(): Uint8Array { + return this.handshakeOut; + } + + /** + * Initiator only: consume the peer's `STREAM_OPEN_ACK` and derive the + * ratchet. Idempotent-safe to call exactly once; throws if called on + * a responder, out of order, or with a mismatched streamId. + */ + async handleHandshake(ackBytes: Uint8Array): Promise { + if (this._role !== 'initiator') { + throw new StreamHandshakeError('handleHandshake is initiator-only'); + } + if (this.state !== 'await-ack' || !this.complete) { + throw new StreamHandshakeError('handshake already completed or stream closed'); + } + const hs = decodeStreamHandshake(ackBytes); + if (hs.kind !== 'open-ack') { + throw new StreamHandshakeError(`expected STREAM_OPEN_ACK, got ${hs.kind}`); + } + if (!streamIdsEqual(hs.streamId, this._streamId)) { + throw new StreamHandshakeError('STREAM_OPEN_ACK streamId mismatch'); + } + this.ratchet = await this.complete(hs.ephemeralPub); + this.state = 'open'; + } + + /** + * Seal one logical frame. Returns `STREAM_FRAME` wire bytes — put + * exactly one in one WS frame. Advances the sending chain one step. + */ + async seal(plaintext: Uint8Array): Promise { + if (!this.ratchet || this.state !== 'open') { + throw new StreamHandshakeError('stream not open (complete the handshake first)'); + } + const msg = await this.ratchet.seal(plaintext); + return encodeStreamFrame(this._streamId, msg); + } + + /** + * Open one inbound `STREAM_FRAME`. Correct and memory-bounded across + * long one-directional bursts; replays / counter-rewinds are rejected + * by the underlying ratchet. + */ + async open(wire: Uint8Array): Promise { + if (!this.ratchet || this.state !== 'open') { + throw new StreamHandshakeError('stream not open (complete the handshake first)'); + } + const frame: { streamId: Uint8Array; message: RatchetMessage } = decodeStreamFrame(wire); + if (!streamIdsEqual(frame.streamId, this._streamId)) { + throw new StreamHandshakeError('STREAM_FRAME streamId mismatch'); + } + return this.ratchet.open(frame.message); + } + + /** Zeroize and drop the ratchet. Idempotent. */ + async close(): Promise { + if (this.state === 'closed') return; + this.state = 'closed'; + if (this.ratchet) { + await this.ratchet.close(); + this.ratchet = null; + } + this.events.emit('stream.closed', { address: this._peer }); + } +} + function bytesToBase64Std(bytes: Uint8Array): string { let bin = ''; for (let i = 0; i < bytes.length; i++) bin += String.fromCharCode(bytes[i]!); diff --git a/packages/shade-server/package.json b/packages/shade-server/package.json index f0495d0..794b4d3 100644 --- a/packages/shade-server/package.json +++ b/packages/shade-server/package.json @@ -1,6 +1,6 @@ { "name": "@shade/server", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-storage-encrypted/package.json b/packages/shade-storage-encrypted/package.json index 3559e12..8dc79bd 100644 --- a/packages/shade-storage-encrypted/package.json +++ b/packages/shade-storage-encrypted/package.json @@ -1,6 +1,6 @@ { "name": "@shade/storage-encrypted", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-storage-indexeddb/package.json b/packages/shade-storage-indexeddb/package.json index 7221817..664460d 100644 --- a/packages/shade-storage-indexeddb/package.json +++ b/packages/shade-storage-indexeddb/package.json @@ -1,6 +1,6 @@ { "name": "@shade/storage-indexeddb", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-storage-postgres/package.json b/packages/shade-storage-postgres/package.json index ade9c99..b807ac7 100644 --- a/packages/shade-storage-postgres/package.json +++ b/packages/shade-storage-postgres/package.json @@ -1,6 +1,6 @@ { "name": "@shade/storage-postgres", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-storage-sqlite/package.json b/packages/shade-storage-sqlite/package.json index 7f89762..c82c467 100644 --- a/packages/shade-storage-sqlite/package.json +++ b/packages/shade-storage-sqlite/package.json @@ -1,6 +1,6 @@ { "name": "@shade/storage-sqlite", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-streams/package.json b/packages/shade-streams/package.json index e588766..dec3d27 100644 --- a/packages/shade-streams/package.json +++ b/packages/shade-streams/package.json @@ -1,6 +1,6 @@ { "name": "@shade/streams", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-transfer/package.json b/packages/shade-transfer/package.json index 2cb5fb5..c34ee59 100644 --- a/packages/shade-transfer/package.json +++ b/packages/shade-transfer/package.json @@ -1,6 +1,6 @@ { "name": "@shade/transfer", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-transport-bridge/package.json b/packages/shade-transport-bridge/package.json index 8b79c1b..8c6c8e3 100644 --- a/packages/shade-transport-bridge/package.json +++ b/packages/shade-transport-bridge/package.json @@ -1,6 +1,6 @@ { "name": "@shade/transport-bridge", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-transport-webrtc/package.json b/packages/shade-transport-webrtc/package.json index 09dc680..596c857 100644 --- a/packages/shade-transport-webrtc/package.json +++ b/packages/shade-transport-webrtc/package.json @@ -1,6 +1,6 @@ { "name": "@shade/transport-webrtc", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-transport/package.json b/packages/shade-transport/package.json index f1b0b36..b5248d0 100644 --- a/packages/shade-transport/package.json +++ b/packages/shade-transport/package.json @@ -1,6 +1,6 @@ { "name": "@shade/transport", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts", diff --git a/packages/shade-widgets/package.json b/packages/shade-widgets/package.json index 526cf64..e6d0584 100644 --- a/packages/shade-widgets/package.json +++ b/packages/shade-widgets/package.json @@ -1,6 +1,6 @@ { "name": "@shade/widgets", - "version": "4.10.0", + "version": "4.11.0", "type": "module", "main": "src/index.ts", "types": "src/index.ts",