Shade/docs/streaming-sessions.md

Streaming Double-Ratchet sub-sessions (V4.11)

ShadeStream wraps individual frames on a long-lived, high-frequency, often one-directional channel (e.g. a server→client console-log WebSocket) in an independent Double Ratchet derived from — but never mutating — an already-established parent Shade session.

This is the answer to Vyvern FR shade-ws-streaming-ratchet.md. It is a first-class API, not the "documented contract that send/receive is safe per-frame" fallback: the Double-Ratchet crypto was already safe for that access pattern, but the send/receive wrapper layer was not (per-frame keystore writes; a shared per-peer mutex and a single stored session row coupling the stream to the HTTP path). ShadeStream keeps the proven ratchet and fixes the wrapper.

API

Transport-agnostic, exactly like send/receive: it emits/consumes wire bytes; you own the WebSocket.

// Initiator (the side that calls openStream)
const stream = await shade.openStream(peerAddr);
ws.send(stream.handshakeFrame());            // → STREAM_OPEN
// first inbound WS frame is the peer's STREAM_OPEN_ACK:
await stream.handleHandshake(ackBytes);      // stream now usable
ws.send(await stream.seal(utf8(logLine)));   // outbound frame
onLog(await stream.open(inboundBytes));      // inbound frame
await stream.close();                        // on ws close/error

// Responder
const stream = await shade.acceptStream(peerAddr, openBytes); // usable now
ws.send(stream.handshakeFrame());            // → STREAM_OPEN_ACK
// open()/seal() as above

Route inbound bytes with inspectEnvelopeType(): 'stream-open' | 'stream-open-ack' | 'stream-frame'.

Seeding (no prekey-server round trip)

The stream root key is derived from an identity-bound 3-DH exchange — the X3DH pattern minus signed/one-time prekeys, because the peer's identity is already mutually pinned by the parent session's TOFU. Two ephemerals are exchanged inside the transport (STREAM_OPEN / STREAM_OPEN_ACK); no prekey server is involved.

slotA = DH(initiatorEphemeral, responderIdentity)   — authenticates responder
slotB = DH(initiatorIdentity,  responderEphemeral)   — authenticates initiator
slotC = DH(initiatorEphemeral, responderEphemeral)   — ephemeral forward secrecy
SK    = HKDF(ikm = slotA‖slotB‖slotC, salt = streamId, info = "ShadeStream/v1")

Both peers compute the identical three scalars regardless of role. SK then bootstraps a textbook Double Ratchet by handing the responder's ephemeral to initSenderSession/initReceiverSession exactly the way X3DH hands its signed prekey to the ratchet — so ratchetEncrypt/ratchetDecrypt and every guarantee they carry apply unchanged.

Security contract (answers FR R1–R7)

• R1 — same properties as send/receive. Each frame is one ratchetEncrypt/ratchetDecrypt over the same crypto as the HTTP path: AES-256-GCM confidentiality, per-frame forward secrecy via the one-way HMAC chain-key KDF with in-place zeroize of the spent chain key, and replay/rewind rejection (a re-delivered or counter-rewound frame fails closed). The handshake is mutually authenticated against the identities the parent session already pinned.
• R2 — one-directional resilience. A long server→client burst with no client traffic only advances the symmetric sending chain (no DH step until the peer replies — standard Double Ratchet). Forward secrecy holds per frame in this regime. Over an ordered transport (WebSocket/TCP) zero keys are skipped per frame.
• R3 — bounded memory. Out-of-order arrivals are capped by the ratchet's MAX_SKIP (1000) and MAX_CACHED_SKIPPED_KEYS (2000) with oldest-key eviction. In-order delivery retains nothing. Verified to stay at zero retained keys across a 5000-frame burst.
• R4 — browser parity. Identical API and guarantees in the browser SDK: ShadeStream is on the same Shade class over the same CryptoProvider (SubtleCryptoProvider), so the IndexedDB-backed build behaves identically to the sqlite: server build. No storage is touched at all (see R7), so the keystore backend is irrelevant.
• R5 — independent lifecycle. The stream ratchet is derived without reading or writing the stored parent SessionState, runs on its own private op-mutex (not the per-peer send/receive queues), and is zeroized on close(). Opening, using for thousands of frames, and closing a stream leaves the parent session byte-identical; the HTTP path keeps working concurrently against the same peer. Each openStream gets a fresh streamId and an independent root, so concurrent streams to one peer never share key material.
• R6 — wire framing. @shade/proto defines STREAM_OPEN (0x31), STREAM_OPEN_ACK (0x32), STREAM_FRAME (0x33). A STREAM_FRAME carries one Double-Ratchet message via the exact ratchet inner codec the HTTP path uses. One sealed logical frame ⇒ one self-delimiting wire frame ⇒ one WS text/binary frame.
• R7 — performance. The ratchet lives only in memory and is never persisted. There is therefore zero per-frame storage I/O — the per-frame cost is exactly the symmetric KDF + one AES-GCM, the same primitives the HTTP path runs. This is strictly better than the "doubled CPU" the Vyvern roadmap budgeted, because the dominant cost the naive send/receive-per-frame approach would have paid (a saveSession keystore write per frame) is eliminated, not doubled. Not persisting is also a security property, not a shortcut: writing evolving per-frame ratchet secrets to disk would defeat forward secrecy. A dropped/reconnected stream is re-opened with a fresh handshake, never resumed.

Double-Ratchet ordering note

A responder cannot seal() until it has open()ed at least one frame from the initiator (standard Signal behaviour — the responder has no sending chain until the first DH step). For a server-heavy stream either make the bursty data sender the initiator, or have the initiator send one priming frame immediately after the handshake.

Tests

• packages/shade-core/tests/stream.test.ts — handshake agreement, frame round-trips, 5000-frame one-directional burst (bounded skipped keys + forward-secrecy zeroize), parent-session independence (R5), replay/rewind rejection, mutual authentication against pinned identities, close() zeroize/idempotence.
• packages/shade-proto/tests/stream-wire.test.ts — wire round-trips and type-tag/length rejection for all three stream frame kinds.