Shade/docs/streams.md

Shade Streams 0.2.0

E2EE chunked upload/download for Shade. Drop into any Shade-using app:

const handle = await shade.upload({ to: 'bob', input: file });
const result = await handle.done(); // { sha256, bytesSent, durationMs }

…and on the receiver:

shade.onIncomingTransfer(async (incoming) => {
  const handle = await incoming.accept({ output: { kind: 'file', path: '/uploads/x' } });
  await handle.done();
});

Or in React:

<ShadeRuntimeProvider runtime={shade}>
  <ShadeUploader to="bob" onComplete={(r) => console.log(r.sha256)} />
</ShadeRuntimeProvider>

How it works

A transfer has two planes:

• Control plane — stream-init, stream-finish, stream-abort, and stream-resume-* messages, encoded as JSON plaintext and shipped through the existing Double Ratchet (envelope type 0x02). One ratchet step establishes a stream; the rest is per-chunk AEAD.
• Data plane — stream-chunk envelopes (envelope type 0x11), AES-256-GCM-encrypted under a per-lane key, shipped over HTTP POST (or WebSocket if opted-in). Lanes run in parallel for throughput.

Sender                                  Receiver
──────                                  ────────
streamSecret = randomBytes(32)
streamId     = randomBytes(16)
streamKey    = HKDF(streamSecret, streamId, "shade-stream/v1\0master")
laneKey[i]   = HKDF(streamKey, streamId, "...\0lane\0" || u32(i))

[stream-init JSON over Double Ratchet] ─▶
                                          parses streamSecret, derives same keys
                                          spawns L per-lane receivers

[chunk 0x11 over HTTP]                 ─▶
   AES-GCM(laneKey[i], plaintext, nonce=laneId||seq, aad=streamId||laneId||seq||isLast)
                                          decrypts, verifies, writes to sink
   (× 4 lanes in parallel)

[stream-finish JSON over Double Ratchet] ─▶
                                          verifies per-lane sha256 + overall sha256
                                          throws TransferIntegrityError on mismatch

Partition strategies

• Range (default for known-size inputs) — lane i owns bytes [i·N/L, (i+1)·N/L). Receiver reconstructs by concatenating lane outputs in laneId order.
• Round-robin (default for unknown-size streams) — chunk i goes to lane i mod L. Receiver reorders via a per-stream chunk-index buffer.

Resume

Persistence is opt-in via a ResumeStore (memory, SQLite, Postgres, IndexedDB-ready). State persisted on init; sender's resume queries the receiver's lastSeqAcked per lane via GET /v1/transfer/:streamId/state, then continues from there. The streamSecret is encrypted at rest under a device-key derived from the local identity's signing private key — a stolen DB without the identity key cannot resume.

const handle = await shade.resumeUpload(streamId, sameInputAsBefore);
await handle.done();

Resume across identity rotation is not supported (rotation invalidates the device key — by design, to prevent a stolen pre-rotation DB from deriving keys for any post-rotation transfer). Restart the transfer manually after rotation.

Throughput

• Default 4 lanes × 1 MiB chunks × 4 in-flight chunks per lane = 16 MiB peak in-flight per direction.
• Memory-bounded: receivers stream chunks to the configured sink without buffering the full payload. 1 GB transfer = O(chunkSize) RSS, not O(file).
• AES-GCM is hardware-accelerated via SubtleCrypto; SHA-256 streaming via @noble/hashes.

Security properties

ID	Property
S1	streamSecret never on the wire in plaintext (Double Ratchet only)
S2	Unique per-(streamKey, laneId, seq) AEAD nonce — no nonce reuse
S3	Tampered chunk header / ciphertext / tag → AEAD reject
S4	Per-lane sha256 + overall sha256 verified at finish
S5	streamKey/laneKey zeroized on abort/finish (destroy())
S6	Concurrent streams have independent lane keys
S7	seq overflow practical-impossible (u64 max)
S8	At-rest streamSecret encrypted under device-key

API surface

See package READMEs:

• packages/shade-streams/README.md — crypto + state machines
• packages/shade-transfer/README.md — orchestration, transports, persistence
• packages/shade-sdk/README.md — magic drop-in
• packages/shade-widgets/README.md — React UI