Shade/docs/streams.md

# Shade Streams 0.2.0

E2EE chunked upload/download for Shade. Drop into any Shade-using app:

```ts
const handle = await shade.upload({ to: 'bob', input: file });
const result = await handle.done(); // { sha256, bytesSent, durationMs }
```

…and on the receiver:

```ts
shade.onIncomingTransfer(async (incoming) => {
  const handle = await incoming.accept({ output: { kind: 'file', path: '/uploads/x' } });
  await handle.done();
});
```

Or in React:

```tsx
<ShadeRuntimeProvider runtime={shade}>
  <ShadeUploader to="bob" onComplete={(r) => console.log(r.sha256)} />
</ShadeRuntimeProvider>
```

## How it works

A transfer has two planes:

- **Control plane** — `stream-init`, `stream-finish`, `stream-abort`, and
  `stream-resume-*` messages, encoded as JSON plaintext and shipped through
  the existing Double Ratchet (envelope type `0x02`). One ratchet step
  establishes a stream; the rest is per-chunk AEAD.
- **Data plane** — `stream-chunk` envelopes (envelope type `0x11`),
  AES-256-GCM-encrypted under a per-lane key, shipped over HTTP POST (or
  WebSocket if opted-in). Lanes run in parallel for throughput.

```
Sender                                  Receiver
──────                                  ────────
streamSecret = randomBytes(32)
streamId     = randomBytes(16)
streamKey    = HKDF(streamSecret, streamId, "shade-stream/v1\0master")
laneKey[i]   = HKDF(streamKey, streamId, "...\0lane\0" || u32(i))

[stream-init JSON over Double Ratchet] ─▶
                                          parses streamSecret, derives same keys
                                          spawns L per-lane receivers

[chunk 0x11 over HTTP]                 ─▶
   AES-GCM(laneKey[i], plaintext, nonce=laneId||seq, aad=streamId||laneId||seq||isLast)
                                          decrypts, verifies, writes to sink
   (× 4 lanes in parallel)

[stream-finish JSON over Double Ratchet] ─▶
                                          verifies per-lane sha256 + overall sha256
                                          throws TransferIntegrityError on mismatch
```

## Partition strategies

- **Range** (default for known-size inputs) — lane `i` owns bytes
  `[i·N/L, (i+1)·N/L)`. Receiver reconstructs by concatenating lane outputs
  in laneId order.
- **Round-robin** (default for unknown-size streams) — chunk `i` goes to
  lane `i mod L`. Receiver reorders via a per-stream chunk-index buffer.

## Resume

Persistence is opt-in via a `ResumeStore` (memory, SQLite, Postgres,
IndexedDB-ready). State persisted on init; sender's resume queries the
receiver's `lastSeqAcked` per lane via `GET /v1/transfer/:streamId/state`,
then continues from there. The streamSecret is encrypted at rest under a
device-key derived from the local identity's signing private key — a
stolen DB without the identity key cannot resume.

```ts
const handle = await shade.resumeUpload(streamId, sameInputAsBefore);
await handle.done();
```

Resume across **identity rotation** is not supported (rotation invalidates
the device key — by design, to prevent a stolen pre-rotation DB from
deriving keys for any post-rotation transfer). Restart the transfer
manually after rotation.

## Throughput

- Default 4 lanes × 1 MiB chunks × 4 in-flight chunks per lane =
  16 MiB peak in-flight per direction.
- Memory-bounded: receivers stream chunks to the configured sink without
  buffering the full payload. 1 GB transfer = O(chunkSize) RSS, not O(file).
- AES-GCM is hardware-accelerated via `SubtleCrypto`; SHA-256 streaming via
  `@noble/hashes`.

## Security properties

| ID | Property |
|---|---|
| S1 | streamSecret never on the wire in plaintext (Double Ratchet only) |
| S2 | Unique per-(streamKey, laneId, seq) AEAD nonce — no nonce reuse |
| S3 | Tampered chunk header / ciphertext / tag → AEAD reject |
| S4 | Per-lane sha256 + overall sha256 verified at finish |
| S5 | streamKey/laneKey zeroized on abort/finish (`destroy()`) |
| S6 | Concurrent streams have independent lane keys |
| S7 | seq overflow practical-impossible (u64 max) |
| S8 | At-rest streamSecret encrypted under device-key |

## API surface

See package READMEs:

- `packages/shade-streams/README.md` — crypto + state machines
- `packages/shade-transfer/README.md` — orchestration, transports, persistence
- `packages/shade-sdk/README.md` — magic drop-in
- `packages/shade-widgets/README.md` — React UI