Shade/CHANGELOG.md

Changelog

All notable changes to Shade are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.3.0] — 2026-05-02 — Shade Files

E2EE filesystem RPC primitive — drop-in entrypoints for any consumer that wants to expose a filesystem (or filesystem-like surface) over Shade. Apps keep their own UI; this layer ships the typed RPC, the streams bridge for content I/O over 256 KiB, and production hooks (rate limit, retention, fingerprint gate, metrics).

Added

@shade/files (NEW)

• Standard ops: list, stat, mkdir, delete, move, read, write, getThumbnail — Zod-validated wire schemas + clean user-handler types.
• Custom ops: client.custom('app.foo', {...}) with full type-safety via TypeScript declaration merging on CustomOpsMap + per-op Zod schemas registered server-side.
• Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams (> 256 KiB) ride @shade/transfer with automatic correlation via userMetadata.shadeFilesWriteId / shadeFilesReadStreamId.
• Directory ops: walk(path, opts) async-iterable depth-first walker; uploadDirectory() / downloadDirectory() with bounded concurrency pool (default 4, cap 16), aggregated progress events, abort support.
• Production hooks (all callback-based, vendor-neutral):
  • Rate limit: token-bucket per sender, op-cost + byte-quota, FsRateLimitError / QuotaExceededError with retryAfterMs.
  • Idempotency cache: per-sender LRU + TTL, in-flight de-dupe, periodic prune via BackgroundHooks.onPruneFiles.
  • Path policy: built-in traversal hardening, percent-decode, forbidden-bytes check, root-scope, symlink toggle, extra predicate.
  • Fingerprint gate: requireFingerprintVerifiedFor(ctx) → 'required' | 'optional' | 'reject' + isFingerprintVerified(sender).
  • Signature verification: pluggable verifySender(sender, canonical, sig) with replay-window enforcement (±5 min signedAt skew rejected).
  • Metrics: onMetric(name, value, tags) with standard names (shade_files_op_duration_ms, _op_total, _bytes_in/out, _idempotency_hit/conflict_total, _rate_limit_reject_total, _fingerprint_reject_total, _signature_reject_total).
• React hooks (subpath import @shade/files/react): <ShadeFilesProvider>, useShadeFiles, useFileList, useFileTransfer / useFileUpload / useFileDownload. SSR-safe; no UI components — apps bring their own.
• High-level entry: Shade.files.serve(handler) and Shade.files.client(peer) in @shade/sdk. Lazy + memoized; one handler per Shade instance.
• Drop-in adapter: createMemoryDirectory() for tests; structurally compatible with browser FileSystemDirectoryHandle.

Wire format bump

• @shade/proto wire VERSION bumped from 0x01 to 0x02. Length prefixes changed from u16 to u32 — previous limit was 64 KiB ratchet payloads, which blocked inline file ops up to 256 KiB. Wire-incompatible with 0.2.x peers. New sessions only.
• Cross-platform Kotlin port (android/shade-android) updated to match.

Concurrency safety

• ShadeSessionManager.encrypt / .decrypt now run under per-peer mutex. Previously, concurrent decryptions of the same peer raced ratchet state (manifested as sporadic Failed to decrypt — wrong key or tampered data under load). Encrypt was already serialized via Shade.send's encryptChains; decrypt is now serialized at the manager layer too.

@shade/streams extension

• StreamMetadata gets optional userMetadata?: Record<string, string> — application-level key/value pairs that round-trip verbatim through stream-init plaintext. Used by @shade/files for write/read correlation but available to any consumer.

@shade/sdk extension

• Shade.files getter (lazy + memoized).
• BackgroundHooks.onPruneFiles?: () => void + periodic timer (default 5 min) for @shade/files retention.
• BackgroundTasks.setHook(name, fn) for runtime hook registration.

Examples

• examples/08-files-browser/ — three-process demo (prekey + Bob server + Alice CLI) covering list/stat/mkdir/delete/upload/download with both inline and streamed paths.

Tests

• 100+ new tests across tests/{unit,integration,security}/ in @shade/files. End-to-end coverage for streams I/O up to 1 MiB, custom-op registration + Zod validation, fingerprint-gate rejection, replay-window enforcement, idempotent retries, rate-limit + quota enforcement, walk
  • bulk transfer aggregated progress.

[0.2.0] — 2026-05-01 — Shade Streams

E2EE chunked upload/download with parallel lanes, resumable transfers, and a "magic drop-in" UX for any Shade-using app. Adds two new packages (@shade/streams, @shade/transfer) and extends @shade/sdk and @shade/widgets with high-level transfer APIs.

Added

Streams crypto layer (@shade/streams)

• HKDF stream/lane key derivation (deriveStreamKey, deriveLaneKey)
• Deterministic AES-GCM nonce construction nonce = laneId(4) || seq(8)
• Streaming SHA-256 via @noble/hashes/sha2.js for memory-bounded integrity
• StreamSender / StreamReceiver per-lane state machines with strict in-order seq + replay detection (StreamReplayError, StreamOutOfOrderError, StreamDecryptionError, StreamProtocolError)
• MultiLaneSender / MultiLaneReceiver coordinators for parallel transfers
• Range and round-robin partitioning helpers (planRangePartition, planRoundRobinPartition, chunkRange)
• Wire format: new envelope type 0x11 (stream-chunk) in @shade/proto, control envelopes (stream-init / -finish / -abort / -resume-*) ride existing 0x02 ratchet messages with JSON kind discriminator

Transfer orchestration (@shade/transfer)

• TransferEngine — single class wrapping outgoing + incoming lifecycle
• Default ShadeTransferHttpTransport for chunk POSTs, opt-in ShadeTransferWsTransport with FallbackTransferTransport for auto-fallback
• createTransferRoutes() Hono factory mounts /v1/transfer/* routes (chunk, state, health)
• IControlChannel + MemoryControlChannel for in-process testing; the SDK provides ShadeControlChannel over Shade.send/receive
• Resume protocol: MemoryResumeStore, StorageBackedResumeStore, deriveDeviceKey() for at-rest streamSecret encryption, engine.resumeUpload(streamId, freshInput) for kill-restart-verify flows
• ProgressTracker with EMA-smoothed throughput + ETA
• Retry/backoff (withRetry) with exponential delay + jitter
• Error hierarchy: TransferError, TransferAbortError, TransferIntegrityError, TransferProtocolError, TransferOfflineError, TransferResumeError, TransferTransportError

SDK (@shade/sdk)

• Shade.upload(opts) — high-level entry; encrypts + chunks + ships
• Shade.onIncomingTransfer(handler) — receiver-side subscription
• Shade.transferRoute() — Hono router to mount on the consumer's HTTP server
• Shade.acceptTransferEnvelope(from, env) — low-level entry for custom transports
• Shade.resumeUpload(streamId, freshInput) — pick up an interrupted transfer
• Shade.listTransfers(filter?) — list resumable / active transfers from storage
• ShadeTransferAuthenticator — Ed25519-signing authenticator for HTTP/WS transports
• Shade.onMessage(handler) now accepts Promise<void>-returning handlers (awaited in sequence) — supports flow-control over the control plane

Storage (all backends)

• New optional StorageProvider methods: saveStreamState, getStreamState, removeStreamState, listActiveStreamStates, pruneStreamStates. Existing v0.1.x providers compile cleanly (optional methods)
• SQLite (stream_state table) and Postgres (shade_stream_state table) schemas with at-rest encrypted streamSecret
• MemoryStorage extended with in-memory stream-state map

Widgets (@shade/widgets)

• <ShadeRuntimeProvider runtime={shade}> — separate React context for upload/download widgets (distinct from the observer-dashboard <ShadeProvider>)
• useShadeUpload() / useShadeDownload() headless hooks
• <ShadeUploader /> / <ShadeDownloader /> composite components with render-prop pattern for full UI replacement
• Sub-components: <DropZone />, <TransferRow />, <ProgressBar />, <SpeedReadout />, <ETAReadout />, <LaneIndicator />
• Theme-token additions for progress, drop zone, and lane indicator colors

Security properties

• Per-chunk AES-256-GCM with deterministic nonce; AAD binds streamId || laneId || seq || isLast so any header tamper invalidates AEAD
• streamSecret never on the wire in plaintext — shipped via Double Ratchet control envelope; lane keys derived locally and never transmitted
• Resume state encrypted at rest with deviceKey derived from identity's signing private key (rotation invalidates in-flight resume — by design)
• Receiver enforces strict in-order seq per lane (StreamOutOfOrderError, StreamReplayError); finish-time integrity check verifies per-lane sha256
  • overall sha256 over original byte order

Tests added (118 new across 47 files; 444 total)

• Unit: KDF, nonce, AEAD, streaming SHA, sender/receiver, partition
• Integration: 1/4/16-lane parity, range vs round-robin parity, Bun.serve loopback at 100 KiB / 1 MiB / 8 MiB, two real Shade instances end-to-end at 64 KiB / 512 KiB / 4 MiB
• Resume: kill-restart-verify on 256 KiB with 4 lanes
• WS fallback: WS connect failure → transparent HTTP completion
• Tamper: bit-flip ciphertext / tag / header field; replay; out-of-order
• Wire: 0x11 envelope encode/decode roundtrip + edge cases

Backward compatibility

• Shade.send/receive/onMessage/fingerprint/rotate unchanged (onMessage widened to support async handlers — sync handlers still work)
• Existing wire types 0x01 (PreKeyMessage) / 0x02 (RatchetMessage) unchanged
• StorageProvider interface extension uses optional methods
• @shade/streams and @shade/transfer are new packages; no migration

[1.0.0] — 2026-04-10

First production release

Shade implements the Signal Protocol (X3DH + Double Ratchet) as a standalone, audit-friendly E2EE library for TypeScript/Bun.

Added

Core protocol

• X3DH key agreement (X25519 + Ed25519, supports asynchronous bundles)
• Double Ratchet with forward secrecy and post-compromise recovery
• Skipped message key cache for out-of-order delivery (max 1000 per chain)
• Header-bound AAD on AES-256-GCM encrypts (tampered headers fail decryption)
• Memory zeroization of message keys, chain keys, root keys, and DH private keys after use

Storage

• MemoryStorage (in-memory, for tests/embedded)
• SQLiteStorage (@shade/storage-sqlite) — bun:sqlite, WAL mode, crash-safe
• PostgresStorage (@shade/storage-postgres) — Drizzle, FOR UPDATE SKIP LOCKED
• All backends survive container restarts and SIGKILL
• Identity history with 7-day grace period for rotation

Prekey server (@shade/server)

• Hono-based REST API with self-authenticated registration (Ed25519 signatures)
• Anonymous bundle fetches (read-only)
• Per-IP and per-identity rate limiting (token bucket)
• Address validation (NFKC normalization, alphanumeric + :_-.)
• ±5 minute replay window on signed requests
• Health endpoints (/health, /healthz, /ready)
• Prometheus metrics (/metrics)
• Structured JSON logging
• Graceful shutdown on SIGTERM/SIGINT
• Production Dockerfile with non-root user, healthcheck, multi-stage build
• docker-compose.yml example for Dokploy

Session manager (@shade/core)

• ShadeSessionManager high-level API (encrypt, decrypt, initSessionFromBundle)
• getIdentityFingerprint() — Signal-style 60-digit safety numbers
• ensurePreKeyStock() — auto-replenish when below threshold
• resetSession() and acceptIdentityChange() for recovery scenarios
• rotateIdentity() with archived previous identities

Transport (@shade/transport)

• ShadeFetchTransport — HTTP client for the prekey server with auto-signing
• ShadeWebSocket — WebSocket wrapper with transparent encrypt/decrypt

Wire format (@shade/proto)

• Compact binary encoding (significantly smaller than JSON)
• Length-prefixed byte arrays, big-endian integers
• Version-tagged envelopes for forward compatibility

Cryptographic hardening

• constantTimeEqual (XOR-accumulator, no early exit)
• randomUint32 via crypto.getRandomValues (no Math.random)
• Timing-attack regression test
• Constant-time trust verification in all storage backends

Errors

• Stable SHADE_* error codes
• errorToHttpStatus for consistent HTTP mapping
• toJSON() for network serialization
• 14 specific error types (Validation, Network, Storage, RateLimit, etc.)

Documentation

• README, SECURITY.md, THREAT-MODEL.md
• 5 runnable examples (basic conversation, prekey server, WebSocket tunnel, identity verification, Dokploy deployment)
• Per-package READMEs
• Inline TSDoc throughout

Testing

• 195+ tests across all packages
• Crash recovery integration test
• Cross-platform PostgreSQL tests (skip without SHADE_TEST_PG_URL)
• CI workflow with PostgreSQL service
• Benchmark suite

Security properties

• Forward secrecy
• Post-compromise security
• Authenticated identity verification
• Replay protection
• Constant-time secret comparisons
• Memory zeroization (best-effort)