Files

Test / test (push) Has been cancelled

Details

feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive

M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).

@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
  Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
  + per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
  (> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
  / shadeFilesReadStreamId correlation. Server-side TransformStream
  bridges accept inbound transfers immediately (engine rejects chunks
  that arrive before accept) and park the readable for the matching
  RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
  uploadDirectory()/downloadDirectory() with bounded concurrency pool
  (default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
  byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
  policy (traversal + percent-decode hardening), fingerprint gate
  (required/optional/reject), pluggable Ed25519 sig verification with
  ±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
  useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
  entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.

Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
  u16 to u32. The previous u16 silently truncated payloads above
  64 KiB — a hard correctness ceiling that blocked inline file ops
  up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
  only. Cross-platform Kotlin port (android/shade-android) updated to
  match; test-vectors/wire-format.json regenerated.

Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
  Concurrent decryptions of the same peer raced ratchet state
  (manifested as sporadic "Failed to decrypt — wrong key or tampered
  data" under load — surfaced once concurrent uploadDirectory pumped
  many writes in flight). Encrypt was already serialized via
  Shade.send's encryptChains; decrypt is now serialized at the
  manager layer too.

@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
  application-level key/value pairs that round-trip verbatim through
  stream-init plaintext. Used by @shade/files for write/read
  correlation; available to any consumer.

@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
  BackgroundTasks.setHook(name, fn) for runtime hook registration.

Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
  shade-sdk streams-bridge + shade-widgets transfer hooks were
  uncommitted prior to this session. Including them keeps the
  workspace consistent at 0.3.0 since @shade/files depends on them.

Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
  3× stable). Coverage spans unit (inline-threshold + concurrency),
  integration (read-write inline + streams up to 1 MiB, walk +
  upload/download directory, custom-op, metrics, SDK namespace
  end-to-end), and security (tampered-envelope sig verification,
  replay window, fingerprint gate, rate-limit + quota).

Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
  topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
  server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-02 14:00:01 +02:00

4.8 KiB

Raw Blame History

Shade by scenario — modular E2EE toolkit

This page is for builders, not cryptography specialists. Shade is packaged so you can drop in small pieces per project instead of importing a heavy “everything” stack.

Plain-language mental model

Identity & session (the hard crypto): Shade establishes a secret channel between two parties using the same kind of cryptographic core as Signal (initial setup + ongoing “ratchet” updates). You mostly call high-level APIs (send, receive, fingerprints) rather than assembling primitives by hand.
Who is “the server”?
The prekey server only helps with public key material (so Alice can fetch Bob’s public bundle before the first message). It is not your general-purpose message relay unless you build that separately. Normal message payloads and file chunks typically flow over your transport (your HTTP routes, websocket, bridge, queue, etc.).
Small vs large payloads:
Short messages ride the usual ratchet envelopes. Very large payloads use Streams + Transfer: secrets are negotiated over the ratchet; ** ciphertext chunks** ship over optimized HTTP/WebSocket transports with parallelism and resume.
Trust: Strong encryption does not replace verifying who you are talking to. For high-stakes use, compare safety numbers out-of-band (see THREAT-MODEL.md).

Scenario → minimum packages

Pull in one row that matches your project; add optional columns only when needed.

Scenario	What you need	Minimum packages / surface	Where to start
Backend or Bun service — encrypted messages between users	Session storage + crypto provider + prekey URL	`@shade/sdk` + `sqlite:` or `@shade/storage-postgres`	`createShade()` → `send` / `receive`
Browser / frontend — same, in the client	Web crypto + durable or memory storage	`@shade/sdk` or `@shade/core` + `@shade/crypto-web` (+ storage you provide)	Same APIs; ensure `prekeyServer` is reachable from the browser (CORS, etc.)
Large files — resumable E2EE upload/download	Above + stream protocol + HTTP (or WS) transport	`@shade/sdk` (re-exports transfer) + mount transfer routes on your HTTP server	`shade.upload` / `onIncomingTransfer` — see streams.md
React UI — upload/download widgets	Runtime from SDK + widgets	`@shade/sdk` + `@shade/widgets`	`ShadeRuntimeProvider`, `useShadeUpload` / `useShadeDownload`
Prekey hosting only — one container per product	No app crypto in the container	Docker image / `@shade/server`	Deploy prekey image; point `prekeyServer` at it from apps
Maximum control — custom wire, custom transport	Wire + session manager	`@shade/core` + `@shade/proto` (+ your storage + crypto provider)	`ShadeSessionManager`, encode/decode envelopes yourself
HTTP or WebSocket convenience	Auto-wrap application bytes	`@shade/transport` on top of your stack	Use when you want transport helpers, not a new protocol
Android	Byte-compatible with TS (roadmap)	`shade-android` module	See android/shade-android/README.md — parity work in progress

You can mix rows: e.g. backend with @shade/sdk + SQLite for sessions, separate service mounting transfer routes, browser clients using @shade/widgets.

New project checklist (lightweight)

Run a prekey server (Docker or embedded @shade/server) for your environment.
Pick storage (sqlite:…, Postgres, or project-specific adapter implementing the core storage interfaces).
Choose surface: usually @shade/sdk unless you truly need @shade/core only.
For files: enable transfer routes and authenticate chunk uploads using the patterns in the SDK (see streams doc).
Run shade doctor when something fails in production-ish setups (install the CLI as in repository Quick start); coverage is evolving — roadmap in V2.2.

Topic	Doc
File transfer architecture	streams.md
Deployment & operations	DEPLOYMENT.md
Threat model	THREAT-MODEL.md
Planned improvements	V2.1, feature backlog V2.2, trust/ops V2.3

Version note

This file describes how Shade is intended to be composed. Package names and re-exports may gain small aliases over time; the scenario table should remain the source of truth for “what to install for what job.” Update this page when adding major surfaces (new transport bridges, richer shade init templates, etc.).

4.8 KiB Raw Blame History Unescape Escape