# `@shade/files` — E2EE filesystem RPC `@shade/files` exposes a typed, end-to-end-encrypted RPC surface for filesystem-style operations between two Shade peers. Both sides ride a single Double Ratchet session for control envelopes; large-file content (`> 256 KiB`) flows through `@shade/transfer` lanes, correlated back to the RPC by an opaque `userMetadata.shadeFilesWriteId` / `shadeFilesReadStreamId`. The package is a **primitive**, not a UI: it ships hooks and clients, not file managers. Consumers (e.g. Dispatch, Mail, Drive-style apps) keep their own UI and plug `@shade/files` underneath. ## Quick start ### Server (Bob exposes a filesystem) ```ts import { createShade } from '@shade/sdk'; const bob = await createShade({ prekeyServer, address: 'bob' }); bob.configureTransfers({ resolveBaseUrl: ... }); Bun.serve({ port: 8080, fetch: (await bob.transferRoute()).fetch }); const stop = await bob.files.serve({ list: async (ctx) => ({ entries: await readdirAt(ctx.path), hasMore: false }), stat: async (ctx) => statAt(ctx.path), mkdir: async (ctx) => mkdirAt(ctx.path, ctx.args.recursive), delete: async (ctx) => deleteAt(ctx.path, ctx.args.recursive), move: async (ctx) => moveAt(ctx.args.src, ctx.args.dst, ctx.args.overwrite), read: async (ctx) => readAt(ctx.path), // returns inline | streams write: async (ctx) => writeAt(ctx.args), // receives inline | streams getThumbnail: async (ctx) => thumbnailAt(ctx.path, ctx.args.size, ctx.args.format), }); // Later: stop the handler. await stop(); ``` ### Client (Alice consumes Bob's filesystem) ```ts const alice = await createShade({ prekeyServer, address: 'alice' }); alice.configureTransfers({ resolveBaseUrl: ... }); Bun.serve({ port: 8081, fetch: (await alice.transferRoute()).fetch }); const fs = await alice.files.client('bob'); const page = await fs.list('/photos'); await fs.write('/photos/cover.png', new Uint8Array(...)); // auto inline/streams const result = await fs.read('/photos/cover.png'); if (result.kind === 'inline') console.log(result.bytes.byteLength); else for await (const _chunk of /* result.stream */) { /* ... */ } ``` ## HTTP RPC — browser-friendly request-response (4.1+) The default `shade.files.client(peer)` requires both peers to be mutually addressable over HTTP — the response to a `list`/`read` etc. round-trips through `Shade.deliverControlEnvelope`, which POSTs to the peer's `/v1/transfer/control` endpoint. **That doesn't work for browsers** — a tab can't host an HTTP server, so the server cannot call back outbound. `@shade/files` 4.1 ships a parallel **request-response** transport that lets browser-style clients fully consume the file-RPC surface without any inbound channel. It mirrors the way `@shade/server`'s `shade-auth-middleware` works: one POST per RPC, encrypted envelope in the request body, encrypted response in the same HTTP response. ### Server side — mount the RPC route ```ts // 1. Register the file handler. `inlineOnly: true` skips the // streams-bridge (which would require @shade/transfer). await shade.files.serve(handlerConfig, { inlineOnly: true }); // 2. Mount the route on your Hono app under any base path. app.route('/api/v1/shade-files', shade.files.rpcRoute()); // ^^^^^^^^^^^^^^ // POST /rpc ``` `rpcRoute()` accepts: | Option | Default | Purpose | |---------------------|---------|----------------------------------------------------------------------------------------------------| | `maxBodyBytes` | 1 MiB | Max request body. The protocol caps inline payloads at 256 KiB; the headroom is for base64 inflation + custom-op envelopes. | | `acceptFirstMessage`| `false` | Accept `0x01` PreKeyMessage envelopes — required when the RPC route also doubles as the X3DH handshake (browser's first-ever request). | ### Browser client ```ts import { createShade } from '@shade/sdk'; const shade = await createShade({ prekeyServer: 'https://shade.example.com', storage: 'memory', address: 'alice@example.com', }); const fs = shade.files.httpClient('bob@example.com', { rpcUrl: 'https://dispatch.example.com/api/v1/shade-files/rpc', // Optional: thread CSRF / auth tokens, override fetch, etc. headers: { 'X-CSRF-Token': csrfToken }, }); await fs.mkdir('/photos'); await fs.write('/photos/cover.png', new Uint8Array([/* ... */]), { contentType: 'image/png', }); const result = await fs.read('/photos/cover.png'); ``` ### What works in HTTP-RPC mode - `list`, `stat`, `mkdir`, `delete`, `move`, `getThumbnail`, `custom` — full parity. - `write` — **inline only** (≤ 256 KiB plaintext). Larger inputs throw `ConflictError`. - `read` — **inline only**. If the server returns a streamed `read` result, the client throws `InternalFileError` directing callers to the stateful pathway. ### Wire contract ``` POST /rpc HTTP/1.1 Content-Type: application/octet-stream X-Shade-Sender-Address: alice@example.com ──── 200 OK Content-Type: application/octet-stream ``` Transport-level failures (no session, undecryptable envelope, body too big) return JSON `{ "error": "..." }` with appropriate 4xx status. Application-level failures (file not found, permission denied) ship encrypted `RpcError` envelopes — the client maps them back to typed `FileError` subclasses (`NotFoundError`, `ConflictError`, etc.). ### Symmetry with `@shade/server` The shape mirrors `@shade/server`'s shade-auth-middleware: encrypted envelope rides the request body, server decrypts via the existing ratchet session, performs the protected operation, returns an encrypted envelope in the response. No bidirectional channel required, no WebSocket, no SSE. ### When to use which | Setup | Use | |-----------------------------------------------|-----------------------------------------------| | Browser client ↔ Bun/Hono server | `httpClient()` + `rpcRoute()` | | Server ↔ server (both can host HTTP) | `client()` (default) — supports streams | | Service-worker / extension ↔ server | `httpClient()` (no inbound listener) | | CLI / daemon ↔ daemon | Either; `client()` if you need streams | ## Op surface | Op | Args | Result | |----------------|------------------------------------------|-----------------------------------------| | `list` | `path`, `cursor?`, `pageSize?`, `filter?`| `{ entries, hasMore, nextCursor? }` | | `stat` | `path` | `FileEntry` | | `mkdir` | `path`, `recursive?` | `{ entry: FileEntry }` | | `delete` | `path`, `recursive?` | `{ deletedCount }` | | `move` | `src`, `dst`, `overwrite?` | `{ entry: FileEntry }` | | `read` | `path`, `range?`, `preferInline?` | inline `{ bytes, sha256, size }` or streams `{ stream, size, sha256 }` | | `write` | `path`, `Uint8Array \| Blob \| Stream` | `{ entry: FileEntry }` | | `getThumbnail` | `path`, `size: 64\|128\|256\|512`, `format?` | `{ bytes, format, width, height, sha256 }` | | `custom` | typed via `CustomOpsMap[K]` | typed via `CustomOpsMap[K]` | `MUTATION_OPS = { mkdir, delete, move, write, custom }` — these auto-generate an idempotency key per logical call so transparent retries under the SDK don't produce duplicates. ## Inline vs streams The threshold is `INLINE_THRESHOLD = 256 * 1024` bytes (plaintext). The client's `decideInline()` runs at `write` time: * `Uint8Array` / `Blob` with known size → direct comparison. * Bare `ReadableStream` → peek the first chunks; promote to streams as soon as the buffered prefix exceeds the threshold. Streams paths kick `shade.upload(...)` with `userMetadata.shadeFilesWriteId` in parallel with the RPC envelope. The server's streams-bridge accepts the inbound transfer immediately into a `TransformStream` and parks the readable side until the matching `write` RPC arrives. ## Custom ops Augment `CustomOpsMap` once globally for type-safe consumer-defined ops: ```ts declare module '@shade/files' { interface CustomOpsMap { 'app.deploy': CustomOpDef<{ jarPath: string }, { deploymentId: string }>; } } // Server registers a Zod-backed handler: await shade.files.serve({ custom: { 'app.deploy': { args: z.object({ jarPath: z.string() }), response: z.object({ deploymentId: z.string() }), handler: async (args, ctx) => ({ deploymentId: deploy(args.jarPath) }), }, }, }); // Client gets typed I/O: const result = await fs.custom('app.deploy', { jarPath: '/mods/foo.jar' }); // ^? { deploymentId: string } ``` ## Production hooks All hooks are callbacks the consumer plugs in — `@shade/files` enforces the *mechanism*; the app owns the *policy*. ```ts await shade.files.serve({ pathPolicy: { rootScope: '/srv/shade-root', forbidTraversal: true }, rateLimits: { maxOpsPerMinutePerSender: 600, maxBytesPerHourPerSender: 10 * 1024 ** 3, opCost: { read: 1, write: 5, delete: 3, default: 1 }, }, idempotency: { ttlMs: 60 * 60 * 1000, maxEntriesPerSender: 10_000 }, requireFingerprintVerifiedFor: (ctx) => ['delete', 'write', 'mkdir'].includes(String(ctx.op)) ? 'required' : 'optional', isFingerprintVerified: (sender) => verifiedSet.has(sender), verifySender: async (sender, canonical, sig) => { return ed25519.verify(base64ToBytes(sig), canonical, getPubKey(sender)); }, onMetric: (name, value, tags) => prometheus.observe(name, value, tags), onError: (err, ctx) => logger.error({ op: ctx.op, sender: ctx.sender }, err), }); ``` ## React ```tsx import { ShadeFilesProvider, useFileList } from '@shade/files/react'; function FileBrowser({ shade, peer }: { shade: Shade; peer: string }) { return ( ); } function Listing({ peer, path }) { const { entries, isLoading, hasMore, loadMore, refresh } = useFileList(peer, path); if (isLoading) return ; return (
    {entries.map((e) =>
  • {e.name} ({e.kind})
    • )} {hasMore && }
); } ``` `useFileTransfer` exposes a generic state machine for `BulkTransferHandle`s returned by `uploadDirectory()` / `downloadDirectory()`: ```tsx const { start, abort, state, filesDone, filesTotal, bytesDone } = useFileUpload(); const handle = uploadDirectory(fs, localDir, '/uploaded'); useEffect(() => { start(handle); return () => void abort(); }, []); ``` ## Path safety The dispatcher applies `validatePath()` before invoking the user handler: 1. Length check on raw input. 2. Forbidden-bytes check (NUL/CR/LF/DEL/backslash). 3. Percent-decode (defends against `%2e%2e` smuggling). 4. POSIX normalization. 5. `..` traversal rejection. 6. Root-scope boundary check. 7. Consumer-supplied `extra` predicate. The user handler receives the *normalized* path; using `args.path` raw is a TOCTOU bug. ## Wire compatibility `@shade/files` 0.3.0 requires `@shade/proto` 0.3.0+. The proto layer's wire VERSION was bumped from `0x01` to `0x02` to lift the 64 KiB length-prefix ceiling that previously capped ratchet payloads. **Sessions are incompatible across the bump**; both peers must run 0.3.0+. ## Rich file metadata + previews (V3.9) `stream-init` carries optional E2EE `fileMetadata` (filename, MIME, thumbnail-stream pointer). `@shade/files` consumers see this on the incoming-transfer side and can render previews via ``. The thumbnail itself rides as a separate AEAD stream — server never sees preview pixels in plaintext. See [streams.md § Rich file metadata + previews](streams.md#rich-file-metadata--previews-v39) for the wire format, format-hardening rules, and renderer trust model. The pattern integrates seamlessly with `@shade/files`'s own write/read RPCs — pass `fileMetadata` in the underlying `shade.upload` and the same `ShadeThumbnailCache` powers previews across all transfer surfaces. ## Related modules * `@shade/streams` — chunk encryption, lane key derivation. Indirect dep. * `@shade/transfer` — multi-lane transport with HTTP / WS fallback. * `@shade/transport-webrtc` (V3.11, optional) — direct P2P chunk delivery via `RTCDataChannel`; large `read`/`write` payloads automatically prefer WebRTC when both peers have called `shade.configureWebRTC()`. * `@shade/sdk` — `Shade.files` getter; `BackgroundHooks.onPruneFiles` for retention.