Shade/docs/files.md

@shade/files — E2EE filesystem RPC

@shade/files exposes a typed, end-to-end-encrypted RPC surface for filesystem-style operations between two Shade peers. Both sides ride a single Double Ratchet session for control envelopes; large-file content (> 256 KiB) flows through @shade/transfer lanes, correlated back to the RPC by an opaque userMetadata.shadeFilesWriteId / shadeFilesReadStreamId.

The package is a primitive, not a UI: it ships hooks and clients, not file managers. Consumers (e.g. Dispatch, Mail, Drive-style apps) keep their own UI and plug @shade/files underneath.

Quick start

Server (Bob exposes a filesystem)

import { createShade } from '@shade/sdk';

const bob = await createShade({ prekeyServer, address: 'bob' });
bob.configureTransfers({ resolveBaseUrl: ... });
Bun.serve({ port: 8080, fetch: (await bob.transferRoute()).fetch });

const stop = await bob.files.serve({
  list: async (ctx) => ({ entries: await readdirAt(ctx.path), hasMore: false }),
  stat: async (ctx) => statAt(ctx.path),
  mkdir: async (ctx) => mkdirAt(ctx.path, ctx.args.recursive),
  delete: async (ctx) => deleteAt(ctx.path, ctx.args.recursive),
  move: async (ctx) => moveAt(ctx.args.src, ctx.args.dst, ctx.args.overwrite),
  read: async (ctx) => readAt(ctx.path),     // returns inline | streams
  write: async (ctx) => writeAt(ctx.args),   // receives inline | streams
  getThumbnail: async (ctx) => thumbnailAt(ctx.path, ctx.args.size, ctx.args.format),
});

// Later: stop the handler.
await stop();

Client (Alice consumes Bob's filesystem)

const alice = await createShade({ prekeyServer, address: 'alice' });
alice.configureTransfers({ resolveBaseUrl: ... });
Bun.serve({ port: 8081, fetch: (await alice.transferRoute()).fetch });

const fs = await alice.files.client('bob');

const page = await fs.list('/photos');
await fs.write('/photos/cover.png', new Uint8Array(...));   // auto inline/streams
const result = await fs.read('/photos/cover.png');
if (result.kind === 'inline') console.log(result.bytes.byteLength);
else for await (const _chunk of /* result.stream */) { /* ... */ }

HTTP RPC — browser-friendly request-response (4.1+)

The default shade.files.client(peer) requires both peers to be mutually addressable over HTTP — the response to a list/read etc. round-trips through Shade.deliverControlEnvelope, which POSTs to the peer's /v1/transfer/control endpoint. That doesn't work for browsers — a tab can't host an HTTP server, so the server cannot call back outbound.

@shade/files 4.1 ships a parallel request-response transport that lets browser-style clients fully consume the file-RPC surface without any inbound channel. It mirrors the way @shade/server's shade-auth-middleware works: one POST per RPC, encrypted envelope in the request body, encrypted response in the same HTTP response.

Server side — mount the RPC route

// 1. Register the file handler. `inlineOnly: true` skips the
//    streams-bridge (which would require @shade/transfer).
await shade.files.serve(handlerConfig, { inlineOnly: true });

// 2. Mount the route on your Hono app under any base path.
app.route('/api/v1/shade-files', shade.files.rpcRoute());
//                                         ^^^^^^^^^^^^^^
//                                         POST <base>/rpc

rpcRoute() accepts:

Option	Default	Purpose
maxBodyBytes	1 MiB	Max request body. The protocol caps inline payloads at 256 KiB; the headroom is for base64 inflation + custom-op envelopes.
acceptFirstMessage	false	Accept 0x01 PreKeyMessage envelopes — required when the RPC route also doubles as the X3DH handshake (browser's first-ever request).

Browser client

import { createShade } from '@shade/sdk';

const shade = await createShade({
  prekeyServer: 'https://shade.example.com',
  storage: 'memory',
  address: 'alice@example.com',
});

const fs = shade.files.httpClient('bob@example.com', {
  rpcUrl: 'https://dispatch.example.com/api/v1/shade-files/rpc',
  // Optional: thread CSRF / auth tokens, override fetch, etc.
  headers: { 'X-CSRF-Token': csrfToken },
});

await fs.mkdir('/photos');
await fs.write('/photos/cover.png', new Uint8Array([/* ... */]), {
  contentType: 'image/png',
});
const result = await fs.read('/photos/cover.png');

What works in HTTP-RPC mode

• list, stat, mkdir, delete, move, getThumbnail, custom<K> — full parity.
• write — inline only (≤ 256 KiB plaintext). Larger inputs throw ConflictError.
• read — inline only. If the server returns a streamed read result, the client throws InternalFileError directing callers to the stateful pathway.

Wire contract

POST /rpc HTTP/1.1
Content-Type: application/octet-stream
X-Shade-Sender-Address: alice@example.com

<wire-encoded ShadeEnvelope (0x01 first-time, 0x02 after) wrapping
 JSON-encoded RpcRequest>

────

200 OK
Content-Type: application/octet-stream

<wire-encoded ShadeEnvelope (0x02) wrapping JSON-encoded RpcResponse | RpcError>

Transport-level failures (no session, undecryptable envelope, body too big) return JSON { "error": "..." } with appropriate 4xx status. Application-level failures (file not found, permission denied) ship encrypted RpcError envelopes — the client maps them back to typed FileError subclasses (NotFoundError, ConflictError, etc.).

Symmetry with @shade/server

The shape mirrors @shade/server's shade-auth-middleware: encrypted envelope rides the request body, server decrypts via the existing ratchet session, performs the protected operation, returns an encrypted envelope in the response. No bidirectional channel required, no WebSocket, no SSE.

When to use which

Setup	Use
Browser client ↔ Bun/Hono server	httpClient() + rpcRoute()
Server ↔ server (both can host HTTP)	client() (default) — supports streams
Service-worker / extension ↔ server	httpClient() (no inbound listener)
CLI / daemon ↔ daemon	Either; client() if you need streams

Op surface

Op	Args	Result
list	path, cursor?, pageSize?, filter?	{ entries, hasMore, nextCursor? }
stat	path	FileEntry
mkdir	path, recursive?	{ entry: FileEntry }
delete	path, recursive?	{ deletedCount }
move	src, dst, overwrite?	{ entry: FileEntry }
read	path, range?, preferInline?	inline { bytes, sha256, size } or streams { stream, size, sha256 }
write	path, Uint8Array | Blob | Stream	{ entry: FileEntry }
getThumbnail	path, size: 64|128|256|512, format?	{ bytes, format, width, height, sha256 }
custom<K>	typed via CustomOpsMap[K]	typed via CustomOpsMap[K]

MUTATION_OPS = { mkdir, delete, move, write, custom } — these auto-generate an idempotency key per logical call so transparent retries under the SDK don't produce duplicates.

Inline vs streams

The threshold is INLINE_THRESHOLD = 256 * 1024 bytes (plaintext). The client's decideInline() runs at write time:

• Uint8Array / Blob with known size → direct comparison.
• Bare ReadableStream → peek the first chunks; promote to streams as soon as the buffered prefix exceeds the threshold.

Streams paths kick shade.upload(...) with userMetadata.shadeFilesWriteId in parallel with the RPC envelope. The server's streams-bridge accepts the inbound transfer immediately into a TransformStream and parks the readable side until the matching write RPC arrives.

Custom ops

Augment CustomOpsMap once globally for type-safe consumer-defined ops:

declare module '@shade/files' {
  interface CustomOpsMap {
    'app.deploy': CustomOpDef<{ jarPath: string }, { deploymentId: string }>;
  }
}

// Server registers a Zod-backed handler:
await shade.files.serve({
  custom: {
    'app.deploy': {
      args: z.object({ jarPath: z.string() }),
      response: z.object({ deploymentId: z.string() }),
      handler: async (args, ctx) => ({ deploymentId: deploy(args.jarPath) }),
    },
  },
});

// Client gets typed I/O:
const result = await fs.custom('app.deploy', { jarPath: '/mods/foo.jar' });
//    ^? { deploymentId: string }

Production hooks

All hooks are callbacks the consumer plugs in — @shade/files enforces the mechanism; the app owns the policy.

await shade.files.serve({
  pathPolicy: { rootScope: '/srv/shade-root', forbidTraversal: true },
  rateLimits: {
    maxOpsPerMinutePerSender: 600,
    maxBytesPerHourPerSender: 10 * 1024 ** 3,
    opCost: { read: 1, write: 5, delete: 3, default: 1 },
  },
  idempotency: { ttlMs: 60 * 60 * 1000, maxEntriesPerSender: 10_000 },
  requireFingerprintVerifiedFor: (ctx) =>
    ['delete', 'write', 'mkdir'].includes(String(ctx.op)) ? 'required' : 'optional',
  isFingerprintVerified: (sender) => verifiedSet.has(sender),
  verifySender: async (sender, canonical, sig) => {
    return ed25519.verify(base64ToBytes(sig), canonical, getPubKey(sender));
  },
  onMetric: (name, value, tags) => prometheus.observe(name, value, tags),
  onError: (err, ctx) => logger.error({ op: ctx.op, sender: ctx.sender }, err),
});

React

import { ShadeFilesProvider, useFileList } from '@shade/files/react';

function FileBrowser({ shade, peer }: { shade: Shade; peer: string }) {
  return (
    <ShadeFilesProvider shade={shade}>
      <Listing peer={peer} path="/" />
    </ShadeFilesProvider>
  );
}

function Listing({ peer, path }) {
  const { entries, isLoading, hasMore, loadMore, refresh } = useFileList(peer, path);
  if (isLoading) return <Spinner />;
  return (
    <ul>
      {entries.map((e) => <li key={e.name}>{e.name} ({e.kind})</li>)}
      {hasMore && <button onClick={loadMore}>More</button>}
    </ul>
  );
}

useFileTransfer exposes a generic state machine for BulkTransferHandles returned by uploadDirectory() / downloadDirectory():

const { start, abort, state, filesDone, filesTotal, bytesDone } = useFileUpload();
const handle = uploadDirectory(fs, localDir, '/uploaded');
useEffect(() => { start(handle); return () => void abort(); }, []);

Path safety

The dispatcher applies validatePath() before invoking the user handler:

1. Length check on raw input.
2. Forbidden-bytes check (NUL/CR/LF/DEL/backslash).
3. Percent-decode (defends against %2e%2e smuggling).
4. POSIX normalization.
5. .. traversal rejection.
6. Root-scope boundary check.
7. Consumer-supplied extra predicate.

The user handler receives the normalized path; using args.path raw is a TOCTOU bug.

Wire compatibility

@shade/files 0.3.0 requires @shade/proto 0.3.0+. The proto layer's wire VERSION was bumped from 0x01 to 0x02 to lift the 64 KiB length-prefix ceiling that previously capped ratchet payloads. Sessions are incompatible across the bump; both peers must run 0.3.0+.

Rich file metadata + previews (V3.9)

stream-init carries optional E2EE fileMetadata (filename, MIME, thumbnail-stream pointer). @shade/files consumers see this on the incoming-transfer side and can render previews via <TransferRow showThumbnail />. The thumbnail itself rides as a separate AEAD stream — server never sees preview pixels in plaintext.

See streams.md § Rich file metadata + previews for the wire format, format-hardening rules, and renderer trust model. The pattern integrates seamlessly with @shade/files's own write/read RPCs — pass fileMetadata in the underlying shade.upload and the same ShadeThumbnailCache powers previews across all transfer surfaces.

Related modules

• @shade/streams — chunk encryption, lane key derivation. Indirect dep.
• @shade/transfer — multi-lane transport with HTTP / WS fallback.
• @shade/transport-webrtc (V3.11, optional) — direct P2P chunk delivery via RTCDataChannel; large read/write payloads automatically prefer WebRTC when both peers have called shade.configureWebRTC().
• @shade/sdk — Shade.files getter; BackgroundHooks.onPruneFiles for retention.