Shade/docs/files.md

@shade/files — E2EE filesystem RPC

@shade/files exposes a typed, end-to-end-encrypted RPC surface for filesystem-style operations between two Shade peers. Both sides ride a single Double Ratchet session for control envelopes; large-file content (> 256 KiB) flows through @shade/transfer lanes, correlated back to the RPC by an opaque userMetadata.shadeFilesWriteId / shadeFilesReadStreamId.

The package is a primitive, not a UI: it ships hooks and clients, not file managers. Consumers (e.g. Dispatch, Mail, Drive-style apps) keep their own UI and plug @shade/files underneath.

Quick start

Server (Bob exposes a filesystem)

import { createShade } from '@shade/sdk';

const bob = await createShade({ prekeyServer, address: 'bob' });
bob.configureTransfers({ resolveBaseUrl: ... });
Bun.serve({ port: 8080, fetch: (await bob.transferRoute()).fetch });

const stop = await bob.files.serve({
  list: async (ctx) => ({ entries: await readdirAt(ctx.path), hasMore: false }),
  stat: async (ctx) => statAt(ctx.path),
  mkdir: async (ctx) => mkdirAt(ctx.path, ctx.args.recursive),
  delete: async (ctx) => deleteAt(ctx.path, ctx.args.recursive),
  move: async (ctx) => moveAt(ctx.args.src, ctx.args.dst, ctx.args.overwrite),
  read: async (ctx) => readAt(ctx.path),     // returns inline | streams
  write: async (ctx) => writeAt(ctx.args),   // receives inline | streams
  getThumbnail: async (ctx) => thumbnailAt(ctx.path, ctx.args.size, ctx.args.format),
});

// Later: stop the handler.
await stop();

Client (Alice consumes Bob's filesystem)

const alice = await createShade({ prekeyServer, address: 'alice' });
alice.configureTransfers({ resolveBaseUrl: ... });
Bun.serve({ port: 8081, fetch: (await alice.transferRoute()).fetch });

const fs = await alice.files.client('bob');

const page = await fs.list('/photos');
await fs.write('/photos/cover.png', new Uint8Array(...));   // auto inline/streams
const result = await fs.read('/photos/cover.png');
if (result.kind === 'inline') console.log(result.bytes.byteLength);
else for await (const _chunk of /* result.stream */) { /* ... */ }

Op surface

Op	Args	Result
list	path, cursor?, pageSize?, filter?	{ entries, hasMore, nextCursor? }
stat	path	FileEntry
mkdir	path, recursive?	{ entry: FileEntry }
delete	path, recursive?	{ deletedCount }
move	src, dst, overwrite?	{ entry: FileEntry }
read	path, range?, preferInline?	inline { bytes, sha256, size } or streams { stream, size, sha256 }
write	path, Uint8Array | Blob | Stream	{ entry: FileEntry }
getThumbnail	path, size: 64|128|256|512, format?	{ bytes, format, width, height, sha256 }
custom<K>	typed via CustomOpsMap[K]	typed via CustomOpsMap[K]

MUTATION_OPS = { mkdir, delete, move, write, custom } — these auto-generate an idempotency key per logical call so transparent retries under the SDK don't produce duplicates.

Inline vs streams

The threshold is INLINE_THRESHOLD = 256 * 1024 bytes (plaintext). The client's decideInline() runs at write time:

• Uint8Array / Blob with known size → direct comparison.
• Bare ReadableStream → peek the first chunks; promote to streams as soon as the buffered prefix exceeds the threshold.

Streams paths kick shade.upload(...) with userMetadata.shadeFilesWriteId in parallel with the RPC envelope. The server's streams-bridge accepts the inbound transfer immediately into a TransformStream and parks the readable side until the matching write RPC arrives.

Custom ops

Augment CustomOpsMap once globally for type-safe consumer-defined ops:

declare module '@shade/files' {
  interface CustomOpsMap {
    'app.deploy': CustomOpDef<{ jarPath: string }, { deploymentId: string }>;
  }
}

// Server registers a Zod-backed handler:
await shade.files.serve({
  custom: {
    'app.deploy': {
      args: z.object({ jarPath: z.string() }),
      response: z.object({ deploymentId: z.string() }),
      handler: async (args, ctx) => ({ deploymentId: deploy(args.jarPath) }),
    },
  },
});

// Client gets typed I/O:
const result = await fs.custom('app.deploy', { jarPath: '/mods/foo.jar' });
//    ^? { deploymentId: string }

Production hooks

All hooks are callbacks the consumer plugs in — @shade/files enforces the mechanism; the app owns the policy.

await shade.files.serve({
  pathPolicy: { rootScope: '/srv/shade-root', forbidTraversal: true },
  rateLimits: {
    maxOpsPerMinutePerSender: 600,
    maxBytesPerHourPerSender: 10 * 1024 ** 3,
    opCost: { read: 1, write: 5, delete: 3, default: 1 },
  },
  idempotency: { ttlMs: 60 * 60 * 1000, maxEntriesPerSender: 10_000 },
  requireFingerprintVerifiedFor: (ctx) =>
    ['delete', 'write', 'mkdir'].includes(String(ctx.op)) ? 'required' : 'optional',
  isFingerprintVerified: (sender) => verifiedSet.has(sender),
  verifySender: async (sender, canonical, sig) => {
    return ed25519.verify(base64ToBytes(sig), canonical, getPubKey(sender));
  },
  onMetric: (name, value, tags) => prometheus.observe(name, value, tags),
  onError: (err, ctx) => logger.error({ op: ctx.op, sender: ctx.sender }, err),
});

React

import { ShadeFilesProvider, useFileList } from '@shade/files/react';

function FileBrowser({ shade, peer }: { shade: Shade; peer: string }) {
  return (
    <ShadeFilesProvider shade={shade}>
      <Listing peer={peer} path="/" />
    </ShadeFilesProvider>
  );
}

function Listing({ peer, path }) {
  const { entries, isLoading, hasMore, loadMore, refresh } = useFileList(peer, path);
  if (isLoading) return <Spinner />;
  return (
    <ul>
      {entries.map((e) => <li key={e.name}>{e.name} ({e.kind})</li>)}
      {hasMore && <button onClick={loadMore}>More</button>}
    </ul>
  );
}

useFileTransfer exposes a generic state machine for BulkTransferHandles returned by uploadDirectory() / downloadDirectory():

const { start, abort, state, filesDone, filesTotal, bytesDone } = useFileUpload();
const handle = uploadDirectory(fs, localDir, '/uploaded');
useEffect(() => { start(handle); return () => void abort(); }, []);

Path safety

The dispatcher applies validatePath() before invoking the user handler:

1. Length check on raw input.
2. Forbidden-bytes check (NUL/CR/LF/DEL/backslash).
3. Percent-decode (defends against %2e%2e smuggling).
4. POSIX normalization.
5. .. traversal rejection.
6. Root-scope boundary check.
7. Consumer-supplied extra predicate.

The user handler receives the normalized path; using args.path raw is a TOCTOU bug.

Wire compatibility

@shade/files 0.3.0 requires @shade/proto 0.3.0+. The proto layer's wire VERSION was bumped from 0x01 to 0x02 to lift the 64 KiB length-prefix ceiling that previously capped ratchet payloads. Sessions are incompatible across the bump; both peers must run 0.3.0+.

Related modules

• @shade/streams — chunk encryption, lane key derivation. Indirect dep.
• @shade/transfer — multi-lane transport with HTTP / WS fallback.
• @shade/sdk — Shade.files getter; BackgroundHooks.onPruneFiles for retention.