Shade/docs/files.md

# `@shade/files` — E2EE filesystem RPC

`@shade/files` exposes a typed, end-to-end-encrypted RPC surface for
filesystem-style operations between two Shade peers. Both sides ride a
single Double Ratchet session for control envelopes; large-file content
(`> 256 KiB`) flows through `@shade/transfer` lanes, correlated back to
the RPC by an opaque `userMetadata.shadeFilesWriteId` /
`shadeFilesReadStreamId`.

The package is a **primitive**, not a UI: it ships hooks and clients, not
file managers. Consumers (e.g. Dispatch, Mail, Drive-style apps) keep
their own UI and plug `@shade/files` underneath.

## Quick start

### Server (Bob exposes a filesystem)

```ts
import { createShade } from '@shade/sdk';

const bob = await createShade({ prekeyServer, address: 'bob' });
bob.configureTransfers({ resolveBaseUrl: ... });
Bun.serve({ port: 8080, fetch: (await bob.transferRoute()).fetch });

const stop = await bob.files.serve({
  list: async (ctx) => ({ entries: await readdirAt(ctx.path), hasMore: false }),
  stat: async (ctx) => statAt(ctx.path),
  mkdir: async (ctx) => mkdirAt(ctx.path, ctx.args.recursive),
  delete: async (ctx) => deleteAt(ctx.path, ctx.args.recursive),
  move: async (ctx) => moveAt(ctx.args.src, ctx.args.dst, ctx.args.overwrite),
  read: async (ctx) => readAt(ctx.path),     // returns inline | streams
  write: async (ctx) => writeAt(ctx.args),   // receives inline | streams
  getThumbnail: async (ctx) => thumbnailAt(ctx.path, ctx.args.size, ctx.args.format),
});

// Later: stop the handler.
await stop();
```

### Client (Alice consumes Bob's filesystem)

```ts
const alice = await createShade({ prekeyServer, address: 'alice' });
alice.configureTransfers({ resolveBaseUrl: ... });
Bun.serve({ port: 8081, fetch: (await alice.transferRoute()).fetch });

const fs = await alice.files.client('bob');

const page = await fs.list('/photos');
await fs.write('/photos/cover.png', new Uint8Array(...));   // auto inline/streams
const result = await fs.read('/photos/cover.png');
if (result.kind === 'inline') console.log(result.bytes.byteLength);
else for await (const _chunk of /* result.stream */) { /* ... */ }
```

## Op surface

| Op             | Args                                     | Result                                  |
|----------------|------------------------------------------|-----------------------------------------|
| `list`         | `path`, `cursor?`, `pageSize?`, `filter?`| `{ entries, hasMore, nextCursor? }`     |
| `stat`         | `path`                                   | `FileEntry`                             |
| `mkdir`        | `path`, `recursive?`                     | `{ entry: FileEntry }`                  |
| `delete`       | `path`, `recursive?`                     | `{ deletedCount }`                      |
| `move`         | `src`, `dst`, `overwrite?`               | `{ entry: FileEntry }`                  |
| `read`         | `path`, `range?`, `preferInline?`        | inline `{ bytes, sha256, size }` or streams `{ stream, size, sha256 }` |
| `write`        | `path`, `Uint8Array \| Blob \| Stream`   | `{ entry: FileEntry }`                  |
| `getThumbnail` | `path`, `size: 64\|128\|256\|512`, `format?` | `{ bytes, format, width, height, sha256 }` |
| `custom<K>`    | typed via `CustomOpsMap[K]`              | typed via `CustomOpsMap[K]`             |

`MUTATION_OPS = { mkdir, delete, move, write, custom }` — these auto-generate
an idempotency key per logical call so transparent retries under the SDK
don't produce duplicates.

## Inline vs streams

The threshold is `INLINE_THRESHOLD = 256 * 1024` bytes (plaintext). The
client's `decideInline()` runs at `write` time:

* `Uint8Array` / `Blob` with known size → direct comparison.
* Bare `ReadableStream` → peek the first chunks; promote to streams as
  soon as the buffered prefix exceeds the threshold.

Streams paths kick `shade.upload(...)` with `userMetadata.shadeFilesWriteId`
in parallel with the RPC envelope. The server's streams-bridge accepts the
inbound transfer immediately into a `TransformStream` and parks the
readable side until the matching `write` RPC arrives.

## Custom ops

Augment `CustomOpsMap` once globally for type-safe consumer-defined ops:

```ts
declare module '@shade/files' {
  interface CustomOpsMap {
    'app.deploy': CustomOpDef<{ jarPath: string }, { deploymentId: string }>;
  }
}

// Server registers a Zod-backed handler:
await shade.files.serve({
  custom: {
    'app.deploy': {
      args: z.object({ jarPath: z.string() }),
      response: z.object({ deploymentId: z.string() }),
      handler: async (args, ctx) => ({ deploymentId: deploy(args.jarPath) }),
    },
  },
});

// Client gets typed I/O:
const result = await fs.custom('app.deploy', { jarPath: '/mods/foo.jar' });
//    ^? { deploymentId: string }
```

## Production hooks

All hooks are callbacks the consumer plugs in — `@shade/files` enforces
the *mechanism*; the app owns the *policy*.

```ts
await shade.files.serve({
  pathPolicy: { rootScope: '/srv/shade-root', forbidTraversal: true },
  rateLimits: {
    maxOpsPerMinutePerSender: 600,
    maxBytesPerHourPerSender: 10 * 1024 ** 3,
    opCost: { read: 1, write: 5, delete: 3, default: 1 },
  },
  idempotency: { ttlMs: 60 * 60 * 1000, maxEntriesPerSender: 10_000 },
  requireFingerprintVerifiedFor: (ctx) =>
    ['delete', 'write', 'mkdir'].includes(String(ctx.op)) ? 'required' : 'optional',
  isFingerprintVerified: (sender) => verifiedSet.has(sender),
  verifySender: async (sender, canonical, sig) => {
    return ed25519.verify(base64ToBytes(sig), canonical, getPubKey(sender));
  },
  onMetric: (name, value, tags) => prometheus.observe(name, value, tags),
  onError: (err, ctx) => logger.error({ op: ctx.op, sender: ctx.sender }, err),
});
```

## React

```tsx
import { ShadeFilesProvider, useFileList } from '@shade/files/react';

function FileBrowser({ shade, peer }: { shade: Shade; peer: string }) {
  return (
    <ShadeFilesProvider shade={shade}>
      <Listing peer={peer} path="/" />
    </ShadeFilesProvider>
  );
}

function Listing({ peer, path }) {
  const { entries, isLoading, hasMore, loadMore, refresh } = useFileList(peer, path);
  if (isLoading) return <Spinner />;
  return (
    <ul>
      {entries.map((e) => <li key={e.name}>{e.name} ({e.kind})</li>)}
      {hasMore && <button onClick={loadMore}>More</button>}
    </ul>
  );
}
```

`useFileTransfer` exposes a generic state machine for `BulkTransferHandle`s
returned by `uploadDirectory()` / `downloadDirectory()`:

```tsx
const { start, abort, state, filesDone, filesTotal, bytesDone } = useFileUpload();
const handle = uploadDirectory(fs, localDir, '/uploaded');
useEffect(() => { start(handle); return () => void abort(); }, []);
```

## Path safety

The dispatcher applies `validatePath()` before invoking the user handler:

1. Length check on raw input.
2. Forbidden-bytes check (NUL/CR/LF/DEL/backslash).
3. Percent-decode (defends against `%2e%2e` smuggling).
4. POSIX normalization.
5. `..` traversal rejection.
6. Root-scope boundary check.
7. Consumer-supplied `extra` predicate.

The user handler receives the *normalized* path; using `args.path` raw is a
TOCTOU bug.

## Wire compatibility

`@shade/files` 0.3.0 requires `@shade/proto` 0.3.0+. The proto layer's wire
VERSION was bumped from `0x01` to `0x02` to lift the 64 KiB length-prefix
ceiling that previously capped ratchet payloads. **Sessions are
incompatible across the bump**; both peers must run 0.3.0+.

## Rich file metadata + previews (V3.9)

`stream-init` carries optional E2EE `fileMetadata` (filename, MIME,
thumbnail-stream pointer). `@shade/files` consumers see this on the
incoming-transfer side and can render previews via `<TransferRow
showThumbnail />`. The thumbnail itself rides as a separate AEAD
stream — server never sees preview pixels in plaintext.

See [streams.md § Rich file metadata + previews](streams.md#rich-file-metadata--previews-v39)
for the wire format, format-hardening rules, and renderer trust
model. The pattern integrates seamlessly with `@shade/files`'s own
write/read RPCs — pass `fileMetadata` in the underlying
`shade.upload` and the same `ShadeThumbnailCache` powers previews
across all transfer surfaces.

## Related modules

* `@shade/streams` — chunk encryption, lane key derivation. Indirect dep.
* `@shade/transfer` — multi-lane transport with HTTP / WS fallback.
* `@shade/transport-webrtc` (V3.11, optional) — direct P2P chunk
  delivery via `RTCDataChannel`; large `read`/`write` payloads
  automatically prefer WebRTC when both peers have called
  `shade.configureWebRTC()`.
* `@shade/sdk` — `Shade.files` getter; `BackgroundHooks.onPruneFiles` for
  retention.