Stian/Shade
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
Some checks failed
Test / test (push) Has been cancelled
Details

Browse Source 
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).

@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
  Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
  + per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
  (> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
  / shadeFilesReadStreamId correlation. Server-side TransformStream
  bridges accept inbound transfers immediately (engine rejects chunks
  that arrive before accept) and park the readable for the matching
  RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
  uploadDirectory()/downloadDirectory() with bounded concurrency pool
  (default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
  byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
  policy (traversal + percent-decode hardening), fingerprint gate
  (required/optional/reject), pluggable Ed25519 sig verification with
  ±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
  useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
  entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.

Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
  u16 to u32. The previous u16 silently truncated payloads above
  64 KiB — a hard correctness ceiling that blocked inline file ops
  up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
  only. Cross-platform Kotlin port (android/shade-android) updated to
  match; test-vectors/wire-format.json regenerated.

Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
  Concurrent decryptions of the same peer raced ratchet state
  (manifested as sporadic "Failed to decrypt — wrong key or tampered
  data" under load — surfaced once concurrent uploadDirectory pumped
  many writes in flight). Encrypt was already serialized via
  Shade.send's encryptChains; decrypt is now serialized at the
  manager layer too.

@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
  application-level key/value pairs that round-trip verbatim through
  stream-init plaintext. Used by @shade/files for write/read
  correlation; available to any consumer.

@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
  BackgroundTasks.setHook(name, fn) for runtime hook registration.

Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
  shade-sdk streams-bridge + shade-widgets transfer hooks were
  uncommitted prior to this session. Including them keeps the
  workspace consistent at 0.3.0 since @shade/files depends on them.

Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
  3× stable). Coverage spans unit (inline-threshold + concurrency),
  integration (read-write inline + streams up to 1 MiB, walk +
  upload/download directory, custom-op, metrics, SDK namespace
  end-to-end), and security (tampered-envelope sig verification,
  replay window, fingerprint gate, rate-limit + quota).

Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
  topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
  server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sterister
2026-05-02 14:00:01 +02:00
parent 7e0f7320a9
commit fa770d3063
198 changed files with 20412 additions and 256 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
packages/shade-files/tests/integration/custom-op.test.ts Normal file
View File

@@ -0,0 +1,77 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { z } from 'zod';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
import { CustomOpRejectedError, NotImplementedError } from '../../src/index.js';
// Module augmentation: register a typed custom op for the test.
declare module '../../src/index.js' {
interface CustomOpsMap {
'test.echo': { args: { message: string }; response: { echoed: string } };
'test.add': { args: { a: number; b: number }; response: { sum: number } };
}
}
describe('Custom ops — registry + Zod validation + typed I/O', () => {
let rig: FileTestRig;
const callLog: string[] = [];
beforeAll(async () => {
rig = await setupFileRig({
custom: {
'test.echo': {
args: z.object({ message: z.string().min(1).max(64) }),
response: z.object({ echoed: z.string() }),
handler: async (args, ctx) => {
callLog.push(`echo:${ctx.sender}:${args.message}`);
return { echoed: args.message.toUpperCase() };
},
},
'test.add': {
args: z.object({ a: z.number(), b: z.number() }),
response: z.object({ sum: z.number() }),
handler: async (args) => ({ sum: args.a + args.b }),
},
'test.bad-response': {
args: z.object({}),
response: z.object({ x: z.number() }),
// Returns wrong shape on purpose.
handler: async () => ({ y: 'not-a-number' }) as unknown as { x: number },
},
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('typed echo round-trips through registered Zod schemas', async () => {
const result = await rig.fs.custom('test.echo', { message: 'hello' });
expect(result.echoed).toBe('HELLO');
expect(callLog).toContain('echo:alice:hello');
});
test('typed add', async () => {
const result = await rig.fs.custom('test.add', { a: 3, b: 4 });
expect(result.sum).toBe(7);
});
test('invalid args (Zod-rejected payload) → InvalidArgsError', async () => {
await expect(
// message: '' violates min(1) — TypeScript still allows it since string
rig.fs.custom('test.echo', { message: '' }),
).rejects.toThrow();
});
test('unknown custom op name → NotImplementedError', async () => {
await expect(
rig.fs.custom('test.unknown' as never, {} as never),
).rejects.toBeInstanceOf(NotImplementedError);
});
test('handler returns wrong shape → CustomOpRejectedError', async () => {
await expect(
rig.fs.custom('test.bad-response' as never, {} as never),
).rejects.toBeInstanceOf(CustomOpRejectedError);
});
});

210
packages/shade-files/tests/integration/download-directory.test.ts Normal file
View File

@@ -0,0 +1,210 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { sha256 } from '@noble/hashes/sha2.js';
import {
NotFoundError,
downloadDirectory,
createMemoryDirectory,
walk,
type DirectoryHandleLike,
type FileEntry,
} from '../../src/index.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
interface StoredFile {
bytes: Uint8Array;
contentType?: string;
sha256: string;
}
function bytesToHex(b: Uint8Array): string {
return Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
}
async function streamToBytes(s: ReadableStream<Uint8Array>): Promise<Uint8Array> {
const reader = s.getReader();
const parts: Uint8Array[] = [];
let total = 0;
while (true) {
const { value, done } = await reader.read();
if (done) break;
if (value === undefined) continue;
parts.push(value);
total += value.byteLength;
}
reader.releaseLock();
const out = new Uint8Array(total);
let offset = 0;
for (const p of parts) {
out.set(p, offset);
offset += p.byteLength;
}
return out;
}
function streamFromBytes(bytes: Uint8Array): ReadableStream<Uint8Array> {
return new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(bytes);
controller.close();
},
});
}
describe('downloadDirectory — bulk download from remote', () => {
let rig: FileTestRig;
const blobs = new Map<string, StoredFile>();
const dirs = new Set<string>();
beforeAll(async () => {
blobs.clear();
dirs.clear();
// Build remote tree:
// /src/
// ├── small.txt ('hello world\n', 12 bytes)
// ├── img.bin (50 KiB random)
// └── nested/
// ├── big.bin (400 KiB random)
// └── tiny.bin (3 bytes)
dirs.add('/');
dirs.add('/src');
dirs.add('/src/nested');
const small = new TextEncoder().encode('hello world\n');
const mid = new Uint8Array(50 * 1024); crypto.getRandomValues(mid);
const big = new Uint8Array(400 * 1024); crypto.getRandomValues(big);
const tiny = new Uint8Array([1, 2, 3]);
blobs.set('/src/small.txt', { bytes: small, sha256: bytesToHex(sha256(small)), contentType: 'text/plain' });
blobs.set('/src/img.bin', { bytes: mid, sha256: bytesToHex(sha256(mid)) });
blobs.set('/src/nested/big.bin', { bytes: big, sha256: bytesToHex(sha256(big)) });
blobs.set('/src/nested/tiny.bin', { bytes: tiny, sha256: bytesToHex(sha256(tiny)) });
rig = await setupFileRig({
list: async (ctx) => {
if (!dirs.has(ctx.path)) throw new NotFoundError(ctx.path);
const entries: FileEntry[] = [];
const dirPrefix = ctx.path === '/' ? '/' : ctx.path + '/';
// Subdirs
for (const d of dirs) {
if (d === ctx.path) continue;
if (!d.startsWith(dirPrefix)) continue;
const rest = d.slice(dirPrefix.length);
if (rest.includes('/')) continue;
entries.push({ name: rest, kind: 'dir', size: 0, mtime: 0, metadata: {} });
}
// Files
for (const [path, blob] of blobs) {
if (!path.startsWith(dirPrefix)) continue;
const rest = path.slice(dirPrefix.length);
if (rest.includes('/')) continue;
entries.push({
name: rest,
kind: 'file',
size: blob.bytes.byteLength,
mtime: 0,
...(blob.contentType !== undefined ? { contentType: blob.contentType } : {}),
metadata: {},
});
}
return { entries, hasMore: false };
},
read: async (ctx) => {
const blob = blobs.get(ctx.path);
if (blob === undefined) throw new NotFoundError(ctx.path);
if (blob.bytes.byteLength > 256 * 1024) {
return blob.contentType !== undefined
? {
kind: 'streams',
stream: streamFromBytes(blob.bytes),
size: blob.bytes.byteLength,
sha256: blob.sha256,
contentType: blob.contentType,
}
: {
kind: 'streams',
stream: streamFromBytes(blob.bytes),
size: blob.bytes.byteLength,
sha256: blob.sha256,
};
}
return blob.contentType !== undefined
? { kind: 'inline', bytes: blob.bytes, sha256: blob.sha256, contentType: blob.contentType }
: { kind: 'inline', bytes: blob.bytes, sha256: blob.sha256 };
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('downloads entire tree, sha256 matches per file', async () => {
const local = createMemoryDirectory('local');
const handle = downloadDirectory(rig.fs, '/src', local);
const result = await handle.done();
expect(result.filesDone).toBe(4);
expect(result.bytesDone).toBe(12 + 50 * 1024 + 400 * 1024 + 3);
// Verify local tree contents
const downloadedFiles = new Map<string, Uint8Array>();
async function dump(dir: DirectoryHandleLike, prefix: string): Promise<void> {
for await (const [name, child] of dir.entries()) {
const path = prefix === '' ? name : `${prefix}/${name}`;
if (child.kind === 'directory') {
await dump(child as DirectoryHandleLike, path);
} else {
const file = await (child as { getFile: () => Promise<{ arrayBuffer: () => Promise<ArrayBuffer> }> }).getFile();
downloadedFiles.set(path, new Uint8Array(await file.arrayBuffer()));
}
}
}
await dump(local, '');
expect(downloadedFiles.size).toBe(4);
expect(downloadedFiles.has('small.txt')).toBe(true);
expect(downloadedFiles.has('img.bin')).toBe(true);
expect(downloadedFiles.has('nested/big.bin')).toBe(true);
expect(downloadedFiles.has('nested/tiny.bin')).toBe(true);
expect(bytesToHex(sha256(downloadedFiles.get('nested/big.bin')!))).toBe(
blobs.get('/src/nested/big.bin')!.sha256,
);
expect(bytesToHex(sha256(downloadedFiles.get('img.bin')!))).toBe(
blobs.get('/src/img.bin')!.sha256,
);
});
test('aggregated progress events fire monotonically', async () => {
const local = createMemoryDirectory('local');
const handle = downloadDirectory(rig.fs, '/src', local);
const progresses: { filesDone: number; bytesDone: number }[] = [];
(async () => {
for await (const ev of handle.events) {
if (ev.type === 'progress') {
progresses.push({ filesDone: ev.filesDone, bytesDone: ev.bytesDone });
}
}
})().catch(() => undefined);
await handle.done();
await new Promise((r) => setTimeout(r, 30));
for (let i = 1; i < progresses.length; i++) {
expect(progresses[i]!.filesDone).toBeGreaterThanOrEqual(progresses[i - 1]!.filesDone);
expect(progresses[i]!.bytesDone).toBeGreaterThanOrEqual(progresses[i - 1]!.bytesDone);
}
});
test('aborts via handle.abort()', async () => {
const local = createMemoryDirectory('local');
const handle = downloadDirectory(rig.fs, '/src', local);
setTimeout(() => void handle.abort('test-cancel'), 5);
await expect(handle.done()).rejects.toThrow();
});
test('walk + downloadDirectory are consistent', async () => {
const local = createMemoryDirectory('local');
const remoteFiles: string[] = [];
for await (const item of walk(rig.fs, '/src')) {
if (item.entry.kind === 'file') remoteFiles.push(item.relativePath);
}
const handle = downloadDirectory(rig.fs, '/src', local);
const result = await handle.done();
expect(result.filesDone).toBe(remoteFiles.length);
});
});

142
packages/shade-files/tests/integration/helpers/rig.ts Normal file
View File

@@ -0,0 +1,142 @@
import { createShade, type Shade } from '@shade/sdk';
import {
createPrekeyServer,
MemoryPrekeyStore,
PrekeyServerEvents,
} from '@shade/server';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import {
PendingRpcRegistry,
ShadeFileRpcChannel,
attachClientRouting,
attachFileHandler,
createClientStreamsBridge,
createFileClient,
createFileHandler,
createServerStreamsBridge,
type ClientStreamsBridge,
type FileClient,
type FileHandler,
type FileHandlerConfig,
type ServerStreamsBridge,
} from '../../../src/index.js';
const crypto = new SubtleCryptoProvider();
export interface FileTestRig {
alice: Shade;
bob: Shade;
fs: FileClient;
bobHandler: FileHandler;
/** Server-side streams bridge (Bob). */
bobStreamsBridge: ServerStreamsBridge;
/** Client-side streams bridge (Alice). */
aliceStreamsBridge: ClientStreamsBridge;
/** Tear everything down (kills servers, shuts down shades). */
teardown(): Promise<void>;
}
/**
* Setup options.
*
* Defaults to wiring streams-bridges on both sides so content I/O tests
* (`read-write-streams.test.ts`) work transparently. Pass `withStreams: false`
* to skip — useful for the legacy `std-ops` tests that don't need them.
*/
export interface SetupRigOptions {
withStreams?: boolean;
}
export async function setupFileRig(
bobConfig: Omit<FileHandlerConfig, 'streamsBridge'>,
options: SetupRigOptions = {},
): Promise<FileTestRig> {
const withStreams = options.withStreams ?? true;
// 1. Prekey server
const prekeyEvents = new PrekeyServerEvents();
const prekey = createPrekeyServer({
crypto,
store: new MemoryPrekeyStore(),
disableRateLimit: true,
events: prekeyEvents,
});
const prekeyServer = Bun.serve({ port: 0, fetch: prekey.fetch });
const prekeyUrl = `http://localhost:${prekeyServer.port}`;
// 2. Two Shades
const alice = await createShade({ prekeyServer: prekeyUrl, address: 'alice' });
const bob = await createShade({ prekeyServer: prekeyUrl, address: 'bob' });
// 3. BOTH sides need a transferRoute mounted because file-RPC is
// request/response — Bob's reply must reach Alice's HTTP endpoint
// just as Alice's request reached Bob's.
let bobBaseUrl = '';
let aliceBaseUrl = '';
alice.configureTransfers({
resolveBaseUrl: async (peer) => {
if (peer === 'bob') return bobBaseUrl;
throw new Error(`alice: unknown peer ${peer}`);
},
});
bob.configureTransfers({
resolveBaseUrl: async (peer) => {
if (peer === 'alice') return aliceBaseUrl;
throw new Error(`bob: unknown peer ${peer}`);
},
});
const bobApp = await bob.transferRoute();
const aliceApp = await alice.transferRoute();
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
const aliceServer = Bun.serve({ port: 0, fetch: aliceApp.fetch });
bobBaseUrl = `http://localhost:${bobServer.port}`;
aliceBaseUrl = `http://localhost:${aliceServer.port}`;
// 4. Streams bridges (both sides) — required for content I/O > 256 KiB.
let bobStreamsBridge: ServerStreamsBridge | undefined;
let aliceStreamsBridge: ClientStreamsBridge | undefined;
if (withStreams) {
bobStreamsBridge = await createServerStreamsBridge(bob);
aliceStreamsBridge = await createClientStreamsBridge(alice);
}
// 5. Bob: file handler + channel
const bobChannel = new ShadeFileRpcChannel(bob);
const fullBobConfig: FileHandlerConfig = {
...bobConfig,
...(bobStreamsBridge !== undefined ? { streamsBridge: bobStreamsBridge } : {}),
};
const bobHandler = createFileHandler(bob, fullBobConfig);
attachFileHandler(bobChannel, bobHandler);
// 6. Alice: client + channel + pending registry
const aliceChannel = new ShadeFileRpcChannel(alice);
const alicePending = new PendingRpcRegistry();
attachClientRouting(aliceChannel, alicePending);
const fs = createFileClient(alice, aliceChannel, alicePending, 'bob', {
defaultTimeoutMs: 5000,
...(aliceStreamsBridge !== undefined ? { streamsBridge: aliceStreamsBridge } : {}),
});
return {
alice,
bob,
fs,
bobHandler,
bobStreamsBridge: bobStreamsBridge as ServerStreamsBridge,
aliceStreamsBridge: aliceStreamsBridge as ClientStreamsBridge,
async teardown() {
bobChannel.destroy();
aliceChannel.destroy();
bobHandler.destroy();
if (bobStreamsBridge !== undefined) await bobStreamsBridge.destroy();
if (aliceStreamsBridge !== undefined) await aliceStreamsBridge.destroy();
await alice.shutdown();
await bob.shutdown();
bobServer.stop();
aliceServer.stop();
prekeyServer.stop();
},
};
}

72
packages/shade-files/tests/integration/metrics.test.ts Normal file
View File

@@ -0,0 +1,72 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
import {
METRIC_OP_DURATION_MS,
METRIC_OP_TOTAL,
METRIC_RATE_LIMIT_REJECT_TOTAL,
type MetricSink,
type MetricTags,
type FileEntry,
} from '../../src/index.js';
interface MetricEvent {
name: string;
value: number;
tags: MetricTags;
}
describe('Metrics — onMetric on success', () => {
let rig: FileTestRig;
const events: MetricEvent[] = [];
const sink: MetricSink = (name, value, tags) => events.push({ name, value, tags });
beforeAll(async () => {
rig = await setupFileRig({
onMetric: sink,
list: async () => ({ entries: [], hasMore: false }),
});
});
afterAll(async () => { await rig.teardown(); });
test('emits op_total + op_duration_ms with result=ok on success', async () => {
events.length = 0;
await rig.fs.list('/');
const totals = events.filter((e) => e.name === METRIC_OP_TOTAL);
expect(totals.length).toBeGreaterThanOrEqual(1);
expect(totals[0]!.tags.result).toBe('ok');
expect(totals[0]!.tags.op).toBe('list');
const durations = events.filter((e) => e.name === METRIC_OP_DURATION_MS);
expect(durations.length).toBeGreaterThanOrEqual(1);
expect(durations[0]!.value).toBeGreaterThanOrEqual(0);
});
});
describe('Metrics — onMetric rate-limit reject', () => {
let rig: FileTestRig;
const events: MetricEvent[] = [];
const sink: MetricSink = (name, value, tags) => events.push({ name, value, tags });
beforeAll(async () => {
rig = await setupFileRig({
onMetric: sink,
rateLimits: { maxOpsPerMinutePerSender: 3 },
stat: async (_ctx) => {
const e: FileEntry = { name: 'x', kind: 'file', size: 1, mtime: 0, metadata: {} };
return e;
},
});
});
afterAll(async () => { await rig.teardown(); });
test('emits rate_limit_reject_total when capacity exhausted', async () => {
events.length = 0;
// Cap is 3; first 3 stats succeed, 4th rejected.
await rig.fs.stat('/x');
await rig.fs.stat('/x');
await rig.fs.stat('/x');
await expect(rig.fs.stat('/x')).rejects.toThrow();
const rejects = events.filter((e) => e.name === METRIC_RATE_LIMIT_REJECT_TOTAL);
expect(rejects.length).toBeGreaterThanOrEqual(1);
expect(rejects[0]!.tags.op).toBe('stat');
});
});

138
packages/shade-files/tests/integration/read-write-inline.test.ts Normal file
View File

@@ -0,0 +1,138 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { sha256 } from '@noble/hashes/sha2.js';
import { NotFoundError, type FileEntry } from '../../src/index.js';
import type { UserReadResult, UserWriteArgs, WriteResult } from '../../src/server/io-types.js';
import type { OpContext } from '../../src/server/handler-context.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
interface StoredBlob {
bytes: Uint8Array;
contentType?: string;
sha256: string;
mtime: number;
}
function bytesToHex(b: Uint8Array): string {
return Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
}
describe('Content I/O — inline read/write E2E', () => {
let rig: FileTestRig;
const blobs = new Map<string, StoredBlob>();
beforeAll(async () => {
blobs.clear();
rig = await setupFileRig({
read: async (ctx: OpContext<{ path: string }>): Promise<UserReadResult> => {
const blob = blobs.get(ctx.path);
if (blob === undefined) throw new NotFoundError(ctx.path);
return blob.contentType !== undefined
? { kind: 'inline', bytes: blob.bytes, sha256: blob.sha256, contentType: blob.contentType }
: { kind: 'inline', bytes: blob.bytes, sha256: blob.sha256 };
},
write: async (ctx: OpContext<UserWriteArgs>): Promise<WriteResult> => {
const args = ctx.args;
if (args.content.kind !== 'inline') {
throw new Error('expected inline content for this test');
}
if (blobs.has(args.path) && !args.overwrite) {
throw new Error('exists');
}
const stored: StoredBlob = {
bytes: args.content.bytes,
...(args.contentType !== undefined ? { contentType: args.contentType } : {}),
sha256: args.content.sha256,
mtime: Date.now(),
};
blobs.set(args.path, stored);
const entry: FileEntry = {
name: args.path.split('/').filter(Boolean).pop() ?? '',
kind: 'file',
size: args.content.size,
mtime: stored.mtime,
...(args.contentType !== undefined ? { contentType: args.contentType } : {}),
metadata: { sha256: args.content.sha256 },
};
return { entry };
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('write 1 KiB inline → read it back, sha256 matches', async () => {
const data = new Uint8Array(1024);
for (let i = 0; i < data.length; i++) data[i] = (i * 7) & 0xff;
const expectedSha = bytesToHex(sha256(data));
const writeResult = await rig.fs.write('/small.bin', data, { contentType: 'application/octet-stream' });
expect(writeResult.entry.size).toBe(1024);
expect(writeResult.entry.metadata.sha256).toBe(expectedSha);
const readResult = await rig.fs.read('/small.bin');
expect(readResult.kind).toBe('inline');
if (readResult.kind === 'inline') {
expect(readResult.bytes.byteLength).toBe(1024);
expect(readResult.sha256).toBe(expectedSha);
expect(readResult.contentType).toBe('application/octet-stream');
expect(Array.from(readResult.bytes)).toEqual(Array.from(data));
}
});
test('write 100 KiB inline → read it back, sha256 matches', async () => {
const data = new Uint8Array(100 * 1024);
crypto.getRandomValues(data);
const expectedSha = bytesToHex(sha256(data));
await rig.fs.write('/big.bin', data);
const readResult = await rig.fs.read('/big.bin');
expect(readResult.kind).toBe('inline');
if (readResult.kind === 'inline') {
expect(readResult.bytes.byteLength).toBe(100 * 1024);
expect(readResult.sha256).toBe(expectedSha);
expect(Array.from(readResult.bytes)).toEqual(Array.from(data));
}
});
test('overwrite without flag → server-defined error; with flag → succeeds', async () => {
const a = new Uint8Array([1, 2, 3]);
const b = new Uint8Array([4, 5, 6]);
await rig.fs.write('/dup.bin', a);
await expect(rig.fs.write('/dup.bin', b)).rejects.toThrow();
await rig.fs.write('/dup.bin', b, { overwrite: true });
const out = await rig.fs.read('/dup.bin');
expect(out.kind).toBe('inline');
if (out.kind === 'inline') {
expect(Array.from(out.bytes)).toEqual([4, 5, 6]);
}
});
test('write Blob input → inferred contentType, round-trips', async () => {
const blob = new Blob([new Uint8Array([0xde, 0xad, 0xbe, 0xef])], { type: 'image/png' });
await rig.fs.write('/blobby.png', blob);
const out = await rig.fs.read('/blobby.png');
expect(out.kind).toBe('inline');
if (out.kind === 'inline') {
expect(out.contentType).toBe('image/png');
expect(Array.from(out.bytes)).toEqual([0xde, 0xad, 0xbe, 0xef]);
}
});
test('read non-existent → NotFoundError', async () => {
await expect(rig.fs.read('/missing.bin')).rejects.toThrow();
});
test('inline path also handles 256 KiB exactly (boundary)', async () => {
const data = new Uint8Array(256 * 1024);
crypto.getRandomValues(data);
const expectedSha = bytesToHex(sha256(data));
await rig.fs.write('/boundary.bin', data);
const out = await rig.fs.read('/boundary.bin');
expect(out.kind).toBe('inline');
if (out.kind === 'inline') {
expect(out.sha256).toBe(expectedSha);
}
});
});

175
packages/shade-files/tests/integration/read-write-streams.test.ts Normal file
View File

@@ -0,0 +1,175 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { sha256 } from '@noble/hashes/sha2.js';
import { NotFoundError, type FileEntry } from '../../src/index.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
interface StoredBlob {
bytes: Uint8Array;
contentType?: string;
sha256: string;
}
function bytesToHex(b: Uint8Array): string {
return Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
}
async function streamToBytes(s: ReadableStream<Uint8Array>): Promise<Uint8Array> {
const reader = s.getReader();
const parts: Uint8Array[] = [];
let total = 0;
while (true) {
const { value, done } = await reader.read();
if (done) break;
if (value === undefined) continue;
parts.push(value);
total += value.byteLength;
}
reader.releaseLock();
const out = new Uint8Array(total);
let offset = 0;
for (const p of parts) {
out.set(p, offset);
offset += p.byteLength;
}
return out;
}
function streamFromBytes(bytes: Uint8Array): ReadableStream<Uint8Array> {
return new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(bytes);
controller.close();
},
});
}
describe('Content I/O — streamed read/write E2E (>256 KiB)', () => {
let rig: FileTestRig;
const blobs = new Map<string, StoredBlob>();
beforeAll(async () => {
blobs.clear();
rig = await setupFileRig({
read: async (ctx) => {
const blob = blobs.get(ctx.path);
if (blob === undefined) throw new NotFoundError(ctx.path);
// Return as streams when blob ≥ 256 KiB.
if (blob.bytes.byteLength > 256 * 1024) {
return blob.contentType !== undefined
? {
kind: 'streams',
stream: streamFromBytes(blob.bytes),
size: blob.bytes.byteLength,
sha256: blob.sha256,
contentType: blob.contentType,
}
: {
kind: 'streams',
stream: streamFromBytes(blob.bytes),
size: blob.bytes.byteLength,
sha256: blob.sha256,
};
}
return blob.contentType !== undefined
? { kind: 'inline', bytes: blob.bytes, sha256: blob.sha256, contentType: blob.contentType }
: { kind: 'inline', bytes: blob.bytes, sha256: blob.sha256 };
},
write: async (ctx) => {
const args = ctx.args;
let bytes: Uint8Array;
let resolvedSha: string;
if (args.content.kind === 'inline') {
bytes = args.content.bytes;
resolvedSha = args.content.sha256;
} else {
bytes = await streamToBytes(args.content.stream);
resolvedSha = await args.content.sha256;
if (bytes.byteLength !== args.content.size) {
throw new Error(`stream produced ${bytes.byteLength} bytes; declared ${args.content.size}`);
}
}
const stored: StoredBlob = {
bytes,
...(args.contentType !== undefined ? { contentType: args.contentType } : {}),
sha256: resolvedSha,
};
blobs.set(args.path, stored);
const entry: FileEntry = {
name: args.path.split('/').filter(Boolean).pop() ?? '',
kind: 'file',
size: bytes.byteLength,
mtime: Date.now(),
...(args.contentType !== undefined ? { contentType: args.contentType } : {}),
metadata: { sha256: resolvedSha },
};
return { entry };
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('write 1 MiB streamed → server sees streams content + sha256 matches', async () => {
const data = new Uint8Array(1024 * 1024);
crypto.getRandomValues(data);
const expectedSha = bytesToHex(sha256(data));
const writeResult = await rig.fs.write('/big1mb.bin', data);
expect(writeResult.entry.size).toBe(data.byteLength);
expect(writeResult.entry.metadata.sha256).toBe(expectedSha);
});
test('read 1 MiB streamed → client gets streams output, drains correctly', async () => {
const out = await rig.fs.read('/big1mb.bin');
expect(out.kind).toBe('streams');
if (out.kind === 'streams') {
expect(out.size).toBe(1024 * 1024);
const drained = await streamToBytes(out.stream);
expect(drained.byteLength).toBe(1024 * 1024);
expect(bytesToHex(sha256(drained))).toBe(out.sha256);
await out.done();
}
});
test('boundary 256 KiB + 1 → streams write + read round-trip', async () => {
const data = new Uint8Array(256 * 1024 + 1);
for (let i = 0; i < data.length; i++) data[i] = (i * 31) & 0xff;
const expectedSha = bytesToHex(sha256(data));
await rig.fs.write('/boundary-plus-one.bin', data);
const out = await rig.fs.read('/boundary-plus-one.bin');
expect(out.kind).toBe('streams');
if (out.kind === 'streams') {
expect(out.sha256).toBe(expectedSha);
const drained = await streamToBytes(out.stream);
expect(drained.byteLength).toBe(256 * 1024 + 1);
expect(Array.from(drained.slice(0, 4))).toEqual(Array.from(data.slice(0, 4)));
expect(Array.from(drained.slice(-4))).toEqual(Array.from(data.slice(-4)));
await out.done();
}
});
test('write streams via { stream, size } wrapper → ok', async () => {
const data = new Uint8Array(500 * 1024);
crypto.getRandomValues(data);
const expectedSha = bytesToHex(sha256(data));
const wrapped = { stream: streamFromBytes(data), size: data.byteLength, contentType: 'application/octet-stream' };
const result = await rig.fs.write('/wrapped.bin', wrapped);
expect(result.entry.size).toBe(data.byteLength);
expect(result.entry.metadata.sha256).toBe(expectedSha);
expect(result.entry.contentType).toBe('application/octet-stream');
});
test('write streams from Blob > 256 KiB → ok', async () => {
const data = new Uint8Array(400 * 1024);
crypto.getRandomValues(data);
const expectedSha = bytesToHex(sha256(data));
const blob = new Blob([data], { type: 'image/png' });
const result = await rig.fs.write('/blobby-big.png', blob);
expect(result.entry.size).toBe(data.byteLength);
expect(result.entry.metadata.sha256).toBe(expectedSha);
expect(result.entry.contentType).toBe('image/png');
});
});

100
packages/shade-files/tests/integration/sdk-namespace.test.ts Normal file
View File

@@ -0,0 +1,100 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { createShade, type Shade } from '@shade/sdk';
import { createPrekeyServer, MemoryPrekeyStore, PrekeyServerEvents } from '@shade/server';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import type { FileEntry } from '../../src/index.js';
const crypto = new SubtleCryptoProvider();
/**
* End-to-end test of `Shade.files` — the high-level SDK entrypoint.
* Verifies that `shade.files.serve(...)` and `shade.files.client(peer)`
* compose correctly and share a single channel + bridges per Shade.
*/
describe('Shade.files namespace — end-to-end via SDK getter', () => {
let alice: Shade;
let bob: Shade;
let prekeyServer: { stop(): void };
let aliceServer: { stop(): void };
let bobServer: { stop(): void };
let stopBobFiles: (() => Promise<void>) | null = null;
beforeAll(async () => {
const prekey = createPrekeyServer({
crypto,
store: new MemoryPrekeyStore(),
disableRateLimit: true,
events: new PrekeyServerEvents(),
});
prekeyServer = Bun.serve({ port: 0, fetch: prekey.fetch });
const prekeyUrl = `http://localhost:${(prekeyServer as unknown as { port: number }).port}`;
alice = await createShade({ prekeyServer: prekeyUrl, address: 'alice' });
bob = await createShade({ prekeyServer: prekeyUrl, address: 'bob' });
let aliceUrl = '';
let bobUrl = '';
alice.configureTransfers({
resolveBaseUrl: async (peer) => {
if (peer === 'bob') return bobUrl;
throw new Error(`unknown peer: ${peer}`);
},
});
bob.configureTransfers({
resolveBaseUrl: async (peer) => {
if (peer === 'alice') return aliceUrl;
throw new Error(`unknown peer: ${peer}`);
},
});
const aliceApp = await alice.transferRoute();
const bobApp = await bob.transferRoute();
aliceServer = Bun.serve({ port: 0, fetch: aliceApp.fetch });
bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
aliceUrl = `http://localhost:${(aliceServer as unknown as { port: number }).port}`;
bobUrl = `http://localhost:${(bobServer as unknown as { port: number }).port}`;
});
afterAll(async () => {
if (stopBobFiles !== null) await stopBobFiles();
await alice.files.destroy();
await bob.files.destroy();
await alice.shutdown();
await bob.shutdown();
aliceServer.stop();
bobServer.stop();
prekeyServer.stop();
});
test('shade.files getter is memoized', () => {
const a = bob.files;
const b = bob.files;
expect(a).toBe(b);
});
test('serve() + client() round-trip stat through the SDK', async () => {
stopBobFiles = await bob.files.serve({
stat: async (ctx) => {
const e: FileEntry = {
name: ctx.path.split('/').filter(Boolean).pop() ?? '',
kind: 'file',
size: 42,
mtime: 1234,
metadata: {},
};
return e;
},
});
const fs = await alice.files.client('bob');
const result = await fs.stat('/answer.txt');
expect(result.name).toBe('answer.txt');
expect(result.size).toBe(42);
expect(result.mtime).toBe(1234);
});
test('second serve() throws (one handler per Shade)', async () => {
await expect(
bob.files.serve({ stat: async () => ({ name: 'x', kind: 'file', size: 0, mtime: 0, metadata: {} }) }),
).rejects.toThrow(/handler is already registered/);
});
});

199
packages/shade-files/tests/integration/std-ops.test.ts Normal file
View File

@@ -0,0 +1,199 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import {
ConflictError,
IdempotencyConflictError,
NotFoundError,
PermissionDeniedError,
type FileEntry,
} from '../../src/index.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
describe('Standard ops — list/stat/mkdir/delete/move E2E', () => {
let rig: FileTestRig;
// Simple in-memory backing store on Bob.
const tree = new Map<string, FileEntry>();
beforeAll(async () => {
tree.clear();
tree.set('/', { name: '', kind: 'dir', size: 0, mtime: 0, metadata: {} });
tree.set('/foo', { name: 'foo', kind: 'dir', size: 0, mtime: 100, metadata: {} });
tree.set('/foo/bar.txt', {
name: 'bar.txt',
kind: 'file',
size: 12,
mtime: 200,
contentType: 'text/plain',
metadata: {},
});
tree.set('/foo/baz.txt', {
name: 'baz.txt',
kind: 'file',
size: 5,
mtime: 300,
metadata: {},
});
rig = await setupFileRig({
list: async (ctx) => {
const dir = ctx.path;
const entries: FileEntry[] = [];
for (const [path, entry] of tree) {
if (path === dir) continue;
if (!path.startsWith(dir === '/' ? '/' : dir + '/')) continue;
const rest = path.slice(dir === '/' ? 1 : dir.length + 1);
if (rest.includes('/')) continue;
entries.push(entry);
}
return { entries, hasMore: false };
},
stat: async (ctx) => {
const e = tree.get(ctx.path);
if (e === undefined) throw new NotFoundError(`${ctx.path} not found`);
return e;
},
mkdir: async (ctx) => {
if (tree.has(ctx.path)) throw new ConflictError(`${ctx.path} exists`);
const name = ctx.path.split('/').filter(Boolean).pop() ?? '';
const entry: FileEntry = {
name,
kind: 'dir',
size: 0,
mtime: Date.now(),
metadata: {},
};
tree.set(ctx.path, entry);
return { entry };
},
delete: async (ctx) => {
if (!tree.has(ctx.path)) throw new NotFoundError(ctx.path);
let count = 0;
if (ctx.args.recursive) {
for (const path of [...tree.keys()]) {
if (path === ctx.path || path.startsWith(ctx.path + '/')) {
tree.delete(path);
count++;
}
}
} else {
tree.delete(ctx.path);
count = 1;
}
return { deletedCount: count };
},
move: async (ctx) => {
const src = ctx.args.src;
const dst = ctx.args.dst;
const e = tree.get(src);
if (e === undefined) throw new NotFoundError(src);
if (tree.has(dst) && !ctx.args.overwrite) {
throw new ConflictError(`${dst} exists`);
}
const newName = dst.split('/').filter(Boolean).pop() ?? e.name;
tree.delete(src);
tree.set(dst, { ...e, name: newName });
return { entry: tree.get(dst)! };
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('list /', async () => {
const page = await rig.fs.list('/');
expect(page.hasMore).toBe(false);
const names = page.entries.map((e) => e.name).sort();
expect(names).toEqual(['foo']);
});
test('list /foo', async () => {
const page = await rig.fs.list('/foo');
const names = page.entries.map((e) => e.name).sort();
expect(names).toEqual(['bar.txt', 'baz.txt']);
});
test('stat existing file', async () => {
const e = await rig.fs.stat('/foo/bar.txt');
expect(e.size).toBe(12);
expect(e.contentType).toBe('text/plain');
});
test('stat missing → NotFoundError', async () => {
let caught: unknown = null;
try {
await rig.fs.stat('/no/such/file');
} catch (err) {
caught = err;
}
expect(caught instanceof NotFoundError).toBe(true);
expect((caught as NotFoundError).payload.code).toBe('NOT_FOUND');
});
test('mkdir creates a new directory', async () => {
const result = await rig.fs.mkdir('/created');
expect(result.entry.kind).toBe('dir');
expect(tree.has('/created')).toBe(true);
});
test('mkdir on existing path → ConflictError', async () => {
await expect(rig.fs.mkdir('/foo')).rejects.toBeInstanceOf(ConflictError);
});
test('delete file', async () => {
tree.set('/temp.txt', { name: 'temp.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
const r = await rig.fs.delete('/temp.txt');
expect(r.deletedCount).toBe(1);
expect(tree.has('/temp.txt')).toBe(false);
});
test('delete missing → NotFoundError', async () => {
await expect(rig.fs.delete('/doesnt-exist')).rejects.toBeInstanceOf(NotFoundError);
});
test('move file', async () => {
tree.set('/x.txt', { name: 'x.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
const r = await rig.fs.move('/x.txt', '/y.txt');
expect(r.entry.name).toBe('y.txt');
expect(tree.has('/x.txt')).toBe(false);
expect(tree.has('/y.txt')).toBe(true);
});
test('move overwrite=false → ConflictError on collision', async () => {
tree.set('/a.txt', { name: 'a.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
tree.set('/b.txt', { name: 'b.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
await expect(rig.fs.move('/a.txt', '/b.txt')).rejects.toBeInstanceOf(ConflictError);
});
test('idempotent retry: same key + same args → cached response', async () => {
tree.set('/idem.txt', { name: 'idem.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
const key = 'IdemKey1AaBbCcDdEeFfGg'; // 22 chars
const r1 = await rig.fs.delete('/idem.txt', { idempotencyKey: key });
expect(r1.deletedCount).toBe(1);
// 2nd call with same key → same result without throwing NotFound
const r2 = await rig.fs.delete('/idem.txt', { idempotencyKey: key });
expect(r2.deletedCount).toBe(1);
});
test('idempotency conflict: same key + different args', async () => {
tree.set('/c1.txt', { name: 'c1.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
tree.set('/c2.txt', { name: 'c2.txt', kind: 'file', size: 0, mtime: 0, metadata: {} });
const key = 'ConflictKeyAaBbCcDdEeF'; // 22 chars
await rig.fs.delete('/c1.txt', { idempotencyKey: key });
await expect(
rig.fs.delete('/c2.txt', { idempotencyKey: key }),
).rejects.toBeInstanceOf(IdempotencyConflictError);
});
test('path validation: traversal rejected by server', async () => {
await expect(rig.fs.list('/foo')).resolves.toBeDefined();
// Client-side schema rejects traversal too — bypass via direct bad path:
await expect(rig.fs.stat('/foo/../etc')).rejects.toThrow();
});
// We don't test PermissionDenied here (no beforeOp gate configured),
// but the type is referenced to ensure it's exported properly.
test('PermissionDeniedError is exported', () => {
expect(PermissionDeniedError).toBeDefined();
});
});

123
packages/shade-files/tests/integration/thumbnail.test.ts Normal file
View File

@@ -0,0 +1,123 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { sha256 } from '@noble/hashes/sha2.js';
import { ConflictError, NotFoundError } from '../../src/index.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
function bytesToHex(b: Uint8Array): string {
return Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
}
// Minimal valid PNG: 8-byte signature + IHDR + zero-byte IDAT + IEND. We
// use realistic magic bytes so the format-hardening check accepts it.
function tinyPng(): Uint8Array {
const sig = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
const filler = new Uint8Array(20);
const out = new Uint8Array(sig.length + filler.length);
out.set(sig, 0);
out.set(filler, sig.length);
return out;
}
function tinyJpeg(): Uint8Array {
const head = new Uint8Array([0xff, 0xd8, 0xff, 0xe0]);
const filler = new Uint8Array(12);
const out = new Uint8Array(head.length + filler.length);
out.set(head, 0);
out.set(filler, head.length);
return out;
}
function tinyWebp(): Uint8Array {
// 'RIFF' + sizeLE + 'WEBP' + 'VP8 ' (or 'VP8L' / 'VP8X')
const out = new Uint8Array(20);
out.set([0x52, 0x49, 0x46, 0x46], 0); // 'RIFF'
out.set([0x0c, 0x00, 0x00, 0x00], 4); // size little-endian (12 bytes follow)
out.set([0x57, 0x45, 0x42, 0x50], 8); // 'WEBP'
out.set([0x56, 0x50, 0x38, 0x20], 12); // 'VP8 '
return out;
}
describe('Content I/O — getThumbnail E2E with format hardening', () => {
let rig: FileTestRig;
// Map: path → (size → { bytes, format })
const thumbnails = new Map<string, Map<number, { bytes: Uint8Array; format: 'png' | 'webp' | 'jpeg' }>>();
beforeAll(async () => {
thumbnails.clear();
thumbnails.set(
'/photo.png',
new Map([
[64, { bytes: tinyPng(), format: 'png' }],
[128, { bytes: tinyPng(), format: 'png' }],
]),
);
thumbnails.set(
'/holiday.jpg',
new Map([[256, { bytes: tinyJpeg(), format: 'jpeg' }]]),
);
thumbnails.set(
'/icon.webp',
new Map([[64, { bytes: tinyWebp(), format: 'webp' }]]),
);
// Mismatched format: server returns PNG bytes but claims JPEG.
thumbnails.set('/lying.png', new Map([[64, { bytes: tinyPng(), format: 'jpeg' }]]));
rig = await setupFileRig({
getThumbnail: async (ctx) => {
const sizes = thumbnails.get(ctx.path);
if (sizes === undefined) throw new NotFoundError(ctx.path);
const entry = sizes.get(ctx.args.size);
if (entry === undefined) throw new NotFoundError(`${ctx.path}@${ctx.args.size}`);
return {
bytes: entry.bytes,
format: entry.format,
width: ctx.args.size,
height: ctx.args.size,
};
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('PNG thumbnail round-trips with computed sha256', async () => {
const result = await rig.fs.getThumbnail('/photo.png', 64);
expect(result.format).toBe('png');
expect(result.width).toBe(64);
expect(result.height).toBe(64);
expect(result.sha256).toBe(bytesToHex(sha256(result.bytes)));
// Verify magic bytes survived
expect(Array.from(result.bytes.slice(0, 8))).toEqual([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
});
test('JPEG thumbnail round-trips', async () => {
const result = await rig.fs.getThumbnail('/holiday.jpg', 256);
expect(result.format).toBe('jpeg');
expect(result.width).toBe(256);
expect(Array.from(result.bytes.slice(0, 3))).toEqual([0xff, 0xd8, 0xff]);
});
test('WebP thumbnail round-trips', async () => {
const result = await rig.fs.getThumbnail('/icon.webp', 64);
expect(result.format).toBe('webp');
expect(Array.from(result.bytes.slice(0, 4))).toEqual([0x52, 0x49, 0x46, 0x46]);
expect(Array.from(result.bytes.slice(8, 12))).toEqual([0x57, 0x45, 0x42, 0x50]);
});
test('PNG bytes claimed as JPEG → format-hardening rejects', async () => {
await expect(rig.fs.getThumbnail('/lying.png', 64)).rejects.toBeInstanceOf(ConflictError);
});
test('non-existent path → NotFoundError', async () => {
await expect(rig.fs.getThumbnail('/missing.png', 64)).rejects.toBeInstanceOf(NotFoundError);
});
test('different sizes return different thumbnails', async () => {
const small = await rig.fs.getThumbnail('/photo.png', 64);
const big = await rig.fs.getThumbnail('/photo.png', 128);
expect(small.width).toBe(64);
expect(big.width).toBe(128);
});
});

238
packages/shade-files/tests/integration/upload-directory.test.ts Normal file
View File

@@ -0,0 +1,238 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { sha256 } from '@noble/hashes/sha2.js';
import {
ConflictError,
NotFoundError,
uploadDirectory,
createMemoryDirectory,
type FileEntry,
type BulkTransferEvent,
} from '../../src/index.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
interface StoredFile {
bytes: Uint8Array;
contentType?: string;
sha256: string;
}
function bytesToHex(b: Uint8Array): string {
return Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
}
async function streamToBytes(s: ReadableStream<Uint8Array>): Promise<Uint8Array> {
const reader = s.getReader();
const parts: Uint8Array[] = [];
let total = 0;
while (true) {
const { value, done } = await reader.read();
if (done) break;
if (value === undefined) continue;
parts.push(value);
total += value.byteLength;
}
reader.releaseLock();
const out = new Uint8Array(total);
let offset = 0;
for (const p of parts) {
out.set(p, offset);
offset += p.byteLength;
}
return out;
}
describe('uploadDirectory — bulk upload to remote', () => {
let rig: FileTestRig;
const blobs = new Map<string, StoredFile>();
const dirs = new Set<string>(['/']);
beforeAll(async () => {
blobs.clear();
dirs.clear();
dirs.add('/');
rig = await setupFileRig({
mkdir: async (ctx) => {
const path = ctx.path;
if (dirs.has(path)) {
if (!ctx.args.recursive) throw new ConflictError('exists');
// Idempotent for recursive
}
// Recursive: add ancestors
if (ctx.args.recursive) {
const segments = path.split('/').filter(Boolean);
let acc = '';
for (const seg of segments) {
acc += '/' + seg;
dirs.add(acc);
}
} else {
dirs.add(path);
}
return {
entry: {
name: path.split('/').filter(Boolean).pop() ?? '',
kind: 'dir',
size: 0,
mtime: Date.now(),
metadata: {},
},
};
},
write: async (ctx) => {
const args = ctx.args;
let bytes: Uint8Array;
let storedSha: string;
if (args.content.kind === 'inline') {
bytes = args.content.bytes;
storedSha = args.content.sha256;
} else {
bytes = await streamToBytes(args.content.stream);
storedSha = await args.content.sha256;
}
if (blobs.has(args.path) && !args.overwrite) {
throw new ConflictError(`${args.path} exists`);
}
blobs.set(args.path, {
bytes,
...(args.contentType !== undefined ? { contentType: args.contentType } : {}),
sha256: storedSha,
});
const entry: FileEntry = {
name: args.path.split('/').filter(Boolean).pop() ?? '',
kind: 'file',
size: bytes.byteLength,
mtime: Date.now(),
...(args.contentType !== undefined ? { contentType: args.contentType } : {}),
metadata: { sha256: storedSha },
};
return { entry };
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('uploads a small tree with mixed inline + streams', async () => {
const local = createMemoryDirectory('local');
const sub = local.addDir('sub');
const small = new Uint8Array(100); for (let i = 0; i < small.length; i++) small[i] = i & 0xff;
const big = new Uint8Array(400 * 1024); crypto.getRandomValues(big);
local.addFile('hello.txt', new TextEncoder().encode('hello world'), 'text/plain');
local.addFile('small.bin', small);
sub.addFile('big.bin', big, 'application/octet-stream');
const handle = uploadDirectory(rig.fs, local, '/upload-target');
const events: BulkTransferEvent[] = [];
(async () => {
for await (const ev of handle.events) events.push(ev);
})().catch(() => undefined);
const result = await handle.done();
expect(result.filesDone).toBe(3);
expect(result.bytesDone).toBe(11 + 100 + 400 * 1024);
// Verify remote tree
expect(blobs.has('/upload-target/hello.txt')).toBe(true);
expect(blobs.has('/upload-target/small.bin')).toBe(true);
expect(blobs.has('/upload-target/sub/big.bin')).toBe(true);
expect(dirs.has('/upload-target/sub')).toBe(true);
// Sha256 paritet for streamed file
expect(blobs.get('/upload-target/sub/big.bin')!.sha256).toBe(bytesToHex(sha256(big)));
expect(blobs.get('/upload-target/hello.txt')!.contentType).toBe('text/plain');
// Wait a tick for events to flush
await new Promise((r) => setTimeout(r, 30));
const planEvent = events.find((e) => e.type === 'plan');
expect(planEvent).toBeDefined();
if (planEvent && planEvent.type === 'plan') {
expect(planEvent.totalFiles).toBe(3);
expect(planEvent.totalBytes).toBe(11 + 100 + 400 * 1024);
}
const completes = events.filter((e) => e.type === 'complete');
expect(completes.length).toBe(1);
});
test('aggregated progress is monotonically non-decreasing', async () => {
const local = createMemoryDirectory('local');
for (let i = 0; i < 10; i++) {
local.addFile(`f${i}.bin`, new Uint8Array(50));
}
const handle = uploadDirectory(rig.fs, local, '/progress');
const progresses: { filesDone: number; bytesDone: number }[] = [];
(async () => {
for await (const ev of handle.events) {
if (ev.type === 'progress') {
progresses.push({ filesDone: ev.filesDone, bytesDone: ev.bytesDone });
}
}
})().catch(() => undefined);
await handle.done();
await new Promise((r) => setTimeout(r, 30));
for (let i = 1; i < progresses.length; i++) {
expect(progresses[i]!.filesDone).toBeGreaterThanOrEqual(progresses[i - 1]!.filesDone);
expect(progresses[i]!.bytesDone).toBeGreaterThanOrEqual(progresses[i - 1]!.bytesDone);
}
expect(progresses[progresses.length - 1]!.filesDone).toBe(10);
});
test('fail-fast: first error aborts the bulk', async () => {
const local = createMemoryDirectory('local');
for (let i = 0; i < 5; i++) {
local.addFile(`x${i}.bin`, new Uint8Array(10));
}
// Pre-create a conflicting file at /conflict/x0.bin so the first write fails.
blobs.set('/conflict/x0.bin', { bytes: new Uint8Array(0), sha256: 'x' });
const handle = uploadDirectory(rig.fs, local, '/conflict', { concurrency: 1 });
await expect(handle.done()).rejects.toThrow();
});
test('continueOnError: completes despite per-file errors', async () => {
const local = createMemoryDirectory('local');
for (let i = 0; i < 5; i++) {
local.addFile(`y${i}.bin`, new Uint8Array(10));
}
blobs.set('/cont/y2.bin', { bytes: new Uint8Array(0), sha256: 'x' });
const handle = uploadDirectory(rig.fs, local, '/cont', {
concurrency: 1,
continueOnError: true,
});
const errors: string[] = [];
(async () => {
for await (const ev of handle.events) {
if (ev.type === 'file-error') errors.push(ev.path);
}
})().catch(() => undefined);
const result = await handle.done();
await new Promise((r) => setTimeout(r, 30));
expect(errors).toEqual(['y2.bin']);
expect(result.filesDone).toBe(4);
});
test('concurrency cap respected', async () => {
const local = createMemoryDirectory('local');
for (let i = 0; i < 30; i++) {
local.addFile(`z${i}.bin`, new Uint8Array(10));
}
// Concurrency above MAX (16) should be clamped.
const handle = uploadDirectory(rig.fs, local, '/cap', { concurrency: 100 });
const result = await handle.done();
expect(result.filesDone).toBe(30);
});
test('aborts mid-flight via handle.abort()', async () => {
const local = createMemoryDirectory('local');
for (let i = 0; i < 50; i++) {
local.addFile(`q${i}.bin`, new Uint8Array(50 * 1024)); // 50 KiB each
}
const handle = uploadDirectory(rig.fs, local, '/abort');
setTimeout(() => void handle.abort('test-cancel'), 20);
await expect(handle.done()).rejects.toThrow();
});
});

126
packages/shade-files/tests/integration/walk.test.ts Normal file
View File

@@ -0,0 +1,126 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { walk, type FileEntry, NotFoundError } from '../../src/index.js';
import { setupFileRig, type FileTestRig } from './helpers/rig.js';
describe('walk — async-iterable depth-first directory traversal', () => {
let rig: FileTestRig;
// In-memory tree:
// /
// ├── a/
// │ ├── 1.txt
// │ └── b/
// │ └── 2.txt
// ├── c/
// │ └── 3.txt
// └── 4.txt
const tree = new Map<string, FileEntry>();
function setEntry(path: string, kind: 'file' | 'dir'): void {
const name = path === '/' ? '' : path.split('/').filter(Boolean).pop() ?? '';
tree.set(path, { name, kind, size: kind === 'file' ? 10 : 0, mtime: 0, metadata: {} });
}
beforeAll(async () => {
tree.clear();
setEntry('/', 'dir');
setEntry('/a', 'dir');
setEntry('/a/1.txt', 'file');
setEntry('/a/b', 'dir');
setEntry('/a/b/2.txt', 'file');
setEntry('/c', 'dir');
setEntry('/c/3.txt', 'file');
setEntry('/4.txt', 'file');
rig = await setupFileRig({
list: async (ctx) => {
if (!tree.has(ctx.path)) throw new NotFoundError(ctx.path);
const entries: FileEntry[] = [];
for (const [path, entry] of tree) {
if (path === ctx.path) continue;
if (!path.startsWith(ctx.path === '/' ? '/' : ctx.path + '/')) continue;
const rest = path.slice(ctx.path === '/' ? 1 : ctx.path.length + 1);
if (rest.includes('/')) continue;
entries.push(entry);
}
return { entries, hasMore: false };
},
});
});
afterAll(async () => {
await rig.teardown();
});
test('walks the entire tree depth-first', async () => {
const items: { path: string; depth: number }[] = [];
for await (const item of walk(rig.fs, '/')) {
items.push({ path: item.relativePath, depth: item.depth });
}
// Depth-first order: visit a, then descend into a/* (1.txt + a/b → a/b/2.txt), then c, c/3.txt, then 4.txt
expect(items.map((i) => i.path)).toEqual([
'a',
'a/1.txt',
'a/b',
'a/b/2.txt',
'c',
'c/3.txt',
'4.txt',
]);
// Depth values
expect(items.find((i) => i.path === 'a')?.depth).toBe(1);
expect(items.find((i) => i.path === 'a/1.txt')?.depth).toBe(2);
expect(items.find((i) => i.path === 'a/b/2.txt')?.depth).toBe(3);
});
test('respects maxDepth', async () => {
const items: string[] = [];
for await (const item of walk(rig.fs, '/', { maxDepth: 1 })) {
items.push(item.relativePath);
}
// Only direct children — no descent into /a or /c.
expect(items.sort()).toEqual(['4.txt', 'a', 'c']);
});
test('breaks cleanly when consumer stops iterating', async () => {
const items: string[] = [];
for await (const item of walk(rig.fs, '/')) {
items.push(item.relativePath);
if (items.length === 2) break;
}
expect(items).toEqual(['a', 'a/1.txt']);
});
test('filter callback skips entries (and excludes their subtree)', async () => {
const items: string[] = [];
for await (const item of walk(rig.fs, '/', {
filter: (entry, _rel) => entry.name !== 'a',
})) {
items.push(item.relativePath);
}
expect(items.sort()).toEqual(['4.txt', 'c', 'c/3.txt']);
});
test('aborts via signal mid-walk', async () => {
const ctrl = new AbortController();
const items: string[] = [];
setTimeout(() => ctrl.abort(), 5);
let threw = false;
try {
for await (const item of walk(rig.fs, '/', { signal: ctrl.signal })) {
items.push(item.relativePath);
await new Promise((r) => setTimeout(r, 10));
}
} catch {
threw = true;
}
expect(threw).toBe(true);
});
test('walking non-existent path → throws on first list', async () => {
await expect(async () => {
for await (const _item of walk(rig.fs, '/nope')) {
/* unreachable */
}
}).toThrow();
});
});