feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
Some checks failed
Test / test (push) Has been cancelled
Some checks failed
Test / test (push) Has been cancelled
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
350
packages/shade-files/tests/unit/schemas.test.ts
Normal file
350
packages/shade-files/tests/unit/schemas.test.ts
Normal file
@@ -0,0 +1,350 @@
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
PathSchema,
|
||||
RequestIdSchema,
|
||||
CursorSchema,
|
||||
Sha256HexSchema,
|
||||
FileEntrySchema,
|
||||
FileKindSchema,
|
||||
ListPageSchema,
|
||||
ListArgsSchema,
|
||||
StatArgsSchema,
|
||||
MkdirArgsSchema,
|
||||
DeleteArgsSchema,
|
||||
MoveArgsSchema,
|
||||
ReadArgsSchema,
|
||||
ReadResultSchema,
|
||||
WriteArgsSchema,
|
||||
GetThumbnailArgsSchema,
|
||||
GetThumbnailResultSchema,
|
||||
CustomArgsSchema,
|
||||
RpcRequestSchema,
|
||||
RpcResponseSchema,
|
||||
RpcErrorSchema,
|
||||
RpcCancelSchema,
|
||||
FileErrorCodeSchema,
|
||||
FileErrorPayloadSchema,
|
||||
} from '../../src/index.js';
|
||||
|
||||
describe('PathSchema', () => {
|
||||
test('accepts absolute paths', () => {
|
||||
expect(PathSchema.parse('/foo')).toBe('/foo');
|
||||
expect(PathSchema.parse('/foo/bar/baz.txt')).toBe('/foo/bar/baz.txt');
|
||||
expect(PathSchema.parse('/')).toBe('/');
|
||||
});
|
||||
test('rejects relative paths', () => {
|
||||
expect(() => PathSchema.parse('foo')).toThrow();
|
||||
expect(() => PathSchema.parse('./foo')).toThrow();
|
||||
});
|
||||
test('rejects NUL/CR/LF/DEL/backslash', () => {
|
||||
expect(() => PathSchema.parse('/foo\x00bar')).toThrow();
|
||||
expect(() => PathSchema.parse('/foo\r\n')).toThrow();
|
||||
expect(() => PathSchema.parse('/foo\x7f')).toThrow();
|
||||
expect(() => PathSchema.parse('/foo\\bar')).toThrow();
|
||||
});
|
||||
test('rejects empty + over-length', () => {
|
||||
expect(() => PathSchema.parse('')).toThrow();
|
||||
expect(() => PathSchema.parse('/' + 'a'.repeat(4096))).toThrow();
|
||||
});
|
||||
test('accepts UTF-8 in filenames', () => {
|
||||
expect(PathSchema.parse('/Документы/файл.txt')).toBe('/Документы/файл.txt');
|
||||
expect(PathSchema.parse('/πρόβλημα/αρχείο')).toBe('/πρόβλημα/αρχείο');
|
||||
});
|
||||
});
|
||||
|
||||
describe('RequestIdSchema', () => {
|
||||
test('accepts 22-char base64url', () => {
|
||||
expect(RequestIdSchema.parse('AbCdEfGhIjKlMnOpQrStUv')).toBe('AbCdEfGhIjKlMnOpQrStUv');
|
||||
});
|
||||
test('rejects wrong length', () => {
|
||||
expect(() => RequestIdSchema.parse('AbCd')).toThrow();
|
||||
expect(() => RequestIdSchema.parse('AbCdEfGhIjKlMnOpQrStUvWxYz')).toThrow();
|
||||
});
|
||||
test('rejects non-base64url chars', () => {
|
||||
expect(() => RequestIdSchema.parse('AbCd/EfGh+IjKlMnOpQrST')).toThrow();
|
||||
expect(() => RequestIdSchema.parse('AbCd=EfGhIjKlMnOpQrSTUV')).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('CursorSchema', () => {
|
||||
test('accepts up to 2048 chars', () => {
|
||||
expect(CursorSchema.parse('a')).toBe('a');
|
||||
expect(CursorSchema.parse('x'.repeat(2048)).length).toBe(2048);
|
||||
});
|
||||
test('rejects empty + over-length', () => {
|
||||
expect(() => CursorSchema.parse('')).toThrow();
|
||||
expect(() => CursorSchema.parse('x'.repeat(2049))).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Sha256HexSchema', () => {
|
||||
test('accepts 64 hex chars', () => {
|
||||
const h = '0'.repeat(64);
|
||||
expect(Sha256HexSchema.parse(h)).toBe(h);
|
||||
});
|
||||
test('rejects wrong length / non-hex', () => {
|
||||
expect(() => Sha256HexSchema.parse('0'.repeat(63))).toThrow();
|
||||
expect(() => Sha256HexSchema.parse('0'.repeat(65))).toThrow();
|
||||
expect(() => Sha256HexSchema.parse('g'.repeat(64))).toThrow();
|
||||
expect(() => Sha256HexSchema.parse('A'.repeat(64))).toThrow(); // uppercase rejected
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileKind / FileEntry', () => {
|
||||
test('FileKind accepts file/dir', () => {
|
||||
expect(FileKindSchema.parse('file')).toBe('file');
|
||||
expect(FileKindSchema.parse('dir')).toBe('dir');
|
||||
expect(() => FileKindSchema.parse('symlink')).toThrow();
|
||||
});
|
||||
test('FileEntry roundtrip', () => {
|
||||
const e = {
|
||||
name: 'foo.txt',
|
||||
kind: 'file' as const,
|
||||
size: 1024,
|
||||
mtime: 1730000000000,
|
||||
contentType: 'text/plain',
|
||||
};
|
||||
const parsed = FileEntrySchema.parse(e);
|
||||
expect(parsed.name).toBe('foo.txt');
|
||||
expect(parsed.metadata).toEqual({}); // default
|
||||
});
|
||||
test('FileEntry rejects path separators in name', () => {
|
||||
expect(() =>
|
||||
FileEntrySchema.parse({ name: 'foo/bar', kind: 'file', size: 0, mtime: 0 }),
|
||||
).toThrow();
|
||||
expect(() =>
|
||||
FileEntrySchema.parse({ name: 'foo\\bar', kind: 'file', size: 0, mtime: 0 }),
|
||||
).toThrow();
|
||||
});
|
||||
test('FileEntry rejects negative size', () => {
|
||||
expect(() =>
|
||||
FileEntrySchema.parse({ name: 'a', kind: 'file', size: -1, mtime: 0 }),
|
||||
).toThrow();
|
||||
});
|
||||
test('FileEntry passes through metadata', () => {
|
||||
const parsed = FileEntrySchema.parse({
|
||||
name: 'mod.jar',
|
||||
kind: 'file',
|
||||
size: 100,
|
||||
mtime: 0,
|
||||
metadata: { modrinthId: 'abc', version: '1.0' },
|
||||
});
|
||||
expect(parsed.metadata).toEqual({ modrinthId: 'abc', version: '1.0' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('ListPage', () => {
|
||||
test('hasMore + nextCursor when more pages', () => {
|
||||
const p = ListPageSchema.parse({
|
||||
entries: [],
|
||||
hasMore: true,
|
||||
nextCursor: 'abc',
|
||||
});
|
||||
expect(p.hasMore).toBe(true);
|
||||
expect(p.nextCursor).toBe('abc');
|
||||
});
|
||||
test('no nextCursor when hasMore false', () => {
|
||||
const p = ListPageSchema.parse({ entries: [], hasMore: false });
|
||||
expect(p.nextCursor).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('ListArgs', () => {
|
||||
test('defaults pageSize to 100', () => {
|
||||
const a = ListArgsSchema.parse({ path: '/foo' });
|
||||
expect(a.pageSize).toBe(100);
|
||||
});
|
||||
test('rejects pageSize > 1000', () => {
|
||||
expect(() => ListArgsSchema.parse({ path: '/foo', pageSize: 1001 })).toThrow();
|
||||
});
|
||||
test('rejects pageSize < 1', () => {
|
||||
expect(() => ListArgsSchema.parse({ path: '/foo', pageSize: 0 })).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('StatArgs', () => {
|
||||
test('requires path', () => {
|
||||
expect(() => StatArgsSchema.parse({})).toThrow();
|
||||
expect(StatArgsSchema.parse({ path: '/foo' }).path).toBe('/foo');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Mkdir/Delete/Move args', () => {
|
||||
test('Mkdir defaults recursive=false', () => {
|
||||
expect(MkdirArgsSchema.parse({ path: '/a' }).recursive).toBe(false);
|
||||
});
|
||||
test('Delete defaults recursive=false', () => {
|
||||
expect(DeleteArgsSchema.parse({ path: '/a' }).recursive).toBe(false);
|
||||
});
|
||||
test('Move defaults overwrite=false', () => {
|
||||
expect(
|
||||
MoveArgsSchema.parse({ src: '/a', dst: '/b' }).overwrite,
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('ReadArgs / ReadResult', () => {
|
||||
test('Read accepts optional range', () => {
|
||||
const a = ReadArgsSchema.parse({ path: '/a', range: { start: 0, end: 100 } });
|
||||
expect(a.range).toEqual({ start: 0, end: 100 });
|
||||
});
|
||||
test('ReadResult inline', () => {
|
||||
const r = ReadResultSchema.parse({
|
||||
kind: 'inline',
|
||||
bytesB64: 'YQ==',
|
||||
size: 1,
|
||||
sha256: 'a'.repeat(64),
|
||||
});
|
||||
expect(r.kind).toBe('inline');
|
||||
});
|
||||
test('ReadResult streams', () => {
|
||||
const r = ReadResultSchema.parse({
|
||||
kind: 'streams',
|
||||
streamId: 'sid-123',
|
||||
size: 1024,
|
||||
sha256: 'b'.repeat(64),
|
||||
});
|
||||
expect(r.kind).toBe('streams');
|
||||
});
|
||||
test('ReadResult rejects unknown kind', () => {
|
||||
expect(() =>
|
||||
ReadResultSchema.parse({ kind: 'magic', bytesB64: 'YQ==' }),
|
||||
).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('WriteArgs', () => {
|
||||
test('inline shape', () => {
|
||||
const w = WriteArgsSchema.parse({
|
||||
kind: 'inline',
|
||||
path: '/foo',
|
||||
bytesB64: 'YQ==',
|
||||
});
|
||||
expect(w.kind).toBe('inline');
|
||||
if (w.kind === 'inline') expect(w.overwrite).toBe(false);
|
||||
});
|
||||
test('streams shape', () => {
|
||||
const w = WriteArgsSchema.parse({
|
||||
kind: 'streams',
|
||||
path: '/foo',
|
||||
size: 1000,
|
||||
writeId: 'AbCdEfGhIjKlMnOpQrStUv',
|
||||
});
|
||||
if (w.kind === 'streams') expect(w.size).toBe(1000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GetThumbnailArgs / Result', () => {
|
||||
test('size must be enum 64/128/256/512', () => {
|
||||
expect(() => GetThumbnailArgsSchema.parse({ path: '/a', size: 100 })).toThrow();
|
||||
expect(GetThumbnailArgsSchema.parse({ path: '/a', size: 64 }).size).toBe(64);
|
||||
});
|
||||
test('format defaults to png', () => {
|
||||
const a = GetThumbnailArgsSchema.parse({ path: '/a', size: 128 });
|
||||
expect(a.format).toBe('png');
|
||||
});
|
||||
test('Result requires width/height/sha256', () => {
|
||||
const r = GetThumbnailResultSchema.parse({
|
||||
bytesB64: 'YQ==',
|
||||
format: 'png',
|
||||
width: 128,
|
||||
height: 128,
|
||||
sha256: 'c'.repeat(64),
|
||||
});
|
||||
expect(r.width).toBe(128);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CustomArgs', () => {
|
||||
test('accepts dotted op names', () => {
|
||||
expect(
|
||||
CustomArgsSchema.parse({ name: 'dispatch.deploy-mod', payload: {} }).name,
|
||||
).toBe('dispatch.deploy-mod');
|
||||
});
|
||||
test('rejects names with spaces or slashes', () => {
|
||||
expect(() => CustomArgsSchema.parse({ name: 'foo bar', payload: {} })).toThrow();
|
||||
expect(() => CustomArgsSchema.parse({ name: 'foo/bar', payload: {} })).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('RPC envelopes', () => {
|
||||
test('RpcRequest valid', () => {
|
||||
const env = RpcRequestSchema.parse({
|
||||
kind: 'shade.fs.list/v1',
|
||||
id: 'AbCdEfGhIjKlMnOpQrStUv',
|
||||
args: { path: '/' },
|
||||
sig: 'sig',
|
||||
signedAt: 1730000000000,
|
||||
});
|
||||
expect(env.kind).toBe('shade.fs.list/v1');
|
||||
});
|
||||
test('RpcRequest rejects malformed kind', () => {
|
||||
expect(() =>
|
||||
RpcRequestSchema.parse({
|
||||
kind: 'shade.fs.list',
|
||||
id: 'AbCdEfGhIjKlMnOpQrStUv',
|
||||
args: {},
|
||||
sig: 's',
|
||||
signedAt: 1,
|
||||
}),
|
||||
).toThrow();
|
||||
});
|
||||
test('RpcResponse expects .response suffix', () => {
|
||||
const env = RpcResponseSchema.parse({
|
||||
kind: 'shade.fs.list/v1.response',
|
||||
id: 'AbCdEfGhIjKlMnOpQrStUv',
|
||||
result: { entries: [], hasMore: false },
|
||||
});
|
||||
expect(env.id.length).toBe(22);
|
||||
});
|
||||
test('RpcError fixed kind', () => {
|
||||
const env = RpcErrorSchema.parse({
|
||||
kind: 'shade.fs.error/v1',
|
||||
id: 'AbCdEfGhIjKlMnOpQrStUv',
|
||||
error: { code: 'NOT_FOUND', message: 'gone' },
|
||||
});
|
||||
expect(env.error.code).toBe('NOT_FOUND');
|
||||
});
|
||||
test('RpcCancel fixed kind', () => {
|
||||
const env = RpcCancelSchema.parse({
|
||||
kind: 'shade.fs.cancel/v1',
|
||||
id: 'AbCdEfGhIjKlMnOpQrStUv',
|
||||
reason: 'user-cancel',
|
||||
});
|
||||
expect(env.reason).toBe('user-cancel');
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileError envelope', () => {
|
||||
test('all known codes parse', () => {
|
||||
const codes: ReadonlyArray<unknown> = [
|
||||
'NOT_FOUND',
|
||||
'PERMISSION_DENIED',
|
||||
'CONFLICT',
|
||||
'QUOTA_EXCEEDED',
|
||||
'RATE_LIMIT',
|
||||
'PATH_VALIDATION',
|
||||
'FINGERPRINT_REQUIRED',
|
||||
'OPERATION_TIMEOUT',
|
||||
'IDEMPOTENCY_CONFLICT',
|
||||
'CANCELLED',
|
||||
'INTERNAL',
|
||||
'NOT_IMPLEMENTED',
|
||||
'CUSTOM_OP_REJECTED',
|
||||
'INVALID_SIGNATURE',
|
||||
'INVALID_ARGS',
|
||||
];
|
||||
for (const c of codes) expect(FileErrorCodeSchema.parse(c)).toBe(c);
|
||||
});
|
||||
test('rejects unknown code', () => {
|
||||
expect(() => FileErrorCodeSchema.parse('NOT_REAL')).toThrow();
|
||||
});
|
||||
test('FileErrorPayload includes optional fields', () => {
|
||||
const p = FileErrorPayloadSchema.parse({
|
||||
code: 'RATE_LIMIT',
|
||||
message: 'too fast',
|
||||
retryAfterMs: 1000,
|
||||
});
|
||||
expect(p.retryAfterMs).toBe(1000);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user