feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
import type { StorageProvider } from '@shade/core';
|
|
|
|
|
import { ConfigurationError } from '@shade/core';
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
import type { ObservabilityHook } from '@shade/observability';
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Shade SDK configuration.
|
|
|
|
|
*
|
|
|
|
|
* All fields have sane defaults except `prekeyServer` which is required
|
|
|
|
|
* for any networked usage.
|
|
|
|
|
*/
|
|
|
|
|
export interface ShadeConfig {
|
|
|
|
|
/** Required: base URL of the prekey server */
|
|
|
|
|
prekeyServer: string;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Storage backend. Accepts:
|
|
|
|
|
* - "memory" — in-memory only (lost on restart, good for tests)
|
|
|
|
|
* - "sqlite:/path/to/file.db" — SQLite backend
|
|
|
|
|
* - { type: 'postgres', url: 'postgres://...' } — PostgreSQL backend
|
|
|
|
|
* - An explicit StorageProvider instance
|
|
|
|
|
*
|
|
|
|
|
* Default: "memory"
|
|
|
|
|
*/
|
|
|
|
|
storage?: string | StorageProvider | { type: 'postgres'; url: string };
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Your address on the prekey server (e.g. "alice@example.com" or "device:abc123").
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
* If omitted (undefined), a random UUID is generated and persisted.
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
*/
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
address?: string | undefined;
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Auto-replenish configuration. When the one-time prekey stock drops
|
|
|
|
|
* below `min`, the SDK will generate enough to reach `target` and
|
|
|
|
|
* upload them to the prekey server.
|
|
|
|
|
*
|
|
|
|
|
* Default: { min: 5, target: 20, intervalMs: 60_000 }
|
|
|
|
|
* Pass `false` to disable.
|
|
|
|
|
*/
|
|
|
|
|
autoReplenish?: { min: number; target: number; intervalMs: number } | false;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Auto identity rotation. Default OFF.
|
|
|
|
|
* Pass an interval string like '7d' or '30d' to enable periodic rotation.
|
|
|
|
|
*/
|
|
|
|
|
autoRotate?: false | '1d' | '7d' | '30d' | '90d';
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Optional observer configuration. If provided, starts a Shade Observer
|
|
|
|
|
* endpoint for the dashboard.
|
|
|
|
|
*/
|
|
|
|
|
observer?: {
|
|
|
|
|
/** Bearer token for the observer. Must be at least 16 chars. */
|
|
|
|
|
token: string;
|
|
|
|
|
/** Port to listen on. If not set, observer is created but not served. */
|
|
|
|
|
port?: number;
|
|
|
|
|
/** Path prefix to mount under (default: "/shade-observer") */
|
|
|
|
|
basePath?: string;
|
|
|
|
|
};
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Optional OpenTelemetry observability hook. Build with
|
|
|
|
|
* `withTracer(otelTracer)` from `@shade/observability`. Default: off.
|
|
|
|
|
*
|
|
|
|
|
* The hook propagates to the session manager (encrypt/decrypt spans),
|
|
|
|
|
* transfer engine (upload/download), and `@shade/files` (per-op).
|
|
|
|
|
* `withTracer()` is itself a no-op unless `SHADE_OTEL_ENABLED` is set,
|
|
|
|
|
* so leaving this configured in code stays free in production until
|
|
|
|
|
* the env-var flips it on.
|
|
|
|
|
*/
|
|
|
|
|
observability?: ObservabilityHook;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Optional Key-Transparency verifier (V3.12). When set, every
|
|
|
|
|
* `fetchBundle` validates the server's inclusion proof against the
|
|
|
|
|
* pinned `logPublicKey` and feeds the STH into a `LightWitness` for
|
|
|
|
|
* split-view detection.
|
|
|
|
|
*
|
|
|
|
|
* Modes:
|
|
|
|
|
* - `'observe'` — verify proofs when present, do not fail when missing.
|
|
|
|
|
* - `'observe-strict'` — require a proof on every successful and 404 response.
|
|
|
|
|
*
|
|
|
|
|
* Default: KT verification disabled. Set this to enable.
|
|
|
|
|
*/
|
|
|
|
|
keyTransparency?: ShadeKTConfig;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Public configuration shape for `Shade.keyTransparency`. */
|
|
|
|
|
export interface ShadeKTConfig {
|
|
|
|
|
mode: 'observe' | 'observe-strict';
|
|
|
|
|
/** Operator's pinned signing public key (32-byte Ed25519) — base64 or raw bytes. */
|
|
|
|
|
logPublicKey: Uint8Array | string;
|
|
|
|
|
/** Reject STHs older than this many ms. Default 24h. */
|
|
|
|
|
maxStaleMs?: number;
|
|
|
|
|
/** Cap on observed-STH cache. Default 1024. */
|
|
|
|
|
witnessMaxStored?: number;
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
export interface ResolvedConfig {
|
|
|
|
|
prekeyServer: string;
|
|
|
|
|
storage: string | StorageProvider | { type: 'postgres'; url: string };
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
address?: string | undefined;
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
autoReplenish: { min: number; target: number; intervalMs: number } | false;
|
|
|
|
|
autoRotate: false | '1d' | '7d' | '30d' | '90d';
|
|
|
|
|
observer?: {
|
|
|
|
|
token: string;
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
port?: number | undefined;
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
basePath: string;
|
|
|
|
|
};
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
observability?: ObservabilityHook;
|
|
|
|
|
keyTransparency?: ResolvedKTConfig;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
export interface ResolvedKTConfig {
|
|
|
|
|
mode: 'observe' | 'observe-strict';
|
|
|
|
|
logPublicKey: Uint8Array;
|
|
|
|
|
maxStaleMs: number;
|
|
|
|
|
witnessMaxStored: number;
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Parse and validate a ShadeConfig, resolving defaults and env var overrides */
|
|
|
|
|
export function resolveConfig(input: ShadeConfig): ResolvedConfig {
|
|
|
|
|
if (!input.prekeyServer) {
|
|
|
|
|
throw new ConfigurationError('prekeyServer is required');
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const autoReplenish = input.autoReplenish === false
|
|
|
|
|
? false
|
|
|
|
|
: {
|
|
|
|
|
min: input.autoReplenish?.min ?? 5,
|
|
|
|
|
target: input.autoReplenish?.target ?? 20,
|
|
|
|
|
intervalMs: input.autoReplenish?.intervalMs ?? 60_000,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
const resolved: ResolvedConfig = {
|
|
|
|
|
prekeyServer: input.prekeyServer.replace(/\/$/, ''),
|
|
|
|
|
storage: input.storage ?? 'memory',
|
|
|
|
|
address: input.address,
|
|
|
|
|
autoReplenish,
|
|
|
|
|
autoRotate: input.autoRotate ?? false,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
if (input.observer) {
|
|
|
|
|
if (!input.observer.token || input.observer.token.length < 16) {
|
|
|
|
|
throw new ConfigurationError('observer.token must be at least 16 characters');
|
|
|
|
|
}
|
|
|
|
|
resolved.observer = {
|
|
|
|
|
token: input.observer.token,
|
|
|
|
|
port: input.observer.port,
|
|
|
|
|
basePath: input.observer.basePath ?? '/shade-observer',
|
|
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
if (input.observability !== undefined) {
|
|
|
|
|
resolved.observability = input.observability;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (input.keyTransparency !== undefined) {
|
|
|
|
|
const kt = input.keyTransparency;
|
|
|
|
|
if (kt.mode !== 'observe' && kt.mode !== 'observe-strict') {
|
|
|
|
|
throw new ConfigurationError(
|
|
|
|
|
`keyTransparency.mode must be 'observe' or 'observe-strict'`,
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
let logKey: Uint8Array;
|
|
|
|
|
if (typeof kt.logPublicKey === 'string') {
|
|
|
|
|
try {
|
|
|
|
|
logKey = new Uint8Array(Buffer.from(kt.logPublicKey, 'base64'));
|
|
|
|
|
} catch {
|
|
|
|
|
throw new ConfigurationError(
|
|
|
|
|
'keyTransparency.logPublicKey must be base64 or Uint8Array',
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
logKey = kt.logPublicKey;
|
|
|
|
|
}
|
|
|
|
|
if (logKey.length !== 32) {
|
|
|
|
|
throw new ConfigurationError(
|
|
|
|
|
`keyTransparency.logPublicKey must be 32 bytes (got ${logKey.length})`,
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
resolved.keyTransparency = {
|
|
|
|
|
mode: kt.mode,
|
|
|
|
|
logPublicKey: logKey,
|
|
|
|
|
maxStaleMs: kt.maxStaleMs ?? 24 * 60 * 60 * 1000,
|
|
|
|
|
witnessMaxStored: kt.witnessMaxStored ?? 1024,
|
|
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:27:59 +02:00
|
|
|
return resolved;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Parse a rotation interval string ('7d', '30d', etc.) into milliseconds */
|
|
|
|
|
export function parseRotationInterval(
|
|
|
|
|
interval: '1d' | '7d' | '30d' | '90d',
|
|
|
|
|
): number {
|
|
|
|
|
const map: Record<string, number> = {
|
|
|
|
|
'1d': 24 * 60 * 60 * 1000,
|
|
|
|
|
'7d': 7 * 24 * 60 * 60 * 1000,
|
|
|
|
|
'30d': 30 * 24 * 60 * 60 * 1000,
|
|
|
|
|
'90d': 90 * 24 * 60 * 60 * 1000,
|
|
|
|
|
};
|
|
|
|
|
return map[interval]!;
|
|
|
|
|
}
|