Stian/Shade
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Details
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Details
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Details
Docker build and publish / docker (push) Has been cancelled
Details
Publish / publish (push) Has been cancelled
Details

Browse Source 
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.

Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
  Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
  bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
  shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
  non-realtime stack.

Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
  version 4.0.0 on /health.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sterister
2026-05-03 18:35:35 +02:00
parent 8b055912b7
commit e6fdf31b49
298 changed files with 37909 additions and 256 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
packages/shade-sdk/src/backup.ts
View File

@@ -132,15 +132,15 @@ export async function exportBackup(
}
/**
* Import a backup blob, decrypt with the passphrase, and write all state
* to the given storage (overwriting existing state).
* Decrypt a backup blob and return its payload without touching storage.
* Useful for peeking at the embedded identity (e.g. to fingerprint it for
* a verification gate) before any state is overwritten.
*/
export async function importBackup(
export async function decryptBackup(
crypto: CryptoProvider,
storage: StorageProvider,
blob: BackupBlob,
passphrase: string,
): Promise<void> {
): Promise<BackupPayload> {
if (blob.version !== BACKUP_VERSION) {
throw new Error(`Unsupported backup version: ${blob.version}`);
}
@@ -149,7 +149,6 @@ export async function importBackup(
const nonce = fromBase64(blob.nonce);
const ciphertext = fromBase64(blob.ciphertext);
// Derive the same key
const key = await crypto.hkdf(
new TextEncoder().encode(passphrase),
salt,
@@ -164,38 +163,56 @@ export async function importBackup(
crypto.zeroize(key);
throw new Error('Wrong passphrase or corrupted backup');
}
crypto.zeroize(key);
const payload = JSON.parse(new TextDecoder().decode(plaintext)) as BackupPayload;
return JSON.parse(new TextDecoder().decode(plaintext)) as BackupPayload;
}
// Restore identity
/**
* Write a previously-decrypted backup payload to storage. Splitting this
* from `decryptBackup` lets callers run an interstitial verification gate
* (V3.3) before any pinned-trust entries are overwritten.
*/
export async function applyBackupPayload(
storage: StorageProvider,
payload: BackupPayload,
): Promise<void> {
if (payload.identity) {
await storage.saveIdentityKeyPair(deserializeIdentityKeyPair(payload.identity));
}
await storage.saveLocalRegistrationId(payload.registrationId);
// Restore signed prekeys
for (const spk of payload.signedPreKeys) {
await storage.saveSignedPreKey(deserializeSignedPreKey(spk.data));
}
// Restore one-time prekeys
for (const otpk of payload.oneTimePreKeys) {
await storage.saveOneTimePreKey(deserializeOneTimePreKey(otpk.data));
}
// Restore sessions
for (const s of payload.sessions) {
await storage.saveSession(s.address, deserializeSessionState(s.state));
}
// Restore trust
for (const t of payload.trustedIdentities) {
await storage.saveTrustedIdentity(t.address, fromBase64(t.key));
}
}
/**
* Import a backup blob, decrypt with the passphrase, and write all state
* to the given storage (overwriting existing state).
*/
export async function importBackup(
crypto: CryptoProvider,
storage: StorageProvider,
blob: BackupBlob,
passphrase: string,
): Promise<void> {
const payload = await decryptBackup(crypto, blob, passphrase);
await applyBackupPayload(storage, payload);
}
/** Serialize a backup blob to a compact single-string form (for copy/paste or QR). */
export function backupToString(blob: BackupBlob): string {
return `shade-backup:v${blob.version}:${blob.salt}:${blob.nonce}:${blob.ciphertext}`;

83
packages/shade-sdk/src/config.ts
View File

@@ -1,5 +1,6 @@
import type { StorageProvider } from '@shade/core';
import { ConfigurationError } from '@shade/core';
import type { ObservabilityHook } from '@shade/observability';
/**
* Shade SDK configuration.
@@ -56,6 +57,43 @@ export interface ShadeConfig {
/** Path prefix to mount under (default: "/shade-observer") */
basePath?: string;
};
/**
* Optional OpenTelemetry observability hook. Build with
* `withTracer(otelTracer)` from `@shade/observability`. Default: off.
*
* The hook propagates to the session manager (encrypt/decrypt spans),
* transfer engine (upload/download), and `@shade/files` (per-op).
* `withTracer()` is itself a no-op unless `SHADE_OTEL_ENABLED` is set,
* so leaving this configured in code stays free in production until
* the env-var flips it on.
*/
observability?: ObservabilityHook;
/**
* Optional Key-Transparency verifier (V3.12). When set, every
* `fetchBundle` validates the server's inclusion proof against the
* pinned `logPublicKey` and feeds the STH into a `LightWitness` for
* split-view detection.
*
* Modes:
* - `'observe'` — verify proofs when present, do not fail when missing.
* - `'observe-strict'` — require a proof on every successful and 404 response.
*
* Default: KT verification disabled. Set this to enable.
*/
keyTransparency?: ShadeKTConfig;
}
/** Public configuration shape for `Shade.keyTransparency`. */
export interface ShadeKTConfig {
mode: 'observe' | 'observe-strict';
/** Operator's pinned signing public key (32-byte Ed25519) — base64 or raw bytes. */
logPublicKey: Uint8Array | string;
/** Reject STHs older than this many ms. Default 24h. */
maxStaleMs?: number;
/** Cap on observed-STH cache. Default 1024. */
witnessMaxStored?: number;
}
export interface ResolvedConfig {
@@ -69,6 +107,15 @@ export interface ResolvedConfig {
port?: number | undefined;
basePath: string;
};
observability?: ObservabilityHook;
keyTransparency?: ResolvedKTConfig;
}
export interface ResolvedKTConfig {
mode: 'observe' | 'observe-strict';
logPublicKey: Uint8Array;
maxStaleMs: number;
witnessMaxStored: number;
}
/** Parse and validate a ShadeConfig, resolving defaults and env var overrides */
@@ -104,6 +151,42 @@ export function resolveConfig(input: ShadeConfig): ResolvedConfig {
};
}
if (input.observability !== undefined) {
resolved.observability = input.observability;
}
if (input.keyTransparency !== undefined) {
const kt = input.keyTransparency;
if (kt.mode !== 'observe' && kt.mode !== 'observe-strict') {
throw new ConfigurationError(
`keyTransparency.mode must be 'observe' or 'observe-strict'`,
);
}
let logKey: Uint8Array;
if (typeof kt.logPublicKey === 'string') {
try {
logKey = new Uint8Array(Buffer.from(kt.logPublicKey, 'base64'));
} catch {
throw new ConfigurationError(
'keyTransparency.logPublicKey must be base64 or Uint8Array',
);
}
} else {
logKey = kt.logPublicKey;
}
if (logKey.length !== 32) {
throw new ConfigurationError(
`keyTransparency.logPublicKey must be 32 bytes (got ${logKey.length})`,
);
}
resolved.keyTransparency = {
mode: kt.mode,
logPublicKey: logKey,
maxStaleMs: kt.maxStaleMs ?? 24 * 60 * 60 * 1000,
witnessMaxStored: kt.witnessMaxStored ?? 1024,
};
}
return resolved;
}

216
packages/shade-sdk/src/gates.ts Normal file
View File

@@ -0,0 +1,216 @@
import type { StorageProvider, PeerVerificationSource } from '@shade/core';
import { FingerprintNotVerifiedError } from '@shade/core';
/**
* Reason a gate was triggered. Surfaced to the registered handler so apps
* can render different UI per gate.
*/
export type FingerprintGate =
| 'first-large-file'
| 'backup-import'
| 'new-device-trust'
| 'inbox-fanout';
/**
* Context handed to a fingerprint-gate handler. The handler decides
* whether the operation may proceed (return `true`) or must be aborted
* (return `false` or throw).
*/
export interface FingerprintGateContext {
/** Peer address being acted on (own address for `backup-import`). */
peerAddress: string;
/** Safety number to display. 60 digits, 12 groups of 5. */
fingerprint: string;
/** Which gate fired. */
gate: FingerprintGate;
/** For `first-large-file`, the file size in bytes. */
fileSize?: number;
}
/** Sync or async predicate. `true` allows; `false` (or throw) rejects. */
export type FingerprintGateHandler = (
ctx: FingerprintGateContext,
) => boolean | Promise<boolean>;
/**
* Registry that tracks gate handlers and the persisted verification state
* for peers. Used internally by `Shade` to wrap critical operations.
*
* Decision tree for every gate check:
* 1. Peer already verified (fingerprint + identityVersion match) → allow.
* 2. Handler registered → invoke; on `true` mark verified, on `false`
* throw `FingerprintNotVerifiedError`.
* 3. No handler registered → log a one-time warning, mark
* `tofu-after-warning`, and allow.
*
* Step 3 satisfies the V3.3 acceptance criterion that apps without
* registered gates get sane defaults instead of hard-failing.
*/
export class FingerprintGateRegistry {
private firstLargeFile: { threshold: number; handler: FingerprintGateHandler } | null = null;
private backupImport: FingerprintGateHandler | null = null;
private newDeviceTrust: FingerprintGateHandler | null = null;
private inboxFanout: FingerprintGateHandler | null = null;
private warnedPeers = new Set<string>();
constructor(private readonly storage: StorageProvider) {}
registerFirstLargeFile(threshold: number, handler: FingerprintGateHandler): void {
if (!Number.isFinite(threshold) || threshold < 0) {
throw new Error('beforeFirstLargeFile: threshold must be a non-negative number');
}
this.firstLargeFile = { threshold, handler };
}
registerBackupImport(handler: FingerprintGateHandler): void {
this.backupImport = handler;
}
registerNewDeviceTrust(handler: FingerprintGateHandler): void {
this.newDeviceTrust = handler;
}
registerInboxFanout(handler: FingerprintGateHandler): void {
this.inboxFanout = handler;
}
/** Default first-large-file threshold: 10 MiB. */
static readonly DEFAULT_LARGE_FILE_THRESHOLD = 10 * 1024 * 1024;
/** Returns the configured threshold (default 10 MiB if not configured). */
getFirstLargeFileThreshold(): number {
return this.firstLargeFile?.threshold ?? FingerprintGateRegistry.DEFAULT_LARGE_FILE_THRESHOLD;
}
/**
* Check whether the recorded verification for `address` is still valid
* against `currentFingerprint` and the peer's current identity-version.
*/
async isVerified(address: string, currentFingerprint: string): Promise<boolean> {
const v = await this.storage.getPeerVerification(address);
if (v === null) return false;
if (v.fingerprint !== currentFingerprint) return false;
const currentVersion = await this.storage.getPeerIdentityVersion(address);
return v.identityVersion === currentVersion;
}
/**
* Persist that `address` has been verified at `fingerprint`. Called
* by the SDK after a handler returns `true`, or directly by the app
* via `Shade.markPeerVerified`.
*/
async markVerified(
address: string,
fingerprint: string,
source: PeerVerificationSource = 'user',
): Promise<void> {
const identityVersion = await this.storage.getPeerIdentityVersion(address);
await this.storage.savePeerVerification({
peerAddress: address,
fingerprint,
verifiedAt: Date.now(),
verifiedBy: source,
identityVersion,
});
}
/** Removes any persisted verification for `address`. */
async revoke(address: string): Promise<void> {
await this.storage.removePeerVerification(address);
}
/**
* Run the first-large-file gate. Returns silently when the file is
* under threshold, when the peer is already verified, or when the
* handler approves. Throws `FingerprintNotVerifiedError` on rejection.
*/
async checkFirstLargeFile(
address: string,
fingerprint: string,
fileSize: number,
): Promise<void> {
const threshold = this.getFirstLargeFileThreshold();
if (fileSize < threshold) return;
await this.runGate({
peerAddress: address,
fingerprint,
gate: 'first-large-file',
fileSize,
}, this.firstLargeFile?.handler ?? null);
}
/**
* Run the backup-import gate. Always invoked regardless of any
* configurable threshold — the spec mandates an irremovable minimum
* gate for backup-import.
*/
async checkBackupImport(address: string, fingerprint: string): Promise<void> {
await this.runGate(
{ peerAddress: address, fingerprint, gate: 'backup-import' },
this.backupImport,
);
}
/** Run the new-device (post-rotation) gate. Always invoked. */
async checkNewDeviceTrust(address: string, fingerprint: string): Promise<void> {
await this.runGate(
{ peerAddress: address, fingerprint, gate: 'new-device-trust' },
this.newDeviceTrust,
);
}
/** Run the inbox-fanout gate (V3.6). Always invoked per recipient. */
async checkInboxFanout(address: string, fingerprint: string): Promise<void> {
await this.runGate(
{ peerAddress: address, fingerprint, gate: 'inbox-fanout' },
this.inboxFanout,
);
}
private async runGate(
ctx: FingerprintGateContext,
handler: FingerprintGateHandler | null,
): Promise<void> {
if (await this.isVerified(ctx.peerAddress, ctx.fingerprint)) return;
if (handler !== null) {
let approved: boolean;
try {
approved = await handler(ctx);
} catch (err) {
throw new FingerprintNotVerifiedError(
ctx.peerAddress,
ctx.gate,
`gate handler threw: ${(err as Error).message}`,
);
}
if (!approved) {
throw new FingerprintNotVerifiedError(ctx.peerAddress, ctx.gate);
}
await this.markVerified(ctx.peerAddress, ctx.fingerprint, 'user');
return;
}
if (!this.warnedPeers.has(ctx.peerAddress)) {
this.warnedPeers.add(ctx.peerAddress);
console.warn(
`[Shade] gate=${ctx.gate} fired for ${ctx.peerAddress} but no handler is registered — ` +
`allowing on TOFU. Register Shade.before${gateMethodSuffix(ctx.gate)}() to require explicit verification.`,
);
}
await this.markVerified(ctx.peerAddress, ctx.fingerprint, 'tofu-after-warning');
}
}
function gateMethodSuffix(gate: FingerprintGate): string {
switch (gate) {
case 'first-large-file':
return 'FirstLargeFile';
case 'backup-import':
return 'BackupImport';
case 'new-device-trust':
return 'NewDeviceTrust';
case 'inbox-fanout':
return 'InboxFanout';
}
}

58
packages/shade-sdk/src/index.ts
View File

@@ -1,14 +1,33 @@
export { createShade } from './create-shade.js';
export { Shade } from './shade.js';
export type {
ShadeUploadOptions,
ShadeWebRtcConfig,
ShadeWebRtcRuntime,
} from './shade.js';
export { generateThumbnail } from './thumbnail.js';
export type { GeneratedThumbnail, ThumbnailGenerationOptions } from './thumbnail.js';
export { ShadeThumbnailCache } from './thumbnail-cache.js';
export type { ThumbnailHit } from './thumbnail-cache.js';
export { resolveConfig, parseRotationInterval } from './config.js';
export { BackgroundTasks } from './background.js';
export {
exportBackup,
importBackup,
decryptBackup,
applyBackupPayload,
backupToString,
backupFromString,
} from './backup.js';
export type { ShadeConfig, ResolvedConfig } from './config.js';
export {
FingerprintGateRegistry,
} from './gates.js';
export type {
FingerprintGate,
FingerprintGateContext,
FingerprintGateHandler,
} from './gates.js';
export type { ShadeConfig, ResolvedConfig, ShadeKTConfig, ResolvedKTConfig } from './config.js';
export type { BackgroundHooks } from './background.js';
export type { BackupBlob, BackupPayload } from './backup.js';
@@ -25,6 +44,7 @@ export {
MemoryControlChannel,
MemoryTransferTransport,
ShadeTransferHttpTransport,
MultiTransportFallback,
createTransferRoutes,
PermissiveAuthenticator,
TransferError,
@@ -34,6 +54,7 @@ export {
TransferOfflineError,
TransferResumeError,
} from '@shade/transfer';
export type { NamedTransport } from '@shade/transfer';
export type {
TransferOptions,
TransferProgress,
@@ -54,4 +75,37 @@ export type {
TransferResumeState,
LaneProgress,
} from '@shade/transfer';
export type { StreamMetadata, LaneInitSpec, LanePartition } from '@shade/streams';
export type {
StreamMetadata,
StreamFileMetadata,
ThumbnailMime,
LaneInitSpec,
LanePartition,
} from '@shade/streams';
export {
THUMBNAIL_MAX_BYTES,
THUMBNAIL_MIME_ALLOWLIST,
isAllowedThumbnailMime,
validateFileMetadata,
declaresThumbnail,
thumbnailStreamIdFor,
mainStreamIdForThumbnail,
} from '@shade/streams';
// ─── Web Workers crypto (V3.8) ─────────────────────────────
export {
createWorkerCryptoProvider,
WorkerCryptoProvider,
WorkerStreamSender,
WorkerStreamReceiver,
createEncryptStream,
createDecryptStream,
DEFAULT_STREAM_CHUNK_SIZE,
WORKER_PROTOCOL_VERSION,
} from '@shade/crypto-web';
export type {
WorkerCryptoProviderOptions,
WorkerLike,
CreateEncryptStreamOptions,
CreateDecryptStreamOptions,
} from '@shade/crypto-web';

721
packages/shade-sdk/src/shade.ts
View File

@@ -4,12 +4,28 @@ import {
ShadeEventEmitter,
NoSessionError,
} from '@shade/core';
import { SubtleCryptoProvider, MemoryStorage } from '@shade/crypto-web';
import {
FingerprintGateRegistry,
type FingerprintGateHandler,
} from './gates.js';
import {
SubtleCryptoProvider,
MemoryStorage,
createWorkerCryptoProvider,
type WorkerCryptoProvider,
createEncryptStream,
createDecryptStream,
type CreateEncryptStreamOptions,
type CreateDecryptStreamOptions,
} from '@shade/crypto-web';
import { encodeEnvelope, decodeEnvelope, inspectEnvelopeType } from '@shade/proto';
import { ShadeFetchTransport } from '@shade/transport';
import { ShadeFetchTransport, type KTVerifierOptions } from '@shade/transport';
import { LightWitness } from '@shade/key-transparency';
import type { SignedTreeHead } from '@shade/key-transparency';
import {
TransferEngine,
ShadeTransferHttpTransport,
MultiTransportFallback,
type ITransferTransport,
type IncomingTransfer,
type TransferHandle,
@@ -18,7 +34,14 @@ import {
} from '@shade/transfer';
import type { Hono } from 'hono';
import { BackgroundTasks } from './background.js';
import { exportBackup, importBackup, backupToString, backupFromString } from './backup.js';
import {
exportBackup,
applyBackupPayload,
decryptBackup,
backupToString,
backupFromString,
} from './backup.js';
import { computeFingerprint, deserializeIdentityKeyPair } from '@shade/core';
import type { ResolvedConfig } from './config.js';
import {
ShadeControlChannel,
@@ -26,6 +49,75 @@ import {
type ControlEnvelopeTransport,
} from './streams-bridge.js';
import { createFilesNamespace, type FilesNamespace } from '@shade/files';
import type { ObservabilityHook } from '@shade/observability';
import {
isAllowedThumbnailMime,
sha256Once,
THUMBNAIL_MAX_BYTES,
type StreamFileMetadata,
type ThumbnailMime,
} from '@shade/streams';
import {
generateThumbnail as generateThumbnailFromBlob,
type ThumbnailGenerationOptions,
} from './thumbnail.js';
/**
* V3.9 — extended upload options. The base `TransferOptions` is forwarded
* verbatim to `@shade/transfer`; the extra fields control the thumbnail
* companion-stream and never leak past the SDK boundary.
*/
/**
* V3.11 — opt-in WebRTC P2P transport. Pass to `shade.configureWebRTC()`
* before the engine is built. The shape mirrors `WebRtcConnectionManager`'s
* options without forcing the SDK to import `@shade/transport-webrtc` at
* the value layer (the import happens lazily inside `engine()`).
*/
export interface ShadeWebRtcConfig {
/**
* WebRTC adapter. Use `nativeRtcFactory()` from `@shade/transport-webrtc`
* in browsers / Deno / Cloudflare Workers; supply your own
* `IRtcFactory` for Node-class environments (`node-datachannel`, `wrtc`).
*/
factory: import('@shade/transport-webrtc').IRtcFactory;
iceServers?: import('@shade/transport-webrtc').ShadeRtcConfig['iceServers'];
iceTransportPolicy?: import('@shade/transport-webrtc').ShadeRtcConfig['iceTransportPolicy'];
bundlePolicy?: import('@shade/transport-webrtc').ShadeRtcConfig['bundlePolicy'];
/** Default 30s. */
connectTimeoutMs?: number;
/** Default 30s. */
requestTimeoutMs?: number;
/** Default 4 MiB. */
backpressureThresholdBytes?: number;
}
/** Live WebRTC runtime returned by `shade.getWebRtcRuntime()`. */
export interface ShadeWebRtcRuntime {
signaling: import('@shade/transport-webrtc').WebRtcSignalingChannel;
manager: import('@shade/transport-webrtc').WebRtcConnectionManager;
transport: import('@shade/transport-webrtc').WebRtcTransferTransport;
fallback: MultiTransportFallback;
/** Internal — wires `engine` into the receiver hooks once it's built. */
attachEngine(engine: TransferEngine): void;
/** Tear down the manager, the signaling channel, and any open peer connections. */
destroy(): void;
}
export interface ShadeUploadOptions extends TransferOptions {
/**
* Pre-generated thumbnail bytes + MIME. Use this on server runtimes
* where in-process image processing is already part of your pipeline,
* or when you have a richer thumbnail than the auto-generator would
* produce (e.g. center-cropped, branded watermark, etc.).
*/
thumbnail?: { bytes: Uint8Array; mime: ThumbnailMime };
/**
* Browser auto-generation. Set to `true` for defaults, or pass a
* config object. Returns `null` (silently skips the thumbnail) on
* runtimes lacking `OffscreenCanvas` + `createImageBitmap`.
*/
generateThumbnail?: boolean | ThumbnailGenerationOptions;
}
/**
* The high-level Shade API.
@@ -63,6 +155,25 @@ export class Shade {
// `@shade/files` namespace, lazy + memoized.
private filesNamespace: FilesNamespace | null = null;
// V3.12 — light-witness for split-view detection.
private ktWitness: LightWitness | null = null;
// V3.11 WebRTC P2P transport. Lazy-built on first engine() if configured.
private webrtcConfig: ShadeWebRtcConfig | null = null;
private webrtcRuntime: ShadeWebRtcRuntime | null = null;
// V3.8 Worker-Crypto. Lazy: configured via `configureWorkerCrypto()`,
// spawned the first time `encryptStream`/`decryptStream` is used.
private workerCryptoConfig: {
workerUrl: URL | string;
idleTimeoutMs?: number;
} | null = null;
private workerCrypto: WorkerCryptoProvider | null = null;
private workerCryptoBoot: Promise<WorkerCryptoProvider> | null = null;
// V3.3 fingerprint gates. Created in `initialize()` once storage is up.
private gates!: FingerprintGateRegistry;
constructor(private readonly config: ResolvedConfig) {}
/**
@@ -83,6 +194,9 @@ export class Shade {
// Step 2: Session manager with event bus attached
this.manager = new ShadeSessionManager(this.crypto, this.storage, {
events: this.events,
...(this.config.observability !== undefined
? { observability: this.config.observability }
: {}),
});
await this.manager.initialize();
@@ -92,10 +206,49 @@ export class Shade {
// Step 4: Transport with our signing key
const identity = await this.storage.getIdentityKeyPair();
if (!identity) throw new Error('Identity not available after initialize');
// V3.12 — wire up KT verifier + light-witness if configured.
let ktOpts: KTVerifierOptions | undefined;
if (this.config.keyTransparency) {
const baseUrl = this.config.prekeyServer;
this.ktWitness = new LightWitness({
crypto: this.crypto,
logPublicKey: this.config.keyTransparency.logPublicKey,
maxStaleMs: this.config.keyTransparency.maxStaleMs,
maxStored: this.config.keyTransparency.witnessMaxStored,
fetcher: {
async fetchLatestSTH() {
const res = await fetch(`${baseUrl}/v1/kt/sth`);
if (!res.ok) throw new Error(`KT /sth: ${res.status}`);
return res.json();
},
async fetchConsistencyProof(from, to) {
const res = await fetch(`${baseUrl}/v1/kt/consistency?from=${from}&to=${to}`);
if (!res.ok) throw new Error(`KT /consistency: ${res.status}`);
return res.json();
},
},
});
ktOpts = {
mode: this.config.keyTransparency.mode,
logPublicKey: this.config.keyTransparency.logPublicKey,
maxStaleMs: this.config.keyTransparency.maxStaleMs,
onObserveSth: async (sth: SignedTreeHead) => {
if (this.ktWitness) {
// The fetched STH was already verified by the transport; feed
// it to the witness for split-view tracking. `observe` may also
// throw on split-view — we let it propagate to the caller.
await this.ktWitness.observe(sth);
}
},
};
}
this.transport = new ShadeFetchTransport({
baseUrl: this.config.prekeyServer,
crypto: this.crypto,
signingPrivateKey: identity.signingPrivateKey,
...(ktOpts ? { keyTransparency: ktOpts } : {}),
});
// Step 5: Initial prekeys + register
@@ -124,6 +277,9 @@ export class Shade {
);
this._background.start();
// Step 7: V3.3 fingerprint gates
this.gates = new FingerprintGateRegistry(this.storage);
this.initialized = true;
}
@@ -183,6 +339,25 @@ export class Shade {
return this.transport;
}
/**
* V3.12 — access the configured Key-Transparency light-witness, or
* `null` when KT was not configured. Useful for surfacing observed
* STHs to UI / dashboards, or for manual gossip checks against
* trusted peers.
*/
getKTWitness(): LightWitness | null {
return this.ktWitness;
}
/**
* Returns the OTel observability hook the SDK was configured with, or
* `undefined` if observability is off. Used by `@shade/files` and other
* sub-modules to inherit the same tracer the rest of the SDK uses.
*/
getObservability(): ObservabilityHook | undefined {
return this.config.observability;
}
/**
* Encrypt a message to a peer. Auto-establishes a session if none exists.
* Returns the ShadeEnvelope ready to send over any transport.
@@ -251,6 +426,97 @@ export class Shade {
return normalize(remote) === normalize(fingerprint);
}
// ─── V3.3 fingerprint gates ───────────────────────────────
/**
* Register a handler that runs before `upload()` proceeds when the file
* is at or above `threshold` bytes and the peer is not yet verified.
* Return `true` to allow + persist the verification, `false` to abort.
*
* Default threshold (when this method is never called): 10 MiB.
*/
beforeFirstLargeFile(threshold: number, handler: FingerprintGateHandler): void {
if (!this.initialized) throw new Error('Not initialized');
this.gates.registerFirstLargeFile(threshold, handler);
}
/**
* Register a handler that runs before `importBackup()` writes to storage.
* The handler receives the fingerprint of the identity *embedded in the
* backup blob*, so the user can OOB-confirm the backup is theirs.
*/
beforeBackupImport(handler: FingerprintGateHandler): void {
if (!this.initialized) throw new Error('Not initialized');
this.gates.registerBackupImport(handler);
}
/**
* Register a handler that runs the first time a peer's rotated identity
* is observed (via `acceptIdentityChange` or X3DH against a new bundle).
*/
beforeNewDeviceTrust(handler: FingerprintGateHandler): void {
if (!this.initialized) throw new Error('Not initialized');
this.gates.registerNewDeviceTrust(handler);
}
/**
* Register a handler that runs per-recipient before an inbox fan-out
* delivery (V3.6). Reserved hook — wired here so apps can register it
* today and have it active automatically when V3.6 ships.
*/
beforeInboxFanout(handler: FingerprintGateHandler): void {
if (!this.initialized) throw new Error('Not initialized');
this.gates.registerInboxFanout(handler);
}
/**
* Mark a peer as verified at their current fingerprint. Call this from
* your own UI (e.g. after the user scans a QR code or reads the safety
* number aloud) to satisfy any gate without going through the handler.
*/
async markPeerVerified(address: string): Promise<void> {
if (!this.initialized) throw new Error('Not initialized');
const fingerprint = await this.manager.getRemoteFingerprint(address);
await this.gates.markVerified(address, fingerprint, 'user');
}
/**
* Returns whether `address` has a current verification (fingerprint and
* identity-version both still match).
*/
async isPeerVerified(address: string): Promise<boolean> {
if (!this.initialized) throw new Error('Not initialized');
const fingerprint = await this.manager.getRemoteFingerprint(address);
return this.gates.isVerified(address, fingerprint);
}
/** Drop any persisted verification for `address`. */
async unmarkPeerVerified(address: string): Promise<void> {
if (!this.initialized) throw new Error('Not initialized');
await this.gates.revoke(address);
}
/**
* Accept a peer's rotated identity. Bumps the per-peer identity-version
* counter so any earlier verification automatically goes stale, then
* runs the `beforeNewDeviceTrust` gate before the new key is pinned.
*/
async acceptIdentityChange(address: string, newIdentityKey: Uint8Array): Promise<void> {
if (!this.initialized) throw new Error('Not initialized');
await this.storage.bumpPeerIdentityVersion(address);
const newFingerprint = await computeFingerprint(
this.crypto,
// X3DH stores DH-only "trusted identity"; in this SDK the trusted
// entry IS the DH public key. We feed it as both args so the
// fingerprint binds to the rotated key material the user is asked
// to confirm.
newIdentityKey,
newIdentityKey,
);
await this.gates.checkNewDeviceTrust(address, newFingerprint);
await this.manager.acceptIdentityChange(address, newIdentityKey);
}
/** Manually rotate the identity (destructive — see docs) */
async rotate(): Promise<void> {
if (!this.initialized) throw new Error('Not initialized');
@@ -311,16 +577,29 @@ export class Shade {
/**
* Restore state from a backup string. Overwrites existing state.
* Call this BEFORE initialize() on a fresh device, or after shutdown() + re-init.
*
* V3.3: invokes the `beforeBackupImport` gate. The handler receives the
* fingerprint of the identity *embedded in the backup* — this lets the
* user OOB-confirm that the backup is theirs before sessions and
* pinned-trust entries are written to disk.
*/
async importBackup(backupString: string, passphrase: string): Promise<void> {
if (!this.initialized) throw new Error('Not initialized');
const blob = backupFromString(backupString);
await importBackup(this.crypto, this.storage, blob, passphrase);
const payload = await decryptBackup(this.crypto, blob, passphrase);
const backupFingerprint = await fingerprintFromBackupPayload(this.crypto, payload);
await this.gates.checkBackupImport(this.address, backupFingerprint);
await applyBackupPayload(this.storage, payload);
// Reload identity after restore
const restored = await this.storage.getIdentityKeyPair();
if (restored) {
// Rebuild the manager and transport with the restored identity
this.manager = new ShadeSessionManager(this.crypto, this.storage, { events: this.events });
this.manager = new ShadeSessionManager(this.crypto, this.storage, {
events: this.events,
...(this.config.observability !== undefined
? { observability: this.config.observability }
: {}),
});
await this.manager.initialize();
this.transport = new ShadeFetchTransport({
baseUrl: this.config.prekeyServer,
@@ -335,6 +614,14 @@ export class Shade {
this._background?.stop();
if (this.transferEngine !== null) this.transferEngine.destroy();
if (this.controlChannel !== null) this.controlChannel.destroy();
if (this.webrtcRuntime !== null) {
this.webrtcRuntime.destroy();
this.webrtcRuntime = null;
}
if (this.workerCrypto !== null) {
await this.workerCrypto.destroy();
this.workerCrypto = null;
}
// Close storage if it has a close method (SQLite)
const closable = this.storage as unknown as { close?: () => void | Promise<void> };
if (typeof closable.close === 'function') {
@@ -343,6 +630,110 @@ export class Shade {
this.initialized = false;
}
// ─── Worker-Crypto streams (V3.8) ──────────────────────────
/**
* Opt in to Web Workers crypto: subsequent `encryptStream` /
* `decryptStream` calls offload all AEAD work to a dedicated worker so
* the main thread stays under the 16 ms-per-frame budget for big
* uploads. The worker is spawned on first use and self-terminates
* after `idleTimeoutMs` of inactivity (default 30 s).
*
* Bundlers resolve worker URLs differently — the recommended idiom is:
*
* ```ts
* shade.configureWorkerCrypto({
* workerUrl: new URL('@shade/crypto-web/worker', import.meta.url),
* });
* ```
*
* See `docs/web-workers.md` for Vite / Webpack / Rollup recipes and
* Safari notes.
*/
configureWorkerCrypto(opts: {
workerUrl: URL | string;
idleTimeoutMs?: number;
}): void {
this.workerCryptoConfig = opts;
}
/**
* Encrypt a `ReadableStream<Uint8Array>` of plaintext into stream-chunk
* wire envelopes via a Web Worker.
*
* The caller pre-negotiates `streamId` + `streamSecret` with the peer
* (typically through `shade.upload()` for HTTP-based delivery, or any
* other channel). The returned `stream` is a TransformStream:
* pipe plaintext in, get encrypted chunks out.
*
* `laneSha256` resolves once the stream finishes (final chunk emitted
* with `isLast=true`). Compare it against the receiver's lane sha256
* for end-to-end integrity proof.
*
* Requires `configureWorkerCrypto()` to be called first.
*/
encryptStream(
opts: Omit<CreateEncryptStreamOptions, 'provider'>,
): Promise<{
stream: TransformStream<Uint8Array, Uint8Array>;
laneSha256: Promise<Uint8Array>;
}> {
return this.ensureWorkerCrypto().then((provider) =>
createEncryptStream({ provider, ...opts }),
);
}
/**
* Inverse of {@link Shade.encryptStream} — decrypt incoming wire
* envelopes back into plaintext. Each input chunk MUST be a complete
* stream-chunk envelope (the wire framing is the caller's job).
*
* Requires `configureWorkerCrypto()` to be called first.
*/
decryptStream(
opts: Omit<CreateDecryptStreamOptions, 'provider'>,
): Promise<{
stream: TransformStream<Uint8Array, Uint8Array>;
laneSha256: Promise<Uint8Array>;
}> {
return this.ensureWorkerCrypto().then((provider) =>
createDecryptStream({ provider, ...opts }),
);
}
/**
* Direct access to the worker-backed `CryptoProvider`. Use when you
* want to run a one-off heavy crypto op (X25519 batch DH, big HKDF
* derivation, etc.) off the main thread without setting up a stream.
*/
async getWorkerCrypto(): Promise<WorkerCryptoProvider> {
return this.ensureWorkerCrypto();
}
private async ensureWorkerCrypto(): Promise<WorkerCryptoProvider> {
if (this.workerCrypto !== null) return this.workerCrypto;
if (this.workerCryptoBoot !== null) return this.workerCryptoBoot;
if (this.workerCryptoConfig === null) {
throw new Error(
'Call shade.configureWorkerCrypto({ workerUrl }) before encryptStream()/decryptStream(). See docs/web-workers.md.',
);
}
const cfg = this.workerCryptoConfig;
this.workerCryptoBoot = (async () => {
const provider = await createWorkerCryptoProvider({
workerUrl: cfg.workerUrl,
...(cfg.idleTimeoutMs !== undefined ? { idleTimeoutMs: cfg.idleTimeoutMs } : {}),
});
this.workerCrypto = provider;
return provider;
})();
try {
return await this.workerCryptoBoot;
} finally {
this.workerCryptoBoot = null;
}
}
// ─── Stream transfers (v0.2.0) ─────────────────────────────
/**
@@ -380,9 +771,120 @@ export class Shade {
/**
* Upload bytes to a peer. Returns a `TransferHandle` that can be paused/
* aborted and awaited. Requires `configureTransfers` to be called first.
*
* V3.3: when the file size is at or above the configured threshold
* (default 10 MiB) and the peer is not yet verified, the registered
* `beforeFirstLargeFile` handler is invoked. Rejection throws
* `FingerprintNotVerifiedError` before any bytes hit the wire.
*
* V3.9: pass `thumbnail: { bytes, mime }` to attach a pre-generated
* preview, or `generateThumbnail: true` to auto-derive a 256x256 preview
* from an image input in browser-class runtimes (no-op elsewhere). The
* thumbnail is shipped as a *separate* E2EE stream and referenced from
* the main stream's `fileMetadata`.
*/
async upload(opts: TransferOptions): Promise<TransferHandle> {
return (await this.engine()).upload(opts);
async upload(opts: ShadeUploadOptions): Promise<TransferHandle> {
if (!this.initialized) throw new Error('Not initialized');
const size = inferTransferSize(opts);
if (size !== null && size >= this.gates.getFirstLargeFileThreshold()) {
// Establish the session up-front so we have a fingerprint to gate on.
// For peers we've never contacted, this is the TOFU moment where the
// gate matters most.
if ((await this.storage.getSession(opts.to)) === null) {
await this.ensureSession(opts.to);
}
const fingerprint = await this.manager.getRemoteFingerprint(opts.to);
await this.gates.checkFirstLargeFile(opts.to, fingerprint, size);
}
const engine = await this.engine();
const thumbnail = await this.resolveThumbnail(opts);
if (thumbnail !== null) {
const fileMeta: StreamFileMetadata = {
...(opts.metadata?.fileMetadata ?? {}),
thumbnailStreamId: thumbnail.streamId,
thumbnailHash: thumbnail.hashB64,
thumbnailMime: thumbnail.mime,
thumbnailBytes: thumbnail.bytes,
};
const merged: TransferOptions = {
...opts,
metadata: {
...(opts.metadata ?? {}),
fileMetadata: fileMeta,
},
};
return engine.upload(merged);
}
return engine.upload(opts);
}
/**
* Coordinate the thumbnail-side of a V3.9 upload. Resolves to either
* - `null` — no thumbnail will be attached (caller passed neither
* `thumbnail` nor a generator that produced bytes), or
* - the streamId + sha256 + mime + bytes of the thumbnail-stream that
* has now been kicked off (it runs to completion in the background;
* the main upload's `done()` is independent).
*/
private async resolveThumbnail(opts: ShadeUploadOptions): Promise<{
streamId: string;
hashB64: string;
mime: ThumbnailMime;
bytes: number;
} | null> {
let bytes: Uint8Array | null = null;
let mime: ThumbnailMime | null = null;
if (opts.thumbnail !== undefined) {
bytes = opts.thumbnail.bytes;
mime = opts.thumbnail.mime;
} else if (opts.generateThumbnail !== undefined && opts.generateThumbnail !== false) {
const genOpts: ThumbnailGenerationOptions =
opts.generateThumbnail === true ? {} : opts.generateThumbnail;
const gen = await generateThumbnailFromBlob(opts.input, genOpts);
if (gen !== null) {
bytes = gen.bytes;
mime = gen.mime;
}
}
if (bytes === null || mime === null) return null;
if (bytes.byteLength > THUMBNAIL_MAX_BYTES) {
throw new Error(
`thumbnail size ${bytes.byteLength} exceeds THUMBNAIL_MAX_BYTES (${THUMBNAIL_MAX_BYTES})`,
);
}
if (!isAllowedThumbnailMime(mime)) {
throw new Error(`thumbnail mime ${mime} not in allowlist`);
}
const hash = sha256Once(bytes);
const hashB64 = bytesToBase64Std(hash);
const engine = await this.engine();
// Ship the thumbnail FIRST so the receiver can present a preview the
// moment the main `stream-init` references it. Single lane, single
// chunk — at ≤ 64 KiB the parallelism overhead would dominate.
const handle = await engine.upload({
to: opts.to,
input: bytes,
lanes: 1,
chunkSize: Math.max(1, bytes.byteLength),
metadata: {
contentType: mime,
userMetadata: {
shadeThumbnail: '1',
},
},
});
// Don't await `done()` — the main upload should not block on the
// thumbnail finishing. Errors on the preview are surfaced via the
// returned handle's events (consumer can listen if they care).
handle.done().catch((err) => {
console.warn('[Shade] thumbnail transfer failed:', err);
});
return {
streamId: handle.streamId,
hashB64,
mime,
bytes: bytes.byteLength,
};
}
/**
@@ -456,6 +958,57 @@ export class Shade {
await this.controlChannel!.acceptEnvelope(from, env);
}
// ─── V3.11 WebRTC P2P transport ────────────────────────────
/**
* Opt in to direct peer-to-peer chunk delivery via WebRTC.
*
* When configured, `upload()` builds a `[WebRTC, HTTP]`
* {@link MultiTransportFallback}: P2P first, HTTP as automatic
* fallback. Signaling (SDP offer/answer + trickle-ICE) rides on top
* of `Shade.send` / `Shade.onMessage` — no out-of-band server needed.
*
* Must be called BEFORE the first `upload()` / `onIncomingTransfer()`
* (those instantiate the transfer engine, which captures the
* transport stack at construction time). Calling later throws.
*
* The `factory` argument is the WebRTC adapter — `nativeRtcFactory()`
* for browsers, a custom one for Node-class environments
* (`node-datachannel`, `wrtc`, etc.). Set `iceServers` to override the
* default public STUN list, or supply TURN credentials for paranoid
* NATs:
*
* ```ts
* import { nativeRtcFactory } from '@shade/transport-webrtc';
* shade.configureWebRTC({
* factory: nativeRtcFactory(),
* iceServers: [
* { urls: 'stun:stun.l.google.com:19302' },
* { urls: 'turn:turn.example.com:3478', username: 'u', credential: 'p' },
* ],
* });
* ```
*/
configureWebRTC(opts: ShadeWebRtcConfig): void {
if (this.transferEngine !== null) {
throw new Error(
'shade.configureWebRTC() must be called before upload()/onIncomingTransfer() builds the engine',
);
}
this.webrtcConfig = opts;
}
/**
* Returns the live WebRTC runtime (signaling channel + connection
* manager + transport) if `configureWebRTC` was called and `engine()`
* has been instantiated. Useful for diagnostics: peek
* `runtime.manager.isConnected('alice')` to see whether a P2P link is
* live, or wire `runtime.fallback.onSwitch(...)` to log demotions.
*/
getWebRtcRuntime(): ShadeWebRtcRuntime | null {
return this.webrtcRuntime;
}
// ─── Internals ─────────────────────────────────────────────
private async engine(): Promise<TransferEngine> {
@@ -466,19 +1019,114 @@ export class Shade {
);
}
this.controlChannel = new ShadeControlChannel(this, this.envelopeOutboxes);
const transport: ITransferTransport = new ShadeTransferHttpTransport({
const httpTransport: ITransferTransport = new ShadeTransferHttpTransport({
resolveBaseUrl: this.peerBaseUrlResolver,
authenticator: await this.makeAuthenticator(),
});
let transport: ITransferTransport = httpTransport;
let webrtcRuntime: ShadeWebRtcRuntime | null = null;
if (this.webrtcConfig !== null) {
webrtcRuntime = await this.buildWebRtcRuntime(this.webrtcConfig, httpTransport);
transport = webrtcRuntime.fallback;
}
this.transferEngine = new TransferEngine({
crypto: this.crypto,
controlChannel: this.controlChannel,
transport,
myAddress: this.address,
...(this.config.observability !== undefined
? { observability: this.config.observability }
: {}),
});
if (webrtcRuntime !== null) {
// Receiver-hooks need to dispatch into the freshly-built engine.
webrtcRuntime.attachEngine(this.transferEngine);
this.webrtcRuntime = webrtcRuntime;
}
return this.transferEngine;
}
/**
* Dynamically import `@shade/transport-webrtc`, wire its signaling
* channel onto our `Shade.send`/`Shade.onMessage`, and build a
* MultiTransportFallback that prefers WebRTC then falls back to HTTP.
*/
private async buildWebRtcRuntime(
cfg: ShadeWebRtcConfig,
httpTransport: ITransferTransport,
): Promise<ShadeWebRtcRuntime> {
// `@shade/transport-webrtc` is an optional peer dep — keep the
// import lazy so consumers that don't use WebRTC don't pay for it.
const moduleId = '@shade/transport-webrtc';
const mod = (await import(moduleId)) as typeof import('@shade/transport-webrtc');
const {
WebRtcSignalingChannel,
WebRtcConnectionManager,
WebRtcTransferTransport,
createShadeBridgeFromShade,
} = mod;
const signaling = new WebRtcSignalingChannel(createShadeBridgeFromShade(this));
let engineRef: TransferEngine | null = null;
const manager = new WebRtcConnectionManager({
factory: cfg.factory,
signaling,
...(cfg.iceServers !== undefined ||
cfg.iceTransportPolicy !== undefined ||
cfg.bundlePolicy !== undefined
? {
config: {
...(cfg.iceServers !== undefined ? { iceServers: cfg.iceServers } : {}),
...(cfg.iceTransportPolicy !== undefined
? { iceTransportPolicy: cfg.iceTransportPolicy }
: {}),
...(cfg.bundlePolicy !== undefined ? { bundlePolicy: cfg.bundlePolicy } : {}),
},
}
: {}),
...(cfg.connectTimeoutMs !== undefined ? { connectTimeoutMs: cfg.connectTimeoutMs } : {}),
receiver: {
onChunk: async (from, streamId, laneId, seq, envelope) => {
if (engineRef === null) {
throw new Error('webrtc receiver hook fired before engine attached');
}
return engineRef.receiveChunk(from, streamId, laneId, seq, envelope);
},
onResumeQuery: async (from, streamId) => {
if (engineRef === null) return null;
return engineRef.getResumeState(from, streamId);
},
},
});
const webrtcTransport = new WebRtcTransferTransport({
manager,
...(cfg.requestTimeoutMs !== undefined ? { requestTimeoutMs: cfg.requestTimeoutMs } : {}),
...(cfg.backpressureThresholdBytes !== undefined
? { backpressureThresholdBytes: cfg.backpressureThresholdBytes }
: {}),
});
const fallback = new MultiTransportFallback([
{ name: 'webrtc', transport: webrtcTransport },
{ name: 'http', transport: httpTransport },
]);
return {
signaling,
manager,
transport: webrtcTransport,
fallback,
attachEngine(engine) {
engineRef = engine;
},
destroy() {
manager.destroy();
signaling.destroy();
},
};
}
private async makeAuthenticator(): Promise<ShadeTransferAuthenticator> {
const identity = await this.storage.getIdentityKeyPair();
if (identity === null) throw new Error('Identity not initialized');
@@ -503,6 +1151,21 @@ export class Shade {
}));
}
/**
* Drop persisted stream-state records whose `updatedAt` is strictly
* less than `olderThan` (Unix ms). Idempotent. Returns silently when
* the configured storage backend does not implement stream-state
* persistence (e.g. memory storage in tests).
*
* Recommended usage: schedule on a daily cron with a 14-day horizon
* — see `docs/streams.md` § Retention. The `bun-server` SDK template
* wires this up by default.
*/
async pruneStreamStates(olderThan: number): Promise<void> {
if (this.storage.pruneStreamStates === undefined) return;
await this.storage.pruneStreamStates(olderThan);
}
private async ensureSession(address: string): Promise<void> {
// Deduplicate concurrent establishment requests
const existing = this.establishing.get(address);
@@ -525,6 +1188,12 @@ export class Shade {
}
}
function bytesToBase64Std(bytes: Uint8Array): string {
let bin = '';
for (let i = 0; i < bytes.length; i++) bin += String.fromCharCode(bytes[i]!);
return btoa(bin);
}
function tryParseMetadata(json: string): import('@shade/streams').StreamMetadata | null {
try {
return JSON.parse(json) as import('@shade/streams').StreamMetadata;
@@ -533,6 +1202,42 @@ function tryParseMetadata(json: string): import('@shade/streams').StreamMetadata
}
}
/**
* Best-effort plaintext-size inference for a `TransferOptions.input`.
* Returns null when the size is genuinely unknowable (raw `ReadableStream`
* without a metadata hint), so the caller can decide whether to gate.
*/
function inferTransferSize(opts: TransferOptions): number | null {
if (typeof opts.metadata?.sizeBytes === 'number') return opts.metadata.sizeBytes;
const input = opts.input;
if (input instanceof Uint8Array) return input.byteLength;
// Blob and File both expose `.size`. Use a structural check so we don't
// depend on lib.dom typings inside the SDK build.
if (typeof (input as unknown as { size?: unknown }).size === 'number') {
return (input as unknown as { size: number }).size;
}
return null;
}
/**
* Compute the safety-number fingerprint for the identity embedded in a
* decrypted backup payload. Used by `Shade.importBackup` to drive the
* `beforeBackupImport` gate before any state is overwritten.
*/
async function fingerprintFromBackupPayload(
crypto: SubtleCryptoProvider,
payload: import('./backup.js').BackupPayload,
): Promise<string> {
if (payload.identity === null) {
// No identity in the backup means there's nothing to fingerprint.
// Return a stable sentinel so the gate handler can still display
// something meaningful instead of throwing here.
return 'no-identity-in-backup';
}
const id = deserializeIdentityKeyPair(payload.identity);
return computeFingerprint(crypto, id.signingPublicKey, id.dhPublicKey);
}
function parseChunkHeader(bytes: Uint8Array): {
streamId: string;
laneId: number;

158
packages/shade-sdk/src/thumbnail-cache.ts Normal file
View File

@@ -0,0 +1,158 @@
/**
* V3.9 — receiver-side thumbnail buffer.
*
* Holds thumbnail-stream bytes keyed by streamId so widgets can render a
* preview as soon as the corresponding *main* stream-init arrives. The
* cache is intentionally tiny (LRU at 32 entries / 1 MiB total) — it
* exists for "show me a preview while I decide whether to accept this
* file"; long-term storage is the consuming app's job.
*
* The cache verifies sha256 *before* surfacing bytes to consumers (the
* declared `thumbnailHash` from the main stream's `fileMetadata` is
* passed in via `setExpectedHash`). Bytes that don't match are
* dropped — a hostile peer cannot make the widget render arbitrary
* pixels by claiming "this is the preview for that other transfer".
*/
import {
isAllowedThumbnailMime,
sha256Once,
THUMBNAIL_MAX_BYTES,
type ThumbnailMime,
} from '@shade/streams';
const MAX_ENTRIES = 32;
const MAX_BYTES = 1024 * 1024; // 1 MiB total
interface CacheEntry {
bytes: Uint8Array;
mime: ThumbnailMime;
/** base64 sha256 of bytes (for matching against declared hash). */
hash: string;
insertedAt: number;
}
export interface ThumbnailHit {
bytes: Uint8Array;
mime: ThumbnailMime;
}
export class ShadeThumbnailCache {
private entries = new Map<string, CacheEntry>();
private bytesUsed = 0;
/** streamId → expected hash (set when the main stream-init arrives). */
private expected = new Map<string, string>();
private listeners = new Set<(streamId: string) => void>();
/**
* Store thumbnail bytes for `streamId`. No-op when:
* - bytes exceed `THUMBNAIL_MAX_BYTES`
* - mime is not in the allowlist
* - the main stream has already declared an `expectedHash` and the
* bytes don't hash to it
*
* Returns `true` when the bytes were accepted into the cache.
*/
put(streamId: string, bytes: Uint8Array, mime: string): boolean {
if (bytes.byteLength > THUMBNAIL_MAX_BYTES) return false;
if (!isAllowedThumbnailMime(mime)) return false;
const hash = bytesToBase64Std(sha256Once(bytes));
const expected = this.expected.get(streamId);
if (expected !== undefined && expected !== hash) return false;
this.evictIfNeeded(bytes.byteLength);
this.entries.set(streamId, {
bytes,
mime,
hash,
insertedAt: Date.now(),
});
this.bytesUsed += bytes.byteLength;
for (const fn of this.listeners) {
try {
fn(streamId);
} catch {
/* listener errors are not the cache's problem */
}
}
return true;
}
/** Inverse of `put`. Returns true when an entry was evicted. */
delete(streamId: string): boolean {
const ex = this.entries.get(streamId);
if (ex === undefined) return false;
this.entries.delete(streamId);
this.bytesUsed -= ex.bytes.byteLength;
return true;
}
/**
* Lookup helper. When `expectedHash` is provided and doesn't match,
* returns null *and* drops the entry (it must have been spoofed).
*/
get(streamId: string, expectedHash?: string): ThumbnailHit | null {
const ex = this.entries.get(streamId);
if (ex === undefined) return null;
if (expectedHash !== undefined && expectedHash !== ex.hash) {
this.delete(streamId);
return null;
}
return { bytes: ex.bytes, mime: ex.mime };
}
/**
* Record the hash the main stream declared for the thumbnail keyed by
* `streamId`. If a cached entry already exists with a mismatching hash,
* it's dropped — only matching bytes ever get rendered.
*/
setExpectedHash(streamId: string, expectedHash: string): void {
this.expected.set(streamId, expectedHash);
const ex = this.entries.get(streamId);
if (ex !== undefined && ex.hash !== expectedHash) {
this.delete(streamId);
}
}
/** Subscribe to "new thumbnail available" events. */
onChange(fn: (streamId: string) => void): () => void {
this.listeners.add(fn);
return () => {
this.listeners.delete(fn);
};
}
/** Number of entries currently held. */
get size(): number {
return this.entries.size;
}
/** Total bytes currently held. */
get totalBytes(): number {
return this.bytesUsed;
}
/** Drop everything. */
clear(): void {
this.entries.clear();
this.expected.clear();
this.bytesUsed = 0;
}
private evictIfNeeded(incomingBytes: number): void {
while (
this.entries.size >= MAX_ENTRIES ||
this.bytesUsed + incomingBytes > MAX_BYTES
) {
const oldestKey = this.entries.keys().next().value;
if (oldestKey === undefined) return;
this.delete(oldestKey);
if (this.entries.size === 0) return;
}
}
}
function bytesToBase64Std(bytes: Uint8Array): string {
let bin = '';
for (let i = 0; i < bytes.length; i++) bin += String.fromCharCode(bytes[i]!);
return btoa(bin);
}

144
packages/shade-sdk/src/thumbnail.ts Normal file
View File

@@ -0,0 +1,144 @@
/**
* V3.9 — thumbnail generation helper.
*
* Browsers expose `OffscreenCanvas` (Workers + main thread) and a Blob /
* File API capable of decoding common image formats; we lean on those to
* synthesize a 256x256 preview without pulling in `sharp` or `node-canvas`.
*
* Server-side runtimes (Bun/Node) typically already have an upstream
* thumbnail (e.g. computed by an image-processing pipeline) — they should
* pass `{ thumbnail: { bytes, mime } }` to `shade.upload()` directly
* instead of `{ generateThumbnail: true }`. `generateThumbnail` returns
* `null` when no in-process generator is available, which the SDK treats
* as "skip the thumbnail" rather than crashing.
*
* The format-hardening invariants (`THUMBNAIL_MAX_BYTES`,
* `THUMBNAIL_MIME_ALLOWLIST`) are enforced *here* so we never produce
* something the receiver would reject.
*/
import {
THUMBNAIL_MAX_BYTES,
isAllowedThumbnailMime,
type ThumbnailMime,
} from '@shade/streams';
export interface GeneratedThumbnail {
bytes: Uint8Array;
mime: ThumbnailMime;
}
export interface ThumbnailGenerationOptions {
/** Output longest-edge in pixels (default 256). */
maxEdge?: number;
/** Preferred output MIME (default `image/webp`, falls back to `image/jpeg`). */
preferMime?: ThumbnailMime;
/** JPEG/WebP quality in [0, 1]; default 0.78. */
quality?: number;
}
interface BlobLike {
size: number;
readonly type: string;
arrayBuffer(): Promise<ArrayBuffer>;
}
interface OffscreenCanvasCtor {
new (width: number, height: number): OffscreenCanvasLike;
}
interface OffscreenCanvasLike {
getContext(kind: '2d'): {
drawImage(image: unknown, dx: number, dy: number, dw: number, dh: number): void;
} | null;
convertToBlob(opts?: { type?: string; quality?: number }): Promise<BlobLike>;
}
interface CreateImageBitmapFn {
(source: BlobLike): Promise<{ width: number; height: number; close(): void }>;
}
function getOffscreenCanvasCtor(): OffscreenCanvasCtor | null {
const g = globalThis as { OffscreenCanvas?: OffscreenCanvasCtor };
return g.OffscreenCanvas ?? null;
}
function getCreateImageBitmap(): CreateImageBitmapFn | null {
const g = globalThis as { createImageBitmap?: CreateImageBitmapFn };
return g.createImageBitmap ?? null;
}
function isImageBlob(input: unknown): input is BlobLike {
if (typeof input !== 'object' || input === null) return false;
const b = input as BlobLike;
return (
typeof b.size === 'number' &&
typeof b.type === 'string' &&
typeof b.arrayBuffer === 'function' &&
b.type.startsWith('image/')
);
}
/**
* Generate a thumbnail from an image input. Returns `null` when:
* - The input is not an image Blob/File.
* - The runtime lacks `OffscreenCanvas` + `createImageBitmap` (Node, Deno
* without polyfills). Server callers should pass `thumbnail` directly.
* - The encoded thumbnail exceeds `THUMBNAIL_MAX_BYTES` even after
* quality back-off (a pathological input — caller should fall back to
* "no thumbnail").
*/
export async function generateThumbnail(
input: unknown,
opts: ThumbnailGenerationOptions = {},
): Promise<GeneratedThumbnail | null> {
if (!isImageBlob(input)) return null;
const Ctor = getOffscreenCanvasCtor();
const createBitmap = getCreateImageBitmap();
if (Ctor === null || createBitmap === null) return null;
const maxEdge = opts.maxEdge ?? 256;
const preferMime = opts.preferMime ?? 'image/webp';
const quality = clamp01(opts.quality ?? 0.78);
const bitmap = await createBitmap(input);
try {
const scale = Math.min(1, maxEdge / Math.max(bitmap.width, bitmap.height));
const w = Math.max(1, Math.round(bitmap.width * scale));
const h = Math.max(1, Math.round(bitmap.height * scale));
const canvas = new Ctor(w, h);
const ctx = canvas.getContext('2d');
if (ctx === null) return null;
ctx.drawImage(bitmap, 0, 0, w, h);
const order: ThumbnailMime[] = preferMime === 'image/webp'
? ['image/webp', 'image/jpeg']
: preferMime === 'image/jpeg'
? ['image/jpeg', 'image/webp']
: ['image/png', 'image/webp', 'image/jpeg'];
for (const mime of order) {
let q = quality;
// Try up to 3 quality back-offs before giving up on this format.
for (let attempt = 0; attempt < 3; attempt++) {
const blob = await canvas.convertToBlob({ type: mime, quality: q });
if (blob.size <= THUMBNAIL_MAX_BYTES) {
if (!isAllowedThumbnailMime(mime)) continue;
const buf = new Uint8Array(await blob.arrayBuffer());
return { bytes: buf, mime };
}
q = Math.max(0.2, q * 0.7);
}
}
return null;
} finally {
bitmap.close();
}
}
function clamp01(n: number): number {
if (!Number.isFinite(n)) return 0.78;
if (n < 0) return 0;
if (n > 1) return 1;
return n;
}