release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled

V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.

Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
  Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
  bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
  shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
  non-realtime stack.

Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
  version 4.0.0 on /health.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-03 18:35:35 +02:00
parent 8b055912b7
commit e6fdf31b49
298 changed files with 37909 additions and 256 deletions

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/cli",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/cli.ts",
"bin": {
@@ -9,7 +9,9 @@
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/keychain": "workspace:*",
"@shade/sdk": "workspace:*",
"@shade/storage-encrypted": "workspace:*",
"@shade/storage-sqlite": "workspace:*",
"@shade/transport": "workspace:*"
},

View File

@@ -12,6 +12,11 @@ import {
import { dashboardCommand } from './commands/dashboard.js';
import { doctorCommand } from './commands/doctor.js';
import { backupExportCommand, backupRestoreCommand } from './commands/backup.js';
import {
migrateStorageCommand,
rotateStorageKeyCommand,
parseStorageArgs,
} from './commands/storage.js';
const VERSION = '0.1.0';
@@ -24,6 +29,7 @@ Commands:
init [name] Scaffold a new Shade project
--template <name> Template to use (default: bun-server)
--prekey-server <url> Override prekey server URL
--doctor Run shade doctor against the new project
fingerprint [address] Print your own or a peer's fingerprint
publish Re-upload your bundle to the prekey server
rotate Rotate the signed prekey
@@ -37,6 +43,19 @@ Commands:
doctor Diagnose setup issues
backup export <file> Export an encrypted backup (prompts for passphrase)
backup restore <file> Restore from a backup file
migrate-storage Encrypt the local SQLite store at-rest (V3.2)
--key-source <kind> passphrase | keychain | injected
--passphrase <s> passphrase for KDF (or env SHADE_STORAGE_PASSPHRASE)
--keychain-service <s> keychain service name (default: shade.storage)
--keychain-account <s> keychain account name (default: default)
--key-hex <hex> inject a raw 32-byte key as hex
--salt-file <path> passphrase salt file (default: <db>.salt)
--dry-run validate + report without writing
--keep-original keep unencrypted tables after migration
--no-backup skip the .bak snapshot
rotate-storage-key Re-key the encrypted SQLite store (V3.2)
--key-source / --passphrase / … (current key — same flags as migrate)
--new-key-source / --new-passphrase / … (target key)
help Show this message
Config:
@@ -98,6 +117,16 @@ async function main(): Promise<void> {
}
break;
}
case 'migrate-storage': {
const opts = parseStorageArgs(args.slice(1));
await migrateStorageCommand(opts);
break;
}
case 'rotate-storage-key': {
const opts = parseStorageArgs(args.slice(1));
await rotateStorageKeyCommand(opts);
break;
}
case 'help':
case '--help':
case '-h':
@@ -125,11 +154,13 @@ function parseInitArgs(args: string[]): {
name?: string;
template?: string;
prekeyServer?: string;
runDoctor?: boolean;
} {
const options: ReturnType<typeof parseInitArgs> = {};
for (let i = 0; i < args.length; i++) {
if (args[i] === '--template') options.template = args[++i];
else if (args[i] === '--prekey-server') options.prekeyServer = args[++i];
else if (args[i] === '--doctor') options.runDoctor = true;
else if (!args[i]!.startsWith('--')) options.name = args[i];
}
return options;

View File

@@ -1,15 +1,19 @@
import { tryLoadConfig } from '../config.js';
import { existsSync } from 'fs';
export interface DoctorOptions {
cwd?: string;
}
/**
* Diagnose common setup issues.
*/
export async function doctorCommand(): Promise<void> {
export async function doctorCommand(opts: DoctorOptions = {}): Promise<void> {
let ok = true;
console.log('\x1b[33mShade doctor\x1b[0m\n');
// 1. Config loadable?
const configResult = tryLoadConfig();
const configResult = tryLoadConfig(opts.cwd);
if (configResult.ok) {
console.log(' \x1b[32m✓\x1b[0m Config loaded from .shaderc.json or env vars');
const config = configResult.config;

View File

@@ -1,6 +1,7 @@
import { existsSync, mkdirSync, writeFileSync, readdirSync, statSync, readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { doctorCommand } from './doctor.js';
const here = dirname(fileURLToPath(import.meta.url));
const TEMPLATES_DIR = join(here, '..', '..', 'templates');
@@ -10,6 +11,7 @@ export interface InitOptions {
template?: string;
prekeyServer?: string;
cwd?: string;
runDoctor?: boolean;
}
export async function initCommand(opts: InitOptions = {}): Promise<void> {
@@ -42,6 +44,11 @@ export async function initCommand(opts: InitOptions = {}): Promise<void> {
console.log(` cd ${name}`);
console.log(' bun install');
console.log(' bun run start');
if (opts.runDoctor) {
console.log('');
await doctorCommand({ cwd: target });
}
}
export function listTemplates(): string[] {

View File

@@ -0,0 +1,208 @@
/**
* `shade migrate-storage` and `shade rotate-storage-key` — at-rest encryption
* lifecycle commands (V3.2).
*
* Both rely on @shade/storage-encrypted; key sources mirror what KeyManager
* accepts: passphrase | keychain | injected (env-injected).
*/
import { existsSync, readFileSync, writeFileSync } from 'node:fs';
import { dirname, resolve } from 'node:path';
import { mkdirSync } from 'node:fs';
import { KeyManager, type KeySource, migrateSqliteToEncrypted, rotateSqliteEncryptionKey } from '@shade/storage-encrypted';
interface CommonStorageOptions {
dbPath?: string;
keySource: 'passphrase' | 'keychain' | 'injected';
passphrase?: string;
keychainService?: string;
keychainAccount?: string;
injectedKeyHex?: string;
saltFile?: string;
}
export interface MigrateCliOptions extends CommonStorageOptions {
dryRun?: boolean;
keepOriginal?: boolean;
noBackup?: boolean;
}
export interface RotateCliOptions extends CommonStorageOptions {
newKeySource: 'passphrase' | 'keychain' | 'injected';
newPassphrase?: string;
newKeychainService?: string;
newKeychainAccount?: string;
newInjectedKeyHex?: string;
newSaltFile?: string;
}
export async function migrateStorageCommand(opts: MigrateCliOptions): Promise<void> {
const dbPath = opts.dbPath ?? process.env.SHADE_DB_PATH ?? '/data/shade-client.db';
const km = await openKeyManager({ ...opts, dbPath, role: 'open-or-create' });
const log = (m: string) => console.log(`${m}`);
console.log(`Migrating ${dbPath} → encrypted at-rest schema…`);
if (opts.dryRun) console.log(' (dry-run: no writes will be made)');
const dryRun = opts.dryRun ?? false;
// Respect explicit --no-backup; otherwise inherit migrate's default
// (which is "no backup during dry-run, backup otherwise").
const backup = opts.noBackup ? false : !dryRun;
const report = await migrateSqliteToEncrypted({
dbPath,
keyManager: km,
log,
dryRun,
keepOriginal: opts.keepOriginal ?? false,
backup,
});
console.log('\nResult:');
console.log(` identity ${report.identity}`);
console.log(` config ${report.config}`);
console.log(` signed_prekeys ${report.signedPrekeys}`);
console.log(` one_time_prekeys ${report.oneTimePrekeys}`);
console.log(` sessions ${report.sessions}`);
console.log(` trusted_identities ${report.trustedIdentities}`);
console.log(` retired_identities ${report.retiredIdentities}`);
console.log(` stream_state ${report.streamStates}`);
console.log(` duration ${report.durationMs}ms`);
if (report.backupPath) console.log(` backup ${report.backupPath}`);
if (report.dryRun) console.log('\nDry-run complete; no data was modified.');
else if (!report.keptOriginal) console.log('\nUnencrypted tables dropped.');
}
export async function rotateStorageKeyCommand(opts: RotateCliOptions): Promise<void> {
const dbPath = opts.dbPath ?? process.env.SHADE_DB_PATH ?? '/data/shade-client.db';
if (!existsSync(dbPath)) throw new Error(`rotate-storage-key: DB not found at ${dbPath}`);
console.log(`Rotating storage key for ${dbPath}`);
const oldKm = await openKeyManager({
dbPath,
role: 'open-existing',
keySource: opts.keySource,
...(opts.passphrase !== undefined ? { passphrase: opts.passphrase } : {}),
...(opts.keychainService !== undefined ? { keychainService: opts.keychainService } : {}),
...(opts.keychainAccount !== undefined ? { keychainAccount: opts.keychainAccount } : {}),
...(opts.injectedKeyHex !== undefined ? { injectedKeyHex: opts.injectedKeyHex } : {}),
...(opts.saltFile !== undefined ? { saltFile: opts.saltFile } : {}),
});
const newKm = await openKeyManager({
dbPath,
role: 'open-or-create',
keySource: opts.newKeySource,
...(opts.newPassphrase !== undefined ? { passphrase: opts.newPassphrase } : {}),
...(opts.newKeychainService !== undefined ? { keychainService: opts.newKeychainService } : {}),
...(opts.newKeychainAccount !== undefined ? { keychainAccount: opts.newKeychainAccount } : {}),
...(opts.newInjectedKeyHex !== undefined ? { injectedKeyHex: opts.newInjectedKeyHex } : {}),
...(opts.newSaltFile !== undefined ? { saltFile: opts.newSaltFile } : {}),
});
const log = (m: string) => console.log(`${m}`);
const result = await rotateSqliteEncryptionKey({ dbPath, oldKeyManager: oldKm, newKeyManager: newKm, log });
console.log(`\nRotation complete: ${result.rowsRotated} rows re-keyed.`);
}
// ─── helpers ──────────────────────────────────────────────────────
interface OpenKmArgs extends CommonStorageOptions {
/** open-existing: salt file MUST already exist for passphrase mode. open-or-create: generate salt if missing. */
role: 'open-existing' | 'open-or-create';
}
async function openKeyManager(args: OpenKmArgs): Promise<KeyManager> {
const source = await buildKeySource(args);
if (source.kind === 'keychain') {
const { getDefaultKeychain } = await import('@shade/keychain');
return KeyManager.open(source, { keychain: getDefaultKeychain() });
}
return KeyManager.open(source);
}
async function buildKeySource(args: OpenKmArgs): Promise<KeySource> {
switch (args.keySource) {
case 'passphrase': {
const passphrase = args.passphrase ?? process.env.SHADE_STORAGE_PASSPHRASE;
if (!passphrase) throw new Error('passphrase required: pass --passphrase or set SHADE_STORAGE_PASSPHRASE');
const dbPath = args.dbPath ?? process.env.SHADE_DB_PATH ?? '/data/shade-client.db';
const saltPath = args.saltFile ?? `${dbPath}.salt`;
const salt = await loadOrCreateSalt(saltPath, args.role);
return { kind: 'passphrase', passphrase, salt };
}
case 'keychain': {
const service = args.keychainService ?? process.env.SHADE_KEYCHAIN_SERVICE ?? 'shade.storage';
const account = args.keychainAccount ?? process.env.SHADE_KEYCHAIN_ACCOUNT ?? 'default';
return {
kind: 'keychain', service, account,
createIfMissing: args.role === 'open-or-create',
};
}
case 'injected': {
const hex = args.injectedKeyHex ?? process.env.SHADE_STORAGE_KEY_HEX;
if (!hex) throw new Error('injected key required: pass --key-hex or set SHADE_STORAGE_KEY_HEX');
const key = hexToBytes(hex);
if (key.length !== 32) throw new Error(`injected key must be 32 bytes (got ${key.length})`);
return { kind: 'injected', key };
}
}
}
async function loadOrCreateSalt(saltPath: string, role: OpenKmArgs['role']): Promise<Uint8Array> {
if (existsSync(saltPath)) {
const buf = readFileSync(saltPath);
if (buf.length < 16) throw new Error(`salt file ${saltPath} is too short (need ≥ 16 bytes)`);
return new Uint8Array(buf);
}
if (role === 'open-existing') {
throw new Error(`salt file not found at ${saltPath} — re-run with the original salt or create new with migrate-storage`);
}
const fresh = new Uint8Array(16);
globalThis.crypto.getRandomValues(fresh);
mkdirSync(dirname(resolve(saltPath)), { recursive: true });
writeFileSync(saltPath, fresh, { mode: 0o600 });
console.log(` • generated new salt at ${saltPath}`);
return fresh;
}
function hexToBytes(hex: string): Uint8Array {
const clean = hex.startsWith('0x') ? hex.slice(2) : hex;
if (clean.length % 2 !== 0) throw new Error('hex string must have even length');
const out = new Uint8Array(clean.length / 2);
for (let i = 0; i < out.length; i++) {
out[i] = parseInt(clean.slice(i * 2, i * 2 + 2), 16);
}
return out;
}
export function parseStorageArgs(args: string[]): MigrateCliOptions & RotateCliOptions {
const out: MigrateCliOptions & RotateCliOptions = {
keySource: 'passphrase',
newKeySource: 'passphrase',
};
for (let i = 0; i < args.length; i++) {
const a = args[i];
switch (a) {
case '--db-path': out.dbPath = args[++i]; break;
case '--key-source': out.keySource = args[++i] as MigrateCliOptions['keySource']; break;
case '--passphrase': out.passphrase = args[++i]; break;
case '--keychain-service': out.keychainService = args[++i]; break;
case '--keychain-account': out.keychainAccount = args[++i]; break;
case '--key-hex': out.injectedKeyHex = args[++i]; break;
case '--salt-file': out.saltFile = args[++i]; break;
case '--dry-run': out.dryRun = true; break;
case '--keep-original': out.keepOriginal = true; break;
case '--no-backup': out.noBackup = true; break;
// rotation
case '--new-key-source': out.newKeySource = args[++i] as RotateCliOptions['newKeySource']; break;
case '--new-passphrase': out.newPassphrase = args[++i]; break;
case '--new-keychain-service': out.newKeychainService = args[++i]; break;
case '--new-keychain-account': out.newKeychainAccount = args[++i]; break;
case '--new-key-hex': out.newInjectedKeyHex = args[++i]; break;
case '--new-salt-file': out.newSaltFile = args[++i]; break;
default:
if (a?.startsWith('--')) throw new Error(`unknown flag: ${a}`);
}
}
return out;
}

View File

@@ -39,8 +39,23 @@ curl -X POST http://localhost:3000/receive \
Returns the decrypted plaintext.
## Stream-state retention
Resumable file transfers persist a per-stream record. The template runs
a daily cron that drops anything idle for more than
`SHADE_STREAM_RETENTION_DAYS` days (default **14**). Override at
deploy:
```bash
SHADE_STREAM_RETENTION_DAYS=7 bun run start
```
See [`docs/streams.md`](../../../../docs/streams.md) § Retention for the
full guidance and tuning per use case.
## Next steps
- Wire a real delivery layer (WebSocket, HTTP push, etc.)
- Run `shade dashboard` to watch live activity
- Compare fingerprints with peers out-of-band before trusting sessions
- Walk through [`docs/PRODUCTION-CHECKLIST.md`](../../../../docs/PRODUCTION-CHECKLIST.md) before going live

View File

@@ -22,6 +22,28 @@ shade.onMessage((from, msg) => {
console.log(`[${from}] ${msg}`);
});
// ─── Resumable-stream retention ──────────────────────────────────
//
// Drop stream-state records older than SHADE_STREAM_RETENTION_DAYS
// (default 14d). See docs/streams.md § Retention. Override the
// horizon with the env var; cron interval is fixed at 24h.
const STREAM_RETENTION_DAYS = Number(process.env.SHADE_STREAM_RETENTION_DAYS ?? 14);
const ONE_DAY_MS = 24 * 60 * 60 * 1000;
const HORIZON_MS = STREAM_RETENTION_DAYS * ONE_DAY_MS;
async function pruneStreamStatesOnce(): Promise<void> {
try {
await shade.pruneStreamStates(Date.now() - HORIZON_MS);
} catch (err) {
console.error('[shade] pruneStreamStates failed:', err);
}
}
await pruneStreamStatesOnce();
const streamPruneTimer = setInterval(pruneStreamStatesOnce, ONE_DAY_MS);
// Don't keep the process alive for the timer alone.
if (typeof streamPruneTimer.unref === 'function') streamPruneTimer.unref();
const app = new Hono();
app.get('/', (c) => c.text(`__PROJECT_NAME__ — Shade-enabled backend`));

View File

@@ -71,6 +71,37 @@ describe('CLI: init command', () => {
);
});
test('init with --doctor runs diagnostics against the new project', async () => {
const port = 19500 + Math.floor(Math.random() * 200);
const app = createPrekeyServer({
crypto,
store: new MemoryPrekeyStore(),
disableRateLimit: true,
});
const server = Bun.serve({ port, fetch: app.fetch });
const logs: string[] = [];
const originalLog = console.log;
console.log = (...args) => logs.push(args.join(' '));
try {
await initCommand({
name: 'doc-app',
template: 'bun-server',
cwd: tmpDir,
prekeyServer: `http://localhost:${port}`,
runDoctor: true,
});
} finally {
console.log = originalLog;
server.stop();
}
const out = logs.join('\n');
expect(out).toContain('Shade doctor');
expect(out).toContain('Prekey server is reachable');
});
test('chat-demo template scaffolds correctly', async () => {
await initCommand({ name: 'chat', template: 'chat-demo', cwd: tmpDir });
@@ -81,6 +112,26 @@ describe('CLI: init command', () => {
const alice = readFileSync(join(target, 'src/alice.ts'), 'utf-8');
expect(alice).not.toContain('__PROJECT_NAME__');
});
test('bun-server template wires the resumable-stream prune cron by default', async () => {
// V3.1 acceptance: a freshly-scaffolded bun-server must call
// shade.pruneStreamStates on startup + on a daily interval, with
// the SHADE_STREAM_RETENTION_DAYS env var as the override knob.
await initCommand({ name: 'cron-app', template: 'bun-server', cwd: tmpDir });
const indexSrc = readFileSync(
join(tmpDir, 'cron-app', 'src/index.ts'),
'utf-8',
);
expect(indexSrc).toContain('shade.pruneStreamStates(');
expect(indexSrc).toContain('SHADE_STREAM_RETENTION_DAYS');
expect(indexSrc).toContain('setInterval');
// Default horizon stays 14 days unless overridden.
expect(indexSrc).toMatch(/\?\?\s*14/);
// Cron timer must not keep the process alive on its own.
expect(indexSrc).toContain('.unref');
});
});
describe('CLI: config loading', () => {

View File

@@ -1,9 +1,12 @@
{
"name": "@shade/core",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"dependencies": {
"@shade/observability": "workspace:*"
},
"peerDependencies": {
"@shade/crypto-web": "workspace:*"
},

View File

@@ -88,6 +88,25 @@ export class IdentityRotationError extends ShadeError {
}
}
/**
* Thrown when a fingerprint gate (V3.3) blocks an operation because the
* peer's safety number has not been verified, or the registered handler
* returned `false`.
*/
export class FingerprintNotVerifiedError extends ShadeError {
constructor(
public readonly peerAddress: string,
public readonly gate: 'first-large-file' | 'backup-import' | 'new-device-trust' | 'inbox-fanout',
message?: string,
) {
super(
'SHADE_FINGERPRINT_NOT_VERIFIED',
message ?? `Fingerprint not verified for ${peerAddress} (gate: ${gate})`,
);
this.name = 'FingerprintNotVerifiedError';
}
}
// ─── Infrastructure Errors ───────────────────────────────────
export class NetworkError extends ShadeError {
@@ -152,6 +171,7 @@ export function errorToHttpStatus(error: unknown): number {
case 'SHADE_UNAUTHORIZED':
return 401;
case 'SHADE_UNTRUSTED_IDENTITY':
case 'SHADE_FINGERPRINT_NOT_VERIFIED':
return 403;
case 'SHADE_NO_SESSION':
case 'SHADE_PREKEY_NOT_FOUND':

View File

@@ -71,7 +71,11 @@ export async function initSenderSession(
* Initialize a session as the receiver (Bob, after X3DH).
*
* Bob knows the root key and his own signed prekey (which was used as
* the initial DH ratchet keypair).
* the initial DH ratchet keypair). The keypair is COPIED into the
* session — the receiving side's DH ratchet will eventually rotate
* `dhSend` and zeroize the previous private key, and that scratch
* buffer must NOT be the same memory as the persisted signed prekey
* (which is shared with future X3DH establishments from other senders).
*/
export function initReceiverSession(
rootKey: Uint8Array,
@@ -83,7 +87,10 @@ export function initReceiverSession(
rootKey,
sendChain: { chainKey: new Uint8Array(32), counter: 0 },
receiveChain: null,
dhSend: localDHKeyPair,
dhSend: {
publicKey: new Uint8Array(localDHKeyPair.publicKey),
privateKey: new Uint8Array(localDHKeyPair.privateKey),
},
dhReceive: null,
previousSendCounter: 0,
skippedKeys: new Map(),

View File

@@ -26,6 +26,16 @@ import {
import { NoSessionError } from './errors.js';
import { computeFingerprint, shortFingerprint } from './fingerprint.js';
import { ShadeEventEmitter, shortHash } from './events.js';
import {
ATTR_ERROR_CODE,
ATTR_OP,
ATTR_PEER_HASH,
ATTR_RESULT,
NOOP_HOOK,
peerHash,
type ObservabilityHook,
type Span,
} from '@shade/observability';
const enc = new TextEncoder();
const dec = new TextDecoder();
@@ -60,6 +70,7 @@ export class ShadeSessionManager {
private registrationId: number = 0;
private currentSignedPreKeyId: number = 0;
private readonly events?: ShadeEventEmitter;
private readonly observability: ObservabilityHook;
/**
* Per-address operation chain. Both encrypt and decrypt mutate ratchet
* state in place (counter, DH key, skipped-keys cache); concurrent
@@ -72,11 +83,46 @@ export class ShadeSessionManager {
constructor(
private readonly crypto: CryptoProvider,
private readonly storage: StorageProvider,
options: { events?: ShadeEventEmitter } = {},
options: { events?: ShadeEventEmitter; observability?: ObservabilityHook } = {},
) {
if (options.events !== undefined) {
this.events = options.events;
}
this.observability = options.observability ?? NOOP_HOOK;
}
/**
* Wrap a per-peer crypto op in a PII-safe span. The span captures the
* mutex-acquire latency separately from the inner crypto work so a
* "ratchet contention" pathology shows up clearly in traces.
*/
private async withSpan<T>(
op: 'encrypt' | 'decrypt',
address: string,
fn: () => Promise<T>,
): Promise<T> {
const span: Span = this.observability.startSpan(`shade.session.${op}`, {
[ATTR_OP]: op,
[ATTR_PEER_HASH]: peerHash(address),
});
const lockStart = nowMs();
try {
return await this.runUnderPeerLock(address, async () => {
span.setAttribute('shade.lock.wait_ms', Math.round(nowMs() - lockStart));
const result = await fn();
span.setAttribute(ATTR_RESULT, 'ok');
span.setStatus('ok');
return result;
});
} catch (err) {
span.setAttribute(ATTR_RESULT, 'error');
span.setAttribute(ATTR_ERROR_CODE, sessionErrorCodeOf(err));
span.recordException(err);
span.setStatus('error');
throw err;
} finally {
span.end();
}
}
/**
@@ -396,7 +442,7 @@ export class ShadeSessionManager {
* Subsequent messages are standard RatchetMessages.
*/
async encrypt(address: string, plaintext: string): Promise<ShadeEnvelope> {
return this.runUnderPeerLock(address, async () => {
return this.withSpan('encrypt', address, async () => {
const session = await this.storage.getSession(address);
if (!session) throw new NoSessionError(address);
@@ -444,7 +490,7 @@ export class ShadeSessionManager {
* Decrypt a message from a peer. Handles both PreKeyMessage and RatchetMessage.
*/
async decrypt(address: string, envelope: ShadeEnvelope): Promise<string> {
return this.runUnderPeerLock(address, async () => {
return this.withSpan('decrypt', address, async () => {
if (envelope.type === 'prekey') {
return this.decryptPreKeyMessage(address, envelope.content as PreKeyMessage);
}
@@ -522,3 +568,18 @@ function arraysEqual(a: Uint8Array, b: Uint8Array): boolean {
}
return true;
}
function nowMs(): number {
return typeof performance !== 'undefined' ? performance.now() : Date.now();
}
function sessionErrorCodeOf(err: unknown): string {
if (err === null || err === undefined) return 'SHADE_UNKNOWN';
if (typeof err === 'object') {
const code = (err as { code?: unknown }).code;
if (typeof code === 'string' && code.length > 0) return code;
const name = (err as { name?: unknown }).name;
if (typeof name === 'string' && name.length > 0) return name;
}
return 'SHADE_UNKNOWN';
}

View File

@@ -38,6 +38,29 @@ export interface PersistedStreamState {
updatedAt: number;
}
/**
* Why a peer's fingerprint was deemed verified.
* - `user`: human did the side-channel comparison and confirmed.
* - `transitive`: trust derived from another verified party (reserved for V3.10).
* - `tofu-after-warning`: caller bypassed gate after seeing a warning.
*/
export type PeerVerificationSource = 'user' | 'transitive' | 'tofu-after-warning';
/**
* Persistent record that a peer's safety number was verified at a point
* in time. `identityVersion` is the local counter for that peer's identity:
* incrementing it (e.g. via `bumpPeerIdentityVersion` on `acceptIdentityChange`)
* invalidates the saved verification because `isPeerVerified` requires the
* stored version to equal the current version.
*/
export interface PeerVerification {
peerAddress: string;
fingerprint: string;
verifiedAt: number;
verifiedBy: PeerVerificationSource;
identityVersion: number;
}
/**
* StorageProvider — abstract interface for persisting cryptographic state.
*
@@ -115,6 +138,34 @@ export interface StorageProvider {
/** Remove retired identities older than the given timestamp */
pruneRetiredIdentities(olderThan: number): Promise<void>;
// ─── Peer verifications (V3.3) ────────────────────────────
/**
* Persist or replace the verification record for a peer. Idempotent
* upsert on `peerAddress`.
*/
savePeerVerification(verification: PeerVerification): Promise<void>;
/** Look up the saved verification for a peer (null if never verified). */
getPeerVerification(address: string): Promise<PeerVerification | null>;
/** Remove a peer's verification record (e.g. user revoked trust). */
removePeerVerification(address: string): Promise<void>;
/**
* Returns the current local identity-version counter for a peer.
* Defaults to 1 when the peer has never been seen. Bumped by
* `bumpPeerIdentityVersion` whenever the peer rotates identity.
*/
getPeerIdentityVersion(address: string): Promise<number>;
/**
* Increment the peer's identity-version counter and return the new value.
* Called from `acceptIdentityChange` so previous verification rows (which
* carry the old version number) become stale.
*/
bumpPeerIdentityVersion(address: string): Promise<number>;
// ─── Stream-transfer resume state (optional, added in v0.2.0) ──
/**

View File

@@ -8,12 +8,23 @@ import {
kdfChainKey,
deriveInitialRootKey,
} from '../src/index.js';
import { encodeEnvelope, decodeEnvelope } from '@shade/proto';
import type { RatchetMessage, ShadeEnvelope } from '../src/index.js';
import { encodeEnvelope, decodeEnvelope, encodeStreamChunk, decodeStreamChunk } from '@shade/proto';
import type { PreKeyMessage, RatchetMessage, ShadeEnvelope } from '../src/index.js';
// Imported via relative path: shade-streams depends on shade-core, so adding
// it to shade-core's dependencies would create a workspace cycle.
import {
deriveStreamKey,
deriveLaneKey,
buildChunkNonce,
buildChunkAad,
aesGcmEncryptWithNonce,
aesGcmDecryptWithNonce,
} from '../../shade-streams/src/index.js';
const crypto = new SubtleCryptoProvider();
const VECTORS_DIR = join(import.meta.dir, '..', '..', '..', 'test-vectors');
const EXPECTED_VECTOR_VERSION = 2;
function hex(bytes: Uint8Array): string {
return Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('');
@@ -27,8 +38,49 @@ function fromHex(str: string): Uint8Array {
return bytes;
}
function loadVectors(name: string): any {
return JSON.parse(readFileSync(join(VECTORS_DIR, name), 'utf-8'));
function loadVectors(name: string): { version: number; vectors: any[] } {
const parsed = JSON.parse(readFileSync(join(VECTORS_DIR, name), 'utf-8'));
expect(parsed.version).toBe(EXPECTED_VECTOR_VERSION);
return parsed;
}
async function aesGcmEncryptDeterministic(
key: Uint8Array,
nonce: Uint8Array,
plaintext: Uint8Array,
aad: Uint8Array,
): Promise<Uint8Array> {
const subtle = globalThis.crypto.subtle;
const aesKey = await subtle.importKey(
'raw',
key as unknown as ArrayBuffer,
'AES-GCM',
false,
['encrypt', 'decrypt'],
);
const out = await subtle.encrypt(
{
name: 'AES-GCM',
iv: nonce as unknown as ArrayBuffer,
additionalData: aad as unknown as ArrayBuffer,
},
aesKey,
plaintext as unknown as ArrayBuffer,
);
return new Uint8Array(out);
}
function encodeRatchetHeader(
dhPublicKey: Uint8Array,
previousCounter: number,
counter: number,
): Uint8Array {
const buf = new Uint8Array(40);
buf.set(dhPublicKey, 0);
const view = new DataView(buf.buffer);
view.setUint32(32, previousCounter, false);
view.setUint32(36, counter, false);
return buf;
}
describe('Cross-platform test vectors', () => {
@@ -82,9 +134,10 @@ describe('Cross-platform test vectors', () => {
}
});
test('Wire format vectors match', () => {
test('Wire format: RatchetMessage encode + decode', () => {
const { vectors } = loadVectors('wire-format.json');
const v = vectors[0];
const v = vectors.find((x: any) => x.kind === 'ratchet');
expect(v).toBeDefined();
const msg: RatchetMessage = {
dhPublicKey: fromHex(v.message.dhPublicKey),
@@ -102,11 +155,295 @@ describe('Cross-platform test vectors', () => {
const encoded = encodeEnvelope(envelope);
expect(hex(encoded)).toBe(v.encoded);
// Also verify round-trip decode
const decoded = decodeEnvelope(encoded);
expect(decoded.type).toBe('ratchet');
const rm = decoded.content as RatchetMessage;
expect(rm.counter).toBe(msg.counter);
expect(hex(rm.ciphertext)).toBe(hex(msg.ciphertext));
});
test('Wire format: PreKeyMessage encode + decode (with and without OTPK)', () => {
const { vectors } = loadVectors('wire-format.json');
const preKeyVectors = vectors.filter((x: any) => x.kind === 'prekey');
expect(preKeyVectors.length).toBeGreaterThanOrEqual(2);
for (const v of preKeyVectors) {
const inner: RatchetMessage = {
dhPublicKey: fromHex(v.message.inner.dhPublicKey),
previousCounter: v.message.inner.previousCounter,
counter: v.message.inner.counter,
ciphertext: fromHex(v.message.inner.ciphertext),
nonce: fromHex(v.message.inner.nonce),
};
const pre: PreKeyMessage = {
registrationId: v.message.registrationId,
preKeyId: v.message.preKeyId === null ? undefined : v.message.preKeyId,
signedPreKeyId: v.message.signedPreKeyId,
ephemeralKey: fromHex(v.message.ephemeralKey),
identityDHKey: fromHex(v.message.identityDHKey),
message: inner,
};
const envelope: ShadeEnvelope = {
type: 'prekey',
content: pre,
timestamp: 0,
senderAddress: '',
};
const encoded = encodeEnvelope(envelope);
expect(hex(encoded)).toBe(v.encoded);
const decoded = decodeEnvelope(encoded);
expect(decoded.type).toBe('prekey');
const dm = decoded.content as PreKeyMessage;
expect(dm.registrationId).toBe(pre.registrationId);
expect(dm.preKeyId).toBe(pre.preKeyId);
expect(dm.signedPreKeyId).toBe(pre.signedPreKeyId);
expect(hex(dm.ephemeralKey)).toBe(hex(pre.ephemeralKey));
expect(hex(dm.message.ciphertext)).toBe(hex(inner.ciphertext));
}
});
test('Streams 0x11: deriveStreamKey + deriveLaneKey + nonce/AAD + chunk encrypt + wire roundtrip', async () => {
const { vectors } = loadVectors('streams.json');
// 1. deriveStreamKey
const sk = vectors.find((v: any) => v.description.startsWith('deriveStreamKey'));
expect(sk).toBeDefined();
const streamKey = await deriveStreamKey(crypto, fromHex(sk.streamSecret), fromHex(sk.streamId));
expect(hex(streamKey)).toBe(sk.streamKey);
// 2. deriveLaneKey for each laneId
const lk = vectors.find((v: any) => v.description.startsWith('deriveLaneKey'));
expect(lk).toBeDefined();
for (const lane of lk.lanes) {
const k = await deriveLaneKey(crypto, fromHex(lk.streamKey), fromHex(lk.streamId), lane.laneId);
expect(hex(k)).toBe(lane.laneKey);
}
// 3. buildChunkNonce
const nv = vectors.find((v: any) => v.description.startsWith('buildChunkNonce'));
expect(nv).toBeDefined();
for (const n of nv.nonces) {
const seq = BigInt(n.seq);
const out = buildChunkNonce(n.laneId, seq);
expect(hex(out)).toBe(n.nonce);
}
// 4. buildChunkAad
const av = vectors.find((v: any) => v.description.startsWith('buildChunkAad'));
expect(av).toBeDefined();
for (const c of av.cases) {
const seq = BigInt(c.seq);
const out = buildChunkAad(fromHex(av.streamId), c.laneId, seq, c.isLast);
expect(hex(out)).toBe(c.aad);
}
// 5. End-to-end chunk encrypt + decrypt
const ev = vectors.find((v: any) => v.description.startsWith('End-to-end chunk encrypt'));
expect(ev).toBeDefined();
const ct = await aesGcmEncryptWithNonce(
fromHex(ev.laneKey),
fromHex(ev.nonce),
fromHex(ev.plaintext),
fromHex(ev.aad),
);
expect(hex(ct)).toBe(ev.ciphertext);
const pt = await aesGcmDecryptWithNonce(
fromHex(ev.laneKey),
fromHex(ev.nonce),
fromHex(ev.ciphertext),
fromHex(ev.aad),
);
expect(hex(pt)).toBe(ev.plaintext);
// 6. Wire 0x11 envelope encode/decode
const wv = vectors.find((v: any) => v.description.startsWith('Wire 0x11'));
expect(wv).toBeDefined();
const encoded = encodeStreamChunk({
streamId: fromHex(wv.streamId),
laneId: wv.laneId,
seq: BigInt(wv.seq),
isLast: wv.isLast,
nonce: fromHex(wv.nonce),
aad: fromHex(wv.extraAad),
ciphertext: fromHex(wv.ciphertext),
});
expect(hex(encoded)).toBe(wv.encoded);
const decoded = decodeStreamChunk(encoded);
expect(hex(decoded.streamId)).toBe(wv.streamId);
expect(decoded.laneId).toBe(wv.laneId);
expect(decoded.seq.toString()).toBe(wv.seq);
expect(decoded.isLast).toBe(wv.isLast);
expect(hex(decoded.nonce)).toBe(wv.nonce);
expect(hex(decoded.ciphertext)).toBe(wv.ciphertext);
});
test('Backup v1: HKDF backupKey + AES-GCM roundtrip', async () => {
const { vectors } = loadVectors('backup.json');
const kv = vectors.find((v: any) => v.description.startsWith('Backup v1: HKDF'));
expect(kv).toBeDefined();
const backupKey = await crypto.hkdf(
new TextEncoder().encode(kv.passphrase),
fromHex(kv.salt),
new TextEncoder().encode(kv.info),
32,
);
expect(hex(backupKey)).toBe(kv.backupKey);
const ev = vectors.find((v: any) => v.description.startsWith('Backup v1: AES-256-GCM'));
expect(ev).toBeDefined();
const ct = await aesGcmEncryptDeterministic(
fromHex(ev.backupKey),
fromHex(ev.nonce),
fromHex(ev.plaintext),
new Uint8Array(0),
);
expect(hex(ct)).toBe(ev.ciphertext);
const pt = await crypto.aesGcmDecrypt(
fromHex(ev.backupKey),
fromHex(ev.ciphertext),
fromHex(ev.nonce),
);
expect(hex(pt)).toBe(ev.plaintext);
});
test('Group sender-keys: header AAD + step + Ed25519 signature', async () => {
const { vectors } = loadVectors('group.json');
// 1. Header AAD encoding
const hv = vectors.find((v: any) => v.description.startsWith('Sender header AAD'));
expect(hv).toBeDefined();
const enc = new TextEncoder();
const gBytes = enc.encode(hv.groupId);
const sBytes = enc.encode(hv.senderAddress);
const aad = new Uint8Array(2 + gBytes.length + 2 + sBytes.length + 4);
const view = new DataView(aad.buffer);
let off = 0;
view.setUint16(off, gBytes.length, false); off += 2;
aad.set(gBytes, off); off += gBytes.length;
view.setUint16(off, sBytes.length, false); off += 2;
aad.set(sBytes, off); off += sBytes.length;
view.setUint32(off, hv.iteration, false);
expect(hex(aad)).toBe(hv.aad);
// 2. Sender-key step
const sv = vectors.find((v: any) => v.description.startsWith('Sender-key step'));
expect(sv).toBeDefined();
const chain = await kdfChainKey(crypto, fromHex(sv.chainKey));
expect(hex(chain.newChainKey)).toBe(sv.newChainKey);
expect(hex(chain.messageKey)).toBe(sv.messageKey);
const ct = await aesGcmEncryptDeterministic(
chain.messageKey,
fromHex(sv.nonce),
fromHex(sv.plaintext),
fromHex(sv.aad),
);
expect(hex(ct)).toBe(sv.ciphertext);
// Ed25519 sign(aad || ct) — verify signature is valid for the recorded keys
const signed = new Uint8Array(fromHex(sv.aad).length + ct.length);
signed.set(fromHex(sv.aad), 0);
signed.set(ct, fromHex(sv.aad).length);
const ok = await crypto.verify(
fromHex(sv.signingPublicKey),
signed,
fromHex(sv.signature),
);
expect(ok).toBe(true);
// Decrypt roundtrip
const pt = await crypto.aesGcmDecrypt(
chain.messageKey,
fromHex(sv.ciphertext),
fromHex(sv.nonce),
fromHex(sv.aad),
);
expect(hex(pt)).toBe(sv.plaintext);
});
test('Storage encryption HKDF subset: storageKey + fieldKey + rowNonce', async () => {
const { vectors } = loadVectors('storage-hkdf.json');
const sv = vectors.find((v: any) => v.description.startsWith('Storage HKDF: storageKey'));
expect(sv).toBeDefined();
const storageKey = await crypto.hkdf(
fromHex(sv.masterKey),
new Uint8Array(0),
new TextEncoder().encode('shade-storage-v1'),
32,
);
expect(hex(storageKey)).toBe(sv.storageKey);
const fv = vectors.find((v: any) => v.description.startsWith('Storage HKDF: fieldKey'));
expect(fv).toBeDefined();
for (const f of fv.fields) {
const k = await crypto.hkdf(
fromHex(fv.storageKey),
new Uint8Array(0),
new TextEncoder().encode(`shade-field-v1:${f.table}:${f.column}`),
32,
);
expect(hex(k)).toBe(f.fieldKey);
}
const nv = vectors.find((v: any) => v.description.startsWith('Storage HKDF: rowNonce'));
expect(nv).toBeDefined();
for (const n of nv.nonces) {
const out = await crypto.hkdf(
fromHex(nv.rowKey),
new Uint8Array(0),
new TextEncoder().encode(`shade-row-nonce-v1:${n.table}:${n.pk}`),
12,
);
expect(hex(out)).toBe(n.nonce);
}
});
test('Ratchet step: deterministic encrypt + decrypt roundtrip', async () => {
const { vectors } = loadVectors('ratchet-step.json');
for (const v of vectors) {
const rootKey = fromHex(v.inputs.rootKey);
const dhSendPriv = fromHex(v.inputs.dhSendPrivateKey);
const dhSendPub = fromHex(v.inputs.dhSendPublicKey);
const dhRemotePub = fromHex(v.inputs.dhRemotePublicKey);
const plaintext = fromHex(v.inputs.plaintext);
const nonce = fromHex(v.inputs.nonce);
const previousCounter: number = v.inputs.previousCounter;
const counter: number = v.inputs.counter;
// 1. DH
const dhOutput = await crypto.x25519(dhSendPriv, dhRemotePub);
expect(hex(dhOutput)).toBe(v.derived.dhOutput);
// 2. kdfRootKey
const root = await kdfRootKey(crypto, rootKey, dhOutput);
expect(hex(root.newRootKey)).toBe(v.derived.newRootKey);
expect(hex(root.chainKey)).toBe(v.derived.chainKey);
// 3. kdfChainKey
const chain = await kdfChainKey(crypto, root.chainKey);
expect(hex(chain.newChainKey)).toBe(v.derived.newChainKey);
expect(hex(chain.messageKey)).toBe(v.derived.messageKey);
// 4. Header AAD
const aad = encodeRatchetHeader(dhSendPub, previousCounter, counter);
expect(hex(aad)).toBe(v.derived.aad);
// 5. AES-GCM encrypt with fixed nonce
const ciphertext = await aesGcmEncryptDeterministic(chain.messageKey, nonce, plaintext, aad);
expect(hex(ciphertext)).toBe(v.ciphertext);
// 6. Roundtrip decrypt — verify the recorded ciphertext recovers the plaintext
const recovered = await crypto.aesGcmDecrypt(
chain.messageKey,
fromHex(v.ciphertext),
nonce,
aad,
);
expect(hex(recovered)).toBe(v.inputs.plaintext);
}
});
});

View File

@@ -31,6 +31,51 @@ async function setupPair(): Promise<{ alice: SessionState; bob: SessionState }>
describe('Double Ratchet', () => {
// ─── Basic Send/Receive ──────────────────────────────────
describe('initReceiverSession isolation', () => {
/**
* Regression — the V3.10 multi-sender recovery flow surfaced a
* bug where `initReceiverSession` shared a reference to the
* receiver's signed-prekey keypair with the new session. The
* first DH ratchet step zeroed the session's stale send-key
* private bytes — which were the SAME backing buffer as the
* persisted signed prekey. A subsequent X3DH from a different
* sender then derived a divergent root key and decryption
* failed.
*
* Fix: `initReceiverSession` copies the keypair into the
* session. Verify here.
*/
test('does not mutate the caller-provided keypair after a DH ratchet step', async () => {
const rootKey = crypto.randomBytes(32);
const remoteIdentityKey = crypto.randomBytes(32);
const bobDH = await crypto.generateX25519KeyPair();
const originalPrivate = new Uint8Array(bobDH.privateKey);
const originalPublic = new Uint8Array(bobDH.publicKey);
const bob = initReceiverSession(rootKey, remoteIdentityKey, bobDH);
// Drive a full receive that triggers `performDHRatchetStep`
// via a message with a fresh dhPublicKey.
const aliceFirst = await initSenderSession(crypto, rootKey, remoteIdentityKey, bobDH.publicKey);
const msg = await ratchetEncrypt(crypto, aliceFirst, enc.encode('hi'));
await ratchetDecrypt(crypto, bob, msg);
// The ORIGINAL keypair must not have been touched, so a
// second X3DH-style establishment using the same prekey
// material still succeeds.
expect(Array.from(bobDH.privateKey)).toEqual(Array.from(originalPrivate));
expect(Array.from(bobDH.publicKey)).toEqual(Array.from(originalPublic));
// Sanity-check: a second receiver session built from the
// same keypair should still decrypt fresh sender traffic.
const bob2 = initReceiverSession(rootKey, remoteIdentityKey, bobDH);
const aliceSecond = await initSenderSession(crypto, rootKey, remoteIdentityKey, bobDH.publicKey);
const msg2 = await ratchetEncrypt(crypto, aliceSecond, enc.encode('hi again'));
const plain2 = await ratchetDecrypt(crypto, bob2, msg2);
expect(dec.decode(plain2)).toBe('hi again');
});
});
describe('basic send/receive', () => {
test('Alice encrypts, Bob decrypts', async () => {
const { alice, bob } = await setupPair();

View File

@@ -1,12 +1,27 @@
{
"name": "@shade/crypto-web",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"exports": {
".": {
"types": "./src/index.ts",
"default": "./src/index.ts"
},
"./worker": {
"types": "./src/worker.ts",
"default": "./src/worker.ts"
},
"./worker-protocol": {
"types": "./src/worker-protocol.ts",
"default": "./src/worker-protocol.ts"
}
},
"dependencies": {
"@noble/curves": "^2.0.1",
"@noble/hashes": "^2.0.1",
"@shade/core": "workspace:*"
"@shade/core": "workspace:*",
"@shade/streams": "workspace:*"
}
}

View File

@@ -1,2 +1,24 @@
export { SubtleCryptoProvider } from './provider.js';
export { MemoryStorage } from './memory-storage.js';
// ─── Web Workers crypto (V3.8) ────────────────────────────
export {
createWorkerCryptoProvider,
WorkerCryptoProvider,
WorkerStreamSender,
WorkerStreamReceiver,
} from './worker-client.js';
export type {
WorkerCryptoProviderOptions,
WorkerLike,
} from './worker-client.js';
export {
createEncryptStream,
createDecryptStream,
DEFAULT_STREAM_CHUNK_SIZE,
} from './worker-streams.js';
export type {
CreateEncryptStreamOptions,
CreateDecryptStreamOptions,
} from './worker-streams.js';
export { WORKER_PROTOCOL_VERSION } from './worker-protocol.js';

View File

@@ -1,4 +1,4 @@
import type { StorageProvider, IdentityKeyPair, SignedPreKey, OneTimePreKey, SessionState, RetiredIdentity, PersistedStreamState } from '@shade/core';
import type { StorageProvider, IdentityKeyPair, SignedPreKey, OneTimePreKey, SessionState, RetiredIdentity, PersistedStreamState, PeerVerification } from '@shade/core';
import { constantTimeEqual } from '@shade/core';
/**
@@ -104,6 +104,34 @@ export class MemoryStorage implements StorageProvider {
this.retiredIdentities = this.retiredIdentities.filter((r) => r.retiredAt >= olderThan);
}
// ─── Peer verifications (V3.3) ────────────────────────────
private peerVerifications = new Map<string, PeerVerification>();
private peerIdentityVersions = new Map<string, number>();
async savePeerVerification(v: PeerVerification): Promise<void> {
this.peerVerifications.set(v.peerAddress, { ...v });
}
async getPeerVerification(address: string): Promise<PeerVerification | null> {
const v = this.peerVerifications.get(address);
return v ? { ...v } : null;
}
async removePeerVerification(address: string): Promise<void> {
this.peerVerifications.delete(address);
}
async getPeerIdentityVersion(address: string): Promise<number> {
return this.peerIdentityVersions.get(address) ?? 1;
}
async bumpPeerIdentityVersion(address: string): Promise<number> {
const next = (this.peerIdentityVersions.get(address) ?? 1) + 1;
this.peerIdentityVersions.set(address, next);
return next;
}
// ─── Stream-transfer resume state (v0.2.0) ────────────────
private streamStates = new Map<string, PersistedStreamState>();

View File

@@ -0,0 +1,513 @@
import type { CryptoProvider } from '@shade/core';
import {
WORKER_PROTOCOL_VERSION,
fromTransferable,
toTransferableCopy,
type WorkerRequest,
type WorkerResponse,
type WorkerResult,
} from './worker-protocol.js';
/** Distributive omit of `id` from each variant of {@link WorkerRequest}. */
type WorkerRequestBody = WorkerRequest extends infer T
? T extends { id: number }
? Omit<T, 'id'>
: never
: never;
/**
* Minimal Worker shape we depend on. Lets the main-thread proxy work
* against both browser `Worker` and Bun's `Worker` without dragging in
* `lib.dom.d.ts`.
*/
export interface WorkerLike {
postMessage(message: unknown, transfer?: ArrayBuffer[]): void;
addEventListener(
type: 'message',
listener: (ev: { data: WorkerResponse }) => void,
): void;
removeEventListener(
type: 'message',
listener: (ev: { data: WorkerResponse }) => void,
): void;
addEventListener(
type: 'error',
listener: (ev: { message?: string; error?: unknown }) => void,
): void;
terminate(): void;
}
export interface WorkerCryptoProviderOptions {
/**
* URL of the bundled worker entry. Required because every bundler
* resolves worker URLs differently — supply yours and stop guessing.
*
* // Vite / Webpack 5 / Rspack:
* workerUrl: new URL('@shade/crypto-web/worker', import.meta.url)
*/
workerUrl: URL | string;
/**
* How long the worker may stay idle before it self-terminates. Set to
* `Infinity` to opt out (e.g. for SharedArrayBuffer / persistent UI).
* Default: 30_000 ms — matches the V3.8 acceptance criterion.
*/
idleTimeoutMs?: number;
/**
* Override the worker factory. Useful in tests with `bun:test`'s
* `Worker` global, or to inject a polyfill.
*/
spawn?: (url: URL | string) => WorkerLike;
}
/**
* Public factory. Resolves once the worker has acknowledged the protocol
* version handshake — so a stale bundle blows up here rather than in the
* middle of an encrypt call.
*/
export async function createWorkerCryptoProvider(
opts: WorkerCryptoProviderOptions,
): Promise<WorkerCryptoProvider> {
const provider = new WorkerCryptoProvider(opts);
await provider.handshake();
return provider;
}
let SENDER_SEQ = 1;
let RECEIVER_SEQ = 1;
/**
* `CryptoProvider` implementation that forwards every async call to a
* dedicated Web Worker. Sync methods (`randomBytes`, `randomUint32`,
* `constantTimeEqual`, `zeroize`) execute on the calling thread — they
* are pure and instantaneous, so a worker round-trip would be silly.
*
* The worker is spawned on construction (lazy: see `createWorkerCryptoProvider`)
* and terminated automatically after `idleTimeoutMs` of inactivity.
* Subsequent calls re-spawn transparently.
*
* Stream sender/receiver state lives on the worker — the provider
* exposes `createStreamSender` / `createStreamReceiver` factories that
* return main-thread proxies (`WorkerStreamSender` / `WorkerStreamReceiver`).
*/
export class WorkerCryptoProvider implements CryptoProvider {
private worker: WorkerLike | null = null;
private nextRequestId = 1;
private readonly inflight = new Map<
number,
{ resolve: (r: WorkerResult) => void; reject: (e: Error) => void }
>();
private idleTimer: ReturnType<typeof setTimeout> | null = null;
private destroyed = false;
private readonly idleTimeoutMs: number;
constructor(private readonly opts: WorkerCryptoProviderOptions) {
this.idleTimeoutMs = opts.idleTimeoutMs ?? 30_000;
}
// ─── lifecycle ───────────────────────────────────────────
/** Force-spawn + complete the protocol handshake. Idempotent. */
async handshake(): Promise<void> {
await this.ensureWorker();
await this.send({ method: 'init', protocolVersion: WORKER_PROTOCOL_VERSION });
}
/** Permanently terminate the worker. After this, every call rejects. */
async destroy(): Promise<void> {
this.destroyed = true;
this.terminateWorker(new Error('WorkerCryptoProvider destroyed'));
}
/**
* Tear down the current worker and (lazily) spawn a fresh one. Use
* after rotating identity keys so leaked-state worst case is bounded
* by one rotation interval.
*/
async rotate(): Promise<void> {
this.terminateWorker(new Error('WorkerCryptoProvider rotated'));
}
// ─── async CryptoProvider methods ────────────────────────
async generateX25519KeyPair(): Promise<{ publicKey: Uint8Array; privateKey: Uint8Array }> {
const r = await this.send({ method: 'crypto.generateX25519KeyPair' });
if (r.kind !== 'keypair') throw new Error('protocol: expected keypair');
return { publicKey: fromTransferable(r.publicKey), privateKey: fromTransferable(r.privateKey) };
}
async x25519(privateKey: Uint8Array, publicKey: Uint8Array): Promise<Uint8Array> {
const r = await this.send(
{
method: 'crypto.x25519',
privateKey: toTransferableCopy(privateKey),
publicKey: toTransferableCopy(publicKey),
},
// No transferables from caller's owned buffers — we copied above.
[],
);
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
async generateEd25519KeyPair(): Promise<{ publicKey: Uint8Array; privateKey: Uint8Array }> {
const r = await this.send({ method: 'crypto.generateEd25519KeyPair' });
if (r.kind !== 'keypair') throw new Error('protocol: expected keypair');
return { publicKey: fromTransferable(r.publicKey), privateKey: fromTransferable(r.privateKey) };
}
async sign(privateKey: Uint8Array, message: Uint8Array): Promise<Uint8Array> {
const r = await this.send({
method: 'crypto.sign',
privateKey: toTransferableCopy(privateKey),
message: toTransferableCopy(message),
});
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
async verify(
publicKey: Uint8Array,
message: Uint8Array,
signature: Uint8Array,
): Promise<boolean> {
const r = await this.send({
method: 'crypto.verify',
publicKey: toTransferableCopy(publicKey),
message: toTransferableCopy(message),
signature: toTransferableCopy(signature),
});
if (r.kind !== 'verify') throw new Error('protocol: expected verify');
return r.valid;
}
async aesGcmEncrypt(
key: Uint8Array,
plaintext: Uint8Array,
aad?: Uint8Array,
): Promise<{ ciphertext: Uint8Array; nonce: Uint8Array }> {
const r = await this.send({
method: 'crypto.aesGcmEncrypt',
key: toTransferableCopy(key),
plaintext: toTransferableCopy(plaintext),
aad: aad ? toTransferableCopy(aad) : null,
});
if (r.kind !== 'aead-encrypt') throw new Error('protocol: expected aead-encrypt');
return {
ciphertext: fromTransferable(r.ciphertext),
nonce: fromTransferable(r.nonce),
};
}
async aesGcmDecrypt(
key: Uint8Array,
ciphertext: Uint8Array,
nonce: Uint8Array,
aad?: Uint8Array,
): Promise<Uint8Array> {
const r = await this.send({
method: 'crypto.aesGcmDecrypt',
key: toTransferableCopy(key),
ciphertext: toTransferableCopy(ciphertext),
nonce: toTransferableCopy(nonce),
aad: aad ? toTransferableCopy(aad) : null,
});
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
async hkdf(
ikm: Uint8Array,
salt: Uint8Array,
info: Uint8Array,
length: number,
): Promise<Uint8Array> {
const r = await this.send({
method: 'crypto.hkdf',
ikm: toTransferableCopy(ikm),
salt: toTransferableCopy(salt),
info: toTransferableCopy(info),
length,
});
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
async hmacSha256(key: Uint8Array, data: Uint8Array): Promise<Uint8Array> {
const r = await this.send({
method: 'crypto.hmacSha256',
key: toTransferableCopy(key),
data: toTransferableCopy(data),
});
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
// ─── sync — local execution (no round-trip) ──────────────
randomBytes(length: number): Uint8Array {
const buf = new Uint8Array(length);
globalThis.crypto.getRandomValues(buf);
return buf;
}
constantTimeEqual(a: Uint8Array, b: Uint8Array): boolean {
if (a.length !== b.length) return false;
let diff = 0;
for (let i = 0; i < a.length; i++) diff |= a[i]! ^ b[i]!;
return diff === 0;
}
zeroize(buf: Uint8Array): void {
buf.fill(0);
}
randomUint32(): number {
const buf = this.randomBytes(4);
return new DataView(buf.buffer, buf.byteOffset, 4).getUint32(0, false);
}
// ─── stream factories ────────────────────────────────────
async createStreamSender(opts: {
streamId: Uint8Array;
streamSecret: Uint8Array;
laneId: number;
startSeq?: number;
}): Promise<WorkerStreamSender> {
const senderId = SENDER_SEQ++;
await this.send({
method: 'stream.createSender',
senderId,
streamId: toTransferableCopy(opts.streamId),
streamSecret: toTransferableCopy(opts.streamSecret),
laneId: opts.laneId,
startSeq: opts.startSeq ?? 0,
});
return new WorkerStreamSender(this, senderId);
}
async createStreamReceiver(opts: {
streamId: Uint8Array;
streamSecret: Uint8Array;
laneId: number;
startSeq?: number;
}): Promise<WorkerStreamReceiver> {
const receiverId = RECEIVER_SEQ++;
await this.send({
method: 'stream.createReceiver',
receiverId,
streamId: toTransferableCopy(opts.streamId),
streamSecret: toTransferableCopy(opts.streamSecret),
laneId: opts.laneId,
startSeq: opts.startSeq ?? 0,
});
return new WorkerStreamReceiver(this, receiverId);
}
// ─── internals ───────────────────────────────────────────
/** @internal — used by `WorkerStreamSender` / `WorkerStreamReceiver`. */
async send(
body: WorkerRequestBody,
extraTransferables?: ArrayBuffer[],
): Promise<WorkerResult> {
if (this.destroyed) throw new Error('WorkerCryptoProvider destroyed');
await this.ensureWorker();
const id = this.nextRequestId++;
const req = { id, ...body } as WorkerRequest;
// Auto-collect transferable buffers from the request payload — every
// ArrayBuffer-typed field is fair game.
const transferables = extraTransferables ?? collectArrayBuffers(req);
return new Promise<WorkerResult>((resolve, reject) => {
this.inflight.set(id, { resolve, reject });
try {
this.worker!.postMessage(req, transferables);
} catch (err) {
this.inflight.delete(id);
reject(err instanceof Error ? err : new Error(String(err)));
return;
}
this.bumpIdleTimer();
});
}
private async ensureWorker(): Promise<void> {
if (this.destroyed) throw new Error('WorkerCryptoProvider destroyed');
if (this.worker !== null) return;
const spawn = this.opts.spawn ?? defaultSpawn;
const w = spawn(this.opts.workerUrl);
w.addEventListener('message', this.onMessage);
w.addEventListener('error', this.onError);
this.worker = w;
}
private readonly onMessage = (ev: { data: WorkerResponse }): void => {
const res = ev.data;
const pending = this.inflight.get(res.id);
if (pending === undefined) return;
this.inflight.delete(res.id);
if (res.ok) pending.resolve(res.result);
else {
const err = new Error(res.error.message);
err.name = res.error.name;
pending.reject(err);
}
this.bumpIdleTimer();
};
private readonly onError = (ev: { message?: string; error?: unknown }): void => {
const msg = ev.message ?? (ev.error instanceof Error ? ev.error.message : String(ev.error));
this.terminateWorker(new Error(`worker error: ${msg}`));
};
private bumpIdleTimer(): void {
if (this.idleTimer !== null) clearTimeout(this.idleTimer);
if (!isFinite(this.idleTimeoutMs)) return;
if (this.inflight.size > 0) return;
this.idleTimer = setTimeout(() => {
// No outstanding work — recycle the worker. Calls after this
// re-spawn lazily.
this.terminateWorker(null);
}, this.idleTimeoutMs);
// Don't keep node-style event loops alive solely on this timer.
const t = this.idleTimer as unknown as { unref?: () => void };
if (typeof t.unref === 'function') t.unref();
}
private terminateWorker(reason: Error | null): void {
if (this.idleTimer !== null) {
clearTimeout(this.idleTimer);
this.idleTimer = null;
}
const w = this.worker;
this.worker = null;
// Reject every in-flight request so callers don't hang.
if (reason !== null) {
for (const [, pending] of this.inflight) pending.reject(reason);
}
this.inflight.clear();
if (w !== null) {
try {
w.removeEventListener('message', this.onMessage);
} catch {
// ignore
}
try {
w.terminate();
} catch {
// ignore
}
}
}
}
/**
* Walk the request body, collecting every `ArrayBuffer` reference so we
* can hand them to `postMessage(_, transfer)`. Cheap because the request
* objects are flat — at most a handful of fields.
*/
function collectArrayBuffers(req: WorkerRequest): ArrayBuffer[] {
const out: ArrayBuffer[] = [];
for (const v of Object.values(req as Record<string, unknown>)) {
if (v instanceof ArrayBuffer) out.push(v);
}
return out;
}
function defaultSpawn(url: URL | string): WorkerLike {
const ctor = (globalThis as unknown as { Worker?: new (u: URL | string, o?: unknown) => unknown })
.Worker;
if (typeof ctor !== 'function') {
throw new Error('Worker is not available in this runtime');
}
return new ctor(url, { type: 'module' }) as WorkerLike;
}
/**
* Main-thread handle on a `StreamSender` that lives in the worker. The
* lane key never crosses thread boundaries — this proxy only ever ships
* plaintext slices and receives wire bytes.
*/
export class WorkerStreamSender {
private destroyed = false;
constructor(
private readonly provider: WorkerCryptoProvider,
private readonly senderId: number,
) {}
async encryptChunk(
plaintext: Uint8Array,
isLast: boolean,
): Promise<{ bytes: Uint8Array; seq: number }> {
if (this.destroyed) throw new Error('WorkerStreamSender destroyed');
const r = await this.provider.send({
method: 'stream.encryptChunk',
senderId: this.senderId,
plaintext: toTransferableCopy(plaintext),
isLast,
});
if (r.kind !== 'chunk-encrypt') throw new Error('protocol: expected chunk-encrypt');
return { bytes: fromTransferable(r.bytes), seq: r.seq };
}
async getLaneSha256(): Promise<Uint8Array> {
const r = await this.provider.send({
method: 'stream.getSenderLaneSha256',
senderId: this.senderId,
});
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
async destroy(): Promise<void> {
if (this.destroyed) return;
this.destroyed = true;
await this.provider.send({ method: 'stream.destroySender', senderId: this.senderId });
}
}
export class WorkerStreamReceiver {
private destroyed = false;
constructor(
private readonly provider: WorkerCryptoProvider,
private readonly receiverId: number,
) {}
async decryptChunk(
wireBytes: Uint8Array,
): Promise<{ plaintext: Uint8Array; seq: number; isLast: boolean }> {
if (this.destroyed) throw new Error('WorkerStreamReceiver destroyed');
const r = await this.provider.send({
method: 'stream.decryptChunk',
receiverId: this.receiverId,
wireBytes: toTransferableCopy(wireBytes),
});
if (r.kind !== 'chunk-decrypt') throw new Error('protocol: expected chunk-decrypt');
return {
plaintext: fromTransferable(r.plaintext),
seq: r.seq,
isLast: r.isLast,
};
}
async getLaneSha256(): Promise<Uint8Array> {
const r = await this.provider.send({
method: 'stream.getReceiverLaneSha256',
receiverId: this.receiverId,
});
if (r.kind !== 'bytes') throw new Error('protocol: expected bytes');
return fromTransferable(r.bytes);
}
async destroy(): Promise<void> {
if (this.destroyed) return;
this.destroyed = true;
await this.provider.send({ method: 'stream.destroyReceiver', receiverId: this.receiverId });
}
}

View File

@@ -0,0 +1,165 @@
/**
* Wire protocol between `WorkerCryptoProvider` (main thread) and the
* worker entry (`worker.ts`).
*
* Shape:
* main → worker: WorkerRequest
* worker → main: WorkerResponse (matched by `id`)
*
* Binary inputs are passed as `ArrayBuffer` (transferable) — never
* `Uint8Array` — so we can hand them off without copying. The worker
* wraps them in `Uint8Array` for use, and the response transfers result
* buffers the same way.
*
* Versioning: `__protocolVersion` is bumped on any breaking change.
* `worker-init` echoes the version so a mismatched bundle aborts
* deterministically instead of silently producing garbage.
*/
export const WORKER_PROTOCOL_VERSION = 1;
export type WorkerRequest =
// ─── lifecycle ────────────────────────────────────────────
| { id: number; method: 'init'; protocolVersion: number }
| { id: number; method: 'ping' }
// ─── crypto.* — generic CryptoProvider ────────────────────
| { id: number; method: 'crypto.generateX25519KeyPair' }
| { id: number; method: 'crypto.x25519'; privateKey: ArrayBuffer; publicKey: ArrayBuffer }
| { id: number; method: 'crypto.generateEd25519KeyPair' }
| { id: number; method: 'crypto.sign'; privateKey: ArrayBuffer; message: ArrayBuffer }
| {
id: number;
method: 'crypto.verify';
publicKey: ArrayBuffer;
message: ArrayBuffer;
signature: ArrayBuffer;
}
| {
id: number;
method: 'crypto.aesGcmEncrypt';
key: ArrayBuffer;
plaintext: ArrayBuffer;
aad: ArrayBuffer | null;
}
| {
id: number;
method: 'crypto.aesGcmDecrypt';
key: ArrayBuffer;
ciphertext: ArrayBuffer;
nonce: ArrayBuffer;
aad: ArrayBuffer | null;
}
| {
id: number;
method: 'crypto.hkdf';
ikm: ArrayBuffer;
salt: ArrayBuffer;
info: ArrayBuffer;
length: number;
}
| { id: number; method: 'crypto.hmacSha256'; key: ArrayBuffer; data: ArrayBuffer }
// ─── stream.* — host StreamSender / StreamReceiver ────────
| {
id: number;
method: 'stream.createSender';
senderId: number;
streamId: ArrayBuffer;
streamSecret: ArrayBuffer;
laneId: number;
startSeq: number;
}
| {
id: number;
method: 'stream.encryptChunk';
senderId: number;
plaintext: ArrayBuffer;
isLast: boolean;
}
| { id: number; method: 'stream.getSenderLaneSha256'; senderId: number }
| { id: number; method: 'stream.destroySender'; senderId: number }
| {
id: number;
method: 'stream.createReceiver';
receiverId: number;
streamId: ArrayBuffer;
streamSecret: ArrayBuffer;
laneId: number;
startSeq: number;
}
| {
id: number;
method: 'stream.decryptChunk';
receiverId: number;
wireBytes: ArrayBuffer;
}
| { id: number; method: 'stream.getReceiverLaneSha256'; receiverId: number }
| { id: number; method: 'stream.destroyReceiver'; receiverId: number };
export type WorkerResponse =
| { id: number; ok: true; result: WorkerResult }
| {
id: number;
ok: false;
error: { name: string; message: string; code?: string };
};
export type WorkerResult =
| { kind: 'ack' } // void/init/destroy
| { kind: 'pong' }
| { kind: 'keypair'; publicKey: ArrayBuffer; privateKey: ArrayBuffer }
| { kind: 'bytes'; bytes: ArrayBuffer }
| { kind: 'aead-encrypt'; ciphertext: ArrayBuffer; nonce: ArrayBuffer }
| { kind: 'verify'; valid: boolean }
| { kind: 'chunk-encrypt'; bytes: ArrayBuffer; seq: number }
| { kind: 'chunk-decrypt'; plaintext: ArrayBuffer; seq: number; isLast: boolean };
/**
* Pull every transferable `ArrayBuffer` out of a result so the runtime
* can hand ownership to the receiving thread. Order doesn't matter; the
* structured-clone algorithm matches buffers by reference.
*/
export function transferablesOf(result: WorkerResult): ArrayBuffer[] {
switch (result.kind) {
case 'keypair':
return [result.publicKey, result.privateKey];
case 'bytes':
return [result.bytes];
case 'aead-encrypt':
return [result.ciphertext, result.nonce];
case 'chunk-encrypt':
return [result.bytes];
case 'chunk-decrypt':
return [result.plaintext];
default:
return [];
}
}
/**
* Wrap a `Uint8Array` as an `ArrayBuffer` suitable for transfer. If the
* view doesn't span its underlying buffer, copy into a fresh one so we
* never transfer slack the caller still owns.
*/
export function toTransferable(u: Uint8Array): ArrayBuffer {
if (u.byteOffset === 0 && u.byteLength === u.buffer.byteLength) {
return u.buffer as ArrayBuffer;
}
const copy = new Uint8Array(u.byteLength);
copy.set(u);
return copy.buffer;
}
/**
* Like `toTransferable`, but always copies. Use when the original buffer
* must remain valid on the calling thread (e.g. when the caller owns a
* key the worker should not mutate).
*/
export function toTransferableCopy(u: Uint8Array): ArrayBuffer {
const copy = new Uint8Array(u.byteLength);
copy.set(u);
return copy.buffer;
}
export function fromTransferable(buf: ArrayBuffer): Uint8Array {
return new Uint8Array(buf);
}

View File

@@ -0,0 +1,217 @@
import type {
WorkerCryptoProvider,
WorkerStreamReceiver,
WorkerStreamSender,
} from './worker-client.js';
/** Default plaintext chunk size — 256 KiB. Matches `@shade/transfer`. */
export const DEFAULT_STREAM_CHUNK_SIZE = 256 * 1024;
export interface CreateEncryptStreamOptions {
provider: WorkerCryptoProvider;
streamId: Uint8Array;
streamSecret: Uint8Array;
laneId?: number;
/**
* Plaintext bytes per AEAD chunk. Smaller = lower latency per chunk +
* more postMessage overhead; larger = higher per-chunk RAM in the
* worker. Default 256 KiB.
*/
chunkSize?: number;
/**
* First sequence number this sender will emit. Default 0.
* Use for resume.
*/
startSeq?: number;
/** First seq this receiver will accept; defaults to 0. */
signal?: AbortSignal;
}
export interface CreateDecryptStreamOptions {
provider: WorkerCryptoProvider;
streamId: Uint8Array;
streamSecret: Uint8Array;
laneId?: number;
startSeq?: number;
signal?: AbortSignal;
}
/**
* Build a `TransformStream<Uint8Array, Uint8Array>` that encrypts every
* passing byte as a stream-chunk wire envelope. The actual AEAD work
* happens in the worker — the main thread only buffers, slices, and
* forwards.
*
* Output: one wire chunk per `enqueue`. Concatenation is the responsibility
* of the downstream consumer (typically an HTTP-shipping `TransformStream`).
*/
export function createEncryptStream(opts: CreateEncryptStreamOptions): {
stream: TransformStream<Uint8Array, Uint8Array>;
/** Promise that resolves to the final lane sha256 once the stream finishes. */
laneSha256: Promise<Uint8Array>;
} {
const chunkSize = opts.chunkSize ?? DEFAULT_STREAM_CHUNK_SIZE;
if (chunkSize <= 0) throw new Error('chunkSize must be positive');
// Plaintext slices accumulate here until we have at least `chunkSize`
// bytes (so we emit fixed-size chunks except for the very last one).
let pending: Uint8Array = new Uint8Array(0);
let sender: WorkerStreamSender | null = null;
let resolveLaneSha: (b: Uint8Array) => void;
let rejectLaneSha: (e: Error) => void;
const laneSha256 = new Promise<Uint8Array>((res, rej) => {
resolveLaneSha = res;
rejectLaneSha = rej;
});
// Cast to `Transformer<I,O>` because some TS lib versions still ship
// the pre-2023 shape without `cancel`. Runtime supports it (Bun, all
// modern browsers).
const transformer = {
async start(): Promise<void> {
sender = await opts.provider.createStreamSender({
streamId: opts.streamId,
streamSecret: opts.streamSecret,
laneId: opts.laneId ?? 0,
startSeq: opts.startSeq ?? 0,
});
},
async transform(
chunk: Uint8Array,
controller: TransformStreamDefaultController<Uint8Array>,
): Promise<void> {
if (sender === null) throw new Error('encryptStream: sender not initialized');
if (chunk.byteLength === 0) return;
pending = concat(pending, chunk);
// Emit complete chunks. Hold back the trailing partial — we don't
// know yet whether it's the last one (which gets isLast=true).
while (pending.byteLength >= chunkSize) {
const slice = pending.subarray(0, chunkSize);
const rest = pending.subarray(chunkSize);
const out = await sender.encryptChunk(slice, false);
controller.enqueue(out.bytes);
// Detach `rest` from the larger backing buffer so it can be GCed.
pending = new Uint8Array(rest);
}
},
async flush(controller: TransformStreamDefaultController<Uint8Array>): Promise<void> {
if (sender === null) throw new Error('encryptStream: sender not initialized');
try {
// Always emit a final chunk with isLast=true. Even if `pending`
// is empty: receivers rely on a trailing isLast envelope to
// mark stream completion.
const out = await sender.encryptChunk(pending, true);
controller.enqueue(out.bytes);
pending = new Uint8Array(0);
const sha = await sender.getLaneSha256();
resolveLaneSha(sha);
} catch (err) {
rejectLaneSha(err instanceof Error ? err : new Error(String(err)));
throw err;
} finally {
await sender.destroy();
sender = null;
}
},
async cancel(reason: unknown): Promise<void> {
try {
if (sender !== null) await sender.destroy();
} finally {
sender = null;
rejectLaneSha(reason instanceof Error ? reason : new Error(String(reason)));
}
},
};
const stream = new TransformStream<Uint8Array, Uint8Array>(
transformer as unknown as Transformer<Uint8Array, Uint8Array>,
);
if (opts.signal) {
const abort = (): void => {
stream.writable.abort(opts.signal!.reason).catch(() => {});
};
if (opts.signal.aborted) abort();
else opts.signal.addEventListener('abort', abort, { once: true });
}
return { stream, laneSha256 };
}
/**
* Build a `TransformStream<Uint8Array, Uint8Array>` that decrypts wire
* stream-chunk envelopes back into plaintext. The input chunks must be
* complete envelopes — the caller is responsible for framing on the wire
* (one envelope per write).
*/
export function createDecryptStream(opts: CreateDecryptStreamOptions): {
stream: TransformStream<Uint8Array, Uint8Array>;
/** Promise that resolves to the final lane sha256 once decryption finishes. */
laneSha256: Promise<Uint8Array>;
} {
let receiver: WorkerStreamReceiver | null = null;
let resolveLaneSha: (b: Uint8Array) => void;
let rejectLaneSha: (e: Error) => void;
const laneSha256 = new Promise<Uint8Array>((res, rej) => {
resolveLaneSha = res;
rejectLaneSha = rej;
});
const transformer = {
async start(): Promise<void> {
receiver = await opts.provider.createStreamReceiver({
streamId: opts.streamId,
streamSecret: opts.streamSecret,
laneId: opts.laneId ?? 0,
startSeq: opts.startSeq ?? 0,
});
},
async transform(
chunk: Uint8Array,
controller: TransformStreamDefaultController<Uint8Array>,
): Promise<void> {
if (receiver === null) throw new Error('decryptStream: receiver not initialized');
const dec = await receiver.decryptChunk(chunk);
if (dec.plaintext.byteLength > 0) controller.enqueue(dec.plaintext);
if (dec.isLast) {
const sha = await receiver.getLaneSha256();
resolveLaneSha(sha);
}
},
async flush(): Promise<void> {
if (receiver !== null) await receiver.destroy();
receiver = null;
},
async cancel(reason: unknown): Promise<void> {
try {
if (receiver !== null) await receiver.destroy();
} finally {
receiver = null;
rejectLaneSha(reason instanceof Error ? reason : new Error(String(reason)));
}
},
};
const stream = new TransformStream<Uint8Array, Uint8Array>(
transformer as unknown as Transformer<Uint8Array, Uint8Array>,
);
if (opts.signal) {
const abort = (): void => {
stream.writable.abort(opts.signal!.reason).catch(() => {});
};
if (opts.signal.aborted) abort();
else opts.signal.addEventListener('abort', abort, { once: true });
}
return { stream, laneSha256 };
}
function concat(a: Uint8Array, b: Uint8Array): Uint8Array {
if (a.byteLength === 0) return b;
if (b.byteLength === 0) return a;
const out = new Uint8Array(a.byteLength + b.byteLength);
out.set(a, 0);
out.set(b, a.byteLength);
return out;
}

View File

@@ -0,0 +1,231 @@
/**
* Dedicated Web Worker entry. Bundle this as a module worker:
*
* // Vite / modern Webpack / Rspack
* const w = new Worker(new URL('@shade/crypto-web/worker', import.meta.url),
* { type: 'module' });
*
* The main thread talks to this worker via the protocol in
* `worker-protocol.ts`. All heavy crypto (AES-GCM, HKDF, HMAC, X25519,
* Ed25519) and stream state (per-lane keys + seq counters + running
* sha256) live here so the main thread is never blocked.
*
* Lifecycle: every request is one `postMessage` round-trip. Sender /
* receiver state is keyed by numeric ids handed in by the main thread —
* the worker never invents ids. `destroy*` calls zeroize lane keys.
*/
import { StreamSender, StreamReceiver } from '@shade/streams';
import { SubtleCryptoProvider } from './provider.js';
import {
WORKER_PROTOCOL_VERSION,
fromTransferable,
toTransferable,
transferablesOf,
type WorkerRequest,
type WorkerResponse,
type WorkerResult,
} from './worker-protocol.js';
interface DedicatedWorkerScope {
postMessage(data: unknown, transfer?: ArrayBuffer[]): void;
addEventListener(
type: 'message',
listener: (ev: { data: WorkerRequest }) => void,
): void;
}
const scope = globalThis as unknown as DedicatedWorkerScope;
const provider = new SubtleCryptoProvider();
const senders = new Map<number, StreamSender>();
const receivers = new Map<number, StreamReceiver>();
scope.addEventListener('message', (ev) => {
void handle(ev.data);
});
async function handle(req: WorkerRequest): Promise<void> {
try {
const result = await dispatch(req);
const transfer = transferablesOf(result);
const res: WorkerResponse = { id: req.id, ok: true, result };
scope.postMessage(res, transfer);
} catch (err) {
const error = err instanceof Error
? { name: err.name, message: err.message }
: { name: 'Error', message: String(err) };
const res: WorkerResponse = { id: req.id, ok: false, error };
scope.postMessage(res);
}
}
async function dispatch(req: WorkerRequest): Promise<WorkerResult> {
switch (req.method) {
case 'init': {
if (req.protocolVersion !== WORKER_PROTOCOL_VERSION) {
throw new Error(
`worker protocol version mismatch: main=${req.protocolVersion} worker=${WORKER_PROTOCOL_VERSION}`,
);
}
return { kind: 'ack' };
}
case 'ping':
return { kind: 'pong' };
// ─── crypto.* ─────────────────────────────────────────
case 'crypto.generateX25519KeyPair': {
const kp = await provider.generateX25519KeyPair();
return {
kind: 'keypair',
publicKey: toTransferable(kp.publicKey),
privateKey: toTransferable(kp.privateKey),
};
}
case 'crypto.x25519': {
const out = await provider.x25519(
fromTransferable(req.privateKey),
fromTransferable(req.publicKey),
);
return { kind: 'bytes', bytes: toTransferable(out) };
}
case 'crypto.generateEd25519KeyPair': {
const kp = await provider.generateEd25519KeyPair();
return {
kind: 'keypair',
publicKey: toTransferable(kp.publicKey),
privateKey: toTransferable(kp.privateKey),
};
}
case 'crypto.sign': {
const sig = await provider.sign(
fromTransferable(req.privateKey),
fromTransferable(req.message),
);
return { kind: 'bytes', bytes: toTransferable(sig) };
}
case 'crypto.verify': {
const valid = await provider.verify(
fromTransferable(req.publicKey),
fromTransferable(req.message),
fromTransferable(req.signature),
);
return { kind: 'verify', valid };
}
case 'crypto.aesGcmEncrypt': {
const out = await provider.aesGcmEncrypt(
fromTransferable(req.key),
fromTransferable(req.plaintext),
req.aad ? fromTransferable(req.aad) : undefined,
);
return {
kind: 'aead-encrypt',
ciphertext: toTransferable(out.ciphertext),
nonce: toTransferable(out.nonce),
};
}
case 'crypto.aesGcmDecrypt': {
const out = await provider.aesGcmDecrypt(
fromTransferable(req.key),
fromTransferable(req.ciphertext),
fromTransferable(req.nonce),
req.aad ? fromTransferable(req.aad) : undefined,
);
return { kind: 'bytes', bytes: toTransferable(out) };
}
case 'crypto.hkdf': {
const out = await provider.hkdf(
fromTransferable(req.ikm),
fromTransferable(req.salt),
fromTransferable(req.info),
req.length,
);
return { kind: 'bytes', bytes: toTransferable(out) };
}
case 'crypto.hmacSha256': {
const out = await provider.hmacSha256(
fromTransferable(req.key),
fromTransferable(req.data),
);
return { kind: 'bytes', bytes: toTransferable(out) };
}
// ─── stream.* (sender) ────────────────────────────────
case 'stream.createSender': {
if (senders.has(req.senderId)) {
throw new Error(`senderId ${req.senderId} already exists`);
}
const sender = await StreamSender.create({
crypto: provider,
streamId: fromTransferable(req.streamId),
streamSecret: fromTransferable(req.streamSecret),
laneId: req.laneId,
startSeq: req.startSeq,
});
senders.set(req.senderId, sender);
return { kind: 'ack' };
}
case 'stream.encryptChunk': {
const sender = senders.get(req.senderId);
if (sender === undefined) throw new Error(`unknown senderId ${req.senderId}`);
const chunk = await sender.encryptChunk(fromTransferable(req.plaintext), req.isLast);
return {
kind: 'chunk-encrypt',
bytes: toTransferable(chunk.bytes),
seq: chunk.seq,
};
}
case 'stream.getSenderLaneSha256': {
const sender = senders.get(req.senderId);
if (sender === undefined) throw new Error(`unknown senderId ${req.senderId}`);
return { kind: 'bytes', bytes: toTransferable(sender.getLaneSha256Digest()) };
}
case 'stream.destroySender': {
const sender = senders.get(req.senderId);
if (sender !== undefined) {
sender.destroy();
senders.delete(req.senderId);
}
return { kind: 'ack' };
}
// ─── stream.* (receiver) ──────────────────────────────
case 'stream.createReceiver': {
if (receivers.has(req.receiverId)) {
throw new Error(`receiverId ${req.receiverId} already exists`);
}
const receiver = await StreamReceiver.create({
crypto: provider,
streamId: fromTransferable(req.streamId),
streamSecret: fromTransferable(req.streamSecret),
laneId: req.laneId,
startSeq: req.startSeq,
});
receivers.set(req.receiverId, receiver);
return { kind: 'ack' };
}
case 'stream.decryptChunk': {
const receiver = receivers.get(req.receiverId);
if (receiver === undefined) throw new Error(`unknown receiverId ${req.receiverId}`);
const dec = await receiver.decryptChunk(fromTransferable(req.wireBytes));
return {
kind: 'chunk-decrypt',
plaintext: toTransferable(dec.plaintext),
seq: dec.seq,
isLast: dec.isLast,
};
}
case 'stream.getReceiverLaneSha256': {
const receiver = receivers.get(req.receiverId);
if (receiver === undefined) throw new Error(`unknown receiverId ${req.receiverId}`);
return { kind: 'bytes', bytes: toTransferable(receiver.getLaneSha256Digest()) };
}
case 'stream.destroyReceiver': {
const receiver = receivers.get(req.receiverId);
if (receiver !== undefined) {
receiver.destroy();
receivers.delete(req.receiverId);
}
return { kind: 'ack' };
}
}
}

View File

@@ -0,0 +1,218 @@
import { describe, expect, test, afterEach } from 'bun:test';
import {
createWorkerCryptoProvider,
SubtleCryptoProvider,
WorkerCryptoProvider,
} from '../src/index.js';
const WORKER_URL = new URL('../src/worker.ts', import.meta.url);
const subtle = new SubtleCryptoProvider();
let provider: WorkerCryptoProvider | null = null;
afterEach(async () => {
if (provider) {
await provider.destroy();
provider = null;
}
});
async function makeProvider(idleTimeoutMs = 30_000): Promise<WorkerCryptoProvider> {
provider = await createWorkerCryptoProvider({
workerUrl: WORKER_URL,
idleTimeoutMs,
});
return provider;
}
describe('WorkerCryptoProvider — roundtrip and parity', () => {
test('handshake completes', async () => {
const p = await makeProvider();
expect(p).toBeInstanceOf(WorkerCryptoProvider);
});
test('AES-GCM encrypt → worker, decrypt locally — produces same plaintext', async () => {
const p = await makeProvider();
const key = subtle.randomBytes(32);
const plaintext = new TextEncoder().encode('hello shade workers — large enough payload');
const enc = await p.aesGcmEncrypt(key, plaintext);
expect(enc.nonce.length).toBe(12);
// Decrypt with the local SubtleCryptoProvider — proves wire compatibility
const dec = await subtle.aesGcmDecrypt(key, enc.ciphertext, enc.nonce);
expect(dec).toEqual(plaintext);
});
test('AES-GCM with AAD round-trips through worker', async () => {
const p = await makeProvider();
const key = subtle.randomBytes(32);
const plaintext = subtle.randomBytes(1024);
const aad = subtle.randomBytes(16);
const enc = await p.aesGcmEncrypt(key, plaintext, aad);
const dec = await p.aesGcmDecrypt(key, enc.ciphertext, enc.nonce, aad);
expect(dec).toEqual(plaintext);
});
test('AES-GCM decrypt rejects tampered ciphertext', async () => {
const p = await makeProvider();
const key = subtle.randomBytes(32);
const plaintext = new TextEncoder().encode('untampered');
const enc = await p.aesGcmEncrypt(key, plaintext);
enc.ciphertext[0]! ^= 0x01;
await expect(p.aesGcmDecrypt(key, enc.ciphertext, enc.nonce)).rejects.toThrow();
});
test('HKDF parity with SubtleCryptoProvider', async () => {
const p = await makeProvider();
const ikm = subtle.randomBytes(32);
const salt = subtle.randomBytes(16);
const info = new TextEncoder().encode('test info');
const a = await p.hkdf(ikm, salt, info, 64);
const b = await subtle.hkdf(ikm, salt, info, 64);
expect(a).toEqual(b);
});
test('HMAC-SHA256 parity with SubtleCryptoProvider', async () => {
const p = await makeProvider();
const key = subtle.randomBytes(32);
const data = subtle.randomBytes(256);
const a = await p.hmacSha256(key, data);
const b = await subtle.hmacSha256(key, data);
expect(a).toEqual(b);
});
test('X25519 DH agrees with SubtleCryptoProvider', async () => {
const p = await makeProvider();
const alice = await p.generateX25519KeyPair();
const bob = await subtle.generateX25519KeyPair();
const ab = await p.x25519(alice.privateKey, bob.publicKey);
const ba = await subtle.x25519(bob.privateKey, alice.publicKey);
expect(ab).toEqual(ba);
});
test('Ed25519 sign in worker, verify locally', async () => {
const p = await makeProvider();
const kp = await p.generateEd25519KeyPair();
const msg = new TextEncoder().encode('please sign me');
const sig = await p.sign(kp.privateKey, msg);
expect(await subtle.verify(kp.publicKey, msg, sig)).toBe(true);
});
test('Ed25519 verify rejects tampered signature', async () => {
const p = await makeProvider();
const kp = await subtle.generateEd25519KeyPair();
const msg = new TextEncoder().encode('msg');
const sig = await subtle.sign(kp.privateKey, msg);
sig[0]! ^= 0x01;
expect(await p.verify(kp.publicKey, msg, sig)).toBe(false);
});
test('local sync helpers do not round-trip', async () => {
const p = await makeProvider();
const a = p.randomBytes(16);
expect(a.length).toBe(16);
expect(p.constantTimeEqual(a, a)).toBe(true);
expect(p.constantTimeEqual(a, new Uint8Array(16))).toBe(false);
expect(typeof p.randomUint32()).toBe('number');
});
test('errors from worker propagate as rejected promises', async () => {
const p = await makeProvider();
const wrongKey = subtle.randomBytes(32);
const ct = subtle.randomBytes(48);
const nonce = subtle.randomBytes(12);
await expect(p.aesGcmDecrypt(wrongKey, ct, nonce)).rejects.toThrow();
});
test('parallel calls do not interleave incorrectly', async () => {
const p = await makeProvider();
const key = subtle.randomBytes(32);
const inputs = Array.from({ length: 16 }, (_, i) =>
new TextEncoder().encode(`payload-${i}-${'x'.repeat(50 * i)}`),
);
const encs = await Promise.all(inputs.map((pt) => p.aesGcmEncrypt(key, pt)));
const decs = await Promise.all(
encs.map((e) => p.aesGcmDecrypt(key, e.ciphertext, e.nonce)),
);
decs.forEach((d, i) => expect(d).toEqual(inputs[i]!));
});
test('after destroy(), calls reject', async () => {
const p = await makeProvider();
await p.destroy();
await expect(p.aesGcmEncrypt(subtle.randomBytes(32), new Uint8Array(8))).rejects.toThrow(
/destroyed/,
);
provider = null;
});
test('rotate() respawns transparently', async () => {
const p = await makeProvider();
const key = subtle.randomBytes(32);
await p.aesGcmEncrypt(key, new Uint8Array(8));
await p.rotate();
const out = await p.aesGcmEncrypt(key, new TextEncoder().encode('still works'));
const dec = await subtle.aesGcmDecrypt(key, out.ciphertext, out.nonce);
expect(new TextDecoder().decode(dec)).toBe('still works');
});
test('idle-timeout terminates worker but next call respawns', async () => {
const p = await makeProvider(120);
const key = subtle.randomBytes(32);
await p.aesGcmEncrypt(key, new Uint8Array(8));
// Wait for the idle timer to fire.
await new Promise((r) => setTimeout(r, 250));
// Next call should still succeed — proves respawn works.
const out = await p.aesGcmEncrypt(key, new TextEncoder().encode('respawned'));
const dec = await subtle.aesGcmDecrypt(key, out.ciphertext, out.nonce);
expect(new TextDecoder().decode(dec)).toBe('respawned');
});
test('configureWorkerCrypto throws on protocol mismatch', async () => {
// Spawn with a fake "spawn" that returns a worker echoing the wrong version.
const fakeProvider = new WorkerCryptoProvider({
workerUrl: WORKER_URL,
spawn: () => {
type Listener = (ev: { data: unknown }) => void;
const listeners: Listener[] = [];
return {
postMessage(msg: unknown): void {
const m = msg as { id: number; method: string };
if (m.method === 'init') {
setTimeout(() => {
for (const l of listeners) {
l({
data: {
id: m.id,
ok: false,
error: { name: 'Error', message: 'protocol version mismatch' },
},
});
}
}, 0);
}
},
addEventListener(type: string, listener: Listener): void {
if (type === 'message') listeners.push(listener);
},
removeEventListener(): void {
// no-op
},
terminate(): void {
// no-op
},
};
},
});
await expect(fakeProvider.handshake()).rejects.toThrow(/protocol/);
await fakeProvider.destroy();
});
});

View File

@@ -0,0 +1,230 @@
import { afterEach, describe, expect, test } from 'bun:test';
import { sha256 } from '@noble/hashes/sha2.js';
import {
createDecryptStream,
createEncryptStream,
createWorkerCryptoProvider,
SubtleCryptoProvider,
WorkerCryptoProvider,
} from '../src/index.js';
const WORKER_URL = new URL('../src/worker.ts', import.meta.url);
const subtle = new SubtleCryptoProvider();
let provider: WorkerCryptoProvider | null = null;
afterEach(async () => {
if (provider) {
await provider.destroy();
provider = null;
}
});
async function makeProvider(): Promise<WorkerCryptoProvider> {
provider = await createWorkerCryptoProvider({ workerUrl: WORKER_URL });
return provider;
}
async function readAll(rs: ReadableStream<Uint8Array>): Promise<Uint8Array> {
const reader = rs.getReader();
const parts: Uint8Array[] = [];
let total = 0;
for (;;) {
const { done, value } = await reader.read();
if (done) break;
parts.push(value);
total += value.byteLength;
}
const out = new Uint8Array(total);
let off = 0;
for (const p of parts) {
out.set(p, off);
off += p.byteLength;
}
return out;
}
function streamFromChunks(chunks: Uint8Array[]): ReadableStream<Uint8Array> {
let i = 0;
return new ReadableStream<Uint8Array>({
pull(controller) {
if (i < chunks.length) controller.enqueue(chunks[i++]!);
else controller.close();
},
});
}
describe('encryptStream / decryptStream — round-trip', () => {
test('round-trips small payload exactly', async () => {
const p = await makeProvider();
const streamId = subtle.randomBytes(16);
const streamSecret = subtle.randomBytes(32);
const plaintext = new TextEncoder().encode('hello stream');
const enc = await createEncryptStream({
provider: p,
streamId,
streamSecret,
chunkSize: 1024,
});
const wireBytes = await readAll(
streamFromChunks([plaintext]).pipeThrough(enc.stream),
);
// Frame: each enqueue is one wire envelope. We can't trivially split
// a concatenated buffer back into envelopes, but we know how many
// chunks were emitted (len/chunkSize, plus the final isLast). Easier
// path: collect them as separate writes through a side channel.
const chunks: Uint8Array[] = [];
await streamFromChunks([plaintext])
.pipeThrough(
(
await createEncryptStream({
provider: p,
streamId,
streamSecret,
chunkSize: 1024,
})
).stream,
)
.pipeTo(
new WritableStream<Uint8Array>({
write(c) {
chunks.push(c);
},
}),
);
const dec = await createDecryptStream({ provider: p, streamId, streamSecret });
const recovered = await readAll(streamFromChunks(chunks).pipeThrough(dec.stream));
expect(recovered).toEqual(plaintext);
expect(wireBytes.byteLength).toBeGreaterThan(plaintext.byteLength); // overhead
});
test('round-trips multi-chunk payload with sha256 parity', async () => {
const p = await makeProvider();
const streamId = subtle.randomBytes(16);
const streamSecret = subtle.randomBytes(32);
const total = 750 * 1024; // 750 KiB → forces 3+ chunks at 256 KiB
const plaintext = subtle.randomBytes(total);
const expectedSha = sha256(plaintext);
const enc = await createEncryptStream({
provider: p,
streamId,
streamSecret,
chunkSize: 256 * 1024,
});
const wireChunks: Uint8Array[] = [];
await streamFromChunks([plaintext])
.pipeThrough(enc.stream)
.pipeTo(
new WritableStream<Uint8Array>({
write(c) {
wireChunks.push(c);
},
}),
);
// 750 KiB / 256 KiB = 2 full chunks + 1 final (238 KiB, isLast=true)
expect(wireChunks.length).toBe(3);
const senderLaneSha = await enc.laneSha256;
expect(senderLaneSha).toEqual(expectedSha);
const dec = await createDecryptStream({
provider: p,
streamId,
streamSecret,
});
const recovered = await readAll(streamFromChunks(wireChunks).pipeThrough(dec.stream));
expect(recovered).toEqual(plaintext);
expect(await dec.laneSha256).toEqual(expectedSha);
});
test('fragmented input produces same output as single-shot', async () => {
const p = await makeProvider();
const streamId = subtle.randomBytes(16);
const streamSecret = subtle.randomBytes(32);
const plaintext = subtle.randomBytes(50_000);
async function run(parts: Uint8Array[]): Promise<Uint8Array[]> {
const wire: Uint8Array[] = [];
const e = await createEncryptStream({
provider: p!,
streamId,
streamSecret,
chunkSize: 8 * 1024,
});
await streamFromChunks(parts)
.pipeThrough(e.stream)
.pipeTo(new WritableStream({ write: (c) => void wire.push(c) }));
return wire;
}
const single = await run([plaintext]);
const split = await run([
plaintext.subarray(0, 17_000),
plaintext.subarray(17_000, 33_000),
plaintext.subarray(33_000),
]);
expect(split.length).toBe(single.length);
for (let i = 0; i < single.length; i++) {
// Same chunk size, same lane key, same seq — wire bytes match
// byte-for-byte (deterministic nonces + AEAD).
expect(split[i]).toEqual(single[i]!);
}
});
test('100 KiB stream end-to-end completes', async () => {
const p = await makeProvider();
const streamId = subtle.randomBytes(16);
const streamSecret = subtle.randomBytes(32);
const plaintext = subtle.randomBytes(100 * 1024);
const enc = await createEncryptStream({
provider: p,
streamId,
streamSecret,
chunkSize: 16 * 1024,
});
const wire: Uint8Array[] = [];
await streamFromChunks([plaintext])
.pipeThrough(enc.stream)
.pipeTo(new WritableStream({ write: (c) => void wire.push(c) }));
const dec = await createDecryptStream({ provider: p, streamId, streamSecret });
const out = await readAll(streamFromChunks(wire).pipeThrough(dec.stream));
expect(out).toEqual(plaintext);
expect(await dec.laneSha256).toEqual(await enc.laneSha256);
});
test('decryptStream rejects out-of-order chunks', async () => {
const p = await makeProvider();
const streamId = subtle.randomBytes(16);
const streamSecret = subtle.randomBytes(32);
const plaintext = subtle.randomBytes(20_000);
const enc = await createEncryptStream({
provider: p,
streamId,
streamSecret,
chunkSize: 4 * 1024,
});
const wire: Uint8Array[] = [];
await streamFromChunks([plaintext])
.pipeThrough(enc.stream)
.pipeTo(new WritableStream({ write: (c) => void wire.push(c) }));
expect(wire.length).toBeGreaterThan(2);
// Swap first and second chunk
[wire[0], wire[1]] = [wire[1]!, wire[0]!];
const dec = await createDecryptStream({ provider: p, streamId, streamSecret });
await expect(
streamFromChunks(wire).pipeThrough(dec.stream).pipeTo(
new WritableStream({ write() {} }),
),
).rejects.toThrow();
});
});

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/dashboard",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"scripts": {
"dev": "vite",

View File

@@ -0,0 +1,12 @@
// Browser-build stub for `bun:sqlite`. The dashboard never instantiates a
// SQLite database — it talks to the prekey server over HTTP. Importing
// transitive code paths from `@shade/storage-sqlite` is fine; calling
// `new Database(...)` would throw, but the dashboard does not.
export class Database {
constructor() {
throw new Error(
'bun:sqlite is not available in the dashboard browser bundle. The dashboard talks to the prekey server over HTTP — instantiating a SQLite database here is a bug.',
);
}
}

View File

@@ -1,9 +1,22 @@
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { resolve } from 'node:path';
// `@shade/widgets` re-exports from `@shade/sdk`, which transitively imports
// `@shade/storage-sqlite` (and therefore `bun:sqlite`). Vite externalizes
// `bun:sqlite` for browser builds, but the `import { Database } from
// 'bun:sqlite'` named-import then fails to resolve. Alias the module to an
// in-tree stub — the dashboard never executes the storage code path
// (it talks to the prekey server via `Shade.send` / `fetch`), so the stub
// is never reached at runtime.
export default defineConfig({
plugins: [react()],
base: '/dashboard/',
resolve: {
alias: {
'bun:sqlite': resolve(__dirname, 'src/stubs/bun-sqlite.ts'),
},
},
build: {
outDir: 'dist',
emptyOutDir: true,

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/files",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
@@ -17,6 +17,7 @@
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/observability": "workspace:*",
"@shade/proto": "workspace:*",
"@shade/sdk": "workspace:*",
"@shade/streams": "workspace:*",

View File

@@ -80,9 +80,13 @@ export function createFilesNamespace(shade: Shade): FilesNamespace {
if (state.serverBridge === null) {
state.serverBridge = await createServerStreamsBridge(shade);
}
const inheritedObservability = shade.getObservability?.();
const handler = createFileHandler(shade, {
...handlerConfig,
streamsBridge: state.serverBridge,
...(handlerConfig.observability === undefined && inheritedObservability !== undefined
? { observability: inheritedObservability }
: {}),
});
const detach = attachFileHandler(state.channel, handler);
state.serverHandler = handler;

View File

@@ -79,6 +79,17 @@ import {
NOOP_METRIC_SINK,
type MetricSink,
} from './metrics.js';
import {
ATTR_BYTES_BIN,
ATTR_ERROR_CODE,
ATTR_OP,
ATTR_PEER_HASH,
ATTR_RESULT,
bytesBin,
NOOP_HOOK,
peerHash,
type ObservabilityHook,
} from '@shade/observability';
import {
CustomArgsSchema,
CustomResultSchema,
@@ -153,6 +164,12 @@ export interface FileHandlerConfig extends FileHandlerOps {
isFingerprintVerified?: (sender: string) => boolean | Promise<boolean>;
/** Vendor-neutral metrics sink. */
onMetric?: MetricSink;
/**
* Optional OTel observability hook. When supplied, each op is wrapped in
* a `shade.files.op` span with PII-safe attributes (peer.hash, op,
* bytes.bin, result, error.code). Defaults to no-op when omitted.
*/
observability?: ObservabilityHook;
/** Called BEFORE the handler runs. Throw to deny. */
beforeOp?: (ctx: OpContext<unknown>) => void | Promise<void>;
/** Called AFTER the handler returns. Result is the validated response. */
@@ -207,12 +224,34 @@ export function createFileHandler(
const defaultTimeoutMs = config.defaultTimeoutMs ?? 60_000;
const ioTimeoutMs = config.ioTimeoutMs ?? 60_000;
const metrics: MetricSink = config.onMetric ?? NOOP_METRIC_SINK;
const observability: ObservabilityHook = config.observability ?? NOOP_HOOK;
const customRegistrations = config.custom ?? {};
const isCustomKind = (kind: string): boolean => kind === 'shade.fs.custom/v1';
async function handleRequest(
from: string,
request: RpcRequest,
): Promise<RpcResponse | RpcError> {
const span = observability.startSpan('shade.files.op', {
[ATTR_PEER_HASH]: peerHash(from),
});
try {
const out = await runHandleRequest(from, request, span);
return out;
} catch (err) {
span.recordException(err);
span.setAttribute(ATTR_ERROR_CODE, errorCodeOf(err));
span.setStatus('error');
throw err;
} finally {
span.end();
}
}
async function runHandleRequest(
from: string,
request: RpcRequest,
span: import('@shade/observability').Span,
): Promise<RpcResponse | RpcError> {
// 0. Replay-window check (independent of sig — defends against
// intercept-and-resend even when sig verification is disabled).
@@ -316,6 +355,7 @@ export function createFileHandler(
// Replace payload with validated value (Zod may apply defaults).
(parsedArgs as CustomArgs).payload = payloadParse.data;
}
span.setAttribute(ATTR_OP, resolvedOpKind);
// 3. Path validation (skip ops without a path)
let primaryPath = '';
@@ -488,6 +528,10 @@ export function createFileHandler(
const durationMs = Date.now() - startedAt;
metrics(METRIC_OP_DURATION_MS, durationMs, { op: resolvedOpKind, result: 'error' });
metrics(METRIC_OP_TOTAL, 1, { op: resolvedOpKind, result: 'error' });
span.setAttribute(ATTR_RESULT, 'error');
span.setAttribute(ATTR_ERROR_CODE, errorCodeOf(err));
span.recordException(err);
span.setStatus('error');
cleanup({ release: true });
if (config.onError !== undefined) {
try {
@@ -543,6 +587,7 @@ export function createFileHandler(
const durationMs = Date.now() - startedAt;
metrics(METRIC_OP_DURATION_MS, durationMs, { op: resolvedOpKind, result: 'ok' });
metrics(METRIC_OP_TOTAL, 1, { op: resolvedOpKind, result: 'ok' });
span.setAttribute(ATTR_RESULT, 'ok');
if (estimatedBytes > 0) {
// Inbound bytes (write) vs outbound (read) — both reuse the same
// pre-call `estimatedBytes`, since post-execution reconciliation
@@ -551,7 +596,9 @@ export function createFileHandler(
if (direction !== null) {
metrics(direction, estimatedBytes, { op: resolvedOpKind });
}
span.setAttribute(ATTR_BYTES_BIN, bytesBin(estimatedBytes));
}
span.setStatus('ok');
return makeResponseEnvelope(request, resultParse.data);
@@ -654,6 +701,17 @@ function makeResponseEnvelope(req: RpcRequest, result: unknown): RpcResponse {
};
}
function errorCodeOf(err: unknown): string {
if (err === null || err === undefined) return 'SHADE_UNKNOWN';
if (typeof err === 'object') {
const code = (err as { code?: unknown }).code;
if (typeof code === 'string' && code.length > 0) return code;
const name = (err as { name?: unknown }).name;
if (typeof name === 'string' && name.length > 0) return name;
}
return 'SHADE_UNKNOWN';
}
function makeErrorEnvelope(req: RpcRequest, err: unknown): RpcError {
return {
kind: 'shade.fs.error/v1',

View File

@@ -0,0 +1,21 @@
{
"name": "@shade/inbox-server",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/observability": "workspace:*",
"@shade/server": "workspace:*",
"hono": "^4.12.12"
},
"optionalDependencies": {
"@shade/crypto-web": "workspace:*",
"@shade/storage-postgres": "workspace:*",
"@shade/storage-sqlite": "workspace:*"
},
"devDependencies": {
"@shade/crypto-web": "workspace:*"
}
}

View File

@@ -0,0 +1,461 @@
/**
* Bridge routes — V3.7.
*
* Three transports, one delivery semantic. Each one streams the same
* not-yet-acked inbox blobs for an authenticated address:
*
* GET /v1/bridge/stream — SSE feed, one envelope per `event: envelope`
* GET /v1/bridge/poll — long-poll, returns at most one batch then closes
* GET /v1/bridge/ws — WebSocket, JSON frame per envelope
*
* Auth: signed query string (`address`, `kind`, `since`, `signedAt`,
* `signature`). The signature is verified against the address's owner key
* registered via `/v1/inbox/register`. The `kind` field is bound into the
* canonical signed payload to prevent cross-endpoint replay.
*
* Cursor semantics: `since` is the highest `receivedAt` the client already
* processed. The server returns blobs strictly greater than that cursor and
* advances the client's cursor by emitting a fresh `id:` (SSE) or by
* including the highest seen `receivedAt` in the JSON response (poll/ws).
*
* The implementations subscribe to {@link InboxServerEvents} so that newly
* stored blobs land on connected clients without polling the store. The
* fallback path (no events configured) relies on a small in-process polling
* timer with a configurable interval.
*/
import { Hono, type Context } from 'hono';
import { streamSSE } from 'hono/streaming';
import { createBunWebSocket } from 'hono/bun';
import type { CryptoProvider } from '@shade/core';
import {
errorToHttpStatus,
ShadeError,
ValidationError,
UnauthorizedError,
toBase64,
} from '@shade/core';
import { verifyPayload, validateAddress } from '@shade/server';
import type { InboxStore } from './store.js';
import type { InboxServerEvents } from './events.js';
export type BridgeKind = 'stream' | 'poll' | 'ws';
export interface BridgeRoutesOptions {
store: InboxStore;
crypto: CryptoProvider;
/** Optional events emitter — enables push-style delivery. */
events?: InboxServerEvents;
/** Maximum blobs returned per fetch page. Default 50. */
pageLimit?: number;
/** Default long-poll hold (ms). Default 25_000 (under typical proxy cutoffs). */
longPollTimeoutMs?: number;
/** Maximum long-poll hold (ms). Hard cap. Default 55_000. */
longPollMaxTimeoutMs?: number;
/** SSE heartbeat interval (ms). Default 15_000. */
heartbeatIntervalMs?: number;
/**
* Fallback poll interval (ms) used when no `events` emitter is wired in.
* The bridge will re-check the store at this cadence to detect new blobs.
* Default 1_000.
*/
fallbackPollIntervalMs?: number;
}
interface VerifiedBridgeRequest {
address: string;
kind: BridgeKind;
since: number;
}
/**
* Build the bridge Hono router and a paired Bun-WebSocket handler.
*
* The HTTP routes (`/v1/bridge/stream`, `/v1/bridge/poll`) work on every
* Hono runtime. The `/v1/bridge/ws` route requires `hono/adapter/bun` to be
* available — we lazy-require it so that non-Bun deployments aren't
* forced to ship the import.
*/
export function createBridgeRoutes(opts: BridgeRoutesOptions): {
app: Hono;
/** Pass to `Bun.serve({ websocket })`. Undefined if Bun adapter is missing. */
websocket: unknown;
} {
const app = new Hono();
const pageLimit = opts.pageLimit ?? 50;
const heartbeatIntervalMs = opts.heartbeatIntervalMs ?? 15_000;
const longPollDefault = opts.longPollTimeoutMs ?? 25_000;
const longPollMax = opts.longPollMaxTimeoutMs ?? 55_000;
const fallbackPollIntervalMs = opts.fallbackPollIntervalMs ?? 1_000;
app.onError((err, c) => {
if (err instanceof ShadeError) {
const status = errorToHttpStatus(err);
return c.json(err.toJSON(), status as any);
}
console.error('[Shade] Unhandled bridge error:', err);
return c.json({ error: 'Internal server error' }, 500);
});
// ─── SSE ──────────────────────────────────────────────────────
app.get('/v1/bridge/stream', async (c) => {
const verified = await verifyBridgeAuth(c, opts, 'stream');
return streamSSE(c, async (stream) => {
const address = verified.address;
let cursor = verified.since;
const writer = makeBlobWriter(opts.store, pageLimit);
// Initial backlog drain.
const flushed = await flushTo(writer, address, cursor, async (blob) => {
await stream.writeSSE({
id: String(blob.receivedAt),
event: 'envelope',
data: JSON.stringify(serializeBlob(blob)),
});
});
cursor = Math.max(cursor, flushed);
// Hook up event-driven push if available, else fall back to a poll
// timer that does the same scan.
let cleanupSubscription: (() => void) | null = null;
let signalled = false;
let pendingFlushPromise: Promise<void> = Promise.resolve();
const triggerFlush = (): void => {
signalled = true;
// Serialize fan-in so concurrent triggers don't double-fetch.
pendingFlushPromise = pendingFlushPromise.then(async () => {
while (signalled) {
signalled = false;
const drained = await flushTo(writer, address, cursor, async (blob) => {
await stream.writeSSE({
id: String(blob.receivedAt),
event: 'envelope',
data: JSON.stringify(serializeBlob(blob)),
});
});
if (drained > cursor) cursor = drained;
}
});
};
if (opts.events) {
cleanupSubscription = opts.events.on((e) => {
if (e.name === 'inbox.blob_stored' && e.data.address === address) {
triggerFlush();
}
});
}
const fallbackTimer = setInterval(() => triggerFlush(), fallbackPollIntervalMs);
const heartbeat = setInterval(() => {
// Comment lines are valid SSE keepalives.
stream.write(`: ping ${Date.now()}\n\n`).catch(() => {});
}, heartbeatIntervalMs);
// Wait for the request to abort (client disconnect).
await new Promise<void>((resolve) => {
const sig = c.req.raw.signal;
if (sig.aborted) return resolve();
sig.addEventListener('abort', () => resolve(), { once: true });
});
cleanupSubscription?.();
clearInterval(fallbackTimer);
clearInterval(heartbeat);
await pendingFlushPromise.catch(() => {});
});
});
// ─── Long-poll ────────────────────────────────────────────────
app.get('/v1/bridge/poll', async (c) => {
const verified = await verifyBridgeAuth(c, opts, 'poll');
const requestedTimeout = Number(c.req.query('timeoutMs') ?? longPollDefault);
const timeoutMs = Math.min(
Math.max(0, Number.isFinite(requestedTimeout) ? requestedTimeout : longPollDefault),
longPollMax,
);
// Try immediate fetch first.
let blobs = await opts.store.fetchBlobs({
address: verified.address,
sinceCursor: verified.since,
now: Date.now(),
limit: pageLimit,
});
if (blobs.length > 0) {
return c.json(buildPollResponse(blobs, verified.since));
}
// Otherwise, wait for either a new event or the timeout.
blobs = await waitForBlobs({
events: opts.events ?? null,
store: opts.store,
address: verified.address,
since: verified.since,
timeoutMs,
pageLimit,
fallbackPollIntervalMs,
abortSignal: c.req.raw.signal,
});
return c.json(buildPollResponse(blobs, verified.since));
});
// ─── WebSocket ────────────────────────────────────────────────
// Hono's Bun adapter resolves `getBunServer` from the request's `env`
// (the second argument of Bun.serve's fetch). On non-Bun runtimes the
// upgrade simply fails at runtime; the SSE/long-poll routes still work.
const { upgradeWebSocket, websocket } = createBunWebSocket();
app.get(
'/v1/bridge/ws',
upgradeWebSocket(async (c: Context) => {
let verified: VerifiedBridgeRequest | null = null;
let upgradeError: Error | null = null;
try {
verified = await verifyBridgeAuth(c, opts, 'ws');
} catch (err) {
upgradeError = err as Error;
}
if (!verified) {
// Hono's API doesn't let us reject the upgrade with a status code
// before opening the socket; close immediately on open with a 4xxx
// policy code so the client can fall back to a different bridge.
return {
onOpen(_evt: unknown, ws: { close: (code?: number, reason?: string) => void }) {
const status =
upgradeError instanceof ShadeError ? errorToHttpStatus(upgradeError) : 500;
ws.close(4000 + (status % 1000), upgradeError?.message ?? 'unauthorized');
},
};
}
const address = verified.address;
let cursor = verified.since;
const writer = makeBlobWriter(opts.store, pageLimit);
let unsubscribe: (() => void) | null = null;
let fallbackTimer: ReturnType<typeof setInterval> | null = null;
let pendingFlushPromise: Promise<void> = Promise.resolve();
let signalled = false;
let connected = true;
return {
onOpen(_evt: unknown, ws: {
send: (data: string) => void;
close: (code?: number, reason?: string) => void;
}) {
const triggerFlush = (): void => {
signalled = true;
pendingFlushPromise = pendingFlushPromise.then(async () => {
while (signalled && connected) {
signalled = false;
const drained = await flushTo(writer, address, cursor, async (blob) => {
ws.send(JSON.stringify(serializeBlob(blob)));
});
if (drained > cursor) cursor = drained;
}
});
};
if (opts.events) {
unsubscribe = opts.events.on((e) => {
if (e.name === 'inbox.blob_stored' && e.data.address === address) {
triggerFlush();
}
});
}
fallbackTimer = setInterval(() => triggerFlush(), fallbackPollIntervalMs);
triggerFlush();
},
onClose() {
connected = false;
unsubscribe?.();
if (fallbackTimer) clearInterval(fallbackTimer);
},
};
}),
);
return { app, websocket };
}
// ─── helpers ──────────────────────────────────────────────────
async function verifyBridgeAuth(
c: Context,
opts: BridgeRoutesOptions,
expectedKind: BridgeKind,
): Promise<VerifiedBridgeRequest> {
const url = new URL(c.req.url);
const qs = url.searchParams;
const address = validateAddress(qs.get('address'));
const kind = qs.get('kind');
if (kind !== expectedKind) {
throw new ValidationError(`bridge kind mismatch: expected ${expectedKind}`, 'kind');
}
const sinceStr = qs.get('since');
const signedAtStr = qs.get('signedAt');
const signature = qs.get('signature');
if (sinceStr === null) throw new ValidationError('missing since', 'since');
if (signedAtStr === null) throw new ValidationError('missing signedAt', 'signedAt');
if (!signature) throw new ValidationError('missing signature', 'signature');
const since = Number(sinceStr);
const signedAt = Number(signedAtStr);
if (!Number.isFinite(since) || since < 0) {
throw new ValidationError('since must be a non-negative number', 'since');
}
if (!Number.isFinite(signedAt)) {
throw new ValidationError('signedAt must be a number', 'signedAt');
}
const owner = await opts.store.getAddressOwner(address);
if (!owner) {
throw new UnauthorizedError(`address ${address} is not registered`);
}
await verifyPayload(opts.crypto, owner, {
address,
kind,
since,
signedAt,
signature,
});
return { address, kind: kind as BridgeKind, since };
}
interface BlobRow {
msgId: string;
ciphertext: Uint8Array;
receivedAt: number;
expiresAt: number;
}
interface BlobWriter {
fetchPage(address: string, cursor: number): Promise<BlobRow[]>;
}
function makeBlobWriter(store: InboxStore, pageLimit: number): BlobWriter {
return {
async fetchPage(address, cursor) {
return store.fetchBlobs({
address,
sinceCursor: cursor,
now: Date.now(),
limit: pageLimit,
});
},
};
}
async function flushTo(
writer: BlobWriter,
address: string,
startCursor: number,
emit: (blob: BlobRow) => Promise<void>,
): Promise<number> {
let cursor = startCursor;
// Drain page-by-page so a backlog larger than `pageLimit` still flushes.
// eslint-disable-next-line no-constant-condition
while (true) {
const page = await writer.fetchPage(address, cursor);
if (page.length === 0) break;
for (const row of page) {
await emit(row);
if (row.receivedAt > cursor) cursor = row.receivedAt;
}
if (page.length === 0) break;
}
return cursor;
}
function serializeBlob(blob: BlobRow): {
msgId: string;
ciphertext: string;
receivedAt: number;
expiresAt: number;
} {
return {
msgId: blob.msgId,
ciphertext: toBase64(blob.ciphertext),
receivedAt: blob.receivedAt,
expiresAt: blob.expiresAt,
};
}
function buildPollResponse(blobs: BlobRow[], sinceFallback: number): {
blobs: ReturnType<typeof serializeBlob>[];
cursor: number;
hasMore: boolean;
} {
const out = blobs.map(serializeBlob);
const cursor = blobs.length > 0 ? blobs[blobs.length - 1]!.receivedAt : sinceFallback;
return { blobs: out, cursor, hasMore: false };
}
interface WaitForBlobsArgs {
events: InboxServerEvents | null;
store: InboxStore;
address: string;
since: number;
timeoutMs: number;
pageLimit: number;
fallbackPollIntervalMs: number;
abortSignal?: AbortSignal;
}
async function waitForBlobs(args: WaitForBlobsArgs): Promise<BlobRow[]> {
if (args.timeoutMs === 0) return [];
return new Promise<BlobRow[]>((resolve) => {
let resolved = false;
let unsubscribe: (() => void) | null = null;
let timer: ReturnType<typeof setTimeout> | null = null;
let fallback: ReturnType<typeof setInterval> | null = null;
let abortHandler: (() => void) | null = null;
const finish = (blobs: BlobRow[]) => {
if (resolved) return;
resolved = true;
if (timer) clearTimeout(timer);
if (fallback) clearInterval(fallback);
if (unsubscribe) unsubscribe();
if (abortHandler && args.abortSignal) {
args.abortSignal.removeEventListener('abort', abortHandler);
}
resolve(blobs);
};
const tryFetch = async (): Promise<void> => {
try {
const rows = await args.store.fetchBlobs({
address: args.address,
sinceCursor: args.since,
now: Date.now(),
limit: args.pageLimit,
});
if (rows.length > 0) finish(rows);
} catch {
// swallow — let timeout handle it
}
};
if (args.events) {
unsubscribe = args.events.on((e) => {
if (e.name === 'inbox.blob_stored' && e.data.address === args.address) {
void tryFetch();
}
});
}
fallback = setInterval(tryFetch, args.fallbackPollIntervalMs);
timer = setTimeout(() => finish([]), args.timeoutMs);
if (args.abortSignal) {
if (args.abortSignal.aborted) {
finish([]);
return;
}
abortHandler = () => finish([]);
args.abortSignal.addEventListener('abort', abortHandler, { once: true });
}
});
}

View File

@@ -0,0 +1,69 @@
import type { InboxStore } from './store.js';
import type { InboxServerEvents } from './events.js';
/**
* Periodic prune task — drops every blob whose `expires_at <= now`.
*
* Configurable via env vars:
* SHADE_INBOX_PRUNE_INTERVAL_MINUTES (default 5)
*/
export class InboxPruneTask {
private timer: ReturnType<typeof setInterval> | null = null;
private running = false;
private readonly intervalMs: number;
constructor(
private readonly store: InboxStore,
options: {
intervalMinutes?: number;
events?: InboxServerEvents;
logger?: { info: (msg: string, meta?: unknown) => void; error: (msg: string, meta?: unknown) => void };
} = {},
) {
const minutes = options.intervalMinutes
?? Number(process.env.SHADE_INBOX_PRUNE_INTERVAL_MINUTES ?? 5);
this.intervalMs = minutes * 60 * 1000;
this.events = options.events;
this.logger = options.logger ?? null;
}
private readonly events: InboxServerEvents | undefined;
private readonly logger: {
info: (msg: string, meta?: unknown) => void;
error: (msg: string, meta?: unknown) => void;
} | null;
start(): void {
if (this.running) return;
this.running = true;
this.runOnce().catch((err) =>
this.logger?.error('Initial inbox prune failed', { error: String(err) }),
);
this.timer = setInterval(() => {
this.runOnce().catch((err) =>
this.logger?.error('Inbox prune failed', { error: String(err) }),
);
}, this.intervalMs);
}
stop(): void {
this.running = false;
if (this.timer) {
clearInterval(this.timer);
this.timer = null;
}
}
async runOnce(): Promise<number> {
const removed = await this.store.purgeExpired(Date.now());
if (removed > 0) {
this.logger?.info('Inbox prune removed expired blobs', { count: removed });
this.events?.emit('inbox.expired_purged', { count: removed });
}
return removed;
}
get isRunning(): boolean {
return this.running;
}
}

View File

@@ -0,0 +1,90 @@
/**
* Inbox server event emitter.
*
* Mirrors `PrekeyServerEvents`. Emits structural facts only — no plaintext,
* no signatures, no key material. Used by the observer dashboard and
* operator metrics.
*/
export interface InboxServerEventBase {
seq: number;
timestamp: number;
}
export interface InboxServerEventMap {
'inbox.address_registered': { address: string; signingKeyHash: string };
'inbox.address_deleted': { address: string };
'inbox.blob_stored': { address: string; msgId: string; bytes: number; ttlSeconds: number };
'inbox.blob_idempotent_replay': { address: string; msgId: string };
'inbox.blob_fetched': { address: string; count: number; bytes: number };
'inbox.blob_acked': { address: string; msgId: string };
'inbox.expired_purged': { count: number };
'inbox.rate_limited': { route: string; key: string };
'inbox.quota_rejected': { address: string; reason: 'address-quota' | 'sender-quota' | 'body-too-large' };
}
export type InboxServerEventName = keyof InboxServerEventMap;
export type InboxServerEvent = {
[K in InboxServerEventName]: InboxServerEventBase & { name: K; data: InboxServerEventMap[K] };
}[InboxServerEventName];
export type InboxServerEventListener = (event: InboxServerEvent) => void;
export class InboxServerEvents {
private listeners = new Set<InboxServerEventListener>();
private nextSeq = 1;
private buffer: InboxServerEvent[] = [];
private readonly maxBuffer: number;
constructor(options: { bufferSize?: number } = {}) {
this.maxBuffer = options.bufferSize ?? 1000;
}
on(listener: InboxServerEventListener): () => void {
this.listeners.add(listener);
return () => this.listeners.delete(listener);
}
off(listener: InboxServerEventListener): void {
this.listeners.delete(listener);
}
emit<K extends InboxServerEventName>(name: K, data: InboxServerEventMap[K]): void {
const event = {
seq: this.nextSeq++,
timestamp: Date.now(),
name,
data,
} as InboxServerEvent;
this.buffer.push(event);
if (this.buffer.length > this.maxBuffer) this.buffer.shift();
for (const listener of this.listeners) {
try {
listener(event);
} catch (err) {
console.error('[Shade] Inbox event listener threw:', err);
}
}
}
getBufferedSince(since: number): InboxServerEvent[] {
return this.buffer.filter((e) => e.seq > since);
}
getRecent(n: number): InboxServerEvent[] {
return this.buffer.slice(-n);
}
get currentSeq(): number {
return this.nextSeq - 1;
}
}
export async function shortHash(key: Uint8Array): Promise<string> {
const buf = await globalThis.crypto.subtle.digest('SHA-256', key as unknown as ArrayBuffer);
const arr = new Uint8Array(buf).slice(0, 8);
return Array.from(arr, (b) => b.toString(16).padStart(2, '0')).join('');
}

View File

@@ -0,0 +1,60 @@
import type { Hono } from 'hono';
import type { CryptoProvider } from '@shade/core';
import { createInboxRoutes, type InboxRoutesOptions } from './routes.js';
import { MemoryInboxStore } from './memory-store.js';
import type { InboxStore } from './store.js';
import { InboxServerEvents } from './events.js';
export { createInboxRoutes } from './routes.js';
export type { InboxRoutesOptions } from './routes.js';
export { MemoryInboxStore } from './memory-store.js';
export type { InboxStore } from './store.js';
export {
InboxServerEvents,
shortHash as inboxShortHash,
} from './events.js';
export type {
InboxServerEvent,
InboxServerEventName,
InboxServerEventMap,
InboxServerEventListener,
} from './events.js';
export { InboxPruneTask } from './cleanup.js';
export {
computeMsgId,
isValidMsgId,
constantTimeStringEqual,
} from './msg-id.js';
export {
DEFAULT_INBOX_QUOTA,
clampTtl,
} from './quota.js';
export type { InboxQuotaConfig } from './quota.js';
export { createBridgeRoutes } from './bridge.js';
export type { BridgeRoutesOptions, BridgeKind } from './bridge.js';
/**
* Create a standalone Shade Inbox Server.
*
* const crypto = new SubtleCryptoProvider();
* const inbox = createInboxServer({ crypto });
* export default { port: 3901, fetch: inbox.fetch };
*
* Or compose into an existing Hono app:
* const app = new Hono();
* app.route('/', createInboxServer({ crypto }));
*/
export function createInboxServer(options: {
crypto: CryptoProvider;
store?: InboxStore;
disableRateLimit?: boolean;
events?: InboxServerEvents;
} & Pick<InboxRoutesOptions, 'observability' | 'quota'>): Hono {
const store = options.store ?? new MemoryInboxStore();
const routesOptions: InboxRoutesOptions = {};
if (options.disableRateLimit !== undefined) routesOptions.disableRateLimit = options.disableRateLimit;
if (options.events !== undefined) routesOptions.events = options.events;
if (options.observability !== undefined) routesOptions.observability = options.observability;
if (options.quota !== undefined) routesOptions.quota = options.quota;
return createInboxRoutes(store, options.crypto, routesOptions);
}

View File

@@ -0,0 +1,105 @@
import type { InboxStore } from './store.js';
interface BlobRow {
msgId: string;
ciphertext: Uint8Array;
receivedAt: number;
expiresAt: number;
}
/**
* In-memory InboxStore — used in tests and as the default fallback when
* neither SHADE_INBOX_DB_PATH nor SHADE_INBOX_PG_URL is set.
*
* Blobs are organized in per-address insertion-ordered arrays so cursor
* pagination is just a `>` filter on `receivedAt`.
*/
export class MemoryInboxStore implements InboxStore {
private owners = new Map<string, Uint8Array>();
private blobs = new Map<string, BlobRow[]>();
private nextReceivedAt = 0;
async saveAddressOwner(address: string, signingKey: Uint8Array): Promise<void> {
this.owners.set(address, new Uint8Array(signingKey));
}
async getAddressOwner(address: string): Promise<Uint8Array | null> {
const k = this.owners.get(address);
return k ? new Uint8Array(k) : null;
}
async putBlob(args: {
address: string;
msgId: string;
ciphertext: Uint8Array;
expiresAt: number;
}): Promise<{ created: boolean; receivedAt: number }> {
const list = this.blobs.get(args.address) ?? [];
const existing = list.find((r) => r.msgId === args.msgId);
if (existing) return { created: false, receivedAt: existing.receivedAt };
// Monotonic `receivedAt` so cursor compare is total-order even when
// multiple blobs land in the same millisecond.
const receivedAt = Math.max(this.nextReceivedAt + 1, Date.now());
this.nextReceivedAt = receivedAt;
list.push({
msgId: args.msgId,
ciphertext: new Uint8Array(args.ciphertext),
receivedAt,
expiresAt: args.expiresAt,
});
this.blobs.set(args.address, list);
return { created: true, receivedAt };
}
async fetchBlobs(args: {
address: string;
sinceCursor: number;
now: number;
limit: number;
}): Promise<Array<{ msgId: string; ciphertext: Uint8Array; receivedAt: number; expiresAt: number }>> {
const list = this.blobs.get(args.address) ?? [];
return list
.filter((r) => r.receivedAt > args.sinceCursor && r.expiresAt > args.now)
.sort((a, b) => a.receivedAt - b.receivedAt)
.slice(0, args.limit)
.map((r) => ({
msgId: r.msgId,
ciphertext: new Uint8Array(r.ciphertext),
receivedAt: r.receivedAt,
expiresAt: r.expiresAt,
}));
}
async deleteBlob(address: string, msgId: string): Promise<boolean> {
const list = this.blobs.get(address);
if (!list) return false;
const idx = list.findIndex((r) => r.msgId === msgId);
if (idx === -1) return false;
list.splice(idx, 1);
return true;
}
async countBlobs(address: string, now: number): Promise<number> {
const list = this.blobs.get(address) ?? [];
return list.filter((r) => r.expiresAt > now).length;
}
async purgeExpired(now: number): Promise<number> {
let removed = 0;
for (const [addr, list] of this.blobs) {
const filtered = list.filter((r) => r.expiresAt > now);
removed += list.length - filtered.length;
if (filtered.length === 0) {
this.blobs.delete(addr);
} else {
this.blobs.set(addr, filtered);
}
}
return removed;
}
async deleteAddress(address: string): Promise<void> {
this.owners.delete(address);
this.blobs.delete(address);
}
}

View File

@@ -0,0 +1,37 @@
/**
* msgId derivation: deterministic SHA-256 of the ciphertext blob.
*
* `msgId = lowercase-hex( sha256(ciphertext) )`
*
* Both client and server compute it independently and check that the
* client's claimed msgId equals the recomputed hash — that gives us
* idempotency on PUT (same ciphertext → same row) and replay-protection
* for free (a tampered re-upload changes the hash and lands in a new
* slot the recipient simply ignores when decrypt fails).
*/
export async function computeMsgId(ciphertext: Uint8Array): Promise<string> {
const buf = await globalThis.crypto.subtle.digest(
'SHA-256',
ciphertext as unknown as ArrayBuffer,
);
const arr = new Uint8Array(buf);
return Array.from(arr, (b) => b.toString(16).padStart(2, '0')).join('');
}
/** Constant-time string equality on the hex form, both lowercased first. */
export function constantTimeStringEqual(a: string, b: string): boolean {
const aa = a.toLowerCase();
const bb = b.toLowerCase();
if (aa.length !== bb.length) return false;
let diff = 0;
for (let i = 0; i < aa.length; i++) {
diff |= aa.charCodeAt(i) ^ bb.charCodeAt(i);
}
return diff === 0;
}
/** Validate hex form: 64 lowercase hex chars (32-byte digest). */
export function isValidMsgId(s: unknown): s is string {
return typeof s === 'string' && /^[0-9a-f]{64}$/.test(s);
}

View File

@@ -0,0 +1,47 @@
import { ValidationError } from '@shade/core';
/**
* Inbox quota policy. The relay limits per-address and per-sender storage
* so a single ondsinnet sender can't exhaust capacity.
*/
export interface InboxQuotaConfig {
/** Hard cap on the body size of a single PUT (bytes). Default: 1 MiB. */
maxBlobBytes: number;
/**
* Hard cap on the number of non-expired blobs queued for a single
* recipient address. PUTs past this cap return SHADE_VALIDATION.
* Default: 1000.
*/
maxBlobsPerAddress: number;
/** Default TTL when sender omits ttlSeconds. Default: 7 days. */
defaultTtlSeconds: number;
/** Maximum TTL the relay will accept. Default: 30 days. */
maxTtlSeconds: number;
/** Minimum TTL. Default: 60 seconds. */
minTtlSeconds: number;
/** Maximum number of blobs returned in one GET. Default: 100. */
fetchPageLimit: number;
}
export const DEFAULT_INBOX_QUOTA: InboxQuotaConfig = {
maxBlobBytes: 1 * 1024 * 1024,
maxBlobsPerAddress: 1000,
defaultTtlSeconds: 7 * 24 * 60 * 60,
maxTtlSeconds: 30 * 24 * 60 * 60,
minTtlSeconds: 60,
fetchPageLimit: 100,
};
export function clampTtl(ttl: number | undefined, q: InboxQuotaConfig): number {
const v = ttl ?? q.defaultTtlSeconds;
if (!Number.isFinite(v) || v <= 0) {
throw new ValidationError('ttlSeconds must be positive', 'ttlSeconds');
}
if (v < q.minTtlSeconds) {
throw new ValidationError(`ttlSeconds < min (${q.minTtlSeconds})`, 'ttlSeconds');
}
if (v > q.maxTtlSeconds) {
throw new ValidationError(`ttlSeconds > max (${q.maxTtlSeconds})`, 'ttlSeconds');
}
return Math.floor(v);
}

View File

@@ -0,0 +1,372 @@
import { Hono } from 'hono';
import type { CryptoProvider } from '@shade/core';
import {
errorToHttpStatus,
ShadeError,
ValidationError,
RateLimitError,
UnauthorizedError,
fromBase64,
toBase64,
constantTimeEqual,
} from '@shade/core';
import {
verifyPayload,
validateAddress,
RateLimiter,
MemoryRateLimitStore,
type RateLimitConfig,
} from '@shade/server';
import {
ATTR_ERROR_CODE,
ATTR_HTTP_STATUS,
ATTR_ROUTE,
NOOP_HOOK,
type ObservabilityHook,
} from '@shade/observability';
import type { InboxStore } from './store.js';
import { InboxServerEvents, shortHash } from './events.js';
import { computeMsgId, isValidMsgId, constantTimeStringEqual } from './msg-id.js';
import { clampTtl, DEFAULT_INBOX_QUOTA, type InboxQuotaConfig } from './quota.js';
/** Max metadata-only body size (no ciphertext). 64 KB is enough headroom. */
const MAX_META_BODY_SIZE = 64 * 1024;
/**
* Per-route token-bucket presets. PUT is intentionally generous (senders
* may burst) but bound on the recipient side (per-address quota in the
* store). FETCH and DELETE are per-address.
*/
const INBOX_PUT_LIMIT: RateLimitConfig = { capacity: 60, refillPerSecond: 1 };
const INBOX_FETCH_LIMIT: RateLimitConfig = { capacity: 60, refillPerSecond: 1 };
const INBOX_DELETE_LIMIT: RateLimitConfig = { capacity: 60, refillPerSecond: 1 };
const INBOX_REGISTER_LIMIT: RateLimitConfig = {
capacity: 5,
refillPerSecond: 5 / 3600,
};
export interface InboxRoutesOptions {
/** Disable rate limiting (used in tests). */
disableRateLimit?: boolean;
/** Optional event emitter. */
events?: InboxServerEvents;
/** OTel observability hook. */
observability?: ObservabilityHook;
/** Override quota policy. */
quota?: Partial<InboxQuotaConfig>;
}
export function createInboxRoutes(
store: InboxStore,
crypto: CryptoProvider,
options: InboxRoutesOptions = {},
): Hono {
const app = new Hono();
const events = options.events;
const observability = options.observability ?? NOOP_HOOK;
const quota: InboxQuotaConfig = { ...DEFAULT_INBOX_QUOTA, ...(options.quota ?? {}) };
app.use('*', async (c, next) => {
const route = c.req.routePath ?? c.req.path ?? '<unknown>';
const span = observability.startSpan('shade.inbox.request', {
[ATTR_ROUTE]: route,
});
try {
await next();
span.setAttribute(ATTR_HTTP_STATUS, c.res.status);
span.setStatus(c.res.status >= 500 ? 'error' : 'ok');
} catch (err) {
const code =
err instanceof ShadeError ? err.code ?? 'SHADE_ERROR' : 'SHADE_INTERNAL';
span.setAttribute(ATTR_ERROR_CODE, code);
span.recordException(err);
span.setStatus('error', code);
throw err;
} finally {
span.end();
}
});
const rlStore = new MemoryRateLimitStore();
const putRL = new RateLimiter(rlStore, INBOX_PUT_LIMIT);
const fetchRL = new RateLimiter(rlStore, INBOX_FETCH_LIMIT);
const deleteRL = new RateLimiter(rlStore, INBOX_DELETE_LIMIT);
const registerRL = new RateLimiter(rlStore, INBOX_REGISTER_LIMIT);
const rateLimitEnabled = !options.disableRateLimit;
const getClientIp = (c: any): string =>
c.req.header('x-forwarded-for')?.split(',')[0]?.trim() ??
c.req.header('x-real-ip') ??
'unknown';
app.get('/health', (c) => c.json({ status: 'ok', service: 'shade-inbox-server' }));
app.onError((err, c) => {
if (err instanceof RateLimitError) {
events?.emit('inbox.rate_limited', {
route: c.req.routePath ?? c.req.path,
key: getClientIp(c),
});
}
if (err instanceof ShadeError) {
const status = errorToHttpStatus(err);
const body: any = err.toJSON();
if ((err as any).retryAfterSeconds) {
c.header('Retry-After', String((err as any).retryAfterSeconds));
}
return c.json(body, status as any);
}
console.error('[Shade] Unhandled inbox error:', err);
return c.json({ error: 'Internal server error' }, 500);
});
// ─── Register address (TOFU) ───────────────────────────────
// Recipient claims an address by uploading its signing key and a
// signature over the canonical body. Subsequent PUT/GET/DELETE for the
// address are authenticated against this key. Idempotent if the same key
// re-registers; rejects if a different key tries to take an existing slot.
app.post('/v1/inbox/register', async (c) => {
if (rateLimitEnabled) await registerRL.consume(`inbox-register:${getClientIp(c)}`);
const rawBody = await c.req.text();
if (rawBody.length > MAX_META_BODY_SIZE) {
throw new ValidationError(`Request body too large (max ${MAX_META_BODY_SIZE} bytes)`);
}
const body = JSON.parse(rawBody);
const { address, signingKey } = body;
const addr = validateAddress(address);
if (typeof signingKey !== 'string') {
throw new ValidationError('Missing signingKey', 'signingKey');
}
const key = b64ToBytes(signingKey);
// Verify signature against the asserted key (TOFU).
await verifyPayload(crypto, key, body);
const existing = await store.getAddressOwner(addr);
if (existing && !constantTimeEqual(existing, key)) {
throw new UnauthorizedError(`Address already claimed by a different key`);
}
await store.saveAddressOwner(addr, key);
if (events) {
events.emit('inbox.address_registered', {
address: addr,
signingKeyHash: await shortHash(key),
});
}
return c.json({ ok: true });
});
// ─── Unregister (signed) ───────────────────────────────────
app.delete('/v1/inbox/register/:address', async (c) => {
const address = validateAddress(c.req.param('address'));
if (rateLimitEnabled) await registerRL.consume(`inbox-unregister:${address}`);
const owner = await store.getAddressOwner(address);
if (!owner) {
return c.json({ error: 'Address not registered', code: 'SHADE_NOT_FOUND' }, 404);
}
const body = await c.req.json();
await verifyPayload(crypto, owner, { ...body, address });
await store.deleteAddress(address);
events?.emit('inbox.address_deleted', { address });
return c.json({ ok: true });
});
// ─── PUT a blob (signed by sender) ─────────────────────────
// Body format:
// {
// senderSigningKey: b64,
// msgId: hex(sha256(ciphertext)),
// ciphertext: b64,
// ttlSeconds?: number,
// signedAt: number,
// signature: b64, // over the canonical body sans signature
// }
// The recipient address is the path parameter. The sender authenticates
// itself via `senderSigningKey` (TOFU per request — the *recipient*
// determines whether to accept the sender, via the encrypted envelope).
app.post('/v1/inbox/:address', async (c) => {
if (rateLimitEnabled) await putRL.consume(`inbox-put:${getClientIp(c)}`);
const address = validateAddress(c.req.param('address'));
const rawBody = await c.req.text();
// Allow up to (maxBlobBytes * 4/3) for base64 + JSON overhead.
const hardLimit = Math.ceil(quota.maxBlobBytes * 1.4) + MAX_META_BODY_SIZE;
if (rawBody.length > hardLimit) {
events?.emit('inbox.quota_rejected', { address, reason: 'body-too-large' });
throw new ValidationError(`Request body too large`);
}
const body = JSON.parse(rawBody);
const { senderSigningKey, msgId, ciphertext, ttlSeconds } = body;
if (typeof senderSigningKey !== 'string') {
throw new ValidationError('Missing senderSigningKey', 'senderSigningKey');
}
if (typeof ciphertext !== 'string') {
throw new ValidationError('Missing ciphertext', 'ciphertext');
}
if (!isValidMsgId(msgId)) {
throw new ValidationError('msgId must be 64 lowercase hex chars', 'msgId');
}
const senderKey = b64ToBytes(senderSigningKey);
const ctBytes = b64ToBytes(ciphertext);
if (ctBytes.length === 0) {
throw new ValidationError('ciphertext is empty', 'ciphertext');
}
if (ctBytes.length > quota.maxBlobBytes) {
events?.emit('inbox.quota_rejected', { address, reason: 'body-too-large' });
throw new ValidationError(
`ciphertext exceeds maxBlobBytes (${ctBytes.length} > ${quota.maxBlobBytes})`,
);
}
// Verify the claimed msgId matches the actual ciphertext digest.
const recomputed = await computeMsgId(ctBytes);
if (!constantTimeStringEqual(recomputed, msgId)) {
throw new ValidationError('msgId does not match sha256(ciphertext)', 'msgId');
}
// Verify sender signature.
await verifyPayload(crypto, senderKey, body);
// Recipient address must be registered (avoids DoS against unclaimed
// slots — see THREAT-MODEL).
const recipient = await store.getAddressOwner(address);
if (!recipient) {
return c.json({ error: 'Recipient not registered', code: 'SHADE_NOT_FOUND' }, 404);
}
const ttl = clampTtl(typeof ttlSeconds === 'number' ? ttlSeconds : undefined, quota);
const expiresAt = Date.now() + ttl * 1000;
// Per-address quota check before the write so the cap is enforced.
const currentCount = await store.countBlobs(address, Date.now());
if (currentCount >= quota.maxBlobsPerAddress) {
events?.emit('inbox.quota_rejected', { address, reason: 'address-quota' });
throw new ValidationError(
`Recipient inbox is full (${currentCount} >= ${quota.maxBlobsPerAddress})`,
);
}
const result = await store.putBlob({
address,
msgId,
ciphertext: ctBytes,
expiresAt,
});
if (result.created) {
events?.emit('inbox.blob_stored', {
address,
msgId,
bytes: ctBytes.length,
ttlSeconds: ttl,
});
} else {
events?.emit('inbox.blob_idempotent_replay', { address, msgId });
}
return c.json({
ok: true,
msgId,
receivedAt: result.receivedAt,
idempotent: !result.created,
});
});
// ─── GET blobs (signed challenge by recipient) ─────────────
// Auth model: recipient signs the canonical (address, sinceCursor,
// signedAt) tuple. Server verifies against the address's registered
// signing key. Cursor is opaque — clients pass back the highest
// `receivedAt` seen so far.
app.post('/v1/inbox/:address/fetch', async (c) => {
const address = validateAddress(c.req.param('address'));
if (rateLimitEnabled) await fetchRL.consume(`inbox-fetch:${address}`);
const owner = await store.getAddressOwner(address);
if (!owner) {
return c.json({ error: 'Address not registered', code: 'SHADE_NOT_FOUND' }, 404);
}
const rawBody = await c.req.text();
if (rawBody.length > MAX_META_BODY_SIZE) {
throw new ValidationError(`Request body too large`);
}
const body = JSON.parse(rawBody);
let { sinceCursor } = body;
if (sinceCursor === undefined || sinceCursor === null) sinceCursor = 0;
if (typeof sinceCursor !== 'number' || !Number.isFinite(sinceCursor) || sinceCursor < 0) {
throw new ValidationError('sinceCursor must be a non-negative number', 'sinceCursor');
}
// Bind the address to the signed payload to prevent cross-address replay.
await verifyPayload(crypto, owner, { ...body, address });
const now = Date.now();
const rows = await store.fetchBlobs({
address,
sinceCursor,
now,
limit: quota.fetchPageLimit,
});
let bytes = 0;
const blobs = rows.map((r) => {
bytes += r.ciphertext.length;
return {
msgId: r.msgId,
ciphertext: toBase64(r.ciphertext),
receivedAt: r.receivedAt,
expiresAt: r.expiresAt,
};
});
const nextCursor = rows.length > 0 ? rows[rows.length - 1]!.receivedAt : sinceCursor;
events?.emit('inbox.blob_fetched', { address, count: blobs.length, bytes });
return c.json({
blobs,
cursor: nextCursor,
hasMore: rows.length === quota.fetchPageLimit,
});
});
// ─── DELETE a single blob (signed challenge by recipient) ──
app.delete('/v1/inbox/:address/:msgId', async (c) => {
const address = validateAddress(c.req.param('address'));
if (rateLimitEnabled) await deleteRL.consume(`inbox-delete:${address}`);
const msgId = c.req.param('msgId');
if (!isValidMsgId(msgId)) {
throw new ValidationError('msgId must be 64 lowercase hex chars', 'msgId');
}
const owner = await store.getAddressOwner(address);
if (!owner) {
return c.json({ error: 'Address not registered', code: 'SHADE_NOT_FOUND' }, 404);
}
const body = await c.req.json();
await verifyPayload(crypto, owner, { ...body, address, msgId });
const removed = await store.deleteBlob(address, msgId);
if (removed) {
events?.emit('inbox.blob_acked', { address, msgId });
}
return c.json({ ok: removed });
});
return app;
}
// ─── Base64 helpers ──────────────────────────────────────────
function b64ToBytes(s: string): Uint8Array {
return fromBase64(s);
}

View File

@@ -0,0 +1,87 @@
/**
* InboxStore — server-side storage interface for the async store-and-forward
* relay (V3.6).
*
* The relay stores ciphertext blobs only. It never sees plaintext, never
* holds private keys, and never decrypts anything. The address-owner table
* binds an address to a recipient signing key (Ed25519) so GET/DELETE
* authentication can verify that the caller actually owns the address.
*
* Per-blob row layout (mandated by V3.6 spec):
* address || msgId || ciphertext-bytes || expires_at
*
* `received_at` is also stored per blob to support cursor-based GET. The
* cursor is opaque to the caller — it is just the highest `received_at`
* the client has seen, encoded as a string.
*/
export interface InboxStore {
// ─── Address ownership (TOFU on first register) ───────────
/**
* Register or update the signing key that owns `address`. First call
* "claims" the address (TOFU). Subsequent calls with the same key are
* no-ops. A different key trying to claim the same address is rejected
* by the route layer (the store itself just upserts).
*/
saveAddressOwner(address: string, signingKey: Uint8Array): Promise<void>;
/** Look up the owner signing key for `address`, or null if unregistered. */
getAddressOwner(address: string): Promise<Uint8Array | null>;
// ─── Blob CRUD ───────────────────────────────────────────
/**
* Store an inbox blob.
*
* **Idempotent**: if a row already exists for `(address, msgId)` the
* implementation MUST return `{ created: false }` and leave the existing
* row untouched. A fresh insert returns `{ created: true, receivedAt }`.
*/
putBlob(args: {
address: string;
msgId: string;
ciphertext: Uint8Array;
expiresAt: number;
}): Promise<{ created: boolean; receivedAt: number }>;
/**
* Fetch all non-expired blobs for `address` whose `received_at > sinceCursor`,
* ordered ascending by `received_at`. The caller passes 0 (or `null`-equiv
* "") to fetch from the beginning.
*
* Implementations must filter out expired rows so a slow consumer never
* sees a payload past TTL. Pruning of expired rows happens out-of-band.
*/
fetchBlobs(args: {
address: string;
sinceCursor: number;
now: number;
limit: number;
}): Promise<Array<{ msgId: string; ciphertext: Uint8Array; receivedAt: number; expiresAt: number }>>;
/**
* Delete a single blob by `(address, msgId)`. Returns true if a row was
* removed. Used by clients to ack a fetched message for early prune.
*/
deleteBlob(address: string, msgId: string): Promise<boolean>;
/**
* Count non-expired blobs for `address`. Used by quota enforcement and
* the inbox-fanout fingerprint gate.
*/
countBlobs(address: string, now: number): Promise<number>;
// ─── Maintenance ─────────────────────────────────────────
/**
* Purge every blob whose `expires_at <= now`. Returns count removed.
* Called periodically by a cron task.
*/
purgeExpired(now: number): Promise<number>;
/**
* Drop the address owner record and any remaining blobs for `address`.
* Used by the unregister route.
*/
deleteAddress(address: string): Promise<void>;
}

View File

@@ -0,0 +1,260 @@
import { describe, test, expect } from 'bun:test';
import {
createInboxServer,
MemoryInboxStore,
computeMsgId,
InboxPruneTask,
} from '../src/index.js';
import { signPayload } from '@shade/server';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { generateIdentityKeyPair, toBase64, fromBase64 } from '@shade/core';
const crypto = new SubtleCryptoProvider();
async function makeIdentity() {
return generateIdentityKeyPair(crypto);
}
function randBytes(n: number): Uint8Array {
const buf = new Uint8Array(n);
globalThis.crypto.getRandomValues(buf);
return buf;
}
describe('Inbox lifecycle', () => {
test('100 messages delivered without online overlap', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const bob = await makeIdentity();
const alice = await makeIdentity();
// Bob registers, then goes "offline".
const reg = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
await app.request('/v1/inbox/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reg),
});
// Alice puts 100 unique blobs while Bob is offline.
const sentMsgIds = new Set<string>();
for (let i = 0; i < 100; i++) {
const ct = randBytes(64 + (i % 8));
const msgId = await computeMsgId(ct);
sentMsgIds.add(msgId);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const r = await app.request(`/v1/inbox/bob`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
expect(r.status).toBe(200);
}
// Bob comes online and pulls everything in pages.
const seen = new Set<string>();
let cursor = 0;
let safety = 0;
while (safety++ < 50) {
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
sinceCursor: cursor,
});
const r = await app.request(`/v1/inbox/bob/fetch`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(fetchBody),
});
const j: any = await r.json();
for (const b of j.blobs) seen.add(b.msgId);
cursor = j.cursor;
if (!j.hasMore) break;
}
expect(seen.size).toBe(100);
for (const msgId of sentMsgIds) expect(seen.has(msgId)).toBe(true);
});
test('persistence across "restart" — same store, fresh app object', async () => {
const store = new MemoryInboxStore();
const bob = await makeIdentity();
const alice = await makeIdentity();
// Stage 1: register + put 5 blobs.
{
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const reg = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
await app.request('/v1/inbox/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reg),
});
for (let i = 0; i < 5; i++) {
const ct = randBytes(48 + i);
const msgId = await computeMsgId(ct);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const r = await app.request(`/v1/inbox/bob`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
expect(r.status).toBe(200);
}
}
// Stage 2: simulate a restart by building a brand-new Hono app on top
// of the same persistent store, then verify fetches still see the data.
{
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
sinceCursor: 0,
});
const r = await app.request('/v1/inbox/bob/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(fetchBody),
});
const j: any = await r.json();
expect(j.blobs.length).toBe(5);
}
});
test('prune removes expired blobs but keeps live ones', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const bob = await makeIdentity();
const alice = await makeIdentity();
const reg = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
await app.request('/v1/inbox/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reg),
});
// One blob with min TTL, one with default TTL (well in future).
const shortCt = randBytes(64);
const shortMsgId = await computeMsgId(shortCt);
const shortBody = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId: shortMsgId,
ciphertext: toBase64(shortCt),
ttlSeconds: 60,
});
await app.request('/v1/inbox/bob', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(shortBody),
});
const longCt = randBytes(64);
const longMsgId = await computeMsgId(longCt);
const longBody = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId: longMsgId,
ciphertext: toBase64(longCt),
});
await app.request('/v1/inbox/bob', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(longBody),
});
// Force-expire the short blob by mutating expires_at.
const list: any = (store as any).blobs.get('bob');
list[0].expiresAt = Date.now() - 1000;
const prune = new InboxPruneTask(store, { intervalMinutes: 60 });
const removed = await prune.runOnce();
expect(removed).toBe(1);
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
sinceCursor: 0,
});
const r = await app.request('/v1/inbox/bob/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(fetchBody),
});
const j: any = await r.json();
expect(j.blobs.length).toBe(1);
expect(j.blobs[0].msgId).toBe(longMsgId);
});
});
describe('Tamper resistance', () => {
test('bit-flip on stored ciphertext is reported as decode/decrypt failure on the client', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const bob = await makeIdentity();
const alice = await makeIdentity();
const reg = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
await app.request('/v1/inbox/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reg),
});
const ct = randBytes(64);
const msgId = await computeMsgId(ct);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const putRes = await app.request('/v1/inbox/bob', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
expect(putRes.status).toBe(200);
// Tamper directly in the store.
const blobs: any = (store as any).blobs.get('bob');
blobs[0].ciphertext[5] ^= 0x01;
// Fetch returns the tampered blob — server is oblivious to integrity.
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
sinceCursor: 0,
});
const r = await app.request('/v1/inbox/bob/fetch', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(fetchBody),
});
const j: any = await r.json();
expect(j.blobs.length).toBe(1);
// Recipient recomputes msgId; tampered ciphertext now hashes to a
// value different from the stored msgId — that's the client-side
// canary the V3.6 spec requires.
const tampered = fromBase64(j.blobs[0].ciphertext);
const recomputed = await computeMsgId(tampered);
expect(recomputed).not.toBe(msgId);
});
});

View File

@@ -0,0 +1,383 @@
import { describe, test, expect, beforeEach } from 'bun:test';
import { Hono } from 'hono';
import {
createInboxServer,
MemoryInboxStore,
computeMsgId,
type InboxStore,
} from '../src/index.js';
import { signPayload } from '@shade/server';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { generateIdentityKeyPair, toBase64 } from '@shade/core';
const crypto = new SubtleCryptoProvider();
async function makeIdentity() {
return generateIdentityKeyPair(crypto);
}
function randBytes(n: number): Uint8Array {
const buf = new Uint8Array(n);
globalThis.crypto.getRandomValues(buf);
return buf;
}
describe('Shade Inbox Server', () => {
let store: InboxStore;
let app: Hono;
beforeEach(() => {
store = new MemoryInboxStore();
app = createInboxServer({ crypto, store, disableRateLimit: true });
});
function req(method: string, path: string, body?: any) {
const init: RequestInit = { method, headers: { 'Content-Type': 'application/json' } };
if (body !== undefined) init.body = JSON.stringify(body);
return app.request(path, init);
}
async function registerBob(address = 'bob') {
const bob = await makeIdentity();
const body = await signPayload(crypto, bob.signingPrivateKey, {
address,
signingKey: toBase64(bob.signingPublicKey),
});
const res = await req('POST', '/v1/inbox/register', body);
expect(res.status).toBe(200);
return bob;
}
async function putMsg(args: {
sender: Awaited<ReturnType<typeof makeIdentity>>;
recipient: string;
ciphertext: Uint8Array;
ttlSeconds?: number;
}) {
const msgId = await computeMsgId(args.ciphertext);
const body: Record<string, unknown> = {
senderSigningKey: toBase64(args.sender.signingPublicKey),
msgId,
ciphertext: toBase64(args.ciphertext),
};
if (args.ttlSeconds !== undefined) body.ttlSeconds = args.ttlSeconds;
const signed = await signPayload(crypto, args.sender.signingPrivateKey, body);
const res = await req('POST', `/v1/inbox/${args.recipient}`, signed);
return { res, msgId };
}
// ─── Health ─────────────────────────────────────────────────
test('health endpoint responds', async () => {
const res = await req('GET', '/health');
expect(res.status).toBe(200);
const json = await res.json();
expect(json.service).toBe('shade-inbox-server');
});
// ─── Registration (TOFU) ────────────────────────────────────
describe('POST /v1/inbox/register', () => {
test('accepts valid registration', async () => {
await registerBob();
});
test('idempotent re-register with same key', async () => {
const bob = await registerBob('bob');
const body = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
const res = await req('POST', '/v1/inbox/register', body);
expect(res.status).toBe(200);
});
test('rejects different key claiming same address', async () => {
await registerBob('bob');
const eve = await makeIdentity();
const body = await signPayload(crypto, eve.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(eve.signingPublicKey),
});
const res = await req('POST', '/v1/inbox/register', body);
expect(res.status).toBe(401);
});
test('rejects unsigned body', async () => {
const bob = await makeIdentity();
const res = await req('POST', '/v1/inbox/register', {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
expect(res.status).toBe(400);
});
test('rejects bad signature', async () => {
const bob = await makeIdentity();
const res = await req('POST', '/v1/inbox/register', {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
signedAt: Date.now(),
signature: toBase64(randBytes(64)),
});
expect(res.status).toBe(401);
});
});
// ─── PUT blob ───────────────────────────────────────────────
describe('POST /v1/inbox/:address (PUT blob)', () => {
test('stores a signed blob from sender', async () => {
await registerBob();
const alice = await makeIdentity();
const ct = randBytes(128);
const { res, msgId } = await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct });
expect(res.status).toBe(200);
const json = await res.json();
expect(json.msgId).toBe(msgId);
expect(json.idempotent).toBe(false);
});
test('idempotent on duplicate ciphertext', async () => {
await registerBob();
const alice = await makeIdentity();
const ct = randBytes(64);
const first = await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct });
expect(first.res.status).toBe(200);
const second = await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct });
expect(second.res.status).toBe(200);
const j2 = await second.res.json();
expect(j2.idempotent).toBe(true);
expect(j2.msgId).toBe(first.msgId);
});
test('rejects mismatched msgId', async () => {
await registerBob();
const alice = await makeIdentity();
const ct = randBytes(64);
const wrongId = '0'.repeat(64);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId: wrongId,
ciphertext: toBase64(ct),
});
const res = await req('POST', '/v1/inbox/bob', body);
expect(res.status).toBe(400);
});
test('rejects PUT to unregistered address', async () => {
const alice = await makeIdentity();
const ct = randBytes(64);
const { res } = await putMsg({ sender: alice, recipient: 'nobody', ciphertext: ct });
expect(res.status).toBe(404);
});
test('rejects bad sender signature', async () => {
await registerBob();
const alice = await makeIdentity();
const eve = await makeIdentity();
const ct = randBytes(64);
const msgId = await computeMsgId(ct);
// Sign with Eve, claim Alice's key.
const body = await signPayload(crypto, eve.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const res = await req('POST', '/v1/inbox/bob', body);
expect(res.status).toBe(401);
});
test('rejects ciphertext > maxBlobBytes', async () => {
const small = createInboxServer({
crypto,
store: new MemoryInboxStore(),
disableRateLimit: true,
quota: { maxBlobBytes: 256 },
});
// Register bob in this fresh app.
const bob = await makeIdentity();
const reg = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
await small.request('/v1/inbox/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reg),
});
const alice = await makeIdentity();
const ct = randBytes(257);
const msgId = await computeMsgId(ct);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const res = await small.request('/v1/inbox/bob', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
expect(res.status).toBe(400);
});
test('rejects stale signature (replay window)', async () => {
await registerBob();
const alice = await makeIdentity();
const ct = randBytes(64);
const msgId = await computeMsgId(ct);
// Hand-craft: sign normally, then mutate signedAt to 10 minutes ago.
const signed = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
(signed as any).signedAt = Date.now() - 10 * 60 * 1000;
const res = await req('POST', '/v1/inbox/bob', signed);
// signedAt mutated → signature invalid → 401, OR replay → 409.
expect([401, 409]).toContain(res.status);
});
test('enforces per-address quota', async () => {
const small = createInboxServer({
crypto,
store: new MemoryInboxStore(),
disableRateLimit: true,
quota: { maxBlobsPerAddress: 2 },
});
const bob = await makeIdentity();
const reg = await signPayload(crypto, bob.signingPrivateKey, {
address: 'bob',
signingKey: toBase64(bob.signingPublicKey),
});
await small.request('/v1/inbox/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reg),
});
const alice = await makeIdentity();
for (let i = 0; i < 2; i++) {
const ct = randBytes(32 + i);
const msgId = await computeMsgId(ct);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const r = await small.request('/v1/inbox/bob', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
expect(r.status).toBe(200);
}
// Third should be quota-rejected.
const ct = randBytes(99);
const msgId = await computeMsgId(ct);
const body = await signPayload(crypto, alice.signingPrivateKey, {
senderSigningKey: toBase64(alice.signingPublicKey),
msgId,
ciphertext: toBase64(ct),
});
const r = await small.request('/v1/inbox/bob', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
expect(r.status).toBe(400);
});
});
// ─── FETCH ──────────────────────────────────────────────────
describe('POST /v1/inbox/:address/fetch', () => {
test('returns blobs after registration', async () => {
const bob = await registerBob();
const alice = await makeIdentity();
const ct1 = randBytes(64);
const ct2 = randBytes(80);
await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct1 });
await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct2 });
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, { address: 'bob', sinceCursor: 0 });
const res = await req('POST', '/v1/inbox/bob/fetch', fetchBody);
expect(res.status).toBe(200);
const json = await res.json();
expect(json.blobs.length).toBe(2);
expect(typeof json.cursor).toBe('number');
});
test('cursor pagination skips already-seen blobs', async () => {
const bob = await registerBob();
const alice = await makeIdentity();
await putMsg({ sender: alice, recipient: 'bob', ciphertext: randBytes(20) });
const firstFetch = await signPayload(crypto, bob.signingPrivateKey, { address: 'bob', sinceCursor: 0 });
const r1 = await req('POST', '/v1/inbox/bob/fetch', firstFetch);
const j1 = await r1.json();
const cursor = j1.cursor;
expect(j1.blobs.length).toBe(1);
// Add a second blob.
await putMsg({ sender: alice, recipient: 'bob', ciphertext: randBytes(30) });
const secondFetch = await signPayload(crypto, bob.signingPrivateKey, { address: 'bob', sinceCursor: cursor });
const r2 = await req('POST', '/v1/inbox/bob/fetch', secondFetch);
const j2 = await r2.json();
expect(j2.blobs.length).toBe(1);
});
test('rejects fetch from a different signing key', async () => {
await registerBob();
const eve = await makeIdentity();
const fetchBody = await signPayload(crypto, eve.signingPrivateKey, { address: 'bob', sinceCursor: 0 });
const res = await req('POST', '/v1/inbox/bob/fetch', fetchBody);
expect(res.status).toBe(401);
});
test('rejects fetch on unregistered address', async () => {
const eve = await makeIdentity();
const fetchBody = await signPayload(crypto, eve.signingPrivateKey, { address: 'nobody', sinceCursor: 0 });
const res = await req('POST', '/v1/inbox/nobody/fetch', fetchBody);
expect(res.status).toBe(404);
});
});
// ─── DELETE / ack ───────────────────────────────────────────
describe('DELETE /v1/inbox/:address/:msgId', () => {
test('removes a blob after ack', async () => {
const bob = await registerBob();
const alice = await makeIdentity();
const ct = randBytes(64);
const { msgId } = await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct });
const ackBody = await signPayload(crypto, bob.signingPrivateKey, { address: 'bob', msgId });
const res = await req('DELETE', `/v1/inbox/bob/${msgId}`, ackBody);
expect(res.status).toBe(200);
const j = await res.json();
expect(j.ok).toBe(true);
// Subsequent fetch should return zero.
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, { address: 'bob', sinceCursor: 0 });
const r2 = await req('POST', '/v1/inbox/bob/fetch', fetchBody);
const j2 = await r2.json();
expect(j2.blobs.length).toBe(0);
});
test('rejects ack from a different signing key', async () => {
const bob = await registerBob();
const alice = await makeIdentity();
const eve = await makeIdentity();
const ct = randBytes(64);
const { msgId } = await putMsg({ sender: alice, recipient: 'bob', ciphertext: ct });
const ackBody = await signPayload(crypto, eve.signingPrivateKey, { address: 'bob', msgId });
const res = await req('DELETE', `/v1/inbox/bob/${msgId}`, ackBody);
expect(res.status).toBe(401);
// and the blob must still be there
const fetchBody = await signPayload(crypto, bob.signingPrivateKey, { address: 'bob', sinceCursor: 0 });
const r2 = await req('POST', '/v1/inbox/bob/fetch', fetchBody);
const j2 = await r2.json();
expect(j2.blobs.length).toBe(1);
});
});
});

View File

@@ -0,0 +1,8 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src"
},
"include": ["src"]
}

View File

@@ -0,0 +1,16 @@
{
"name": "@shade/inbox",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/proto": "workspace:*",
"@shade/server": "workspace:*"
},
"devDependencies": {
"@shade/crypto-web": "workspace:*",
"@shade/inbox-server": "workspace:*"
}
}

View File

@@ -0,0 +1,219 @@
import type { CryptoProvider, ShadeEnvelope } from '@shade/core';
import {
NetworkError,
toBase64,
fromBase64,
ShadeError,
ValidationError,
} from '@shade/core';
import { signPayload } from '@shade/server';
import { encodeEnvelope, decodeEnvelope } from '@shade/proto';
import { computeMsgId } from './msg-id.js';
/**
* Low-level HTTP client for `@shade/inbox-server`.
*
* Stateless and reusable across many recipients. Higher-level orchestration
* (queue, poll loop, ack-on-decrypt) lives in `Inbox` (see `inbox.ts`),
* which composes this client.
*/
export interface InboxClientOptions {
baseUrl: string;
crypto: CryptoProvider;
/** Used to sign requests on behalf of *this* identity. */
signingPrivateKey: Uint8Array;
/** Optional fetch override (defaults to globalThis.fetch). */
fetch?: typeof fetch;
}
export interface PutResult {
msgId: string;
receivedAt: number;
idempotent: boolean;
}
export interface FetchedBlob {
msgId: string;
/** Wire-encoded ShadeEnvelope bytes. */
ciphertext: Uint8Array;
/** Server-assigned monotonic timestamp; pass back as `sinceCursor`. */
receivedAt: number;
/** Absolute expiry time (ms since epoch) reported by the server. */
expiresAt: number;
}
export interface FetchResult {
blobs: FetchedBlob[];
cursor: number;
hasMore: boolean;
}
export class InboxClient {
private readonly fetchImpl: typeof fetch;
constructor(private readonly options: InboxClientOptions) {
this.fetchImpl = options.fetch ?? globalThis.fetch;
}
/**
* TOFU-register the address that this signing key will own.
* Idempotent if the same key re-registers; rejected by the server if a
* different key has already claimed the address.
*/
async register(args: {
address: string;
signingKey: Uint8Array;
}): Promise<void> {
const body = await signPayload(this.options.crypto, this.options.signingPrivateKey, {
address: args.address,
signingKey: toBase64(args.signingKey),
});
await this.postJson(`/v1/inbox/register`, body);
}
/**
* Unregister the address. Drops every queued blob. Signed by the
* registered signing key.
*/
async unregister(address: string): Promise<void> {
const body = await signPayload(this.options.crypto, this.options.signingPrivateKey, {
address,
});
await this.requestJson('DELETE', `/v1/inbox/register/${encodeURIComponent(address)}`, body);
}
/**
* PUT a ShadeEnvelope to a recipient's inbox. Idempotent: same
* ciphertext yields the same msgId and the server folds the second PUT
* into a 200 with `idempotent: true`.
*/
async put(args: {
/** Recipient address (inbox owner). */
recipientAddress: string;
/** Sender's identity signing public key (the one matching `signingPrivateKey`). */
senderSigningKey: Uint8Array;
/** Encrypted Shade envelope. Either pre-encoded bytes or a parsed envelope. */
envelope: ShadeEnvelope | Uint8Array;
ttlSeconds?: number;
}): Promise<PutResult> {
const ciphertext =
args.envelope instanceof Uint8Array ? args.envelope : encodeEnvelope(args.envelope);
if (ciphertext.length === 0) {
throw new ValidationError('Empty ciphertext');
}
const msgId = await computeMsgId(ciphertext);
const payload: Record<string, unknown> = {
senderSigningKey: toBase64(args.senderSigningKey),
msgId,
ciphertext: toBase64(ciphertext),
};
if (args.ttlSeconds !== undefined) payload.ttlSeconds = args.ttlSeconds;
const signed = await signPayload(
this.options.crypto,
this.options.signingPrivateKey,
payload,
);
const json = await this.postJson(
`/v1/inbox/${encodeURIComponent(args.recipientAddress)}`,
signed,
);
return {
msgId: String(json.msgId),
receivedAt: Number(json.receivedAt),
idempotent: Boolean(json.idempotent),
};
}
/**
* Fetch all blobs for `address` whose `received_at > sinceCursor`.
* Returns at most one server page; clients keep calling with the new
* cursor until `hasMore === false`.
*/
async fetch(args: { address: string; sinceCursor?: number }): Promise<FetchResult> {
const sinceCursor = args.sinceCursor ?? 0;
const body = await signPayload(this.options.crypto, this.options.signingPrivateKey, {
address: args.address,
sinceCursor,
});
const json = await this.postJson(
`/v1/inbox/${encodeURIComponent(args.address)}/fetch`,
body,
);
const blobs = Array.isArray(json.blobs) ? json.blobs : [];
return {
blobs: blobs.map((b: any) => ({
msgId: String(b.msgId),
ciphertext: fromBase64(String(b.ciphertext)),
receivedAt: Number(b.receivedAt),
expiresAt: Number(b.expiresAt),
})),
cursor: Number(json.cursor ?? sinceCursor),
hasMore: Boolean(json.hasMore),
};
}
/**
* Acknowledge — delete a fetched blob. Should be called after the
* caller has successfully decrypted (or persisted) the message.
*/
async ack(args: { address: string; msgId: string }): Promise<boolean> {
const body = await signPayload(this.options.crypto, this.options.signingPrivateKey, {
address: args.address,
msgId: args.msgId,
});
const json = await this.requestJson(
'DELETE',
`/v1/inbox/${encodeURIComponent(args.address)}/${encodeURIComponent(args.msgId)}`,
body,
);
return Boolean(json.ok);
}
// ─── HTTP plumbing ──────────────────────────────────────────
private async postJson(path: string, body: unknown): Promise<any> {
return this.requestJson('POST', path, body);
}
private async requestJson(method: string, path: string, body: unknown): Promise<any> {
const url = joinUrl(this.options.baseUrl, path);
let res: Response;
try {
res = await this.fetchImpl(url, {
method,
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
});
} catch (err) {
throw new NetworkError(`Inbox request failed: ${(err as Error).message}`);
}
const text = await res.text();
let json: any;
try {
json = text.length > 0 ? JSON.parse(text) : {};
} catch {
throw new NetworkError(`Inbox response not JSON: ${text.slice(0, 200)}`, res.status);
}
if (!res.ok) {
// Surface server-mapped Shade errors in their original shape.
const code = String(json.code ?? '');
const message = String(json.message ?? text);
throw new ShadeError(code || 'SHADE_NETWORK', message);
}
return json;
}
}
function joinUrl(base: string, path: string): string {
if (base.endsWith('/') && path.startsWith('/')) return base + path.slice(1);
if (!base.endsWith('/') && !path.startsWith('/')) return base + '/' + path;
return base + path;
}
/** Decode a fetched blob into a ShadeEnvelope. */
export function decodeFetchedEnvelope(b: FetchedBlob): ShadeEnvelope {
return decodeEnvelope(b.ciphertext);
}

View File

@@ -0,0 +1,26 @@
/**
* Persistent receive cursor.
*
* Tracks the highest `received_at` we've consumed from the inbox per
* (ownAddress, ourSelf-poll context). The InboxClient pulls everything
* strictly greater than this on each poll.
*
* The default in-memory implementation is sufficient for short-lived
* processes (CLIs, tests). Long-lived services should plug in a SQLite,
* Postgres, or IndexedDB backed store so a restart doesn't redownload all
* messages still in TTL.
*/
export interface CursorStore {
load(address: string): Promise<number>;
save(address: string, cursor: number): Promise<void>;
}
export class MemoryCursorStore implements CursorStore {
private cursors = new Map<string, number>();
async load(address: string): Promise<number> {
return this.cursors.get(address) ?? 0;
}
async save(address: string, cursor: number): Promise<void> {
this.cursors.set(address, cursor);
}
}

View File

@@ -0,0 +1,72 @@
/**
* Client-side inbox event emitter.
*
* The high-level `Inbox` orchestrator emits structural events — message
* queued, delivered, polled, decrypted — so apps can drive UI badges,
* push hooks, or telemetry without polling internal state.
*/
export interface InboxClientEventMap {
/**
* A new ciphertext entry was added to the outgoing queue. The push-hook
* mentioned in the V3.6 spec dispatches off this event.
*/
'inbox.message_queued': {
recipientAddress: string;
msgId: string;
bytes: number;
ttlSeconds: number;
};
/** Server confirmed a queued blob landed. */
'inbox.message_delivered': {
recipientAddress: string;
msgId: string;
idempotent: boolean;
};
/** Delivery attempt failed; will retry on next flush. */
'inbox.message_failed': {
recipientAddress: string;
msgId: string;
attempts: number;
error: string;
};
/** A poll cycle completed and pulled `count` blobs. */
'inbox.poll_completed': { ownAddress: string; count: number; cursor: number };
/** Caller successfully decrypted and acked a blob. */
'inbox.message_received': {
senderHint: string | null;
msgId: string;
};
/** Caller failed to decrypt — typically tampering or stale ratchet. */
'inbox.message_decrypt_failed': {
msgId: string;
error: string;
};
}
export type InboxClientEventName = keyof InboxClientEventMap;
export type InboxClientEvent = {
[K in InboxClientEventName]: { name: K; data: InboxClientEventMap[K]; timestamp: number };
}[InboxClientEventName];
export type InboxClientListener = (e: InboxClientEvent) => void;
export class InboxClientEvents {
private readonly listeners = new Set<InboxClientListener>();
on(listener: InboxClientListener): () => void {
this.listeners.add(listener);
return () => this.listeners.delete(listener);
}
emit<K extends InboxClientEventName>(name: K, data: InboxClientEventMap[K]): void {
const event = { name, data, timestamp: Date.now() } as InboxClientEvent;
for (const l of this.listeners) {
try {
l(event);
} catch (err) {
console.error('[Shade] Inbox client listener threw:', err);
}
}
}
}

View File

@@ -0,0 +1,407 @@
import type { CryptoProvider, ShadeEnvelope } from '@shade/core';
import { encodeEnvelope } from '@shade/proto';
import { InboxClient, type FetchedBlob } from './client.js';
import {
MemoryOutgoingQueueStore,
type OutgoingEntry,
type OutgoingQueueStore,
} from './queue-store.js';
import { MemoryCursorStore, type CursorStore } from './cursor-store.js';
import { computeMsgId } from './msg-id.js';
import { InboxClientEvents, type InboxClientListener } from './events.js';
/**
* Caller-supplied incoming-blob handler. Receives raw wire bytes; the app
* is expected to call `decodeEnvelope` + `Shade.receive` (or whatever
* decrypt path it owns) and either return a sender-hint for telemetry
* (the address the SDK extracted, or `null`) or throw to keep the blob
* on the server for a later retry.
*/
export type DecryptHandler = (
raw: { msgId: string; ciphertext: Uint8Array; receivedAt: number; expiresAt: number },
) => Promise<string | null | undefined> | string | null | undefined;
export interface InboxOptions {
/** Inbox-server base URL (e.g. "https://inbox.example.com"). */
baseUrl: string;
/** Recipient address — the address that owns this queue. */
ownAddress: string;
/** Crypto provider — used for signing requests. */
crypto: CryptoProvider;
/** Identity signing private key. */
signingPrivateKey: Uint8Array;
/** Identity signing public key. */
signingPublicKey: Uint8Array;
/**
* Default TTL for outgoing PUTs (seconds). The server clamps to its own
* min/max. Defaults to 7 days as mandated by the V3.6 spec.
*/
defaultTtlSeconds?: number;
/** Polling interval (ms). Default: 30s. Set to 0 to disable auto-poll. */
pollIntervalMs?: number;
/** Outgoing queue persistence. Defaults to in-memory. */
queueStore?: OutgoingQueueStore;
/** Cursor persistence. Defaults to in-memory. */
cursorStore?: CursorStore;
/** Override the underlying fetch (tests). */
fetch?: typeof fetch;
/** Maximum delivery attempts before dropping a queued entry. Default: 10. */
maxAttempts?: number;
}
const DEFAULT_TTL_SECONDS = 7 * 24 * 60 * 60;
const DEFAULT_POLL_INTERVAL_MS = 30_000;
const DEFAULT_MAX_ATTEMPTS = 10;
/**
* High-level inbox orchestrator.
*
* Responsibilities:
* - Outgoing: serialize ciphertext → queue → flush to server with retry.
* - Incoming: poll → decrypt-handler → ack on success.
* - Push hook: `onMessageQueued(handler)` lets a transport vendor (FCM,
* APNs) wake the recipient out-of-band when a blob is enqueued for them
* here. The hook is called with the recipient address — the actual
* push-delivery wire is left to the integrator.
*
* The class never *encrypts* — that's `Shade.send`'s job. Apps wire it up
* like:
*
* const envelope = await shade.send(bob, "hi");
* await inbox.send(bob, envelope);
*
* On Bob's device:
*
* inbox.onIncoming(async (env) => {
* await shade.receive(senderAddress, env);
* });
* inbox.start();
*/
export class Inbox {
private readonly client: InboxClient;
private readonly queueStore: OutgoingQueueStore;
private readonly cursorStore: CursorStore;
private readonly events = new InboxClientEvents();
private readonly defaultTtlSeconds: number;
private readonly pollIntervalMs: number;
private readonly maxAttempts: number;
private incomingHandler: DecryptHandler | null = null;
private flushTimer: ReturnType<typeof setTimeout> | null = null;
private pollTimer: ReturnType<typeof setTimeout> | null = null;
private flushing = false;
private polling = false;
private started = false;
private registered = false;
constructor(private readonly options: InboxOptions) {
const clientOptions: ConstructorParameters<typeof InboxClient>[0] = {
baseUrl: options.baseUrl,
crypto: options.crypto,
signingPrivateKey: options.signingPrivateKey,
};
if (options.fetch !== undefined) clientOptions.fetch = options.fetch;
this.client = new InboxClient(clientOptions);
this.queueStore = options.queueStore ?? new MemoryOutgoingQueueStore();
this.cursorStore = options.cursorStore ?? new MemoryCursorStore();
this.defaultTtlSeconds = options.defaultTtlSeconds ?? DEFAULT_TTL_SECONDS;
this.pollIntervalMs = options.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS;
this.maxAttempts = options.maxAttempts ?? DEFAULT_MAX_ATTEMPTS;
}
/** Subscribe to client events (queued / delivered / received / failed). */
on(listener: InboxClientListener): () => void {
return this.events.on(listener);
}
/**
* Push-hook: invoked once per new blob enqueued for delivery. Vendors
* implement FCM/APNs/email wake-ups inside the handler. The handler
* fires *after* the blob is durably in the local queue but before the
* server has confirmed PUT.
*/
onMessageQueued(handler: (recipientAddress: string, msgId: string) => void | Promise<void>): () => void {
return this.events.on((e) => {
if (e.name === 'inbox.message_queued') {
Promise.resolve(handler(e.data.recipientAddress, e.data.msgId)).catch((err) => {
console.error('[Shade] onMessageQueued handler threw:', err);
});
}
});
}
/** Register a handler that processes incoming blobs. Replaces any prior. */
onIncoming(handler: DecryptHandler): () => void {
this.incomingHandler = handler;
return () => {
if (this.incomingHandler === handler) this.incomingHandler = null;
};
}
/**
* Idempotent TOFU registration with the server. Called automatically by
* `start()`; can also be invoked directly e.g. during onboarding.
*/
async register(): Promise<void> {
if (this.registered) return;
await this.client.register({
address: this.options.ownAddress,
signingKey: this.options.signingPublicKey,
});
this.registered = true;
}
/** Drop the address from the server. Local queue/cursor are preserved. */
async unregister(): Promise<void> {
await this.client.unregister(this.options.ownAddress);
this.registered = false;
}
/**
* Enqueue an envelope for delivery. The actual PUT happens
* asynchronously — the call returns once the entry is durably in the
* outgoing queue. Returns the deterministic msgId.
*
* `envelope` accepts a `ShadeEnvelope` (the type returned from
* `Shade.send`) or pre-encoded wire bytes (`Uint8Array`).
*/
async send(args: {
recipientAddress: string;
envelope: ShadeEnvelope | Uint8Array;
ttlSeconds?: number;
}): Promise<string> {
const ciphertext =
args.envelope instanceof Uint8Array ? args.envelope : encodeEnvelope(args.envelope);
const msgId = await computeMsgId(ciphertext);
const ttlSeconds = args.ttlSeconds ?? this.defaultTtlSeconds;
const entry: OutgoingEntry = {
recipientAddress: args.recipientAddress,
msgId,
ciphertext,
ttlSeconds,
queuedAt: Date.now(),
attempts: 0,
};
await this.queueStore.enqueue(entry);
this.events.emit('inbox.message_queued', {
recipientAddress: args.recipientAddress,
msgId,
bytes: ciphertext.length,
ttlSeconds,
});
// Kick the flush loop. Don't await — caller doesn't need to block on
// network for a "queued" return.
this.scheduleFlush();
return msgId;
}
/**
* Force a flush + poll cycle now (useful right after registering or
* after a push-trigger arrives). Does not throw on transient errors.
*/
async tick(): Promise<{ flushed: number; received: number }> {
const flushed = await this.flushOnce();
const received = await this.pollOnce();
return { flushed, received };
}
/** Start background flush + poll timers. Idempotent. */
start(): void {
if (this.started) return;
this.started = true;
this.register().catch((err) => {
console.warn('[Shade] Inbox register failed (will retry):', (err as Error).message);
this.scheduleRegisterRetry();
});
this.scheduleFlush();
this.schedulePoll(0);
}
/** Stop background timers. Pending entries remain in the queue. */
stop(): void {
this.started = false;
if (this.flushTimer) {
clearTimeout(this.flushTimer);
this.flushTimer = null;
}
if (this.pollTimer) {
clearTimeout(this.pollTimer);
this.pollTimer = null;
}
}
/** Number of entries currently waiting to be flushed. */
async pendingCount(): Promise<number> {
return this.queueStore.size();
}
// ─── internals ──────────────────────────────────────────────
private scheduleRegisterRetry(): void {
if (!this.started) return;
setTimeout(() => {
if (!this.started) return;
this.register().catch(() => this.scheduleRegisterRetry());
}, 5000);
}
private scheduleFlush(delayMs = 0): void {
if (this.flushTimer) return;
this.flushTimer = setTimeout(() => {
this.flushTimer = null;
this.flushOnce()
.then(() => {
// If anything is still queued, retry with backoff.
this.queueStore.size().then((n) => {
if (n > 0 && this.started) this.scheduleFlush(15_000);
});
})
.catch(() => {
if (this.started) this.scheduleFlush(15_000);
});
}, delayMs);
}
private schedulePoll(delayMs: number): void {
if (!this.started || this.pollIntervalMs === 0) return;
if (this.pollTimer) clearTimeout(this.pollTimer);
this.pollTimer = setTimeout(() => {
this.pollTimer = null;
this.pollOnce()
.catch(() => {})
.finally(() => this.schedulePoll(this.pollIntervalMs));
}, delayMs);
}
private async flushOnce(): Promise<number> {
if (this.flushing) return 0;
this.flushing = true;
let delivered = 0;
try {
const entries = await this.queueStore.list();
for (const entry of entries) {
try {
const result = await this.client.put({
recipientAddress: entry.recipientAddress,
senderSigningKey: this.options.signingPublicKey,
envelope: entry.ciphertext,
ttlSeconds: entry.ttlSeconds,
});
await this.queueStore.remove(entry.recipientAddress, entry.msgId);
delivered++;
this.events.emit('inbox.message_delivered', {
recipientAddress: entry.recipientAddress,
msgId: result.msgId,
idempotent: result.idempotent,
});
} catch (err) {
await this.queueStore.bumpAttempts(entry.recipientAddress, entry.msgId);
const attempts = entry.attempts + 1;
this.events.emit('inbox.message_failed', {
recipientAddress: entry.recipientAddress,
msgId: entry.msgId,
attempts,
error: (err as Error).message,
});
if (attempts >= this.maxAttempts) {
await this.queueStore.remove(entry.recipientAddress, entry.msgId);
}
}
}
} finally {
this.flushing = false;
}
return delivered;
}
private async pollOnce(): Promise<number> {
if (this.polling) return 0;
if (!this.incomingHandler) return 0;
this.polling = true;
let total = 0;
try {
let cursor = await this.cursorStore.load(this.options.ownAddress);
// Pull all pages.
// eslint-disable-next-line no-constant-condition
while (true) {
let page;
try {
page = await this.client.fetch({
address: this.options.ownAddress,
sinceCursor: cursor,
});
} catch (err) {
// Surface but don't crash the loop.
console.warn('[Shade] Inbox fetch failed:', (err as Error).message);
break;
}
for (const blob of page.blobs) {
const handled = await this.handleBlob(blob);
if (handled) total++;
if (blob.receivedAt > cursor) cursor = blob.receivedAt;
}
await this.cursorStore.save(this.options.ownAddress, cursor);
this.events.emit('inbox.poll_completed', {
ownAddress: this.options.ownAddress,
count: page.blobs.length,
cursor,
});
if (!page.hasMore) break;
}
} finally {
this.polling = false;
}
return total;
}
private async handleBlob(blob: FetchedBlob): Promise<boolean> {
if (!this.incomingHandler) return false;
// Defense-in-depth: verify msgId ↔ ciphertext at the client too. A
// server bug or malicious operator can't sneak a different blob past
// the client's hash check.
const recomputed = await computeMsgId(blob.ciphertext);
if (recomputed !== blob.msgId) {
this.events.emit('inbox.message_decrypt_failed', {
msgId: blob.msgId,
error: 'msgId/ciphertext mismatch',
});
// Don't ack — let the operator notice the divergence.
return false;
}
let senderHint: string | null = null;
try {
const result = await this.incomingHandler({
msgId: blob.msgId,
ciphertext: blob.ciphertext,
receivedAt: blob.receivedAt,
expiresAt: blob.expiresAt,
});
senderHint = result ?? null;
} catch (err) {
this.events.emit('inbox.message_decrypt_failed', {
msgId: blob.msgId,
error: (err as Error).message,
});
// Don't ack — caller can retry on next poll.
return false;
}
try {
await this.client.ack({ address: this.options.ownAddress, msgId: blob.msgId });
} catch (err) {
// Decryption succeeded; ack just failed. Will be retried later, and
// the duplicate-message ratchet check on `Shade.receive` will dedupe.
console.warn('[Shade] Inbox ack failed (will retry):', (err as Error).message);
}
this.events.emit('inbox.message_received', {
senderHint,
msgId: blob.msgId,
});
return true;
}
}

View File

@@ -0,0 +1,45 @@
export {
InboxClient,
decodeFetchedEnvelope,
} from './client.js';
export type {
InboxClientOptions,
PutResult,
FetchedBlob,
FetchResult,
} from './client.js';
export {
Inbox,
} from './inbox.js';
export type {
InboxOptions,
DecryptHandler,
} from './inbox.js';
export {
MemoryOutgoingQueueStore,
} from './queue-store.js';
export type {
OutgoingEntry,
OutgoingQueueStore,
} from './queue-store.js';
export {
MemoryCursorStore,
} from './cursor-store.js';
export type {
CursorStore,
} from './cursor-store.js';
export {
InboxClientEvents,
} from './events.js';
export type {
InboxClientEvent,
InboxClientEventName,
InboxClientEventMap,
InboxClientListener,
} from './events.js';
export { computeMsgId } from './msg-id.js';

View File

@@ -0,0 +1,14 @@
/**
* Client-side msgId helper. Mirrors `@shade/inbox-server/msg-id` but lives
* in this package so client code doesn't need to import the server bundle.
*
* `msgId = lowercase-hex( sha256(ciphertext) )`
*/
export async function computeMsgId(ciphertext: Uint8Array): Promise<string> {
const buf = await globalThis.crypto.subtle.digest(
'SHA-256',
ciphertext as unknown as ArrayBuffer,
);
const arr = new Uint8Array(buf);
return Array.from(arr, (b) => b.toString(16).padStart(2, '0')).join('');
}

View File

@@ -0,0 +1,77 @@
/**
* Outgoing-queue persistence interface.
*
* The client buffers ciphertext blobs until the inbox server confirms the
* PUT. If the process restarts mid-flush, the queue must survive — so we
* model it as an explicit interface that apps can back with SQLite, IndexedDB,
* AsyncStorage, etc.
*
* Each entry is keyed by `(recipientAddress, msgId)` (the same key the
* server uses) so retries are naturally idempotent.
*/
export interface OutgoingEntry {
/** Recipient address (the inbox owner). */
recipientAddress: string;
/** Hex SHA-256 of `ciphertext` — server-side msg id. */
msgId: string;
/** Wire-encoded ShadeEnvelope to deliver. */
ciphertext: Uint8Array;
/** Time-to-live in seconds. The server clamps to its allowed range. */
ttlSeconds: number;
/** Local timestamp when the entry was queued (ms). */
queuedAt: number;
/** Number of failed delivery attempts so far. */
attempts: number;
}
export interface OutgoingQueueStore {
/** Insert a new entry. Idempotent on (recipientAddress, msgId). */
enqueue(entry: OutgoingEntry): Promise<void>;
/** List entries in insertion order. Caller filters/limits. */
list(): Promise<OutgoingEntry[]>;
/** Remove an entry by composite key. Returns true if found. */
remove(recipientAddress: string, msgId: string): Promise<boolean>;
/** Update the attempts counter for an entry. */
bumpAttempts(recipientAddress: string, msgId: string): Promise<void>;
/** Total queued count. */
size(): Promise<number>;
}
export class MemoryOutgoingQueueStore implements OutgoingQueueStore {
private entries: OutgoingEntry[] = [];
async enqueue(entry: OutgoingEntry): Promise<void> {
const dup = this.entries.some(
(e) => e.recipientAddress === entry.recipientAddress && e.msgId === entry.msgId,
);
if (dup) return;
this.entries.push({ ...entry, ciphertext: new Uint8Array(entry.ciphertext) });
}
async list(): Promise<OutgoingEntry[]> {
return this.entries.map((e) => ({
...e,
ciphertext: new Uint8Array(e.ciphertext),
}));
}
async remove(recipientAddress: string, msgId: string): Promise<boolean> {
const idx = this.entries.findIndex(
(e) => e.recipientAddress === recipientAddress && e.msgId === msgId,
);
if (idx === -1) return false;
this.entries.splice(idx, 1);
return true;
}
async bumpAttempts(recipientAddress: string, msgId: string): Promise<void> {
const e = this.entries.find(
(x) => x.recipientAddress === recipientAddress && x.msgId === msgId,
);
if (e) e.attempts++;
}
async size(): Promise<number> {
return this.entries.length;
}
}

View File

@@ -0,0 +1,283 @@
import { describe, test, expect } from 'bun:test';
import { Inbox, InboxClient, computeMsgId, MemoryOutgoingQueueStore } from '../src/index.js';
import {
createInboxServer,
MemoryInboxStore,
} from '@shade/inbox-server';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { generateIdentityKeyPair } from '@shade/core';
import type { Hono } from 'hono';
const crypto = new SubtleCryptoProvider();
async function makeIdentity() {
return generateIdentityKeyPair(crypto);
}
function randBytes(n: number): Uint8Array {
const buf = new Uint8Array(n);
globalThis.crypto.getRandomValues(buf);
return buf;
}
/**
* Wrap a Hono app as a fetch implementation. Strips the protocol/host so
* `app.request(path, init)` works.
*/
function honoFetch(app: Hono): typeof fetch {
return (async (input: RequestInfo | URL, init?: RequestInit) => {
const url = typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url;
const path = url.startsWith('http://localhost') ? url.slice('http://localhost'.length) : url;
return app.request(path, init);
}) as typeof fetch;
}
describe('InboxClient', () => {
test('register + put + fetch + ack roundtrip', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const bob = await makeIdentity();
const alice = await makeIdentity();
const bobClient = new InboxClient({
baseUrl: 'http://localhost',
crypto,
signingPrivateKey: bob.signingPrivateKey,
fetch: honoFetch(app),
});
const aliceClient = new InboxClient({
baseUrl: 'http://localhost',
crypto,
signingPrivateKey: alice.signingPrivateKey,
fetch: honoFetch(app),
});
await bobClient.register({ address: 'bob', signingKey: bob.signingPublicKey });
const ct = randBytes(64);
const msgId = await computeMsgId(ct);
const result = await aliceClient.put({
recipientAddress: 'bob',
senderSigningKey: alice.signingPublicKey,
envelope: ct,
});
expect(result.msgId).toBe(msgId);
expect(result.idempotent).toBe(false);
const second = await aliceClient.put({
recipientAddress: 'bob',
senderSigningKey: alice.signingPublicKey,
envelope: ct,
});
expect(second.idempotent).toBe(true);
const fetched = await bobClient.fetch({ address: 'bob' });
expect(fetched.blobs.length).toBe(1);
expect(fetched.blobs[0]!.msgId).toBe(msgId);
expect(fetched.blobs[0]!.ciphertext).toEqual(ct);
const acked = await bobClient.ack({ address: 'bob', msgId });
expect(acked).toBe(true);
const second2 = await bobClient.fetch({ address: 'bob' });
expect(second2.blobs.length).toBe(0);
});
});
describe('Inbox orchestrator', () => {
test('queue → flush → server-side blob shows up', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const bob = await makeIdentity();
const alice = await makeIdentity();
const aliceInbox = new Inbox({
baseUrl: 'http://localhost',
ownAddress: 'alice',
crypto,
signingPrivateKey: alice.signingPrivateKey,
signingPublicKey: alice.signingPublicKey,
pollIntervalMs: 0,
fetch: honoFetch(app),
});
const bobInbox = new Inbox({
baseUrl: 'http://localhost',
ownAddress: 'bob',
crypto,
signingPrivateKey: bob.signingPrivateKey,
signingPublicKey: bob.signingPublicKey,
pollIntervalMs: 0,
fetch: honoFetch(app),
});
// Bob registers so he can receive.
await bobInbox.register();
// Alice queues a message.
const ct = randBytes(64);
const msgId = await aliceInbox.send({ recipientAddress: 'bob', envelope: ct });
expect(await aliceInbox.pendingCount()).toBe(1);
// Alice ticks: flushes + (no incoming because no handler).
await aliceInbox.tick();
expect(await aliceInbox.pendingCount()).toBe(0);
// Bob ticks: should see the blob via incoming handler.
let received: { msgId: string; bytes: number } | null = null;
bobInbox.onIncoming(async (raw) => {
received = { msgId: raw.msgId, bytes: raw.ciphertext.length };
return 'alice';
});
const result = await bobInbox.tick();
expect(result.received).toBe(1);
expect(received).not.toBeNull();
expect(received!.msgId).toBe(msgId);
expect(received!.bytes).toBe(ct.length);
// No re-delivery on second tick (cursor advanced + ack performed).
const r2 = await bobInbox.tick();
expect(r2.received).toBe(0);
});
test('onMessageQueued hook fires for each enqueue', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const alice = await makeIdentity();
const inbox = new Inbox({
baseUrl: 'http://localhost',
ownAddress: 'alice',
crypto,
signingPrivateKey: alice.signingPrivateKey,
signingPublicKey: alice.signingPublicKey,
pollIntervalMs: 0,
fetch: honoFetch(app),
});
const seen: Array<{ to: string; msgId: string }> = [];
inbox.onMessageQueued((to, msgId) => {
seen.push({ to, msgId });
});
await inbox.send({ recipientAddress: 'bob', envelope: randBytes(10) });
await inbox.send({ recipientAddress: 'carol', envelope: randBytes(20) });
// Wait for the (sync) hook to flush.
await new Promise((r) => setTimeout(r, 5));
expect(seen.length).toBe(2);
expect(seen[0]!.to).toBe('bob');
expect(seen[1]!.to).toBe('carol');
});
test('flush retries on transient server failure', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const alice = await makeIdentity();
const bob = await makeIdentity();
// Register bob via direct API.
const bobClient = new InboxClient({
baseUrl: 'http://localhost',
crypto,
signingPrivateKey: bob.signingPrivateKey,
fetch: honoFetch(app),
});
await bobClient.register({ address: 'bob', signingKey: bob.signingPublicKey });
// Wrap fetch so first PUT fails, subsequent succeed.
let failsLeft = 1;
const flakyFetch: typeof fetch = (async (input, init) => {
const m = (init as RequestInit | undefined)?.method ?? 'GET';
const u = typeof input === 'string' ? input : input instanceof URL ? input.toString() : (input as Request).url;
if (m === 'POST' && u.includes('/v1/inbox/bob') && !u.includes('/fetch') && failsLeft > 0) {
failsLeft--;
throw new Error('transient network');
}
return honoFetch(app)(input, init);
}) as typeof fetch;
const aliceInbox = new Inbox({
baseUrl: 'http://localhost',
ownAddress: 'alice',
crypto,
signingPrivateKey: alice.signingPrivateKey,
signingPublicKey: alice.signingPublicKey,
pollIntervalMs: 0,
fetch: flakyFetch,
queueStore: new MemoryOutgoingQueueStore(),
});
await aliceInbox.send({ recipientAddress: 'bob', envelope: randBytes(40) });
// First flush fails.
await aliceInbox.tick();
expect(await aliceInbox.pendingCount()).toBe(1);
// Second flush succeeds.
await aliceInbox.tick();
expect(await aliceInbox.pendingCount()).toBe(0);
});
});
describe('tamper detection', () => {
test('client rejects blob whose msgId does not match recomputed hash', async () => {
const store = new MemoryInboxStore();
const app = createInboxServer({ crypto, store, disableRateLimit: true });
const bob = await makeIdentity();
const alice = await makeIdentity();
// Register Bob.
const bobClient = new InboxClient({
baseUrl: 'http://localhost',
crypto,
signingPrivateKey: bob.signingPrivateKey,
fetch: honoFetch(app),
});
await bobClient.register({ address: 'bob', signingKey: bob.signingPublicKey });
// Alice puts a real blob.
const ct = randBytes(64);
const aliceClient = new InboxClient({
baseUrl: 'http://localhost',
crypto,
signingPrivateKey: alice.signingPrivateKey,
fetch: honoFetch(app),
});
await aliceClient.put({
recipientAddress: 'bob',
senderSigningKey: alice.signingPublicKey,
envelope: ct,
});
// Tamper: flip a byte in the in-memory store.
const list: any = (store as any).blobs.get('bob');
list[0].ciphertext[0] ^= 0xff;
const bobInbox = new Inbox({
baseUrl: 'http://localhost',
ownAddress: 'bob',
crypto,
signingPrivateKey: bob.signingPrivateKey,
signingPublicKey: bob.signingPublicKey,
pollIntervalMs: 0,
fetch: honoFetch(app),
});
let decryptCalls = 0;
let failures = 0;
bobInbox.onIncoming(() => {
decryptCalls++;
return null;
});
bobInbox.on((e) => {
if (e.name === 'inbox.message_decrypt_failed') failures++;
});
const result = await bobInbox.tick();
// Tampered blob: handler must NOT be called; decrypt-failed event fires.
expect(decryptCalls).toBe(0);
expect(failures).toBeGreaterThan(0);
expect(result.received).toBe(0);
});
});

View File

@@ -0,0 +1,8 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src"
},
"include": ["src"]
}

View File

@@ -0,0 +1,21 @@
{
"name": "@shade/key-transparency",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"exports": {
".": {
"types": "./src/index.ts",
"import": "./src/index.ts"
}
},
"dependencies": {
"@shade/core": "workspace:*",
"@noble/hashes": "^2.0.1"
},
"devDependencies": {
"@shade/crypto-web": "workspace:*",
"fast-check": "^3.22.0"
}
}

View File

@@ -0,0 +1,36 @@
import { ShadeError } from '@shade/core';
export class KTError extends ShadeError {
constructor(code: string, message: string) {
super(code, message);
this.name = 'KTError';
}
}
export class KTVerificationError extends KTError {
constructor(message: string) {
super('SHADE_KT_VERIFICATION', message);
this.name = 'KTVerificationError';
}
}
export class KTSplitViewError extends KTError {
constructor(message: string) {
super('SHADE_KT_SPLIT_VIEW', message);
this.name = 'KTSplitViewError';
}
}
export class KTStaleSTHError extends KTError {
constructor(message: string) {
super('SHADE_KT_STALE_STH', message);
this.name = 'KTStaleSTHError';
}
}
export class KTLogIdMismatchError extends KTError {
constructor(message: string) {
super('SHADE_KT_LOG_ID_MISMATCH', message);
this.name = 'KTLogIdMismatchError';
}
}

View File

@@ -0,0 +1,137 @@
/**
* RFC 6962 §2.1 hash construction for an append-only Merkle log.
*
* leaf_hash(d) = SHA-256(0x00 || d)
* node_hash(left, r) = SHA-256(0x01 || left || r)
*
* The 0x00 / 0x01 prefixes are critical — they make leaf-hashes and
* node-hashes distinct, which prevents second-preimage attacks where
* an attacker re-interprets a leaf as an internal node.
*/
import { sha256Sync } from './sha256.js';
const DOMAIN_LEAF = 0x00;
const DOMAIN_NODE = 0x01;
/** Bundle commitment domain prefix. Must be stable across versions. */
export const DOMAIN_BUNDLE = 0x01;
/** STH canonical-bytes domain prefix. */
export const DOMAIN_STH = 0x02;
/** Operation byte values inside a leaf. */
export const OP_REGISTER = 0x01;
export const OP_REPLENISH = 0x02;
export const OP_DELETE = 0x03;
export function leafHash(data: Uint8Array): Uint8Array {
const buf = new Uint8Array(1 + data.length);
buf[0] = DOMAIN_LEAF;
buf.set(data, 1);
return sha256Sync(buf);
}
export function nodeHash(left: Uint8Array, right: Uint8Array): Uint8Array {
const buf = new Uint8Array(1 + left.length + right.length);
buf[0] = DOMAIN_NODE;
buf.set(left, 1);
buf.set(right, 1 + left.length);
return sha256Sync(buf);
}
/** RFC 6962 MTH for an empty list: SHA-256 of the empty string. */
export function emptyRootHash(): Uint8Array {
return sha256Sync(new Uint8Array(0));
}
/**
* Encode a leaf describing an `address → bundle_hash` event.
*
* Layout:
* uint64_be timestamp_ms
* byte operation
* uint16_be addr_len
* bytes address (utf-8)
* uint16_be hash_len
* bytes bundle_hash (32 bytes for register/replenish, may be 0 for delete)
*/
export function encodeLeafData(
timestampMs: number,
operation: number,
address: string,
bundleHash: Uint8Array,
): Uint8Array {
const addrBytes = new TextEncoder().encode(address);
if (addrBytes.length > 0xffff) {
throw new Error('address too long for KT leaf encoding');
}
if (bundleHash.length > 0xffff) {
throw new Error('bundleHash too long for KT leaf encoding');
}
const len = 8 + 1 + 2 + addrBytes.length + 2 + bundleHash.length;
const out = new Uint8Array(len);
const view = new DataView(out.buffer);
let off = 0;
// uint64 BE — split into two uint32 halves
view.setUint32(off, Math.floor(timestampMs / 0x100000000));
view.setUint32(off + 4, timestampMs >>> 0);
off += 8;
out[off++] = operation;
view.setUint16(off, addrBytes.length);
off += 2;
out.set(addrBytes, off);
off += addrBytes.length;
view.setUint16(off, bundleHash.length);
off += 2;
out.set(bundleHash, off);
return out;
}
/**
* Compute the canonical bundle commitment hash.
*
* bundle_hash = SHA-256(
* 0x01 ||
* identitySigningKey (32) ||
* identityDHKey (32) ||
* uint32_be signedPreKey.keyId ||
* signedPreKey.publicKey (32) ||
* signedPreKey.signature (64)
* )
*
* One-time prekeys are NOT included — they are ephemeral and including
* them would force a log mutation per OTP rotation.
*/
export function computeBundleHash(input: {
identitySigningKey: Uint8Array;
identityDHKey: Uint8Array;
signedPreKey: { keyId: number; publicKey: Uint8Array; signature: Uint8Array };
}): Uint8Array {
if (input.identitySigningKey.length !== 32) {
throw new Error('identitySigningKey must be 32 bytes');
}
if (input.identityDHKey.length !== 32) {
throw new Error('identityDHKey must be 32 bytes');
}
if (input.signedPreKey.publicKey.length !== 32) {
throw new Error('signedPreKey.publicKey must be 32 bytes');
}
if (input.signedPreKey.signature.length !== 64) {
throw new Error('signedPreKey.signature must be 64 bytes');
}
const buf = new Uint8Array(1 + 32 + 32 + 4 + 32 + 64);
let off = 0;
buf[off++] = DOMAIN_BUNDLE;
buf.set(input.identitySigningKey, off);
off += 32;
buf.set(input.identityDHKey, off);
off += 32;
new DataView(buf.buffer).setUint32(off, input.signedPreKey.keyId >>> 0);
off += 4;
buf.set(input.signedPreKey.publicKey, off);
off += 32;
buf.set(input.signedPreKey.signature, off);
return sha256Sync(buf);
}

View File

@@ -0,0 +1,339 @@
/**
* Address-index commitment.
*
* The Merkle log itself records mutation events (`address → bundle_hash`
* at time T), but doesn't natively answer "what's the *current* state for
* `address`?" or "does `address` exist?".
*
* The address index is a **lexicographically sorted snapshot** of the
* current `(address, latest_leaf_index)` mapping. Its commitment hash —
* `index_root` — is part of every Signed Tree Head.
*
* Inclusion proof: the entry exists at sorted index `i`, prove it via
* audit path (same Merkle construction as the main log).
*
* Absence proof: the address would sort between two adjacent existing
* entries; prove inclusion of those two adjacent entries
* and that the queried address sorts strictly between them.
*
* V1 representation: a flat sorted array. We re-hash the whole index per
* STH (cheap up to ~1M entries). V2 will move to a sparse Merkle tree if
* the dataset grows enough that flat re-hash becomes a bottleneck.
*/
import { leafHash, nodeHash, emptyRootHash } from './hashes.js';
import { sha256Sync } from './sha256.js';
import { constantTimeEqual } from './util.js';
import { mth, auditPath, recomputeRootFromAuditPath } from './log.js';
export interface AddressIndexEntry {
/** The address (UTF-8 string). */
address: string;
/** Most recent log leaf index that mutated this address. */
latestLeafIndex: number;
/** Most recent bundle hash committed for this address. */
bundleHash: Uint8Array;
/** Whether the latest event was a delete (tombstone). */
deleted: boolean;
}
/** Encode an index entry into the bytes that go into a Merkle leaf. */
export function encodeIndexEntry(entry: AddressIndexEntry): Uint8Array {
const addrBytes = new TextEncoder().encode(entry.address);
if (addrBytes.length > 0xffff) throw new Error('address too long');
if (entry.bundleHash.length > 0xffff) throw new Error('bundleHash too long');
const len = 2 + addrBytes.length + 4 + 1 + 2 + entry.bundleHash.length;
const out = new Uint8Array(len);
const view = new DataView(out.buffer);
let off = 0;
view.setUint16(off, addrBytes.length);
off += 2;
out.set(addrBytes, off);
off += addrBytes.length;
view.setUint32(off, entry.latestLeafIndex >>> 0);
off += 4;
out[off++] = entry.deleted ? 1 : 0;
view.setUint16(off, entry.bundleHash.length);
off += 2;
out.set(entry.bundleHash, off);
return out;
}
/** Compute index_root over a sorted entry list. */
export function computeIndexRoot(sortedEntries: AddressIndexEntry[]): Uint8Array {
if (sortedEntries.length === 0) return emptyRootHash();
const leaves = sortedEntries.map((e) => leafHash(encodeIndexEntry(e)));
return mth(leaves, 0, leaves.length);
}
/** Compare two addresses lexicographically (by UTF-8 byte order). */
export function compareAddresses(a: string, b: string): number {
const ab = new TextEncoder().encode(a);
const bb = new TextEncoder().encode(b);
const len = Math.min(ab.length, bb.length);
for (let i = 0; i < len; i++) {
if (ab[i]! !== bb[i]!) return ab[i]! - bb[i]!;
}
return ab.length - bb.length;
}
/**
* In-memory address index. Maintains the canonical sorted ordering; on
* mutate, the operator re-computes index_root for the next STH.
*/
export class AddressIndex {
private entries: AddressIndexEntry[] = [];
private positionByAddress = new Map<string, number>();
get size(): number {
return this.entries.length;
}
/** Idempotently set an entry; re-sorts only when a new address is added. */
upsert(entry: AddressIndexEntry): void {
const existingPos = this.positionByAddress.get(entry.address);
if (existingPos !== undefined) {
this.entries[existingPos] = { ...entry };
return;
}
// Insert keeping sort order
let lo = 0;
let hi = this.entries.length;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (compareAddresses(this.entries[mid]!.address, entry.address) < 0) lo = mid + 1;
else hi = mid;
}
this.entries.splice(lo, 0, { ...entry });
// Rebuild position map (positions shift after insert)
this.positionByAddress.clear();
for (let i = 0; i < this.entries.length; i++) {
this.positionByAddress.set(this.entries[i]!.address, i);
}
}
/** Mark an address tombstoned. Keeps the entry in sorted order. */
tombstone(address: string, latestLeafIndex: number): void {
const pos = this.positionByAddress.get(address);
if (pos === undefined) return;
const e = this.entries[pos]!;
this.entries[pos] = {
...e,
deleted: true,
latestLeafIndex,
bundleHash: new Uint8Array(0),
};
}
/** Snapshot ordered list (defensive copy). */
snapshot(): AddressIndexEntry[] {
return this.entries.map((e) => ({ ...e, bundleHash: new Uint8Array(e.bundleHash) }));
}
/** Compute the index commitment root over the current sorted list. */
rootHash(): Uint8Array {
return computeIndexRoot(this.entries);
}
/** Look up an entry. */
get(address: string): AddressIndexEntry | undefined {
const pos = this.positionByAddress.get(address);
if (pos === undefined) return undefined;
return { ...this.entries[pos]!, bundleHash: new Uint8Array(this.entries[pos]!.bundleHash) };
}
/** Build inclusion proof: returns sorted-position + audit path. */
inclusionProof(address: string): IndexInclusionProof | null {
const pos = this.positionByAddress.get(address);
if (pos === undefined) return null;
const leaves = this.entries.map((e) => leafHash(encodeIndexEntry(e)));
return {
kind: 'inclusion',
position: pos,
treeSize: this.entries.length,
entry: { ...this.entries[pos]!, bundleHash: new Uint8Array(this.entries[pos]!.bundleHash) },
auditPath: auditPath(leaves, pos, leaves.length),
};
}
/**
* Build absence proof: returns the two adjacent entries that bracket the
* queried address (or boundary case for first/last).
*/
absenceProof(address: string): IndexAbsenceProof | null {
if (this.positionByAddress.has(address)) return null;
if (this.entries.length === 0) {
return {
kind: 'absence',
treeSize: 0,
queryAddress: address,
prev: null,
next: null,
};
}
// Find insertion position
let lo = 0;
let hi = this.entries.length;
while (lo < hi) {
const mid = (lo + hi) >>> 1;
if (compareAddresses(this.entries[mid]!.address, address) < 0) lo = mid + 1;
else hi = mid;
}
const leaves = this.entries.map((e) => leafHash(encodeIndexEntry(e)));
const prevPos = lo - 1;
const nextPos = lo;
const prev =
prevPos >= 0
? {
position: prevPos,
entry: {
...this.entries[prevPos]!,
bundleHash: new Uint8Array(this.entries[prevPos]!.bundleHash),
},
auditPath: auditPath(leaves, prevPos, leaves.length),
}
: null;
const next =
nextPos < this.entries.length
? {
position: nextPos,
entry: {
...this.entries[nextPos]!,
bundleHash: new Uint8Array(this.entries[nextPos]!.bundleHash),
},
auditPath: auditPath(leaves, nextPos, leaves.length),
}
: null;
return {
kind: 'absence',
treeSize: this.entries.length,
queryAddress: address,
prev,
next,
};
}
/** Hot-load from a sorted entry array (used by persistent stores). */
static fromEntries(sortedEntries: AddressIndexEntry[]): AddressIndex {
const idx = new AddressIndex();
for (const e of sortedEntries) {
idx.entries.push({ ...e, bundleHash: new Uint8Array(e.bundleHash) });
}
for (let i = 0; i < idx.entries.length; i++) {
idx.positionByAddress.set(idx.entries[i]!.address, i);
}
return idx;
}
}
export interface IndexInclusionProof {
kind: 'inclusion';
position: number;
treeSize: number;
entry: AddressIndexEntry;
auditPath: Uint8Array[];
}
export interface IndexAbsenceProof {
kind: 'absence';
treeSize: number;
queryAddress: string;
/**
* Largest existing entry less than the query (null if the query would
* be the first entry).
*/
prev: { position: number; entry: AddressIndexEntry; auditPath: Uint8Array[] } | null;
/**
* Smallest existing entry greater than the query (null if the query
* would be appended after the last entry).
*/
next: { position: number; entry: AddressIndexEntry; auditPath: Uint8Array[] } | null;
}
export type IndexProof = IndexInclusionProof | IndexAbsenceProof;
/**
* Verify an inclusion proof against an `index_root` commitment.
*/
export function verifyInclusionProof(
proof: IndexInclusionProof,
indexRoot: Uint8Array,
): boolean {
const lh = leafHash(encodeIndexEntry(proof.entry));
let recomputed: Uint8Array;
try {
recomputed = recomputeRootFromAuditPath(lh, proof.position, proof.treeSize, proof.auditPath);
} catch {
return false;
}
return constantTimeEqual(recomputed, indexRoot);
}
/**
* Verify an absence proof:
* - prev exists and address(prev) < query
* - next exists and query < address(next)
* - prev.position + 1 === next.position (they are adjacent)
* - both inclusion sub-proofs verify against indexRoot
*
* Boundary cases:
* - tree empty (treeSize === 0): valid if both prev and next are null
* - query smaller than all entries: prev is null, next.position === 0
* - query larger than all entries: next is null, prev.position === treeSize - 1
*/
export function verifyAbsenceProof(
proof: IndexAbsenceProof,
indexRoot: Uint8Array,
): boolean {
if (proof.treeSize === 0) {
if (proof.prev !== null || proof.next !== null) return false;
return constantTimeEqual(emptyRootHash(), indexRoot);
}
const queryAddr = proof.queryAddress;
if (proof.prev) {
if (compareAddresses(proof.prev.entry.address, queryAddr) >= 0) return false;
const lh = leafHash(encodeIndexEntry(proof.prev.entry));
let r: Uint8Array;
try {
r = recomputeRootFromAuditPath(lh, proof.prev.position, proof.treeSize, proof.prev.auditPath);
} catch {
return false;
}
if (!constantTimeEqual(r, indexRoot)) return false;
}
if (proof.next) {
if (compareAddresses(queryAddr, proof.next.entry.address) >= 0) return false;
const lh = leafHash(encodeIndexEntry(proof.next.entry));
let r: Uint8Array;
try {
r = recomputeRootFromAuditPath(lh, proof.next.position, proof.treeSize, proof.next.auditPath);
} catch {
return false;
}
if (!constantTimeEqual(r, indexRoot)) return false;
}
// Boundary checks
if (proof.prev === null) {
if (proof.next === null) return false; // already handled treeSize===0
if (proof.next.position !== 0) return false;
} else if (proof.next === null) {
if (proof.prev.position !== proof.treeSize - 1) return false;
} else {
if (proof.prev.position + 1 !== proof.next.position) return false;
}
return true;
}
/** sha256 helper export for callers that need the same hash function. */
export { sha256Sync };

View File

@@ -0,0 +1,100 @@
/**
* `@shade/key-transparency` — verifiable prekey distribution (V3.12).
*
* Public surface:
* - Hash primitives: `leafHash`, `nodeHash`, `computeBundleHash`, `encodeLeafData`.
* - Merkle log: `MerkleLog`, `auditPath`, `recomputeRootFromAuditPath`,
* `consistencyProof`, `verifyConsistencyProof`.
* - Address index: `AddressIndex`, `verifyInclusionProof`, `verifyAbsenceProof`.
* - Signed Tree Head: `SignedTreeHead`, `signSth`, `verifySthSignature`,
* `canonicalSthBytes`, `computeLogId`, `STHWire`.
* - Bundle proofs: `KTProof`, `verifyBundleInclusion`, `verifyBundleAbsence`,
* `verifyBundleTombstone`, `ktProofToWire`, `ktProofFromWire`.
* - Manager (server-side orchestration): `KTLogManager`.
* - Stores: `KTLogStore` interface + `MemoryKTLogStore`.
* - Witness: `LightWitness`, `WitnessFetcher`.
* - Errors: `KTError` and subclasses.
*/
export {
DOMAIN_BUNDLE,
DOMAIN_STH,
OP_DELETE,
OP_REGISTER,
OP_REPLENISH,
computeBundleHash,
emptyRootHash,
encodeLeafData,
leafHash,
nodeHash,
} from './hashes.js';
export {
MerkleLog,
auditPath,
consistencyProof,
mth,
recomputeRootFromAuditPath,
verifyConsistencyProof,
} from './log.js';
export {
AddressIndex,
compareAddresses,
computeIndexRoot,
encodeIndexEntry,
verifyAbsenceProof,
verifyInclusionProof,
} from './index-tree.js';
export type {
AddressIndexEntry,
IndexAbsenceProof,
IndexInclusionProof,
IndexProof,
} from './index-tree.js';
export {
canonicalSthBytes,
computeLogId,
signSth,
sthFromWire,
sthToWire,
verifySthSignature,
} from './sth.js';
export type { SignedTreeHead, STHWire } from './sth.js';
export {
ktProofFromWire,
ktProofToWire,
verifyBundleAbsence,
verifyBundleInclusion,
verifyBundleTombstone,
} from './proof.js';
export type {
KTBundleAbsenceProof,
KTBundleInclusionProof,
KTBundleTombstoneProof,
KTProof,
KTProofBody,
KTProofWire,
KTVerifyOptions,
} from './proof.js';
export { KTLogManager } from './manager.js';
export type { KTLogManagerOptions } from './manager.js';
export { MemoryKTLogStore } from './memory-store.js';
export type { KTLogLeaf, KTLogStore } from './store.js';
export { LightWitness } from './witness.js';
export type { LightWitnessOptions, WitnessFetcher, WitnessObservation } from './witness.js';
export {
KTError,
KTLogIdMismatchError,
KTSplitViewError,
KTStaleSTHError,
KTVerificationError,
} from './errors.js';
export { fromBase64 as ktFromBase64, toBase64 as ktToBase64 } from './util.js';

View File

@@ -0,0 +1,273 @@
/**
* RFC 6962-compatible Merkle Hash Tree (MTH) over an append-only list
* of pre-hashed leaves.
*
* Recurrence (RFC 6962 §2.1):
*
* MTH({}) = SHA-256()
* MTH({d(0)}) = leaf_hash(d(0))
* MTH(D[n]) = node_hash( MTH(D[0:k]), MTH(D[k:n]) )
* where k = largest power of 2 < n
*
* `MerkleLog` stores **leaf hashes** (already prefixed with 0x00) and
* recomputes the tree on demand. Storage is O(N); audit-path / consistency
* computation is O(log N) per leaf. This is acceptable for prekey-server
* scale (≤ ~5M leaves over a decade for 100k addresses).
*/
import { emptyRootHash, leafHash, nodeHash } from './hashes.js';
import { constantTimeEqual } from './util.js';
/** Largest power of two strictly less than n (for n ≥ 2). */
function largestPow2LessThan(n: number): number {
if (n < 2) throw new Error('largestPow2LessThan requires n >= 2');
let k = 1;
while (k < n) k <<= 1;
return k >>> 1;
}
/**
* Compute the Merkle Tree Hash (MTH) over a slice of pre-hashed leaves.
* Used internally by audit-path / consistency-proof builders.
*/
export function mth(leaves: Uint8Array[], lo: number, hi: number): Uint8Array {
const n = hi - lo;
if (n === 0) return emptyRootHash();
if (n === 1) return leaves[lo]!;
const k = largestPow2LessThan(n);
return nodeHash(mth(leaves, lo, lo + k), mth(leaves, lo + k, hi));
}
/**
* Build the audit path for the leaf at index `m` in a tree of size `n`.
*
* RFC 6962 §2.1.1:
* PATH(m, D[n]) = PATH(m, D[0:k]) : MTH(D[k:n]) if m < k
* PATH(m, D[n]) = PATH(m-k, D[k:n]) : MTH(D[0:k]) if m >= k
*
* The returned array is ordered from leaf-sibling outward to root-sibling.
* Each entry is the *hash of the sibling subtree*; the verifier reconstructs
* the root using `audit_path_hash` below.
*/
export function auditPath(leaves: Uint8Array[], m: number, n: number): Uint8Array[] {
if (n <= 0) throw new Error('auditPath requires n > 0');
if (m < 0 || m >= n) throw new Error(`m out of range: ${m} of ${n}`);
return auditPathInner(leaves, m, 0, n);
}
function auditPathInner(
leaves: Uint8Array[],
m: number,
lo: number,
hi: number,
): Uint8Array[] {
const n = hi - lo;
if (n === 1) return [];
const k = largestPow2LessThan(n);
if (m < k) {
return [...auditPathInner(leaves, m, lo, lo + k), mth(leaves, lo + k, hi)];
}
return [...auditPathInner(leaves, m - k, lo + k, hi), mth(leaves, lo, lo + k)];
}
/**
* Reconstruct the Merkle root from a leaf, its index, the tree size, and
* an audit path. RFC 6962 §2.1.1.
*
* Returns the recomputed root; the caller compares (constant-time) against
* the STH root to verify inclusion.
*/
export function recomputeRootFromAuditPath(
leaf: Uint8Array,
m: number,
n: number,
path: Uint8Array[],
): Uint8Array {
if (n <= 0) throw new Error('recomputeRoot requires n > 0');
if (m < 0 || m >= n) throw new Error(`m out of range: ${m} of ${n}`);
return recomputeRootInner(leaf, m, 0, n, path, 0).root;
}
function recomputeRootInner(
leaf: Uint8Array,
m: number,
lo: number,
hi: number,
path: Uint8Array[],
pathIdx: number,
): { root: Uint8Array; pathIdx: number } {
const n = hi - lo;
if (n === 1) return { root: leaf, pathIdx };
const k = largestPow2LessThan(n);
if (m < k) {
const left = recomputeRootInner(leaf, m, lo, lo + k, path, pathIdx);
const sibling = path[left.pathIdx];
if (!sibling) throw new Error('audit path too short');
return { root: nodeHash(left.root, sibling), pathIdx: left.pathIdx + 1 };
}
const right = recomputeRootInner(leaf, m - k, lo + k, hi, path, pathIdx);
const sibling = path[right.pathIdx];
if (!sibling) throw new Error('audit path too short');
return { root: nodeHash(sibling, right.root), pathIdx: right.pathIdx + 1 };
}
/**
* Build a consistency proof between tree sizes m (older) and n (newer).
* RFC 6962 §2.1.2.
*/
export function consistencyProof(
leaves: Uint8Array[],
m: number,
n: number,
): Uint8Array[] {
if (m < 0 || n < m) throw new Error(`invalid m,n: ${m},${n}`);
if (m === 0 || m === n) return [];
return subProof(leaves, m, 0, n, true);
}
function subProof(
leaves: Uint8Array[],
m: number,
lo: number,
hi: number,
isOriginalRoot: boolean,
): Uint8Array[] {
const n = hi - lo;
if (m === n) {
return isOriginalRoot ? [] : [mth(leaves, lo, hi)];
}
const k = largestPow2LessThan(n);
if (m <= k) {
return [...subProof(leaves, m, lo, lo + k, isOriginalRoot), mth(leaves, lo + k, hi)];
}
return [...subProof(leaves, m - k, lo + k, hi, false), mth(leaves, lo, lo + k)];
}
/**
* Verify a consistency proof. Given:
* - oldRoot = MTH(D[0:m])
* - newRoot = MTH(D[0:n]) with n >= m
* - proof = consistencyProof(leaves, m, n)
*
* Returns true if the proof is valid (i.e. D[0:n] really is an extension
* of D[0:m]). RFC 6962 §2.1.2.
*/
export function verifyConsistencyProof(
m: number,
n: number,
oldRoot: Uint8Array,
newRoot: Uint8Array,
proof: Uint8Array[],
): boolean {
if (m < 0 || n < m) return false;
if (m === 0) return true; // any newRoot is consistent with empty old tree
if (m === n) return proof.length === 0 && constantTimeEqual(oldRoot, newRoot);
// RFC 6962 verification recurrence
let path = proof;
if (isPowerOfTwo(m)) {
path = [oldRoot, ...path];
}
let fn = m - 1;
let sn = n - 1;
while ((fn & 1) === 1) {
fn >>= 1;
sn >>= 1;
}
if (path.length === 0) return false;
let fr = path[0]!;
let sr = path[0]!;
let i = 1;
while (sn > 0) {
if ((fn & 1) === 1 || fn === sn) {
if (i >= path.length) return false;
const c = path[i++]!;
fr = nodeHash(c, fr);
sr = nodeHash(c, sr);
while ((fn & 1) === 0 && fn !== 0) {
fn >>= 1;
sn >>= 1;
}
} else {
if (i >= path.length) return false;
sr = nodeHash(sr, path[i++]!);
}
fn >>= 1;
sn >>= 1;
}
if (i !== path.length) return false;
return constantTimeEqual(fr, oldRoot) && constantTimeEqual(sr, newRoot);
}
function isPowerOfTwo(n: number): boolean {
return n > 0 && (n & (n - 1)) === 0;
}
/**
* Append-only Merkle log used server-side. Holds leaf hashes in memory
* and recomputes paths on demand.
*
* For production deployments the caller wraps this with a persistent
* `KTLogStore` (see `store.ts`) — this class is the algorithmic core.
*/
export class MerkleLog {
private readonly leaves: Uint8Array[] = [];
/** Number of leaves currently in the tree. */
get size(): number {
return this.leaves.length;
}
/** Append a *raw leaf data* (will be domain-separated and hashed). */
appendData(data: Uint8Array): { index: number; leafHash: Uint8Array } {
const lh = leafHash(data);
const index = this.leaves.length;
this.leaves.push(lh);
return { index, leafHash: lh };
}
/** Append a leaf that has *already been hashed* (rebuild path). */
appendLeafHash(lh: Uint8Array): number {
const index = this.leaves.length;
this.leaves.push(lh);
return index;
}
/** Current root hash (MTH of all leaves). */
rootHash(): Uint8Array {
return mth(this.leaves, 0, this.leaves.length);
}
/** Audit path for leaf at `index`. */
auditPath(index: number): Uint8Array[] {
return auditPath(this.leaves, index, this.leaves.length);
}
/** Consistency proof from `oldSize` to current size. */
consistencyProof(oldSize: number): Uint8Array[] {
return consistencyProof(this.leaves, oldSize, this.leaves.length);
}
/** Snapshot the leaf hash at `index` (read-only). */
leafHashAt(index: number): Uint8Array {
const lh = this.leaves[index];
if (!lh) throw new Error(`no leaf at index ${index}`);
return lh;
}
/** Defensive copy of all leaves (used by persistent stores on hot-load). */
exportLeaves(): Uint8Array[] {
return this.leaves.map((l) => new Uint8Array(l));
}
/** Hot-load from persisted leaf hashes. */
static fromLeaves(leaves: Uint8Array[]): MerkleLog {
const log = new MerkleLog();
for (const l of leaves) log.appendLeafHash(l);
return log;
}
}

View File

@@ -0,0 +1,274 @@
/**
* KTLogManager — server-side orchestration of the log + address index.
*
* Wraps a `KTLogStore` with the algorithmic primitives so that callers
* (the prekey-server integration layer) never have to think about Merkle
* paths, index commitments, or STH signing in isolation.
*
* Lifecycle:
* const mgr = await KTLogManager.create({ crypto, store, signingKey });
* await mgr.recordRegister(address, bundleHash, timestampMs);
* const sth = await mgr.publishSTH();
* const proof = await mgr.buildBundleInclusionProof(address);
*
* Concurrency: the manager is single-writer. Server callers must serialize
* mutations behind a mutex (the integration layer does this with an in-process
* lock that is sufficient for the single-instance default deployment;
* multi-instance HA requires external coordination — documented in
* docs/key-transparency.md §"Recovery and HA").
*/
import type { CryptoProvider } from '@shade/core';
import { OP_DELETE, OP_REGISTER, OP_REPLENISH, encodeLeafData, leafHash } from './hashes.js';
import { MerkleLog, auditPath } from './log.js';
import {
AddressIndex,
type AddressIndexEntry,
type IndexAbsenceProof,
type IndexInclusionProof,
} from './index-tree.js';
import {
type SignedTreeHead,
computeLogId,
signSth,
} from './sth.js';
import type { KTLogStore } from './store.js';
import type {
KTBundleAbsenceProof,
KTBundleInclusionProof,
KTBundleTombstoneProof,
KTProof,
} from './proof.js';
import { consistencyProof, verifyConsistencyProof } from './log.js';
export interface KTLogManagerOptions {
crypto: CryptoProvider;
store: KTLogStore;
/** Operator's Ed25519 signing key (private, 32-byte seed). */
signingPrivateKey: Uint8Array;
/** Operator's Ed25519 signing public key (pinned by clients). */
signingPublicKey: Uint8Array;
/** Time source — defaults to `Date.now()`. */
now?: () => number;
}
export class KTLogManager {
private readonly crypto: CryptoProvider;
private readonly store: KTLogStore;
private readonly signingPrivateKey: Uint8Array;
private readonly signingPublicKey: Uint8Array;
private readonly logId: Uint8Array;
private readonly now: () => number;
// In-memory mirror of the persistent state for fast proof generation.
private merkleLog: MerkleLog;
private addressIndex: AddressIndex;
private constructor(opts: KTLogManagerOptions, log: MerkleLog, idx: AddressIndex) {
this.crypto = opts.crypto;
this.store = opts.store;
this.signingPrivateKey = opts.signingPrivateKey;
this.signingPublicKey = opts.signingPublicKey;
this.logId = computeLogId(opts.signingPublicKey);
this.now = opts.now ?? (() => Date.now());
this.merkleLog = log;
this.addressIndex = idx;
}
static async create(opts: KTLogManagerOptions): Promise<KTLogManager> {
const size = await opts.store.size();
const leaves = await opts.store.getLeaves(0, size);
const log = MerkleLog.fromLeaves(leaves.map((l) => l.leafHash));
const idx = AddressIndex.fromEntries(await opts.store.getAllIndexEntries());
return new KTLogManager(opts, log, idx);
}
/** Operator's pinned signing public key, for callers to ship to clients. */
getSigningPublicKey(): Uint8Array {
return new Uint8Array(this.signingPublicKey);
}
/** log_id (== sha256(signingPublicKey)). */
getLogId(): Uint8Array {
return new Uint8Array(this.logId);
}
/** Current tree size (number of leaves). */
getTreeSize(): number {
return this.merkleLog.size;
}
/** Record a register/rotate event. Bundle hash is the canonical bundle commit. */
async recordRegister(
address: string,
bundleHash: Uint8Array,
timestampMs?: number,
): Promise<{ leafIndex: number }> {
return this.recordOperation(address, OP_REGISTER, bundleHash, timestampMs);
}
/**
* Record a "replenish" event. Per §11 of the design notat, replenish does
* NOT mutate bundle_hash (one-time prekeys aren't part of the commitment).
* In practice this method is rarely called — kept for cases where the
* operator wants liveness-evidence of OTP top-ups in the log. When used,
* the leaf's bundle_hash equals the current index entry's bundle_hash.
*/
async recordReplenish(
address: string,
timestampMs?: number,
): Promise<{ leafIndex: number } | null> {
const existing = await this.store.getIndexEntry(address);
if (!existing) return null;
return this.recordOperation(address, OP_REPLENISH, existing.bundleHash, timestampMs);
}
/** Record an unregister/tombstone event. */
async recordDelete(address: string, timestampMs?: number): Promise<{ leafIndex: number }> {
const result = await this.recordOperation(
address,
OP_DELETE,
new Uint8Array(0),
timestampMs,
);
await this.store.tombstoneIndexEntry(address, result.leafIndex);
this.addressIndex.tombstone(address, result.leafIndex);
return result;
}
private async recordOperation(
address: string,
operation: number,
bundleHash: Uint8Array,
timestampMsOpt?: number,
): Promise<{ leafIndex: number }> {
const timestampMs = timestampMsOpt ?? this.now();
const data = encodeLeafData(timestampMs, operation, address, bundleHash);
const lh = leafHash(data);
const index = await this.store.appendLeaf({
leafHash: lh,
timestampMs,
operation,
address,
bundleHash,
});
this.merkleLog.appendLeafHash(lh);
if (operation !== OP_DELETE) {
const entry: AddressIndexEntry = {
address,
latestLeafIndex: index,
bundleHash,
deleted: false,
};
await this.store.upsertIndexEntry(entry);
this.addressIndex.upsert(entry);
}
return { leafIndex: index };
}
/** Re-sign and persist the current STH. Idempotent if no change since last call. */
async publishSTH(timestampMsOpt?: number): Promise<SignedTreeHead> {
const treeSize = this.merkleLog.size;
const rootHash = this.merkleLog.rootHash();
const indexRoot = this.addressIndex.rootHash();
const timestampMs = timestampMsOpt ?? this.now();
const sth = await signSth(this.crypto, this.signingPrivateKey, {
treeSize,
timestampMs,
rootHash,
indexRoot,
logId: this.logId,
});
await this.store.saveSTH(sth);
return sth;
}
/** Build an inclusion proof for the address's *latest* event. */
async buildBundleInclusionProof(
address: string,
sth: SignedTreeHead,
): Promise<KTProof | null> {
const indexEntry = this.addressIndex.get(address);
if (!indexEntry) return null;
const leaf = await this.store.getLeaf(indexEntry.latestLeafIndex);
if (!leaf) return null;
if (sth.treeSize <= indexEntry.latestLeafIndex) return null;
// Audit path is over the snapshot at sth.treeSize. The current in-memory
// log is exactly that size when the manager produced this STH.
const path = this.auditPathAt(indexEntry.latestLeafIndex, sth.treeSize);
const indexProof = this.addressIndex.inclusionProof(address);
if (!indexProof) return null;
if (indexEntry.deleted) {
const body: KTBundleTombstoneProof = {
kind: 'tombstone',
leafIndex: indexEntry.latestLeafIndex,
leafTimestampMs: leaf.timestampMs,
operation: leaf.operation,
auditPath: path,
indexProof,
};
return { sth, body };
}
const body: KTBundleInclusionProof = {
kind: 'inclusion',
leafIndex: indexEntry.latestLeafIndex,
leafTimestampMs: leaf.timestampMs,
operation: leaf.operation,
auditPath: path,
indexProof,
};
return { sth, body };
}
/** Build an absence proof for an address that does not exist. */
buildBundleAbsenceProof(address: string, sth: SignedTreeHead): KTProof | null {
const indexProof = this.addressIndex.absenceProof(address);
if (!indexProof) return null; // address actually exists
const body: KTBundleAbsenceProof = { kind: 'absence', indexProof };
return { sth, body };
}
/**
* Compute a consistency proof from `oldTreeSize` to current. Works against
* the in-memory log; for very-large logs this becomes O(N) and a future
* persistent-only path may be needed.
*/
async buildConsistencyProof(oldTreeSize: number): Promise<{
proof: Uint8Array[];
fromTreeSize: number;
toTreeSize: number;
}> {
const size = this.merkleLog.size;
const leaves = this.merkleLog.exportLeaves();
const proof = consistencyProof(leaves, oldTreeSize, size);
return { proof, fromTreeSize: oldTreeSize, toTreeSize: size };
}
/**
* Compute a consistency proof between two arbitrary historical sizes.
* Reads leaves from the persistent store (so older snapshots can be proven
* even if the in-memory log has grown further).
*/
async buildHistoricalConsistencyProof(oldSize: number, newSize: number): Promise<Uint8Array[]> {
if (newSize > this.merkleLog.size) {
throw new Error(`newSize ${newSize} exceeds current tree size ${this.merkleLog.size}`);
}
const leaves = this.merkleLog.exportLeaves();
return consistencyProof(leaves, oldSize, newSize);
}
// ─── Helpers ──────────────────────────────────────────
private auditPathAt(leafIndex: number, treeSize: number): Uint8Array[] {
const leaves = this.merkleLog.exportLeaves().slice(0, treeSize);
return auditPath(leaves, leafIndex, leaves.length);
}
}
export { verifyConsistencyProof };

View File

@@ -0,0 +1,149 @@
import type { KTLogLeaf, KTLogStore } from './store.js';
import type { AddressIndexEntry } from './index-tree.js';
import type { SignedTreeHead } from './sth.js';
import { compareAddresses } from './index-tree.js';
import { constantTimeEqual } from './util.js';
/**
* In-memory KTLogStore for testing and embedded servers.
*
* Maintains the same append-only invariants as a persistent store —
* `appendLeaf` cannot mutate prior entries, only push.
*/
export class MemoryKTLogStore implements KTLogStore {
private leaves: KTLogLeaf[] = [];
private indexByAddress = new Map<string, AddressIndexEntry>();
private sthsByTreeSize = new Map<number, SignedTreeHead[]>();
private latestSth: SignedTreeHead | null = null;
async appendLeaf(input: Omit<KTLogLeaf, 'index'>): Promise<number> {
const index = this.leaves.length;
this.leaves.push({
...input,
index,
leafHash: new Uint8Array(input.leafHash),
bundleHash: new Uint8Array(input.bundleHash),
});
return index;
}
async getLeaves(fromIndex: number, toIndex: number): Promise<KTLogLeaf[]> {
return this.leaves.slice(fromIndex, toIndex).map((l) => ({
...l,
leafHash: new Uint8Array(l.leafHash),
bundleHash: new Uint8Array(l.bundleHash),
}));
}
async getLeaf(index: number): Promise<KTLogLeaf | null> {
const l = this.leaves[index];
if (!l) return null;
return {
...l,
leafHash: new Uint8Array(l.leafHash),
bundleHash: new Uint8Array(l.bundleHash),
};
}
async size(): Promise<number> {
return this.leaves.length;
}
async upsertIndexEntry(entry: AddressIndexEntry): Promise<void> {
this.indexByAddress.set(entry.address, {
...entry,
bundleHash: new Uint8Array(entry.bundleHash),
});
}
async tombstoneIndexEntry(address: string, latestLeafIndex: number): Promise<void> {
const e = this.indexByAddress.get(address);
if (!e) return;
this.indexByAddress.set(address, {
...e,
deleted: true,
latestLeafIndex,
bundleHash: new Uint8Array(0),
});
}
async getAllIndexEntries(): Promise<AddressIndexEntry[]> {
const all = Array.from(this.indexByAddress.values()).map((e) => ({
...e,
bundleHash: new Uint8Array(e.bundleHash),
}));
all.sort((a, b) => compareAddresses(a.address, b.address));
return all;
}
async getIndexEntry(address: string): Promise<AddressIndexEntry | null> {
const e = this.indexByAddress.get(address);
if (!e) return null;
return { ...e, bundleHash: new Uint8Array(e.bundleHash) };
}
async saveSTH(sth: SignedTreeHead): Promise<void> {
const cloned: SignedTreeHead = {
treeSize: sth.treeSize,
timestampMs: sth.timestampMs,
rootHash: new Uint8Array(sth.rootHash),
indexRoot: new Uint8Array(sth.indexRoot),
logId: new Uint8Array(sth.logId),
signature: new Uint8Array(sth.signature),
};
const list = this.sthsByTreeSize.get(sth.treeSize) ?? [];
// De-duplicate (same root_hash + signature == same STH)
const dup = list.find(
(existing) =>
existing.timestampMs === cloned.timestampMs &&
constantTimeEqual(existing.rootHash, cloned.rootHash) &&
constantTimeEqual(existing.signature, cloned.signature),
);
if (!dup) list.push(cloned);
this.sthsByTreeSize.set(sth.treeSize, list);
if (
!this.latestSth ||
cloned.treeSize > this.latestSth.treeSize ||
(cloned.treeSize === this.latestSth.treeSize && cloned.timestampMs > this.latestSth.timestampMs)
) {
this.latestSth = cloned;
}
}
async getLatestSTH(): Promise<SignedTreeHead | null> {
return this.latestSth ? cloneSth(this.latestSth) : null;
}
async getSTHByTreeSize(treeSize: number): Promise<SignedTreeHead | null> {
const list = this.sthsByTreeSize.get(treeSize);
if (!list || list.length === 0) return null;
// Pick the most recent one for this tree size.
let best = list[0]!;
for (const s of list) if (s.timestampMs > best.timestampMs) best = s;
return cloneSth(best);
}
async listSTHs(fromTimestampMs?: number, toTimestampMs?: number): Promise<SignedTreeHead[]> {
const all: SignedTreeHead[] = [];
for (const list of this.sthsByTreeSize.values()) {
for (const s of list) {
if (fromTimestampMs !== undefined && s.timestampMs < fromTimestampMs) continue;
if (toTimestampMs !== undefined && s.timestampMs > toTimestampMs) continue;
all.push(cloneSth(s));
}
}
all.sort((a, b) => a.timestampMs - b.timestampMs);
return all;
}
}
function cloneSth(sth: SignedTreeHead): SignedTreeHead {
return {
treeSize: sth.treeSize,
timestampMs: sth.timestampMs,
rootHash: new Uint8Array(sth.rootHash),
indexRoot: new Uint8Array(sth.indexRoot),
logId: new Uint8Array(sth.logId),
signature: new Uint8Array(sth.signature),
};
}

View File

@@ -0,0 +1,453 @@
/**
* Combined KT proof attached to a bundle response.
*
* Wire shape (returned by GET /v1/keys/bundle/:address when KT is on):
*
* {
* bundle: { ... },
* ktProof: {
* sth: STHWire,
* inclusion: {
* leafIndex: number,
* leafTimestampMs: number,
* operation: number,
* auditPath: string[] // base64 sibling hashes
* },
* indexProof: IndexInclusionProof | IndexAbsenceProof (wire-encoded)
* }
* }
*
* The verifier:
* 1. Confirms STH signature against pinned log_public_key.
* 2. Confirms STH timestamp is fresh (<= maxStaleMs old).
* 3. Re-derives bundle_hash from the bundle and re-builds the leaf.
* 4. Verifies inclusion against sth.root_hash via auditPath.
* 5. Verifies index proof (inclusion w/ matching bundle hash, or absence
* proof for tombstoned address) against sth.index_root.
*/
import type { CryptoProvider } from '@shade/core';
import { computeBundleHash, encodeLeafData, leafHash, OP_DELETE } from './hashes.js';
import { recomputeRootFromAuditPath } from './log.js';
import {
type SignedTreeHead,
type STHWire,
sthFromWire,
sthToWire,
verifySthSignature,
} from './sth.js';
import {
type AddressIndexEntry,
type IndexAbsenceProof,
type IndexInclusionProof,
verifyInclusionProof,
verifyAbsenceProof,
} from './index-tree.js';
import {
KTLogIdMismatchError,
KTStaleSTHError,
KTVerificationError,
} from './errors.js';
import { computeLogId } from './sth.js';
import { constantTimeEqual, fromBase64, toBase64 } from './util.js';
/** Exists-style proof: leaf is in the log, address is in the index. */
export interface KTBundleInclusionProof {
kind: 'inclusion';
leafIndex: number;
leafTimestampMs: number;
operation: number;
auditPath: Uint8Array[];
indexProof: IndexInclusionProof;
}
/** Tombstone proof: latest leaf for the address is a delete; index entry is deleted. */
export interface KTBundleTombstoneProof {
kind: 'tombstone';
leafIndex: number;
leafTimestampMs: number;
/** operation will be OP_DELETE here. */
operation: number;
auditPath: Uint8Array[];
indexProof: IndexInclusionProof;
}
/** Absence proof: address has never been registered. */
export interface KTBundleAbsenceProof {
kind: 'absence';
indexProof: IndexAbsenceProof;
}
export type KTProofBody = KTBundleInclusionProof | KTBundleTombstoneProof | KTBundleAbsenceProof;
export interface KTProof {
sth: SignedTreeHead;
body: KTProofBody;
}
// ─── Wire encoding ────────────────────────────────────────
interface IndexInclusionWire {
kind: 'inclusion';
position: number;
treeSize: number;
entry: { address: string; latestLeafIndex: number; bundleHash: string; deleted: boolean };
auditPath: string[];
}
interface IndexAbsenceWire {
kind: 'absence';
treeSize: number;
queryAddress: string;
prev: {
position: number;
entry: { address: string; latestLeafIndex: number; bundleHash: string; deleted: boolean };
auditPath: string[];
} | null;
next: {
position: number;
entry: { address: string; latestLeafIndex: number; bundleHash: string; deleted: boolean };
auditPath: string[];
} | null;
}
type IndexProofWire = IndexInclusionWire | IndexAbsenceWire;
interface BundleInclusionWire {
kind: 'inclusion' | 'tombstone';
leafIndex: number;
leafTimestampMs: number;
operation: number;
auditPath: string[];
indexProof: IndexInclusionWire;
}
interface BundleAbsenceWire {
kind: 'absence';
indexProof: IndexAbsenceWire;
}
type KTProofBodyWire = BundleInclusionWire | BundleAbsenceWire;
export interface KTProofWire {
sth: STHWire;
body: KTProofBodyWire;
}
function entryToWire(e: AddressIndexEntry) {
return {
address: e.address,
latestLeafIndex: e.latestLeafIndex,
bundleHash: toBase64(e.bundleHash),
deleted: e.deleted,
};
}
function entryFromWire(w: {
address: string;
latestLeafIndex: number;
bundleHash: string;
deleted: boolean;
}): AddressIndexEntry {
return {
address: w.address,
latestLeafIndex: w.latestLeafIndex,
bundleHash: fromBase64(w.bundleHash),
deleted: w.deleted,
};
}
function indexInclusionToWire(p: IndexInclusionProof): IndexInclusionWire {
return {
kind: 'inclusion',
position: p.position,
treeSize: p.treeSize,
entry: entryToWire(p.entry),
auditPath: p.auditPath.map(toBase64),
};
}
function indexInclusionFromWire(w: IndexInclusionWire): IndexInclusionProof {
return {
kind: 'inclusion',
position: w.position,
treeSize: w.treeSize,
entry: entryFromWire(w.entry),
auditPath: w.auditPath.map(fromBase64),
};
}
function indexAbsenceToWire(p: IndexAbsenceProof): IndexAbsenceWire {
return {
kind: 'absence',
treeSize: p.treeSize,
queryAddress: p.queryAddress,
prev: p.prev
? {
position: p.prev.position,
entry: entryToWire(p.prev.entry),
auditPath: p.prev.auditPath.map(toBase64),
}
: null,
next: p.next
? {
position: p.next.position,
entry: entryToWire(p.next.entry),
auditPath: p.next.auditPath.map(toBase64),
}
: null,
};
}
function indexAbsenceFromWire(w: IndexAbsenceWire): IndexAbsenceProof {
return {
kind: 'absence',
treeSize: w.treeSize,
queryAddress: w.queryAddress,
prev: w.prev
? {
position: w.prev.position,
entry: entryFromWire(w.prev.entry),
auditPath: w.prev.auditPath.map(fromBase64),
}
: null,
next: w.next
? {
position: w.next.position,
entry: entryFromWire(w.next.entry),
auditPath: w.next.auditPath.map(fromBase64),
}
: null,
};
}
export function ktProofToWire(proof: KTProof): KTProofWire {
const sth = sthToWire(proof.sth, toBase64);
let body: KTProofBodyWire;
if (proof.body.kind === 'absence') {
body = { kind: 'absence', indexProof: indexAbsenceToWire(proof.body.indexProof) };
} else {
body = {
kind: proof.body.kind,
leafIndex: proof.body.leafIndex,
leafTimestampMs: proof.body.leafTimestampMs,
operation: proof.body.operation,
auditPath: proof.body.auditPath.map(toBase64),
indexProof: indexInclusionToWire(proof.body.indexProof),
};
}
return { sth, body };
}
export function ktProofFromWire(wire: KTProofWire): KTProof {
const sth = sthFromWire(wire.sth, fromBase64);
let body: KTProofBody;
if (wire.body.kind === 'absence') {
body = { kind: 'absence', indexProof: indexAbsenceFromWire(wire.body.indexProof) };
} else if (wire.body.kind === 'tombstone') {
body = {
kind: 'tombstone',
leafIndex: wire.body.leafIndex,
leafTimestampMs: wire.body.leafTimestampMs,
operation: wire.body.operation,
auditPath: wire.body.auditPath.map(fromBase64),
indexProof: indexInclusionFromWire(wire.body.indexProof),
};
} else {
body = {
kind: 'inclusion',
leafIndex: wire.body.leafIndex,
leafTimestampMs: wire.body.leafTimestampMs,
operation: wire.body.operation,
auditPath: wire.body.auditPath.map(fromBase64),
indexProof: indexInclusionFromWire(wire.body.indexProof),
};
}
return { sth, body };
}
// ─── Verifier ────────────────────────────────────────────
export interface KTVerifyOptions {
crypto: CryptoProvider;
/** Pinned log signing public key (Ed25519, 32 bytes). */
logPublicKey: Uint8Array;
/** Reject STH older than this many milliseconds. Default 24h. */
maxStaleMs?: number;
/** `now` for time-checks. Defaults to `Date.now()`. */
nowMs?: number;
/** Allow STH timestamps slightly in the future (clock-skew). Default 60 s. */
futureSkewMs?: number;
}
const DEFAULT_MAX_STALE_MS = 24 * 60 * 60 * 1000;
const DEFAULT_FUTURE_SKEW_MS = 60_000;
/**
* Verify an inclusion KT proof for a freshly fetched bundle.
*
* Throws on any failure (signature mismatch, stale STH, broken audit
* path, address-mismatch, bundle-hash mismatch). On success returns the
* STH so callers can cache it for split-view detection.
*/
export async function verifyBundleInclusion(
options: KTVerifyOptions,
address: string,
bundle: {
identitySigningKey: Uint8Array;
identityDHKey: Uint8Array;
signedPreKey: { keyId: number; publicKey: Uint8Array; signature: Uint8Array };
},
proof: KTProof,
): Promise<SignedTreeHead> {
if (proof.body.kind !== 'inclusion') {
throw new KTVerificationError(`expected inclusion proof, got ${proof.body.kind}`);
}
await verifyStaticSth(options, proof.sth);
// 1. Re-derive bundle_hash and confirm it matches the index entry.
const bundleHash = computeBundleHash(bundle);
const indexEntry = proof.body.indexProof.entry;
if (indexEntry.address !== address) {
throw new KTVerificationError(`index entry address mismatch: ${indexEntry.address} != ${address}`);
}
if (indexEntry.deleted) {
throw new KTVerificationError('index entry marked deleted, but inclusion proof claims live');
}
if (!constantTimeEqual(indexEntry.bundleHash, bundleHash)) {
throw new KTVerificationError('bundle hash does not match committed index entry');
}
// 2. Verify log inclusion.
const leafBytes = encodeLeafData(
proof.body.leafTimestampMs,
proof.body.operation,
address,
bundleHash,
);
const expectedLeafHash = leafHash(leafBytes);
let recomputedRoot: Uint8Array;
try {
recomputedRoot = recomputeRootFromAuditPath(
expectedLeafHash,
proof.body.leafIndex,
proof.sth.treeSize,
proof.body.auditPath,
);
} catch (err) {
throw new KTVerificationError(`audit path malformed: ${(err as Error).message}`);
}
if (!constantTimeEqual(recomputedRoot, proof.sth.rootHash)) {
throw new KTVerificationError('audit path does not yield STH root_hash');
}
// 3. Verify the index inclusion proof.
if (proof.body.indexProof.entry.latestLeafIndex !== proof.body.leafIndex) {
throw new KTVerificationError('index entry latestLeafIndex mismatch with log leaf');
}
if (!verifyInclusionProof(proof.body.indexProof, proof.sth.indexRoot)) {
throw new KTVerificationError('index inclusion proof failed');
}
return proof.sth;
}
/**
* Verify an absence KT proof — used when the server replies "no such
* address". The verifier returns `null` to indicate absence (vs. a
* verified live STH). On any failure it throws.
*/
export async function verifyBundleAbsence(
options: KTVerifyOptions,
address: string,
proof: KTProof,
): Promise<SignedTreeHead> {
if (proof.body.kind !== 'absence') {
throw new KTVerificationError(`expected absence proof, got ${proof.body.kind}`);
}
await verifyStaticSth(options, proof.sth);
if (proof.body.indexProof.queryAddress !== address) {
throw new KTVerificationError('absence proof query address mismatch');
}
if (!verifyAbsenceProof(proof.body.indexProof, proof.sth.indexRoot)) {
throw new KTVerificationError('absence proof failed');
}
return proof.sth;
}
/**
* Verify a tombstone proof — the address used to exist but has been
* deleted. The verifier returns the STH so split-view caching still
* applies; callers treat the bundle as "not available".
*/
export async function verifyBundleTombstone(
options: KTVerifyOptions,
address: string,
proof: KTProof,
): Promise<SignedTreeHead> {
if (proof.body.kind !== 'tombstone') {
throw new KTVerificationError(`expected tombstone proof, got ${proof.body.kind}`);
}
await verifyStaticSth(options, proof.sth);
if (proof.body.operation !== OP_DELETE) {
throw new KTVerificationError('tombstone proof must reference an OP_DELETE leaf');
}
if (proof.body.indexProof.entry.address !== address) {
throw new KTVerificationError('tombstone index entry address mismatch');
}
if (!proof.body.indexProof.entry.deleted) {
throw new KTVerificationError('tombstone proof index entry is not marked deleted');
}
// Deleted entries have empty bundleHash but the leaf data still uses an
// empty hash too; encode and verify the audit path with that.
const leafBytes = encodeLeafData(
proof.body.leafTimestampMs,
proof.body.operation,
address,
proof.body.indexProof.entry.bundleHash,
);
const expectedLeafHash = leafHash(leafBytes);
let recomputedRoot: Uint8Array;
try {
recomputedRoot = recomputeRootFromAuditPath(
expectedLeafHash,
proof.body.leafIndex,
proof.sth.treeSize,
proof.body.auditPath,
);
} catch (err) {
throw new KTVerificationError(`audit path malformed: ${(err as Error).message}`);
}
if (!constantTimeEqual(recomputedRoot, proof.sth.rootHash)) {
throw new KTVerificationError('tombstone audit path does not yield STH root_hash');
}
if (!verifyInclusionProof(proof.body.indexProof, proof.sth.indexRoot)) {
throw new KTVerificationError('tombstone index inclusion proof failed');
}
return proof.sth;
}
async function verifyStaticSth(options: KTVerifyOptions, sth: SignedTreeHead): Promise<void> {
const expectedLogId = computeLogId(options.logPublicKey);
if (!constantTimeEqual(expectedLogId, sth.logId)) {
throw new KTLogIdMismatchError('STH log_id does not match pinned log_public_key');
}
const sigOk = await verifySthSignature(options.crypto, sth, options.logPublicKey);
if (!sigOk) {
throw new KTVerificationError('STH signature did not verify');
}
// Encode entry into the in-memory canonical bytes used by the leaf and
// confirm the leaf actually re-encodes to the right bytes when the entry
// says it's present (we let the caller do this to keep per-mode flow
// small).
const now = options.nowMs ?? Date.now();
const maxStale = options.maxStaleMs ?? DEFAULT_MAX_STALE_MS;
const futureSkew = options.futureSkewMs ?? DEFAULT_FUTURE_SKEW_MS;
if (sth.timestampMs > now + futureSkew) {
throw new KTStaleSTHError(`STH timestamp in the future: ${sth.timestampMs} > ${now + futureSkew}`);
}
if (sth.timestampMs + maxStale < now) {
throw new KTStaleSTHError(`STH older than maxStale: ${now - sth.timestampMs}ms`);
}
}

View File

@@ -0,0 +1,12 @@
import { sha256 } from '@noble/hashes/sha2.js';
/**
* Synchronous SHA-256 — required because every Merkle hash composes
* many leaf/node hashes and an async API would force callers into
* promise chains for hot paths. `@noble/hashes` is the same dependency
* used elsewhere in the workspace (see `@shade/files`,
* `@shade/observability`).
*/
export function sha256Sync(data: Uint8Array): Uint8Array {
return sha256(data);
}

View File

@@ -0,0 +1,120 @@
/**
* Signed Tree Head (STH) — server's commitment to a tree state.
*
* canonical layout for signing:
* 0x02 (DOMAIN_STH) ||
* uint64_be tree_size ||
* uint64_be timestamp_ms ||
* root_hash (32 bytes) ||
* index_root (32 bytes) ||
* log_id (32 bytes)
*
* `log_id` is `SHA-256(log_public_key)` — a stable identifier that
* doesn't change unless the operator rotates the signing key.
*/
import type { CryptoProvider } from '@shade/core';
import { DOMAIN_STH } from './hashes.js';
import { sha256Sync } from './sha256.js';
import { constantTimeEqual } from './util.js';
export interface SignedTreeHead {
treeSize: number;
timestampMs: number;
rootHash: Uint8Array;
indexRoot: Uint8Array;
logId: Uint8Array;
signature: Uint8Array;
}
/** Compute log_id = SHA-256(public_key). */
export function computeLogId(logPublicKey: Uint8Array): Uint8Array {
return sha256Sync(logPublicKey);
}
/** Canonical bytes covered by the STH signature. */
export function canonicalSthBytes(sth: Omit<SignedTreeHead, 'signature'>): Uint8Array {
if (sth.rootHash.length !== 32) throw new Error('rootHash must be 32 bytes');
if (sth.indexRoot.length !== 32) throw new Error('indexRoot must be 32 bytes');
if (sth.logId.length !== 32) throw new Error('logId must be 32 bytes');
if (sth.treeSize < 0 || !Number.isFinite(sth.treeSize)) {
throw new Error('treeSize must be a non-negative integer');
}
const buf = new Uint8Array(1 + 8 + 8 + 32 + 32 + 32);
const view = new DataView(buf.buffer);
let off = 0;
buf[off++] = DOMAIN_STH;
view.setUint32(off, Math.floor(sth.treeSize / 0x100000000));
view.setUint32(off + 4, sth.treeSize >>> 0);
off += 8;
view.setUint32(off, Math.floor(sth.timestampMs / 0x100000000));
view.setUint32(off + 4, sth.timestampMs >>> 0);
off += 8;
buf.set(sth.rootHash, off);
off += 32;
buf.set(sth.indexRoot, off);
off += 32;
buf.set(sth.logId, off);
return buf;
}
/** Sign an STH with the operator's Ed25519 signing key. */
export async function signSth(
crypto: CryptoProvider,
signingPrivateKey: Uint8Array,
sth: Omit<SignedTreeHead, 'signature'>,
): Promise<SignedTreeHead> {
const message = canonicalSthBytes(sth);
const signature = await crypto.sign(signingPrivateKey, message);
return { ...sth, signature };
}
/**
* Verify the STH signature against a pinned `logPublicKey`.
*
* Also checks `logId === SHA-256(logPublicKey)` so a forged STH that
* claims a different log_id is rejected.
*/
export async function verifySthSignature(
crypto: CryptoProvider,
sth: SignedTreeHead,
logPublicKey: Uint8Array,
): Promise<boolean> {
const expectedLogId = computeLogId(logPublicKey);
if (!constantTimeEqual(expectedLogId, sth.logId)) return false;
const message = canonicalSthBytes(sth);
return crypto.verify(logPublicKey, message, sth.signature);
}
/** JSON-friendly STH for the wire (base64-encoded byte fields). */
export interface STHWire {
treeSize: number;
timestampMs: number;
rootHash: string;
indexRoot: string;
logId: string;
signature: string;
}
export function sthToWire(sth: SignedTreeHead, b64: (b: Uint8Array) => string): STHWire {
return {
treeSize: sth.treeSize,
timestampMs: sth.timestampMs,
rootHash: b64(sth.rootHash),
indexRoot: b64(sth.indexRoot),
logId: b64(sth.logId),
signature: b64(sth.signature),
};
}
export function sthFromWire(wire: STHWire, fromB64: (s: string) => Uint8Array): SignedTreeHead {
return {
treeSize: wire.treeSize,
timestampMs: wire.timestampMs,
rootHash: fromB64(wire.rootHash),
indexRoot: fromB64(wire.indexRoot),
logId: fromB64(wire.logId),
signature: fromB64(wire.signature),
};
}

View File

@@ -0,0 +1,59 @@
/**
* Persistent store interface for the server-side KT log + address index.
*
* Append-only invariant: implementations MUST NEVER overwrite or delete a
* row in the leaf table. The only mutation allowed is `appendLeaf`. Index
* entries (`upsertIndex`, `tombstoneIndex`) replace in place because the
* sorted index is a *projection* of the log — its history lives in the
* log itself.
*/
import type { SignedTreeHead } from './sth.js';
import type { AddressIndexEntry } from './index-tree.js';
export interface KTLogLeaf {
index: number;
leafHash: Uint8Array;
timestampMs: number;
operation: number;
address: string;
bundleHash: Uint8Array;
}
export interface KTLogStore {
/** Append a leaf. Returns assigned `index` (= prior size). */
appendLeaf(input: Omit<KTLogLeaf, 'index'>): Promise<number>;
/** Fetch leaves in [fromIndex, toIndex). For audit-path / consistency-proof builds. */
getLeaves(fromIndex: number, toIndex: number): Promise<KTLogLeaf[]>;
/** Fetch a single leaf. */
getLeaf(index: number): Promise<KTLogLeaf | null>;
/** Number of leaves currently in the log. */
size(): Promise<number>;
/** Upsert an address-index entry. */
upsertIndexEntry(entry: AddressIndexEntry): Promise<void>;
/** Tombstone an index entry (mark deleted). */
tombstoneIndexEntry(address: string, latestLeafIndex: number): Promise<void>;
/** Fetch the entire sorted index snapshot. */
getAllIndexEntries(): Promise<AddressIndexEntry[]>;
/** Fetch a single index entry by address. */
getIndexEntry(address: string): Promise<AddressIndexEntry | null>;
/** Persist a freshly-signed STH. */
saveSTH(sth: SignedTreeHead): Promise<void>;
/** Latest STH (most recent treeSize, most recent timestamp). */
getLatestSTH(): Promise<SignedTreeHead | null>;
/** STH at a specific tree size (for consistency proofs). */
getSTHByTreeSize(treeSize: number): Promise<SignedTreeHead | null>;
/** All STHs in (fromTimestamp, toTimestamp], sorted ascending. */
listSTHs(fromTimestampMs?: number, toTimestampMs?: number): Promise<SignedTreeHead[]>;
}

View File

@@ -0,0 +1,42 @@
/**
* Constant-time byte comparison. Mirrors `@shade/core` `constantTimeEqual`
* but is duplicated here so the KT package has no runtime dependency on
* `@shade/core` for primitive comparisons (it depends on it for error
* types only).
*/
export function constantTimeEqual(a: Uint8Array, b: Uint8Array): boolean {
if (a.length !== b.length) return false;
let diff = 0;
for (let i = 0; i < a.length; i++) {
diff |= a[i]! ^ b[i]!;
}
return diff === 0;
}
/**
* Encode bytes to standard base64 (no URL variant). Avoids platform-specific
* `Buffer` so the KT package works in browsers, Bun, and Workers.
*/
export function toBase64(bytes: Uint8Array): string {
if (typeof Buffer !== 'undefined') return Buffer.from(bytes).toString('base64');
let str = '';
for (let i = 0; i < bytes.length; i++) str += String.fromCharCode(bytes[i]!);
return btoa(str);
}
export function fromBase64(s: string): Uint8Array {
if (typeof Buffer !== 'undefined') return new Uint8Array(Buffer.from(s, 'base64'));
const str = atob(s);
const out = new Uint8Array(str.length);
for (let i = 0; i < str.length; i++) out[i] = str.charCodeAt(i);
return out;
}
/** Stable hex encoding for log_id rendering. */
export function toHex(bytes: Uint8Array): string {
let s = '';
for (let i = 0; i < bytes.length; i++) {
s += bytes[i]!.toString(16).padStart(2, '0');
}
return s;
}

View File

@@ -0,0 +1,239 @@
/**
* Light-witness — a passive observer of one or more KT logs.
*
* Responsibilities (V1):
* 1. Pin a log_public_key.
* 2. Periodically poll the server's `/v1/kt/sth` endpoint.
* 3. Verify each new STH's signature.
* 4. Maintain a chain of observed STHs and verify consistency proofs
* between successive observations.
* 5. Expose a "compare" API: given an STH another party has seen
* (e.g. delivered with a bundle-fetch), return whether *we* have
* seen the same `tree_size → root_hash → index_root` triple. A
* mismatch is a *split-view alarm*.
*
* V1 is library-only; deployments embed it in long-running processes
* (CLI tools, security-research auditors, server-to-server). V2 will
* add an HTTP `GET /witness/sth` endpoint for peer-to-peer gossip.
*/
import type { CryptoProvider } from '@shade/core';
import {
type SignedTreeHead,
type STHWire,
computeLogId,
sthFromWire,
verifySthSignature,
} from './sth.js';
import { verifyConsistencyProof } from './log.js';
import { fromBase64 } from './util.js';
import { constantTimeEqual } from './util.js';
import {
KTLogIdMismatchError,
KTSplitViewError,
KTStaleSTHError,
KTVerificationError,
} from './errors.js';
export interface WitnessFetcher {
/** GET latest STH. */
fetchLatestSTH(): Promise<STHWire>;
/** GET consistency proof from `fromTreeSize` to `toTreeSize`. */
fetchConsistencyProof(fromTreeSize: number, toTreeSize: number): Promise<{ proof: string[] }>;
}
export interface LightWitnessOptions {
crypto: CryptoProvider;
/** Pinned log signing public key (Ed25519, 32 bytes). */
logPublicKey: Uint8Array;
/** Source of STHs and consistency proofs. */
fetcher: WitnessFetcher;
/** Reject STH older than this many ms. Default 24h. */
maxStaleMs?: number;
/** Allowed clock-skew for STH future timestamps. Default 60 s. */
futureSkewMs?: number;
/** Time source. Defaults to `Date.now()`. */
now?: () => number;
/** Cap on stored STHs (LRU on tree_size). Default 1024. */
maxStored?: number;
}
const DEFAULT_MAX_STALE_MS = 24 * 60 * 60 * 1000;
const DEFAULT_FUTURE_SKEW_MS = 60_000;
const DEFAULT_MAX_STORED = 1024;
/** A piece of evidence that a particular STH was observed. */
export interface WitnessObservation {
sth: SignedTreeHead;
observedAtMs: number;
}
export class LightWitness {
private readonly crypto: CryptoProvider;
private readonly logPublicKey: Uint8Array;
private readonly fetcher: WitnessFetcher;
private readonly logId: Uint8Array;
private readonly maxStaleMs: number;
private readonly futureSkewMs: number;
private readonly now: () => number;
private readonly maxStored: number;
/** STHs we've observed, indexed by tree_size. */
private observed = new Map<number, SignedTreeHead>();
private observedOrder: number[] = []; // insertion order for LRU eviction
constructor(opts: LightWitnessOptions) {
this.crypto = opts.crypto;
this.logPublicKey = opts.logPublicKey;
this.fetcher = opts.fetcher;
this.logId = computeLogId(opts.logPublicKey);
this.maxStaleMs = opts.maxStaleMs ?? DEFAULT_MAX_STALE_MS;
this.futureSkewMs = opts.futureSkewMs ?? DEFAULT_FUTURE_SKEW_MS;
this.now = opts.now ?? (() => Date.now());
this.maxStored = opts.maxStored ?? DEFAULT_MAX_STORED;
}
/** Pinned log_id for callers that want to verify message routing. */
getLogId(): Uint8Array {
return new Uint8Array(this.logId);
}
/**
* Poll the server for the latest STH. Verifies signature, freshness, and
* consistency with the most recent observation we hold. Adds the STH to
* the local set on success.
*/
async pollOnce(): Promise<SignedTreeHead> {
const wire = await this.fetcher.fetchLatestSTH();
const sth = sthFromWire(wire, fromBase64);
await this.observe(sth);
return sth;
}
/**
* Ingest an STH supplied externally (e.g. embedded in a bundle-fetch
* response). Re-uses all the same verification & comparison logic so
* proofs returned by the bundle-fetch path also feed into split-view
* detection.
*/
async observe(sth: SignedTreeHead): Promise<void> {
if (!constantTimeEqual(sth.logId, this.logId)) {
throw new KTLogIdMismatchError(`STH log_id does not match pinned key`);
}
const ok = await verifySthSignature(this.crypto, sth, this.logPublicKey);
if (!ok) throw new KTVerificationError('STH signature did not verify');
const now = this.now();
if (sth.timestampMs > now + this.futureSkewMs) {
throw new KTStaleSTHError(`STH timestamp in the future`);
}
if (sth.timestampMs + this.maxStaleMs < now) {
throw new KTStaleSTHError(`STH older than maxStale`);
}
// Split-view check — same tree_size, different root or index_root → fork.
const prior = this.observed.get(sth.treeSize);
if (prior) {
if (
!constantTimeEqual(prior.rootHash, sth.rootHash) ||
!constantTimeEqual(prior.indexRoot, sth.indexRoot)
) {
throw new KTSplitViewError(
`Split view: two STHs at tree_size=${sth.treeSize} disagree`,
);
}
// Same STH content; nothing to insert (might just be a refreshed timestamp).
// Keep the freshest one for staleness checks.
if (sth.timestampMs > prior.timestampMs) {
this.observed.set(sth.treeSize, sth);
}
return;
}
// Consistency check against our most recent prior observation.
const latest = this.latestObserved();
if (latest && latest.treeSize !== sth.treeSize) {
const [oldSth, newSth] = latest.treeSize < sth.treeSize ? [latest, sth] : [sth, latest];
if (oldSth.treeSize > 0 && oldSth.treeSize < newSth.treeSize) {
const { proof } = await this.fetcher.fetchConsistencyProof(
oldSth.treeSize,
newSth.treeSize,
);
const proofBytes = proof.map(fromBase64);
const consistent = verifyConsistencyProof(
oldSth.treeSize,
newSth.treeSize,
oldSth.rootHash,
newSth.rootHash,
proofBytes,
);
if (!consistent) {
throw new KTVerificationError(
`Consistency proof failed: ${oldSth.treeSize}${newSth.treeSize}`,
);
}
}
}
this.insertObservation(sth);
}
/**
* Compare an external STH (e.g. one another client claims to have seen)
* against our stored set. Returns:
* - 'agree' if we've observed the same tree_size with the same roots,
* - 'unknown' if we have no STH at that tree_size (caller may want to
* poll once more before deciding),
* - 'split-view' if our roots differ.
*/
compare(sth: SignedTreeHead): 'agree' | 'unknown' | 'split-view' {
const ours = this.observed.get(sth.treeSize);
if (!ours) return 'unknown';
const sameRoot = constantTimeEqual(ours.rootHash, sth.rootHash);
const sameIndex = constantTimeEqual(ours.indexRoot, sth.indexRoot);
if (sameRoot && sameIndex) return 'agree';
return 'split-view';
}
/** Latest observed STH (highest tree_size). */
latestObserved(): SignedTreeHead | null {
let best: SignedTreeHead | null = null;
for (const sth of this.observed.values()) {
if (!best || sth.treeSize > best.treeSize) best = sth;
}
return best;
}
/** Snapshot all observed STHs (defensive copy). */
observations(): WitnessObservation[] {
return Array.from(this.observed.values()).map((sth) => ({
sth: {
treeSize: sth.treeSize,
timestampMs: sth.timestampMs,
rootHash: new Uint8Array(sth.rootHash),
indexRoot: new Uint8Array(sth.indexRoot),
logId: new Uint8Array(sth.logId),
signature: new Uint8Array(sth.signature),
},
observedAtMs: this.now(),
}));
}
private insertObservation(sth: SignedTreeHead): void {
if (!this.observed.has(sth.treeSize)) {
this.observedOrder.push(sth.treeSize);
}
this.observed.set(sth.treeSize, sth);
while (this.observedOrder.length > this.maxStored) {
const evict = this.observedOrder.shift();
if (evict !== undefined && evict !== sth.treeSize) {
// Always keep the latest STH even under aggressive eviction.
if (this.latestObserved()?.treeSize !== evict) {
this.observed.delete(evict);
}
}
}
}
}

View File

@@ -0,0 +1,127 @@
import { describe, expect, test } from 'bun:test';
import {
DOMAIN_BUNDLE,
computeBundleHash,
encodeLeafData,
leafHash,
nodeHash,
OP_REGISTER,
} from '../src/hashes.js';
import { sha256Sync } from '../src/sha256.js';
describe('RFC 6962 hash primitives', () => {
test('leafHash applies 0x00 prefix', () => {
const data = new Uint8Array([1, 2, 3]);
const expected = sha256Sync(new Uint8Array([0x00, 1, 2, 3]));
expect(leafHash(data)).toEqual(expected);
});
test('nodeHash applies 0x01 prefix', () => {
const left = new Uint8Array(32).fill(0xaa);
const right = new Uint8Array(32).fill(0xbb);
const concat = new Uint8Array(1 + 32 + 32);
concat[0] = 0x01;
concat.set(left, 1);
concat.set(right, 33);
const expected = sha256Sync(concat);
expect(nodeHash(left, right)).toEqual(expected);
});
test('leafHash and nodeHash never collide', () => {
// Same content but different domain → different hash
const x = new Uint8Array(32).fill(0x42);
const lh = leafHash(x);
const concat = new Uint8Array(64).fill(0x42);
const nh = nodeHash(concat.slice(0, 32), concat.slice(32));
expect(Buffer.from(lh).toString('hex')).not.toBe(Buffer.from(nh).toString('hex'));
});
});
describe('encodeLeafData', () => {
test('encodes timestamp + operation + address + bundleHash', () => {
const buf = encodeLeafData(
0x010203040506,
OP_REGISTER,
'alice',
new Uint8Array([0xde, 0xad, 0xbe, 0xef]),
);
// 8 + 1 + 2 + 5 + 2 + 4 = 22
expect(buf.length).toBe(22);
// last 4 bytes = bundleHash
expect(buf[buf.length - 4]).toBe(0xde);
expect(buf[buf.length - 1]).toBe(0xef);
});
test('rejects address > 65535 bytes', () => {
const huge = 'a'.repeat(0x10000);
expect(() => encodeLeafData(0, OP_REGISTER, huge, new Uint8Array(0))).toThrow();
});
});
describe('computeBundleHash', () => {
test('deterministic over equal input', () => {
const sk = new Uint8Array(32).fill(0x11);
const dk = new Uint8Array(32).fill(0x22);
const pk = new Uint8Array(32).fill(0x33);
const sig = new Uint8Array(64).fill(0x44);
const a = computeBundleHash({
identitySigningKey: sk,
identityDHKey: dk,
signedPreKey: { keyId: 7, publicKey: pk, signature: sig },
});
const b = computeBundleHash({
identitySigningKey: sk,
identityDHKey: dk,
signedPreKey: { keyId: 7, publicKey: pk, signature: sig },
});
expect(a).toEqual(b);
});
test('changing keyId changes hash', () => {
const sk = new Uint8Array(32).fill(0x11);
const dk = new Uint8Array(32).fill(0x22);
const pk = new Uint8Array(32).fill(0x33);
const sig = new Uint8Array(64).fill(0x44);
const a = computeBundleHash({
identitySigningKey: sk,
identityDHKey: dk,
signedPreKey: { keyId: 1, publicKey: pk, signature: sig },
});
const b = computeBundleHash({
identitySigningKey: sk,
identityDHKey: dk,
signedPreKey: { keyId: 2, publicKey: pk, signature: sig },
});
expect(a).not.toEqual(b);
});
test('rejects wrong-length keys', () => {
expect(() =>
computeBundleHash({
identitySigningKey: new Uint8Array(31),
identityDHKey: new Uint8Array(32),
signedPreKey: {
keyId: 0,
publicKey: new Uint8Array(32),
signature: new Uint8Array(64),
},
}),
).toThrow();
});
test('uses domain prefix 0x01', () => {
const sk = new Uint8Array(32);
const dk = new Uint8Array(32);
const pk = new Uint8Array(32);
const sig = new Uint8Array(64);
const expected = sha256Sync(
Buffer.concat([Buffer.from([DOMAIN_BUNDLE]), sk, dk, Buffer.alloc(4), pk, sig]),
);
const got = computeBundleHash({
identitySigningKey: sk,
identityDHKey: dk,
signedPreKey: { keyId: 0, publicKey: pk, signature: sig },
});
expect(Buffer.from(got).toString('hex')).toBe(Buffer.from(expected).toString('hex'));
});
});

View File

@@ -0,0 +1,221 @@
import { describe, expect, test } from 'bun:test';
import {
AddressIndex,
compareAddresses,
computeIndexRoot,
emptyRootHash,
verifyAbsenceProof,
verifyInclusionProof,
} from '../src/index.js';
describe('AddressIndex', () => {
test('empty index has emptyRootHash root', () => {
const idx = new AddressIndex();
expect(idx.rootHash()).toEqual(emptyRootHash());
});
test('upsert keeps entries lexicographically sorted', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'charlie',
latestLeafIndex: 1,
bundleHash: new Uint8Array(32).fill(3),
deleted: false,
});
idx.upsert({
address: 'alice',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
idx.upsert({
address: 'bob',
latestLeafIndex: 2,
bundleHash: new Uint8Array(32).fill(2),
deleted: false,
});
const snap = idx.snapshot();
expect(snap.map((e) => e.address)).toEqual(['alice', 'bob', 'charlie']);
});
test('compareAddresses is byte-lex', () => {
expect(compareAddresses('alice', 'bob') < 0).toBe(true);
expect(compareAddresses('bob', 'alice') > 0).toBe(true);
expect(compareAddresses('alice', 'alice')).toBe(0);
expect(compareAddresses('alice', 'aliceb')).toBeLessThan(0);
});
test('inclusion proof verifies against rootHash', () => {
const idx = new AddressIndex();
for (const a of ['alice', 'bob', 'charlie', 'dave', 'eve']) {
idx.upsert({
address: a,
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(a.charCodeAt(0)),
deleted: false,
});
}
const proof = idx.inclusionProof('charlie');
expect(proof).not.toBeNull();
expect(verifyInclusionProof(proof!, idx.rootHash())).toBe(true);
});
test('inclusion proof fails against tampered root', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'alice',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
const proof = idx.inclusionProof('alice')!;
const tampered = new Uint8Array(idx.rootHash());
tampered[0] ^= 0xff;
expect(verifyInclusionProof(proof, tampered)).toBe(false);
});
test('absence proof: query between two entries', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'alice',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
idx.upsert({
address: 'charlie',
latestLeafIndex: 1,
bundleHash: new Uint8Array(32).fill(3),
deleted: false,
});
const absence = idx.absenceProof('bob');
expect(absence).not.toBeNull();
expect(verifyAbsenceProof(absence!, idx.rootHash())).toBe(true);
});
test('absence proof: query before first entry', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'm',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
idx.upsert({
address: 'z',
latestLeafIndex: 1,
bundleHash: new Uint8Array(32).fill(2),
deleted: false,
});
const absence = idx.absenceProof('a');
expect(absence!.prev).toBeNull();
expect(absence!.next).not.toBeNull();
expect(verifyAbsenceProof(absence!, idx.rootHash())).toBe(true);
});
test('absence proof: query after last entry', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'a',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
idx.upsert({
address: 'm',
latestLeafIndex: 1,
bundleHash: new Uint8Array(32).fill(2),
deleted: false,
});
const absence = idx.absenceProof('z');
expect(absence!.prev).not.toBeNull();
expect(absence!.next).toBeNull();
expect(verifyAbsenceProof(absence!, idx.rootHash())).toBe(true);
});
test('absence proof: empty tree', () => {
const idx = new AddressIndex();
const absence = idx.absenceProof('alice');
expect(absence).not.toBeNull();
expect(absence!.treeSize).toBe(0);
expect(verifyAbsenceProof(absence!, idx.rootHash())).toBe(true);
});
test('absence proof returns null for existing address', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'alice',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
expect(idx.absenceProof('alice')).toBeNull();
});
test('absence proof can be forged-detected: claim adjacent but not adjacent', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'alice',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
idx.upsert({
address: 'bob',
latestLeafIndex: 1,
bundleHash: new Uint8Array(32).fill(2),
deleted: false,
});
idx.upsert({
address: 'charlie',
latestLeafIndex: 2,
bundleHash: new Uint8Array(32).fill(3),
deleted: false,
});
const absence = idx.absenceProof('aaron')!;
// Tamper: replace prev with non-adjacent neighbor (charlie)
const charlieProof = idx.inclusionProof('charlie')!;
const forged = {
...absence,
prev: {
position: charlieProof.position,
entry: charlieProof.entry,
auditPath: charlieProof.auditPath,
},
};
expect(verifyAbsenceProof(forged, idx.rootHash())).toBe(false);
});
test('tombstone marks entry deleted', () => {
const idx = new AddressIndex();
idx.upsert({
address: 'alice',
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(1),
deleted: false,
});
idx.tombstone('alice', 5);
const e = idx.get('alice')!;
expect(e.deleted).toBe(true);
expect(e.latestLeafIndex).toBe(5);
expect(e.bundleHash.length).toBe(0);
});
test('computeIndexRoot equals AddressIndex.rootHash for the same sorted snapshot', () => {
const idx = new AddressIndex();
for (const a of ['carol', 'alice', 'bob']) {
idx.upsert({
address: a,
latestLeafIndex: 0,
bundleHash: new Uint8Array(32).fill(a.charCodeAt(0)),
deleted: false,
});
}
expect(idx.rootHash()).toEqual(computeIndexRoot(idx.snapshot()));
});
});

View File

@@ -0,0 +1,189 @@
import { describe, expect, test } from 'bun:test';
import fc from 'fast-check';
import {
MerkleLog,
auditPath,
consistencyProof,
emptyRootHash,
leafHash,
nodeHash,
recomputeRootFromAuditPath,
verifyConsistencyProof,
} from '../src/index.js';
import { mth } from '../src/log.js';
function buildLog(n: number): MerkleLog {
const log = new MerkleLog();
for (let i = 0; i < n; i++) {
log.appendData(new Uint8Array([i & 0xff, (i >> 8) & 0xff]));
}
return log;
}
describe('MerkleLog basics', () => {
test('empty tree root = SHA-256(empty)', () => {
const log = new MerkleLog();
expect(log.rootHash()).toEqual(emptyRootHash());
});
test('single-leaf tree root = leaf_hash(d)', () => {
const log = new MerkleLog();
const d = new Uint8Array([0xab, 0xcd]);
log.appendData(d);
expect(log.rootHash()).toEqual(leafHash(d));
});
test('two-leaf tree root = node_hash(leaf0, leaf1)', () => {
const log = new MerkleLog();
const d0 = new Uint8Array([1]);
const d1 = new Uint8Array([2]);
log.appendData(d0);
log.appendData(d1);
expect(log.rootHash()).toEqual(nodeHash(leafHash(d0), leafHash(d1)));
});
test('append-only ordering preserved', () => {
const log = new MerkleLog();
log.appendData(new Uint8Array([1]));
log.appendData(new Uint8Array([2]));
expect(log.size).toBe(2);
log.appendData(new Uint8Array([3]));
expect(log.size).toBe(3);
});
});
describe('Audit path verification', () => {
test('valid audit path reconstructs root for every leaf, every size 1..32', () => {
for (let n = 1; n <= 32; n++) {
const log = buildLog(n);
const root = log.rootHash();
for (let m = 0; m < n; m++) {
const path = log.auditPath(m);
const lh = log.leafHashAt(m);
const reconstructed = recomputeRootFromAuditPath(lh, m, n, path);
expect(Buffer.from(reconstructed).toString('hex')).toBe(
Buffer.from(root).toString('hex'),
);
}
}
});
test('property: tampered leaf fails verification', () => {
const log = buildLog(7);
const root = log.rootHash();
const path = log.auditPath(3);
const tampered = new Uint8Array(log.leafHashAt(3));
tampered[0] ^= 0xff;
const reconstructed = recomputeRootFromAuditPath(tampered, 3, 7, path);
expect(Buffer.from(reconstructed).toString('hex')).not.toBe(
Buffer.from(root).toString('hex'),
);
});
test('property: tampered audit path fails verification', () => {
const log = buildLog(11);
const root = log.rootHash();
const path = log.auditPath(5);
if (path.length === 0) return;
const tampered = path.map((p, i) => {
if (i === 0) {
const x = new Uint8Array(p);
x[0] ^= 0xff;
return x;
}
return p;
});
const reconstructed = recomputeRootFromAuditPath(log.leafHashAt(5), 5, 11, tampered);
expect(Buffer.from(reconstructed).toString('hex')).not.toBe(
Buffer.from(root).toString('hex'),
);
});
test('property-based: random N and m', () => {
fc.assert(
fc.property(fc.integer({ min: 1, max: 100 }), (n) => {
const log = buildLog(n);
const root = log.rootHash();
const m = Math.min(n - 1, Math.floor(Math.random() * n));
const path = log.auditPath(m);
const reconstructed = recomputeRootFromAuditPath(log.leafHashAt(m), m, n, path);
return Buffer.from(reconstructed).toString('hex') === Buffer.from(root).toString('hex');
}),
{ numRuns: 50 },
);
});
});
describe('Consistency proofs', () => {
test('m === 0 always consistent', () => {
const log = buildLog(5);
const newRoot = log.rootHash();
expect(verifyConsistencyProof(0, 5, emptyRootHash(), newRoot, [])).toBe(true);
});
test('m === n consistent only when both roots match and proof empty', () => {
const log = buildLog(5);
const root = log.rootHash();
expect(verifyConsistencyProof(5, 5, root, root, [])).toBe(true);
const wrong = new Uint8Array(root);
wrong[0] ^= 0xff;
expect(verifyConsistencyProof(5, 5, wrong, root, [])).toBe(false);
});
test('valid proof verifies for every (m, n) up to 16', () => {
for (let n = 1; n <= 16; n++) {
const newLog = buildLog(n);
const newRoot = newLog.rootHash();
for (let m = 0; m <= n; m++) {
const oldLog = buildLog(m);
const oldRoot = oldLog.rootHash();
const proof = newLog.consistencyProof(m);
expect(verifyConsistencyProof(m, n, oldRoot, newRoot, proof)).toBe(true);
}
}
});
test('fork detection — re-write history fails consistency', () => {
const original = buildLog(5);
const oldRoot = original.rootHash();
// Server "rewrites" leaf 2: build a new log where leaf 2 has different data.
const tampered = new MerkleLog();
for (let i = 0; i < 5; i++) {
tampered.appendData(
i === 2 ? new Uint8Array([0x99, 0x99]) : new Uint8Array([i & 0xff, (i >> 8) & 0xff]),
);
}
tampered.appendData(new Uint8Array([0x77]));
const tamperedRoot = tampered.rootHash();
const proof = tampered.consistencyProof(5);
expect(verifyConsistencyProof(5, 6, oldRoot, tamperedRoot, proof)).toBe(false);
});
});
describe('mth helper', () => {
test('mth over slice == sub-tree root', () => {
const log = buildLog(4);
const leaves = log.exportLeaves();
const left = mth(leaves, 0, 2);
const right = mth(leaves, 2, 4);
const root = mth(leaves, 0, 4);
expect(root).toEqual(nodeHash(left, right));
});
});
describe('Direct auditPath helper (tree size 1)', () => {
test('singleton tree audit path is empty', () => {
const log = buildLog(1);
const path = log.auditPath(0);
expect(path.length).toBe(0);
expect(consistencyProof([log.leafHashAt(0)], 1, 1)).toEqual([]);
});
test('auditPath out-of-range throws', () => {
expect(() => auditPath([], 0, 0)).toThrow();
const log = buildLog(3);
expect(() => log.auditPath(3)).toThrow();
});
});

View File

@@ -0,0 +1,223 @@
import { describe, expect, test } from 'bun:test';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import {
KTLogManager,
MemoryKTLogStore,
computeBundleHash,
ktProofFromWire,
ktProofToWire,
verifyBundleAbsence,
verifyBundleInclusion,
verifyBundleTombstone,
} from '../src/index.js';
import { KTSplitViewError, KTVerificationError } from '../src/errors.js';
const crypto = new SubtleCryptoProvider();
async function makeManager() {
const kp = await crypto.generateEd25519KeyPair();
const store = new MemoryKTLogStore();
const mgr = await KTLogManager.create({
crypto,
store,
signingPrivateKey: kp.privateKey,
signingPublicKey: kp.publicKey,
});
return { mgr, kp, store };
}
function fakeBundle(seed: number) {
return {
identitySigningKey: new Uint8Array(32).fill(seed),
identityDHKey: new Uint8Array(32).fill(seed + 1),
signedPreKey: {
keyId: 1,
publicKey: new Uint8Array(32).fill(seed + 2),
signature: new Uint8Array(64).fill(seed + 3),
},
};
}
describe('KTLogManager — happy paths', () => {
test('register + buildBundleInclusionProof + verify', async () => {
const { mgr, kp } = await makeManager();
const bundle = fakeBundle(0x10);
const bundleHash = computeBundleHash(bundle);
await mgr.recordRegister('alice', bundleHash);
const sth = await mgr.publishSTH();
const proof = await mgr.buildBundleInclusionProof('alice', sth);
expect(proof).not.toBeNull();
expect(proof!.body.kind).toBe('inclusion');
const verified = await verifyBundleInclusion(
{ crypto, logPublicKey: kp.publicKey },
'alice',
bundle,
proof!,
);
expect(verified.treeSize).toBe(1);
});
test('absence proof for unknown address verifies', async () => {
const { mgr, kp } = await makeManager();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth = await mgr.publishSTH();
const proof = mgr.buildBundleAbsenceProof('zeta', sth);
expect(proof).not.toBeNull();
expect(proof!.body.kind).toBe('absence');
await verifyBundleAbsence({ crypto, logPublicKey: kp.publicKey }, 'zeta', proof!);
});
test('tombstone proof verifies after delete', async () => {
const { mgr, kp } = await makeManager();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
await mgr.recordDelete('alice');
const sth = await mgr.publishSTH();
const proof = await mgr.buildBundleInclusionProof('alice', sth);
expect(proof!.body.kind).toBe('tombstone');
const verified = await verifyBundleTombstone(
{ crypto, logPublicKey: kp.publicKey },
'alice',
proof!,
);
expect(verified.treeSize).toBe(2);
});
test('multiple addresses + STH at increasing tree sizes', async () => {
const { mgr, kp } = await makeManager();
const aliceBundle = fakeBundle(0x10);
const bobBundle = fakeBundle(0x20);
await mgr.recordRegister('alice', computeBundleHash(aliceBundle));
await mgr.recordRegister('bob', computeBundleHash(bobBundle));
const sth = await mgr.publishSTH();
expect(sth.treeSize).toBe(2);
const proofAlice = await mgr.buildBundleInclusionProof('alice', sth);
const proofBob = await mgr.buildBundleInclusionProof('bob', sth);
await verifyBundleInclusion(
{ crypto, logPublicKey: kp.publicKey },
'alice',
aliceBundle,
proofAlice!,
);
await verifyBundleInclusion(
{ crypto, logPublicKey: kp.publicKey },
'bob',
bobBundle,
proofBob!,
);
});
test('rotation: new register replaces old', async () => {
const { mgr, kp } = await makeManager();
const v1 = fakeBundle(0x10);
const v2 = fakeBundle(0x55);
await mgr.recordRegister('alice', computeBundleHash(v1));
await mgr.recordRegister('alice', computeBundleHash(v2));
const sth = await mgr.publishSTH();
const proof = await mgr.buildBundleInclusionProof('alice', sth);
// Latest is v2; verifying with v1's bundle should fail.
await expect(
verifyBundleInclusion({ crypto, logPublicKey: kp.publicKey }, 'alice', v1, proof!),
).rejects.toBeInstanceOf(KTVerificationError);
await verifyBundleInclusion(
{ crypto, logPublicKey: kp.publicKey },
'alice',
v2,
proof!,
);
});
});
describe('KTLogManager — wire encoding roundtrip', () => {
test('inclusion proof survives wire roundtrip', async () => {
const { mgr, kp } = await makeManager();
const bundle = fakeBundle(0x42);
await mgr.recordRegister('alice', computeBundleHash(bundle));
const sth = await mgr.publishSTH();
const proof = await mgr.buildBundleInclusionProof('alice', sth);
const wire = ktProofToWire(proof!);
const json = JSON.stringify(wire);
const back = ktProofFromWire(JSON.parse(json));
await verifyBundleInclusion(
{ crypto, logPublicKey: kp.publicKey },
'alice',
bundle,
back,
);
});
});
describe('Tampering detection', () => {
test('forged bundle (different signing key) is rejected', async () => {
const { mgr, kp } = await makeManager();
const real = fakeBundle(0x10);
await mgr.recordRegister('alice', computeBundleHash(real));
const sth = await mgr.publishSTH();
const proof = await mgr.buildBundleInclusionProof('alice', sth);
const forged = { ...real, identitySigningKey: new Uint8Array(32).fill(0xff) };
await expect(
verifyBundleInclusion({ crypto, logPublicKey: kp.publicKey }, 'alice', forged, proof!),
).rejects.toBeInstanceOf(KTVerificationError);
});
test('proof for alice cannot be re-used for bob (address mismatch)', async () => {
const { mgr, kp } = await makeManager();
const aliceBundle = fakeBundle(0x10);
await mgr.recordRegister('alice', computeBundleHash(aliceBundle));
const sth = await mgr.publishSTH();
const proof = await mgr.buildBundleInclusionProof('alice', sth);
await expect(
verifyBundleInclusion(
{ crypto, logPublicKey: kp.publicKey },
'bob',
aliceBundle,
proof!,
),
).rejects.toBeInstanceOf(KTVerificationError);
});
test('split-view: same tree_size with different roots is detected by witness', async () => {
const { mgr, kp } = await makeManager();
const bundle = fakeBundle(0x10);
await mgr.recordRegister('alice', computeBundleHash(bundle));
const sth = await mgr.publishSTH();
// Forge a *different* STH at the same tree_size — pretend the server
// signed two divergent versions.
const sth2 = await mgr.publishSTH();
expect(sth2.treeSize).toBe(sth.treeSize);
// To simulate a conflicting STH, sign one with a tampered root_hash.
const tampered = { ...sth, rootHash: new Uint8Array(sth.rootHash) };
tampered.rootHash[0] ^= 0xff;
// Re-sign with the same key so it would individually verify…
const forged = await (await import('../src/sth.js')).signSth(crypto, kp.privateKey, {
treeSize: tampered.treeSize,
timestampMs: tampered.timestampMs,
rootHash: tampered.rootHash,
indexRoot: tampered.indexRoot,
logId: tampered.logId,
});
const { LightWitness } = await import('../src/witness.js');
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return (await import('../src/sth.js')).sthToWire(sth, (b) =>
Buffer.from(b).toString('base64'),
);
},
async fetchConsistencyProof() {
return { proof: [] };
},
},
});
await witness.observe(sth);
await expect(witness.observe(forged)).rejects.toBeInstanceOf(KTSplitViewError);
});
});

View File

@@ -0,0 +1,83 @@
import { describe, expect, test } from 'bun:test';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import {
canonicalSthBytes,
computeLogId,
signSth,
sthFromWire,
sthToWire,
verifySthSignature,
} from '../src/index.js';
import { fromBase64, toBase64 } from '../src/util.js';
const crypto = new SubtleCryptoProvider();
describe('STH signing & verification', () => {
test('signSth + verifySthSignature roundtrip', async () => {
const kp = await crypto.generateEd25519KeyPair();
const sth = await signSth(crypto, kp.privateKey, {
treeSize: 42,
timestampMs: 1700000000000,
rootHash: new Uint8Array(32).fill(0xaa),
indexRoot: new Uint8Array(32).fill(0xbb),
logId: computeLogId(kp.publicKey),
});
expect(await verifySthSignature(crypto, sth, kp.publicKey)).toBe(true);
});
test('verify fails with wrong public key', async () => {
const kp = await crypto.generateEd25519KeyPair();
const other = await crypto.generateEd25519KeyPair();
const sth = await signSth(crypto, kp.privateKey, {
treeSize: 1,
timestampMs: 1700000000000,
rootHash: new Uint8Array(32),
indexRoot: new Uint8Array(32),
logId: computeLogId(kp.publicKey),
});
expect(await verifySthSignature(crypto, sth, other.publicKey)).toBe(false);
});
test('verify fails when log_id is forged', async () => {
const kp = await crypto.generateEd25519KeyPair();
const other = await crypto.generateEd25519KeyPair();
const sth = await signSth(crypto, kp.privateKey, {
treeSize: 1,
timestampMs: 1700000000000,
rootHash: new Uint8Array(32),
indexRoot: new Uint8Array(32),
logId: computeLogId(other.publicKey), // mismatched
});
// The signature was made over a log_id that doesn't match the supplied
// public key — verifySthSignature should refuse.
expect(await verifySthSignature(crypto, sth, kp.publicKey)).toBe(false);
});
test('canonical bytes layout is stable', () => {
const bytes = canonicalSthBytes({
treeSize: 0x0102030405,
timestampMs: 0x06070809,
rootHash: new Uint8Array(32).fill(0x11),
indexRoot: new Uint8Array(32).fill(0x22),
logId: new Uint8Array(32).fill(0x33),
});
// 1 prefix + 8 treeSize + 8 timestamp + 32 + 32 + 32
expect(bytes.length).toBe(113);
expect(bytes[0]).toBe(0x02);
});
test('wire roundtrip', async () => {
const kp = await crypto.generateEd25519KeyPair();
const sth = await signSth(crypto, kp.privateKey, {
treeSize: 7,
timestampMs: 1700000000000,
rootHash: new Uint8Array(32).fill(0x77),
indexRoot: new Uint8Array(32).fill(0x88),
logId: computeLogId(kp.publicKey),
});
const wire = sthToWire(sth, toBase64);
const back = sthFromWire(wire, fromBase64);
expect(back).toEqual(sth);
expect(await verifySthSignature(crypto, back, kp.publicKey)).toBe(true);
});
});

View File

@@ -0,0 +1,223 @@
import { describe, expect, test } from 'bun:test';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import {
KTLogManager,
LightWitness,
MemoryKTLogStore,
computeBundleHash,
computeLogId,
signSth,
sthToWire,
} from '../src/index.js';
import {
KTLogIdMismatchError,
KTSplitViewError,
KTStaleSTHError,
KTVerificationError,
} from '../src/errors.js';
import { toBase64, fromBase64 } from '../src/util.js';
const crypto = new SubtleCryptoProvider();
async function setup() {
const kp = await crypto.generateEd25519KeyPair();
const store = new MemoryKTLogStore();
const mgr = await KTLogManager.create({
crypto,
store,
signingPrivateKey: kp.privateKey,
signingPublicKey: kp.publicKey,
});
return { kp, mgr };
}
function fakeBundle(seed: number) {
return {
identitySigningKey: new Uint8Array(32).fill(seed),
identityDHKey: new Uint8Array(32).fill(seed + 1),
signedPreKey: {
keyId: 1,
publicKey: new Uint8Array(32).fill(seed + 2),
signature: new Uint8Array(64).fill(seed + 3),
},
};
}
describe('LightWitness', () => {
test('observes valid STH and stores it', async () => {
const { kp, mgr } = await setup();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth = await mgr.publishSTH();
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return sthToWire(sth, toBase64);
},
async fetchConsistencyProof() {
return { proof: [] };
},
},
});
const polled = await witness.pollOnce();
expect(polled.treeSize).toBe(1);
expect(witness.compare(sth)).toBe('agree');
});
test('rejects STH whose log_id does not match pinned key', async () => {
const { mgr } = await setup();
const wrong = await crypto.generateEd25519KeyPair();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth = await mgr.publishSTH();
const witness = new LightWitness({
crypto,
logPublicKey: wrong.publicKey, // pinned to wrong key
fetcher: {
async fetchLatestSTH() {
return sthToWire(sth, toBase64);
},
async fetchConsistencyProof() {
return { proof: [] };
},
},
});
await expect(witness.pollOnce()).rejects.toBeInstanceOf(KTLogIdMismatchError);
});
test('rejects STH older than maxStaleMs', async () => {
const { kp, mgr } = await setup();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth = await mgr.publishSTH(1000); // far in the past
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return sthToWire(sth, toBase64);
},
async fetchConsistencyProof() {
return { proof: [] };
},
},
maxStaleMs: 1000,
now: () => 10_000_000,
});
await expect(witness.pollOnce()).rejects.toBeInstanceOf(KTStaleSTHError);
});
test('detects split-view at same tree_size', async () => {
const { kp, mgr } = await setup();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth1 = await mgr.publishSTH();
// Forge another signed STH with same tree_size but different rootHash
const tamperedRoot = new Uint8Array(sth1.rootHash);
tamperedRoot[0] ^= 0xff;
const sth2 = await signSth(crypto, kp.privateKey, {
treeSize: sth1.treeSize,
timestampMs: sth1.timestampMs,
rootHash: tamperedRoot,
indexRoot: sth1.indexRoot,
logId: computeLogId(kp.publicKey),
});
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return sthToWire(sth1, toBase64);
},
async fetchConsistencyProof() {
return { proof: [] };
},
},
});
await witness.observe(sth1);
await expect(witness.observe(sth2)).rejects.toBeInstanceOf(KTSplitViewError);
});
test('verifies consistency between two successive STHs', async () => {
const { kp, mgr } = await setup();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth1 = await mgr.publishSTH();
await mgr.recordRegister('bob', computeBundleHash(fakeBundle(0x20)));
const sth2 = await mgr.publishSTH();
const consistency = await mgr.buildConsistencyProof(sth1.treeSize);
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return sthToWire(sth2, toBase64);
},
async fetchConsistencyProof() {
return { proof: consistency.proof.map(toBase64) };
},
},
});
await witness.observe(sth1);
await witness.observe(sth2);
expect(witness.compare(sth2)).toBe('agree');
});
test('rejects STH where log re-wrote history (consistency proof fails)', async () => {
const { kp, mgr } = await setup();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth1 = await mgr.publishSTH();
// Build a forked log where leaf 0 is different.
const forkStore = new MemoryKTLogStore();
const forkMgr = await KTLogManager.create({
crypto,
store: forkStore,
signingPrivateKey: kp.privateKey,
signingPublicKey: kp.publicKey,
});
await forkMgr.recordRegister('mallory', computeBundleHash(fakeBundle(0xee)));
await forkMgr.recordRegister('bob', computeBundleHash(fakeBundle(0x20)));
const forkedSth2 = await forkMgr.publishSTH();
const forkedConsistency = await forkMgr.buildConsistencyProof(sth1.treeSize);
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return sthToWire(forkedSth2, toBase64);
},
async fetchConsistencyProof() {
return { proof: forkedConsistency.proof.map(toBase64) };
},
},
});
await witness.observe(sth1);
await expect(witness.observe(forkedSth2)).rejects.toBeInstanceOf(KTVerificationError);
});
test('compare returns "unknown" for tree_size we have not seen', async () => {
const { kp, mgr } = await setup();
await mgr.recordRegister('alice', computeBundleHash(fakeBundle(0x10)));
const sth = await mgr.publishSTH();
const witness = new LightWitness({
crypto,
logPublicKey: kp.publicKey,
fetcher: {
async fetchLatestSTH() {
return sthToWire(sth, toBase64);
},
async fetchConsistencyProof() {
return { proof: [] };
},
},
});
expect(witness.compare(sth)).toBe('unknown');
});
});
// Make TS happy about unused fromBase64
void fromBase64;

View File

@@ -0,0 +1,8 @@
{
"name": "@shade/keychain",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"dependencies": {}
}

View File

@@ -0,0 +1,151 @@
/**
* @shade/keychain — OS-keychain backend for @shade/storage-encrypted.
*
* No native deps: shells out to the platform's standard credential CLI:
* - macOS: `security` (CLI shipped with the OS)
* - Linux: `secret-tool` (libsecret CLI; available on most distros)
* - Windows: PowerShell + CredentialManager module (built into Windows 10+)
*
* Values are stored as base64 strings on platforms whose native APIs only
* accept strings (macOS, libsecret) and as raw bytes where supported.
*/
import { spawn } from 'node:child_process';
import { Buffer } from 'node:buffer';
export interface KeychainBackend {
get(service: string, account: string): Promise<Uint8Array | null>;
set(service: string, account: string, value: Uint8Array): Promise<void>;
delete(service: string, account: string): Promise<void>;
}
/** Pick a backend appropriate for the current platform; throw if unsupported. */
export function getDefaultKeychain(): KeychainBackend {
switch (process.platform) {
case 'darwin': return new MacOSKeychain();
case 'linux': return new LibSecretKeychain();
case 'win32': return new WindowsCredentialManager();
default:
throw new Error(`@shade/keychain: unsupported platform ${process.platform}`);
}
}
// ─── macOS ──────────────────────────────────────────────────
export class MacOSKeychain implements KeychainBackend {
async get(service: string, account: string): Promise<Uint8Array | null> {
const { code, stdout } = await runCmd('security', [
'find-generic-password', '-s', service, '-a', account, '-w',
]);
if (code !== 0) return null;
const b64 = stdout.trim();
if (!b64) return null;
return Buffer.from(b64, 'base64');
}
async set(service: string, account: string, value: Uint8Array): Promise<void> {
const b64 = Buffer.from(value).toString('base64');
// -U: update if exists. Send password via stdin to avoid leaking via argv.
const { code, stderr } = await runCmd(
'security',
['add-generic-password', '-U', '-s', service, '-a', account, '-w', b64],
);
if (code !== 0) throw new Error(`security add-generic-password failed: ${stderr}`);
}
async delete(service: string, account: string): Promise<void> {
await runCmd('security', ['delete-generic-password', '-s', service, '-a', account]);
}
}
// ─── Linux (libsecret via secret-tool) ──────────────────────
export class LibSecretKeychain implements KeychainBackend {
async get(service: string, account: string): Promise<Uint8Array | null> {
const { code, stdout } = await runCmd('secret-tool', ['lookup', 'service', service, 'account', account]);
if (code !== 0) return null;
const b64 = stdout.trim();
if (!b64) return null;
return Buffer.from(b64, 'base64');
}
async set(service: string, account: string, value: Uint8Array): Promise<void> {
const b64 = Buffer.from(value).toString('base64');
const { code, stderr } = await runCmd(
'secret-tool',
['store', '--label', `shade:${service}:${account}`, 'service', service, 'account', account],
b64,
);
if (code !== 0) throw new Error(`secret-tool store failed: ${stderr}`);
}
async delete(service: string, account: string): Promise<void> {
await runCmd('secret-tool', ['clear', 'service', service, 'account', account]);
}
}
// ─── Windows Credential Manager ────────────────────────────
export class WindowsCredentialManager implements KeychainBackend {
private target(service: string, account: string): string {
return `shade:${service}:${account}`;
}
async get(service: string, account: string): Promise<Uint8Array | null> {
const target = this.target(service, account);
const ps = `
$ErrorActionPreference = 'Stop'
Import-Module CredentialManager -ErrorAction SilentlyContinue
$c = Get-StoredCredential -Target '${target}'
if ($null -eq $c) { exit 1 }
$bstr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($c.Password)
try { [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($bstr) }
finally { [System.Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) }
`.trim();
const { code, stdout } = await runCmd('powershell', ['-NoProfile', '-Command', ps]);
if (code !== 0) return null;
const b64 = stdout.trim();
return b64 ? Buffer.from(b64, 'base64') : null;
}
async set(service: string, account: string, value: Uint8Array): Promise<void> {
const target = this.target(service, account);
const b64 = Buffer.from(value).toString('base64');
const ps = `
$ErrorActionPreference = 'Stop'
Import-Module CredentialManager
$sec = ConvertTo-SecureString -String '${b64}' -AsPlainText -Force
New-StoredCredential -Target '${target}' -UserName '${account}' -SecurePassword $sec -Persist LocalMachine | Out-Null
`.trim();
const { code, stderr } = await runCmd('powershell', ['-NoProfile', '-Command', ps]);
if (code !== 0) throw new Error(`Windows CredentialManager store failed: ${stderr}`);
}
async delete(service: string, account: string): Promise<void> {
const target = this.target(service, account);
const ps = `Remove-StoredCredential -Target '${target}' -ErrorAction SilentlyContinue`;
await runCmd('powershell', ['-NoProfile', '-Command', ps]);
}
}
// ─── Helpers ────────────────────────────────────────────────
interface CmdResult { code: number; stdout: string; stderr: string }
function runCmd(cmd: string, args: string[], stdin?: string): Promise<CmdResult> {
return new Promise((resolve) => {
const child = spawn(cmd, args, { stdio: ['pipe', 'pipe', 'pipe'] });
let out = '';
let err = '';
child.stdout.on('data', (d: Buffer) => { out += d.toString('utf8'); });
child.stderr.on('data', (d: Buffer) => { err += d.toString('utf8'); });
child.on('error', () => resolve({ code: 127, stdout: '', stderr: 'spawn failed' }));
child.on('close', (code: number | null) => resolve({ code: code ?? 0, stdout: out, stderr: err }));
if (stdin !== undefined) {
child.stdin.write(stdin);
child.stdin.end();
} else {
child.stdin.end();
}
});
}

View File

@@ -0,0 +1,4 @@
{
"extends": "../../tsconfig.json",
"include": ["src/**/*", "tests/**/*"]
}

View File

@@ -0,0 +1,23 @@
{
"name": "@shade/observability",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"dependencies": {
"@noble/hashes": "^2.0.1"
},
"peerDependencies": {
"@opentelemetry/api": ">=1.7.0"
},
"peerDependenciesMeta": {
"@opentelemetry/api": {
"optional": true
}
},
"devDependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/server": "workspace:*"
}
}

View File

@@ -0,0 +1,122 @@
/**
* Safe-attribute helpers — these are the ONLY attribute keys/values that
* Shade internals are allowed to put on spans. The PII-policy guarantees:
*
* - No plaintext peer addresses (use `peerHash`).
* - No plaintext payloads.
* - No exact byte counts for stream content (use `bytesBin`).
* - Counters/codes are fine.
*
* Custom-op authors who want to add their own attributes MUST go through
* `safeAttribute()`, which rejects keys/values that look like PII.
*/
import { sha256 } from '@noble/hashes/sha2.js';
// ─── Standard attribute keys ─────────────────────────────────
export const ATTR_PEER_HASH = 'shade.peer.hash';
export const ATTR_BYTES_BIN = 'shade.bytes.bin';
export const ATTR_LANE_COUNT = 'shade.lane.count';
export const ATTR_LANE_ID = 'shade.lane.id';
export const ATTR_RETRY_COUNT = 'shade.retry.count';
export const ATTR_ERROR_CODE = 'shade.error.code';
export const ATTR_OP = 'shade.op';
export const ATTR_ROUTE = 'shade.route';
export const ATTR_HTTP_STATUS = 'shade.http.status';
export const ATTR_DIRECTION = 'shade.direction';
export const ATTR_PARTITION = 'shade.partition';
export const ATTR_RESULT = 'shade.result';
/**
* Forbidden substrings — if these appear in attribute keys we refuse.
* Mirrors the intent of the PII policy doc: never log addresses or
* exact-byte sizes.
*/
const FORBIDDEN_KEY_FRAGMENTS = [
'peer.address',
'peer_address',
'plaintext',
'payload',
'bytes.exact',
'bytes_exact',
];
/** 8-byte stable pseudonym derived from a peer address. */
export function peerHash(address: string): string {
const enc = new TextEncoder().encode(address);
const digest = sha256(enc);
let hex = '';
for (let i = 0; i < 4; i++) hex += digest[i]!.toString(16).padStart(2, '0');
return hex;
}
/** Bin a byte count into a coarse PII-safe bucket. */
export function bytesBin(n: number): string {
if (!Number.isFinite(n) || n < 0) return 'unknown';
if (n <= 4 * 1024) return '≤4KB';
if (n <= 64 * 1024) return '464KB';
if (n <= 1024 * 1024) return '64KB1MB';
if (n <= 10 * 1024 * 1024) return '110MB';
if (n <= 100 * 1024 * 1024) return '10100MB';
if (n <= 1024 * 1024 * 1024) return '100MB1GB';
return '≥1GB';
}
/** Bin a lane count into a stable bucket. */
export function laneCountBin(n: number): number {
if (n <= 1) return 1;
if (n <= 4) return 4;
if (n <= 16) return 16;
if (n <= 64) return 64;
return 64;
}
export class UnsafeAttributeError extends Error {
override readonly name = 'UnsafeAttributeError';
constructor(reason: string) {
super(reason);
}
}
/**
* Validate a user-supplied custom attribute. Returns the key/value pair
* untouched on success, or throws `UnsafeAttributeError`. Use this in
* any code path that accepts attributes from outside Shade's own
* helpers (e.g. plugin-supplied tags).
*/
export function safeAttribute(
key: string,
value: string | number | boolean,
): { key: string; value: string | number | boolean } {
const lower = key.toLowerCase();
for (const frag of FORBIDDEN_KEY_FRAGMENTS) {
if (lower.includes(frag)) {
throw new UnsafeAttributeError(`attribute key "${key}" is PII-unsafe (contains "${frag}")`);
}
}
if (typeof value === 'string') {
if (value.length > 256) {
throw new UnsafeAttributeError(
`attribute "${key}" value too long (${value.length}B); cap at 256B to avoid embedded PII`,
);
}
if (looksLikeAddress(value)) {
throw new UnsafeAttributeError(
`attribute "${key}" value looks like a peer address — use peerHash() first`,
);
}
}
return { key, value };
}
function looksLikeAddress(s: string): boolean {
// Heuristic: emails, "device:UUID", and DID-style identifiers all look
// like PII to the grep tester. Hashes (8 hex chars) and small ints are
// fine.
if (/^[a-f0-9]{1,16}$/i.test(s)) return false;
if (/@/.test(s)) return true;
if (/^device:/i.test(s)) return true;
if (/^did:/i.test(s)) return true;
return false;
}

View File

@@ -0,0 +1,36 @@
export type {
Attributes,
AttrValue,
ObservabilityHook,
Span,
} from './types.js';
export { NOOP_HOOK, noopSpan } from './types.js';
export {
ATTR_BYTES_BIN,
ATTR_DIRECTION,
ATTR_ERROR_CODE,
ATTR_HTTP_STATUS,
ATTR_LANE_COUNT,
ATTR_LANE_ID,
ATTR_OP,
ATTR_PARTITION,
ATTR_PEER_HASH,
ATTR_RESULT,
ATTR_RETRY_COUNT,
ATTR_ROUTE,
bytesBin,
laneCountBin,
peerHash,
safeAttribute,
UnsafeAttributeError,
} from './attributes.js';
export {
withTracer,
type OtelSpanLike,
type OtelTracerLike,
type WithTracerOptions,
} from './with-tracer.js';
export { createRecorder, type RecordedSpan, type SpanRecorder } from './recorder.js';

View File

@@ -0,0 +1,99 @@
/**
* In-memory `ObservabilityHook` for tests. Records every span name +
* attribute mutation so the PII grep test can scrub the recording for
* forbidden values.
*/
import type { Attributes, AttrValue, ObservabilityHook, Span } from './types.js';
export interface RecordedSpan {
name: string;
attributes: Attributes;
status: 'ok' | 'error' | 'unset';
statusMessage?: string;
exceptions: unknown[];
ended: boolean;
startedAtMs: number;
endedAtMs?: number;
}
export interface SpanRecorder extends ObservabilityHook {
/** All spans started so far (in order). */
readonly spans: readonly RecordedSpan[];
/** Drop all recorded spans — convenient between test cases. */
clear(): void;
/**
* Search recorded attributes/values for any of `forbiddenSubstrings`.
* Returns the offending hits. Use in PII guard tests.
*/
scanForPII(forbiddenSubstrings: readonly string[]): Array<{
spanName: string;
key: string;
value: AttrValue;
match: string;
}>;
}
export function createRecorder(): SpanRecorder {
const spans: RecordedSpan[] = [];
const hook: ObservabilityHook = {
startSpan(name, attrs) {
const rec: RecordedSpan = {
name,
attributes: attrs !== undefined ? { ...attrs } : {},
status: 'unset',
exceptions: [],
ended: false,
startedAtMs: Date.now(),
};
spans.push(rec);
const span: Span = {
setAttribute(key, value) {
if (rec.ended) return;
rec.attributes[key] = value;
},
setAttributes(more) {
if (rec.ended) return;
for (const [k, v] of Object.entries(more)) rec.attributes[k] = v;
},
recordException(err) {
if (rec.ended) return;
rec.exceptions.push(err);
},
setStatus(status, message) {
if (rec.ended) return;
rec.status = status;
if (message !== undefined) rec.statusMessage = message;
},
end() {
if (rec.ended) return;
rec.ended = true;
rec.endedAtMs = Date.now();
},
};
return span;
},
};
return Object.assign(hook, {
get spans() {
return spans as readonly RecordedSpan[];
},
clear() {
spans.length = 0;
},
scanForPII(forbidden: readonly string[]) {
const hits: Array<{ spanName: string; key: string; value: AttrValue; match: string }> = [];
for (const span of spans) {
for (const [key, value] of Object.entries(span.attributes)) {
const haystack = `${key}=${String(value)}`.toLowerCase();
for (const f of forbidden) {
if (haystack.includes(f.toLowerCase())) {
hits.push({ spanName: span.name, key, value, match: f });
}
}
}
}
return hits;
},
});
}

View File

@@ -0,0 +1,46 @@
/**
* Vendor-neutral observability hook surface.
*
* Shade's internals never depend on `@opentelemetry/api` directly — they
* consume an `ObservabilityHook` through which spans are started/ended.
* `withTracer()` adapts an OTel tracer to this hook; tests use
* `createRecorder()` for in-memory inspection.
*/
export type AttrValue = string | number | boolean;
export type Attributes = Record<string, AttrValue>;
export interface Span {
/** Set/overwrite a single attribute on this span. */
setAttribute(key: string, value: AttrValue): void;
/** Bulk-set attributes. */
setAttributes(attrs: Attributes): void;
/** Mark the span as failed with the given error. */
recordException(err: unknown): void;
/** Set the span status. */
setStatus(status: 'ok' | 'error', message?: string): void;
/** Close the span. Idempotent. */
end(): void;
}
export interface ObservabilityHook {
/** Start a new span. The returned object MUST always have `.end()` called. */
startSpan(name: string, attrs?: Attributes): Span;
}
const NOOP_SPAN: Span = {
setAttribute: () => undefined,
setAttributes: () => undefined,
recordException: () => undefined,
setStatus: () => undefined,
end: () => undefined,
};
export const NOOP_HOOK: ObservabilityHook = {
startSpan: () => NOOP_SPAN,
};
/** Returned by `Span` factories that get sampled-out. */
export function noopSpan(): Span {
return NOOP_SPAN;
}

View File

@@ -0,0 +1,140 @@
/**
* `withTracer()` — adapt an `@opentelemetry/api` Tracer (or any
* structurally-compatible tracer) into the vendor-neutral
* `ObservabilityHook` that Shade's internals consume.
*
* No-op behavior:
* - When `tracer` is `undefined`.
* - When `process.env.SHADE_OTEL_ENABLED` is not set to a truthy value
* ("1"/"true") AND `opts.force` isn't passed.
* - When `opts.sample` is provided and the per-span dice roll fails.
*
* The OTel structural surface we depend on is intentionally tiny — only
* `tracer.startSpan(name)` and the resulting span's
* `setAttribute/setStatus/recordException/end` methods. This keeps us
* compatible with any OTel-flavoured tracer (Jaeger, Honeycomb, Tempo,
* Sentry's OTel adapter, etc.) without locking into one vendor.
*/
import {
NOOP_HOOK,
noopSpan,
type Attributes,
type AttrValue,
type ObservabilityHook,
type Span,
} from './types.js';
/** The structural subset of an OTel `Span` that we use. */
export interface OtelSpanLike {
setAttribute(key: string, value: AttrValue): unknown;
setAttributes?(attrs: Attributes): unknown;
recordException?(err: unknown): unknown;
setStatus?(status: { code: number; message?: string }): unknown;
end(endTime?: number): unknown;
}
/** The structural subset of an OTel `Tracer` that we use. */
export interface OtelTracerLike {
startSpan(name: string, options?: { attributes?: Attributes }): OtelSpanLike;
}
export interface WithTracerOptions {
/**
* Per-span sample rate in `[0, 1]`. Default `1` (sample everything when
* the hook is active). Sampled-out spans return a no-op span.
*/
sample?: number;
/**
* Bypass the `SHADE_OTEL_ENABLED` env-var gate. Useful for tests and
* for embeddings where the hosting application makes the on/off
* decision itself.
*/
force?: boolean;
/** Override the env-var name. Defaults to `SHADE_OTEL_ENABLED`. */
envVar?: string;
/** Override the random source (used in tests for determinism). */
random?: () => number;
}
/**
* Per the OTel spec: `SpanStatusCode.OK = 1`, `ERROR = 2`. We hardcode
* the integers so we don't pull in `@opentelemetry/api` at runtime.
*/
const OTEL_STATUS_OK = 1;
const OTEL_STATUS_ERROR = 2;
export function withTracer(
tracer: OtelTracerLike | null | undefined,
opts: WithTracerOptions = {},
): ObservabilityHook {
if (tracer === null || tracer === undefined) return NOOP_HOOK;
const envVar = opts.envVar ?? 'SHADE_OTEL_ENABLED';
if (!opts.force && !envEnabled(envVar)) return NOOP_HOOK;
const sample = opts.sample ?? 1;
if (sample <= 0) return NOOP_HOOK;
const random = opts.random ?? Math.random;
return {
startSpan(name: string, attrs?: Attributes): Span {
if (sample < 1 && random() >= sample) return noopSpan();
const otelSpan = attrs !== undefined
? tracer.startSpan(name, { attributes: attrs })
: tracer.startSpan(name);
return adaptSpan(otelSpan);
},
};
}
function adaptSpan(otel: OtelSpanLike): Span {
let ended = false;
return {
setAttribute(key, value) {
if (ended) return;
otel.setAttribute(key, value);
},
setAttributes(attrs) {
if (ended) return;
if (typeof otel.setAttributes === 'function') {
otel.setAttributes(attrs);
} else {
for (const [k, v] of Object.entries(attrs)) otel.setAttribute(k, v);
}
},
recordException(err) {
if (ended) return;
if (typeof otel.recordException === 'function') {
otel.recordException(err);
} else {
otel.setAttribute(
'exception.message',
err instanceof Error ? err.message : String(err),
);
}
},
setStatus(status, message) {
if (ended) return;
if (typeof otel.setStatus === 'function') {
const code = status === 'ok' ? OTEL_STATUS_OK : OTEL_STATUS_ERROR;
otel.setStatus(message !== undefined ? { code, message } : { code });
}
},
end() {
if (ended) return;
ended = true;
otel.end();
},
};
}
function envEnabled(name: string): boolean {
const proc =
typeof globalThis !== 'undefined'
? (globalThis as unknown as { process?: { env?: Record<string, string | undefined> } }).process
: undefined;
const v = proc?.env?.[name];
if (v === undefined || v === '') return false;
return v === '1' || v.toLowerCase() === 'true';
}

View File

@@ -0,0 +1,96 @@
import { describe, expect, test } from 'bun:test';
import {
bytesBin,
laneCountBin,
peerHash,
safeAttribute,
UnsafeAttributeError,
} from '../src/index.ts';
describe('peerHash', () => {
test('produces stable 8-char hex', () => {
const a = peerHash('alice@example.com');
const b = peerHash('alice@example.com');
expect(a).toBe(b);
expect(a).toMatch(/^[0-9a-f]{8}$/);
});
test('different addresses produce different hashes', () => {
expect(peerHash('alice@example.com')).not.toBe(peerHash('bob@example.com'));
});
test('never echoes the address', () => {
const addr = 'alice@example.com';
expect(peerHash(addr).includes('alice')).toBe(false);
expect(peerHash(addr).includes('@')).toBe(false);
});
});
describe('bytesBin', () => {
test('bins by order of magnitude', () => {
expect(bytesBin(0)).toBe('≤4KB');
expect(bytesBin(4096)).toBe('≤4KB');
expect(bytesBin(4097)).toBe('464KB');
expect(bytesBin(64 * 1024)).toBe('464KB');
expect(bytesBin(64 * 1024 + 1)).toBe('64KB1MB');
expect(bytesBin(1024 * 1024)).toBe('64KB1MB');
expect(bytesBin(10 * 1024 * 1024)).toBe('110MB');
expect(bytesBin(100 * 1024 * 1024)).toBe('10100MB');
expect(bytesBin(1024 * 1024 * 1024)).toBe('100MB1GB');
expect(bytesBin(2 * 1024 * 1024 * 1024)).toBe('≥1GB');
});
test('handles invalid input', () => {
expect(bytesBin(-1)).toBe('unknown');
expect(bytesBin(NaN)).toBe('unknown');
expect(bytesBin(Infinity)).toBe('unknown');
});
});
describe('laneCountBin', () => {
test('snaps to {1, 4, 16, 64}', () => {
expect(laneCountBin(1)).toBe(1);
expect(laneCountBin(2)).toBe(4);
expect(laneCountBin(4)).toBe(4);
expect(laneCountBin(5)).toBe(16);
expect(laneCountBin(16)).toBe(16);
expect(laneCountBin(32)).toBe(64);
expect(laneCountBin(128)).toBe(64);
});
});
describe('safeAttribute', () => {
test('rejects PII-flavoured keys', () => {
expect(() => safeAttribute('shade.peer.address', 'x')).toThrow(UnsafeAttributeError);
expect(() => safeAttribute('shade.bytes.exact', 1)).toThrow(UnsafeAttributeError);
expect(() => safeAttribute('shade.plaintext', 'x')).toThrow(UnsafeAttributeError);
});
test('rejects address-like values', () => {
expect(() => safeAttribute('custom.tag', 'alice@example.com')).toThrow(UnsafeAttributeError);
expect(() => safeAttribute('custom.tag', 'device:abc-123')).toThrow(UnsafeAttributeError);
expect(() => safeAttribute('custom.tag', 'did:web:example.com')).toThrow(UnsafeAttributeError);
});
test('rejects oversized strings', () => {
expect(() => safeAttribute('ok', 'x'.repeat(257))).toThrow(UnsafeAttributeError);
});
test('accepts safe values', () => {
expect(safeAttribute('shade.bytes.bin', '464KB')).toEqual({
key: 'shade.bytes.bin',
value: '464KB',
});
expect(safeAttribute('shade.lane.count', 4)).toEqual({ key: 'shade.lane.count', value: 4 });
expect(safeAttribute('shade.retry.count', 0)).toEqual({ key: 'shade.retry.count', value: 0 });
expect(safeAttribute('shade.error.code', 'SHADE_TIMEOUT')).toEqual({
key: 'shade.error.code',
value: 'SHADE_TIMEOUT',
});
// hashes pass through
expect(safeAttribute('shade.peer.hash', 'abcdef01')).toEqual({
key: 'shade.peer.hash',
value: 'abcdef01',
});
});
});

View File

@@ -0,0 +1,104 @@
/**
* End-to-end PII guard test.
*
* Exercises real Shade entry points (session encrypt/decrypt, transfer
* upload + receive, prekey HTTP routes, files RPC) with a recorder hook
* and asserts that NONE of the recorded span attributes echo the
* sensitive plaintext we deliberately fed in (peer address, message
* content, exact byte counts).
*/
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
import { ShadeSessionManager, type StorageProvider } from '@shade/core';
import { MemoryStorage, SubtleCryptoProvider } from '@shade/crypto-web';
import { createRecorder } from '../src/index.ts';
import { createPrekeyRoutes, MemoryPrekeyStore } from '@shade/server';
const DANGER_FRAGMENTS = [
// Peer addresses we feed into APIs:
'alice@danger.test',
'bob@danger.test',
'device:hot-secret-12345',
// Plaintext message bodies:
'sekret-payload-XYZ',
'CLASSIFIED-7777',
// Exact byte counts that we'd never want leaked:
'1048577',
];
describe('observability — PII guard for ShadeSessionManager', () => {
test('encrypt/decrypt spans never echo address or plaintext', async () => {
const rec = createRecorder();
const crypto = new SubtleCryptoProvider();
const aliceStorage: StorageProvider = new MemoryStorage();
const bobStorage: StorageProvider = new MemoryStorage();
const alice = new ShadeSessionManager(crypto, aliceStorage, { observability: rec });
const bob = new ShadeSessionManager(crypto, bobStorage, { observability: rec });
await alice.initialize();
await bob.initialize();
// Alice -> Bob handshake (X3DH)
const bobBundle = await bob.createPreKeyBundle();
await alice.initSessionFromBundle('bob@danger.test', bobBundle);
const env1 = await alice.encrypt('bob@danger.test', 'sekret-payload-XYZ');
await bob.decrypt('alice@danger.test', env1);
// Round-trip a second time so a steady-state ratchet step also runs.
const env2 = await bob.encrypt('alice@danger.test', 'CLASSIFIED-7777');
await alice.decrypt('bob@danger.test', env2);
expect(rec.spans.length).toBeGreaterThan(0);
const hits = rec.scanForPII(DANGER_FRAGMENTS);
if (hits.length > 0) {
throw new Error(`PII leak in spans: ${JSON.stringify(hits, null, 2)}`);
}
});
});
describe('observability — PII guard for prekey routes', () => {
let port: number;
let server: ReturnType<typeof Bun.serve>;
let rec: ReturnType<typeof createRecorder>;
beforeAll(async () => {
rec = createRecorder();
const crypto = new SubtleCryptoProvider();
const store = new MemoryPrekeyStore();
const app = createPrekeyRoutes(store, crypto, {
observability: rec,
disableRateLimit: true,
});
server = Bun.serve({
fetch: app.fetch,
port: 0,
});
port = (server as unknown as { port: number }).port;
});
afterAll(async () => {
await server.stop();
});
test('GET /v1/keys/bundle/<address> never logs the address verbatim', async () => {
const addr = 'device:hot-secret-12345';
// Anonymous fetch — bundle endpoint will 404 since we never registered,
// but the route still emits a span with the route template (not the
// raw address path).
await fetch(`http://localhost:${port}/v1/keys/bundle/${encodeURIComponent(addr)}`);
expect(rec.spans.length).toBeGreaterThan(0);
const hits = rec.scanForPII([addr, 'hot-secret']);
if (hits.length > 0) {
throw new Error(`PII leak in prekey-route spans: ${JSON.stringify(hits, null, 2)}`);
}
// The span name should reference the route TEMPLATE, not the raw path.
const seenRoutes = rec.spans.flatMap((s) => {
const r = s.attributes['shade.route'];
return typeof r === 'string' ? [r] : [];
});
// Route should be `/v1/keys/bundle/:address` (or empty if Hono didn't
// resolve it; what we MUST NOT see is the literal device:... value).
for (const r of seenRoutes) {
expect(r.includes('hot-secret')).toBe(false);
}
});
});

View File

@@ -0,0 +1,52 @@
import { describe, expect, test } from 'bun:test';
import { createRecorder } from '../src/index.ts';
describe('createRecorder', () => {
test('captures attributes and end-state', () => {
const rec = createRecorder();
const span = rec.startSpan('shade.test', { initial: 'on' });
span.setAttribute('extra', 1);
span.setAttributes({ batch1: 'a', batch2: 'b' });
span.setStatus('ok');
span.end();
expect(rec.spans).toHaveLength(1);
const s = rec.spans[0]!;
expect(s.name).toBe('shade.test');
expect(s.attributes).toEqual({ initial: 'on', extra: 1, batch1: 'a', batch2: 'b' });
expect(s.status).toBe('ok');
expect(s.ended).toBe(true);
});
test('records exceptions', () => {
const rec = createRecorder();
const span = rec.startSpan('shade.test');
const err = new Error('boom');
span.recordException(err);
span.setStatus('error', 'boom');
span.end();
expect(rec.spans[0]?.exceptions).toEqual([err]);
expect(rec.spans[0]?.status).toBe('error');
expect(rec.spans[0]?.statusMessage).toBe('boom');
});
test('scanForPII catches forbidden substrings', () => {
const rec = createRecorder();
const safe = rec.startSpan('shade.upload', { 'shade.peer.hash': 'abc12345' });
safe.end();
const leaky = rec.startSpan('shade.upload', { 'shade.peer.address': 'alice@example.com' });
leaky.end();
const hits = rec.scanForPII(['@', 'alice', 'peer.address']);
expect(hits.length).toBeGreaterThan(0);
// The safe span should not be in the hits.
const safeHit = hits.find((h) => h.spanName === 'shade.upload' && h.value === 'abc12345');
expect(safeHit).toBeUndefined();
});
test('clear() drops the buffer', () => {
const rec = createRecorder();
rec.startSpan('a').end();
expect(rec.spans).toHaveLength(1);
rec.clear();
expect(rec.spans).toHaveLength(0);
});
});

View File

@@ -0,0 +1,127 @@
import { describe, expect, test, beforeEach, afterEach } from 'bun:test';
import { NOOP_HOOK, withTracer, type OtelTracerLike } from '../src/index.ts';
interface RecordedSpan {
name: string;
attrs: Record<string, unknown>;
ended: boolean;
}
function makeFakeTracer(): { tracer: OtelTracerLike; spans: RecordedSpan[] } {
const spans: RecordedSpan[] = [];
const tracer: OtelTracerLike = {
startSpan(name, options) {
const rec: RecordedSpan = {
name,
attrs: { ...(options?.attributes ?? {}) },
ended: false,
};
spans.push(rec);
return {
setAttribute(k, v) {
rec.attrs[k] = v;
return undefined;
},
end() {
rec.ended = true;
return undefined;
},
};
},
};
return { tracer, spans };
}
describe('withTracer (off-by-default)', () => {
beforeEach(() => {
delete (globalThis as unknown as { process?: { env?: Record<string, string | undefined> } }).process?.env?.SHADE_OTEL_ENABLED;
});
test('returns NOOP_HOOK when tracer is undefined', () => {
const hook = withTracer(undefined);
expect(hook).toBe(NOOP_HOOK);
});
test('returns NOOP_HOOK when env-var is not set (default)', () => {
const { tracer, spans } = makeFakeTracer();
const hook = withTracer(tracer);
const span = hook.startSpan('shade.test');
span.setAttribute('foo', 'bar');
span.end();
expect(spans.length).toBe(0); // never reached the OTel tracer
});
test('force=true bypasses the env gate', () => {
const { tracer, spans } = makeFakeTracer();
const hook = withTracer(tracer, { force: true });
hook.startSpan('shade.test', { foo: 'bar' }).end();
expect(spans.length).toBe(1);
expect(spans[0]?.name).toBe('shade.test');
expect(spans[0]?.attrs.foo).toBe('bar');
expect(spans[0]?.ended).toBe(true);
});
});
describe('withTracer (env-enabled)', () => {
beforeEach(() => {
process.env.SHADE_OTEL_ENABLED = '1';
});
afterEach(() => {
delete process.env.SHADE_OTEL_ENABLED;
});
test('emits spans through the tracer when env is set', () => {
const { tracer, spans } = makeFakeTracer();
const hook = withTracer(tracer);
const span = hook.startSpan('shade.upload', { 'shade.bytes.bin': '110MB' });
span.setAttribute('shade.result', 'ok');
span.setStatus('ok');
span.end();
expect(spans.length).toBe(1);
expect(spans[0]?.name).toBe('shade.upload');
expect(spans[0]?.attrs['shade.bytes.bin']).toBe('110MB');
expect(spans[0]?.attrs['shade.result']).toBe('ok');
expect(spans[0]?.ended).toBe(true);
});
test('respects per-span sampling', () => {
const { tracer, spans } = makeFakeTracer();
let n = 0;
const random = () => {
// Alternates: 0.1 (sampled in), 0.9 (sampled out)
const v = n % 2 === 0 ? 0.1 : 0.9;
n++;
return v;
};
const hook = withTracer(tracer, { sample: 0.5, random });
for (let i = 0; i < 10; i++) hook.startSpan(`s${i}`).end();
// Half (5 of 10) should reach the OTel tracer.
expect(spans.length).toBe(5);
});
test('sample=0 means no spans even when env is on', () => {
const { tracer, spans } = makeFakeTracer();
const hook = withTracer(tracer, { sample: 0 });
hook.startSpan('shade.test').end();
expect(spans.length).toBe(0);
});
test('end() is idempotent', () => {
const { tracer, spans } = makeFakeTracer();
const hook = withTracer(tracer);
const span = hook.startSpan('shade.test');
span.end();
span.end();
expect(spans.length).toBe(1);
expect(spans[0]?.ended).toBe(true);
});
test('attribute mutations after end() are no-op', () => {
const { tracer, spans } = makeFakeTracer();
const hook = withTracer(tracer);
const span = hook.startSpan('shade.test', { a: 1 });
span.end();
span.setAttribute('after_end', 'oops');
expect(spans[0]?.attrs.after_end).toBeUndefined();
});
});

View File

@@ -0,0 +1,8 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "dist",
"rootDir": "src"
},
"include": ["src"]
}

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/observer",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/proto",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -0,0 +1,63 @@
# `@shade/recovery`
Social key recovery for Shade — V3.10.
Shamir Secret Sharing over GF(2^8) splits the user's identity backup
key into `n` shares; any threshold-many `k` together reconstruct the
identity onto a new device. Distribution and reconstruction ride
existing 1:1 Shade sessions — no centralized recovery agent.
## Install
```bash
bun add @shade/recovery
```
## Quick wire-up
```ts
import {
setupRecovery,
attachGuardian,
requestRecovery,
MemoryRecoveryStore,
} from '@shade/recovery';
// Primary (Alice's existing device)
await setupRecovery({
shade,
guardians: ['bob', 'carol', 'dan', 'eve', 'faythe'],
threshold: 3,
deliver: async (to, envelope) => myOutbox.send(to, envelope),
});
// Each guardian
attachGuardian({
shade,
store: new MemoryRecoveryStore(), // swap for persistent store in prod
approve: async (ctx) => askUser(ctx),
deliver: async (to, envelope) => myOutbox.send(to, envelope),
});
// New device (Alice on a fresh phone)
await requestRecovery({
shade: tempShade,
originalAddress: 'alice',
setupId: '<from recovery card>',
threshold: 3,
guardians: ['bob', 'carol', 'dan', 'eve', 'faythe'],
deliver: async (to, envelope) => myOutbox.send(to, envelope),
});
```
See [`docs/recovery.md`](../../docs/recovery.md) for the full
threat model, persistence recommendations, and guardian-UX guidance.
## Tests
```bash
bun test # all
bun test tests/shamir # Shamir primitives
bun test tests/integration # 3-of-5 end-to-end
bun test tests/adversarial # k-1 collusion + forged shares + OOB-gate
```

View File

@@ -0,0 +1,22 @@
{
"name": "@shade/recovery",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"exports": {
".": {
"types": "./src/index.ts",
"import": "./src/index.ts"
}
},
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/sdk": "workspace:*"
},
"devDependencies": {
"@shade/server": "workspace:*",
"fast-check": "^3.22.0"
}
}

View File

@@ -0,0 +1,45 @@
/**
* Encoding helpers shared by the setup, request, and guardian modules.
* Kept as a tiny standalone module so individual flows don't carry
* private base64 helpers; consistent encoding across send/receive
* sides.
*/
/** Base64url (no padding) — used for both `recoveryKey → passphrase` and arbitrary share bytes. */
export function bytesToBase64Url(bytes: Uint8Array): string {
let bin = '';
for (let i = 0; i < bytes.length; i++) bin += String.fromCharCode(bytes[i]!);
return btoa(bin).replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
}
export function base64UrlToBytes(s: string): Uint8Array {
const padded = s.replace(/-/g, '+').replace(/_/g, '/');
const padding = padded.length % 4 === 0 ? 0 : 4 - (padded.length % 4);
const bin = atob(padded + '='.repeat(padding));
const out = new Uint8Array(bin.length);
for (let i = 0; i < bin.length; i++) out[i] = bin.charCodeAt(i);
return out;
}
/**
* Convert a `recoveryKey` (32 random bytes) to the passphrase that
* `Shade.exportBackup` / `Shade.importBackup` expect. We use base64url
* because:
* - it's a string, satisfying the export/import API,
* - 32 bytes encodes to 43 characters, comfortably above the 12-char
* minimum the exportBackup helper enforces,
* - the encoding is deterministic so split + reconstruct + decode
* yields the identical passphrase the original device used.
*
* The HKDF inside `exportBackup` is a deterministic KDF that's
* cryptographically appropriate for a 32-byte uniformly-random IKM
* (this is exactly the standard HKDF use case). The fact that the
* passphrase API was designed for human-typed passwords does not
* weaken the construction here.
*/
export function recoveryKeyToBackupPassphrase(key: Uint8Array): string {
if (key.length !== 32) {
throw new Error(`recoveryKey must be 32 bytes (got ${key.length})`);
}
return `shade-rk:${bytesToBase64Url(key)}`;
}

View File

@@ -0,0 +1,90 @@
/**
* Errors emitted by `@shade/recovery`. All of them subclass the same base
* so consumers can catch any recovery-related failure in one block, then
* branch on the concrete type for messaging.
*/
export class RecoveryError extends Error {
constructor(message: string, options?: ErrorOptions) {
super(message, options);
this.name = 'RecoveryError';
}
}
/**
* The new-device flow timed out while waiting for the threshold number
* of guardians to respond. Retryable — the caller can ask the user to
* nudge guardians offline and retry.
*/
export class RecoveryTimeoutError extends RecoveryError {
constructor(
public readonly received: number,
public readonly threshold: number,
) {
super(
`Recovery timed out: received ${received} guardian responses, need ${threshold}`,
);
this.name = 'RecoveryTimeoutError';
}
}
/**
* One or more guardians explicitly declined the recovery request. Listed
* in `declines` (guardian addresses). The new-device flow keeps running
* with the remaining guardians; this error fires only when too many
* decline to ever reach the threshold.
*/
export class RecoveryDeclinedError extends RecoveryError {
constructor(
public readonly declines: ReadonlyArray<string>,
public readonly threshold: number,
public readonly remaining: number,
) {
super(
`Recovery aborted: ${declines.length} guardian(s) declined and ${remaining} are left, ` +
`which is below the threshold of ${threshold}`,
);
this.name = 'RecoveryDeclinedError';
}
}
/**
* The reconstructed `recoveryKey` did not authenticate the encrypted
* `shareSecret` (AEAD tag mismatch). Most likely cause: a guardian
* supplied a forged share, OR the user supplied the wrong original
* address. Treat as adversarial — abort and notify the user, do not
* retry with the same shares.
*/
export class RecoveryReconstructionError extends RecoveryError {
constructor(message: string, options?: ErrorOptions) {
super(message, options);
this.name = 'RecoveryReconstructionError';
}
}
/**
* The guardian-side approve callback returned `false` (or threw),
* indicating the user did not match the OOB safety number. The peer
* receives a `share-decline` envelope; this error is thrown locally on
* the new device when its requestRecovery() detects too many declines.
*/
export class RecoveryGuardianRejectedError extends RecoveryError {
constructor(
public readonly guardianAddress: string,
public readonly reason: string,
) {
super(`Guardian ${guardianAddress} rejected the recovery request: ${reason}`);
this.name = 'RecoveryGuardianRejectedError';
}
}
/**
* A protocol envelope arrived that could not be parsed (malformed JSON,
* missing required field, unknown version, etc.). Always treat as a bug
* or as malicious input — never silently ignore.
*/
export class RecoveryProtocolError extends RecoveryError {
constructor(message: string, options?: ErrorOptions) {
super(message, options);
this.name = 'RecoveryProtocolError';
}
}

View File

@@ -0,0 +1,250 @@
/**
* Guardian-side receiver.
*
* `attachGuardian` wires a `Shade.onMessage` handler that handles every
* recovery envelope addressed to this device:
*
* - `share-deposit`: persist the share + backup blob in the supplied
* `RecoveryStore`. Idempotent on (originalAddress, setupId).
*
* - `recovery-request`: invoke the `approve` callback with the
* ratcheted requester address + the safety number embedded in the
* envelope. The approve callback is the user-confronted gate — it
* SHOULD pop UI showing both fingerprints and require an explicit
* OOB-confirmed click. Returning `true` ships a `share-grant`;
* returning `false` (or throwing) ships a `share-decline` with the
* reason. Recovery-requests for unknown originalAddress/setupId
* pairs are auto-declined with `"unknown setup"`.
*
* - All other recovery types are ignored (we only care about the
* guardian's inbound side; share-grant + share-decline target the
* new device, not the guardian).
*
* The handler returns nothing — it never blocks `Shade.receive`. All
* outbound replies happen via the same `deliver` callback the caller
* supplied, so the caller's transport layer is the only piece that
* touches the wire.
*
* Returned function is the detach handle — calling it removes the
* onMessage handler and frees the registry.
*/
import type { Shade } from '@shade/sdk';
import {
encodeRecoveryEnvelope,
tryParseRecoveryEnvelope,
type RecoveryEnvelope,
type ShareDeclineEnvelope,
type ShareGrantEnvelope,
} from './protocol.js';
import type { GuardianShareEntry, RecoveryStore } from './store.js';
import type { RecoveryDeliver } from './setup.js';
export interface GuardianApproveContext {
/** Address of the device requesting recovery (the new device's temporary identity). */
requesterAddress: string;
/** Address of the original (lost) device whose share is being requested. */
originalAddress: string;
/** Setup id from the request — matches a deposit in the store. */
setupId: string;
/** Safety number of the new device's TEMPORARY identity. Show this to the user. */
requesterFingerprint: string;
/** Safety number of the original device at deposit time. Show this for comparison. */
setupFingerprint: string;
/** When the deposit was originally made. */
depositCreatedAt: number;
/** When the request was received. */
requestReceivedAt: number;
}
/**
* Async predicate the caller registers to authorize a `recovery-request`.
* Return `true` to release the share, `false` (or throw) to decline.
*
* Implementations MUST be irrevocably user-driven — never auto-approve,
* because the social-engineering threat (V3.10 risk #2) is exactly that
* an attacker imitates the original user. The default
* `<RecoveryApprove />` widget enforces an explicit OOB-confirmation
* checkbox + "I have verified the safety numbers match" gate before
* resolving `true`.
*/
export type GuardianApproveHandler = (ctx: GuardianApproveContext) => Promise<boolean>;
export interface AttachGuardianOptions {
/** Initialized Shade instance whose onMessage will be subscribed. */
shade: Shade;
/** Persistent storage for received shares. */
store: RecoveryStore;
/** User-driven approval predicate. See {@link GuardianApproveHandler}. */
approve: GuardianApproveHandler;
/** Outbound transport for share-grant / share-decline replies. */
deliver: RecoveryDeliver;
/**
* Optional hook fired when the guardian persists a fresh deposit.
* Used by widget layers to reactively re-render the deposit list.
*/
onDeposit?: (entry: GuardianShareEntry) => void;
/**
* Optional logger for protocol-level anomalies. Defaults to
* `console.warn`. Pass a no-op for silent operation in tests.
*/
onProtocolError?: (err: Error, source: string) => void;
/** Wall-clock source for `receivedAt`. Defaults to `Date.now`. */
now?: () => number;
}
export interface AttachedGuardian {
/** Detach the onMessage handler. Idempotent. */
stop: () => void;
}
/**
* Wire a Shade instance to act as a guardian. Returns a detach handle
* the caller invokes on shutdown.
*/
export function attachGuardian(opts: AttachGuardianOptions): AttachedGuardian {
if (typeof opts.deliver !== 'function') {
throw new TypeError('attachGuardian: deliver must be a function');
}
if (typeof opts.approve !== 'function') {
throw new TypeError('attachGuardian: approve must be a function');
}
const onProtocolError = opts.onProtocolError ?? defaultProtocolErrorLogger;
const now = opts.now ?? Date.now;
const detach = opts.shade.onMessage(async (from, plaintext) => {
let env: RecoveryEnvelope | null;
try {
env = tryParseRecoveryEnvelope(plaintext);
} catch (err) {
onProtocolError(err as Error, `from=${from}`);
return;
}
if (env === null) return; // not a recovery message — ignore.
try {
switch (env.type) {
case 'share-deposit':
await handleDeposit(opts, env, now());
return;
case 'recovery-request':
await handleRequest(opts, from, env, now());
return;
case 'share-grant':
case 'share-decline':
// Replies belong to the new-device flow — guardians don't act on them.
return;
}
} catch (err) {
onProtocolError(err as Error, `type=${env.type} from=${from}`);
}
});
let stopped = false;
return {
stop: () => {
if (stopped) return;
stopped = true;
detach();
},
};
}
async function handleDeposit(
opts: AttachGuardianOptions,
env: import('./protocol.js').ShareDepositEnvelope,
receivedAt: number,
): Promise<void> {
const entry: GuardianShareEntry = {
originalAddress: env.originalAddress,
setupId: env.setupId,
shareIndex: env.shareIndex,
shareBytes: env.shareBytes,
// The backupBlob string contains the AES-GCM-protected payload; we
// split it into "shareSecret" fields per the V3.10 schema for
// legibility while keeping the original blob string intact.
shareSecretCiphertext: env.backupBlob,
shareSecretNonce: '',
setupFingerprint: env.setupFingerprint,
guardianCount: env.guardianCount,
threshold: env.threshold,
receivedAt,
};
await opts.store.save(entry);
opts.onDeposit?.(entry);
}
async function handleRequest(
opts: AttachGuardianOptions,
from: string,
env: import('./protocol.js').RecoveryRequestEnvelope,
receivedAt: number,
): Promise<void> {
const stored = await opts.store.get(env.originalAddress, env.setupId);
if (stored === null) {
await sendDecline(opts, from, env, 'unknown setup');
return;
}
let approved: boolean;
try {
approved = await opts.approve({
requesterAddress: from,
originalAddress: env.originalAddress,
setupId: env.setupId,
requesterFingerprint: env.requesterFingerprint,
setupFingerprint: stored.setupFingerprint,
depositCreatedAt: stored.receivedAt,
requestReceivedAt: receivedAt,
});
} catch (err) {
await sendDecline(
opts,
from,
env,
`approve handler threw: ${(err as Error).message}`,
);
return;
}
if (!approved) {
await sendDecline(opts, from, env, 'user declined');
return;
}
const grant: ShareGrantEnvelope = {
shadeRecovery: 1,
type: 'share-grant',
flowId: env.flowId,
originalAddress: env.originalAddress,
setupId: env.setupId,
shareIndex: stored.shareIndex,
shareBytes: stored.shareBytes,
backupBlob: stored.shareSecretCiphertext,
};
const plaintext = encodeRecoveryEnvelope(grant);
const envelope = await opts.shade.send(from, plaintext);
await opts.deliver(from, envelope);
}
async function sendDecline(
opts: AttachGuardianOptions,
to: string,
env: import('./protocol.js').RecoveryRequestEnvelope,
reason: string,
): Promise<void> {
const decline: ShareDeclineEnvelope = {
shadeRecovery: 1,
type: 'share-decline',
flowId: env.flowId,
originalAddress: env.originalAddress,
setupId: env.setupId,
reason,
};
const plaintext = encodeRecoveryEnvelope(decline);
const envelope = await opts.shade.send(to, plaintext);
await opts.deliver(to, envelope);
}
function defaultProtocolErrorLogger(err: Error, source: string): void {
console.warn(`[shade-recovery] guardian dropped malformed envelope (${source}): ${err.message}`);
}

View File

@@ -0,0 +1,77 @@
/**
* `@shade/recovery` — social key recovery for Shade (V3.10).
*
* Public surface:
* - {@link setupRecovery}: distribute Shamir shares to guardians.
* - {@link attachGuardian}: wire a guardian-side receiver.
* - {@link requestRecovery}: rebuild a lost identity from threshold guardians.
* - {@link splitSecret} / {@link combineShares}: low-level Shamir primitives
* (exported for advanced callers and test harnesses).
* - Errors, store interface, and protocol envelope types.
*/
// Core flows
export { setupRecovery } from './setup.js';
export type {
SetupRecoveryOptions,
SetupRecoveryResult,
GuardianDelivery,
RecoveryDeliver,
} from './setup.js';
export { attachGuardian } from './guardian.js';
export type {
AttachGuardianOptions,
AttachedGuardian,
GuardianApproveContext,
GuardianApproveHandler,
} from './guardian.js';
export { requestRecovery } from './request.js';
export type {
RequestRecoveryOptions,
RecoveryProgress,
RecoveryResult,
} from './request.js';
// Storage
export { MemoryRecoveryStore } from './store.js';
export type { GuardianShareEntry, RecoveryStore } from './store.js';
// Errors
export {
RecoveryError,
RecoveryDeclinedError,
RecoveryGuardianRejectedError,
RecoveryProtocolError,
RecoveryReconstructionError,
RecoveryTimeoutError,
} from './errors.js';
// Protocol — exported for apps that need to inspect or relay envelopes.
export {
encodeRecoveryEnvelope,
tryParseRecoveryEnvelope,
RECOVERY_DISCRIMINATOR,
RECOVERY_PROTOCOL_VERSION,
} from './protocol.js';
export type {
RecoveryEnvelope,
RecoveryMessageType,
RecoveryRequestEnvelope,
ShareDeclineEnvelope,
ShareDepositEnvelope,
ShareGrantEnvelope,
} from './protocol.js';
// Shamir primitives — exported for tests and advanced callers (e.g.
// hardware-token integrations that want to split a different secret).
export { splitSecret, combineShares, encodeShare, decodeShare } from './shamir.js';
export type { ShamirShare } from './shamir.js';
// Encoding helpers — used by widget-layer code and integration tests.
export {
bytesToBase64Url,
base64UrlToBytes,
recoveryKeyToBackupPassphrase,
} from './encoding.js';

View File

@@ -0,0 +1,245 @@
/**
* Wire protocol for `@shade/recovery`.
*
* Every payload is a JSON object that fits inside a single
* `Shade.send(plaintext)` call (we travel over the existing 1:1
* Double-Ratchet sessions — no new transport). All envelopes carry a
* `shadeRecovery` discriminator so a guardian-side `Shade.onMessage`
* handler can cheaply skip non-recovery traffic without misparsing it.
*
* Envelope versions:
* - v1: initial release (V3.10).
*
* Message types:
*
* share-deposit primary → guardian (during setup)
* The guardian receives one Shamir share for the primary user's
* identity, plus the AEAD-protected `backupBlob` string (duplicated
* to every guardian so any threshold subset can reconstruct).
*
* recovery-request new-device → guardian
* Asks the guardian to release its stored share. The new device's
* temporary identity fingerprint is included so the guardian can
* OOB-confirm before approving.
*
* share-grant guardian → new-device
* The guardian's response when its approve handler returns true:
* ships the share + backupBlob back to the new device.
*
* share-decline guardian → new-device
* The guardian's response when the user (or a hard policy) refused
* to release the share. Carries a short reason for diagnostics.
*
* The encoder is deliberately a thin JSON serializer with stable keys —
* we do NOT need canonical hashing here because every payload is already
* authenticated by the underlying Double-Ratchet AEAD. Stability matters
* only for forward-compatible parsing (tolerate future, additive fields
* but reject malformed ones).
*
* `backupBlob` is opaque from this module's perspective — it's whatever
* `Shade.exportBackup(passphrase, addresses)` produces, where the
* `passphrase` is a base64url encoding of the random `recoveryKey` that
* was Shamir-split. The new device combines shares to recover the key,
* derives the same passphrase, and calls `Shade.importBackup`. The
* AES-GCM authentication tag inside the backup blob doubles as the
* sentinel for a successful reconstruction.
*/
import { RecoveryProtocolError } from './errors.js';
export const RECOVERY_PROTOCOL_VERSION = 1;
export const RECOVERY_DISCRIMINATOR = 'shadeRecovery';
export type RecoveryMessageType =
| 'share-deposit'
| 'recovery-request'
| 'share-grant'
| 'share-decline';
interface BaseEnvelope<T extends RecoveryMessageType> {
shadeRecovery: 1;
type: T;
/** Stable per-flow identifier; correlates request → grant/decline. */
flowId: string;
}
export interface ShareDepositEnvelope extends BaseEnvelope<'share-deposit'> {
/** Address of the primary (the user being protected). */
originalAddress: string;
/** Stable session id — primary chooses; same across all guardians in this setup. */
setupId: string;
/** Threshold this setup was created with. */
threshold: number;
/** Number of guardians in this setup. */
guardianCount: number;
/** Index of THIS guardian in 1..guardianCount. Matches the Shamir x-coordinate. */
shareIndex: number;
/** Base64-encoded `encodeShare(...)` bytes (1 byte x + N bytes y). */
shareBytes: string;
/**
* Output of `Shade.exportBackup(passphrase, knownAddresses)`. Opaque
* string of the form `shade-backup:v1:<salt>:<nonce>:<ciphertext>`.
* Stored verbatim by the guardian and shipped back on
* `share-grant`. Identical for every guardian in the same setup.
*/
backupBlob: string;
/** Original-device fingerprint at setup time — guardians can persist for sanity-checks. */
setupFingerprint: string;
/** Unix-ms timestamp at setup time. */
createdAt: number;
}
export interface RecoveryRequestEnvelope extends BaseEnvelope<'recovery-request'> {
/** Address of the original (lost) identity. The guardian uses this to look up its stored share. */
originalAddress: string;
/** Setup id for the share the new device wants. Lets a guardian disambiguate multiple deposits. */
setupId: string;
/** Safety number of the *temporary* identity making the request. */
requesterFingerprint: string;
/** Unix-ms timestamp; receiver MAY enforce a freshness window. */
requestedAt: number;
}
export interface ShareGrantEnvelope extends BaseEnvelope<'share-grant'> {
/** Echoes the request's originalAddress so the receiver can route it. */
originalAddress: string;
/** Echoes setupId. */
setupId: string;
/** Index 1..guardianCount — same as the deposit's shareIndex. */
shareIndex: number;
/** Base64-encoded `encodeShare(...)` bytes. */
shareBytes: string;
/** Verbatim copy of the backup blob the guardian stored at setup time. */
backupBlob: string;
}
export interface ShareDeclineEnvelope extends BaseEnvelope<'share-decline'> {
originalAddress: string;
setupId: string;
/** Short, user-visible reason ("fingerprint mismatch", "user declined", "no share for this address"). */
reason: string;
}
export type RecoveryEnvelope =
| ShareDepositEnvelope
| RecoveryRequestEnvelope
| ShareGrantEnvelope
| ShareDeclineEnvelope;
/**
* Serialize an envelope to the JSON string that goes through
* `Shade.send`. Throws if the input is structurally invalid (defensive —
* shouldn't fire under normal flows, but cheap to keep).
*/
export function encodeRecoveryEnvelope(env: RecoveryEnvelope): string {
validateEnvelope(env);
return JSON.stringify(env);
}
/**
* Parse a string payload from `Shade.onMessage`. Returns `null` when the
* payload is not a recovery envelope (so guardian apps can multiplex
* recovery + non-recovery traffic on the same handler). Throws
* `RecoveryProtocolError` on payloads that *claim* to be recovery
* envelopes but are malformed.
*/
export function tryParseRecoveryEnvelope(plaintext: string): RecoveryEnvelope | null {
let parsed: unknown;
try {
parsed = JSON.parse(plaintext);
} catch {
return null;
}
if (!isRecord(parsed)) return null;
if (parsed[RECOVERY_DISCRIMINATOR] !== 1) return null;
if (typeof parsed.type !== 'string') {
throw new RecoveryProtocolError('recovery envelope missing string `type` field');
}
validateEnvelope(parsed as unknown as RecoveryEnvelope);
return parsed as unknown as RecoveryEnvelope;
}
function validateEnvelope(envIn: RecoveryEnvelope): void {
const env = envIn as unknown as Record<string, unknown>;
if (env.shadeRecovery !== 1) {
throw new RecoveryProtocolError(
`unsupported recovery envelope version: ${env.shadeRecovery as unknown as string}`,
);
}
if (typeof env.flowId !== 'string' || (env.flowId as string).length === 0) {
throw new RecoveryProtocolError('recovery envelope missing flowId');
}
switch (env.type) {
case 'share-deposit': {
requireString(env, 'originalAddress');
requireString(env, 'setupId');
requireFiniteInt(env, 'threshold', 1, 255);
requireFiniteInt(env, 'guardianCount', 1, 255);
const threshold = env.threshold as number;
const guardianCount = env.guardianCount as number;
if (threshold > guardianCount) {
throw new RecoveryProtocolError('share-deposit: threshold > guardianCount');
}
requireFiniteInt(env, 'shareIndex', 1, guardianCount);
requireString(env, 'shareBytes');
requireString(env, 'backupBlob');
requireString(env, 'setupFingerprint');
requireFinite(env, 'createdAt');
break;
}
case 'recovery-request':
requireString(env, 'originalAddress');
requireString(env, 'setupId');
requireString(env, 'requesterFingerprint');
requireFinite(env, 'requestedAt');
break;
case 'share-grant':
requireString(env, 'originalAddress');
requireString(env, 'setupId');
requireFiniteInt(env, 'shareIndex', 1, 255);
requireString(env, 'shareBytes');
requireString(env, 'backupBlob');
break;
case 'share-decline':
requireString(env, 'originalAddress');
requireString(env, 'setupId');
requireString(env, 'reason');
break;
default:
throw new RecoveryProtocolError(
`unknown recovery envelope type: ${env.type as unknown as string}`,
);
}
}
function isRecord(v: unknown): v is Record<string, unknown> {
return typeof v === 'object' && v !== null && !Array.isArray(v);
}
function requireString(env: Record<string, unknown>, key: string): void {
const v = env[key];
if (typeof v !== 'string' || v.length === 0) {
throw new RecoveryProtocolError(`recovery envelope missing string field "${key}"`);
}
}
function requireFinite(env: Record<string, unknown>, key: string): void {
const v = env[key];
if (typeof v !== 'number' || !Number.isFinite(v)) {
throw new RecoveryProtocolError(`recovery envelope missing numeric field "${key}"`);
}
}
function requireFiniteInt(
env: Record<string, unknown>,
key: string,
min: number,
max: number,
): void {
const v = env[key];
if (typeof v !== 'number' || !Number.isInteger(v) || v < min || v > max) {
throw new RecoveryProtocolError(
`recovery envelope field "${key}" must be an integer in [${min}, ${max}]`,
);
}
}

View File

@@ -0,0 +1,387 @@
/**
* New-device flow: rebuild the original identity from threshold-many
* guardian shares.
*
* Sequence:
*
* 1. The new device boots a Shade with a temporary identity (caller
* must do this BEFORE invoking `requestRecovery`; the new device
* needs a published prekey bundle so guardians can reply).
* 2. The new device's safety number is read off `shade.fingerprint`
* and embedded in every `recovery-request` envelope so the
* guardian's user can OOB-confirm before approving.
* 3. For each guardian in the supplied list, we send one
* `recovery-request` envelope. We register a transient
* `Shade.onMessage` handler that collects the matching
* `share-grant` and `share-decline` replies.
* 4. When `threshold` distinct grants have arrived, we Shamir-combine
* them, re-derive the backup passphrase, and call
* `Shade.importBackup` — which atomically swaps the temporary
* identity for the recovered one.
* 5. If too many guardians decline (so the threshold can no longer
* be reached) or the timeout elapses, we abort with a typed
* error.
*
* The reconstruction is authenticated end-to-end: a forged share is
* detected when the AES-GCM tag inside the backup blob fails to
* verify. We retry combination with subsets of size `threshold` from
* the received-grants pool until one succeeds OR every subset fails;
* the latter is a {@link RecoveryReconstructionError} the caller
* MUST treat as adversarial (do not retry blindly — at least one
* guardian is malicious).
*/
import type { ShadeEnvelope } from '@shade/core';
import type { Shade } from '@shade/sdk';
import {
base64UrlToBytes,
bytesToBase64Url,
recoveryKeyToBackupPassphrase,
} from './encoding.js';
import {
RecoveryDeclinedError,
RecoveryReconstructionError,
RecoveryTimeoutError,
} from './errors.js';
import {
encodeRecoveryEnvelope,
tryParseRecoveryEnvelope,
type RecoveryRequestEnvelope,
} from './protocol.js';
import type { RecoveryDeliver } from './setup.js';
import { combineShares, decodeShare, type ShamirShare } from './shamir.js';
export interface RequestRecoveryOptions {
/** Initialized Shade with a temporary identity. */
shade: Shade;
/** Address of the original (lost) identity to recover. */
originalAddress: string;
/**
* Guardians to query, in any order. Caller can supply a superset of
* the threshold — we'll continue collecting until we have enough
* grants, then stop early. Length must be ≥ threshold.
*/
guardians: ReadonlyArray<string>;
/** Reconstruction threshold `k`. Must match the value used at setup time. */
threshold: number;
/** Setup id from the original setupRecovery() call. */
setupId: string;
/** Outbound transport callback, same shape as setupRecovery. */
deliver: RecoveryDeliver;
/**
* Timeout for the entire flow in milliseconds. Defaults to 5 min.
* Pass `Infinity` to disable.
*/
timeoutMs?: number;
/**
* Optional progress callback. Fired once per guardian response. The
* caller wires this to UI so the user sees "3/5 guardians responded".
*/
onProgress?: (progress: RecoveryProgress) => void;
/** Wall-clock source. Defaults to Date.now. */
now?: () => number;
}
export interface RecoveryProgress {
granted: number;
declined: number;
pending: number;
threshold: number;
/** Latest guardian to respond. */
fromAddress?: string;
/** Outcome of the latest response. */
latest?: 'grant' | 'decline';
}
export interface RecoveryResult {
/** True iff `Shade.importBackup` ran and returned. */
applied: boolean;
/** Guardians that returned grants. */
granted: ReadonlyArray<string>;
/** Guardians that explicitly declined; absent guardians are not listed here. */
declined: ReadonlyArray<{ address: string; reason: string }>;
/** Setup fingerprint of the original device at setup time. Hand to UI. */
setupFingerprint: string | null;
/**
* Recovered safety number. Will equal the original device's pre-loss
* fingerprint (since we restored the same identity). Compare to the
* pre-loss value the user has on a recovery card / second device for
* sanity-checking.
*/
restoredFingerprint: string;
}
interface PendingShare {
share: ShamirShare;
backupBlob: string;
setupFingerprint?: string;
fromAddress: string;
}
/**
* Drive the recovery flow to completion. On success the new Shade
* instance now hosts the original identity (its prior temporary
* identity is overwritten). On failure throws a typed
* {@link RecoveryError} subclass — never `applied: false`.
*/
export async function requestRecovery(opts: RequestRecoveryOptions): Promise<RecoveryResult> {
validateOptions(opts);
const now = opts.now ?? Date.now;
const timeoutMs = opts.timeoutMs ?? 5 * 60 * 1000;
const requesterFingerprint = await opts.shade.fingerprint;
const flowId = `recover:${opts.setupId}:${bytesToBase64Url(crypto.getRandomValues(new Uint8Array(8)))}`;
// Buckets for collecting responses.
const grants = new Map<string, PendingShare>();
const declines: Array<{ address: string; reason: string }> = [];
const guardianSet = new Set(opts.guardians);
let resolveCollection!: (value: 'enough' | 'declined-out' | 'timeout') => void;
const collected = new Promise<'enough' | 'declined-out' | 'timeout'>((resolve) => {
resolveCollection = resolve;
});
// Fingerprint we're waiting to see at setup-time. Filled as soon as
// any grant lands so subsequent grants can be sanity-checked.
let firstSetupFingerprint: string | null = null;
const detach = opts.shade.onMessage(async (from, plaintext) => {
if (!guardianSet.has(from)) return;
const env = safeParse(plaintext);
if (env === null) return;
if (env.flowId !== flowId) return;
if (env.type === 'share-grant') {
if (grants.has(from)) return; // ignore duplicates
try {
const shareBytes = base64UrlToBytes(env.shareBytes);
const share = decodeShare(shareBytes);
const pending: PendingShare = {
share,
backupBlob: env.backupBlob,
fromAddress: from,
};
if (firstSetupFingerprint === null) {
// Shape from the share-grant doesn't include setupFingerprint;
// we'll fall back to whatever importBackup reports.
firstSetupFingerprint = null;
}
grants.set(from, pending);
} catch {
// Malformed share — count this as an implicit decline.
declines.push({ address: from, reason: 'malformed share-bytes' });
}
} else if (env.type === 'share-decline') {
if (grants.has(from)) return;
declines.push({ address: from, reason: env.reason });
} else {
return;
}
opts.onProgress?.({
granted: grants.size,
declined: declines.length,
pending: opts.guardians.length - grants.size - declines.length,
threshold: opts.threshold,
fromAddress: from,
latest: env.type === 'share-grant' ? 'grant' : 'decline',
});
if (grants.size >= opts.threshold) {
resolveCollection('enough');
return;
}
if (opts.guardians.length - declines.length < opts.threshold) {
resolveCollection('declined-out');
}
});
// Send recovery-request to every guardian. Failures count as
// implicit declines (the network couldn't reach them); we keep
// going so partial reachability still clears the threshold.
const requestedAt = now();
for (const guardian of opts.guardians) {
const env: RecoveryRequestEnvelope = {
shadeRecovery: 1,
type: 'recovery-request',
flowId,
originalAddress: opts.originalAddress,
setupId: opts.setupId,
requesterFingerprint,
requestedAt,
};
const plaintext = encodeRecoveryEnvelope(env);
try {
const envelope: ShadeEnvelope = await opts.shade.send(guardian, plaintext);
await opts.deliver(guardian, envelope);
} catch (err) {
declines.push({
address: guardian,
reason: `delivery failed: ${(err as Error).message}`,
});
opts.onProgress?.({
granted: grants.size,
declined: declines.length,
pending: opts.guardians.length - grants.size - declines.length,
threshold: opts.threshold,
fromAddress: guardian,
latest: 'decline',
});
}
}
// Already over-quota on declines from delivery failures alone.
if (opts.guardians.length - declines.length < opts.threshold) {
resolveCollection('declined-out');
}
// Race the collection against the timeout.
let timer: ReturnType<typeof setTimeout> | null = null;
if (Number.isFinite(timeoutMs)) {
timer = setTimeout(() => resolveCollection('timeout'), timeoutMs);
}
let outcome: 'enough' | 'declined-out' | 'timeout';
try {
outcome = await collected;
} finally {
if (timer !== null) clearTimeout(timer);
detach();
}
if (outcome === 'declined-out') {
throw new RecoveryDeclinedError(
declines.map((d) => d.address),
opts.threshold,
opts.guardians.length - declines.length,
);
}
if (outcome === 'timeout') {
throw new RecoveryTimeoutError(grants.size, opts.threshold);
}
// Reconstruct. Try the natural subset first; if a guardian forged
// their share, retry with `threshold`-sized subsets until one
// authenticates against the AES-GCM tag inside the backup blob.
const granted = Array.from(grants.values());
const result = await tryReconstruct(opts.shade, granted, opts.threshold);
if (result === null) {
throw new RecoveryReconstructionError(
'none of the threshold-sized share subsets authenticated the backup blob — ' +
'at least one guardian likely supplied a forged share',
);
}
const restoredFingerprint = await opts.shade.fingerprint;
return {
applied: true,
granted: granted.map((g) => g.fromAddress),
declined: declines.slice(),
setupFingerprint: firstSetupFingerprint,
restoredFingerprint,
};
}
function safeParse(plaintext: string): ReturnType<typeof tryParseRecoveryEnvelope> {
try {
return tryParseRecoveryEnvelope(plaintext);
} catch {
return null;
}
}
/**
* Attempt reconstruction with successively-chosen `threshold`-sized
* subsets of `granted`. The first subset whose combined recoveryKey
* authenticates the backup blob wins. Returns `null` if every subset
* is rejected by the AEAD.
*
* In the honest case the very first subset (the natural-order
* threshold-many) succeeds and we exit immediately. In the adversarial
* case where one or more guardians supplied forged shares, we
* exhaustively try all C(granted, threshold) subsets — bounded above
* by C(255, 128) which is finite but large, so callers SHOULD cap the
* size of the granted pool to a sane maximum (e.g. threshold + 2)
* before invoking this. Inside `requestRecovery` we never collect more
* than `threshold` grants because we stop the listener as soon as the
* threshold lands; the iterative branch only fires when the caller
* passes a pre-collected set.
*/
async function tryReconstruct(
shade: Shade,
granted: ReadonlyArray<PendingShare>,
threshold: number,
): Promise<{ backupBlob: string } | null> {
if (granted.length < threshold) return null;
// All grants must agree on the backupBlob — if a guardian shipped a
// different blob, we treat it as adversarial and skip its share.
const groups = new Map<string, PendingShare[]>();
for (const g of granted) {
const bucket = groups.get(g.backupBlob);
if (bucket === undefined) groups.set(g.backupBlob, [g]);
else bucket.push(g);
}
for (const [backupBlob, group] of groups) {
if (group.length < threshold) continue;
const subsets = subsetsOfSize(group, threshold);
for (const subset of subsets) {
const recoveryKey = combineShares(subset.map((s) => s.share));
try {
const passphrase = recoveryKeyToBackupPassphrase(recoveryKey);
await shade.importBackup(backupBlob, passphrase);
return { backupBlob };
} catch {
// Wrong combination — keep trying other subsets.
continue;
} finally {
recoveryKey.fill(0);
}
}
}
return null;
}
function* subsetsOfSize<T>(items: ReadonlyArray<T>, size: number): IterableIterator<T[]> {
if (size <= 0) {
yield [];
return;
}
if (items.length < size) return;
const indices = new Array<number>(size);
for (let i = 0; i < size; i++) indices[i] = i;
while (true) {
yield indices.map((i) => items[i]!);
// advance
let i = size - 1;
while (i >= 0 && indices[i]! === items.length - size + i) i--;
if (i < 0) return;
indices[i] = indices[i]! + 1;
for (let j = i + 1; j < size; j++) indices[j] = indices[j - 1]! + 1;
}
}
function validateOptions(opts: RequestRecoveryOptions): void {
if (typeof opts.threshold !== 'number' || !Number.isInteger(opts.threshold) || opts.threshold < 1) {
throw new RangeError('requestRecovery: threshold must be an integer ≥ 1');
}
if (!Array.isArray(opts.guardians) || opts.guardians.length === 0) {
throw new RangeError('requestRecovery: guardians must be a non-empty array');
}
if (opts.guardians.length < opts.threshold) {
throw new RangeError(
`requestRecovery: guardians.length (${opts.guardians.length}) must be ≥ threshold (${opts.threshold})`,
);
}
if (new Set(opts.guardians).size !== opts.guardians.length) {
throw new RangeError('requestRecovery: guardians must be unique');
}
if (opts.guardians.includes(opts.shade.myAddress)) {
throw new RangeError('requestRecovery: cannot include your own address as a guardian');
}
if (typeof opts.deliver !== 'function') {
throw new TypeError('requestRecovery: deliver must be a function');
}
if (typeof opts.originalAddress !== 'string' || opts.originalAddress.length === 0) {
throw new TypeError('requestRecovery: originalAddress must be a non-empty string');
}
if (typeof opts.setupId !== 'string' || opts.setupId.length === 0) {
throw new TypeError('requestRecovery: setupId must be a non-empty string');
}
}

View File

@@ -0,0 +1,229 @@
/**
* Primary-device flow: distribute Shamir shares of an identity backup
* to a quorum of guardians.
*
* The setup happens entirely over existing 1:1 Shade sessions — no
* server-side coordination, no separate transport. The primary calls
* `setupRecovery({ shade, guardians, threshold, deliver })` once;
* under the hood we:
*
* 1. Generate a 32-byte uniformly-random `recoveryKey`.
* 2. Encode it as a base64url passphrase (`shade-rk:<encoded>`).
* 3. Call `shade.exportBackup(passphrase, knownAddresses)` — that
* gives us an AES-GCM-protected backup blob whose key is derived
* from the recoveryKey via HKDF inside the SDK helper.
* 4. Shamir-split the recoveryKey into `guardians.length` shares
* with threshold `threshold`.
* 5. For each guardian, call `shade.send(...)` to encrypt a
* `share-deposit` envelope, then hand it to the caller-supplied
* `deliver` callback for transport. Failures on individual
* guardians do NOT abort the loop — partial-distribution is
* reported back so the caller can surface it.
* 6. Zero the recoveryKey and the in-memory share buffers.
*/
import type { ShadeEnvelope } from '@shade/core';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import type { Shade } from '@shade/sdk';
import {
bytesToBase64Url,
recoveryKeyToBackupPassphrase,
} from './encoding.js';
import { encodeRecoveryEnvelope, type ShareDepositEnvelope } from './protocol.js';
import { encodeShare, splitSecret } from './shamir.js';
/**
* Caller-supplied envelope-delivery callback. The recovery package is
* transport-agnostic by design (Shade itself is); whatever channel
* the host application uses for plaintext messages — WebSocket, push
* notification, polling, in-process pipe in tests — the caller
* implements `deliver` to put the encrypted envelope on it. The
* receiving side is expected to call `shade.receive(from, envelope)`
* which will fire the `onMessage` handler that `attachGuardian` /
* `requestRecovery` register.
*/
export type RecoveryDeliver = (to: string, envelope: ShadeEnvelope) => Promise<void>;
export interface SetupRecoveryOptions {
/** Initialized Shade instance whose identity will be backed up. */
shade: Shade;
/**
* Guardian addresses, in stable order. The order determines each
* guardian's Shamir x-coordinate (1..n), so callers should keep the
* list stable across re-runs of setup if they want predictable share
* indices. Length must equal `n` and be in [1, 255].
*/
guardians: ReadonlyArray<string>;
/**
* Reconstruction threshold `k`. Any subset of `k` guardians can
* recover the identity; any subset of `k-1` reveals nothing. Must
* satisfy `1 ≤ threshold ≤ guardians.length`.
*/
threshold: number;
/** Outbound transport — see {@link RecoveryDeliver}. */
deliver: RecoveryDeliver;
/**
* Peer addresses whose Double-Ratchet sessions should be included in
* the backup. Forwarded verbatim to `Shade.exportBackup`. Pass `[]`
* (default) to back up identity + prekeys only.
*/
knownAddresses?: ReadonlyArray<string>;
/**
* Optional caller-supplied setupId. Defaults to a fresh random id.
* Useful when re-running setup to refresh share material in-place
* without rotating the setupId (e.g. after replacing one guardian).
*/
setupId?: string;
/**
* Wall-clock timestamp embedded in each share-deposit envelope.
* Defaults to `Date.now()`. Tests inject a fixed value for
* determinism.
*/
now?: () => number;
}
export interface SetupRecoveryResult {
/** Stable id the user records alongside their guardian roster. */
setupId: string;
/** k (threshold) and n (guardians.length), echoed for caller convenience. */
threshold: number;
guardianCount: number;
/** Per-guardian outcome of the share-deposit send. */
deliveries: ReadonlyArray<GuardianDelivery>;
/** True iff every guardian got the deposit — convenience flag. */
allDelivered: boolean;
/** Original device's safety number at setup time, for the user's records. */
setupFingerprint: string;
}
export interface GuardianDelivery {
guardianAddress: string;
shareIndex: number;
/**
* `null` when the deposit was sent successfully. Otherwise the
* `Error` raised by `Shade.send` or the `deliver` callback. Failed
* deliveries DO NOT abort the remaining sends — partial-distribution
* is recoverable by retrying just the failures or removing the
* unreachable guardian and re-running setup.
*/
error: Error | null;
}
/**
* Distribute a fresh Shamir-split recovery setup to the supplied
* guardians. See module-level docs for the full flow. Idempotent only
* insofar as the caller pins `setupId` across re-runs; without a
* pinned id every call generates a new (recoveryKey, setupId) pair so
* old shares are silently superseded.
*/
export async function setupRecovery(opts: SetupRecoveryOptions): Promise<SetupRecoveryResult> {
validateOptions(opts);
const crypto = new SubtleCryptoProvider();
const guardianCount = opts.guardians.length;
const threshold = opts.threshold;
const setupId = opts.setupId ?? bytesToBase64Url(crypto.randomBytes(16));
const now = opts.now ?? Date.now;
// 1. Random recoveryKey + 2/3. Backup blob keyed by recoveryKey-derived passphrase.
const recoveryKey = crypto.randomBytes(32);
let setupFingerprint: string;
let backupBlob: string;
try {
const passphrase = recoveryKeyToBackupPassphrase(recoveryKey);
setupFingerprint = await opts.shade.fingerprint;
backupBlob = await opts.shade.exportBackup(passphrase, [...(opts.knownAddresses ?? [])]);
} catch (err) {
crypto.zeroize(recoveryKey);
throw err;
}
// 4. Shamir-split the recoveryKey.
let shares;
try {
shares = splitSecret(
recoveryKey,
threshold,
guardianCount,
(length) => crypto.randomBytes(length),
);
} finally {
// The recoveryKey lives in `splitSecret` only as the constant term
// of each per-byte polynomial; once split returns we can wipe it.
crypto.zeroize(recoveryKey);
}
// 5. Deliver one share-deposit per guardian.
const flowId = `setup:${setupId}`;
const createdAt = now();
const deliveries: GuardianDelivery[] = [];
for (let i = 0; i < guardianCount; i++) {
const guardianAddress = opts.guardians[i]!;
const share = shares[i]!;
const shareBytes = bytesToBase64Url(encodeShare(share));
const envelope: ShareDepositEnvelope = {
shadeRecovery: 1,
type: 'share-deposit',
flowId,
originalAddress: opts.shade.myAddress,
setupId,
threshold,
guardianCount,
shareIndex: share.x,
shareBytes,
backupBlob,
setupFingerprint,
createdAt,
};
const plaintext = encodeRecoveryEnvelope(envelope);
let error: Error | null = null;
try {
const env = await opts.shade.send(guardianAddress, plaintext);
await opts.deliver(guardianAddress, env);
} catch (err) {
error = err instanceof Error ? err : new Error(String(err));
}
deliveries.push({ guardianAddress, shareIndex: share.x, error });
// Wipe the share's y-buffer once it has been encoded + sent so it
// doesn't linger in JS memory longer than necessary.
share.y.fill(0);
}
return {
setupId,
threshold,
guardianCount,
deliveries,
allDelivered: deliveries.every((d) => d.error === null),
setupFingerprint,
};
}
function validateOptions(opts: SetupRecoveryOptions): void {
if (typeof opts.threshold !== 'number' || !Number.isInteger(opts.threshold)) {
throw new TypeError('setupRecovery: threshold must be an integer');
}
if (opts.threshold < 1) {
throw new RangeError('setupRecovery: threshold must be ≥ 1');
}
if (!Array.isArray(opts.guardians) || opts.guardians.length === 0) {
throw new RangeError('setupRecovery: guardians must be a non-empty array');
}
if (opts.guardians.length > 255) {
throw new RangeError('setupRecovery: guardians.length must be ≤ 255 (GF(2^8))');
}
if (opts.threshold > opts.guardians.length) {
throw new RangeError('setupRecovery: threshold must be ≤ guardians.length');
}
if (new Set(opts.guardians).size !== opts.guardians.length) {
throw new RangeError('setupRecovery: guardians must be unique');
}
if (opts.guardians.includes(opts.shade.myAddress)) {
throw new RangeError(
'setupRecovery: cannot use your own address as a guardian (the share would die with the device)',
);
}
if (typeof opts.deliver !== 'function') {
throw new TypeError('setupRecovery: deliver must be a function');
}
}

View File

@@ -0,0 +1,226 @@
/**
* Shamir Secret Sharing over GF(2^8).
*
* Splits a secret byte-array into `n` shares such that any `k` (the
* threshold) reconstruct the original, but any combination of `k-1` or
* fewer reveals nothing about the secret beyond its length. Each byte of
* the secret is shared independently using a random polynomial of degree
* `k-1` whose constant term is the secret byte; shares are points on that
* polynomial evaluated at `x = 1..n`.
*
* Field: GF(2^8) with the irreducible polynomial 0x11b (AES'). Tables for
* exp and log are precomputed at module load time and reused for both
* multiplication and inversion. All field operations are constant-time
* (table lookups + xor / mod), so split + combine leak nothing about the
* secret bytes through timing.
*
* The wire format for a share is:
*
* 1 byte x-coordinate (1..255)
* N bytes y-coordinates, one per secret byte
*
* `splitSecret` returns an array of length `n`. `combineShares` accepts
* any subset of shares; the threshold is implicit in the polynomial
* degree the sender chose. Combining `k-1` shares yields a different
* (random-looking) result — there's no way to detect under-threshold
* combination from the shares alone, so callers must enforce the
* threshold separately (e.g. by waiting for `k` arrivals before combining)
* AND verify the reconstructed key against the AEAD tag of the
* ciphertext it was meant to decrypt.
*/
const FIELD_SIZE = 256;
const EXP = new Uint8Array(FIELD_SIZE * 2);
const LOG = new Uint8Array(FIELD_SIZE);
(function buildTables(): void {
// Generator: 0x03. Standard AES-style table buildup.
let x = 1;
for (let i = 0; i < 255; i++) {
EXP[i] = x;
LOG[x] = i;
// Multiply x by the generator (0x03) in GF(2^8) with the AES
// reduction polynomial 0x1b on overflow.
let next = x ^ ((x << 1) & 0xff);
if (x & 0x80) next ^= 0x1b;
x = next & 0xff;
}
// Mirror the first half so `EXP[i + j]` works for any 0..510 without
// an extra modulus.
for (let i = 255; i < FIELD_SIZE * 2; i++) EXP[i] = EXP[i - 255]!;
})();
function gfMul(a: number, b: number): number {
if (a === 0 || b === 0) return 0;
return EXP[LOG[a]! + LOG[b]!]!;
}
function gfDiv(a: number, b: number): number {
if (b === 0) throw new Error('Shamir: division by zero');
if (a === 0) return 0;
// a/b = exp(log(a) - log(b)) — keep the index non-negative by adding 255.
return EXP[LOG[a]! - LOG[b]! + 255]!;
}
/**
* Evaluate a polynomial `coeffs[0] + coeffs[1]*x + … + coeffs[d]*x^d`
* at point `x` using Horner's method. Constant-time in the coefficients
* (no early-out on zero terms).
*/
function evalPoly(coeffs: Uint8Array, x: number): number {
let acc = coeffs[coeffs.length - 1]!;
for (let i = coeffs.length - 2; i >= 0; i--) {
acc = gfMul(acc, x) ^ coeffs[i]!;
}
return acc;
}
/**
* One Shamir share.
*
* - `x`: the x-coordinate, 1..255 (unique per share within a split).
* - `y`: a y-coordinate for each byte of the original secret.
*/
export interface ShamirShare {
x: number;
y: Uint8Array;
}
/**
* Split `secret` into `n` shares; any `k` reconstructs.
*
* @param secret the bytes to split (any length ≥ 1)
* @param k threshold (1 ≤ k ≤ n ≤ 255)
* @param n total number of shares
* @param random RNG callback (length → bytes). Must be cryptographically
* secure. Inject `crypto.randomBytes.bind(crypto)` from a
* `CryptoProvider` to reuse the SDK's RNG.
* @returns array of `n` shares with x-coordinates 1..n (skipping 0
* because P(0) IS the secret).
*/
export function splitSecret(
secret: Uint8Array,
k: number,
n: number,
random: (length: number) => Uint8Array,
): ShamirShare[] {
if (!Number.isInteger(k) || !Number.isInteger(n)) {
throw new Error('Shamir: k and n must be integers');
}
if (k < 1) throw new Error('Shamir: threshold must be ≥ 1');
if (n < k) throw new Error('Shamir: n must be ≥ k');
if (n > 255) throw new Error('Shamir: n must be ≤ 255 (GF(2^8) limit)');
if (secret.length === 0) throw new Error('Shamir: secret must be non-empty');
// Pre-allocate per-share y-buffers. We fill column-wise: for each byte
// of the secret we draw a fresh random polynomial of degree k-1 with
// constant term equal to the secret byte, then emit y[byteIndex] for
// each share at its x-coordinate.
const shares: ShamirShare[] = [];
for (let i = 0; i < n; i++) {
shares.push({ x: i + 1, y: new Uint8Array(secret.length) });
}
// For each column (byte of the secret), build a fresh polynomial.
const coeffs = new Uint8Array(k);
for (let byteIdx = 0; byteIdx < secret.length; byteIdx++) {
coeffs[0] = secret[byteIdx]!;
if (k > 1) {
const rnd = random(k - 1);
for (let j = 0; j < k - 1; j++) coeffs[j + 1] = rnd[j]!;
}
for (let i = 0; i < n; i++) {
const share = shares[i]!;
share.y[byteIdx] = evalPoly(coeffs, share.x);
}
// Zero the polynomial buffer between columns. Doesn't help against a
// V8 GC adversary but is the right discipline.
coeffs.fill(0);
}
return shares;
}
/**
* Reconstruct the original secret from a subset of shares using
* Lagrange interpolation evaluated at x = 0.
*
* `shares.length` MUST equal the threshold the secret was split with
* (the algorithm doesn't know `k`; supplying fewer or more shares either
* yields garbage or wastes work). The returned secret has the same
* length as each share's y-buffer.
*
* The secret cannot be authenticated from the shares alone — the caller
* is expected to verify the reconstructed key against an AEAD tag (e.g.
* the AES-GCM ciphertext it was used to encrypt). Without that
* authentication, an attacker who supplied a forged share at any of the
* sample points can flip the reconstructed key to anything they want.
*/
export function combineShares(shares: ShamirShare[]): Uint8Array {
if (shares.length === 0) throw new Error('Shamir: need at least one share');
const len = shares[0]!.y.length;
for (const s of shares) {
if (s.y.length !== len) throw new Error('Shamir: share length mismatch');
if (s.x === 0) throw new Error('Shamir: x-coordinate 0 is reserved for the secret');
}
// Reject duplicate x-coordinates: two shares with the same x but
// different y collide as polynomial evaluations and produce nonsense.
const xs = new Set<number>();
for (const s of shares) {
if (xs.has(s.x)) throw new Error('Shamir: duplicate x-coordinate in share set');
xs.add(s.x);
}
const out = new Uint8Array(len);
// Precompute Lagrange basis weights at x=0:
// L_i(0) = ∏_{j ≠ i} -x_j / (x_i - x_j)
// In GF(2^8) negation is identity, so this simplifies to ∏ x_j / (x_i + x_j).
const weights = new Uint8Array(shares.length);
for (let i = 0; i < shares.length; i++) {
let num = 1;
let den = 1;
const xi = shares[i]!.x;
for (let j = 0; j < shares.length; j++) {
if (i === j) continue;
const xj = shares[j]!.x;
num = gfMul(num, xj);
den = gfMul(den, xi ^ xj);
}
weights[i] = gfDiv(num, den);
}
for (let byteIdx = 0; byteIdx < len; byteIdx++) {
let acc = 0;
for (let i = 0; i < shares.length; i++) {
acc ^= gfMul(shares[i]!.y[byteIdx]!, weights[i]!);
}
out[byteIdx] = acc;
}
return out;
}
/**
* Encode a `ShamirShare` to a single Uint8Array on the wire:
*
* 1 byte x-coordinate
* N bytes y-coordinates
*
* `decodeShare` is the inverse. Callers ship this byte sequence as
* opaque payload — the encoding is stable across platforms and is what
* `@shade/recovery` puts on the wire.
*/
export function encodeShare(share: ShamirShare): Uint8Array {
if (share.x < 1 || share.x > 255) throw new Error('Shamir: x out of range [1..255]');
const out = new Uint8Array(1 + share.y.length);
out[0] = share.x;
out.set(share.y, 1);
return out;
}
export function decodeShare(bytes: Uint8Array): ShamirShare {
if (bytes.length < 2) throw new Error('Shamir: encoded share must be ≥ 2 bytes');
const x = bytes[0]!;
if (x === 0) throw new Error('Shamir: x-coordinate 0 in encoded share');
return { x, y: bytes.slice(1) };
}

View File

@@ -0,0 +1,91 @@
/**
* Guardian-side persistence for received shares.
*
* Each guardian holds, per protected user, exactly one record per
* `setupId`:
*
* {
* originalAddress, // the protected user's Shade address
* setupId, // disambiguates if a user re-runs setup
* shareIndex, // their assigned x-coordinate
* shareBytes, // base64-encoded Shamir share
* shareSecretCiphertext, // base64 — encrypted backup payload
* shareSecretNonce, // base64 — AES-GCM nonce
* setupFingerprint, // safety number at deposit time
* guardianCount, threshold, // informational
* receivedAt
* }
*
* The interface is deliberately minimal so consumers can back it with
* whatever they already use for app state — IndexedDB, AsyncStorage,
* SQLite, etc. The package ships an in-memory implementation suitable
* for tests and small one-device demos. Guardian apps that survive
* restart MUST supply a persistent implementation.
*/
export interface GuardianShareEntry {
originalAddress: string;
setupId: string;
shareIndex: number;
shareBytes: string;
shareSecretCiphertext: string;
shareSecretNonce: string;
setupFingerprint: string;
guardianCount: number;
threshold: number;
receivedAt: number;
}
/**
* The minimal contract a guardian-side share store must implement.
*
* Shapes:
* - `save` is upsert by (originalAddress, setupId). If a user re-runs
* setup with a different setupId, both entries persist; the guardian
* can choose which to release based on the recovery-request's
* setupId. If the same (originalAddress, setupId) is saved twice,
* the second write wins (idempotent re-deposit).
* - `get` returns the deposit for the (originalAddress, setupId) pair,
* or `null` when none exists.
* - `list` enumerates everything (used by guardian-UX widgets).
* - `delete` removes a single deposit; the guardian-UX exposes this so
* a user can prune deposits from people they no longer wish to
* protect.
*/
export interface RecoveryStore {
save(entry: GuardianShareEntry): Promise<void>;
get(originalAddress: string, setupId: string): Promise<GuardianShareEntry | null>;
list(): Promise<GuardianShareEntry[]>;
delete(originalAddress: string, setupId: string): Promise<void>;
}
/**
* Process-local in-memory store. Suitable for tests and one-shot demos;
* LOSES STATE ON RESTART. Production guardian apps must supply a
* persistent `RecoveryStore` (see `docs/recovery.md` for backing-store
* recommendations).
*/
export class MemoryRecoveryStore implements RecoveryStore {
private readonly entries = new Map<string, GuardianShareEntry>();
async save(entry: GuardianShareEntry): Promise<void> {
this.entries.set(keyOf(entry.originalAddress, entry.setupId), { ...entry });
}
async get(originalAddress: string, setupId: string): Promise<GuardianShareEntry | null> {
const found = this.entries.get(keyOf(originalAddress, setupId));
return found === undefined ? null : { ...found };
}
async list(): Promise<GuardianShareEntry[]> {
return Array.from(this.entries.values()).map((e) => ({ ...e }));
}
async delete(originalAddress: string, setupId: string): Promise<void> {
this.entries.delete(keyOf(originalAddress, setupId));
}
}
function keyOf(originalAddress: string, setupId: string): string {
return `${originalAddress} ${setupId}`;
}

Some files were not shown because too many files have changed in this diff Show More