2026-04-09 20:12:01 +02:00
|
|
|
import type { CryptoProvider } from './crypto.js';
|
|
|
|
|
import type { StorageProvider } from './storage.js';
|
|
|
|
|
import type {
|
|
|
|
|
IdentityKeyPair,
|
|
|
|
|
SignedPreKey,
|
|
|
|
|
OneTimePreKey,
|
|
|
|
|
PreKeyBundle,
|
|
|
|
|
PreKeyMessage,
|
|
|
|
|
RatchetMessage,
|
|
|
|
|
ShadeEnvelope,
|
|
|
|
|
} from './types.js';
|
|
|
|
|
import {
|
|
|
|
|
generateIdentityKeyPair,
|
|
|
|
|
generateSignedPreKey,
|
|
|
|
|
generateOneTimePreKeys,
|
|
|
|
|
createPreKeyBundle,
|
|
|
|
|
processPreKeyBundle,
|
|
|
|
|
processPreKeyMessage,
|
|
|
|
|
} from './x3dh.js';
|
|
|
|
|
import {
|
|
|
|
|
initSenderSession,
|
|
|
|
|
initReceiverSession,
|
|
|
|
|
ratchetEncrypt,
|
|
|
|
|
ratchetDecrypt,
|
|
|
|
|
} from './ratchet.js';
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
import { NoSessionError } from './errors.js';
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
import { computeFingerprint, shortFingerprint } from './fingerprint.js';
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
import { ShadeEventEmitter, shortHash } from './events.js';
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
import {
|
|
|
|
|
ATTR_ERROR_CODE,
|
|
|
|
|
ATTR_OP,
|
|
|
|
|
ATTR_PEER_HASH,
|
|
|
|
|
ATTR_RESULT,
|
|
|
|
|
NOOP_HOOK,
|
|
|
|
|
peerHash,
|
|
|
|
|
type ObservabilityHook,
|
|
|
|
|
type Span,
|
|
|
|
|
} from '@shade/observability';
|
2026-04-09 20:12:01 +02:00
|
|
|
|
|
|
|
|
const enc = new TextEncoder();
|
|
|
|
|
const dec = new TextDecoder();
|
|
|
|
|
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
/** Default grace period for retired identities: 7 days */
|
|
|
|
|
export const GRACE_PERIOD_MS = 7 * 24 * 60 * 60 * 1000;
|
|
|
|
|
|
2026-04-09 20:12:01 +02:00
|
|
|
/**
|
|
|
|
|
* ShadeSessionManager — the high-level API for using Shade.
|
|
|
|
|
*
|
|
|
|
|
* Wraps X3DH key agreement and Double Ratchet into a simple interface:
|
|
|
|
|
* - `initialize()` — generate or load identity keys
|
|
|
|
|
* - `createPreKeyBundle()` — publish to prekey server
|
|
|
|
|
* - `encrypt(address, plaintext)` — encrypt for a peer
|
|
|
|
|
* - `decrypt(address, envelope)` — decrypt from a peer
|
|
|
|
|
*
|
|
|
|
|
* Usage:
|
|
|
|
|
* ```ts
|
|
|
|
|
* const manager = new ShadeSessionManager(crypto, storage);
|
|
|
|
|
* await manager.initialize();
|
|
|
|
|
*
|
|
|
|
|
* // To initiate: fetch bundle, then encrypt
|
|
|
|
|
* await manager.initSessionFromBundle('bob', bundle);
|
|
|
|
|
* const envelope = await manager.encrypt('bob', 'Hello!');
|
|
|
|
|
*
|
|
|
|
|
* // To receive: decrypt handles everything
|
|
|
|
|
* const plaintext = await manager.decrypt('alice', envelope);
|
|
|
|
|
* ```
|
|
|
|
|
*/
|
|
|
|
|
export class ShadeSessionManager {
|
|
|
|
|
private identity: IdentityKeyPair | null = null;
|
|
|
|
|
private registrationId: number = 0;
|
|
|
|
|
private currentSignedPreKeyId: number = 0;
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
private readonly events?: ShadeEventEmitter;
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
private readonly observability: ObservabilityHook;
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
/**
|
|
|
|
|
* Per-address operation chain. Both encrypt and decrypt mutate ratchet
|
|
|
|
|
* state in place (counter, DH key, skipped-keys cache); concurrent
|
|
|
|
|
* operations on the same peer can corrupt the session. We serialize
|
|
|
|
|
* per-peer by chaining promises — operations to different peers stay
|
|
|
|
|
* fully concurrent.
|
|
|
|
|
*/
|
|
|
|
|
private readonly peerOpChains = new Map<string, Promise<unknown>>();
|
2026-04-09 20:12:01 +02:00
|
|
|
|
|
|
|
|
constructor(
|
|
|
|
|
private readonly crypto: CryptoProvider,
|
|
|
|
|
private readonly storage: StorageProvider,
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
options: { events?: ShadeEventEmitter; observability?: ObservabilityHook } = {},
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
) {
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
if (options.events !== undefined) {
|
|
|
|
|
this.events = options.events;
|
|
|
|
|
}
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
this.observability = options.observability ?? NOOP_HOOK;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Wrap a per-peer crypto op in a PII-safe span. The span captures the
|
|
|
|
|
* mutex-acquire latency separately from the inner crypto work so a
|
|
|
|
|
* "ratchet contention" pathology shows up clearly in traces.
|
|
|
|
|
*/
|
|
|
|
|
private async withSpan<T>(
|
|
|
|
|
op: 'encrypt' | 'decrypt',
|
|
|
|
|
address: string,
|
|
|
|
|
fn: () => Promise<T>,
|
|
|
|
|
): Promise<T> {
|
|
|
|
|
const span: Span = this.observability.startSpan(`shade.session.${op}`, {
|
|
|
|
|
[ATTR_OP]: op,
|
|
|
|
|
[ATTR_PEER_HASH]: peerHash(address),
|
|
|
|
|
});
|
|
|
|
|
const lockStart = nowMs();
|
|
|
|
|
try {
|
|
|
|
|
return await this.runUnderPeerLock(address, async () => {
|
|
|
|
|
span.setAttribute('shade.lock.wait_ms', Math.round(nowMs() - lockStart));
|
|
|
|
|
const result = await fn();
|
|
|
|
|
span.setAttribute(ATTR_RESULT, 'ok');
|
|
|
|
|
span.setStatus('ok');
|
|
|
|
|
return result;
|
|
|
|
|
});
|
|
|
|
|
} catch (err) {
|
|
|
|
|
span.setAttribute(ATTR_RESULT, 'error');
|
|
|
|
|
span.setAttribute(ATTR_ERROR_CODE, sessionErrorCodeOf(err));
|
|
|
|
|
span.recordException(err);
|
|
|
|
|
span.setStatus('error');
|
|
|
|
|
throw err;
|
|
|
|
|
} finally {
|
|
|
|
|
span.end();
|
|
|
|
|
}
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Run `fn` under the per-address mutex so encrypt/decrypt for the same
|
|
|
|
|
* peer never interleave their session-state mutations.
|
|
|
|
|
*/
|
|
|
|
|
private async runUnderPeerLock<T>(address: string, fn: () => Promise<T>): Promise<T> {
|
|
|
|
|
const previous = this.peerOpChains.get(address) ?? Promise.resolve();
|
|
|
|
|
const next = previous.catch(() => undefined).then(fn);
|
|
|
|
|
this.peerOpChains.set(address, next);
|
|
|
|
|
try {
|
|
|
|
|
return await next;
|
|
|
|
|
} finally {
|
|
|
|
|
// Best-effort cleanup so finished chains can be garbage collected
|
|
|
|
|
// when a peer goes idle. If a newer op has chained on, we leave it.
|
|
|
|
|
if (this.peerOpChains.get(address) === next) {
|
|
|
|
|
this.peerOpChains.delete(address);
|
|
|
|
|
}
|
|
|
|
|
}
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Get the event emitter (if observability is enabled) */
|
|
|
|
|
getEvents(): ShadeEventEmitter | undefined {
|
|
|
|
|
return this.events;
|
|
|
|
|
}
|
2026-04-09 20:12:01 +02:00
|
|
|
|
|
|
|
|
// ─── Initialization ────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/** Initialize: load or generate identity keys and a signed prekey */
|
|
|
|
|
async initialize(): Promise<void> {
|
|
|
|
|
// Load or generate identity
|
|
|
|
|
this.identity = await this.storage.getIdentityKeyPair();
|
|
|
|
|
if (!this.identity) {
|
|
|
|
|
this.identity = await generateIdentityKeyPair(this.crypto);
|
|
|
|
|
await this.storage.saveIdentityKeyPair(this.identity);
|
|
|
|
|
}
|
|
|
|
|
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
// Load or generate registration ID (cryptographically secure)
|
2026-04-09 20:12:01 +02:00
|
|
|
this.registrationId = await this.storage.getLocalRegistrationId();
|
|
|
|
|
if (this.registrationId === 0) {
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
// Ensure nonzero (0 is the "unset" sentinel)
|
|
|
|
|
let id = this.crypto.randomUint32();
|
|
|
|
|
if (id === 0) id = 1;
|
|
|
|
|
this.registrationId = id;
|
2026-04-09 20:12:01 +02:00
|
|
|
await this.storage.saveLocalRegistrationId(this.registrationId);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Generate initial signed prekey if none exists
|
|
|
|
|
const spk = await this.storage.getSignedPreKey(1);
|
|
|
|
|
if (!spk) {
|
|
|
|
|
const signedPreKey = await generateSignedPreKey(this.crypto, this.identity, 1);
|
|
|
|
|
await this.storage.saveSignedPreKey(signedPreKey);
|
|
|
|
|
this.currentSignedPreKeyId = 1;
|
|
|
|
|
} else {
|
|
|
|
|
this.currentSignedPreKeyId = spk.keyId;
|
|
|
|
|
}
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
|
|
|
|
|
// Emit identity initialization event
|
|
|
|
|
if (this.events) {
|
|
|
|
|
const fingerprint = await this.getIdentityFingerprint();
|
|
|
|
|
this.events.emit('identity.initialized', {
|
|
|
|
|
fingerprint,
|
|
|
|
|
registrationId: this.registrationId,
|
|
|
|
|
});
|
|
|
|
|
}
|
2026-04-09 20:12:01 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Get our identity's DH public key (for addressing) */
|
|
|
|
|
getPublicIdentity(): { signingKey: Uint8Array; dhKey: Uint8Array } {
|
|
|
|
|
if (!this.identity) throw new Error('Not initialized');
|
|
|
|
|
return {
|
|
|
|
|
signingKey: this.identity.signingPublicKey,
|
|
|
|
|
dhKey: this.identity.dhPublicKey,
|
|
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
// ─── Fingerprints (Safety Numbers) ──────────────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a human-readable fingerprint of our own identity.
|
|
|
|
|
* Format: 12 groups of 5 decimal digits (60 total).
|
|
|
|
|
* Two parties compare these out-of-band to verify no MITM.
|
|
|
|
|
*/
|
|
|
|
|
async getIdentityFingerprint(): Promise<string> {
|
|
|
|
|
if (!this.identity) throw new Error('Not initialized');
|
|
|
|
|
return computeFingerprint(
|
|
|
|
|
this.crypto,
|
|
|
|
|
this.identity.signingPublicKey,
|
|
|
|
|
this.identity.dhPublicKey,
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Short 4-group fingerprint for quick comparison */
|
|
|
|
|
async getShortFingerprint(): Promise<string> {
|
|
|
|
|
return shortFingerprint(await this.getIdentityFingerprint());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get a fingerprint for a remote peer's identity.
|
|
|
|
|
* Throws NoSessionError if we haven't established a session with them.
|
|
|
|
|
*/
|
|
|
|
|
async getRemoteFingerprint(address: string): Promise<string> {
|
|
|
|
|
const session = await this.storage.getSession(address);
|
|
|
|
|
if (!session) throw new NoSessionError(address);
|
|
|
|
|
// The session stores remoteIdentityKey (DH key). We need the signing key too,
|
|
|
|
|
// which we don't store per-session. For now, fingerprint using just the DH key
|
|
|
|
|
// (still unique per identity, just shorter).
|
|
|
|
|
// In the future, store remoteIdentitySigningKey alongside.
|
|
|
|
|
return computeFingerprint(
|
|
|
|
|
this.crypto,
|
|
|
|
|
session.remoteIdentityKey,
|
|
|
|
|
session.remoteIdentityKey,
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ─── Prekey Stock Management ────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Ensure the one-time prekey stock is above a minimum threshold.
|
|
|
|
|
* If below `min`, generates enough to bring it up to `target`.
|
|
|
|
|
* Returns the number of new keys generated (0 if no action needed).
|
|
|
|
|
*/
|
|
|
|
|
async ensurePreKeyStock(min = 5, target = 20): Promise<number> {
|
|
|
|
|
const current = await this.storage.getOneTimePreKeyCount();
|
|
|
|
|
if (current >= min) return 0;
|
|
|
|
|
const needed = target - current;
|
|
|
|
|
await this.generateOneTimePreKeys(needed);
|
|
|
|
|
return needed;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ─── Session Reset / Identity Change ────────────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Delete the session for a peer. The next message will trigger a fresh X3DH.
|
|
|
|
|
* Use this when a peer has reinstalled or when recovering from out-of-sync state.
|
|
|
|
|
*/
|
|
|
|
|
async resetSession(address: string): Promise<void> {
|
|
|
|
|
await this.storage.removeSession(address);
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
this.events?.emit('session.removed', { address });
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
// Note: we keep the trusted identity; new session will verify against it.
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Accept a changed remote identity. This should only be called after
|
|
|
|
|
* verifying the new identity out-of-band (e.g., comparing fingerprints).
|
|
|
|
|
* After this, any pinned trust for this address is replaced.
|
|
|
|
|
*/
|
|
|
|
|
async acceptIdentityChange(address: string, newIdentityKey: Uint8Array): Promise<void> {
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
// Capture old hash for the trust.changed event (TOFU semantics make this messy
|
|
|
|
|
// because isTrustedIdentity() compares not retrieves; we just emit the new hash)
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
await this.storage.saveTrustedIdentity(address, newIdentityKey);
|
|
|
|
|
await this.storage.removeSession(address);
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
|
|
|
|
|
if (this.events) {
|
|
|
|
|
const newHash = await shortHash(this.crypto, newIdentityKey);
|
|
|
|
|
this.events.emit('trust.changed', { address, oldKeyHash: '?', newKeyHash: newHash });
|
|
|
|
|
this.events.emit('session.removed', { address });
|
|
|
|
|
}
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Check whether a remote identity key matches what we have pinned for an address.
|
|
|
|
|
* Returns true on TOFU (no pinned key yet) or exact match.
|
|
|
|
|
*/
|
|
|
|
|
async verifyRemoteIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
|
|
|
|
return this.storage.isTrustedIdentity(address, identityKey);
|
|
|
|
|
}
|
|
|
|
|
|
2026-04-09 20:12:01 +02:00
|
|
|
// ─── Prekey Management ─────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/** Create a prekey bundle to publish to the prekey server */
|
|
|
|
|
async createPreKeyBundle(): Promise<PreKeyBundle> {
|
|
|
|
|
if (!this.identity) throw new Error('Not initialized');
|
|
|
|
|
const spk = await this.storage.getSignedPreKey(this.currentSignedPreKeyId);
|
|
|
|
|
if (!spk) throw new Error('No signed prekey');
|
|
|
|
|
|
|
|
|
|
// Try to include a one-time prekey
|
|
|
|
|
// (In real usage, the prekey server would pick one — here we just check if any exist)
|
|
|
|
|
return createPreKeyBundle(this.registrationId, this.identity, spk);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Generate and store a batch of one-time prekeys */
|
|
|
|
|
async generateOneTimePreKeys(count: number): Promise<OneTimePreKey[]> {
|
|
|
|
|
const existingCount = await this.storage.getOneTimePreKeyCount();
|
|
|
|
|
const startId = existingCount + 1;
|
|
|
|
|
const keys = await generateOneTimePreKeys(this.crypto, startId, count);
|
|
|
|
|
for (const key of keys) {
|
|
|
|
|
await this.storage.saveOneTimePreKey(key);
|
|
|
|
|
}
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
this.events?.emit('prekey.generated', {
|
|
|
|
|
count,
|
|
|
|
|
totalAfter: existingCount + count,
|
|
|
|
|
});
|
2026-04-09 20:12:01 +02:00
|
|
|
return keys;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Rotate the signed prekey (recommended: every 1-7 days) */
|
|
|
|
|
async rotateSignedPreKey(): Promise<SignedPreKey> {
|
|
|
|
|
if (!this.identity) throw new Error('Not initialized');
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
const oldId = this.currentSignedPreKeyId;
|
|
|
|
|
const newId = oldId + 1;
|
2026-04-09 20:12:01 +02:00
|
|
|
const spk = await generateSignedPreKey(this.crypto, this.identity, newId);
|
|
|
|
|
await this.storage.saveSignedPreKey(spk);
|
|
|
|
|
// Keep old one for a grace period (sessions may still reference it)
|
|
|
|
|
this.currentSignedPreKeyId = newId;
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
this.events?.emit('signed_prekey.rotated', { oldKeyId: oldId, newKeyId: newId });
|
2026-04-09 20:12:01 +02:00
|
|
|
return spk;
|
|
|
|
|
}
|
|
|
|
|
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
// ─── Identity Rotation (with grace period) ─────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Rotate the identity keypair.
|
|
|
|
|
*
|
|
|
|
|
* Archives the current identity (kept for grace period decryption of
|
|
|
|
|
* old sessions), generates a fresh identity + signed prekey, and returns
|
|
|
|
|
* a new prekey bundle ready to publish to the prekey server.
|
|
|
|
|
*
|
|
|
|
|
* Callers should:
|
|
|
|
|
* 1. Call this method
|
|
|
|
|
* 2. Re-publish the new bundle via ShadeFetchTransport.register()
|
|
|
|
|
* 3. Optionally broadcast the rotation to known peers out-of-band
|
|
|
|
|
*
|
|
|
|
|
* The old identity is retained for GRACE_PERIOD_MS so existing sessions
|
|
|
|
|
* continue decrypting. Call `pruneExpiredIdentities()` periodically.
|
|
|
|
|
*/
|
|
|
|
|
async rotateIdentity(): Promise<PreKeyBundle> {
|
|
|
|
|
if (!this.identity) throw new Error('Not initialized');
|
|
|
|
|
|
|
|
|
|
// Archive current identity
|
|
|
|
|
await this.storage.addRetiredIdentity({
|
|
|
|
|
keyPair: this.identity,
|
|
|
|
|
retiredAt: Date.now(),
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
// Generate new identity + save
|
|
|
|
|
this.identity = await generateIdentityKeyPair(this.crypto);
|
|
|
|
|
await this.storage.saveIdentityKeyPair(this.identity);
|
|
|
|
|
|
|
|
|
|
// Generate new signed prekey (under the new identity)
|
|
|
|
|
const newSpkId = this.currentSignedPreKeyId + 1;
|
|
|
|
|
const spk = await generateSignedPreKey(this.crypto, this.identity, newSpkId);
|
|
|
|
|
await this.storage.saveSignedPreKey(spk);
|
|
|
|
|
this.currentSignedPreKeyId = newSpkId;
|
|
|
|
|
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
if (this.events) {
|
|
|
|
|
const newFingerprint = await this.getIdentityFingerprint();
|
|
|
|
|
this.events.emit('identity.rotated', { newFingerprint });
|
|
|
|
|
}
|
|
|
|
|
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
// Return a fresh bundle for re-publication
|
|
|
|
|
return createPreKeyBundle(this.registrationId, this.identity, spk);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Get all retired identities that are still within the grace period.
|
|
|
|
|
* Used internally for trying previous identities when X3DH fails.
|
|
|
|
|
*/
|
|
|
|
|
async getActiveRetiredIdentities(gracePeriodMs = GRACE_PERIOD_MS): Promise<IdentityKeyPair[]> {
|
|
|
|
|
const all = await this.storage.getRetiredIdentities();
|
|
|
|
|
const cutoff = Date.now() - gracePeriodMs;
|
|
|
|
|
return all.filter((r) => r.retiredAt >= cutoff).map((r) => r.keyPair);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Delete retired identities older than the grace period.
|
|
|
|
|
* Call this periodically (e.g., daily cleanup task).
|
|
|
|
|
*/
|
|
|
|
|
async pruneExpiredIdentities(gracePeriodMs = GRACE_PERIOD_MS): Promise<void> {
|
|
|
|
|
const cutoff = Date.now() - gracePeriodMs;
|
|
|
|
|
await this.storage.pruneRetiredIdentities(cutoff);
|
|
|
|
|
}
|
|
|
|
|
|
2026-04-09 20:12:01 +02:00
|
|
|
// ─── Session Establishment ─────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Initiate a session with a peer by processing their prekey bundle.
|
|
|
|
|
* Call this before the first `encrypt()` to a new peer.
|
|
|
|
|
*/
|
|
|
|
|
async initSessionFromBundle(address: string, bundle: PreKeyBundle): Promise<void> {
|
|
|
|
|
const x3dhResult = await processPreKeyBundle(this.crypto, this.storage, bundle);
|
|
|
|
|
|
|
|
|
|
const session = await initSenderSession(
|
|
|
|
|
this.crypto,
|
|
|
|
|
x3dhResult.rootKey,
|
|
|
|
|
x3dhResult.remoteIdentityKey,
|
|
|
|
|
x3dhResult.remoteSignedPreKey,
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
await this.storage.saveSession(address, session);
|
|
|
|
|
await this.storage.saveTrustedIdentity(address, x3dhResult.remoteIdentityKey);
|
|
|
|
|
|
|
|
|
|
// Store X3DH metadata for the first message
|
|
|
|
|
// We stash this on the session object for the first encrypt call
|
|
|
|
|
(session as any).__x3dh = {
|
|
|
|
|
ephemeralPublicKey: x3dhResult.ephemeralPublicKey,
|
|
|
|
|
signedPreKeyId: x3dhResult.signedPreKeyId,
|
|
|
|
|
preKeyId: x3dhResult.preKeyId,
|
|
|
|
|
identityDHKey: this.identity!.dhPublicKey,
|
|
|
|
|
registrationId: this.registrationId,
|
|
|
|
|
};
|
|
|
|
|
await this.storage.saveSession(address, session);
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
|
|
|
|
|
if (this.events) {
|
|
|
|
|
const remoteHash = await shortHash(this.crypto, x3dhResult.remoteIdentityKey);
|
|
|
|
|
this.events.emit('session.created', { address, remoteIdentityKeyHash: remoteHash });
|
|
|
|
|
this.events.emit('trust.pinned', { address, identityKeyHash: remoteHash });
|
|
|
|
|
}
|
2026-04-09 20:12:01 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ─── Encrypt / Decrypt ─────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Encrypt a message for a peer. Returns a ShadeEnvelope ready to send.
|
|
|
|
|
*
|
|
|
|
|
* The first message to a new peer will be a PreKeyMessage (includes X3DH info).
|
|
|
|
|
* Subsequent messages are standard RatchetMessages.
|
|
|
|
|
*/
|
|
|
|
|
async encrypt(address: string, plaintext: string): Promise<ShadeEnvelope> {
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
return this.withSpan('encrypt', address, async () => {
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
const session = await this.storage.getSession(address);
|
|
|
|
|
if (!session) throw new NoSessionError(address);
|
2026-04-09 20:12:01 +02:00
|
|
|
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
const ratchetMsg = await ratchetEncrypt(this.crypto, session, enc.encode(plaintext));
|
2026-04-09 20:12:01 +02:00
|
|
|
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
this.events?.emit('message.encrypted', {
|
|
|
|
|
address,
|
|
|
|
|
counter: ratchetMsg.counter,
|
|
|
|
|
ciphertextSize: ratchetMsg.ciphertext.length,
|
|
|
|
|
});
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
// Check if this is the first message (X3DH metadata attached)
|
|
|
|
|
const x3dh = (session as any).__x3dh;
|
|
|
|
|
if (x3dh) {
|
|
|
|
|
delete (session as any).__x3dh;
|
|
|
|
|
await this.storage.saveSession(address, session);
|
|
|
|
|
|
|
|
|
|
const preKeyMsg: PreKeyMessage = {
|
|
|
|
|
registrationId: x3dh.registrationId,
|
|
|
|
|
preKeyId: x3dh.preKeyId,
|
|
|
|
|
signedPreKeyId: x3dh.signedPreKeyId,
|
|
|
|
|
ephemeralKey: x3dh.ephemeralPublicKey,
|
|
|
|
|
identityDHKey: x3dh.identityDHKey,
|
|
|
|
|
message: ratchetMsg,
|
|
|
|
|
};
|
|
|
|
|
return {
|
|
|
|
|
type: 'prekey',
|
|
|
|
|
content: preKeyMsg,
|
|
|
|
|
timestamp: Date.now(),
|
|
|
|
|
senderAddress: address,
|
|
|
|
|
};
|
|
|
|
|
}
|
2026-04-09 20:12:01 +02:00
|
|
|
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
await this.storage.saveSession(address, session);
|
2026-04-09 20:12:01 +02:00
|
|
|
return {
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
type: 'ratchet',
|
|
|
|
|
content: ratchetMsg,
|
2026-04-09 20:12:01 +02:00
|
|
|
timestamp: Date.now(),
|
|
|
|
|
senderAddress: address,
|
|
|
|
|
};
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
});
|
2026-04-09 20:12:01 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Decrypt a message from a peer. Handles both PreKeyMessage and RatchetMessage.
|
|
|
|
|
*/
|
|
|
|
|
async decrypt(address: string, envelope: ShadeEnvelope): Promise<string> {
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
return this.withSpan('decrypt', address, async () => {
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
if (envelope.type === 'prekey') {
|
|
|
|
|
return this.decryptPreKeyMessage(address, envelope.content as PreKeyMessage);
|
|
|
|
|
}
|
|
|
|
|
return this.decryptRatchetMessage(address, envelope.content as RatchetMessage);
|
|
|
|
|
});
|
2026-04-09 20:12:01 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
private async decryptPreKeyMessage(address: string, message: PreKeyMessage): Promise<string> {
|
|
|
|
|
// Process X3DH to establish session
|
|
|
|
|
const x3dhResult = await processPreKeyMessage(this.crypto, this.storage, message);
|
|
|
|
|
|
|
|
|
|
// Find the signed prekey that was used
|
|
|
|
|
const spk = await this.storage.getSignedPreKey(message.signedPreKeyId);
|
|
|
|
|
if (!spk) throw new Error(`Signed prekey ${message.signedPreKeyId} not found`);
|
|
|
|
|
|
|
|
|
|
const session = initReceiverSession(
|
|
|
|
|
x3dhResult.rootKey,
|
|
|
|
|
x3dhResult.remoteIdentityKey,
|
|
|
|
|
spk.keyPair,
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
// Decrypt the embedded first ratchet message
|
|
|
|
|
const plaintext = await ratchetDecrypt(this.crypto, session, x3dhResult.initialMessage);
|
|
|
|
|
|
|
|
|
|
await this.storage.saveSession(address, session);
|
|
|
|
|
await this.storage.saveTrustedIdentity(address, x3dhResult.remoteIdentityKey);
|
|
|
|
|
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
if (this.events) {
|
|
|
|
|
const remoteHash = await shortHash(this.crypto, x3dhResult.remoteIdentityKey);
|
|
|
|
|
this.events.emit('session.created', { address, remoteIdentityKeyHash: remoteHash });
|
|
|
|
|
this.events.emit('trust.pinned', { address, identityKeyHash: remoteHash });
|
|
|
|
|
if (message.preKeyId != null) {
|
|
|
|
|
this.events.emit('prekey.consumed', { keyId: message.preKeyId });
|
|
|
|
|
}
|
|
|
|
|
this.events.emit('message.decrypted', {
|
|
|
|
|
address,
|
|
|
|
|
counter: x3dhResult.initialMessage.counter,
|
|
|
|
|
plaintextSize: plaintext.length,
|
|
|
|
|
});
|
|
|
|
|
}
|
|
|
|
|
|
2026-04-09 20:12:01 +02:00
|
|
|
return dec.decode(plaintext);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
private async decryptRatchetMessage(address: string, message: RatchetMessage): Promise<string> {
|
|
|
|
|
const session = await this.storage.getSession(address);
|
|
|
|
|
if (!session) throw new NoSessionError(address);
|
|
|
|
|
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
// Detect DH ratchet step (new remote DH key)
|
|
|
|
|
const willRatchet = !session.dhReceive ||
|
|
|
|
|
!arraysEqual(message.dhPublicKey, session.dhReceive);
|
|
|
|
|
|
2026-04-09 20:12:01 +02:00
|
|
|
const plaintext = await ratchetDecrypt(this.crypto, session, message);
|
|
|
|
|
await this.storage.saveSession(address, session);
|
|
|
|
|
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
if (this.events) {
|
|
|
|
|
if (willRatchet) {
|
|
|
|
|
this.events.emit('ratchet.dh_step', { address });
|
|
|
|
|
}
|
|
|
|
|
this.events.emit('message.decrypted', {
|
|
|
|
|
address,
|
|
|
|
|
counter: message.counter,
|
|
|
|
|
plaintextSize: plaintext.length,
|
|
|
|
|
});
|
|
|
|
|
}
|
|
|
|
|
|
2026-04-09 20:12:01 +02:00
|
|
|
return dec.decode(plaintext);
|
|
|
|
|
}
|
|
|
|
|
}
|
feat(observer): M-Obs 1-3 — event bus, server hooks, observer backend
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:49:51 +02:00
|
|
|
|
|
|
|
|
function arraysEqual(a: Uint8Array, b: Uint8Array): boolean {
|
|
|
|
|
if (a.length !== b.length) return false;
|
|
|
|
|
for (let i = 0; i < a.length; i++) {
|
|
|
|
|
if (a[i] !== b[i]) return false;
|
|
|
|
|
}
|
|
|
|
|
return true;
|
|
|
|
|
}
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
|
|
|
|
|
function nowMs(): number {
|
|
|
|
|
return typeof performance !== 'undefined' ? performance.now() : Date.now();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function sessionErrorCodeOf(err: unknown): string {
|
|
|
|
|
if (err === null || err === undefined) return 'SHADE_UNKNOWN';
|
|
|
|
|
if (typeof err === 'object') {
|
|
|
|
|
const code = (err as { code?: unknown }).code;
|
|
|
|
|
if (typeof code === 'string' && code.length > 0) return code;
|
|
|
|
|
const name = (err as { name?: unknown }).name;
|
|
|
|
|
if (typeof name === 'string' && name.length > 0) return name;
|
|
|
|
|
}
|
|
|
|
|
return 'SHADE_UNKNOWN';
|
|
|
|
|
}
|