feat: Shade E2EE library — M1-M3 complete
Signal Protocol implementation with full X3DH + Double Ratchet:
- M1: Core types, CryptoProvider interface, KDF chain functions,
SubtleCrypto+noble/curves provider, MemoryStorage
- M2: X3DH key agreement (identity keys, signed prekeys, one-time
prekeys, bundle processing for both initiator and responder)
- M3: Double Ratchet (symmetric-key ratchet, DH ratchet, skipped
message key cache, out-of-order delivery, AAD-bound headers)
68 tests, 0 failures — including full integration test of
X3DH handshake → Double Ratchet conversation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:08:19 +02:00
|
|
|
import type { IdentityKeyPair, SignedPreKey, OneTimePreKey, SessionState } from './types.js';
|
|
|
|
|
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
/** A retired identity kept in history during the rotation grace period */
|
|
|
|
|
export interface RetiredIdentity {
|
|
|
|
|
keyPair: IdentityKeyPair;
|
|
|
|
|
retiredAt: number;
|
|
|
|
|
}
|
|
|
|
|
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
/**
|
|
|
|
|
* Persisted stream-transfer resume record. Holds enough state for either
|
|
|
|
|
* side of a transfer to resume after restart. The `streamSecret` MUST be
|
|
|
|
|
* encrypted before storage (see `@shade/transfer` for the deviceKey-based
|
|
|
|
|
* AES-GCM at-rest scheme).
|
|
|
|
|
*/
|
|
|
|
|
export interface PersistedStreamState {
|
|
|
|
|
streamId: string;
|
|
|
|
|
direction: 'send' | 'receive';
|
|
|
|
|
peerAddress: string;
|
|
|
|
|
status: 'active' | 'paused' | 'finished' | 'aborted';
|
|
|
|
|
/** JSON-serialized `StreamMetadata`. */
|
|
|
|
|
metadataJson: string;
|
|
|
|
|
/** JSON-serialized `LaneInitSpec[]`. */
|
|
|
|
|
partitionJson: string;
|
|
|
|
|
/** JSON-serialized per-lane progress array (laneId/nextSeq/bytesProcessed). */
|
|
|
|
|
laneStateJson: string;
|
|
|
|
|
/** JSON-serialized I/O descriptor (file path / file handle reference / buffer). */
|
|
|
|
|
ioDescriptorJson: string;
|
|
|
|
|
/** AES-GCM-encrypted streamSecret (under deviceKey). */
|
|
|
|
|
secretEnc: Uint8Array;
|
|
|
|
|
/** AES-GCM nonce used for `secretEnc`. */
|
|
|
|
|
secretNonce: Uint8Array;
|
|
|
|
|
/**
|
|
|
|
|
* Reserved for future hasher serialization. Empty in v0.2.0; resume
|
|
|
|
|
* re-hashes received bytes from disk.
|
|
|
|
|
*/
|
|
|
|
|
overallHashState?: string;
|
|
|
|
|
createdAt: number;
|
|
|
|
|
updatedAt: number;
|
|
|
|
|
}
|
|
|
|
|
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
/**
|
|
|
|
|
* Why a peer's fingerprint was deemed verified.
|
|
|
|
|
* - `user`: human did the side-channel comparison and confirmed.
|
|
|
|
|
* - `transitive`: trust derived from another verified party (reserved for V3.10).
|
|
|
|
|
* - `tofu-after-warning`: caller bypassed gate after seeing a warning.
|
|
|
|
|
*/
|
|
|
|
|
export type PeerVerificationSource = 'user' | 'transitive' | 'tofu-after-warning';
|
|
|
|
|
|
2026-05-07 15:55:34 +02:00
|
|
|
/**
|
|
|
|
|
* Persisted broadcast-channel state (V4.6). Holds the sender-key chain that
|
|
|
|
|
* lets the channel owner encrypt a single ciphertext and fan it out to all
|
|
|
|
|
* paired members.
|
|
|
|
|
*
|
|
|
|
|
* - `ownerRole === 'sender'`: this device created the channel; the record
|
|
|
|
|
* carries the full chain (chainKey + iteration + signing keypair).
|
|
|
|
|
* - `ownerRole === 'receiver'`: this device joined a channel that
|
|
|
|
|
* `ownerAddress` owns; we hold a tracking copy of the chain (chainKey +
|
|
|
|
|
* iteration + signing public key only — no private signing key).
|
|
|
|
|
*
|
|
|
|
|
* `generation` is bumped each time `removeMember` rotates the chain. Stale
|
|
|
|
|
* broadcasts at lower generations are silently dropped on receive.
|
|
|
|
|
*/
|
|
|
|
|
export interface BroadcastChannelRecord {
|
|
|
|
|
/** Opaque, stable across restarts. UUID-style. */
|
|
|
|
|
channelId: string;
|
|
|
|
|
ownerRole: 'sender' | 'receiver';
|
|
|
|
|
/** The address that owns the channel (i.e. the only sender). */
|
|
|
|
|
ownerAddress: string;
|
|
|
|
|
label?: string;
|
|
|
|
|
/** Sender-key generation — bumped on rotation. Starts at 0. */
|
|
|
|
|
generation: number;
|
|
|
|
|
/** Current chain key (32 bytes). */
|
|
|
|
|
chainKey: Uint8Array;
|
|
|
|
|
/** Counter advanced by `senderKeyEncrypt` / `senderKeyDecrypt`. */
|
|
|
|
|
iteration: number;
|
|
|
|
|
/** Owner's Ed25519 signing public key (32 bytes). */
|
|
|
|
|
signingPublicKey: Uint8Array;
|
|
|
|
|
/** Owner's Ed25519 signing private key. Present iff `ownerRole === 'sender'`. */
|
|
|
|
|
signingPrivateKey?: Uint8Array;
|
|
|
|
|
createdAt: number;
|
|
|
|
|
updatedAt: number;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/** Membership row for a broadcast channel (sender-side only). */
|
|
|
|
|
export interface BroadcastMemberRecord {
|
|
|
|
|
channelId: string;
|
|
|
|
|
peerAddress: string;
|
|
|
|
|
joinedAt: number;
|
|
|
|
|
/** When this peer was revoked. `null` while still active. */
|
|
|
|
|
removedAt: number | null;
|
|
|
|
|
}
|
|
|
|
|
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
/**
|
|
|
|
|
* Persistent record that a peer's safety number was verified at a point
|
|
|
|
|
* in time. `identityVersion` is the local counter for that peer's identity:
|
|
|
|
|
* incrementing it (e.g. via `bumpPeerIdentityVersion` on `acceptIdentityChange`)
|
|
|
|
|
* invalidates the saved verification because `isPeerVerified` requires the
|
|
|
|
|
* stored version to equal the current version.
|
|
|
|
|
*/
|
|
|
|
|
export interface PeerVerification {
|
|
|
|
|
peerAddress: string;
|
|
|
|
|
fingerprint: string;
|
|
|
|
|
verifiedAt: number;
|
|
|
|
|
verifiedBy: PeerVerificationSource;
|
|
|
|
|
identityVersion: number;
|
|
|
|
|
}
|
|
|
|
|
|
feat: Shade E2EE library — M1-M3 complete
Signal Protocol implementation with full X3DH + Double Ratchet:
- M1: Core types, CryptoProvider interface, KDF chain functions,
SubtleCrypto+noble/curves provider, MemoryStorage
- M2: X3DH key agreement (identity keys, signed prekeys, one-time
prekeys, bundle processing for both initiator and responder)
- M3: Double Ratchet (symmetric-key ratchet, DH ratchet, skipped
message key cache, out-of-order delivery, AAD-bound headers)
68 tests, 0 failures — including full integration test of
X3DH handshake → Double Ratchet conversation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:08:19 +02:00
|
|
|
/**
|
|
|
|
|
* StorageProvider — abstract interface for persisting cryptographic state.
|
|
|
|
|
*
|
|
|
|
|
* Implementations per platform:
|
|
|
|
|
* - In-memory (testing)
|
|
|
|
|
* - IndexedDB (browser)
|
|
|
|
|
* - SQLite/PostgreSQL (server)
|
|
|
|
|
* - EncryptedSharedPreferences (Android)
|
|
|
|
|
*/
|
|
|
|
|
export interface StorageProvider {
|
|
|
|
|
// ─── Identity ──────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/** Get our local identity keypair, or null if not yet generated */
|
|
|
|
|
getIdentityKeyPair(): Promise<IdentityKeyPair | null>;
|
|
|
|
|
|
|
|
|
|
/** Persist our local identity keypair */
|
|
|
|
|
saveIdentityKeyPair(keyPair: IdentityKeyPair): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Get our local registration ID (unique per installation) */
|
|
|
|
|
getLocalRegistrationId(): Promise<number>;
|
|
|
|
|
|
|
|
|
|
/** Save our local registration ID */
|
|
|
|
|
saveLocalRegistrationId(id: number): Promise<void>;
|
|
|
|
|
|
|
|
|
|
// ─── Signed Pre-Keys ──────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/** Get a signed prekey by ID */
|
|
|
|
|
getSignedPreKey(keyId: number): Promise<SignedPreKey | null>;
|
|
|
|
|
|
|
|
|
|
/** Persist a signed prekey */
|
|
|
|
|
saveSignedPreKey(key: SignedPreKey): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Remove a signed prekey (after rotation grace period) */
|
|
|
|
|
removeSignedPreKey(keyId: number): Promise<void>;
|
|
|
|
|
|
|
|
|
|
// ─── One-Time Pre-Keys ────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/** Get a one-time prekey by ID */
|
|
|
|
|
getOneTimePreKey(keyId: number): Promise<OneTimePreKey | null>;
|
|
|
|
|
|
|
|
|
|
/** Persist a one-time prekey */
|
|
|
|
|
saveOneTimePreKey(key: OneTimePreKey): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Remove a consumed one-time prekey */
|
|
|
|
|
removeOneTimePreKey(keyId: number): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Count remaining one-time prekeys */
|
|
|
|
|
getOneTimePreKeyCount(): Promise<number>;
|
|
|
|
|
|
|
|
|
|
// ─── Sessions ─────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
/** Get session state for a peer address (e.g. "device:abc123") */
|
|
|
|
|
getSession(address: string): Promise<SessionState | null>;
|
|
|
|
|
|
|
|
|
|
/** Persist session state for a peer */
|
|
|
|
|
saveSession(address: string, state: SessionState): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Remove session for a peer */
|
|
|
|
|
removeSession(address: string): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Check if we trust a remote identity key (for TOFU or pinned keys) */
|
|
|
|
|
isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean>;
|
|
|
|
|
|
|
|
|
|
/** Save a trusted remote identity key */
|
|
|
|
|
saveTrustedIdentity(address: string, identityKey: Uint8Array): Promise<void>;
|
feat(hardening): M-Hard 1-5 — crypto, auth, rate limit, fingerprints, rotation
M-Hard 1: Cryptographic Hardening
- constantTimeEqual, zeroize, randomUint32 on CryptoProvider
- Fix Math.random() → crypto.randomUint32() for registrationId
- Zero message keys and chain keys after use in ratchet.ts
- Constant-time trust comparison in MemoryStorage + SQLiteStorage
- Timing variance test catches early-exit regressions
M-Hard 2: Self-Authenticated Prekey Server
- Ed25519 signature verification on all write routes
- signPayload/verifyPayload with canonical JSON, ±5 min replay window
- Address validation (NFKC, alphanumeric + :_-.)
- Global ShadeError → HTTP status mapping
- ShadeFetchTransport signs requests when signingPrivateKey provided
- Anonymous bundle fetches still allowed (read-only)
M-Hard 3: Rate Limiting + DoS Protection
- Token bucket rate limiter with pluggable store
- Per-route limits: register 5/h/IP, fetch 60/min/IP, replenish 10/min/id
- 64KB body size limit on POST
- Retry-After header on 429 responses
M-Hard 4: Auto-replenish + Fingerprints + Session Reset
- Safety numbers (12 groups × 5 digits, Signal-style)
- ensurePreKeyStock, resetSession, acceptIdentityChange
- verifyRemoteIdentity for out-of-band comparison
M-Hard 5: Identity Rotation with Grace Period
- rotateIdentity archives old identity, generates fresh signed prekey
- RetiredIdentity storage with addRetired/getRetired/pruneRetired
- 7-day default grace period for decrypting old sessions
- pruneExpiredIdentities for cleanup
M-Hard 8: Error Hierarchy
- New error types: Network, Storage, Validation, Timeout, RateLimit,
Configuration, Unauthorized, Replay, IdentityRotation
- All errors have stable SHADE_* codes
- errorToHttpStatus for consistent HTTP mapping
- toJSON() for network serialization
188 tests passing, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:45:34 +02:00
|
|
|
|
|
|
|
|
// ─── Identity History (rotation with grace period) ──────
|
|
|
|
|
|
|
|
|
|
/** Add an identity to the retired history */
|
|
|
|
|
addRetiredIdentity(identity: RetiredIdentity): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Get all retired identities (for grace-period decryption) */
|
|
|
|
|
getRetiredIdentities(): Promise<RetiredIdentity[]>;
|
|
|
|
|
|
|
|
|
|
/** Remove retired identities older than the given timestamp */
|
|
|
|
|
pruneRetiredIdentities(olderThan: number): Promise<void>;
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
// ─── Peer verifications (V3.3) ────────────────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Persist or replace the verification record for a peer. Idempotent
|
|
|
|
|
* upsert on `peerAddress`.
|
|
|
|
|
*/
|
|
|
|
|
savePeerVerification(verification: PeerVerification): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Look up the saved verification for a peer (null if never verified). */
|
|
|
|
|
getPeerVerification(address: string): Promise<PeerVerification | null>;
|
|
|
|
|
|
|
|
|
|
/** Remove a peer's verification record (e.g. user revoked trust). */
|
|
|
|
|
removePeerVerification(address: string): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Returns the current local identity-version counter for a peer.
|
|
|
|
|
* Defaults to 1 when the peer has never been seen. Bumped by
|
|
|
|
|
* `bumpPeerIdentityVersion` whenever the peer rotates identity.
|
|
|
|
|
*/
|
|
|
|
|
getPeerIdentityVersion(address: string): Promise<number>;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Increment the peer's identity-version counter and return the new value.
|
|
|
|
|
* Called from `acceptIdentityChange` so previous verification rows (which
|
|
|
|
|
* carry the old version number) become stale.
|
|
|
|
|
*/
|
|
|
|
|
bumpPeerIdentityVersion(address: string): Promise<number>;
|
|
|
|
|
|
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
|
|
|
// ─── Stream-transfer resume state (optional, added in v0.2.0) ──
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Persist or update the stream-state row for a given streamId. Idempotent:
|
|
|
|
|
* upserts on `streamId`. Optional — providers that don't support resume
|
|
|
|
|
* can omit this and consumers will fall back to in-memory state.
|
|
|
|
|
*/
|
|
|
|
|
saveStreamState?(state: PersistedStreamState): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Look up the stream-state row by streamId. Returns null if absent. */
|
|
|
|
|
getStreamState?(streamId: string): Promise<PersistedStreamState | null>;
|
|
|
|
|
|
|
|
|
|
/** Remove a stream-state row (e.g. on completion or abort). */
|
|
|
|
|
removeStreamState?(streamId: string): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** List active or paused stream-state rows (for resume on startup). */
|
|
|
|
|
listActiveStreamStates?(
|
|
|
|
|
direction?: 'send' | 'receive',
|
|
|
|
|
): Promise<PersistedStreamState[]>;
|
|
|
|
|
|
|
|
|
|
/** Prune stream-state rows in `'finished' | 'aborted'` status older than `olderThan`. */
|
|
|
|
|
pruneStreamStates?(olderThan: number): Promise<void>;
|
2026-05-07 15:55:34 +02:00
|
|
|
|
|
|
|
|
// ─── Broadcast channels (V4.6) — optional ─────────────────
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Persist or replace the broadcast-channel record. Idempotent upsert on
|
|
|
|
|
* `channelId`. Backends that don't implement broadcast-channel support
|
|
|
|
|
* may omit this; the SDK throws a clear error when the app tries to
|
|
|
|
|
* call `createBroadcastChannel` on such a backend.
|
|
|
|
|
*/
|
|
|
|
|
saveBroadcastChannel?(channel: BroadcastChannelRecord): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Look up a broadcast channel by id. */
|
|
|
|
|
getBroadcastChannel?(channelId: string): Promise<BroadcastChannelRecord | null>;
|
|
|
|
|
|
|
|
|
|
/** Enumerate all broadcast channels persisted on this device. */
|
|
|
|
|
listBroadcastChannels?(): Promise<BroadcastChannelRecord[]>;
|
|
|
|
|
|
|
|
|
|
/** Drop a broadcast channel and all its membership rows. */
|
|
|
|
|
removeBroadcastChannel?(channelId: string): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/** Persist or replace a broadcast-membership row (sender-side only). */
|
|
|
|
|
saveBroadcastMember?(member: BroadcastMemberRecord): Promise<void>;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* List membership rows for a channel. Includes revoked members (with
|
|
|
|
|
* `removedAt !== null`); callers filter as needed.
|
|
|
|
|
*/
|
|
|
|
|
getBroadcastMembers?(channelId: string): Promise<BroadcastMemberRecord[]>;
|
|
|
|
|
|
|
|
|
|
/** Hard-delete a single membership row. */
|
|
|
|
|
removeBroadcastMember?(channelId: string, peerAddress: string): Promise<void>;
|
feat: Shade E2EE library — M1-M3 complete
Signal Protocol implementation with full X3DH + Double Ratchet:
- M1: Core types, CryptoProvider interface, KDF chain functions,
SubtleCrypto+noble/curves provider, MemoryStorage
- M2: X3DH key agreement (identity keys, signed prekeys, one-time
prekeys, bundle processing for both initiator and responder)
- M3: Double Ratchet (symmetric-key ratchet, DH ratchet, skipped
message key cache, out-of-order delivery, AAD-bound headers)
68 tests, 0 failures — including full integration test of
X3DH handshake → Double Ratchet conversation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:08:19 +02:00
|
|
|
}
|