84d3166ca1a01852409ed983a064383c7b2be448
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 96c20cb4b2 |
fix(session): remember where aliasSession moved a session
aliasSession knew that two labels name the same peer, then threw that knowledge away. The binding lived only in the caller's memory, so a restart lost it — and the peer could not repair it from its side. First contact forces the receiver to label a session by the only sender hint a relay surfaces, an 8-byte signing-key fingerprint (`fp:<hex>`). Once the peer announces its canonical address, aliasSession moves the session there. But the peer keeps sending under `fp:<hex>`, because its transport derives the same label from the same hint every time. After a restart the session sat under the canonical address, inbound frames resolved to `fp:<hex>`, and nothing matched. The peer held a valid session so it never re-ran X3DH: the failure was permanent, and only a manual re-link cleared it. Observed in Prism as `No session for address: fp:579c3b335d66e2c0` on every receive for three days, with a phone whose every RPC timed out. StorageProvider gains saveSessionAlias / getSessionAlias / removeSessionAliasesFor, optional so third-party implementations keep compiling, and implemented across all seven backends. Lookups resolve through resolveLabel(), which runs BEFORE the peer mutex — locking the alias while mutating the canonical session would let an aliased and a canonical caller ratchet the same state concurrently. A live session under a label always wins over an alias, and prekey envelopes never resolve: both keep a re-link establishing a fresh session instead of being redirected into the stale one. Aliases are dropped in resetSession and acceptIdentityChange, and memoized so the hot path costs no extra read. The sdk.test.ts case that asserted a dead fp-label encoded the old behaviour; it now pins the new contract. Verified: 1166 tests pass (from 1160). With alias persistence disabled as a negative control, 5 of the 6 new tests fail, including both restart cases. Also drops `baseUrl` from the consumer-strict tsconfig — removed in TS 6.0, and it was failing the typecheck that gates publishing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
|||
| 037f994572 |
release(v4.11.0): streaming Double-Ratchet sub-sessions (ShadeStream)
Some checks failed
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Test / test (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled
Answers Vyvern FR shade-ws-streaming-ratchet.md with a first-class
streaming-session API rather than the documented-contract fallback.
The Double-Ratchet crypto was already safe for high-frequency
one-directional use; the send/receive wrapper was not (per-frame
saveSession keystore write; shared per-peer mutex + single stored
session row coupling reuse to the HTTP path).
- @shade/core: stream.ts — identity-bound 3-DH seeding (X3DH-minus-
prekeys, no prekey-server round trip, mutually authenticated against
the parent session's pinned identities), bootstrapStreamSession
reusing init{Sender,Receiver}Session verbatim, in-memory-only
StreamRatchet (own op-mutex, never persisted, zeroized on close).
beginStream/acceptStream on ShadeSessionManager; Stream{Closed,
Handshake}Error; stream.opened/closed events.
- @shade/proto: STREAM_OPEN/OPEN_ACK/FRAME wire (0x31/0x32/0x33),
additive; inspectEnvelopeType extended.
- @shade/sdk: Shade.openStream/acceptStream → ShadeStream
(handshakeFrame/handleHandshake/seal/open/close), transport-
agnostic, independent of encrypt/decrypt queues + parent session,
identical server (sqlite:) and browser (IndexedDB) — touches no
storage.
- Tests: 5000-frame one-directional burst (bounded skipped keys + FS
zeroize), parent-session independence, replay/rewind rejection,
mutual-auth, proto wire round-trips. Full suite green (1159 pass).
- docs/streaming-sessions.md (R1–R7 contract); SECURITY.md matrix rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| d47774ef1c |
release(v4.8.3): cross-channel msgId dedup + Shade.aliasSession
Two follow-ups to the V4.8.2 duplicate-fan-out fixes Prism filed. 1. `Inbox.acceptBridgeFrame(blob)` + shared 4096-entry msgId LRU. The relay durably stores blobs and pushes them to every active delivery channel; without a cross-channel ack the bridge frame ran first and the next inbox-poll re-dispatched the same blob ~30 s later, tripping on consumed prekeys. Bridge consumers now plumb pushed frames through `acceptBridgeFrame`, which shares the dedup gate + ack path with `pollOnce`. Whichever channel delivers first wins; the other acks-and-skips. Inbox records the msgId before the ack so a parallel poll can't observe an in-flight ack window. 2. `Shade.aliasSession(oldLabel, newLabel)`. First-contact forces the receiver to label the new session by the relay's sender fingerprint hint (`fp:<senderfp>`); the post-decrypt plaintext typically announces the peer's real address. Aliasing moves session, trusted identity, peer-verification, and identity- version under the canonical label. Holds the per-peer mutex on both labels (lexicographic order) so concurrent crypto ops can't observe a half-moved state. Refuses to overwrite an existing session at the new label. Wire change: `IncomingMessage.expiresAt?` now surfaces the relay's expiry so receivers can pass bridge frames straight to `acceptBridgeFrame` without inventing a TTL. Tests cover bridge-then-poll, poll-then-bridge, aliasSession happy path, refuse-to-overwrite, and same-label no-op. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 8c606ad498 |
release(v4.8.2): per-from receive serialization + per-connection bridge dedup
Some checks failed
Test / test (push) Has been cancelled
Two interlocking robustness fixes for the duplicate-fan-out / first-contact
class of failures Prism reported.
1. `Shade.receive(from, env)` now queues its `manager.decrypt` step
per `from` so concurrent dispatches can't race the SessionManager
ratchet or the StorageProvider (sqlite "database is locked", IDB
transaction conflicts). User message handlers run *outside* the
queue so streams + file-RPC's nested `shade.receive` calls don't
self-deadlock.
2. Bridge WS + SSE handlers now run a per-connection bounded msgId
LRU as defense-in-depth against any flushTo re-entry (event-storm,
future refactor). Pending-flush chains are wrapped in `.catch(() =>
{})` so a transient `ws.send` rejection no longer poisons the
connection's flush loop.
Tests: storming `inbox.blob_stored` 10× per PUT yields exactly one WS/
SSE frame; 8 concurrent `bob.receive('alice', envelope)` calls keep
the ratchet intact and never surface "database is locked".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 2c400d7094 |
release(v4.6.0): broadcast channels — Signal sender-keys for one-to-many fan-out
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled
Lands the broadcast-channel primitive Prism asked for in Docs/shade-feature-request-sender-keys.md. The crypto in @shade/core/sender-keys.ts was already in place; this release wires it up as a first-class app-facing API, adds the persistence schema across all six storage backends (memory, sqlite, indexeddb + encrypted variants), introduces wire type 0x21 in @shade/proto, and ships Prism's three acceptance tests verbatim. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| dbb3a090d8 |
release(v4.4.0): public accessor for device identity public key
Expose the local device's 32-byte Ed25519 identity public key on Shade so apps can hand it to their own backend at enrollment time for signature verification, key pinning or per-device safety-number computation. Closes the gap that forced consumers to ship placeholder random bytes their backend could store but never verify against. - @shade/sdk Shade.identityPublicKey: Promise<Uint8Array> — getter mirrors the existing fingerprint accessor. Throws pre-init, reflects the current key after rotate(), retired key preserved in retired-identities storage per existing grace-period contract. Private key remains unreachable. - Test in shade-sdk/tests/sdk.test.ts: round-trip match against the underlying storage's signingPublicKey, plus value updates after rotate(). - Lockstep version bump 4.3.0 → 4.4.0 across all 25 packages. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| f5f42fe557 |
release(v4.3.0): browser persistence via @shade/storage-indexeddb
Ship an official IndexedDB-backed StorageProvider so browser-based Shade
consumers persist identity, prekeys, sessions, retired identities,
peer-verification state and stream-resume rows across tab refresh and
browser restart. Closes the gap that forced browser apps onto
storage:"memory" (regenerated identity each load, orphaned device
records server-side).
- New package @shade/storage-indexeddb (4.3.0): full StorageProvider
conformance, schema v1, idb-backed; bumpPeerIdentityVersion is wrapped
in a single readwrite IDB transaction (atomic, vs SQLite's
read-then-upsert race).
- @shade/sdk resolveStorage() accepts { type: 'indexeddb', dbName? } via
dynamic import (lazy, optional dep — same pattern as
@shade/storage-postgres). Named StorageSpec type now reused by
ResolvedConfig.
- Tests: 16 new tests in shade-storage-indexeddb (StorageProvider
surface + peer-verifications + full E2EE conversation surviving a
simulated tab reload). Run on fake-indexeddb.
- Lockstep version bump 4.2.1 → 4.3.0 across all 25 packages.
- Publish scripts updated to include the new package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 7520b11b25 |
release(v4.2.0): pull-mode streams for browser @shade/files
4.1.0's HTTP RPC for browsers capped at inline payloads (≤ 256 KiB).
4.2.0 unlocks streams: server queues outbound chunks + control
envelopes per peer, browser long-polls the queue. Browser-to-server
writes ride the existing /v1/transfer/<id>/chunk POST routes
unchanged.
For Dispatch this unlocks mod-jar uploads (50 MB) and world-backup
downloads (100+ MB) — the actual reason browser-side @shade/files
matters.
### New API
@shade/sdk:
- shade.transferQueueRoute(opts?) — Hono app with /queue +
/v1/transfer/* routes. Auto-configures the queue transport.
- shade.configureTransfers extended: transport + envelopeTransport
override slots; resolveBaseUrl optional when both supplied.
@shade/transfer:
- OutboundQueue — per-peer monotonic event log with long-poll
semantics, idle-eviction GC, ring-buffered to maxEventsPerPeer.
- QueueTransferTransport — enqueues instead of POSTing.
@shade/files:
- httpClient({ outboundQueueUrl, transferBaseUrl }) — when set,
starts a long-poll drainer + builds a streams-bridge. fs.read /
fs.write of >256 KiB work end-to-end.
- startQueueDrainer(shade, opts) — exported helper for advanced
consumers driving their own drainer.
### Implementation notes
- ClientStreamsBridge's TransformStream had HWM=0 by default which
stalled the drainer's await chain at chunk 4 (writer.write pended
before the consumer's reader was attached). Bumped to HWM=64 so
the receive loop can buffer ahead of the consumer.
### Tests
3 new integration tests in tests/integration/http-rpc-streams.test.ts:
4 MiB streamed read round-trip, inline-only error path, idle-timeout
long-poll behaviour.
Wire-compatible. Source-compatible. Lockstep bump to 4.2.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 70e319fef8 |
release(v4.0.1): strict-TS publishability fixes
4.0.0 shipped TypeScript source as published main/types, but several
files only compiled inside the monorepo. Consumer projects (Dispatch,
etc.) running their own strict tsc against our published source hit:
- @shade/key-transparency: 4 noUnusedLocals violations
(IndexAbsenceProof, IndexInclusionProof, IndexProofWire, nodeHash)
- @shade/sdk: KT verifier callbacks returned Promise<unknown> instead
of Promise<STHWire> / Promise<{ proof: string[] }>
- @shade/sdk: thumbnail.ts globalThis cast collided with consumer's
lib.dom-supplied createImageBitmap signature
- @shade/files: cycle with @shade/sdk produced "this is not assignable
to type 'Shade'" because hoisted node_modules layouts duplicated the
Shade class. Broken by replacing `import type { Shade }` with a
local structural ShadeBridge interface.
- @shade/storage-encrypted: KeyUsage (lib.dom) used under
lib: ["ES2022"]
- @shade/transport-bridge: ReadableStreamDefaultReader<any> ↔
<Uint8Array> mismatch
- @shade/keychain / @shade/dashboard / @shade/storage-encrypted
tsconfig rootDir / include hygiene
Tooling: scripts/typecheck-all.ts runs `bunx tsc --noEmit` against
every workspace package's tsconfig and fails on any error. Wired into
publish:dry / publish:all and publish-shade.sh as a hard gate so this
class of bug cannot recur.
All 24 packages bumped to 4.0.1 in lockstep.
Migration: <ShadeFilesProvider> now requires an explicit `files` prop
(pass `shade.files`). Wire format unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| e6fdf31b49 |
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers byte-for-byte. The version bump is semantic: audit-cycle complete, opt-in surface fully exposed, threat model refreshed for every new surface. Highlights: - All 24 @shade/* packages bumped to 4.0.0 in lockstep. - CHANGELOG 4.0.0 section is the canonical manifest of what landed. - THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12 Web-Worker boundary) + residual-risks table refreshed. - OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox, bridge, observer, /metrics, /healthz, /ready. - MIGRATION 0.3.x → 4.0 documented + smoke-tested against shade migrate-storage on a real SQLite DB. - docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer. - scripts/soak.ts harness for the GA-stable 2-week soak window. - All V*.md plans archived under docs/archive/ with Status: Done. - Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen non-realtime stack. Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green. Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports version 4.0.0 on /health. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| fa770d3063 |
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
Some checks failed
Test / test (push) Has been cancelled
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 467dd5b065 |
feat(advanced): M-Adv 1-3 — multi-device, backup/restore, group messaging
Some checks failed
Test / test (push) Has been cancelled
Phase D complete. Shade is now at parity with Signal libsignal's core
feature set.
M-Adv 1: Multi-device support (simplified Sesame)
- DeviceListManager tracks per-user device lists ("user:deviceId" addresses)
- fanOutEncrypt() sends one message to all known devices via independent
1:1 Double Ratchet sessions
- observeIncoming() auto-registers new devices from received messages
- JSON serialization for persistence
- userOfDevice/deviceIdOf address parsers
M-Adv 2: Backup and restore
- @shade/sdk exports BackupBlob format: version + salt + nonce + ciphertext
- Passphrase-derived key via HKDF (note: upgrade path to Argon2id documented)
- exportBackup()/importBackup() handle identity, prekeys, sessions, trust
- backupToString/backupFromString for single-string transport (copy/paste, QR)
- shade.exportBackup()/importBackup() convenience methods on SDK
- CLI: shade backup export <file> / shade backup restore <file>
- Rebuilds manager + transport after restore so ratchet state is consistent
M-Adv 3: Group messaging (Sender Keys)
- Per-sender chain key + Ed25519 signing key per group
- createSenderKey / buildDistribution / installDistribution for key distribution
- senderKeyEncrypt advances chain and signs ciphertext+header
- senderKeyDecrypt verifies signature then advances the sender's chain
- Out-of-order handling with bounded skip
- O(1) per message (once distributions are installed)
- Defensive ByteArray copies in distribution to prevent zeroize-across-refs
276 tests passing, 0 failures. All 13 SDK/tooling/platform/advanced
milestones complete. Shade is feature-complete for v2.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|||
| c95824f95f |
feat(sdk): M-Magic 1-4 — high-level SDK with magic drop-in
Phase A complete: createShade() one-liner with auto-establish, auto-publish,
and auto-replenish.
M-Magic 1-4 rolled into @shade/sdk:
- createShade() factory with config validation and storage resolution
(memory | sqlite:... | { type: 'postgres', url: ... } | explicit instance)
- Shade class wraps crypto + storage + session manager + transport
- Auto-publish: initialize() automatically registers with the prekey server
- Auto-establish: send() transparently fetches bundles and creates sessions
on first message to a new peer
- Per-address mutex serializes concurrent sends to prevent ratchet corruption
- BackgroundTasks class for periodic replenishment + opt-in identity rotation
- rotate() rebuilds the transport with the new signing key so subsequent
signed operations work after rotation
- onMessage() handler API for incoming plaintext
API:
const shade = await createShade({ prekeyServer, storage });
await shade.send('bob', 'hello');
await shade.receive('alice', envelope);
shade.onMessage((from, msg) => ...);
await shade.rotate();
await shade.shutdown();
13 new SDK tests covering: happy path, auto-publish, two-process
conversation, onMessage handlers, concurrent sends, unknown peer,
fingerprint verification, shutdown, manual replenish, auto-replenish
off, rotate, and config validation.
233 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
