publish-all.ts now does a tsc → dist/ build per package before pack, then
rewrites package.json's main/types/exports to point at the built artefacts
and ensures `files: ["dist"]` so the tarball ships only the built code.
The in-repo package.json is restored in the finally block so dev/typecheck
keep working without a build pass.
Why: strict-mode consumers (Cyndr) were forced to recompile Shade source
under their own tsconfig and tripped on internal `process.env.X` accesses
and implicit-any parameters. Shipping pre-built `.js` + `.d.ts` makes the
strictness contract live entirely inside Shade.
Builds on V4.9's encrypted profile blob: ships the canonical
profile-blob schema (hosts/clients/trustedApproverFingerprints) and
the build/sign/verify trio for proxy-approval frames. Headless servers
can now route a `linkRequest` to a trusted-approver phone, verify the
phone's Ed25519 signature against the fresh profile blob, and complete
pairing without a GUI host being available.
Length-prefixed binary signing payload so any platform (Kotlin, Swift,
Go) can produce byte-identical signing input from test vectors. No
relay or transport changes — entirely SDK-level.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships the Prism FR (encrypted-profile-storage-v4.9.md) as a generic
relay-side encrypted blob primitive: deterministically-located,
AEAD-sealed blobs keyed by a 32-byte slotId derived client-side via
HKDF from the user's master key. Unlocks credential-only bootstrap
of new devices into existing E2EE state — no QR, no physical access.
Server: BlobStore interface + Memory/Sqlite/Postgres impls,
createBlobRoutes for GET/PUT/DELETE /v1/blob/:slotId with TOFU pubkey
auth and If-Match CAS (409/412 semantics). Mounted on the same Hono
app as the inbox; SHADE_BLOB_PG_URL / SHADE_BLOB_DB_PATH /
SHADE_DISABLE_BLOB env-var plumbing in standalone.
SDK: createProfileNamespace high-level wrapper (HKDF derivation,
random-nonce AEAD seal, slotId-bound AAD) + low-level BlobClient.
Cross-platform test vectors in test-vectors/blob-storage.json.
New errors: ConflictError (409), PreconditionFailedError (412).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prism filed a per-recipient-flush-concurrency FR pointing at
serial-per-flush. Investigation surfaced the actual culprit:
`scheduleFlush` was using a 15 s backoff on **both** the success and
failure paths, so envelopes enqueued *during* an in-flight flush
sat ~15 s behind the next drain — visible as "10 s of silence then
25-frame burst" on the receiving side under sustained sender output.
Two fixes:
1. `scheduleFlush` now uses 0 ms delay when `flushOnce` delivered
≥1 envelope and more is queued (network healthy → drain
remainder immediately). 15 s reserved for the actual failure
case where every attempt this round failed. `flushOnce` returns
`{ delivered, remaining } | null` so concurrent-flush early
returns don't double-schedule.
2. `flushOnce` groups the outgoing queue by `recipientAddress` and
drains buckets via `Promise.all`. Per-peer order preserved
(sequential within a bucket); a slow POST to recipient A no
longer head-of-line-blocks frames bound for B.
`Inbox.tick` public shape unchanged. `OutgoingQueueStore`
implementations see the same per-entry list/remove/bumpAttempts/
size contract; only cross-recipient interleaving changes.
Tests cover (1) 25-envelope burst behind a 100 ms slow PUT drains
within 1 s, and (2) carol's PUT lands within 150 ms even when bob's
PUT stalls 200 ms.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V4.8.3 shipped client-side cross-channel dedup hook
(`Inbox.acceptBridgeFrame`), but recipients that didn't migrate to
the new wiring still observed the same envelope twice — once via
WS bridge push, again ~30 s later via inbox-poll. Prism re-verified
the FR after 4.8.3 and asked for a relay-side enforcement so app
code doesn't have to ack-via-DELETE on every bridge frame.
V4.8.4 adds an in-memory `BridgeDeliveryLog` (default 60 s grace,
8192-per-address cap) that records every successful WS / SSE /
long-poll push of `(address, msgId)`. The `/v1/inbox/:addr/fetch`
route filters out blobs in the log's grace window so a recipient
running both a bridge and the 30 s poll cadence sees exactly one
delivery. Cursor advances over the full fetched window so a poll
that straddles a suppressed blob doesn't stall.
The standalone server auto-wires the log between
`createBridgeRoutes` and `createInboxRoutes`. Custom mounts thread
the same instance through `bridgeDeliveryLog` on both factories.
Tests cover WS-then-poll, SSE-then-poll, and a negative control
(non-bridge-pushed blob still comes through inbox-fetch).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to the V4.8.2 duplicate-fan-out fixes Prism filed.
1. `Inbox.acceptBridgeFrame(blob)` + shared 4096-entry msgId LRU.
The relay durably stores blobs and pushes them to every active
delivery channel; without a cross-channel ack the bridge frame
ran first and the next inbox-poll re-dispatched the same blob
~30 s later, tripping on consumed prekeys. Bridge consumers now
plumb pushed frames through `acceptBridgeFrame`, which shares
the dedup gate + ack path with `pollOnce`. Whichever channel
delivers first wins; the other acks-and-skips. Inbox records
the msgId before the ack so a parallel poll can't observe an
in-flight ack window.
2. `Shade.aliasSession(oldLabel, newLabel)`. First-contact forces
the receiver to label the new session by the relay's sender
fingerprint hint (`fp:<senderfp>`); the post-decrypt plaintext
typically announces the peer's real address. Aliasing moves
session, trusted identity, peer-verification, and identity-
version under the canonical label. Holds the per-peer mutex on
both labels (lexicographic order) so concurrent crypto ops can't
observe a half-moved state. Refuses to overwrite an existing
session at the new label.
Wire change: `IncomingMessage.expiresAt?` now surfaces the relay's
expiry so receivers can pass bridge frames straight to
`acceptBridgeFrame` without inventing a TTL.
Tests cover bridge-then-poll, poll-then-bridge, aliasSession happy
path, refuse-to-overwrite, and same-label no-op.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two interlocking robustness fixes for the duplicate-fan-out / first-contact
class of failures Prism reported.
1. `Shade.receive(from, env)` now queues its `manager.decrypt` step
per `from` so concurrent dispatches can't race the SessionManager
ratchet or the StorageProvider (sqlite "database is locked", IDB
transaction conflicts). User message handlers run *outside* the
queue so streams + file-RPC's nested `shade.receive` calls don't
self-deadlock.
2. Bridge WS + SSE handlers now run a per-connection bounded msgId
LRU as defense-in-depth against any flushTo re-entry (event-storm,
future refactor). Pending-flush chains are wrapped in `.catch(() =>
{})` so a transient `ws.send` rejection no longer poisons the
connection's flush loop.
Tests: storming `inbox.blob_stored` 10× per PUT yields exactly one WS/
SSE frame; 8 concurrent `bob.receive('alice', envelope)` calls keep
the ratchet intact and never surface "database is locked".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plumbing fix only — both createPrekeyRoutes and createInboxRoutes
already accepted disableRateLimit; standalone.ts just didn't read
the env. Now SHADE_DISABLE_RATE_LIMIT=1 turns off IP rate-limits on
every prekey + inbox route, with a WARN log on startup so operators
see it.
Single-tenant deployments only — multi-tenant relays must leave it
unset. Documented in docs/DEPLOYMENT.md.
Reported by Prism: ~6 pair attempts/hour from a single dev IP +
the sidecar's register call tripped the 5/hour REGISTER_LIMIT every
dev iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two unblocking changes for first-contact flows.
Sender attribution: relay captures shortHash(senderSigningKey) at
PUT time (after signature verification, no new trust surface) and
surfaces it on bridge push (IncomingMessage.from) + inbox-fetch
(FetchedBlob.from) + DecryptHandler raw arg. Apps receiving a prekey
envelope from a never-before-seen peer can now bootstrap X3DH via
shade.receive('fp:<hex>', env) — pre-4.8 the wire envelope didn't
authenticate the sender and there was no out-of-band hint to use.
Idempotent ALTER TABLE migrations for SQLite + Postgres add a
sender_fp TEXT column; legacy rows surface as from=undefined
(inter-version compat).
Inbox.start() race: pre-4.8 start() called register() fire-and-forget
AND schedulePoll(0) synchronously, so the first poll on a fresh
address often beat the register HTTP RTT and got SHADE_NOT_FOUND.
start() now defers; register() success kicks schedulePoll(0). Manual
tick() is unaffected (deliberate user action, no gating).
Both reported by Prism. Tests cover all five acceptance criteria
from the sender-attribution request (PUT capture, bridge surface,
fetch surface, inter-version compat, end-to-end pair smoke) plus
the three from the race-fix request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the bridge-connection-lifecycle signal that closes Prism's
~45s revoke window down to one server→client round-trip (~50ms).
Server (`@shade/inbox-server`):
- `inbox.peer_connected` / `inbox.peer_disconnected` events on the
0↔1 boundary across WS + SSE bridges. Long-poll deliberately not
tracked (every poll boundary would flap; push transports are also
the only ones where instant revoke matters).
- `PresenceTracker` collapses two parallel bridges (e.g. WS + SSE
during fallback handover) into one connect/disconnect pair.
- `GET /v1/bridge/presence` SSE endpoint: signed query with
`kind: 'presence'`, `watched: string[]`; on open streams a
per-address snapshot, then change frames filtered server-side.
MAX_WATCHED_ADDRESSES = 64. Subscribing does not itself count as
a peer-bridge connection.
- `createBridgeRoutes` now returns `{ app, websocket, presence }`.
Client (`@shade/transport-bridge`):
- `PresenceBridge.subscribe({ watch, onPresenceChange })` →
`{ addPeer, removePeer, watching, unsubscribe }`. addPeer/removePeer
mutate via reconnect with a fresh signed query.
- `signPresenceQuery` helper for non-PresenceBridge consumers.
Tests cover all four acceptance criteria from the Prism request:
server-event smoke, online→offline subscription, address scoping
(carol invisible to a [alice]-only sub), reconnect, plus an
addPeer/removePeer regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Browsers' Window.fetch is a WebIDL bound operation; storing it as
this.fetchImpl / this.fetchFn and calling via the instance receiver
threw "Illegal invocation" on the first request. Bind once at
construction in InboxClient, LongPollBridge, and SseBridge. Reported
by Prism (multi-device E2EE terminal), blocking every browser
consumer of the v4.6 transport stack on inbox.start() / bridge.connect().
WsBridge unaffected (uses WebSocket). Node/Bun fetch tolerates a free
receiver, so the bug never surfaced server-side — added regression
tests that install a strict-receiver globalThis.fetch to catch the
issue without an actual browser harness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the broadcast-channel primitive Prism asked for in
Docs/shade-feature-request-sender-keys.md. The crypto in
@shade/core/sender-keys.ts was already in place; this release wires
it up as a first-class app-facing API, adds the persistence schema
across all six storage backends (memory, sqlite, indexeddb +
encrypted variants), introduces wire type 0x21 in @shade/proto,
and ships Prism's three acceptance tests verbatim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the foundations Prism's web client (and any future browser-based
Shade app) needs: at-rest-encrypted IndexedDB storage that mirrors the
SQLite backend byte-for-byte at the AAD/nonce level, browser-safe
subpath imports so Vite/webpack/esbuild stop hitting bun:sqlite, and
KeyManager support for argon2id and N-factor composite unlock.
@shade/storage-encrypted
- EncryptedIndexedDBStorage (subpath: /idb) — full StorageProvider
using one object store per _enc table; reuses aeadSeal/aeadOpen +
row-codec sealers so a row sealed under the SQLite or Postgres
backend decrypts under IDB given the same KeyManager.
bumpPeerIdentityVersion is atomic under one IDB transaction.
- KeyManager argon2id source — memory-hard KDF for low-entropy
secrets (PINs). Backed by @noble/hashes/argon2 (already a transitive
dep). DEFAULT_ARGON2ID exported (m=64 MiB, t=3, p=1).
- KeyManager composite source — HKDF-combine N sub-sources into one
master. Every source mandatory; order significant by design;
composite-of-composite rejected; optional info string for app-level
domain separation.
- Subpath exports (/crypto, /sqlite, /postgres, /idb) plus a `browser`
condition on the default import that resolves to a barrel
excluding the Bun- and Postgres-specific entries. Browser bundles
no longer pull bun:sqlite transitively.
Tests
- 73 tests in shade-storage-encrypted (was 31). New coverage:
argon2id determinism + reject paths, composite same-factors → same
master, wrong-PIN/passphrase/order-swap → different master, info
domain separation, all 28 StorageProvider methods on
EncryptedIndexedDBStorage, fingerprint-mismatch rejection, and
cross-impl roundtrip with EncryptedSQLiteStorage proving the AAD/
nonce derivation is implementation-agnostic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expose the local device's 32-byte Ed25519 identity public key on Shade
so apps can hand it to their own backend at enrollment time for
signature verification, key pinning or per-device safety-number
computation. Closes the gap that forced consumers to ship placeholder
random bytes their backend could store but never verify against.
- @shade/sdk Shade.identityPublicKey: Promise<Uint8Array> — getter
mirrors the existing fingerprint accessor. Throws pre-init,
reflects the current key after rotate(), retired key preserved in
retired-identities storage per existing grace-period contract.
Private key remains unreachable.
- Test in shade-sdk/tests/sdk.test.ts: round-trip match against the
underlying storage's signingPublicKey, plus value updates after
rotate().
- Lockstep version bump 4.3.0 → 4.4.0 across all 25 packages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ship an official IndexedDB-backed StorageProvider so browser-based Shade
consumers persist identity, prekeys, sessions, retired identities,
peer-verification state and stream-resume rows across tab refresh and
browser restart. Closes the gap that forced browser apps onto
storage:"memory" (regenerated identity each load, orphaned device
records server-side).
- New package @shade/storage-indexeddb (4.3.0): full StorageProvider
conformance, schema v1, idb-backed; bumpPeerIdentityVersion is wrapped
in a single readwrite IDB transaction (atomic, vs SQLite's
read-then-upsert race).
- @shade/sdk resolveStorage() accepts { type: 'indexeddb', dbName? } via
dynamic import (lazy, optional dep — same pattern as
@shade/storage-postgres). Named StorageSpec type now reused by
ResolvedConfig.
- Tests: 16 new tests in shade-storage-indexeddb (StorageProvider
surface + peer-verifications + full E2EE conversation surviving a
simulated tab reload). Run on fake-indexeddb.
- Lockstep version bump 4.2.1 → 4.3.0 across all 25 packages.
- Publish scripts updated to include the new package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pull-mode httpClient + drainer + parallel RPCs against the same peer
deteriorated after ~10s with `DecryptionError`. Two bugs combined:
- `OutboundQueue.enqueue` woke `drain` waiters with a `since=0`
snapshot, replaying already-processed events into
`Shade.acceptTransferEnvelope` → `manager.decrypt` twice. The
duplicate consumed an already-used skipped key and corrupted the
Double Ratchet receive chain.
- `ratchetDecrypt` then propagated the corruption: a same-DH
message behind the chain with no cached skipped key fell through
to `kdfChainKey` on the ahead state and rewound `chain.counter`,
permanently desyncing the chain.
Fix `OutboundQueue` to honor each waiter's `since`, and harden
`ratchetDecrypt` so any future duplicate fails cleanly without
mutating state. Adds regression coverage at all three layers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.1.0's HTTP RPC for browsers capped at inline payloads (≤ 256 KiB).
4.2.0 unlocks streams: server queues outbound chunks + control
envelopes per peer, browser long-polls the queue. Browser-to-server
writes ride the existing /v1/transfer/<id>/chunk POST routes
unchanged.
For Dispatch this unlocks mod-jar uploads (50 MB) and world-backup
downloads (100+ MB) — the actual reason browser-side @shade/files
matters.
### New API
@shade/sdk:
- shade.transferQueueRoute(opts?) — Hono app with /queue +
/v1/transfer/* routes. Auto-configures the queue transport.
- shade.configureTransfers extended: transport + envelopeTransport
override slots; resolveBaseUrl optional when both supplied.
@shade/transfer:
- OutboundQueue — per-peer monotonic event log with long-poll
semantics, idle-eviction GC, ring-buffered to maxEventsPerPeer.
- QueueTransferTransport — enqueues instead of POSTing.
@shade/files:
- httpClient({ outboundQueueUrl, transferBaseUrl }) — when set,
starts a long-poll drainer + builds a streams-bridge. fs.read /
fs.write of >256 KiB work end-to-end.
- startQueueDrainer(shade, opts) — exported helper for advanced
consumers driving their own drainer.
### Implementation notes
- ClientStreamsBridge's TransformStream had HWM=0 by default which
stalled the drainer's await chain at chunk 4 (writer.write pended
before the consumer's reader was attached). Bumped to HWM=64 so
the receive loop can buffer ahead of the consumer.
### Tests
3 new integration tests in tests/integration/http-rpc-streams.test.ts:
4 MiB streamed read round-trip, inline-only error path, idle-timeout
long-poll behaviour.
Wire-compatible. Source-compatible. Lockstep bump to 4.2.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default shade.files.client(peer) requires both peers to be mutually
addressable over HTTP — the response round-trips through
Shade.deliverControlEnvelope (POST to peer's /v1/transfer/control).
Browser tabs can't host an HTTP server, so they couldn't consume
@shade/files at all. Dispatch's filutforsker (admin-panel browser UI)
is the canonical use-case.
This release adds a parallel request-response transport: one POST per
RPC, encrypted envelope in the body, encrypted response in the same
HTTP response. No inbound channel needed on the client.
### New API
- shade.files.rpcRoute(opts?) — Hono app exposing POST /rpc.
- shade.files.httpClient(peer, opts) — request-response FileClient.
- FilesNamespace.serve(handler, { inlineOnly: true }) — skip streams-
bridge (and its configureTransfers pre-condition); also skip
channel-based dispatch so requests aren't double-dispatched.
### Limitations (v1)
Inline only (≤ 256 KiB). Streamed reads/writes throw clear errors
directing to shade.files.client(peer) on a server-to-server deploy.
### Tests
7 integration tests in tests/integration/http-rpc.test.ts covering
round-trip + negative cases (sender header, empty/garbage body,
maxBodyBytes, rpcRoute-without-serve).
### Symmetry
Mirrors @shade/server's shade-auth-middleware: encrypted envelope in
request body, decrypted via existing ratchet, response in same HTTP
roundtrip. No WebSocket, no SSE, no outbound from server.
Wire-compatible. Source-compatible. Lockstep bump to 4.1.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.0.1's typecheck gate compiled each package internally against
lib: ["ES2022"]. That doesn't catch types that only fail when
*consumer* code (lib: ["DOM"] + exactOptionalPropertyTypes) tries to
assign a native browser type into one of our locally-defined narrower
types. Dispatch hit one such case in @shade/files inline-threshold.ts.
This release adds a tests/consumer-strict/ smoke project to the
pre-publish gate. It compiles a tiny "as if I were a downstream app"
TS file against:
lib: ["ES2022", "DOM", "DOM.Iterable"]
types: ["bun-types"]
exactOptionalPropertyTypes: true
strict: true
paths → packages/*/src/index.ts
scripts/typecheck-all.ts now runs the smoke after per-package checks.
Both must pass before publish:dry / publish:all proceeds.
### Fixed
- @shade/files inline-threshold.ts: MinimalReader<T> rewritten as the
explicit disjoint union { done:false, value:T } | { done:true,
value?: T | undefined } that's assignable from every native reader
shape (bun, DOM, node:stream/web). Fixes the
"ReadableStreamReadResult is not assignable" Dispatch reported.
- @shade/files streams-bridge (client + server): stash setTimeout
return in a local before .unref?.() via { unref?: () => void } cast.
Fluent .unref?.() failed under lib: ["DOM"] (setTimeout returns
number there).
- @shade/sdk background.ts: same setInterval .unref?.() fix.
Wire-compatible. No API shape changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.0.0 shipped TypeScript source as published main/types, but several
files only compiled inside the monorepo. Consumer projects (Dispatch,
etc.) running their own strict tsc against our published source hit:
- @shade/key-transparency: 4 noUnusedLocals violations
(IndexAbsenceProof, IndexInclusionProof, IndexProofWire, nodeHash)
- @shade/sdk: KT verifier callbacks returned Promise<unknown> instead
of Promise<STHWire> / Promise<{ proof: string[] }>
- @shade/sdk: thumbnail.ts globalThis cast collided with consumer's
lib.dom-supplied createImageBitmap signature
- @shade/files: cycle with @shade/sdk produced "this is not assignable
to type 'Shade'" because hoisted node_modules layouts duplicated the
Shade class. Broken by replacing `import type { Shade }` with a
local structural ShadeBridge interface.
- @shade/storage-encrypted: KeyUsage (lib.dom) used under
lib: ["ES2022"]
- @shade/transport-bridge: ReadableStreamDefaultReader<any> ↔
<Uint8Array> mismatch
- @shade/keychain / @shade/dashboard / @shade/storage-encrypted
tsconfig rootDir / include hygiene
Tooling: scripts/typecheck-all.ts runs `bunx tsc --noEmit` against
every workspace package's tsconfig and fails on any error. Wired into
publish:dry / publish:all and publish-shade.sh as a hard gate so this
class of bug cannot recur.
All 24 packages bumped to 4.0.1 in lockstep.
Migration: <ShadeFilesProvider> now requires an explicit `files` prop
(pass `shade.files`). Wire format unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V4.0 acceptance §"All docs/V*.md arkivert med DONE-status" requires
every archived plan to carry an explicit Status field. V2.1 / V2.2 /
V2.3 inherited their pre-status format; V3.12-DESIGN was still
"Approved". Mark all four as Done with a one-line pointer to where
the work actually landed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
publish-shade.sh used to call `bun run publish:all`, which in turn was
wired to call publish-shade.sh (after the V4.0 cleanup). Point it
directly at scripts/publish-all.ts so the interactive flow runs the
TS publisher without re-entering itself.
Verified: dry-run from publish-shade.sh now packs all 24 @shade/*@4.0.0
packages cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Top-of-file callout for the 0.2.x → 0.3.0 wire incompatibility.
- "What you get" picks up E2EE file transfers + filesystem RPC bullets.
- Quick start gains a `Shade.files.serve()` / `.client()` snippet.
- Packages table adds @shade/streams, @shade/transfer, @shade/files;
@shade/sdk and @shade/widgets descriptions extended for the new APIs.
- Documentation index links docs/files.md, docs/streams.md, examples/07
and 08.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
publish-shade.sh's PACKAGES list was missing shade-streams, shade-transfer,
and shade-files — so the conflict-check + auto-bump loop silently skipped
those three packages. The final `bun run publish:all` step still picked
them up (via scripts/publish-all.ts), but a re-publish on a version that
already existed in the registry would not have bumped the missing
packages, leaving a workspace-wide version mismatch.
Also un-gitignore both publish scripts. publish-all.ts was already tracked
(git overrides ignore for tracked files), and publish-shade.sh contains
no secrets at rest — GITEA_TOKEN is read interactively and stored in
~/.bashrc by the script itself.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shade now ships as a self-contained Docker image. Deploy one container
per project, any stack (Bun, Python, Go, Rust, Kotlin) can talk to it via
plain HTTP. Zero coupling to consumer codebases.
M-Box 1: Stale identity cleanup API
- touchIdentity + purgeStaleIdentities on PrekeyStore interface
- Implemented for Memory, SQLite, and Postgres backends
- SQLite adds last_activity_at column with migration ALTER for existing DBs
- Postgres adds the same via raw SQL with IF NOT EXISTS guards
- Routes call touchIdentity on register, bundle fetch, replenish
- 4 new tests for the cleanup API
M-Box 2: Stale cleanup background task
- StaleCleanupTask runs purge on startup + every 24h (configurable)
- Reads SHADE_STALE_DAYS (default 30) and SHADE_CLEANUP_INTERVAL_HOURS
- Wired into standalone.ts, stopped on graceful shutdown
- 5 new tests for the task
M-Box 3: Observer baked into the container
- standalone.ts conditionally mounts @shade/observer at /shade-observer
when SHADE_OBSERVER_TOKEN is set (and >= 16 chars)
- Shared PrekeyServerEvents emitter feeds both routes and observer
- @shade/observer added as optional dependency of @shade/server
M-Box 4: Dockerfile with dashboard build
- Multi-stage build: oven/bun:1 builder → oven/bun:1-alpine runtime
- COPY packages/ wholesale so workspace lockfile resolves cleanly
- RUN bun run build inside shade-dashboard → dist/ → observer/dist/
- Non-root shade user, /data volume, healthcheck, env defaults
- Final image: 260 MB
M-Box 5: OpenAPI spec for stack-agnostic clients
- packages/shade-server/openapi.yaml documents all 9 endpoints with
request/response schemas, security (Ed25519 signatures + bearer token)
- createOpenApiRoutes serves /openapi.yaml and /docs (Redoc viewer)
- Any language can generate a client with openapi-generator
M-Box 6: Docker CI pipeline
- .gitea/workflows/docker.yml builds + pushes on git tag v*
- scripts/build-docker.ts for local builds, supports --push with GITEA_TOKEN
- Root package.json: build:docker, publish:docker scripts
M-Box 7: Deployment documentation
- packages/shade-server/README rewritten: 5-line quickstart with the image
- docs/DEPLOYMENT.md: full reference, env vars, backup, Dokploy, PG setup
- examples/05-dokploy-deployment/docker-compose.yml updated to pull
published image (gt.zyon.no/stian/shade-prekey:latest)
- Root README deployment section rewritten
M-Box 8: End-to-end verification
- Image builds locally (bun run build:docker)
- /health, /openapi.yaml, /docs, /metrics, /shade-observer all respond
- 401 without observer token, 200 with
- Real SDK client round-trip: Alice → container → Bob → reply → Alice
- Persistence: identity + prekeys survive container restart (count 20→18
as expected from two bundle fetches)
285 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase D complete. Shade is now at parity with Signal libsignal's core
feature set.
M-Adv 1: Multi-device support (simplified Sesame)
- DeviceListManager tracks per-user device lists ("user:deviceId" addresses)
- fanOutEncrypt() sends one message to all known devices via independent
1:1 Double Ratchet sessions
- observeIncoming() auto-registers new devices from received messages
- JSON serialization for persistence
- userOfDevice/deviceIdOf address parsers
M-Adv 2: Backup and restore
- @shade/sdk exports BackupBlob format: version + salt + nonce + ciphertext
- Passphrase-derived key via HKDF (note: upgrade path to Argon2id documented)
- exportBackup()/importBackup() handle identity, prekeys, sessions, trust
- backupToString/backupFromString for single-string transport (copy/paste, QR)
- shade.exportBackup()/importBackup() convenience methods on SDK
- CLI: shade backup export <file> / shade backup restore <file>
- Rebuilds manager + transport after restore so ratchet state is consistent
M-Adv 3: Group messaging (Sender Keys)
- Per-sender chain key + Ed25519 signing key per group
- createSenderKey / buildDistribution / installDistribution for key distribution
- senderKeyEncrypt advances chain and signs ciphertext+header
- senderKeyDecrypt verifies signature then advances the sender's chain
- Out-of-order handling with bounded skip
- O(1) per message (once distributions are installed)
- Defensive ByteArray copies in distribution to prevent zeroize-across-refs
276 tests passing, 0 failures. All 13 SDK/tooling/platform/advanced
milestones complete. Shade is feature-complete for v2.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase B complete: Shade now has a full developer tooling story.
@shade/cli
- shade init with project scaffolding from templates
- shade fingerprint (own or peer)
- shade publish (re-upload bundle)
- shade rotate (--identity for full rotation, otherwise signed prekey)
- shade peer add/list/verify/remove
- shade dashboard (opens observer in browser)
- shade doctor (diagnose config, storage, prekey server reachability)
- Config from .shaderc.json or SHADE_* env vars
Templates (in packages/shade-cli/templates/)
- bun-server — Bun + Hono backend with /send + /receive endpoints
- chat-demo — Two-process Alice/Bob chat over HTTP
Publishing pipeline (Gitea npm registry)
- .gitea/workflows/test.yml — CI on push/PR with PostgreSQL service
- .gitea/workflows/publish.yml — publish on git tag v*
- scripts/publish-all.ts — local publish helper with DRY_RUN support
- scripts/bump-version.ts — lockstep version bump across all packages
- Root package.json scripts: version, publish:dry, publish:all
Also: /health endpoint now lives in createPrekeyRoutes so doctor can
probe it without needing the full standalone setup.
Dry-run verified: all 11 packages pack cleanly.
246 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
M-Obs 4: @shade/widgets React library
- ShadeProvider context with observer URL + token + theme
- useShadeState (polling) + useShadeEvents (SSE) hooks
- 7 widgets: IdentityCard, SessionList, PrekeyStock, RecentActivity,
ServerStatus, FingerprintCompare, WidgetCatalog (meta-widget for
user-selectable layout with localStorage persistence)
- Self-contained CSS via inline styles, no external CSS conflicts
- Light/dark/auto theme via tokens
M-Obs 5: @shade/dashboard standalone SPA
- Vite + React app composing all widgets into a full debugger layout
- Login screen with token persistence to localStorage
- Build script copies dist/ to @shade/observer/dist/ for embedded serving
- 211 KB JS bundle (66 KB gzipped)
M-Obs 6: Documentation + integration example
- READMEs for @shade/observer and @shade/widgets
- examples/06-observer-dashboard runnable demo: spins up prekey server +
observer, runs Alice ↔ Bob conversation loop, dashboard at :3901
- Updated root README and docker-compose.yml with observer integration
M-Obs 7: End-to-end verification
- StateAggregator now drains buffered events on subscription, so
identity.initialized fires before observer construction are still seen
- Verified live: snapshot endpoint returns full state, dashboard serves,
401 without auth, sessions/messages/ratchet steps all tracked
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
M-Hard 6: PostgreSQL Storage Backend
- @shade/storage-postgres with PostgresStorage + PostgresPrekeyStore
- Drizzle-style raw SQL ensureClientTables / ensurePrekeyServerTables
- All tables prefixed `shade_` to avoid collisions in shared databases
- DELETE ... FOR UPDATE SKIP LOCKED for concurrent OTPK consumption
- Tests skip gracefully without SHADE_TEST_PG_URL, run against real PG when set
M-Hard 7: Production Server Infrastructure
- Structured JSON logger (logger.ts) — SHADE_LOG_LEVEL configurable
- Health endpoints (/health, /healthz, /ready) — Kubernetes-friendly
- Prometheus metrics (/metrics) — counters, histograms, middleware
- Graceful shutdown with SIGTERM/SIGINT handlers + store close
- Production Dockerfile with non-root user, healthcheck, multi-stage build
- docker-compose.yml example for Dokploy with optional PostgreSQL
193 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shade sessions and keys now survive server crashes, container restarts,
and power outages via SQLite with WAL mode.
New packages:
- @shade/storage-sqlite: SQLiteStorage (StorageProvider) + SqlitePrekeyStore
(PrekeyStore), both using bun:sqlite with auto-created tables and WAL mode
- Serialization layer in shade-core for SessionState/keys ↔ JSON/base64
Docker usage: mount volume at /data, set SHADE_DB_PATH=/data/shade-client.db
Prekey server auto-detects SHADE_PREKEY_DB_PATH for SQLite persistence
Includes crash recovery integration test: encrypt → close DB → reopen →
conversation continues seamlessly.
129 tests, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ShadeFetchTransport: HTTP client for prekey server
(register, fetchBundle, replenish, getKeyCount)
- ShadeWebSocket: wraps existing WebSocket with auto E2EE
(binary wire format, transparent encrypt/decrypt)
- Full integration test: register → fetch → session → encrypt → decrypt
over real HTTP against in-process Hono prekey server
101 tests, 0 failures across all milestones (M1-M7).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
