Tørrkjøringen fanget to ting: vault manglet i PACKAGES (så @shade/server
feilet på en ukjent workspace-referanse), og jeg hadde innført en syklus
storage-sqlite → vault → server → storage-sqlite. Den siste er løst ved at
storage-sqlite bare bruker VaultStore som TYPE — flyttet til devDependencies,
så det ikke er en runtime-avhengighet.
vault står ved siden av inbox-server i lista: begge er i en syklus med
server (de trenger verifyPayload, standalone.ts mounter rutene deres), og
lista håndterer det allerede ved å liste server først.
publish:dry: 26 pakket, 0 feilet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shade kunne flytte filer mellom peers (@shade/files) og lagre én liten
profil-blob per konto, men hadde ingen alltid-på lagring av krypterte filer.
Uten den kan ingen Shade-app tilby backup, og ingen klient lese data mens
peeren som eier dem er avslått.
Objekter er innholdsadresserte på hashen av CHIFFERTEKSTEN, så relayen kan
lagre, deduplisere og verifisere uten nøkkel — den regner om hashen ved
opplasting og avviser feilnavngitte objekter. Stier bor inne i det krypterte
manifestet, aldri i objektnavn: relayen skal ikke lære hva filene heter.
Loggen er append-only, så historikk og rollback følger av modellen.
SqliteVaultStore har med vilje INGEN minne-fallback, i motsetning til
blob-storen. Den fallbacken slettet Prisms profil ved en rutine-redeploy
2026-08-12 fordi den fungerte helt til containeren ble recreated, uten en
eneste feilmelding. En backup som glemmer er verre enn ingen backup, så uten
SHADE_VAULT_DB_PATH mountes rutene ikke — med en logglinje som sier hvorfor.
Én feil fanget av testene: pubkeyen ble først lagt på UTENFOR signaturen,
som både brøt verifyPayload og ville latt hvem som helst bytte identitet i
transit på den TOFU-pinnende førsteskrivingen.
16 vault- + 7 store-tester, alle mot de ekte rutehåndtererne gjennom Honos
fetch. Kjeden er dessuten kjørt mot en ekte HTTP-server med et ekte
workspace: 491 filer / 7,7 MB, alle bit-identiske etter gjenoppretting,
og andre push etter én endring sendte 0 KB.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
aliasSession knew that two labels name the same peer, then threw that
knowledge away. The binding lived only in the caller's memory, so a
restart lost it — and the peer could not repair it from its side.
First contact forces the receiver to label a session by the only sender
hint a relay surfaces, an 8-byte signing-key fingerprint (`fp:<hex>`).
Once the peer announces its canonical address, aliasSession moves the
session there. But the peer keeps sending under `fp:<hex>`, because its
transport derives the same label from the same hint every time. After a
restart the session sat under the canonical address, inbound frames
resolved to `fp:<hex>`, and nothing matched. The peer held a valid
session so it never re-ran X3DH: the failure was permanent, and only a
manual re-link cleared it.
Observed in Prism as `No session for address: fp:579c3b335d66e2c0` on
every receive for three days, with a phone whose every RPC timed out.
StorageProvider gains saveSessionAlias / getSessionAlias /
removeSessionAliasesFor, optional so third-party implementations keep
compiling, and implemented across all seven backends. Lookups resolve
through resolveLabel(), which runs BEFORE the peer mutex — locking the
alias while mutating the canonical session would let an aliased and a
canonical caller ratchet the same state concurrently.
A live session under a label always wins over an alias, and prekey
envelopes never resolve: both keep a re-link establishing a fresh
session instead of being redirected into the stale one. Aliases are
dropped in resetSession and acceptIdentityChange, and memoized so the
hot path costs no extra read.
The sdk.test.ts case that asserted a dead fp-label encoded the old
behaviour; it now pins the new contract.
Verified: 1166 tests pass (from 1160). With alias persistence disabled
as a negative control, 5 of the 6 new tests fail, including both
restart cases.
Also drops `baseUrl` from the consumer-strict tsconfig — removed in
TS 6.0, and it was failing the typecheck that gates publishing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
publish-all.ts now does a tsc → dist/ build per package before pack, then
rewrites package.json's main/types/exports to point at the built artefacts
and ensures `files: ["dist"]` so the tarball ships only the built code.
The in-repo package.json is restored in the finally block so dev/typecheck
keep working without a build pass.
Why: strict-mode consumers (Cyndr) were forced to recompile Shade source
under their own tsconfig and tripped on internal `process.env.X` accesses
and implicit-any parameters. Shipping pre-built `.js` + `.d.ts` makes the
strictness contract live entirely inside Shade.
Builds on V4.9's encrypted profile blob: ships the canonical
profile-blob schema (hosts/clients/trustedApproverFingerprints) and
the build/sign/verify trio for proxy-approval frames. Headless servers
can now route a `linkRequest` to a trusted-approver phone, verify the
phone's Ed25519 signature against the fresh profile blob, and complete
pairing without a GUI host being available.
Length-prefixed binary signing payload so any platform (Kotlin, Swift,
Go) can produce byte-identical signing input from test vectors. No
relay or transport changes — entirely SDK-level.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships the Prism FR (encrypted-profile-storage-v4.9.md) as a generic
relay-side encrypted blob primitive: deterministically-located,
AEAD-sealed blobs keyed by a 32-byte slotId derived client-side via
HKDF from the user's master key. Unlocks credential-only bootstrap
of new devices into existing E2EE state — no QR, no physical access.
Server: BlobStore interface + Memory/Sqlite/Postgres impls,
createBlobRoutes for GET/PUT/DELETE /v1/blob/:slotId with TOFU pubkey
auth and If-Match CAS (409/412 semantics). Mounted on the same Hono
app as the inbox; SHADE_BLOB_PG_URL / SHADE_BLOB_DB_PATH /
SHADE_DISABLE_BLOB env-var plumbing in standalone.
SDK: createProfileNamespace high-level wrapper (HKDF derivation,
random-nonce AEAD seal, slotId-bound AAD) + low-level BlobClient.
Cross-platform test vectors in test-vectors/blob-storage.json.
New errors: ConflictError (409), PreconditionFailedError (412).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prism filed a per-recipient-flush-concurrency FR pointing at
serial-per-flush. Investigation surfaced the actual culprit:
`scheduleFlush` was using a 15 s backoff on **both** the success and
failure paths, so envelopes enqueued *during* an in-flight flush
sat ~15 s behind the next drain — visible as "10 s of silence then
25-frame burst" on the receiving side under sustained sender output.
Two fixes:
1. `scheduleFlush` now uses 0 ms delay when `flushOnce` delivered
≥1 envelope and more is queued (network healthy → drain
remainder immediately). 15 s reserved for the actual failure
case where every attempt this round failed. `flushOnce` returns
`{ delivered, remaining } | null` so concurrent-flush early
returns don't double-schedule.
2. `flushOnce` groups the outgoing queue by `recipientAddress` and
drains buckets via `Promise.all`. Per-peer order preserved
(sequential within a bucket); a slow POST to recipient A no
longer head-of-line-blocks frames bound for B.
`Inbox.tick` public shape unchanged. `OutgoingQueueStore`
implementations see the same per-entry list/remove/bumpAttempts/
size contract; only cross-recipient interleaving changes.
Tests cover (1) 25-envelope burst behind a 100 ms slow PUT drains
within 1 s, and (2) carol's PUT lands within 150 ms even when bob's
PUT stalls 200 ms.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V4.8.3 shipped client-side cross-channel dedup hook
(`Inbox.acceptBridgeFrame`), but recipients that didn't migrate to
the new wiring still observed the same envelope twice — once via
WS bridge push, again ~30 s later via inbox-poll. Prism re-verified
the FR after 4.8.3 and asked for a relay-side enforcement so app
code doesn't have to ack-via-DELETE on every bridge frame.
V4.8.4 adds an in-memory `BridgeDeliveryLog` (default 60 s grace,
8192-per-address cap) that records every successful WS / SSE /
long-poll push of `(address, msgId)`. The `/v1/inbox/:addr/fetch`
route filters out blobs in the log's grace window so a recipient
running both a bridge and the 30 s poll cadence sees exactly one
delivery. Cursor advances over the full fetched window so a poll
that straddles a suppressed blob doesn't stall.
The standalone server auto-wires the log between
`createBridgeRoutes` and `createInboxRoutes`. Custom mounts thread
the same instance through `bridgeDeliveryLog` on both factories.
Tests cover WS-then-poll, SSE-then-poll, and a negative control
(non-bridge-pushed blob still comes through inbox-fetch).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to the V4.8.2 duplicate-fan-out fixes Prism filed.
1. `Inbox.acceptBridgeFrame(blob)` + shared 4096-entry msgId LRU.
The relay durably stores blobs and pushes them to every active
delivery channel; without a cross-channel ack the bridge frame
ran first and the next inbox-poll re-dispatched the same blob
~30 s later, tripping on consumed prekeys. Bridge consumers now
plumb pushed frames through `acceptBridgeFrame`, which shares
the dedup gate + ack path with `pollOnce`. Whichever channel
delivers first wins; the other acks-and-skips. Inbox records
the msgId before the ack so a parallel poll can't observe an
in-flight ack window.
2. `Shade.aliasSession(oldLabel, newLabel)`. First-contact forces
the receiver to label the new session by the relay's sender
fingerprint hint (`fp:<senderfp>`); the post-decrypt plaintext
typically announces the peer's real address. Aliasing moves
session, trusted identity, peer-verification, and identity-
version under the canonical label. Holds the per-peer mutex on
both labels (lexicographic order) so concurrent crypto ops can't
observe a half-moved state. Refuses to overwrite an existing
session at the new label.
Wire change: `IncomingMessage.expiresAt?` now surfaces the relay's
expiry so receivers can pass bridge frames straight to
`acceptBridgeFrame` without inventing a TTL.
Tests cover bridge-then-poll, poll-then-bridge, aliasSession happy
path, refuse-to-overwrite, and same-label no-op.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two interlocking robustness fixes for the duplicate-fan-out / first-contact
class of failures Prism reported.
1. `Shade.receive(from, env)` now queues its `manager.decrypt` step
per `from` so concurrent dispatches can't race the SessionManager
ratchet or the StorageProvider (sqlite "database is locked", IDB
transaction conflicts). User message handlers run *outside* the
queue so streams + file-RPC's nested `shade.receive` calls don't
self-deadlock.
2. Bridge WS + SSE handlers now run a per-connection bounded msgId
LRU as defense-in-depth against any flushTo re-entry (event-storm,
future refactor). Pending-flush chains are wrapped in `.catch(() =>
{})` so a transient `ws.send` rejection no longer poisons the
connection's flush loop.
Tests: storming `inbox.blob_stored` 10× per PUT yields exactly one WS/
SSE frame; 8 concurrent `bob.receive('alice', envelope)` calls keep
the ratchet intact and never surface "database is locked".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plumbing fix only — both createPrekeyRoutes and createInboxRoutes
already accepted disableRateLimit; standalone.ts just didn't read
the env. Now SHADE_DISABLE_RATE_LIMIT=1 turns off IP rate-limits on
every prekey + inbox route, with a WARN log on startup so operators
see it.
Single-tenant deployments only — multi-tenant relays must leave it
unset. Documented in docs/DEPLOYMENT.md.
Reported by Prism: ~6 pair attempts/hour from a single dev IP +
the sidecar's register call tripped the 5/hour REGISTER_LIMIT every
dev iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two unblocking changes for first-contact flows.
Sender attribution: relay captures shortHash(senderSigningKey) at
PUT time (after signature verification, no new trust surface) and
surfaces it on bridge push (IncomingMessage.from) + inbox-fetch
(FetchedBlob.from) + DecryptHandler raw arg. Apps receiving a prekey
envelope from a never-before-seen peer can now bootstrap X3DH via
shade.receive('fp:<hex>', env) — pre-4.8 the wire envelope didn't
authenticate the sender and there was no out-of-band hint to use.
Idempotent ALTER TABLE migrations for SQLite + Postgres add a
sender_fp TEXT column; legacy rows surface as from=undefined
(inter-version compat).
Inbox.start() race: pre-4.8 start() called register() fire-and-forget
AND schedulePoll(0) synchronously, so the first poll on a fresh
address often beat the register HTTP RTT and got SHADE_NOT_FOUND.
start() now defers; register() success kicks schedulePoll(0). Manual
tick() is unaffected (deliberate user action, no gating).
Both reported by Prism. Tests cover all five acceptance criteria
from the sender-attribution request (PUT capture, bridge surface,
fetch surface, inter-version compat, end-to-end pair smoke) plus
the three from the race-fix request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the bridge-connection-lifecycle signal that closes Prism's
~45s revoke window down to one server→client round-trip (~50ms).
Server (`@shade/inbox-server`):
- `inbox.peer_connected` / `inbox.peer_disconnected` events on the
0↔1 boundary across WS + SSE bridges. Long-poll deliberately not
tracked (every poll boundary would flap; push transports are also
the only ones where instant revoke matters).
- `PresenceTracker` collapses two parallel bridges (e.g. WS + SSE
during fallback handover) into one connect/disconnect pair.
- `GET /v1/bridge/presence` SSE endpoint: signed query with
`kind: 'presence'`, `watched: string[]`; on open streams a
per-address snapshot, then change frames filtered server-side.
MAX_WATCHED_ADDRESSES = 64. Subscribing does not itself count as
a peer-bridge connection.
- `createBridgeRoutes` now returns `{ app, websocket, presence }`.
Client (`@shade/transport-bridge`):
- `PresenceBridge.subscribe({ watch, onPresenceChange })` →
`{ addPeer, removePeer, watching, unsubscribe }`. addPeer/removePeer
mutate via reconnect with a fresh signed query.
- `signPresenceQuery` helper for non-PresenceBridge consumers.
Tests cover all four acceptance criteria from the Prism request:
server-event smoke, online→offline subscription, address scoping
(carol invisible to a [alice]-only sub), reconnect, plus an
addPeer/removePeer regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Browsers' Window.fetch is a WebIDL bound operation; storing it as
this.fetchImpl / this.fetchFn and calling via the instance receiver
threw "Illegal invocation" on the first request. Bind once at
construction in InboxClient, LongPollBridge, and SseBridge. Reported
by Prism (multi-device E2EE terminal), blocking every browser
consumer of the v4.6 transport stack on inbox.start() / bridge.connect().
WsBridge unaffected (uses WebSocket). Node/Bun fetch tolerates a free
receiver, so the bug never surfaced server-side — added regression
tests that install a strict-receiver globalThis.fetch to catch the
issue without an actual browser harness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the broadcast-channel primitive Prism asked for in
Docs/shade-feature-request-sender-keys.md. The crypto in
@shade/core/sender-keys.ts was already in place; this release wires
it up as a first-class app-facing API, adds the persistence schema
across all six storage backends (memory, sqlite, indexeddb +
encrypted variants), introduces wire type 0x21 in @shade/proto,
and ships Prism's three acceptance tests verbatim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the foundations Prism's web client (and any future browser-based
Shade app) needs: at-rest-encrypted IndexedDB storage that mirrors the
SQLite backend byte-for-byte at the AAD/nonce level, browser-safe
subpath imports so Vite/webpack/esbuild stop hitting bun:sqlite, and
KeyManager support for argon2id and N-factor composite unlock.
@shade/storage-encrypted
- EncryptedIndexedDBStorage (subpath: /idb) — full StorageProvider
using one object store per _enc table; reuses aeadSeal/aeadOpen +
row-codec sealers so a row sealed under the SQLite or Postgres
backend decrypts under IDB given the same KeyManager.
bumpPeerIdentityVersion is atomic under one IDB transaction.
- KeyManager argon2id source — memory-hard KDF for low-entropy
secrets (PINs). Backed by @noble/hashes/argon2 (already a transitive
dep). DEFAULT_ARGON2ID exported (m=64 MiB, t=3, p=1).
- KeyManager composite source — HKDF-combine N sub-sources into one
master. Every source mandatory; order significant by design;
composite-of-composite rejected; optional info string for app-level
domain separation.
- Subpath exports (/crypto, /sqlite, /postgres, /idb) plus a `browser`
condition on the default import that resolves to a barrel
excluding the Bun- and Postgres-specific entries. Browser bundles
no longer pull bun:sqlite transitively.
Tests
- 73 tests in shade-storage-encrypted (was 31). New coverage:
argon2id determinism + reject paths, composite same-factors → same
master, wrong-PIN/passphrase/order-swap → different master, info
domain separation, all 28 StorageProvider methods on
EncryptedIndexedDBStorage, fingerprint-mismatch rejection, and
cross-impl roundtrip with EncryptedSQLiteStorage proving the AAD/
nonce derivation is implementation-agnostic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expose the local device's 32-byte Ed25519 identity public key on Shade
so apps can hand it to their own backend at enrollment time for
signature verification, key pinning or per-device safety-number
computation. Closes the gap that forced consumers to ship placeholder
random bytes their backend could store but never verify against.
- @shade/sdk Shade.identityPublicKey: Promise<Uint8Array> — getter
mirrors the existing fingerprint accessor. Throws pre-init,
reflects the current key after rotate(), retired key preserved in
retired-identities storage per existing grace-period contract.
Private key remains unreachable.
- Test in shade-sdk/tests/sdk.test.ts: round-trip match against the
underlying storage's signingPublicKey, plus value updates after
rotate().
- Lockstep version bump 4.3.0 → 4.4.0 across all 25 packages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ship an official IndexedDB-backed StorageProvider so browser-based Shade
consumers persist identity, prekeys, sessions, retired identities,
peer-verification state and stream-resume rows across tab refresh and
browser restart. Closes the gap that forced browser apps onto
storage:"memory" (regenerated identity each load, orphaned device
records server-side).
- New package @shade/storage-indexeddb (4.3.0): full StorageProvider
conformance, schema v1, idb-backed; bumpPeerIdentityVersion is wrapped
in a single readwrite IDB transaction (atomic, vs SQLite's
read-then-upsert race).
- @shade/sdk resolveStorage() accepts { type: 'indexeddb', dbName? } via
dynamic import (lazy, optional dep — same pattern as
@shade/storage-postgres). Named StorageSpec type now reused by
ResolvedConfig.
- Tests: 16 new tests in shade-storage-indexeddb (StorageProvider
surface + peer-verifications + full E2EE conversation surviving a
simulated tab reload). Run on fake-indexeddb.
- Lockstep version bump 4.2.1 → 4.3.0 across all 25 packages.
- Publish scripts updated to include the new package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pull-mode httpClient + drainer + parallel RPCs against the same peer
deteriorated after ~10s with `DecryptionError`. Two bugs combined:
- `OutboundQueue.enqueue` woke `drain` waiters with a `since=0`
snapshot, replaying already-processed events into
`Shade.acceptTransferEnvelope` → `manager.decrypt` twice. The
duplicate consumed an already-used skipped key and corrupted the
Double Ratchet receive chain.
- `ratchetDecrypt` then propagated the corruption: a same-DH
message behind the chain with no cached skipped key fell through
to `kdfChainKey` on the ahead state and rewound `chain.counter`,
permanently desyncing the chain.
Fix `OutboundQueue` to honor each waiter's `since`, and harden
`ratchetDecrypt` so any future duplicate fails cleanly without
mutating state. Adds regression coverage at all three layers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.1.0's HTTP RPC for browsers capped at inline payloads (≤ 256 KiB).
4.2.0 unlocks streams: server queues outbound chunks + control
envelopes per peer, browser long-polls the queue. Browser-to-server
writes ride the existing /v1/transfer/<id>/chunk POST routes
unchanged.
For Dispatch this unlocks mod-jar uploads (50 MB) and world-backup
downloads (100+ MB) — the actual reason browser-side @shade/files
matters.
### New API
@shade/sdk:
- shade.transferQueueRoute(opts?) — Hono app with /queue +
/v1/transfer/* routes. Auto-configures the queue transport.
- shade.configureTransfers extended: transport + envelopeTransport
override slots; resolveBaseUrl optional when both supplied.
@shade/transfer:
- OutboundQueue — per-peer monotonic event log with long-poll
semantics, idle-eviction GC, ring-buffered to maxEventsPerPeer.
- QueueTransferTransport — enqueues instead of POSTing.
@shade/files:
- httpClient({ outboundQueueUrl, transferBaseUrl }) — when set,
starts a long-poll drainer + builds a streams-bridge. fs.read /
fs.write of >256 KiB work end-to-end.
- startQueueDrainer(shade, opts) — exported helper for advanced
consumers driving their own drainer.
### Implementation notes
- ClientStreamsBridge's TransformStream had HWM=0 by default which
stalled the drainer's await chain at chunk 4 (writer.write pended
before the consumer's reader was attached). Bumped to HWM=64 so
the receive loop can buffer ahead of the consumer.
### Tests
3 new integration tests in tests/integration/http-rpc-streams.test.ts:
4 MiB streamed read round-trip, inline-only error path, idle-timeout
long-poll behaviour.
Wire-compatible. Source-compatible. Lockstep bump to 4.2.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Default shade.files.client(peer) requires both peers to be mutually
addressable over HTTP — the response round-trips through
Shade.deliverControlEnvelope (POST to peer's /v1/transfer/control).
Browser tabs can't host an HTTP server, so they couldn't consume
@shade/files at all. Dispatch's filutforsker (admin-panel browser UI)
is the canonical use-case.
This release adds a parallel request-response transport: one POST per
RPC, encrypted envelope in the body, encrypted response in the same
HTTP response. No inbound channel needed on the client.
### New API
- shade.files.rpcRoute(opts?) — Hono app exposing POST /rpc.
- shade.files.httpClient(peer, opts) — request-response FileClient.
- FilesNamespace.serve(handler, { inlineOnly: true }) — skip streams-
bridge (and its configureTransfers pre-condition); also skip
channel-based dispatch so requests aren't double-dispatched.
### Limitations (v1)
Inline only (≤ 256 KiB). Streamed reads/writes throw clear errors
directing to shade.files.client(peer) on a server-to-server deploy.
### Tests
7 integration tests in tests/integration/http-rpc.test.ts covering
round-trip + negative cases (sender header, empty/garbage body,
maxBodyBytes, rpcRoute-without-serve).
### Symmetry
Mirrors @shade/server's shade-auth-middleware: encrypted envelope in
request body, decrypted via existing ratchet, response in same HTTP
roundtrip. No WebSocket, no SSE, no outbound from server.
Wire-compatible. Source-compatible. Lockstep bump to 4.1.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.0.1's typecheck gate compiled each package internally against
lib: ["ES2022"]. That doesn't catch types that only fail when
*consumer* code (lib: ["DOM"] + exactOptionalPropertyTypes) tries to
assign a native browser type into one of our locally-defined narrower
types. Dispatch hit one such case in @shade/files inline-threshold.ts.
This release adds a tests/consumer-strict/ smoke project to the
pre-publish gate. It compiles a tiny "as if I were a downstream app"
TS file against:
lib: ["ES2022", "DOM", "DOM.Iterable"]
types: ["bun-types"]
exactOptionalPropertyTypes: true
strict: true
paths → packages/*/src/index.ts
scripts/typecheck-all.ts now runs the smoke after per-package checks.
Both must pass before publish:dry / publish:all proceeds.
### Fixed
- @shade/files inline-threshold.ts: MinimalReader<T> rewritten as the
explicit disjoint union { done:false, value:T } | { done:true,
value?: T | undefined } that's assignable from every native reader
shape (bun, DOM, node:stream/web). Fixes the
"ReadableStreamReadResult is not assignable" Dispatch reported.
- @shade/files streams-bridge (client + server): stash setTimeout
return in a local before .unref?.() via { unref?: () => void } cast.
Fluent .unref?.() failed under lib: ["DOM"] (setTimeout returns
number there).
- @shade/sdk background.ts: same setInterval .unref?.() fix.
Wire-compatible. No API shape changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4.0.0 shipped TypeScript source as published main/types, but several
files only compiled inside the monorepo. Consumer projects (Dispatch,
etc.) running their own strict tsc against our published source hit:
- @shade/key-transparency: 4 noUnusedLocals violations
(IndexAbsenceProof, IndexInclusionProof, IndexProofWire, nodeHash)
- @shade/sdk: KT verifier callbacks returned Promise<unknown> instead
of Promise<STHWire> / Promise<{ proof: string[] }>
- @shade/sdk: thumbnail.ts globalThis cast collided with consumer's
lib.dom-supplied createImageBitmap signature
- @shade/files: cycle with @shade/sdk produced "this is not assignable
to type 'Shade'" because hoisted node_modules layouts duplicated the
Shade class. Broken by replacing `import type { Shade }` with a
local structural ShadeBridge interface.
- @shade/storage-encrypted: KeyUsage (lib.dom) used under
lib: ["ES2022"]
- @shade/transport-bridge: ReadableStreamDefaultReader<any> ↔
<Uint8Array> mismatch
- @shade/keychain / @shade/dashboard / @shade/storage-encrypted
tsconfig rootDir / include hygiene
Tooling: scripts/typecheck-all.ts runs `bunx tsc --noEmit` against
every workspace package's tsconfig and fails on any error. Wired into
publish:dry / publish:all and publish-shade.sh as a hard gate so this
class of bug cannot recur.
All 24 packages bumped to 4.0.1 in lockstep.
Migration: <ShadeFilesProvider> now requires an explicit `files` prop
(pass `shade.files`). Wire format unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shade now ships as a self-contained Docker image. Deploy one container
per project, any stack (Bun, Python, Go, Rust, Kotlin) can talk to it via
plain HTTP. Zero coupling to consumer codebases.
M-Box 1: Stale identity cleanup API
- touchIdentity + purgeStaleIdentities on PrekeyStore interface
- Implemented for Memory, SQLite, and Postgres backends
- SQLite adds last_activity_at column with migration ALTER for existing DBs
- Postgres adds the same via raw SQL with IF NOT EXISTS guards
- Routes call touchIdentity on register, bundle fetch, replenish
- 4 new tests for the cleanup API
M-Box 2: Stale cleanup background task
- StaleCleanupTask runs purge on startup + every 24h (configurable)
- Reads SHADE_STALE_DAYS (default 30) and SHADE_CLEANUP_INTERVAL_HOURS
- Wired into standalone.ts, stopped on graceful shutdown
- 5 new tests for the task
M-Box 3: Observer baked into the container
- standalone.ts conditionally mounts @shade/observer at /shade-observer
when SHADE_OBSERVER_TOKEN is set (and >= 16 chars)
- Shared PrekeyServerEvents emitter feeds both routes and observer
- @shade/observer added as optional dependency of @shade/server
M-Box 4: Dockerfile with dashboard build
- Multi-stage build: oven/bun:1 builder → oven/bun:1-alpine runtime
- COPY packages/ wholesale so workspace lockfile resolves cleanly
- RUN bun run build inside shade-dashboard → dist/ → observer/dist/
- Non-root shade user, /data volume, healthcheck, env defaults
- Final image: 260 MB
M-Box 5: OpenAPI spec for stack-agnostic clients
- packages/shade-server/openapi.yaml documents all 9 endpoints with
request/response schemas, security (Ed25519 signatures + bearer token)
- createOpenApiRoutes serves /openapi.yaml and /docs (Redoc viewer)
- Any language can generate a client with openapi-generator
M-Box 6: Docker CI pipeline
- .gitea/workflows/docker.yml builds + pushes on git tag v*
- scripts/build-docker.ts for local builds, supports --push with GITEA_TOKEN
- Root package.json: build:docker, publish:docker scripts
M-Box 7: Deployment documentation
- packages/shade-server/README rewritten: 5-line quickstart with the image
- docs/DEPLOYMENT.md: full reference, env vars, backup, Dokploy, PG setup
- examples/05-dokploy-deployment/docker-compose.yml updated to pull
published image (gt.zyon.no/stian/shade-prekey:latest)
- Root README deployment section rewritten
M-Box 8: End-to-end verification
- Image builds locally (bun run build:docker)
- /health, /openapi.yaml, /docs, /metrics, /shade-observer all respond
- 401 without observer token, 200 with
- Real SDK client round-trip: Alice → container → Bob → reply → Alice
- Persistence: identity + prekeys survive container restart (count 20→18
as expected from two bundle fetches)
285 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase B complete: Shade now has a full developer tooling story.
@shade/cli
- shade init with project scaffolding from templates
- shade fingerprint (own or peer)
- shade publish (re-upload bundle)
- shade rotate (--identity for full rotation, otherwise signed prekey)
- shade peer add/list/verify/remove
- shade dashboard (opens observer in browser)
- shade doctor (diagnose config, storage, prekey server reachability)
- Config from .shaderc.json or SHADE_* env vars
Templates (in packages/shade-cli/templates/)
- bun-server — Bun + Hono backend with /send + /receive endpoints
- chat-demo — Two-process Alice/Bob chat over HTTP
Publishing pipeline (Gitea npm registry)
- .gitea/workflows/test.yml — CI on push/PR with PostgreSQL service
- .gitea/workflows/publish.yml — publish on git tag v*
- scripts/publish-all.ts — local publish helper with DRY_RUN support
- scripts/bump-version.ts — lockstep version bump across all packages
- Root package.json scripts: version, publish:dry, publish:all
Also: /health endpoint now lives in createPrekeyRoutes so doctor can
probe it without needing the full standalone setup.
Dry-run verified: all 11 packages pack cleanly.
246 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
M-Obs 1: Event bus in @shade/core
- ShadeEventEmitter with typed event union, ring buffer for replay
- 12 event types covering session lifecycle, ratchet operations,
prekey changes, identity rotation, trust changes
- Wired into ShadeSessionManager (zero overhead when not enabled)
- shortHash helper for safe display of public keys
- Security test: regex-checks event payloads contain no key material
M-Obs 2: Prekey server event hooks
- PrekeyServerEvents emitter mirroring core's pattern
- 5 server event types: registered, fetched, replenished, deleted, rate_limited
- Wired into all routes including the rate-limit error handler
- shortHash helper using crypto.subtle directly (no provider dep)
M-Obs 3: @shade/observer package
- StateAggregator subscribes to client + server events, builds rolling snapshot
- Hono routes: GET /api/state (snapshot), GET /api/events (SSE stream)
- Bearer token auth via SHADE_OBSERVER_TOKEN, query string for SSE
- Refuses to start with token < 16 chars (ConfigurationError)
- Static file serving for bundled dashboard at /dashboard/
- Placeholder dashboard renders when no built SPA present
220 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
M-Hard 6: PostgreSQL Storage Backend
- @shade/storage-postgres with PostgresStorage + PostgresPrekeyStore
- Drizzle-style raw SQL ensureClientTables / ensurePrekeyServerTables
- All tables prefixed `shade_` to avoid collisions in shared databases
- DELETE ... FOR UPDATE SKIP LOCKED for concurrent OTPK consumption
- Tests skip gracefully without SHADE_TEST_PG_URL, run against real PG when set
M-Hard 7: Production Server Infrastructure
- Structured JSON logger (logger.ts) — SHADE_LOG_LEVEL configurable
- Health endpoints (/health, /healthz, /ready) — Kubernetes-friendly
- Prometheus metrics (/metrics) — counters, histograms, middleware
- Graceful shutdown with SIGTERM/SIGINT handlers + store close
- Production Dockerfile with non-root user, healthcheck, multi-stage build
- docker-compose.yml example for Dokploy with optional PostgreSQL
193 tests passing, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shade sessions and keys now survive server crashes, container restarts,
and power outages via SQLite with WAL mode.
New packages:
- @shade/storage-sqlite: SQLiteStorage (StorageProvider) + SqlitePrekeyStore
(PrekeyStore), both using bun:sqlite with auto-created tables and WAL mode
- Serialization layer in shade-core for SessionState/keys ↔ JSON/base64
Docker usage: mount volume at /data, set SHADE_DB_PATH=/data/shade-client.db
Prekey server auto-detects SHADE_PREKEY_DB_PATH for SQLite persistence
Includes crash recovery integration test: encrypt → close DB → reopen →
conversation continues seamlessly.
129 tests, 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
