release(v4.8.2): per-from receive serialization + per-connection bridge dedup
Some checks failed
Test / test (push) Has been cancelled
Some checks failed
Test / test (push) Has been cancelled
Two interlocking robustness fixes for the duplicate-fan-out / first-contact
class of failures Prism reported.
1. `Shade.receive(from, env)` now queues its `manager.decrypt` step
per `from` so concurrent dispatches can't race the SessionManager
ratchet or the StorageProvider (sqlite "database is locked", IDB
transaction conflicts). User message handlers run *outside* the
queue so streams + file-RPC's nested `shade.receive` calls don't
self-deadlock.
2. Bridge WS + SSE handlers now run a per-connection bounded msgId
LRU as defense-in-depth against any flushTo re-entry (event-storm,
future refactor). Pending-flush chains are wrapped in `.catch(() =>
{})` so a transient `ws.send` rejection no longer poisons the
connection's flush loop.
Tests: storming `inbox.blob_stored` 10× per PUT yields exactly one WS/
SSE frame; 8 concurrent `bob.receive('alice', envelope)` calls keep
the ratchet intact and never surface "database is locked".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
81
CHANGELOG.md
81
CHANGELOG.md
@@ -5,6 +5,87 @@ All notable changes to Shade are documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [4.8.2] — 2026-05-08 — Per-`from` decrypt serialization + per-connection bridge dedup
|
||||||
|
|
||||||
|
Two interlocking robustness fixes for the first-contact / duplicate-fan-out
|
||||||
|
class of failures Prism reported. Either fix on its own would help; together
|
||||||
|
they make the receiver path tolerant of any combination of relay duplicates
|
||||||
|
and concurrent dispatchers.
|
||||||
|
|
||||||
|
**(1) `Shade.receive(from, env)` now serializes its ratchet/storage
|
||||||
|
step per `from`.** The send path has had a per-address `encryptChains`
|
||||||
|
mutex since V1 — receive did not. Concurrent decrypts for the same peer
|
||||||
|
raced the `SessionManager` ratchet (mutated in place) and the
|
||||||
|
`StorageProvider` (which is not required to be a concurrent-safe
|
||||||
|
writer — `bun:sqlite` throws `database is locked`, IndexedDB throws
|
||||||
|
transaction conflicts). Symptom in production: a single relay PUT that
|
||||||
|
fans out 8× over a WS bridge gets dispatched as 8 parallel
|
||||||
|
`shade.receive` calls; one wins the X3DH prekey race, the other 7 fail
|
||||||
|
with `database is locked` or `one-time prekey not found: <id>`, and the
|
||||||
|
post-decrypt side effects (`markPeerVerified`,
|
||||||
|
`BroadcastChannel.addMember`, paired-reply `inbox.send`) get lost in
|
||||||
|
the rubble. The decrypt step is now chained off a per-`from` promise
|
||||||
|
queue. Crucially, the user-facing **message handlers run outside the
|
||||||
|
queue** — streams + file-RPC issue nested `shade.receive` calls for the
|
||||||
|
same peer from inside their handlers (e.g. `stream-end` arrives while a
|
||||||
|
write-RPC is still waiting on chunks), and holding the queue across the
|
||||||
|
handler would self-deadlock. Only the atomic ratchet+storage step is
|
||||||
|
protected.
|
||||||
|
|
||||||
|
**(2) Bridge handlers (WS + SSE) now run a per-connection msgId
|
||||||
|
LRU dedup.** Cursor-based delivery already de-duplicates in the happy
|
||||||
|
path, but the gate is a defense-in-depth against any subtle re-entry of
|
||||||
|
`flushTo` (event-storm, future refactor, fallback-timer race). The chain
|
||||||
|
that drives flush is now also wrapped in `.catch(() => {})` so a
|
||||||
|
transient `ws.send` / SSE write rejection doesn't poison every future
|
||||||
|
push on the connection.
|
||||||
|
|
||||||
|
Both reported by Prism (multi-device E2EE terminal). Wave-3 pair
|
||||||
|
handshake is unblocked even when the receiver runs multiple bridges or
|
||||||
|
the relay double-fires `inbox.blob_stored`.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
#### `@shade/sdk` — `Shade.receive` per-`from` serialization
|
||||||
|
- `Shade` gains a private `decryptChains: Map<string, Promise<unknown>>`
|
||||||
|
mirroring the existing `encryptChains` on the send path.
|
||||||
|
- `Shade.receive(from, env)` chains its `manager.decrypt(from, env)`
|
||||||
|
call off the prior decrypt promise for the same `from`. The
|
||||||
|
post-decrypt control-plaintext check and user `messageHandlers` run
|
||||||
|
*outside* the chain so nested `shade.receive` calls from inside a
|
||||||
|
handler don't self-deadlock (streams + file-RPC depend on this).
|
||||||
|
- The stored chain is `decryptPromise.catch(() => undefined)` so a
|
||||||
|
rejection in one decrypt doesn't sabotage the next; this caller
|
||||||
|
still sees its own rejection through the original promise.
|
||||||
|
- External signature unchanged.
|
||||||
|
|
||||||
|
#### `@shade/inbox-server` — bridge per-connection msgId dedup
|
||||||
|
- New internal `DeliveredIdLru` (4096-entry bounded set, FIFO eviction)
|
||||||
|
per WS / SSE connection. `flushTo` skips emit when a row's `msgId` is
|
||||||
|
already in the LRU. Long-poll handlers don't need it (each request is
|
||||||
|
isolated).
|
||||||
|
- `pendingFlushPromise` chains in both WS and SSE handlers now
|
||||||
|
terminate in `.catch(() => {})` so a transient emit failure doesn't
|
||||||
|
silently kill the connection's flush loop.
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
- `packages/shade-transport-bridge/tests/bridge.test.ts` — new
|
||||||
|
"Bridge dedup" describe block: storms `inbox.blob_stored` 10× for one
|
||||||
|
PUT and asserts WS / SSE both deliver exactly one frame.
|
||||||
|
- `packages/shade-sdk/tests/sdk.test.ts` — new
|
||||||
|
"concurrent receive(from, env) for same `from` does not race the
|
||||||
|
ratchet" exercises 8 parallel `bob.receive('alice', env)` for the
|
||||||
|
same envelope and asserts:
|
||||||
|
1. at least one fulfills with the right plaintext;
|
||||||
|
2. no rejection mentions `database is locked`;
|
||||||
|
3. the next legitimate message still decrypts (ratchet intact).
|
||||||
|
|
||||||
|
### Migration
|
||||||
|
|
||||||
|
None. Drop-in. Bridges and receivers behave identically on non-
|
||||||
|
duplicate paths; the new gates only kick in when a duplicate would
|
||||||
|
otherwise have been emitted / dispatched.
|
||||||
|
|
||||||
## [4.8.1] — 2026-05-08 — `SHADE_DISABLE_RATE_LIMIT` env var for single-tenant deploys
|
## [4.8.1] — 2026-05-08 — `SHADE_DISABLE_RATE_LIMIT` env var for single-tenant deploys
|
||||||
|
|
||||||
The standalone server's `routes.ts` and `inbox-server`'s
|
The standalone server's `routes.ts` and `inbox-server`'s
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/cli",
|
"name": "@shade/cli",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/cli.ts",
|
"main": "src/cli.ts",
|
||||||
"bin": {
|
"bin": {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/core",
|
"name": "@shade/core",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/crypto-web",
|
"name": "@shade/crypto-web",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/dashboard",
|
"name": "@shade/dashboard",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "vite",
|
"dev": "vite",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/files",
|
"name": "@shade/files",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/inbox-server",
|
"name": "@shade/inbox-server",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -136,15 +136,22 @@ export function createBridgeRoutes(opts: BridgeRoutesOptions): {
|
|||||||
};
|
};
|
||||||
let cursor = verified.since;
|
let cursor = verified.since;
|
||||||
const writer = makeBlobWriter(opts.store, pageLimit);
|
const writer = makeBlobWriter(opts.store, pageLimit);
|
||||||
|
const delivered = new DeliveredIdLru();
|
||||||
|
|
||||||
// Initial backlog drain.
|
// Initial backlog drain.
|
||||||
const flushed = await flushTo(writer, address, cursor, async (blob) => {
|
const flushed = await flushTo(
|
||||||
await stream.writeSSE({
|
writer,
|
||||||
id: String(blob.receivedAt),
|
address,
|
||||||
event: 'envelope',
|
cursor,
|
||||||
data: JSON.stringify(serializeBlob(blob)),
|
async (blob) => {
|
||||||
});
|
await stream.writeSSE({
|
||||||
});
|
id: String(blob.receivedAt),
|
||||||
|
event: 'envelope',
|
||||||
|
data: JSON.stringify(serializeBlob(blob)),
|
||||||
|
});
|
||||||
|
},
|
||||||
|
delivered,
|
||||||
|
);
|
||||||
cursor = Math.max(cursor, flushed);
|
cursor = Math.max(cursor, flushed);
|
||||||
|
|
||||||
// Hook up event-driven push if available, else fall back to a poll
|
// Hook up event-driven push if available, else fall back to a poll
|
||||||
@@ -156,19 +163,31 @@ export function createBridgeRoutes(opts: BridgeRoutesOptions): {
|
|||||||
const triggerFlush = (): void => {
|
const triggerFlush = (): void => {
|
||||||
signalled = true;
|
signalled = true;
|
||||||
// Serialize fan-in so concurrent triggers don't double-fetch.
|
// Serialize fan-in so concurrent triggers don't double-fetch.
|
||||||
pendingFlushPromise = pendingFlushPromise.then(async () => {
|
// `.catch(() => {})` keeps the chain alive across transient
|
||||||
while (signalled) {
|
// emit failures (e.g. a closed SSE write throws) — without it
|
||||||
signalled = false;
|
// one rejection silently kills every future flush on this
|
||||||
const drained = await flushTo(writer, address, cursor, async (blob) => {
|
// connection.
|
||||||
await stream.writeSSE({
|
pendingFlushPromise = pendingFlushPromise
|
||||||
id: String(blob.receivedAt),
|
.then(async () => {
|
||||||
event: 'envelope',
|
while (signalled) {
|
||||||
data: JSON.stringify(serializeBlob(blob)),
|
signalled = false;
|
||||||
});
|
const drained = await flushTo(
|
||||||
});
|
writer,
|
||||||
if (drained > cursor) cursor = drained;
|
address,
|
||||||
}
|
cursor,
|
||||||
});
|
async (blob) => {
|
||||||
|
await stream.writeSSE({
|
||||||
|
id: String(blob.receivedAt),
|
||||||
|
event: 'envelope',
|
||||||
|
data: JSON.stringify(serializeBlob(blob)),
|
||||||
|
});
|
||||||
|
},
|
||||||
|
delivered,
|
||||||
|
);
|
||||||
|
if (drained > cursor) cursor = drained;
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(() => {});
|
||||||
};
|
};
|
||||||
|
|
||||||
if (opts.events) {
|
if (opts.events) {
|
||||||
@@ -327,6 +346,7 @@ export function createBridgeRoutes(opts: BridgeRoutesOptions): {
|
|||||||
const connId = presence.newConnectionId();
|
const connId = presence.newConnectionId();
|
||||||
let cursor = verified.since;
|
let cursor = verified.since;
|
||||||
const writer = makeBlobWriter(opts.store, pageLimit);
|
const writer = makeBlobWriter(opts.store, pageLimit);
|
||||||
|
const delivered = new DeliveredIdLru();
|
||||||
let unsubscribe: (() => void) | null = null;
|
let unsubscribe: (() => void) | null = null;
|
||||||
let fallbackTimer: ReturnType<typeof setInterval> | null = null;
|
let fallbackTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
let pendingFlushPromise: Promise<void> = Promise.resolve();
|
let pendingFlushPromise: Promise<void> = Promise.resolve();
|
||||||
@@ -347,15 +367,26 @@ export function createBridgeRoutes(opts: BridgeRoutesOptions): {
|
|||||||
presence.markConnected(address, 'ws', connId);
|
presence.markConnected(address, 'ws', connId);
|
||||||
const triggerFlush = (): void => {
|
const triggerFlush = (): void => {
|
||||||
signalled = true;
|
signalled = true;
|
||||||
pendingFlushPromise = pendingFlushPromise.then(async () => {
|
// `.catch(() => {})` mirrors the SSE chain — keeps the
|
||||||
while (signalled && connected) {
|
// pending-flush queue alive across transient ws.send errors
|
||||||
signalled = false;
|
// (e.g. partial close, backpressure overflow).
|
||||||
const drained = await flushTo(writer, address, cursor, async (blob) => {
|
pendingFlushPromise = pendingFlushPromise
|
||||||
ws.send(JSON.stringify(serializeBlob(blob)));
|
.then(async () => {
|
||||||
});
|
while (signalled && connected) {
|
||||||
if (drained > cursor) cursor = drained;
|
signalled = false;
|
||||||
}
|
const drained = await flushTo(
|
||||||
});
|
writer,
|
||||||
|
address,
|
||||||
|
cursor,
|
||||||
|
async (blob) => {
|
||||||
|
ws.send(JSON.stringify(serializeBlob(blob)));
|
||||||
|
},
|
||||||
|
delivered,
|
||||||
|
);
|
||||||
|
if (drained > cursor) cursor = drained;
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(() => {});
|
||||||
};
|
};
|
||||||
if (opts.events) {
|
if (opts.events) {
|
||||||
unsubscribe = opts.events.on((e) => {
|
unsubscribe = opts.events.on((e) => {
|
||||||
@@ -518,11 +549,41 @@ function makeBlobWriter(store: InboxStore, pageLimit: number): BlobWriter {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-connection bounded msgId tracker — defense in depth against duplicate
|
||||||
|
* delivery of the same blob to the same bridge socket. Cursor pagination
|
||||||
|
* already guarantees uniqueness in the happy path, but a dedup gate at the
|
||||||
|
* emit boundary catches any subtle bug (e.g. a flushTo race, a future
|
||||||
|
* refactor, an event-emit retry) without changing wire semantics.
|
||||||
|
*
|
||||||
|
* The cap is intentionally large enough to cover any realistic bridge
|
||||||
|
* pageLimit and small enough to bound memory under long-running streams.
|
||||||
|
*/
|
||||||
|
const DELIVERED_LRU_CAP = 4096;
|
||||||
|
|
||||||
|
class DeliveredIdLru {
|
||||||
|
private readonly seen = new Set<string>();
|
||||||
|
private readonly order: string[] = [];
|
||||||
|
|
||||||
|
/** Returns true if `msgId` has not been seen on this connection yet. */
|
||||||
|
add(msgId: string): boolean {
|
||||||
|
if (this.seen.has(msgId)) return false;
|
||||||
|
this.seen.add(msgId);
|
||||||
|
this.order.push(msgId);
|
||||||
|
if (this.order.length > DELIVERED_LRU_CAP) {
|
||||||
|
const evicted = this.order.shift()!;
|
||||||
|
this.seen.delete(evicted);
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async function flushTo(
|
async function flushTo(
|
||||||
writer: BlobWriter,
|
writer: BlobWriter,
|
||||||
address: string,
|
address: string,
|
||||||
startCursor: number,
|
startCursor: number,
|
||||||
emit: (blob: BlobRow) => Promise<void>,
|
emit: (blob: BlobRow) => Promise<void>,
|
||||||
|
delivered?: DeliveredIdLru,
|
||||||
): Promise<number> {
|
): Promise<number> {
|
||||||
let cursor = startCursor;
|
let cursor = startCursor;
|
||||||
// Drain page-by-page so a backlog larger than `pageLimit` still flushes.
|
// Drain page-by-page so a backlog larger than `pageLimit` still flushes.
|
||||||
@@ -531,7 +592,12 @@ async function flushTo(
|
|||||||
const page = await writer.fetchPage(address, cursor);
|
const page = await writer.fetchPage(address, cursor);
|
||||||
if (page.length === 0) break;
|
if (page.length === 0) break;
|
||||||
for (const row of page) {
|
for (const row of page) {
|
||||||
await emit(row);
|
// Per-connection dedup gate — prevents the same msgId from being
|
||||||
|
// emitted twice if flushTo is somehow re-entered before the cursor
|
||||||
|
// catches up. See comment on `DeliveredIdLru`.
|
||||||
|
if (!delivered || delivered.add(row.msgId)) {
|
||||||
|
await emit(row);
|
||||||
|
}
|
||||||
if (row.receivedAt > cursor) cursor = row.receivedAt;
|
if (row.receivedAt > cursor) cursor = row.receivedAt;
|
||||||
}
|
}
|
||||||
if (page.length === 0) break;
|
if (page.length === 0) break;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/inbox",
|
"name": "@shade/inbox",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/key-transparency",
|
"name": "@shade/key-transparency",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/keychain",
|
"name": "@shade/keychain",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/observability",
|
"name": "@shade/observability",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/observer",
|
"name": "@shade/observer",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/proto",
|
"name": "@shade/proto",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/recovery",
|
"name": "@shade/recovery",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/sdk",
|
"name": "@shade/sdk",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -153,6 +153,13 @@ export class Shade {
|
|||||||
private establishing = new Map<string, Promise<void>>();
|
private establishing = new Map<string, Promise<void>>();
|
||||||
// Per-address encrypt queue to serialize ratchet mutations
|
// Per-address encrypt queue to serialize ratchet mutations
|
||||||
private encryptChains = new Map<string, Promise<unknown>>();
|
private encryptChains = new Map<string, Promise<unknown>>();
|
||||||
|
// Per-`from` decrypt queue: serializes incoming receives so two concurrent
|
||||||
|
// shade.receive(from, env) calls can't race the ratchet/storage. Without
|
||||||
|
// this, parallel deliveries (relay duplicate fan-out, fast pipelined
|
||||||
|
// sends) hit `database is locked` (sqlite) or transaction conflicts (IDB)
|
||||||
|
// because the underlying StorageProvider isn't required to be a
|
||||||
|
// concurrent-safe writer. See V4.8.2 changelog.
|
||||||
|
private decryptChains = new Map<string, Promise<unknown>>();
|
||||||
|
|
||||||
// Message handlers — may be sync or async; receive() awaits each. The
|
// Message handlers — may be sync or async; receive() awaits each. The
|
||||||
// optional third arg distinguishes direct vs broadcast plaintexts;
|
// optional third arg distinguishes direct vs broadcast plaintexts;
|
||||||
@@ -436,7 +443,35 @@ export class Shade {
|
|||||||
*/
|
*/
|
||||||
async receive(from: string, envelope: ShadeEnvelope): Promise<string> {
|
async receive(from: string, envelope: ShadeEnvelope): Promise<string> {
|
||||||
if (!this.initialized) throw new Error('Not initialized');
|
if (!this.initialized) throw new Error('Not initialized');
|
||||||
const plaintext = await this.manager.decrypt(from, envelope);
|
|
||||||
|
// Serialize ONLY the ratchet/storage write portion of receive (the
|
||||||
|
// call into `manager.decrypt`). Concurrent decrypts race the
|
||||||
|
// SessionManager ratchet (mutated in place) and the StorageProvider
|
||||||
|
// (not required to be a concurrent-safe writer — `bun:sqlite`
|
||||||
|
// throws `database is locked`, IDB throws transaction conflicts).
|
||||||
|
// The Prism FR called this out: a relay-duplicated WS fan-out
|
||||||
|
// dispatched 8 parallel `shade.receive(from, env)` calls, one won
|
||||||
|
// the X3DH prekey race and the other 7 failed with
|
||||||
|
// `database is locked` / `one-time prekey not found`. The fix is
|
||||||
|
// to queue per-`from` decrypts so the ratchet step is sequential.
|
||||||
|
//
|
||||||
|
// Crucially the user-visible MESSAGE HANDLERS run *outside* the
|
||||||
|
// queue. Streams + file-RPC issue nested `shade.receive` calls for
|
||||||
|
// the same peer from inside their handlers (e.g. `stream-end`
|
||||||
|
// arrives while a write-RPC is still waiting on chunks); holding
|
||||||
|
// the queue across the handler would self-deadlock. The atomic
|
||||||
|
// unit we have to protect is just the ratchet+storage step, not
|
||||||
|
// the consumer's reaction to it.
|
||||||
|
const previous = this.decryptChains.get(from) ?? Promise.resolve();
|
||||||
|
const decryptPromise = previous
|
||||||
|
.catch(() => undefined) // don't propagate upstream failures
|
||||||
|
.then(() => this.manager.decrypt(from, envelope));
|
||||||
|
// Store a never-rejecting copy so the next chained receive doesn't
|
||||||
|
// see a rejection from this one (we still surface our own rejection
|
||||||
|
// to *this* caller via the original `decryptPromise`).
|
||||||
|
this.decryptChains.set(from, decryptPromise.catch(() => undefined));
|
||||||
|
const plaintext = await decryptPromise;
|
||||||
|
|
||||||
const consumed = await maybeHandleControlPlaintext(
|
const consumed = await maybeHandleControlPlaintext(
|
||||||
this.broadcastHooks(),
|
this.broadcastHooks(),
|
||||||
from,
|
from,
|
||||||
|
|||||||
@@ -131,6 +131,53 @@ describe('createShade — happy path', () => {
|
|||||||
await expect(alice.send('nobody', 'ghost')).rejects.toThrow();
|
await expect(alice.send('nobody', 'ghost')).rejects.toThrow();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('concurrent receive(from, env) for same `from` does not race the ratchet (V4.8.2)', async () => {
|
||||||
|
// Reproduces the Prism FR scenario: a single PUT is fanned out
|
||||||
|
// multiple times by the relay (or any duplicating transport), the
|
||||||
|
// receiver dispatches several `shade.receive(from, env)` in
|
||||||
|
// parallel, and the underlying SessionManager + StorageProvider
|
||||||
|
// would race on the ratchet (and on storage writes — sqlite throws
|
||||||
|
// "database is locked", IDB throws transaction conflicts) without
|
||||||
|
// per-`from` serialization. We pre-establish a session, then fire
|
||||||
|
// the same envelope at `bob.receive` from many concurrent callers
|
||||||
|
// and verify all of them either decrypt to the same plaintext or
|
||||||
|
// surface a benign "already-consumed" error. Crucially: no
|
||||||
|
// unhandled storage races, no ratchet corruption, and the next
|
||||||
|
// legitimate message still decrypts.
|
||||||
|
alice = await createShade({ prekeyServer: server.url, address: 'alice' });
|
||||||
|
bob = await createShade({ prekeyServer: server.url, address: 'bob' });
|
||||||
|
|
||||||
|
const env1 = await alice.send('bob', 'first');
|
||||||
|
expect(await bob.receive('alice', env1)).toBe('first');
|
||||||
|
|
||||||
|
const env2 = await alice.send('bob', 'second');
|
||||||
|
// Fan the same envelope out to 8 concurrent receives — exactly the
|
||||||
|
// shape of the relay duplicate fan-out described in the FR.
|
||||||
|
const dispatches = await Promise.allSettled(
|
||||||
|
Array.from({ length: 8 }, () => bob.receive('alice', env2)),
|
||||||
|
);
|
||||||
|
// At least one must have succeeded with the right plaintext; the
|
||||||
|
// others may legitimately reject (replay protection / OTPK
|
||||||
|
// already-consumed) but MUST NOT corrupt the ratchet or throw
|
||||||
|
// "database is locked".
|
||||||
|
const fulfilled = dispatches.filter((d) => d.status === 'fulfilled') as Array<
|
||||||
|
PromiseFulfilledResult<string>
|
||||||
|
>;
|
||||||
|
expect(fulfilled.length).toBeGreaterThan(0);
|
||||||
|
expect(fulfilled[0]!.value).toBe('second');
|
||||||
|
|
||||||
|
for (const d of dispatches) {
|
||||||
|
if (d.status === 'rejected') {
|
||||||
|
const msg = String((d.reason as Error)?.message ?? d.reason);
|
||||||
|
expect(msg).not.toMatch(/database is locked/i);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ratchet must still advance — the next legitimate message decrypts.
|
||||||
|
const env3 = await alice.send('bob', 'third');
|
||||||
|
expect(await bob.receive('alice', env3)).toBe('third');
|
||||||
|
});
|
||||||
|
|
||||||
test('verify fingerprint matches pinned identity', async () => {
|
test('verify fingerprint matches pinned identity', async () => {
|
||||||
alice = await createShade({ prekeyServer: server.url, address: 'alice' });
|
alice = await createShade({ prekeyServer: server.url, address: 'alice' });
|
||||||
bob = await createShade({ prekeyServer: server.url, address: 'bob' });
|
bob = await createShade({ prekeyServer: server.url, address: 'bob' });
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/server",
|
"name": "@shade/server",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-encrypted",
|
"name": "@shade/storage-encrypted",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-indexeddb",
|
"name": "@shade/storage-indexeddb",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-postgres",
|
"name": "@shade/storage-postgres",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-sqlite",
|
"name": "@shade/storage-sqlite",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/streams",
|
"name": "@shade/streams",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transfer",
|
"name": "@shade/transfer",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transport-bridge",
|
"name": "@shade/transport-bridge",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -951,3 +951,80 @@ describe('Sender attribution — bridge push surfaces IncomingMessage.from', ()
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ─── V4.8.2 — per-connection msgId dedup (Prism FR: duplicate fan-out) ─
|
||||||
|
|
||||||
|
describe('Bridge dedup — single PUT yields exactly one push per connection', () => {
|
||||||
|
test('WS: storming inbox.blob_stored does not duplicate frames for one msgId', async () => {
|
||||||
|
const h = await bootstrap();
|
||||||
|
try {
|
||||||
|
const received: IncomingMessage[] = [];
|
||||||
|
const bridge = new WsBridge({
|
||||||
|
baseUrl: h.baseUrl,
|
||||||
|
auth: bobAuth(h),
|
||||||
|
connectTimeoutMs: 2_000,
|
||||||
|
disableAutoReconnect: true,
|
||||||
|
});
|
||||||
|
await bridge.connect({ onMessage: (m) => received.push(m) });
|
||||||
|
try {
|
||||||
|
// One real PUT + replay the inbox.blob_stored event ten times to
|
||||||
|
// simulate any future code path (or external bug) that double-
|
||||||
|
// fires the trigger. The cursor in flushTo would already cover
|
||||||
|
// the happy case, but the per-connection LRU is the explicit
|
||||||
|
// dedup gate that survives even if cursor logic regresses.
|
||||||
|
const msgId = await putBlob(h, rand(48));
|
||||||
|
for (let i = 0; i < 10; i++) {
|
||||||
|
h.events.emit('inbox.blob_stored', {
|
||||||
|
address: 'bob',
|
||||||
|
msgId,
|
||||||
|
bytes: 48,
|
||||||
|
ttlSeconds: 60,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
await waitFor(() => received.length >= 1, 2_000);
|
||||||
|
// Give any stragglers a chance to arrive and inflate the count.
|
||||||
|
await new Promise((r) => setTimeout(r, 250));
|
||||||
|
expect(received.length).toBe(1);
|
||||||
|
expect(received[0]!.msgId).toBe(msgId);
|
||||||
|
} finally {
|
||||||
|
await bridge.disconnect();
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
h.server.stop(true);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('SSE: same dedup contract', async () => {
|
||||||
|
const h = await bootstrap();
|
||||||
|
try {
|
||||||
|
const received: IncomingMessage[] = [];
|
||||||
|
const bridge = new SseBridge({
|
||||||
|
baseUrl: h.baseUrl,
|
||||||
|
auth: bobAuth(h),
|
||||||
|
initialBackoffMs: 50,
|
||||||
|
maxBackoffMs: 200,
|
||||||
|
disableAutoReconnect: true,
|
||||||
|
});
|
||||||
|
await bridge.connect({ onMessage: (m) => received.push(m) });
|
||||||
|
try {
|
||||||
|
const msgId = await putBlob(h, rand(48));
|
||||||
|
for (let i = 0; i < 10; i++) {
|
||||||
|
h.events.emit('inbox.blob_stored', {
|
||||||
|
address: 'bob',
|
||||||
|
msgId,
|
||||||
|
bytes: 48,
|
||||||
|
ttlSeconds: 60,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
await waitFor(() => received.length >= 1, 2_000);
|
||||||
|
await new Promise((r) => setTimeout(r, 250));
|
||||||
|
expect(received.length).toBe(1);
|
||||||
|
expect(received[0]!.msgId).toBe(msgId);
|
||||||
|
} finally {
|
||||||
|
await bridge.disconnect();
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
h.server.stop(true);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transport-webrtc",
|
"name": "@shade/transport-webrtc",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transport",
|
"name": "@shade/transport",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/widgets",
|
"name": "@shade/widgets",
|
||||||
"version": "4.8.1",
|
"version": "4.8.2",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
Reference in New Issue
Block a user