release(v4.8.2): per-from receive serialization + per-connection bridge dedup
Some checks failed
Test / test (push) Has been cancelled
Some checks failed
Test / test (push) Has been cancelled
Two interlocking robustness fixes for the duplicate-fan-out / first-contact
class of failures Prism reported.
1. `Shade.receive(from, env)` now queues its `manager.decrypt` step
per `from` so concurrent dispatches can't race the SessionManager
ratchet or the StorageProvider (sqlite "database is locked", IDB
transaction conflicts). User message handlers run *outside* the
queue so streams + file-RPC's nested `shade.receive` calls don't
self-deadlock.
2. Bridge WS + SSE handlers now run a per-connection bounded msgId
LRU as defense-in-depth against any flushTo re-entry (event-storm,
future refactor). Pending-flush chains are wrapped in `.catch(() =>
{})` so a transient `ws.send` rejection no longer poisons the
connection's flush loop.
Tests: storming `inbox.blob_stored` 10× per PUT yields exactly one WS/
SSE frame; 8 concurrent `bob.receive('alice', envelope)` calls keep
the ratchet intact and never surface "database is locked".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@shade/sdk",
|
||||
"version": "4.8.1",
|
||||
"version": "4.8.2",
|
||||
"type": "module",
|
||||
"main": "src/index.ts",
|
||||
"types": "src/index.ts",
|
||||
|
||||
@@ -153,6 +153,13 @@ export class Shade {
|
||||
private establishing = new Map<string, Promise<void>>();
|
||||
// Per-address encrypt queue to serialize ratchet mutations
|
||||
private encryptChains = new Map<string, Promise<unknown>>();
|
||||
// Per-`from` decrypt queue: serializes incoming receives so two concurrent
|
||||
// shade.receive(from, env) calls can't race the ratchet/storage. Without
|
||||
// this, parallel deliveries (relay duplicate fan-out, fast pipelined
|
||||
// sends) hit `database is locked` (sqlite) or transaction conflicts (IDB)
|
||||
// because the underlying StorageProvider isn't required to be a
|
||||
// concurrent-safe writer. See V4.8.2 changelog.
|
||||
private decryptChains = new Map<string, Promise<unknown>>();
|
||||
|
||||
// Message handlers — may be sync or async; receive() awaits each. The
|
||||
// optional third arg distinguishes direct vs broadcast plaintexts;
|
||||
@@ -436,7 +443,35 @@ export class Shade {
|
||||
*/
|
||||
async receive(from: string, envelope: ShadeEnvelope): Promise<string> {
|
||||
if (!this.initialized) throw new Error('Not initialized');
|
||||
const plaintext = await this.manager.decrypt(from, envelope);
|
||||
|
||||
// Serialize ONLY the ratchet/storage write portion of receive (the
|
||||
// call into `manager.decrypt`). Concurrent decrypts race the
|
||||
// SessionManager ratchet (mutated in place) and the StorageProvider
|
||||
// (not required to be a concurrent-safe writer — `bun:sqlite`
|
||||
// throws `database is locked`, IDB throws transaction conflicts).
|
||||
// The Prism FR called this out: a relay-duplicated WS fan-out
|
||||
// dispatched 8 parallel `shade.receive(from, env)` calls, one won
|
||||
// the X3DH prekey race and the other 7 failed with
|
||||
// `database is locked` / `one-time prekey not found`. The fix is
|
||||
// to queue per-`from` decrypts so the ratchet step is sequential.
|
||||
//
|
||||
// Crucially the user-visible MESSAGE HANDLERS run *outside* the
|
||||
// queue. Streams + file-RPC issue nested `shade.receive` calls for
|
||||
// the same peer from inside their handlers (e.g. `stream-end`
|
||||
// arrives while a write-RPC is still waiting on chunks); holding
|
||||
// the queue across the handler would self-deadlock. The atomic
|
||||
// unit we have to protect is just the ratchet+storage step, not
|
||||
// the consumer's reaction to it.
|
||||
const previous = this.decryptChains.get(from) ?? Promise.resolve();
|
||||
const decryptPromise = previous
|
||||
.catch(() => undefined) // don't propagate upstream failures
|
||||
.then(() => this.manager.decrypt(from, envelope));
|
||||
// Store a never-rejecting copy so the next chained receive doesn't
|
||||
// see a rejection from this one (we still surface our own rejection
|
||||
// to *this* caller via the original `decryptPromise`).
|
||||
this.decryptChains.set(from, decryptPromise.catch(() => undefined));
|
||||
const plaintext = await decryptPromise;
|
||||
|
||||
const consumed = await maybeHandleControlPlaintext(
|
||||
this.broadcastHooks(),
|
||||
from,
|
||||
|
||||
@@ -131,6 +131,53 @@ describe('createShade — happy path', () => {
|
||||
await expect(alice.send('nobody', 'ghost')).rejects.toThrow();
|
||||
});
|
||||
|
||||
test('concurrent receive(from, env) for same `from` does not race the ratchet (V4.8.2)', async () => {
|
||||
// Reproduces the Prism FR scenario: a single PUT is fanned out
|
||||
// multiple times by the relay (or any duplicating transport), the
|
||||
// receiver dispatches several `shade.receive(from, env)` in
|
||||
// parallel, and the underlying SessionManager + StorageProvider
|
||||
// would race on the ratchet (and on storage writes — sqlite throws
|
||||
// "database is locked", IDB throws transaction conflicts) without
|
||||
// per-`from` serialization. We pre-establish a session, then fire
|
||||
// the same envelope at `bob.receive` from many concurrent callers
|
||||
// and verify all of them either decrypt to the same plaintext or
|
||||
// surface a benign "already-consumed" error. Crucially: no
|
||||
// unhandled storage races, no ratchet corruption, and the next
|
||||
// legitimate message still decrypts.
|
||||
alice = await createShade({ prekeyServer: server.url, address: 'alice' });
|
||||
bob = await createShade({ prekeyServer: server.url, address: 'bob' });
|
||||
|
||||
const env1 = await alice.send('bob', 'first');
|
||||
expect(await bob.receive('alice', env1)).toBe('first');
|
||||
|
||||
const env2 = await alice.send('bob', 'second');
|
||||
// Fan the same envelope out to 8 concurrent receives — exactly the
|
||||
// shape of the relay duplicate fan-out described in the FR.
|
||||
const dispatches = await Promise.allSettled(
|
||||
Array.from({ length: 8 }, () => bob.receive('alice', env2)),
|
||||
);
|
||||
// At least one must have succeeded with the right plaintext; the
|
||||
// others may legitimately reject (replay protection / OTPK
|
||||
// already-consumed) but MUST NOT corrupt the ratchet or throw
|
||||
// "database is locked".
|
||||
const fulfilled = dispatches.filter((d) => d.status === 'fulfilled') as Array<
|
||||
PromiseFulfilledResult<string>
|
||||
>;
|
||||
expect(fulfilled.length).toBeGreaterThan(0);
|
||||
expect(fulfilled[0]!.value).toBe('second');
|
||||
|
||||
for (const d of dispatches) {
|
||||
if (d.status === 'rejected') {
|
||||
const msg = String((d.reason as Error)?.message ?? d.reason);
|
||||
expect(msg).not.toMatch(/database is locked/i);
|
||||
}
|
||||
}
|
||||
|
||||
// Ratchet must still advance — the next legitimate message decrypts.
|
||||
const env3 = await alice.send('bob', 'third');
|
||||
expect(await bob.receive('alice', env3)).toBe('third');
|
||||
});
|
||||
|
||||
test('verify fingerprint matches pinned identity', async () => {
|
||||
alice = await createShade({ prekeyServer: server.url, address: 'alice' });
|
||||
bob = await createShade({ prekeyServer: server.url, address: 'bob' });
|
||||
|
||||
Reference in New Issue
Block a user