Stian/Shade
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test: stop guessing port numbers

Browse Source 
Twenty test servers picked a port with `NNNN + Math.floor(Math.random() * 500)`
and hoped it was free. Several files guessed inside the *same* range —
sdk.test.ts and gates.test.ts both on 19500, three files on 22000, and
webrtc-integration.test.ts used 22000 twice within itself — so with test files
serving in parallel the collision was not unlikely, it was scheduled.

Found from the outside, in Nova, where Shade is vendored: across eight runs of
an unchanged tree, two had failures and six were clean. The visible error was

    Failed to start server. Is port 24287 in use?  EADDRINUSE

and then a second, louder one: `teardownRig` ran on a rig that setup had never
finished building and died on `rig.alice`, so the TypeError from cleanup is
what you read first and the real cause scrolled past above it.

`Bun.serve({ port: 0 })` lets the OS hand out a free port and `server.port`
reads it back. There is no range to collide inside.

teardownRig now tolerates a rig that was never built. Cleanup must never be the
loudest thing in a failing test.

A suite that fails a quarter of the time is worse than a suite that fails: it
teaches everyone to re-run it, and then a real regression looks like the flake.

Verified: 1198 pass / 0 fail here, and the same fix is going into Nova's
vendored copy so the next sync does not bring the guessing back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014489bKUtUEY1Zgs9xN9mt7
This commit is contained in:
Sterister
2026-09-08 16:33:40 +02:00
parent 3c69995097
commit 53ce776c02
12 changed files with 132 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
packages/shade-sdk/tests/broadcast.test.ts
View File

@@ -14,8 +14,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
store: new MemoryPrekeyStore(), store: new MemoryPrekeyStore(),
disableRateLimit: true, disableRateLimit: true,
}); });
const port = 19500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }

8
packages/shade-sdk/tests/gates.test.ts
View File

@@ -12,8 +12,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
store: new MemoryPrekeyStore(), store: new MemoryPrekeyStore(),
disableRateLimit: true, disableRateLimit: true,
}); });
const port = 19500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }

8
packages/shade-sdk/tests/profile.test.ts
View File

@@ -36,8 +36,12 @@ async function startServer(): Promise<ServerHandle> {
blobStore: new MemoryBlobStore(), blobStore: new MemoryBlobStore(),
disableRateLimit: true, disableRateLimit: true,
}); });
const port = 19000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: app.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: app.fetch });
const port = handle.port;
return { return {
url: `http://localhost:${port}`, url: `http://localhost:${port}`,
stop: () => handle.stop(true), stop: () => handle.stop(true),

8
packages/shade-sdk/tests/sdk.test.ts
View File

@@ -25,8 +25,12 @@ async function startPrekeyServer(): Promise<{
disableRateLimit: true, disableRateLimit: true,
events, events,
}); });
const port = 19500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { return {
url: `http://localhost:${port}`, url: `http://localhost:${port}`,
stop: () => handle.stop(), stop: () => handle.stop(),

16
packages/shade-sdk/tests/streams-integration.test.ts
View File

@@ -26,8 +26,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
disableRateLimit: true, disableRateLimit: true,
events, events,
}); });
const port = 21000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }
@@ -46,8 +50,12 @@ async function setupRig(): Promise<TestRig> {
// Spin up Bob's HTTP transfer endpoint. // Spin up Bob's HTTP transfer endpoint.
const bobApp = await bob.transferRoute(); const bobApp = await bob.transferRoute();
const port = 21500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const bobServer = Bun.serve({ port, fetch: bobApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
const port = bobServer.port;
const bobBaseUrl = `http://localhost:${port}`; const bobBaseUrl = `http://localhost:${port}`;
// Wire up Alice's outgoing transfer routing. // Wire up Alice's outgoing transfer routing.

16
packages/shade-sdk/tests/thumbnail.test.ts
View File

@@ -33,8 +33,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
disableRateLimit: true, disableRateLimit: true,
events, events,
}); });
const port = 22000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }
@@ -49,8 +53,12 @@ async function setupRig(): Promise<TestRig> {
}, },
}); });
const bobApp = await bob.transferRoute(); const bobApp = await bob.transferRoute();
const port = 22500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const bobServer = Bun.serve({ port, fetch: bobApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
const port = bobServer.port;
const bobBaseUrl = `http://localhost:${port}`; const bobBaseUrl = `http://localhost:${port}`;
alice.configureTransfers({ alice.configureTransfers({

42
packages/shade-sdk/tests/webrtc-failover.test.ts
View File

@@ -103,8 +103,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
disableRateLimit: true, disableRateLimit: true,
events, events,
}); });
const port = 23000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }
@@ -127,13 +131,21 @@ async function setupRig(connectTimeoutMs: number): Promise<Rig> {
bob.configureWebRTC({ factory: broken, connectTimeoutMs }); bob.configureWebRTC({ factory: broken, connectTimeoutMs });
const bobApp = await bob.transferRoute(); const bobApp = await bob.transferRoute();
const bobPort = 23500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const bobServer = Bun.serve({ port: bobPort, fetch: bobApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
const bobPort = bobServer.port;
baseUrls.set('bob', `http://localhost:${bobPort}`); baseUrls.set('bob', `http://localhost:${bobPort}`);
const aliceApp = await alice.transferRoute(); const aliceApp = await alice.transferRoute();
const alicePort = 24000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const aliceServer = Bun.serve({ port: alicePort, fetch: aliceApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const aliceServer = Bun.serve({ port: 0, fetch: aliceApp.fetch });
const alicePort = aliceServer.port;
baseUrls.set('alice', `http://localhost:${alicePort}`); baseUrls.set('alice', `http://localhost:${alicePort}`);
return { return {
@@ -145,12 +157,18 @@ async function setupRig(connectTimeoutMs: number): Promise<Rig> {
}; };
} }
async function teardownRig(rig: Rig): Promise<void> { async function teardownRig(rig: Rig | undefined): Promise<void> {
await rig.alice.shutdown(); // Tolerates a rig that never finished being built. When setupRig threw —
await rig.bob.shutdown(); // EADDRINUSE, before ports stopped being guessed — this ran anyway and died
rig.bobServerStop(); // on `rig.alice`, so one failure became two and the second one, a TypeError
rig.aliceServerStop(); // in teardown, is what you saw first. Cleanup must never be the loudest
rig.prekeyStop(); // thing in a failing test.
if (!rig) return;
await rig.alice?.shutdown().catch(() => {});
await rig.bob?.shutdown().catch(() => {});
try { rig.bobServerStop?.(); } catch { /* already gone */ }
try { rig.aliceServerStop?.(); } catch { /* already gone */ }
try { rig.prekeyStop?.(); } catch { /* already gone */ }
} }
function hex(b: Uint8Array): string { function hex(b: Uint8Array): string {

24
packages/shade-sdk/tests/webrtc-integration.test.ts
View File

@@ -40,8 +40,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
disableRateLimit: true, disableRateLimit: true,
events, events,
}); });
const port = 22000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }
@@ -70,13 +74,21 @@ async function setupRig(): Promise<Rig> {
bob.configureWebRTC({ factory, connectTimeoutMs: 10_000 }); bob.configureWebRTC({ factory, connectTimeoutMs: 10_000 });
const bobApp = await bob.transferRoute(); const bobApp = await bob.transferRoute();
const bobPort = 22500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const bobServer = Bun.serve({ port: bobPort, fetch: bobApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
const bobPort = bobServer.port;
const bobBaseUrl = `http://localhost:${bobPort}`; const bobBaseUrl = `http://localhost:${bobPort}`;
const aliceApp = await alice.transferRoute(); const aliceApp = await alice.transferRoute();
const alicePort = 22000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const aliceServer = Bun.serve({ port: alicePort, fetch: aliceApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const aliceServer = Bun.serve({ port: 0, fetch: aliceApp.fetch });
const alicePort = aliceServer.port;
const aliceBaseUrl = `http://localhost:${alicePort}`; const aliceBaseUrl = `http://localhost:${alicePort}`;
baseUrls.set('alice', aliceBaseUrl); baseUrls.set('alice', aliceBaseUrl);

24
packages/shade-sdk/tests/webrtc-throughput.test.ts
View File

@@ -44,8 +44,12 @@ async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
disableRateLimit: true, disableRateLimit: true,
events, events,
}); });
const port = 24500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: server.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: server.fetch });
const port = handle.port;
return { url: `http://localhost:${port}`, stop: () => handle.stop() }; return { url: `http://localhost:${port}`, stop: () => handle.stop() };
} }
@@ -70,13 +74,21 @@ async function setupRig(opts: { withWebRTC: boolean }): Promise<Rig> {
} }
const bobApp = await bob.transferRoute(); const bobApp = await bob.transferRoute();
const bobPort = 25000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const bobServer = Bun.serve({ port: bobPort, fetch: bobApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
const bobPort = bobServer.port;
baseUrls.set('bob', `http://localhost:${bobPort}`); baseUrls.set('bob', `http://localhost:${bobPort}`);
const aliceApp = await alice.transferRoute(); const aliceApp = await alice.transferRoute();
const alicePort = 25500 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const aliceServer = Bun.serve({ port: alicePort, fetch: aliceApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const aliceServer = Bun.serve({ port: 0, fetch: aliceApp.fetch });
const alicePort = aliceServer.port;
baseUrls.set('alice', `http://localhost:${alicePort}`); baseUrls.set('alice', `http://localhost:${alicePort}`);
return { return {

8
packages/shade-transfer/tests/resume.test.ts
View File

@@ -41,8 +41,12 @@ describe('Resume protocol — kill-restart-verify', () => {
myAddress: 'bob', myAddress: 'bob',
}); });
const receiverApp = await createTransferRoutes(receiverEngine); const receiverApp = await createTransferRoutes(receiverEngine);
const port = 22000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const server = Bun.serve({ port, fetch: receiverApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const server = Bun.serve({ port: 0, fetch: receiverApp.fetch });
const port = server.port;
const baseUrl = `http://localhost:${port}`; const baseUrl = `http://localhost:${port}`;
// Receiver accepts incoming. // Receiver accepts incoming.

8
packages/shade-transfer/tests/ws-fallback.test.ts
View File

@@ -39,8 +39,12 @@ describe('WS opt-in transport with HTTP fallback', () => {
myAddress: 'bob', myAddress: 'bob',
}); });
const httpApp = await createTransferRoutes(receiverEngine); const httpApp = await createTransferRoutes(receiverEngine);
const httpPort = 23000 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const httpServer = Bun.serve({ port: httpPort, fetch: httpApp.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const httpServer = Bun.serve({ port: 0, fetch: httpApp.fetch });
const httpPort = httpServer.port;
const httpBaseUrl = `http://localhost:${httpPort}`; const httpBaseUrl = `http://localhost:${httpPort}`;
cleanups.push(() => httpServer.stop()); cleanups.push(() => httpServer.stop());

8
packages/shade-transport/tests/kt-transport.test.ts
View File

@@ -31,8 +31,12 @@ describe('ShadeFetchTransport with KT verifier', () => {
}, },
}); });
const port = 20100 + Math.floor(Math.random() * 500); // Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
const handle = Bun.serve({ port, fetch: app.fetch }); // range and hoped — and several files here guessed inside the SAME range,
// so with test files serving in parallel the collision arrived about a
// quarter of the time as EADDRINUSE.
const handle = Bun.serve({ port: 0, fetch: app.fetch });
const port = handle.port;
try { try {
const baseUrl = `http://localhost:${port}`; const baseUrl = `http://localhost:${port}`;