release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
/**
|
|
|
|
|
* V3.11 — full SDK integration: two Shade instances exchange a transfer
|
|
|
|
|
* over the in-process `MemoryRtcFactory`. The WebRTC transport sits on
|
|
|
|
|
* top of `MultiTransportFallback([webrtc, http])`, so this also verifies
|
|
|
|
|
* the SDK wires the fallback chain correctly.
|
|
|
|
|
*/
|
|
|
|
|
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
|
|
|
|
import {
|
|
|
|
|
createShade,
|
|
|
|
|
type Shade,
|
|
|
|
|
type TransferHandle,
|
|
|
|
|
type TransferResult,
|
|
|
|
|
} from '../src/index.js';
|
|
|
|
|
import {
|
|
|
|
|
createPrekeyServer,
|
|
|
|
|
MemoryPrekeyStore,
|
|
|
|
|
PrekeyServerEvents,
|
|
|
|
|
} from '@shade/server';
|
|
|
|
|
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
|
|
|
|
import { sha256Once } from '@shade/streams';
|
|
|
|
|
import { MemoryRtcFactory } from '@shade/transport-webrtc';
|
|
|
|
|
|
|
|
|
|
const crypto = new SubtleCryptoProvider();
|
|
|
|
|
|
|
|
|
|
interface Rig {
|
|
|
|
|
alice: Shade;
|
|
|
|
|
bob: Shade;
|
|
|
|
|
aliceBaseUrl: string;
|
|
|
|
|
bobBaseUrl: string;
|
|
|
|
|
prekeyStop: () => void;
|
|
|
|
|
aliceServerStop: () => void;
|
|
|
|
|
bobServerStop: () => void;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
async function startPrekeyServer(): Promise<{ url: string; stop: () => void }> {
|
|
|
|
|
const events = new PrekeyServerEvents();
|
|
|
|
|
const server = createPrekeyServer({
|
|
|
|
|
crypto,
|
|
|
|
|
store: new MemoryPrekeyStore(),
|
|
|
|
|
disableRateLimit: true,
|
|
|
|
|
events,
|
|
|
|
|
});
|
test: stop guessing port numbers
Twenty test servers picked a port with `NNNN + Math.floor(Math.random() * 500)`
and hoped it was free. Several files guessed inside the *same* range —
sdk.test.ts and gates.test.ts both on 19500, three files on 22000, and
webrtc-integration.test.ts used 22000 twice within itself — so with test files
serving in parallel the collision was not unlikely, it was scheduled.
Found from the outside, in Nova, where Shade is vendored: across eight runs of
an unchanged tree, two had failures and six were clean. The visible error was
Failed to start server. Is port 24287 in use? EADDRINUSE
and then a second, louder one: `teardownRig` ran on a rig that setup had never
finished building and died on `rig.alice`, so the TypeError from cleanup is
what you read first and the real cause scrolled past above it.
`Bun.serve({ port: 0 })` lets the OS hand out a free port and `server.port`
reads it back. There is no range to collide inside.
teardownRig now tolerates a rig that was never built. Cleanup must never be the
loudest thing in a failing test.
A suite that fails a quarter of the time is worse than a suite that fails: it
teaches everyone to re-run it, and then a real regression looks like the flake.
Verified: 1198 pass / 0 fail here, and the same fix is going into Nova's
vendored copy so the next sync does not bring the guessing back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014489bKUtUEY1Zgs9xN9mt7
2026-09-08 16:33:40 +02:00
|
|
|
// Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
|
|
|
|
|
// range and hoped — and several files here guessed inside the SAME range,
|
|
|
|
|
// so with test files serving in parallel the collision arrived about a
|
|
|
|
|
// quarter of the time as EADDRINUSE.
|
|
|
|
|
const handle = Bun.serve({ port: 0, fetch: server.fetch });
|
|
|
|
|
const port = handle.port;
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
return { url: `http://localhost:${port}`, stop: () => handle.stop() };
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
async function setupRig(): Promise<Rig> {
|
|
|
|
|
const prekey = await startPrekeyServer();
|
|
|
|
|
const alice = await createShade({ prekeyServer: prekey.url, address: 'alice' });
|
|
|
|
|
const bob = await createShade({ prekeyServer: prekey.url, address: 'bob' });
|
|
|
|
|
|
|
|
|
|
// Both peers need bidirectional resolveBaseUrl since signaling envelopes
|
|
|
|
|
// ride the control plane in BOTH directions (offer one way, answer
|
|
|
|
|
// back). Static map for this test rig.
|
|
|
|
|
const baseUrls = new Map<string, string>();
|
|
|
|
|
const resolveBaseUrl = async (addr: string): Promise<string> => {
|
|
|
|
|
const url = baseUrls.get(addr);
|
|
|
|
|
if (url === undefined) throw new Error(`unknown peer ${addr}`);
|
|
|
|
|
return url;
|
|
|
|
|
};
|
|
|
|
|
alice.configureTransfers({ resolveBaseUrl });
|
|
|
|
|
bob.configureTransfers({ resolveBaseUrl });
|
|
|
|
|
|
|
|
|
|
// V3.11: opt-in to WebRTC BEFORE the engine is built (transferRoute
|
|
|
|
|
// builds it lazily). Both peers use the same in-process factory so
|
|
|
|
|
// their PCs can pair up via the registry.
|
|
|
|
|
const factory = new MemoryRtcFactory();
|
|
|
|
|
alice.configureWebRTC({ factory, connectTimeoutMs: 10_000 });
|
|
|
|
|
bob.configureWebRTC({ factory, connectTimeoutMs: 10_000 });
|
|
|
|
|
|
|
|
|
|
const bobApp = await bob.transferRoute();
|
test: stop guessing port numbers
Twenty test servers picked a port with `NNNN + Math.floor(Math.random() * 500)`
and hoped it was free. Several files guessed inside the *same* range —
sdk.test.ts and gates.test.ts both on 19500, three files on 22000, and
webrtc-integration.test.ts used 22000 twice within itself — so with test files
serving in parallel the collision was not unlikely, it was scheduled.
Found from the outside, in Nova, where Shade is vendored: across eight runs of
an unchanged tree, two had failures and six were clean. The visible error was
Failed to start server. Is port 24287 in use? EADDRINUSE
and then a second, louder one: `teardownRig` ran on a rig that setup had never
finished building and died on `rig.alice`, so the TypeError from cleanup is
what you read first and the real cause scrolled past above it.
`Bun.serve({ port: 0 })` lets the OS hand out a free port and `server.port`
reads it back. There is no range to collide inside.
teardownRig now tolerates a rig that was never built. Cleanup must never be the
loudest thing in a failing test.
A suite that fails a quarter of the time is worse than a suite that fails: it
teaches everyone to re-run it, and then a real regression looks like the flake.
Verified: 1198 pass / 0 fail here, and the same fix is going into Nova's
vendored copy so the next sync does not bring the guessing back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014489bKUtUEY1Zgs9xN9mt7
2026-09-08 16:33:40 +02:00
|
|
|
// Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
|
|
|
|
|
// range and hoped — and several files here guessed inside the SAME range,
|
|
|
|
|
// so with test files serving in parallel the collision arrived about a
|
|
|
|
|
// quarter of the time as EADDRINUSE.
|
|
|
|
|
const bobServer = Bun.serve({ port: 0, fetch: bobApp.fetch });
|
|
|
|
|
const bobPort = bobServer.port;
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
const bobBaseUrl = `http://localhost:${bobPort}`;
|
|
|
|
|
|
|
|
|
|
const aliceApp = await alice.transferRoute();
|
test: stop guessing port numbers
Twenty test servers picked a port with `NNNN + Math.floor(Math.random() * 500)`
and hoped it was free. Several files guessed inside the *same* range —
sdk.test.ts and gates.test.ts both on 19500, three files on 22000, and
webrtc-integration.test.ts used 22000 twice within itself — so with test files
serving in parallel the collision was not unlikely, it was scheduled.
Found from the outside, in Nova, where Shade is vendored: across eight runs of
an unchanged tree, two had failures and six were clean. The visible error was
Failed to start server. Is port 24287 in use? EADDRINUSE
and then a second, louder one: `teardownRig` ran on a rig that setup had never
finished building and died on `rig.alice`, so the TypeError from cleanup is
what you read first and the real cause scrolled past above it.
`Bun.serve({ port: 0 })` lets the OS hand out a free port and `server.port`
reads it back. There is no range to collide inside.
teardownRig now tolerates a rig that was never built. Cleanup must never be the
loudest thing in a failing test.
A suite that fails a quarter of the time is worse than a suite that fails: it
teaches everyone to re-run it, and then a real regression looks like the flake.
Verified: 1198 pass / 0 fail here, and the same fix is going into Nova's
vendored copy so the next sync does not bring the guessing back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014489bKUtUEY1Zgs9xN9mt7
2026-09-08 16:33:40 +02:00
|
|
|
// Port 0 lets the OS hand out a free one. This guessed inside a 500-wide
|
|
|
|
|
// range and hoped — and several files here guessed inside the SAME range,
|
|
|
|
|
// so with test files serving in parallel the collision arrived about a
|
|
|
|
|
// quarter of the time as EADDRINUSE.
|
|
|
|
|
const aliceServer = Bun.serve({ port: 0, fetch: aliceApp.fetch });
|
|
|
|
|
const alicePort = aliceServer.port;
|
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.
Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
non-realtime stack.
Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
version 4.0.0 on /health.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 18:35:35 +02:00
|
|
|
const aliceBaseUrl = `http://localhost:${alicePort}`;
|
|
|
|
|
|
|
|
|
|
baseUrls.set('alice', aliceBaseUrl);
|
|
|
|
|
baseUrls.set('bob', bobBaseUrl);
|
|
|
|
|
|
|
|
|
|
return {
|
|
|
|
|
alice,
|
|
|
|
|
bob,
|
|
|
|
|
aliceBaseUrl,
|
|
|
|
|
bobBaseUrl,
|
|
|
|
|
prekeyStop: prekey.stop,
|
|
|
|
|
aliceServerStop: () => aliceServer.stop(),
|
|
|
|
|
bobServerStop: () => bobServer.stop(),
|
|
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
async function teardownRig(rig: Rig): Promise<void> {
|
|
|
|
|
await rig.alice.shutdown();
|
|
|
|
|
await rig.bob.shutdown();
|
|
|
|
|
rig.bobServerStop();
|
|
|
|
|
rig.aliceServerStop();
|
|
|
|
|
rig.prekeyStop();
|
|
|
|
|
MemoryRtcFactory.reset();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function hex(b: Uint8Array): string {
|
|
|
|
|
return Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
async function uploadAndAwait(
|
|
|
|
|
rig: Rig,
|
|
|
|
|
input: Uint8Array,
|
|
|
|
|
opts?: { lanes?: number; chunkSize?: number },
|
|
|
|
|
): Promise<{ senderResult: TransferResult; received: Uint8Array }> {
|
|
|
|
|
let resolveRecv!: (h: TransferHandle) => void;
|
|
|
|
|
const recvHandlePromise = new Promise<TransferHandle>((r) => {
|
|
|
|
|
resolveRecv = r;
|
|
|
|
|
});
|
|
|
|
|
const unsubscribe = await rig.bob.onIncomingTransfer(async (incoming) => {
|
|
|
|
|
const h = await incoming.accept({ output: { kind: 'buffer' } });
|
|
|
|
|
resolveRecv(h);
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
const handle = await rig.alice.upload({
|
|
|
|
|
to: 'bob',
|
|
|
|
|
input,
|
|
|
|
|
...(opts?.lanes !== undefined ? { lanes: opts.lanes } : {}),
|
|
|
|
|
...(opts?.chunkSize !== undefined ? { chunkSize: opts.chunkSize } : {}),
|
|
|
|
|
metadata: { name: 'webrtc-test.bin' },
|
|
|
|
|
});
|
|
|
|
|
const recvHandle = await recvHandlePromise;
|
|
|
|
|
const [senderResult, recvResult] = await Promise.all([
|
|
|
|
|
handle.done(),
|
|
|
|
|
recvHandle.done(),
|
|
|
|
|
]);
|
|
|
|
|
unsubscribe();
|
|
|
|
|
const received =
|
|
|
|
|
(recvResult as TransferResult & { bytes?: Uint8Array }).bytes ?? new Uint8Array();
|
|
|
|
|
return { senderResult, received };
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
describe('V3.11 WebRTC integration via MemoryRtcFactory', () => {
|
|
|
|
|
let rig: Rig;
|
|
|
|
|
beforeAll(async () => {
|
|
|
|
|
rig = await setupRig();
|
|
|
|
|
});
|
|
|
|
|
afterAll(async () => {
|
|
|
|
|
await teardownRig(rig);
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
test('256 KiB payload over WebRTC primary', async () => {
|
|
|
|
|
const input = crypto.randomBytes(256 * 1024);
|
|
|
|
|
const { senderResult, received } = await uploadAndAwait(rig, input, {
|
|
|
|
|
lanes: 1,
|
|
|
|
|
chunkSize: 64 * 1024,
|
|
|
|
|
});
|
|
|
|
|
expect(received).toEqual(input);
|
|
|
|
|
expect(senderResult.sha256).toBe(hex(sha256Once(input)));
|
|
|
|
|
|
|
|
|
|
// Verify the WebRTC runtime is alive and the multi-fallback hasn't
|
|
|
|
|
// demoted away from webrtc.
|
|
|
|
|
const runtime = rig.alice.getWebRtcRuntime();
|
|
|
|
|
expect(runtime).not.toBeNull();
|
|
|
|
|
expect(runtime!.fallback.activeName).toBe('webrtc');
|
|
|
|
|
expect(runtime!.fallback.hasFallenBack).toBe(false);
|
|
|
|
|
expect(runtime!.manager.isConnected('bob')).toBe(true);
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
test('1 MiB payload — 4 lanes range partition over WebRTC', async () => {
|
|
|
|
|
const input = crypto.randomBytes(1024 * 1024);
|
|
|
|
|
const { received } = await uploadAndAwait(rig, input, {
|
|
|
|
|
lanes: 4,
|
|
|
|
|
chunkSize: 64 * 1024,
|
|
|
|
|
});
|
|
|
|
|
expect(received).toEqual(input);
|
|
|
|
|
|
|
|
|
|
const runtime = rig.alice.getWebRtcRuntime();
|
|
|
|
|
expect(runtime!.fallback.activeName).toBe('webrtc');
|
|
|
|
|
});
|
|
|
|
|
});
|