release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled

V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.

Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
  Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
  bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
  shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
  non-realtime stack.

Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
  version 4.0.0 on /health.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-03 18:35:35 +02:00
parent 8b055912b7
commit e6fdf31b49
298 changed files with 37909 additions and 256 deletions

View File

@@ -42,10 +42,12 @@ EXPOSE 3900
# Defaults — override via `docker run -e`
ENV SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
ENV SHADE_INBOX_DB_PATH=/data/shade-inbox.db
ENV PORT=3900
ENV SHADE_LOG_LEVEL=info
ENV SHADE_STALE_DAYS=30
ENV SHADE_CLEANUP_INTERVAL_HOURS=24
ENV SHADE_INBOX_PRUNE_INTERVAL_MINUTES=5
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl -fsS http://localhost:${PORT}/health || exit 1

View File

@@ -34,6 +34,45 @@ future-project (Docker container, SQLite volume) ← Any future app
Each project owns its own container, its own volume, its own observer token. Zero cross-project coupling. If one project's Shade is down, the others keep running.
## Keys vs. payloads — what this server is, and isn't
The prekey server is a **public-key directory**. It exists so a brand-
new client can find the right Ed25519 + X25519 bundle to start an
X3DH handshake with a peer it has never talked to. After that, the
peers ratchet directly.
What lives on this server:
- Identity public keys
- Signed prekey + one-time prekey bundles
- Activity timestamps (used by stale cleanup)
- Operator metadata: `/health`, `/metrics`, `/openapi.yaml`,
`/shade-observer/*`
What never lives on this server:
- **Message plaintext.** Ratchet envelopes flow peer-to-peer.
- **Transfer chunks.** `@shade/transfer` POSTs ciphertext directly to
the receiver's `/v1/transfer/:streamId/chunk` route — not here.
- **Identity private keys** or **session state**. Both are device-
local.
- **Resume secrets** for in-flight transfers. Encrypted under a
device-key derived from the identity signing key, never uploaded.
This is the bright line that lets you deploy one shared prekey
container per project even when consumer apps don't trust each other:
the worst a compromised prekey server can do is hand out a fake
bundle (MITM at first contact). Out-of-band fingerprint comparison
detects this — see `THREAT-MODEL.md § 2` and the `getIdentityFingerprint()`
API.
For deployment-time gates (TLS, backup, observer-token rotation, log
level, secret rotation) see
[`docs/PRODUCTION-CHECKLIST.md`](../../docs/PRODUCTION-CHECKLIST.md).
For the wire contract — including the peer-served
`/v1/transfer/*` and `ShadeTransferAuthenticator` security scheme —
see [`openapi.yaml`](./openapi.yaml).
## Environment variables
| Var | Default | Description |

View File

@@ -9,7 +9,7 @@ info:
**Security model:** Write operations (register, replenish, delete) are
authenticated by Ed25519 signatures over the request body. Bundle fetches
are anonymous. See the `SignedPayload` schema for the signing format.
version: "1.0.0"
version: "4.0.0"
license:
name: MIT
url: https://gt.zyon.no/Stian/Shade/raw/branch/main/LICENSE
@@ -223,6 +223,583 @@ paths:
'401':
$ref: '#/components/responses/Unauthorized'
/v1/kt/log_id:
get:
summary: Key Transparency log identity (V3.12)
description: |
Returns the operator's STH-signing public key and derived log_id
(`SHA-256(public_key)`). Clients fetch this once during bootstrap
to confirm they pinned the right key out-of-band. Available when
the server is started with `keyTransparency` configured.
tags: [KeyTransparency]
responses:
'200':
description: Log identity
content:
application/json:
schema:
type: object
required: [logId, publicKey]
properties:
logId: { type: string, description: 'base64 of sha256(publicKey)' }
publicKey: { type: string, description: 'base64 Ed25519 public key (32 bytes)' }
/v1/kt/sth:
get:
summary: Latest Signed Tree Head (V3.12)
tags: [KeyTransparency]
responses:
'200':
description: Most recent STH
content:
application/json:
schema:
$ref: '#/components/schemas/STH'
/v1/kt/sth/{treeSize}:
get:
summary: Historical STH at a specific tree size (V3.12)
tags: [KeyTransparency]
parameters:
- name: treeSize
in: path
required: true
schema:
type: integer
minimum: 0
responses:
'200':
description: STH at requested tree_size
content:
application/json:
schema:
$ref: '#/components/schemas/STH'
'404':
description: No STH at that tree_size
/v1/kt/consistency:
get:
summary: Consistency proof between two tree sizes (V3.12)
description: |
RFC 6962 §2.1.2 consistency proof from `from` to `to`. Used by
clients and witnesses to verify that the log between two STHs
is an honest extension, not a re-write.
tags: [KeyTransparency]
parameters:
- name: from
in: query
required: true
schema: { type: integer, minimum: 0 }
- name: to
in: query
required: false
schema: { type: integer, minimum: 0 }
responses:
'200':
description: Consistency proof
content:
application/json:
schema:
type: object
required: [fromTreeSize, toTreeSize, proof]
properties:
fromTreeSize: { type: integer }
toTreeSize: { type: integer }
proof:
type: array
items: { type: string, description: 'base64 of 32-byte node hash' }
/v1/transfer/health:
get:
summary: Peer transfer-route reachability probe
description: |
Lightweight probe served by **the receiver-side peer** (mounted via
`Shade.transferRoute()` or `createTransferRoutes()`). The prekey
server itself does not serve `/v1/transfer/*` — it is documented
here so any-language clients can generate transfer clients from
the same OpenAPI contract.
tags: [Transfer]
responses:
'200':
description: Peer is reachable
content:
application/json:
schema:
type: object
required: [ok]
properties:
ok: { type: boolean }
/v1/transfer/{streamId}/chunk:
post:
summary: Upload a stream-chunk envelope (signed by sender)
description: |
Per-chunk POST: the sender uploads a wire-encoded `0x11`
stream-chunk envelope, signed via `ShadeTransferAuthenticator`.
The receiver verifies the signature, ratifies lane + seq, and
returns an `{ok, lastSeqAcked}` ACK.
Body is `application/octet-stream`. Body size is bounded by the
receiver's `maxChunkBytes` option (default ≈ 16 MiB + header);
anything over the limit returns `413` before decryption.
tags: [Transfer]
security:
- ShadeTransferAuthenticator: []
parameters:
- name: streamId
in: path
required: true
schema:
$ref: '#/components/schemas/StreamId'
- name: X-Shade-Lane-Id
in: header
required: true
description: Lane index (`u32`, 0-based) the chunk belongs to.
schema:
type: integer
minimum: 0
- name: X-Shade-Seq
in: header
required: true
description: Per-lane chunk sequence number (`u64` decimal).
schema:
type: string
pattern: '^[0-9]+$'
requestBody:
required: true
content:
application/octet-stream:
schema:
type: string
format: binary
description: Wire-encoded `0x11` stream-chunk envelope.
responses:
'200':
description: Chunk accepted
content:
application/json:
schema:
$ref: '#/components/schemas/ChunkAck'
'400':
$ref: '#/components/responses/ValidationError'
'401':
$ref: '#/components/responses/Unauthorized'
'409':
description: Replay or out-of-order chunk
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
'413':
description: Chunk exceeds receiver's `maxChunkBytes`
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
/v1/transfer/{streamId}/state:
get:
summary: Read resume-state for a stream (signed)
description: |
Sender-side resume probe: returns the receiver's per-lane
`lastSeqAcked` so the sender can skip already-ack'd chunks.
Authentication is the same `ShadeTransferAuthenticator` shape
as `/chunk`, but signed over the control canonical form
(method + path) rather than chunk bodies.
tags: [Transfer]
security:
- ShadeTransferAuthenticator: []
parameters:
- name: streamId
in: path
required: true
schema:
$ref: '#/components/schemas/StreamId'
responses:
'200':
description: Resume state
content:
application/json:
schema:
$ref: '#/components/schemas/ResumeState'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
description: No state recorded for this streamId
content:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
/v1/transfer/control:
post:
summary: Deliver a control-plane ratchet envelope
description: |
SDK-internal endpoint that receives a wire-encoded `0x02`
Double-Ratchet envelope from a peer. The body is the raw
envelope bytes; the sender's address is supplied via the
`X-Shade-Sender-Address` header. The receiver decrypts on the
ratchet, parses the control message, and dispatches to the
transfer engine. **Note:** `0x02` envelopes are already
ratchet-authenticated, so this endpoint does not require a
separate signature header.
tags: [Transfer]
parameters:
- name: X-Shade-Sender-Address
in: header
required: true
schema:
$ref: '#/components/schemas/Address'
requestBody:
required: true
content:
application/octet-stream:
schema:
type: string
format: binary
description: Wire-encoded `0x02` ratchet envelope.
responses:
'200':
description: Control envelope accepted
content:
application/json:
schema:
type: object
required: [ok]
properties:
ok: { type: boolean }
'400':
$ref: '#/components/responses/ValidationError'
/v1/inbox/register:
post:
summary: Bind an address to a signing key (TOFU) — inbox relay (V3.6)
description: |
Claims a recipient address on the inbox relay. The first
successful registration binds the address to the supplied
signing key; subsequent fetch / ack / unregister calls must
be Ed25519-signed by the same key. A different key claiming
an existing address is rejected with 401.
tags: [Inbox]
requestBody:
required: true
content:
application/json:
schema:
allOf:
- $ref: '#/components/schemas/SignedPayload'
- type: object
required: [address, signingPublicKey]
properties:
address: { $ref: '#/components/schemas/Address' }
signingPublicKey:
type: string
description: Ed25519 public key, base64
responses:
'200':
description: Address bound to signing key
'401':
description: Wrong signing key for an existing registration
'409':
description: Stale signedAt (replay-window)
'400':
$ref: '#/components/responses/ValidationError'
/v1/inbox/register/{address}:
delete:
summary: Release an inbox registration
description: |
Signed unregister — drops the address ↔ signing-key binding
and any queued blobs for the address.
tags: [Inbox]
parameters:
- name: address
in: path
required: true
schema: { $ref: '#/components/schemas/Address' }
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/SignedPayload'
responses:
'200': { description: Registration removed }
'401': { description: Bad signature for the address }
/v1/inbox/{address}:
post:
summary: Deposit a ciphertext blob for a recipient
description: |
Signed PUT carrying an opaque ciphertext blob plus
`msgId = lower-hex(sha256(ciphertext))`. Idempotent on
`(address, msgId)` — replays return `200 { idempotent: true }`.
The relay rejects bodies past `maxBlobBytes` (default 1 MiB)
and per-recipient quota (default 1000 blobs).
tags: [Inbox]
parameters:
- name: address
in: path
required: true
schema: { $ref: '#/components/schemas/Address' }
requestBody:
required: true
content:
application/json:
schema:
allOf:
- $ref: '#/components/schemas/SignedPayload'
- type: object
required: [msgId, ciphertext, expiresAt, senderPublicKey]
properties:
msgId:
type: string
description: Lower-hex SHA-256 of ciphertext bytes.
ciphertext:
type: string
description: Base64-encoded ratchet envelope (`0x02`).
expiresAt:
type: integer
format: int64
description: Unix epoch ms; clamped to `[minTtl, maxTtl]`.
senderPublicKey:
type: string
description: Per-PUT Ed25519 public key, base64. TOFU-bound for the duration of this blob.
responses:
'200':
description: Blob accepted
content:
application/json:
schema:
type: object
properties:
idempotent: { type: boolean }
'401': { description: Bad sender signature }
'409': { description: Stale signedAt or msgId mismatch }
'413': { description: Body past maxBlobBytes }
'429': { description: Per-address quota exceeded }
/v1/inbox/{address}/fetch:
post:
summary: Cursor-paginated fetch of queued blobs
description: |
Signed challenge that returns up to `limit` blobs newer than the
supplied cursor. Cursors are server-issued opaque strings.
tags: [Inbox]
parameters:
- name: address
in: path
required: true
schema: { $ref: '#/components/schemas/Address' }
requestBody:
required: true
content:
application/json:
schema:
allOf:
- $ref: '#/components/schemas/SignedPayload'
- type: object
properties:
cursor: { type: string, nullable: true }
limit: { type: integer, minimum: 1, maximum: 100 }
responses:
'200':
description: Blob page
content:
application/json:
schema:
type: object
required: [items, nextCursor]
properties:
items:
type: array
items:
type: object
required: [msgId, ciphertext, receivedAt, expiresAt]
properties:
msgId: { type: string }
ciphertext: { type: string }
receivedAt: { type: integer, format: int64 }
expiresAt: { type: integer, format: int64 }
nextCursor:
type: string
nullable: true
'401': { description: Bad signature }
/v1/inbox/{address}/{msgId}:
delete:
summary: Acknowledge (delete) a delivered blob
tags: [Inbox]
parameters:
- name: address
in: path
required: true
schema: { $ref: '#/components/schemas/Address' }
- name: msgId
in: path
required: true
schema: { type: string }
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/SignedPayload'
responses:
'200': { description: Blob acknowledged and removed }
'401': { description: Bad signature for this address }
'404': { description: Blob not found (already acked or expired) }
/v1/bridge/stream:
get:
summary: Server-Sent Events feed of pending envelopes (V3.7)
description: |
Signed query string (no headers — `EventSource` strips them) drives
an SSE feed. Each delivered envelope is emitted as
`event: envelope` with a JSON payload. Heartbeats are sent every
15 s as `: ping` SSE comments. Resume on reconnect via
`Last-Event-ID`. `kind=stream` is bound into the canonical signed
payload to prevent cross-endpoint replay.
tags: [Bridge]
parameters:
- in: query
name: address
required: true
schema: { $ref: '#/components/schemas/Address' }
- in: query
name: signedAt
required: true
schema: { type: integer, format: int64 }
- in: query
name: signature
required: true
schema: { type: string }
- in: query
name: kind
required: true
schema: { type: string, enum: [stream] }
responses:
'200':
description: SSE stream
content:
text/event-stream:
schema:
type: string
'401': { description: Bad signature }
/v1/bridge/poll:
get:
summary: Long-poll fallback for environments without WS or SSE (V3.7)
description: |
Returns at most one batch of pending envelopes; holds the
connection open up to `timeoutMs` (default 25 s, hard cap 55 s)
if nothing is queued. `kind=poll` is bound into the canonical
signed payload.
tags: [Bridge]
parameters:
- in: query
name: address
required: true
schema: { $ref: '#/components/schemas/Address' }
- in: query
name: signedAt
required: true
schema: { type: integer, format: int64 }
- in: query
name: signature
required: true
schema: { type: string }
- in: query
name: kind
required: true
schema: { type: string, enum: [poll] }
- in: query
name: timeoutMs
schema: { type: integer, minimum: 0, maximum: 55000 }
responses:
'200':
description: Pending envelope batch (possibly empty after timeout).
content:
application/json:
schema:
type: object
required: [envelopes]
properties:
envelopes:
type: array
items:
type: object
required: [from, bytes, receivedAt]
properties:
from: { $ref: '#/components/schemas/Address' }
bytes: { type: string, description: 'Base64 envelope' }
receivedAt: { type: integer, format: int64 }
msgId: { type: string, nullable: true }
'401': { description: Bad signature }
/v1/bridge/ws:
get:
summary: WebSocket bridge (Bun WS upgrade) (V3.7)
description: |
Upgrade to a WebSocket. Each delivered envelope is one JSON frame.
Authentication is the same signed-query pattern as the SSE and
long-poll endpoints (`kind=ws`).
tags: [Bridge]
parameters:
- in: query
name: address
required: true
schema: { $ref: '#/components/schemas/Address' }
- in: query
name: signedAt
required: true
schema: { type: integer, format: int64 }
- in: query
name: signature
required: true
schema: { type: string }
- in: query
name: kind
required: true
schema: { type: string, enum: [ws] }
responses:
'101': { description: Switching Protocols (WS upgrade) }
'401': { description: Bad signature }
/metrics:
get:
summary: Prometheus metrics
description: |
Plaintext Prometheus exposition. Counters for HTTP requests,
rate-limit rejections, KT publish events, inbox PUT/FETCH/ACK,
bridge transports, and inbox prune cycles.
tags: [Operations]
responses:
'200':
description: Prometheus exposition.
content:
text/plain:
schema:
type: string
/healthz:
get:
summary: Liveness probe
tags: [Operations]
responses:
'200': { description: Process alive }
/ready:
get:
summary: Readiness probe
description: |
Returns 200 only when the storage backend, the inbox store (if
configured) and any KT log store have all reported ready.
tags: [Operations]
responses:
'200': { description: All backends ready }
'503': { description: At least one backend not ready }
/shade-observer/api/state:
get:
summary: Current observer snapshot (optional)
@@ -292,6 +869,50 @@ components:
type: string
description: X25519 public key, base64
STH:
type: object
description: |
Signed Tree Head — the operator-signed commitment to the current
state of the Key Transparency log (V3.12). Clients verify the
signature against a pinned `logPublicKey` and use `rootHash`
plus an audit path to validate inclusion proofs.
required: [treeSize, timestampMs, rootHash, indexRoot, logId, signature]
properties:
treeSize:
type: integer
minimum: 0
description: Number of leaves in the log
timestampMs:
type: integer
format: int64
description: Unix epoch ms; clients reject STHs older than `maxStaleMs` (default 24h)
rootHash:
type: string
description: 32-byte Merkle root hash, base64
indexRoot:
type: string
description: 32-byte commitment to the address-index, base64
logId:
type: string
description: 32-byte SHA-256 of the operator's signing public key, base64
signature:
type: string
description: Ed25519 signature over canonical bytes (DOMAIN_STH || treeSize || timestampMs || rootHash || indexRoot || logId), base64
KTProof:
type: object
description: |
Combined proof attached to bundle responses when KT is active.
See `@shade/key-transparency` for full structure of the `body`
variant (inclusion / tombstone / absence).
required: [sth, body]
properties:
sth:
$ref: '#/components/schemas/STH'
body:
type: object
description: Variant-specific proof body
PreKeyBundle:
type: object
required: [identitySigningKey, identityDHKey, signedPreKey]
@@ -316,6 +937,54 @@ components:
service:
type: string
StreamId:
type: string
description: |
URL-safe base64 encoding of the 16-byte streamId allocated by
the sender. Always 22 characters, no padding.
pattern: '^[A-Za-z0-9_-]{22}$'
example: "Tg9k0lZc7r2wQwUYP3KX9A"
ChunkAck:
type: object
required: [ok, lastSeqAcked]
properties:
ok:
type: boolean
lastSeqAcked:
type: string
description: |
Highest contiguous `seq` accepted by the receiver for this
lane, encoded as a decimal string (`u64` does not fit in a
JSON number).
pattern: '^[0-9]+$'
example: "42"
ResumeState:
type: object
description: |
Per-stream resume snapshot: each lane's last accepted seq, plus
the stream status. Returned by `GET /v1/transfer/{streamId}/state`.
required: [streamId, status, lanes]
properties:
streamId:
$ref: '#/components/schemas/StreamId'
status:
type: string
enum: [active, paused, finished, aborted]
lanes:
type: array
items:
type: object
required: [laneId, lastSeqAcked]
properties:
laneId:
type: integer
minimum: 0
lastSeqAcked:
type: string
pattern: '^[0-9]+$'
ErrorResponse:
type: object
properties:
@@ -375,3 +1044,44 @@ components:
`Authorization: Bearer <SHADE_OBSERVER_TOKEN>`. The observer also
accepts the token via `?token=...` query string for SSE endpoints
that can't set headers.
ShadeTransferAuthenticator:
type: apiKey
in: header
name: X-Shade-Signature
description: |
Per-request Ed25519 signature over a canonical byte string. Three
headers must be sent together:
- `X-Shade-Sender-Address: <address>`
- `X-Shade-Signed-At: <unix-ms>` (must be within ±5 minutes of
receiver clock, same window as the prekey server)
- `X-Shade-Signature: <base64-of-sig>`
Canonical message for `/v1/transfer/{streamId}/chunk`:
chunk\0
addr=<address>\0
at=<signedAt>\0
sid=<streamId>\0
lane=<laneId>\0
seq=<seq decimal>\0
bodyHash=<hex(sha256(body))>\0
Canonical message for `/v1/transfer/{streamId}/state`:
control\0
addr=<address>\0
at=<signedAt>\0
sid=<streamId>\0
method=GET\0
path=/v1/transfer/<streamId>/state\0
The receiver verifies using the public Ed25519 signing key it
fetched from the prekey server's `/v1/keys/bundle/{address}`
endpoint. SDK reference implementation:
`packages/shade-sdk/src/streams-bridge.ts` (`ShadeTransferAuthenticator`).
OpenAPI 3.1 has no native multi-header security scheme, so this
scheme is declared as `apiKey` on `X-Shade-Signature` with the
sibling headers documented in the description. Generated clients
SHOULD treat all three headers as mandatory.

View File

@@ -1,11 +1,14 @@
{
"name": "@shade/server",
"version": "0.3.0",
"version": "4.0.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/inbox-server": "workspace:*",
"@shade/key-transparency": "workspace:*",
"@shade/observability": "workspace:*",
"hono": "^4.12.12"
},
"optionalDependencies": {

View File

@@ -4,11 +4,18 @@ import { createPrekeyRoutes } from './routes.js';
import { MemoryPrekeyStore } from './memory-store.js';
import type { PrekeyStore } from './store.js';
import type { PrekeyServerEvents } from './events.js';
import {
KeyTransparencyService,
type KeyTransparencyConfig,
} from './kt-integration.js';
export { createPrekeyRoutes } from './routes.js';
export { MemoryPrekeyStore } from './memory-store.js';
export type { PrekeyStore } from './store.js';
export { verifyPayload, signPayload, canonicalizePayload, validateAddress } from './auth.js';
export { KeyTransparencyService, encodeSthForWire, encodeProofForWire } from './kt-integration.js';
export type { KeyTransparencyConfig } from './kt-integration.js';
export { createKTRoutes } from './kt-routes.js';
/**
* Create a standalone Shade Prekey Server.
@@ -29,6 +36,8 @@ export function createPrekeyServer(options: {
store?: PrekeyStore;
disableRateLimit?: boolean;
events?: PrekeyServerEvents;
/** Existing KT service (already initialized via `KeyTransparencyService.create`). */
keyTransparency?: KeyTransparencyService;
}): Hono {
const store = options.store ?? new MemoryPrekeyStore();
const routesOptions: Parameters<typeof createPrekeyRoutes>[2] = {};
@@ -38,9 +47,35 @@ export function createPrekeyServer(options: {
if (options.events !== undefined) {
routesOptions.events = options.events;
}
if (options.keyTransparency !== undefined) {
routesOptions.keyTransparency = options.keyTransparency;
}
return createPrekeyRoutes(store, options.crypto, routesOptions);
}
/**
* Convenience: create both the KT service and the prekey server in one call.
* Async because `KeyTransparencyService.create` reads existing state.
*/
export async function createPrekeyServerWithKT(options: {
crypto: CryptoProvider;
store?: PrekeyStore;
disableRateLimit?: boolean;
events?: PrekeyServerEvents;
keyTransparency: KeyTransparencyConfig;
}): Promise<{ app: Hono; kt: KeyTransparencyService }> {
const kt = await KeyTransparencyService.create(options.crypto, options.keyTransparency);
const passOpts: Parameters<typeof createPrekeyServer>[0] = {
crypto: options.crypto,
keyTransparency: kt,
};
if (options.store !== undefined) passOpts.store = options.store;
if (options.disableRateLimit !== undefined) passOpts.disableRateLimit = options.disableRateLimit;
if (options.events !== undefined) passOpts.events = options.events;
const app = createPrekeyServer(passOpts);
return { app, kt };
}
export { RateLimiter, MemoryRateLimitStore } from './rate-limit.js';
export type { RateLimitStore, RateLimitConfig } from './rate-limit.js';
export { PrekeyServerEvents, shortHash as serverShortHash } from './events.js';

View File

@@ -0,0 +1,216 @@
/**
* Key-Transparency integration for the Shade prekey server.
*
* The prekey server is the *source of truth* for which prekey bundle is
* currently published for each address. Without KT a malicious server
* could swap a bundle without anyone noticing. With KT enabled:
*
* - Every register / delete operation appends a leaf to an append-only
* Merkle log via `KTLogManager.recordRegister` / `recordDelete`.
* - After each mutation the manager re-signs and publishes a fresh STH.
* - GET /v1/keys/bundle/:address attaches a `ktProof` to its response,
* so the client can verify inclusion + freshness.
* - GET /v1/kt/sth and friends expose the log to witnesses.
*
* KT is **opt-in**: pass `keyTransparency` to `createPrekeyServer`. When
* absent, the server behaves exactly as before — proof fields are simply
* not added to the bundle response.
*/
import type { CryptoProvider } from '@shade/core';
import {
KTLogManager,
type KTLogStore,
computeBundleHash,
ktProofToWire,
sthToWire,
type KTProof,
type KTProofWire,
type STHWire,
type SignedTreeHead,
} from '@shade/key-transparency';
export interface KeyTransparencyConfig {
/** Persistent store for the log + index + STH set. */
store: KTLogStore;
/** Operator's STH signing key (32-byte Ed25519 seed). */
signingPrivateKey: Uint8Array;
/** Operator's STH signing public key (32-byte Ed25519). */
signingPublicKey: Uint8Array;
/**
* Heartbeat interval — minimum gap between fresh STHs even when no
* mutations occur. Default 10 minutes; set to 0 to disable.
*/
heartbeatIntervalMs?: number;
/** Time source override (testing). */
now?: () => number;
}
/**
* Wraps a `KTLogManager` with the bookkeeping the server cares about:
* - Serializes mutations (single-writer guarantee).
* - Caches the latest STH so bundle-fetch is hot-path-fast.
* - Schedules / surfaces heartbeats.
* - Lazily backfills index entries from the prekey-server's existing
* state when KT is first turned on.
*/
export class KeyTransparencyService {
private readonly mgr: KTLogManager;
private readonly store: KTLogStore;
private readonly heartbeatIntervalMs: number;
private readonly now: () => number;
private latest: SignedTreeHead | null = null;
private mutex: Promise<unknown> = Promise.resolve();
private constructor(
mgr: KTLogManager,
store: KTLogStore,
opts: { heartbeatIntervalMs?: number; now?: () => number },
) {
this.mgr = mgr;
this.store = store;
this.heartbeatIntervalMs = opts.heartbeatIntervalMs ?? 10 * 60 * 1000;
this.now = opts.now ?? (() => Date.now());
}
static async create(crypto: CryptoProvider, cfg: KeyTransparencyConfig): Promise<KeyTransparencyService> {
const mgr = await KTLogManager.create({
crypto,
store: cfg.store,
signingPrivateKey: cfg.signingPrivateKey,
signingPublicKey: cfg.signingPublicKey,
...(cfg.now ? { now: cfg.now } : {}),
});
const svc = new KeyTransparencyService(mgr, cfg.store, {
...(cfg.heartbeatIntervalMs !== undefined ? { heartbeatIntervalMs: cfg.heartbeatIntervalMs } : {}),
...(cfg.now ? { now: cfg.now } : {}),
});
// Cache or generate the initial STH so bundle responses always have one.
const existing = await cfg.store.getLatestSTH();
if (existing) {
svc.latest = existing;
} else {
svc.latest = await mgr.publishSTH();
}
return svc;
}
/**
* Run a mutation under the manager's serial lock and refresh the STH.
*/
private async withLock<T>(fn: () => Promise<T>): Promise<T> {
const prev = this.mutex;
let resolveNext: () => void;
const next = new Promise<void>((res) => {
resolveNext = res;
});
this.mutex = next;
try {
await prev.catch(() => {});
return await fn();
} finally {
resolveNext!();
}
}
async recordRegister(address: string, bundle: {
identitySigningKey: Uint8Array;
identityDHKey: Uint8Array;
signedPreKey: { keyId: number; publicKey: Uint8Array; signature: Uint8Array };
}): Promise<SignedTreeHead> {
return this.withLock(async () => {
await this.mgr.recordRegister(address, computeBundleHash(bundle));
this.latest = await this.mgr.publishSTH();
return this.latest!;
});
}
async recordDelete(address: string): Promise<SignedTreeHead> {
return this.withLock(async () => {
await this.mgr.recordDelete(address);
this.latest = await this.mgr.publishSTH();
return this.latest!;
});
}
/**
* Build a proof for a freshly-fetched bundle. Returns null if the
* address has no live entry (caller can request an absence proof
* via `buildAbsenceProof` instead).
*/
async buildBundleInclusion(address: string): Promise<KTProof | null> {
const sth = await this.maybeHeartbeat();
return this.mgr.buildBundleInclusionProof(address, sth);
}
async buildAbsence(address: string): Promise<KTProof | null> {
const sth = await this.maybeHeartbeat();
return this.mgr.buildBundleAbsenceProof(address, sth);
}
/** Latest STH — issuing a heartbeat first if the cached one is stale. */
async getLatestSTH(): Promise<SignedTreeHead> {
return this.maybeHeartbeat();
}
/** Historical STH at a specific tree size. */
async getSTHByTreeSize(treeSize: number): Promise<SignedTreeHead | null> {
return this.store.getSTHByTreeSize(treeSize);
}
/** All persisted STHs in a time window — used by witness backfill. */
async listSTHs(fromTimestampMs?: number, toTimestampMs?: number): Promise<SignedTreeHead[]> {
return this.store.listSTHs(fromTimestampMs, toTimestampMs);
}
/** Build a consistency proof for `from → to`. */
async buildConsistencyProof(fromTreeSize: number, toTreeSize?: number): Promise<{
fromTreeSize: number;
toTreeSize: number;
proof: Uint8Array[];
}> {
return this.withLock(async () => {
const targetSize = toTreeSize ?? this.mgr.getTreeSize();
const proof = await this.mgr.buildHistoricalConsistencyProof(fromTreeSize, targetSize);
return { fromTreeSize, toTreeSize: targetSize, proof };
});
}
/** STH signing public key — operators expose this to clients OOB. */
getSigningPublicKey(): Uint8Array {
return this.mgr.getSigningPublicKey();
}
getLogId(): Uint8Array {
return this.mgr.getLogId();
}
private async maybeHeartbeat(): Promise<SignedTreeHead> {
if (!this.latest) {
return this.withLock(async () => {
this.latest = await this.mgr.publishSTH();
return this.latest!;
});
}
if (this.heartbeatIntervalMs <= 0) return this.latest;
const age = this.now() - this.latest.timestampMs;
if (age < this.heartbeatIntervalMs) return this.latest;
return this.withLock(async () => {
// Re-check age inside the lock — another caller may have published.
const ageNow = this.now() - (this.latest?.timestampMs ?? 0);
if (ageNow < this.heartbeatIntervalMs) return this.latest!;
this.latest = await this.mgr.publishSTH();
return this.latest!;
});
}
}
/** Helpers to encode an STH for the wire (base64). */
export function encodeSthForWire(sth: SignedTreeHead): STHWire {
return sthToWire(sth, (b) => Buffer.from(b).toString('base64'));
}
export function encodeProofForWire(proof: KTProof): KTProofWire {
return ktProofToWire(proof);
}

View File

@@ -0,0 +1,74 @@
import { Hono } from 'hono';
import { ValidationError } from '@shade/core';
import type { KeyTransparencyService } from './kt-integration.js';
import { encodeSthForWire } from './kt-integration.js';
/**
* Mountable routes that expose the KT log to clients and witnesses.
*
* GET /v1/kt/log_id — public-key + log_id (operator pinning)
* GET /v1/kt/sth — latest signed tree head
* GET /v1/kt/sth/:treeSize — historical STH at a specific tree_size
* GET /v1/kt/consistency — consistency proof between two tree_sizes
*
* These are intentionally **anonymous & read-only** so witnesses can
* poll without sharing identity. Rate-limiting is the prekey-server's
* existing fetch RL bucket.
*/
export function createKTRoutes(svc: KeyTransparencyService): Hono {
const app = new Hono();
app.get('/log_id', (c) => {
const logId = svc.getLogId();
const pub = svc.getSigningPublicKey();
return c.json({
logId: Buffer.from(logId).toString('base64'),
publicKey: Buffer.from(pub).toString('base64'),
});
});
app.get('/sth', async (c) => {
const sth = await svc.getLatestSTH();
return c.json(encodeSthForWire(sth));
});
app.get('/sth/:treeSize', async (c) => {
const sizeRaw = c.req.param('treeSize');
const size = Number(sizeRaw);
if (!Number.isFinite(size) || size < 0 || size !== Math.floor(size)) {
throw new ValidationError('treeSize must be a non-negative integer');
}
const sth = await svc.getSTHByTreeSize(size);
if (!sth) {
return c.json({ error: 'STH not found at that tree_size', code: 'SHADE_NOT_FOUND' }, 404);
}
return c.json(encodeSthForWire(sth));
});
app.get('/consistency', async (c) => {
const fromRaw = c.req.query('from');
const toRaw = c.req.query('to');
if (fromRaw === undefined) {
throw new ValidationError('from query param required');
}
const from = Number(fromRaw);
if (!Number.isFinite(from) || from < 0 || from !== Math.floor(from)) {
throw new ValidationError('from must be a non-negative integer');
}
let to: number | undefined;
if (toRaw !== undefined) {
to = Number(toRaw);
if (!Number.isFinite(to) || to < from || to !== Math.floor(to)) {
throw new ValidationError('to must be an integer >= from');
}
}
const result = await svc.buildConsistencyProof(from, to);
return c.json({
fromTreeSize: result.fromTreeSize,
toTreeSize: result.toTreeSize,
proof: result.proof.map((p) => Buffer.from(p).toString('base64')),
});
});
return app;
}

View File

@@ -5,6 +5,16 @@ import type { PrekeyStore } from './store.js';
import { verifyPayload, validateAddress } from './auth.js';
import { RateLimiter, MemoryRateLimitStore, REGISTER_LIMIT, FETCH_LIMIT, REPLENISH_LIMIT, DELETE_LIMIT } from './rate-limit.js';
import { PrekeyServerEvents, shortHash } from './events.js';
import {
ATTR_ERROR_CODE,
ATTR_HTTP_STATUS,
ATTR_ROUTE,
NOOP_HOOK,
type ObservabilityHook,
} from '@shade/observability';
import type { KeyTransparencyService } from './kt-integration.js';
import { encodeProofForWire, encodeSthForWire } from './kt-integration.js';
import { createKTRoutes } from './kt-routes.js';
/** Max POST body size in bytes (64KB) */
const MAX_BODY_SIZE = 64 * 1024;
@@ -26,6 +36,22 @@ export interface PrekeyRoutesOptions {
disableRateLimit?: boolean;
/** Optional event emitter for observability. */
events?: PrekeyServerEvents;
/**
* Optional OTel observability hook. When supplied (and the runtime gate
* is on), each request gets a `shade.prekey.<route>` span with route
* + HTTP-status attributes. PII-safe: never logs the address path
* parameter or client IP.
*/
observability?: ObservabilityHook;
/**
* Optional Key-Transparency service (V3.12). When provided:
* - Every `register` and `delete` mutation is committed to the log.
* - `GET /v1/keys/bundle/:address` includes a `ktProof` field.
* - `/v1/kt/*` routes are mounted (latest STH, historical STHs,
* consistency proofs, log_id pinning info).
* When absent, the server is byte-compatible with pre-V3.12 clients.
*/
keyTransparency?: KeyTransparencyService;
}
export function createPrekeyRoutes(
@@ -35,6 +61,35 @@ export function createPrekeyRoutes(
): Hono {
const app = new Hono();
const events = options.events;
const observability = options.observability ?? NOOP_HOOK;
const kt = options.keyTransparency;
// Per-request span middleware — runs first so it covers handlers AND
// the global error handler. Span name is the route template (e.g.
// `/v1/keys/bundle/:address`), so cardinality stays bounded and the
// address itself never enters span data.
app.use('*', async (c, next) => {
const route = c.req.routePath ?? c.req.path ?? '<unknown>';
const span = observability.startSpan('shade.prekey.request', {
[ATTR_ROUTE]: route,
});
try {
await next();
span.setAttribute(ATTR_HTTP_STATUS, c.res.status);
span.setStatus(c.res.status >= 500 ? 'error' : 'ok');
} catch (err) {
const code =
err instanceof ShadeError
? err.code ?? 'SHADE_ERROR'
: 'SHADE_INTERNAL';
span.setAttribute(ATTR_ERROR_CODE, code);
span.recordException(err);
span.setStatus('error', code);
throw err;
} finally {
span.end();
}
});
// Rate limiters (one per route, per IP or per identity)
const rlStore = new MemoryRateLimitStore();
@@ -115,6 +170,21 @@ export function createPrekeyRoutes(
await store.saveOneTimePreKeys(addr, keys);
}
// Commit to KT log (if enabled). The bundle covered by the commitment
// is { signing_key, dh_key, signed_prekey } — one-time prekeys are
// intentionally excluded so OTP rotation doesn't churn the log.
if (kt) {
await kt.recordRegister(addr, {
identitySigningKey: signingKey,
identityDHKey: dhKey,
signedPreKey: {
keyId: signedPreKey.keyId,
publicKey: b64ToBytes(signedPreKey.publicKey),
signature: b64ToBytes(signedPreKey.signature),
},
});
}
if (events) {
const hash = await shortHash(signingKey);
events.emit('server.identity_registered', { address: addr, identityKeyHash: hash });
@@ -130,6 +200,33 @@ export function createPrekeyRoutes(
const identity = await store.getIdentity(address);
if (!identity) {
// KT-enabled: pin the negative answer to a tree state. If the
// address has been tombstoned we serve the tombstone (inclusion)
// proof; otherwise an absence proof.
if (kt) {
const inclusion = await kt.buildBundleInclusion(address);
if (inclusion) {
return c.json(
{
error: 'Address not found',
code: 'SHADE_NOT_FOUND',
ktProof: encodeProofForWire(inclusion),
},
404,
);
}
const absence = await kt.buildAbsence(address);
if (absence) {
return c.json(
{
error: 'Address not found',
code: 'SHADE_NOT_FOUND',
ktProof: encodeProofForWire(absence),
},
404,
);
}
}
return c.json({ error: 'Address not found', code: 'SHADE_NOT_FOUND' }, 404);
}
@@ -157,6 +254,19 @@ export function createPrekeyRoutes(
};
}
if (kt) {
const proof = await kt.buildBundleInclusion(address);
if (proof) {
bundle.ktProof = encodeProofForWire(proof);
} else {
// No live entry in the index — fall back to STH so client at
// least sees a fresh tree-head, then surfaces "no proof available"
// as a soft warning.
const sth = await kt.getLatestSTH();
bundle.ktSth = encodeSthForWire(sth);
}
}
// Update activity so stale cleanup doesn't purge active addresses
await store.touchIdentity(address);
@@ -228,10 +338,17 @@ export function createPrekeyRoutes(
await verifyPayload(crypto, identity.identitySigningKey, { ...body, address });
await store.deleteAll(address);
if (kt) {
await kt.recordDelete(address);
}
events?.emit('server.identity_deleted', { address });
return c.json({ ok: true });
});
if (kt) {
app.route('/v1/kt', createKTRoutes(kt));
}
return app;
}

View File

@@ -1,5 +1,13 @@
import { Hono } from 'hono';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import {
createInboxRoutes,
createBridgeRoutes,
InboxServerEvents,
InboxPruneTask,
MemoryInboxStore,
type InboxStore,
} from '@shade/inbox-server';
import { createPrekeyRoutes } from './routes.js';
import { createHealthRoutes } from './health.js';
import { createMetricsRoutes, metricsMiddleware } from './metrics.js';
@@ -8,8 +16,13 @@ import { PrekeyServerEvents } from './events.js';
import { StaleCleanupTask } from './cleanup.js';
import { logger } from './logger.js';
import type { PrekeyStore } from './store.js';
import { KeyTransparencyService } from './kt-integration.js';
import {
MemoryKTLogStore,
type KTLogStore,
} from '@shade/key-transparency';
const VERSION = '1.0.0';
const VERSION = '4.0.0';
async function createStore(): Promise<PrekeyStore & { close?: () => void | Promise<void> }> {
const sqlitePath = process.env.SHADE_PREKEY_DB_PATH;
@@ -32,6 +45,89 @@ async function createStore(): Promise<PrekeyStore & { close?: () => void | Promi
return new MemoryPrekeyStore();
}
async function createInboxStore(): Promise<InboxStore & { close?: () => void | Promise<void> }> {
const sqlitePath = process.env.SHADE_INBOX_DB_PATH;
const pgUrl = process.env.SHADE_INBOX_PG_URL ?? process.env.SHADE_PREKEY_PG_URL;
if (pgUrl && process.env.SHADE_INBOX_PG_URL) {
const { PostgresInboxStore } = await import('@shade/storage-postgres');
logger.info('Using PostgreSQL inbox store', { url: maskUrl(pgUrl) });
return PostgresInboxStore.create(pgUrl);
}
if (sqlitePath) {
const { SqliteInboxStore } = await import('@shade/storage-sqlite');
logger.info('Using SQLite inbox store', { path: sqlitePath });
return new SqliteInboxStore(sqlitePath);
}
if (pgUrl) {
const { PostgresInboxStore } = await import('@shade/storage-postgres');
logger.info('Using PostgreSQL inbox store (sharing prekey URL)', { url: maskUrl(pgUrl) });
return PostgresInboxStore.create(pgUrl);
}
logger.warn('Using in-memory inbox store — data will not persist across restarts');
return new MemoryInboxStore();
}
async function maybeCreateKT(): Promise<KeyTransparencyService | undefined> {
const skPriv = process.env.SHADE_KT_SIGNING_PRIVATE_KEY;
const skPub = process.env.SHADE_KT_SIGNING_PUBLIC_KEY;
if (!skPriv || !skPub) {
if (skPriv || skPub) {
logger.warn(
'Key Transparency requires BOTH SHADE_KT_SIGNING_PRIVATE_KEY and SHADE_KT_SIGNING_PUBLIC_KEY — KT disabled',
);
} else {
logger.info('Key Transparency disabled (signing keys not configured)');
}
return undefined;
}
let signingPrivateKey: Uint8Array;
let signingPublicKey: Uint8Array;
try {
signingPrivateKey = new Uint8Array(Buffer.from(skPriv, 'base64'));
signingPublicKey = new Uint8Array(Buffer.from(skPub, 'base64'));
} catch {
logger.error('SHADE_KT_SIGNING_*_KEY must be base64 — KT disabled');
return undefined;
}
if (signingPrivateKey.length !== 32 || signingPublicKey.length !== 32) {
logger.error(
`SHADE_KT_SIGNING_*_KEY must decode to 32 bytes (priv=${signingPrivateKey.length}, pub=${signingPublicKey.length}) — KT disabled`,
);
return undefined;
}
let ktStore: KTLogStore;
const ktPg = process.env.SHADE_KT_PG_URL ?? process.env.SHADE_PREKEY_PG_URL;
if (ktPg) {
const { PostgresKTLogStore } = await import('@shade/storage-postgres');
logger.info('Using PostgreSQL KT log store', { url: maskUrl(ktPg) });
ktStore = await PostgresKTLogStore.create(ktPg);
} else {
logger.warn('Using in-memory KT log store — KT data will not persist across restarts');
ktStore = new MemoryKTLogStore();
}
const heartbeatRaw = process.env.SHADE_KT_HEARTBEAT_MS;
const heartbeatIntervalMs = heartbeatRaw ? Number(heartbeatRaw) : 10 * 60 * 1000;
const svc = await KeyTransparencyService.create(crypto, {
store: ktStore,
signingPrivateKey,
signingPublicKey,
heartbeatIntervalMs,
});
logger.info('Key Transparency enabled', {
logId: Buffer.from(svc.getLogId()).toString('hex').slice(0, 16) + '…',
heartbeatIntervalMs,
});
return svc;
}
function maskUrl(url: string): string {
try {
const u = new URL(url);
@@ -46,13 +142,42 @@ const crypto = new SubtleCryptoProvider();
const store = await createStore();
const events = new PrekeyServerEvents();
// Inbox store + events (V3.6 store-and-forward relay)
const inboxStore = await createInboxStore();
const inboxEvents = new InboxServerEvents();
// ─── Optional: Key Transparency (V3.12) ──────────────────────
//
// Enabled when both SHADE_KT_SIGNING_PRIVATE_KEY and
// SHADE_KT_SIGNING_PUBLIC_KEY are set (base64-encoded 32-byte
// Ed25519 seeds). Storage: PostgresKTLogStore when
// SHADE_KT_PG_URL (or SHADE_PREKEY_PG_URL) is set, else memory.
const kt = await maybeCreateKT();
// Compose the full app: metrics middleware + health + metrics + prekey routes
const app = new Hono();
app.use('*', metricsMiddleware());
app.route('/', createHealthRoutes(store, VERSION));
app.route('/', createMetricsRoutes());
app.route('/', createOpenApiRoutes());
app.route('/', createPrekeyRoutes(store, crypto, { events }));
app.route(
'/',
createPrekeyRoutes(store, crypto, {
events,
...(kt ? { keyTransparency: kt } : {}),
}),
);
app.route('/', createInboxRoutes(inboxStore, crypto, { events: inboxEvents }));
// V3.7 transport-bridge — SSE / long-poll / WS fallbacks for the inbox.
// Held as a top-level reference so the WebSocket handler can be passed to
// Bun.serve below.
const bridgeRoutes = createBridgeRoutes({
store: inboxStore,
crypto,
events: inboxEvents,
});
app.route('/', bridgeRoutes.app);
// ─── Optional: Observer + Dashboard ──────────────────────────
@@ -84,13 +209,31 @@ logger.info('Stale cleanup task started', {
intervalHours: Number(process.env.SHADE_CLEANUP_INTERVAL_HOURS ?? 24),
});
// ─── Inbox prune task ────────────────────────────────────────
const inboxPrune = new InboxPruneTask(inboxStore, {
events: inboxEvents,
logger: {
info: (m, d) => logger.info(m, d as Record<string, unknown> | undefined),
error: (m, d) => logger.error(m, d as Record<string, unknown> | undefined),
},
});
inboxPrune.start();
logger.info('Inbox prune task started', {
intervalMinutes: Number(process.env.SHADE_INBOX_PRUNE_INTERVAL_MINUTES ?? 5),
});
// ─── Start HTTP server ───────────────────────────────────────
const port = Number(process.env.PORT ?? 3900);
logger.info('Shade Prekey Server starting', { port, version: VERSION });
const server = Bun.serve({ port, fetch: app.fetch });
const server = Bun.serve({
port,
fetch: (req, srv) => app.fetch(req, srv),
websocket: bridgeRoutes.websocket as any,
});
// ─── Graceful shutdown ───────────────────────────────────────
@@ -102,10 +245,14 @@ async function shutdown(signal: string) {
try {
cleanupTask.stop();
inboxPrune.stop();
server.stop();
if ('close' in store && typeof store.close === 'function') {
await store.close();
}
if ('close' in inboxStore && typeof inboxStore.close === 'function') {
await inboxStore.close();
}
logger.info('Shutdown complete');
process.exit(0);
} catch (err) {

View File

@@ -0,0 +1,260 @@
import { describe, test, expect, beforeEach } from 'bun:test';
import {
createPrekeyServerWithKT,
KeyTransparencyService,
MemoryPrekeyStore,
signPayload,
} from '../src/index.js';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { generateIdentityKeyPair } from '@shade/core';
import {
MemoryKTLogStore,
computeBundleHash,
ktProofFromWire,
sthFromWire,
verifyBundleAbsence,
verifyBundleInclusion,
verifyConsistencyProof,
type KTProofWire,
type STHWire,
} from '@shade/key-transparency';
const crypto = new SubtleCryptoProvider();
function b64(bytes: Uint8Array): string {
return Buffer.from(bytes).toString('base64');
}
function fromB64(s: string): Uint8Array {
return new Uint8Array(Buffer.from(s, 'base64'));
}
function randBytes(n: number): Uint8Array {
const buf = new Uint8Array(n);
globalThis.crypto.getRandomValues(buf);
return buf;
}
async function makeBundleData() {
const identity = await generateIdentityKeyPair(crypto);
const signedPreKeyPub = randBytes(32);
const signedPreKeySig = await crypto.sign(identity.signingPrivateKey, signedPreKeyPub);
return {
identity,
signedPreKey: {
keyId: 1,
publicKey: signedPreKeyPub,
signature: signedPreKeySig,
},
};
}
describe('Prekey server with KT enabled', () => {
let app: any;
let kt: KeyTransparencyService;
let logKp: { publicKey: Uint8Array; privateKey: Uint8Array };
beforeEach(async () => {
logKp = await crypto.generateEd25519KeyPair();
const result = await createPrekeyServerWithKT({
crypto,
store: new MemoryPrekeyStore(),
disableRateLimit: true,
keyTransparency: {
store: new MemoryKTLogStore(),
signingPrivateKey: logKp.privateKey,
signingPublicKey: logKp.publicKey,
},
});
app = result.app;
kt = result.kt;
});
async function registerAddress(address: string) {
const data = await makeBundleData();
const body: any = {
address,
identitySigningKey: b64(data.identity.signingPublicKey),
identityDHKey: b64(data.identity.dhPublicKey),
signedPreKey: {
keyId: data.signedPreKey.keyId,
publicKey: b64(data.signedPreKey.publicKey),
signature: b64(data.signedPreKey.signature),
},
oneTimePreKeys: [{ keyId: 100, publicKey: b64(randBytes(32)) }],
};
const signed = await signPayload(crypto, data.identity.signingPrivateKey, body);
const res = await app.request('/v1/keys/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(signed),
});
expect(res.status).toBe(200);
return data;
}
test('GET /v1/kt/log_id returns logId + publicKey', async () => {
const res = await app.request('/v1/kt/log_id');
expect(res.status).toBe(200);
const body = await res.json();
expect(body.logId).toBeDefined();
expect(body.publicKey).toBeDefined();
expect(b64(kt.getSigningPublicKey())).toBe(body.publicKey);
});
test('GET /v1/kt/sth returns latest STH', async () => {
await registerAddress('alice');
const res = await app.request('/v1/kt/sth');
expect(res.status).toBe(200);
const wire = (await res.json()) as STHWire;
const sth = sthFromWire(wire, fromB64);
expect(sth.treeSize).toBe(1);
});
test('bundle response carries verified inclusion proof', async () => {
const data = await registerAddress('alice');
const res = await app.request('/v1/keys/bundle/alice');
expect(res.status).toBe(200);
const body = (await res.json()) as { ktProof: KTProofWire } & Record<string, unknown>;
expect(body.ktProof).toBeDefined();
const proof = ktProofFromWire(body.ktProof);
expect(proof.body.kind).toBe('inclusion');
await verifyBundleInclusion(
{ crypto, logPublicKey: logKp.publicKey },
'alice',
{
identitySigningKey: data.identity.signingPublicKey,
identityDHKey: data.identity.dhPublicKey,
signedPreKey: data.signedPreKey,
},
proof,
);
// Sanity: bundle hash matches the proof's index commitment
const expected = computeBundleHash({
identitySigningKey: data.identity.signingPublicKey,
identityDHKey: data.identity.dhPublicKey,
signedPreKey: data.signedPreKey,
});
if (proof.body.kind === 'inclusion') {
expect(b64(proof.body.indexProof.entry.bundleHash)).toBe(b64(expected));
}
});
test('bundle for unknown address returns 404 + absence proof', async () => {
await registerAddress('alice');
const res = await app.request('/v1/keys/bundle/zeta');
expect(res.status).toBe(404);
const body = (await res.json()) as { ktProof?: KTProofWire };
expect(body.ktProof).toBeDefined();
const proof = ktProofFromWire(body.ktProof!);
expect(proof.body.kind).toBe('absence');
await verifyBundleAbsence({ crypto, logPublicKey: logKp.publicKey }, 'zeta', proof);
});
test('DELETE /v1/keys/:address commits a tombstone', async () => {
const data = await registerAddress('alice');
const sthBefore = await kt.getLatestSTH();
const signed = await signPayload(crypto, data.identity.signingPrivateKey, { address: 'alice' });
const res = await app.request('/v1/keys/alice', {
method: 'DELETE',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(signed),
});
expect(res.status).toBe(200);
const sthAfter = await kt.getLatestSTH();
expect(sthAfter.treeSize).toBeGreaterThan(sthBefore.treeSize);
const fetched = await app.request('/v1/keys/bundle/alice');
// Identity row was removed by the prekey-store; absence-proof returned.
expect(fetched.status).toBe(404);
const body = (await fetched.json()) as { ktProof?: KTProofWire };
expect(body.ktProof).toBeDefined();
});
test('GET /v1/kt/consistency returns valid proof', async () => {
await registerAddress('alice');
const sth1 = await kt.getLatestSTH();
await registerAddress('bob');
const sth2 = await kt.getLatestSTH();
const res = await app.request(`/v1/kt/consistency?from=${sth1.treeSize}&to=${sth2.treeSize}`);
expect(res.status).toBe(200);
const body = (await res.json()) as { proof: string[] };
const proofBytes = body.proof.map(fromB64);
expect(verifyConsistencyProof(sth1.treeSize, sth2.treeSize, sth1.rootHash, sth2.rootHash, proofBytes)).toBe(true);
});
test('GET /v1/kt/sth/:treeSize returns historical STH', async () => {
await registerAddress('alice');
const sth1 = await kt.getLatestSTH();
await registerAddress('bob');
const res = await app.request(`/v1/kt/sth/${sth1.treeSize}`);
expect(res.status).toBe(200);
const wire = (await res.json()) as STHWire;
const back = sthFromWire(wire, fromB64);
expect(b64(back.rootHash)).toBe(b64(sth1.rootHash));
});
test('rotation: latest STH proof verifies with new bundle, not old', async () => {
const v1Data = await registerAddress('alice');
// Register again with a new identity (rotation)
const v2Data = await makeBundleData();
const body: any = {
address: 'alice',
identitySigningKey: b64(v2Data.identity.signingPublicKey),
identityDHKey: b64(v2Data.identity.dhPublicKey),
signedPreKey: {
keyId: v2Data.signedPreKey.keyId,
publicKey: b64(v2Data.signedPreKey.publicKey),
signature: b64(v2Data.signedPreKey.signature),
},
};
const signed = await signPayload(crypto, v2Data.identity.signingPrivateKey, body);
const reRes = await app.request('/v1/keys/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(signed),
});
expect(reRes.status).toBe(200);
// Now the bundle response should reflect the new identity
const res = await app.request('/v1/keys/bundle/alice');
const body2 = (await res.json()) as {
identitySigningKey: string;
ktProof: KTProofWire;
};
expect(body2.identitySigningKey).toBe(b64(v2Data.identity.signingPublicKey));
const proof = ktProofFromWire(body2.ktProof);
await verifyBundleInclusion(
{ crypto, logPublicKey: logKp.publicKey },
'alice',
{
identitySigningKey: v2Data.identity.signingPublicKey,
identityDHKey: v2Data.identity.dhPublicKey,
signedPreKey: v2Data.signedPreKey,
},
proof,
);
// Verifying with v1 data should reject
await expect(
verifyBundleInclusion(
{ crypto, logPublicKey: logKp.publicKey },
'alice',
{
identitySigningKey: v1Data.identity.signingPublicKey,
identityDHKey: v1Data.identity.dhPublicKey,
signedPreKey: v1Data.signedPreKey,
},
proof,
),
).rejects.toThrow();
});
});

View File

@@ -0,0 +1,146 @@
import { describe, test, expect } from 'bun:test';
import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
const here = dirname(fileURLToPath(import.meta.url));
const specPath = join(here, '..', 'openapi.yaml');
const spec = Bun.YAML.parse(readFileSync(specPath, 'utf-8')) as Record<string, unknown>;
describe('openapi.yaml lint', () => {
test('declares OpenAPI 3.1', () => {
expect(typeof spec.openapi).toBe('string');
expect(String(spec.openapi).startsWith('3.1')).toBe(true);
});
test('has the required top-level fields', () => {
expect(spec).toHaveProperty('info');
expect(spec).toHaveProperty('paths');
expect(spec).toHaveProperty('components');
const info = spec.info as Record<string, unknown>;
expect(typeof info.title).toBe('string');
expect(typeof info.version).toBe('string');
});
test('every operation has summary + responses', () => {
const paths = (spec.paths ?? {}) as Record<string, Record<string, unknown>>;
const httpMethods = new Set([
'get',
'post',
'put',
'patch',
'delete',
'options',
'head',
]);
for (const [routePath, route] of Object.entries(paths)) {
for (const [verb, op] of Object.entries(route)) {
if (!httpMethods.has(verb)) continue;
const operation = op as Record<string, unknown>;
expect(operation.summary, `${verb.toUpperCase()} ${routePath} missing summary`).toBeDefined();
const responses = operation.responses as Record<string, unknown> | undefined;
expect(responses, `${verb.toUpperCase()} ${routePath} missing responses`).toBeDefined();
expect(
Object.keys(responses ?? {}).length,
`${verb.toUpperCase()} ${routePath} has empty responses`,
).toBeGreaterThan(0);
}
}
});
test('every $ref resolves to a defined component', () => {
const refs = collectRefs(spec);
expect(refs.length).toBeGreaterThan(0);
for (const ref of refs) {
expect(
ref.startsWith('#/'),
`non-internal $ref not allowed: ${ref}`,
).toBe(true);
const segments = ref.slice(2).split('/');
let cursor: unknown = spec;
for (const segment of segments) {
expect(
isObject(cursor),
`dangling $ref: ${ref} (failed at "${segment}")`,
).toBe(true);
cursor = (cursor as Record<string, unknown>)[segment];
}
expect(cursor, `unresolved $ref: ${ref}`).toBeDefined();
}
});
test('every security requirement references a defined scheme', () => {
const schemes = (
((spec.components ?? {}) as Record<string, Record<string, unknown>>)
.securitySchemes ?? {}
) as Record<string, unknown>;
const definedSchemes = new Set(Object.keys(schemes));
const requirements = collectSecurityRequirements(spec);
for (const name of requirements) {
expect(
definedSchemes.has(name),
`security requirement "${name}" not defined under components.securitySchemes`,
).toBe(true);
}
});
test('declares the V3.1 transfer surface', () => {
const paths = (spec.paths ?? {}) as Record<string, unknown>;
expect(paths['/v1/transfer/health']).toBeDefined();
expect(paths['/v1/transfer/{streamId}/chunk']).toBeDefined();
expect(paths['/v1/transfer/{streamId}/state']).toBeDefined();
expect(paths['/v1/transfer/control']).toBeDefined();
const schemes = (
((spec.components ?? {}) as Record<string, Record<string, unknown>>)
.securitySchemes ?? {}
) as Record<string, unknown>;
expect(schemes.ShadeTransferAuthenticator).toBeDefined();
});
});
function isObject(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function collectRefs(node: unknown, out: string[] = []): string[] {
if (Array.isArray(node)) {
for (const item of node) collectRefs(item, out);
return out;
}
if (!isObject(node)) return out;
for (const [key, value] of Object.entries(node)) {
if (key === '$ref' && typeof value === 'string') {
out.push(value);
continue;
}
collectRefs(value, out);
}
return out;
}
function collectSecurityRequirements(spec: Record<string, unknown>): Set<string> {
const names = new Set<string>();
const collectFromList = (list: unknown): void => {
if (!Array.isArray(list)) return;
for (const entry of list) {
if (!isObject(entry)) continue;
for (const name of Object.keys(entry)) names.add(name);
}
};
collectFromList(spec.security);
const paths = (spec.paths ?? {}) as Record<string, Record<string, unknown>>;
for (const route of Object.values(paths)) {
for (const value of Object.values(route)) {
if (!isObject(value)) continue;
collectFromList(value.security);
}
}
return names;
}