3c6999509776b84308bbc7bf488796061af5015c
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 96c20cb4b2 |
fix(session): remember where aliasSession moved a session
aliasSession knew that two labels name the same peer, then threw that knowledge away. The binding lived only in the caller's memory, so a restart lost it — and the peer could not repair it from its side. First contact forces the receiver to label a session by the only sender hint a relay surfaces, an 8-byte signing-key fingerprint (`fp:<hex>`). Once the peer announces its canonical address, aliasSession moves the session there. But the peer keeps sending under `fp:<hex>`, because its transport derives the same label from the same hint every time. After a restart the session sat under the canonical address, inbound frames resolved to `fp:<hex>`, and nothing matched. The peer held a valid session so it never re-ran X3DH: the failure was permanent, and only a manual re-link cleared it. Observed in Prism as `No session for address: fp:579c3b335d66e2c0` on every receive for three days, with a phone whose every RPC timed out. StorageProvider gains saveSessionAlias / getSessionAlias / removeSessionAliasesFor, optional so third-party implementations keep compiling, and implemented across all seven backends. Lookups resolve through resolveLabel(), which runs BEFORE the peer mutex — locking the alias while mutating the canonical session would let an aliased and a canonical caller ratchet the same state concurrently. A live session under a label always wins over an alias, and prekey envelopes never resolve: both keep a re-link establishing a fresh session instead of being redirected into the stale one. Aliases are dropped in resetSession and acceptIdentityChange, and memoized so the hot path costs no extra read. The sdk.test.ts case that asserted a dead fp-label encoded the old behaviour; it now pins the new contract. Verified: 1166 tests pass (from 1160). With alias persistence disabled as a negative control, 5 of the 6 new tests fail, including both restart cases. Also drops `baseUrl` from the consumer-strict tsconfig — removed in TS 6.0, and it was failing the typecheck that gates publishing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
|||
| 80c410f518 |
release(v4.9.0): relay-side encrypted blob primitive + SDK Profile namespace
Ships the Prism FR (encrypted-profile-storage-v4.9.md) as a generic relay-side encrypted blob primitive: deterministically-located, AEAD-sealed blobs keyed by a 32-byte slotId derived client-side via HKDF from the user's master key. Unlocks credential-only bootstrap of new devices into existing E2EE state — no QR, no physical access. Server: BlobStore interface + Memory/Sqlite/Postgres impls, createBlobRoutes for GET/PUT/DELETE /v1/blob/:slotId with TOFU pubkey auth and If-Match CAS (409/412 semantics). Mounted on the same Hono app as the inbox; SHADE_BLOB_PG_URL / SHADE_BLOB_DB_PATH / SHADE_DISABLE_BLOB env-var plumbing in standalone. SDK: createProfileNamespace high-level wrapper (HKDF derivation, random-nonce AEAD seal, slotId-bound AAD) + low-level BlobClient. Cross-platform test vectors in test-vectors/blob-storage.json. New errors: ConflictError (409), PreconditionFailedError (412). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| 1fb59a7076 |
release(v4.8.0): sender-fingerprint attribution + Inbox.start race fix
Two unblocking changes for first-contact flows.
Sender attribution: relay captures shortHash(senderSigningKey) at
PUT time (after signature verification, no new trust surface) and
surfaces it on bridge push (IncomingMessage.from) + inbox-fetch
(FetchedBlob.from) + DecryptHandler raw arg. Apps receiving a prekey
envelope from a never-before-seen peer can now bootstrap X3DH via
shade.receive('fp:<hex>', env) — pre-4.8 the wire envelope didn't
authenticate the sender and there was no out-of-band hint to use.
Idempotent ALTER TABLE migrations for SQLite + Postgres add a
sender_fp TEXT column; legacy rows surface as from=undefined
(inter-version compat).
Inbox.start() race: pre-4.8 start() called register() fire-and-forget
AND schedulePoll(0) synchronously, so the first poll on a fresh
address often beat the register HTTP RTT and got SHADE_NOT_FOUND.
start() now defers; register() success kicks schedulePoll(0). Manual
tick() is unaffected (deliberate user action, no gating).
Both reported by Prism. Tests cover all five acceptance criteria
from the sender-attribution request (PUT capture, bridge surface,
fetch surface, inter-version compat, end-to-end pair smoke) plus
the three from the race-fix request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| e6fdf31b49 |
release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled
V3.1 → V3.12 consolidated and tagged for the first GA release. Wire format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers byte-for-byte. The version bump is semantic: audit-cycle complete, opt-in surface fully exposed, threat model refreshed for every new surface. Highlights: - All 24 @shade/* packages bumped to 4.0.0 in lockstep. - CHANGELOG 4.0.0 section is the canonical manifest of what landed. - THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12 Web-Worker boundary) + residual-risks table refreshed. - OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox, bridge, observer, /metrics, /healthz, /ready. - MIGRATION 0.3.x → 4.0 documented + smoke-tested against shade migrate-storage on a real SQLite DB. - docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer. - scripts/soak.ts harness for the GA-stable 2-week soak window. - All V*.md plans archived under docs/archive/ with Status: Done. - Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen non-realtime stack. Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green. Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports version 4.0.0 on /health. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
|||
| fa770d3063 |
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
Some checks failed
Test / test (push) Has been cancelled
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).
@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
+ per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
(> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
/ shadeFilesReadStreamId correlation. Server-side TransformStream
bridges accept inbound transfers immediately (engine rejects chunks
that arrive before accept) and park the readable for the matching
RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
uploadDirectory()/downloadDirectory() with bounded concurrency pool
(default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
policy (traversal + percent-decode hardening), fingerprint gate
(required/optional/reject), pluggable Ed25519 sig verification with
±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.
Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
u16 to u32. The previous u16 silently truncated payloads above
64 KiB — a hard correctness ceiling that blocked inline file ops
up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
only. Cross-platform Kotlin port (android/shade-android) updated to
match; test-vectors/wire-format.json regenerated.
Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
Concurrent decryptions of the same peer raced ratchet state
(manifested as sporadic "Failed to decrypt — wrong key or tampered
data" under load — surfaced once concurrent uploadDirectory pumped
many writes in flight). Encrypt was already serialized via
Shade.send's encryptChains; decrypt is now serialized at the
manager layer too.
@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
application-level key/value pairs that round-trip verbatim through
stream-init plaintext. Used by @shade/files for write/read
correlation; available to any consumer.
@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
BackgroundTasks.setHook(name, fn) for runtime hook registration.
Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
shade-sdk streams-bridge + shade-widgets transfer hooks were
uncommitted prior to this session. Including them keeps the
workspace consistent at 0.3.0 since @shade/files depends on them.
Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
3× stable). Coverage spans unit (inline-threshold + concurrency),
integration (read-write inline + streams up to 1 MiB, walk +
upload/download directory, custom-op, metrics, SDK namespace
end-to-end), and security (tampered-envelope sig verification,
replay window, fingerprint gate, rate-limit + quota).
Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 7e0f7320a9 |
feat(container): M-Box 1-8 — stack-agnostic standalone Docker container
Some checks failed
Test / test (push) Has been cancelled
Shade now ships as a self-contained Docker image. Deploy one container per project, any stack (Bun, Python, Go, Rust, Kotlin) can talk to it via plain HTTP. Zero coupling to consumer codebases. M-Box 1: Stale identity cleanup API - touchIdentity + purgeStaleIdentities on PrekeyStore interface - Implemented for Memory, SQLite, and Postgres backends - SQLite adds last_activity_at column with migration ALTER for existing DBs - Postgres adds the same via raw SQL with IF NOT EXISTS guards - Routes call touchIdentity on register, bundle fetch, replenish - 4 new tests for the cleanup API M-Box 2: Stale cleanup background task - StaleCleanupTask runs purge on startup + every 24h (configurable) - Reads SHADE_STALE_DAYS (default 30) and SHADE_CLEANUP_INTERVAL_HOURS - Wired into standalone.ts, stopped on graceful shutdown - 5 new tests for the task M-Box 3: Observer baked into the container - standalone.ts conditionally mounts @shade/observer at /shade-observer when SHADE_OBSERVER_TOKEN is set (and >= 16 chars) - Shared PrekeyServerEvents emitter feeds both routes and observer - @shade/observer added as optional dependency of @shade/server M-Box 4: Dockerfile with dashboard build - Multi-stage build: oven/bun:1 builder → oven/bun:1-alpine runtime - COPY packages/ wholesale so workspace lockfile resolves cleanly - RUN bun run build inside shade-dashboard → dist/ → observer/dist/ - Non-root shade user, /data volume, healthcheck, env defaults - Final image: 260 MB M-Box 5: OpenAPI spec for stack-agnostic clients - packages/shade-server/openapi.yaml documents all 9 endpoints with request/response schemas, security (Ed25519 signatures + bearer token) - createOpenApiRoutes serves /openapi.yaml and /docs (Redoc viewer) - Any language can generate a client with openapi-generator M-Box 6: Docker CI pipeline - .gitea/workflows/docker.yml builds + pushes on git tag v* - scripts/build-docker.ts for local builds, supports --push with GITEA_TOKEN - Root package.json: build:docker, publish:docker scripts M-Box 7: Deployment documentation - packages/shade-server/README rewritten: 5-line quickstart with the image - docs/DEPLOYMENT.md: full reference, env vars, backup, Dokploy, PG setup - examples/05-dokploy-deployment/docker-compose.yml updated to pull published image (gt.zyon.no/stian/shade-prekey:latest) - Root README deployment section rewritten M-Box 8: End-to-end verification - Image builds locally (bun run build:docker) - /health, /openapi.yaml, /docs, /metrics, /shade-observer all respond - 401 without observer token, 200 with - Real SDK client round-trip: Alice → container → Bob → reply → Alice - Persistence: identity + prekeys survive container restart (count 20→18 as expected from two bundle fetches) 285 tests passing, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
|||
| 1bd5436506 |
feat(hardening): M-Hard 6 + 7 — PostgreSQL backend + production server infra
M-Hard 6: PostgreSQL Storage Backend - @shade/storage-postgres with PostgresStorage + PostgresPrekeyStore - Drizzle-style raw SQL ensureClientTables / ensurePrekeyServerTables - All tables prefixed `shade_` to avoid collisions in shared databases - DELETE ... FOR UPDATE SKIP LOCKED for concurrent OTPK consumption - Tests skip gracefully without SHADE_TEST_PG_URL, run against real PG when set M-Hard 7: Production Server Infrastructure - Structured JSON logger (logger.ts) — SHADE_LOG_LEVEL configurable - Health endpoints (/health, /healthz, /ready) — Kubernetes-friendly - Prometheus metrics (/metrics) — counters, histograms, middleware - Graceful shutdown with SIGTERM/SIGINT handlers + store close - Production Dockerfile with non-root user, healthcheck, multi-stage build - docker-compose.yml example for Dokploy with optional PostgreSQL 193 tests passing, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
