Files
Shade/packages/shade-server
Sterister fa770d3063
Some checks failed
Test / test (push) Has been cancelled
feat(files): @shade/files 0.3.0 — E2EE filesystem RPC primitive
M-Files-1..6 land the full files-RPC layer + everything 0.3.0 needs to
ship. Apps keep their own UI; this layer ships the typed RPC, the
streams bridge for content I/O, and production hooks (rate limit,
retention, fingerprint gate, metrics).

@shade/files (NEW)
- Standard ops: list/stat/mkdir/delete/move/read/write/getThumbnail with
  Zod-validated wire schemas + clean user-handler types.
- Custom ops: typed via TypeScript declaration merging on CustomOpsMap
  + per-op Zod schemas; client.custom('app.foo', {...}) is fully typed.
- Content I/O: inline (≤ 256 KiB plaintext) base64-in-RPC; streams
  (> 256 KiB) ride @shade/transfer via userMetadata.shadeFilesWriteId
  / shadeFilesReadStreamId correlation. Server-side TransformStream
  bridges accept inbound transfers immediately (engine rejects chunks
  that arrive before accept) and park the readable for the matching
  RPC.
- Directory ops: walk(path, opts) async-iterable depth-first walker;
  uploadDirectory()/downloadDirectory() with bounded concurrency pool
  (default 4, cap 16), aggregated progress, abort.
- Production hooks (callback-based, vendor-neutral): rate-limit (op +
  byte), idempotency cache (LRU + TTL + in-flight de-dupe), path
  policy (traversal + percent-decode hardening), fingerprint gate
  (required/optional/reject), pluggable Ed25519 sig verification with
  ±5 min replay window, onMetric sink (standard names).
- React hooks (subpath @shade/files/react): ShadeFilesProvider,
  useShadeFiles, useFileList, useFileTransfer/Upload/Download.
- Shade.files.serve(handler) + Shade.files.client(peer) high-level
  entrypoint in @shade/sdk; lazy + memoized; one handler per Shade.

Wire format bump
- @shade/proto wire VERSION 0x01 → 0x02. Length prefixes changed from
  u16 to u32. The previous u16 silently truncated payloads above
  64 KiB — a hard correctness ceiling that blocked inline file ops
  up to 256 KiB. Wire-incompatible with 0.2.x peers; new sessions
  only. Cross-platform Kotlin port (android/shade-android) updated to
  match; test-vectors/wire-format.json regenerated.

Concurrency safety
- ShadeSessionManager.encrypt/.decrypt now run under per-peer mutex.
  Concurrent decryptions of the same peer raced ratchet state
  (manifested as sporadic "Failed to decrypt — wrong key or tampered
  data" under load — surfaced once concurrent uploadDirectory pumped
  many writes in flight). Encrypt was already serialized via
  Shade.send's encryptChains; decrypt is now serialized at the
  manager layer too.

@shade/streams extension
- StreamMetadata.userMetadata?: Record<string, string> for
  application-level key/value pairs that round-trip verbatim through
  stream-init plaintext. Used by @shade/files for write/read
  correlation; available to any consumer.

@shade/sdk extension
- Shade.files getter (lazy + memoized).
- BackgroundHooks.onPruneFiles + periodic timer (default 5 min) +
  BackgroundTasks.setHook(name, fn) for runtime hook registration.

Bundles in-flight 0.2.0 work
- packages/shade-streams/, packages/shade-transfer/, related
  shade-sdk streams-bridge + shade-widgets transfer hooks were
  uncommitted prior to this session. Including them keeps the
  workspace consistent at 0.3.0 since @shade/files depends on them.

Tests
- 74 new tests in @shade/files (572 → 646 workspace pass; 0 fail;
  3× stable). Coverage spans unit (inline-threshold + concurrency),
  integration (read-write inline + streams up to 1 MiB, walk +
  upload/download directory, custom-op, metrics, SDK namespace
  end-to-end), and security (tampered-envelope sig verification,
  replay window, fingerprint gate, rate-limit + quota).

Release artifacts
- All packages bumped to 0.3.0 via scripts/bump-version.ts.
- scripts/publish-all.ts PACKAGES updated with shade-files in
  topological order (after shade-transfer, before shade-sdk).
- bun run publish:dry clean (14 packed, 0 failed).
- examples/08-files-browser/ — three-process CLI demo (prekey + Bob
  server + Alice CLI) covering list/stat/mkdir/delete/upload/download.
- docs/files.md — full API + design doc.
- CHANGELOG.md 0.3.0 entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 14:00:01 +02:00
..

@shade/server — Shade Prekey Server (standalone container)

A self-contained Docker image that provides the prekey server, OpenAPI contract, observer dashboard, and stale cleanup — everything a project needs to adopt Shade, with zero coupling to the consumer's stack.

Deploy in 2 minutes

docker run -d \
  --name my-project-shade \
  -v my-project-shade:/data \
  -p 3900:3900 \
  -e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
  gt.zyon.no/stian/shade-prekey:latest

Done. Your prekey server is live:

  • http://localhost:3900/health — health check
  • http://localhost:3900/openapi.yaml — API contract for any language
  • http://localhost:3900/docs — interactive API reference (Redoc)
  • http://localhost:3900/shade-observer/dashboard/ — live debugger (token required)
  • http://localhost:3900/v1/keys/* — prekey REST API

Your consumer projects (Nova, Orchestrator, Python apps, anything) then point at http://localhost:3900 as their prekeyServer URL.

One container per project

The recommended architecture is one Shade container per project:

nova-shade          (Docker container, SQLite volume)    ← Nova backend + Android app
orchestrator-shade  (Docker container, SQLite volume)    ← Orchestrator hub + workstations
future-project     (Docker container, SQLite volume)    ← Any future app

Each project owns its own container, its own volume, its own observer token. Zero cross-project coupling. If one project's Shade is down, the others keep running.

Environment variables

Var Default Description
PORT 3900 HTTP port
SHADE_PREKEY_DB_PATH /data/shade-prekeys.db SQLite file path
SHADE_PREKEY_PG_URL unset Postgres connection string. If set, overrides SQLite.
SHADE_OBSERVER_TOKEN unset Bearer token for the dashboard. Min 16 chars. Unset = observer disabled.
SHADE_STALE_DAYS 30 Purge identities with no activity in N days
SHADE_CLEANUP_INTERVAL_HOURS 24 How often the cleanup task runs
SHADE_LOG_LEVEL info debug / info / warn / error

Persistence

The /data volume holds the SQLite database. Back it up by copying the .db file (use SQLite's online backup API or just stop the container briefly).

To switch to Postgres, set SHADE_PREKEY_PG_URL=postgres://user:pass@host/db. Tables will be created automatically with the shade_server_* prefix.

Stale cleanup

Identities that have no activity (no bundle fetches, no replenishments, no registration updates) for more than SHADE_STALE_DAYS days are automatically purged. This keeps the database bounded even if users never unregister cleanly.

Using from your project

Any language can speak to a Shade container — it's just HTTP. See openapi.yaml for the full contract.

TypeScript / Bun:

import { createShade } from '@shade/sdk';
const shade = await createShade({ prekeyServer: 'http://my-project-shade:3900' });

Python / Go / Rust: generate a client from the OpenAPI spec with openapi-generator, or implement the wire protocol directly (8 endpoints, Ed25519 signatures documented in the spec).

Android: use the shade-android Kotlin module. Same wire protocol, verified by cross-platform test vectors.

Building locally

bun run build:docker                          # build shade-prekey:dev
bun run build:docker -- --tag v1.0.0          # custom tag
GITEA_TOKEN=... bun run publish:docker        # build + push to registry

CI publishing

Tag a release and CI publishes automatically:

git tag v1.0.0
git push --tags

.gitea/workflows/docker.yml runs tests, builds the image, and pushes both v1.0.0 and latest tags to gt.zyon.no/stian/shade-prekey.