feat(container): M-Box 1-8 — stack-agnostic standalone Docker container
Some checks failed
Test / test (push) Has been cancelled
Some checks failed
Test / test (push) Has been cancelled
Shade now ships as a self-contained Docker image. Deploy one container per project, any stack (Bun, Python, Go, Rust, Kotlin) can talk to it via plain HTTP. Zero coupling to consumer codebases. M-Box 1: Stale identity cleanup API - touchIdentity + purgeStaleIdentities on PrekeyStore interface - Implemented for Memory, SQLite, and Postgres backends - SQLite adds last_activity_at column with migration ALTER for existing DBs - Postgres adds the same via raw SQL with IF NOT EXISTS guards - Routes call touchIdentity on register, bundle fetch, replenish - 4 new tests for the cleanup API M-Box 2: Stale cleanup background task - StaleCleanupTask runs purge on startup + every 24h (configurable) - Reads SHADE_STALE_DAYS (default 30) and SHADE_CLEANUP_INTERVAL_HOURS - Wired into standalone.ts, stopped on graceful shutdown - 5 new tests for the task M-Box 3: Observer baked into the container - standalone.ts conditionally mounts @shade/observer at /shade-observer when SHADE_OBSERVER_TOKEN is set (and >= 16 chars) - Shared PrekeyServerEvents emitter feeds both routes and observer - @shade/observer added as optional dependency of @shade/server M-Box 4: Dockerfile with dashboard build - Multi-stage build: oven/bun:1 builder → oven/bun:1-alpine runtime - COPY packages/ wholesale so workspace lockfile resolves cleanly - RUN bun run build inside shade-dashboard → dist/ → observer/dist/ - Non-root shade user, /data volume, healthcheck, env defaults - Final image: 260 MB M-Box 5: OpenAPI spec for stack-agnostic clients - packages/shade-server/openapi.yaml documents all 9 endpoints with request/response schemas, security (Ed25519 signatures + bearer token) - createOpenApiRoutes serves /openapi.yaml and /docs (Redoc viewer) - Any language can generate a client with openapi-generator M-Box 6: Docker CI pipeline - .gitea/workflows/docker.yml builds + pushes on git tag v* - scripts/build-docker.ts for local builds, supports --push with GITEA_TOKEN - Root package.json: build:docker, publish:docker scripts M-Box 7: Deployment documentation - packages/shade-server/README rewritten: 5-line quickstart with the image - docs/DEPLOYMENT.md: full reference, env vars, backup, Dokploy, PG setup - examples/05-dokploy-deployment/docker-compose.yml updated to pull published image (gt.zyon.no/stian/shade-prekey:latest) - Root README deployment section rewritten M-Box 8: End-to-end verification - Image builds locally (bun run build:docker) - /health, /openapi.yaml, /docs, /metrics, /shade-observer all respond - 401 without observer token, 200 with - Real SDK client round-trip: Alice → container → Bob → reply → Alice - Persistence: identity + prekeys survive container restart (count 20→18 as expected from two bundle fetches) 285 tests passing, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
52
.gitea/workflows/docker.yml
Normal file
52
.gitea/workflows/docker.yml
Normal file
@@ -0,0 +1,52 @@
|
||||
name: Docker build and publish
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
docker:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install Bun
|
||||
run: curl -fsSL https://bun.sh/install | bash
|
||||
|
||||
- name: Install dependencies
|
||||
run: ~/.bun/bin/bun install --frozen-lockfile
|
||||
|
||||
- name: Run tests (gate)
|
||||
run: ~/.bun/bin/bun test --recursive
|
||||
|
||||
- name: Extract version from tag
|
||||
id: version
|
||||
run: |
|
||||
VERSION="${GITHUB_REF_NAME#v}"
|
||||
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Log in to Gitea container registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: gt.zyon.no
|
||||
username: Stian
|
||||
password: ${{ secrets.GITEA_PUBLISH_TOKEN }}
|
||||
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: packages/shade-server/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
gt.zyon.no/stian/shade-prekey:${{ steps.version.outputs.version }}
|
||||
gt.zyon.no/stian/shade-prekey:latest
|
||||
labels: |
|
||||
org.opencontainers.image.version=${{ steps.version.outputs.version }}
|
||||
org.opencontainers.image.source=https://gt.zyon.no/Stian/Shade
|
||||
org.opencontainers.image.revision=${{ github.sha }}
|
||||
33
README.md
33
README.md
@@ -154,25 +154,28 @@ bun run publish:all
|
||||
- [examples/](./examples/) — Runnable example applications
|
||||
- [MIGRATION.md](./MIGRATION.md) — How to replace existing crypto with Shade
|
||||
|
||||
## Deployment
|
||||
## Deployment — one container per project
|
||||
|
||||
For containerized deployment (Docker/Dokploy):
|
||||
Shade ships as a self-contained Docker image. Deploy one container per project, point your app at it, done. Any stack (Bun, Python, Go, Rust, Kotlin) can use it — the container exposes a plain HTTP API documented in OpenAPI.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
shade-prekey:
|
||||
image: shade-prekey-server:latest
|
||||
ports:
|
||||
- "3900:3900"
|
||||
volumes:
|
||||
- shade-data:/data
|
||||
environment:
|
||||
- SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
||||
volumes:
|
||||
shade-data:
|
||||
```bash
|
||||
docker run -d \
|
||||
--name my-project-shade \
|
||||
-v my-project-shade:/data \
|
||||
-p 3900:3900 \
|
||||
-e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
|
||||
gt.zyon.no/stian/shade-prekey:latest
|
||||
```
|
||||
|
||||
The SQLite database persists to a Docker volume so all keys and prekey bundles survive restarts.
|
||||
The container includes:
|
||||
- **Prekey server** — `/v1/keys/*` REST API
|
||||
- **Observer dashboard** — `/shade-observer/dashboard/` (off unless token is set)
|
||||
- **OpenAPI spec** — `/openapi.yaml` and interactive `/docs` viewer
|
||||
- **Prometheus metrics** — `/metrics`
|
||||
- **Health check** — `/health`
|
||||
- **Stale cleanup** — purges inactive identities automatically
|
||||
|
||||
See [docs/DEPLOYMENT.md](./docs/DEPLOYMENT.md) for the full deployment guide, environment variables, PostgreSQL config, backup strategy, and Dokploy instructions.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
6
bun.lock
6
bun.lock
@@ -106,6 +106,12 @@
|
||||
"devDependencies": {
|
||||
"@shade/crypto-web": "workspace:*",
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@shade/crypto-web": "workspace:*",
|
||||
"@shade/observer": "workspace:*",
|
||||
"@shade/storage-postgres": "workspace:*",
|
||||
"@shade/storage-sqlite": "workspace:*",
|
||||
},
|
||||
},
|
||||
"packages/shade-storage-postgres": {
|
||||
"name": "@shade/storage-postgres",
|
||||
|
||||
136
docs/DEPLOYMENT.md
Normal file
136
docs/DEPLOYMENT.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# Deploying Shade
|
||||
|
||||
Shade ships as a single Docker image that contains the prekey server, observer dashboard, OpenAPI contract, and stale cleanup. You deploy one container per project.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name my-project-shade \
|
||||
-v my-project-shade:/data \
|
||||
-p 3900:3900 \
|
||||
-e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
|
||||
gt.zyon.no/stian/shade-prekey:latest
|
||||
```
|
||||
|
||||
That's it. Your projects can now register identities and exchange prekey bundles via `http://localhost:3900`.
|
||||
|
||||
## Why one container per project
|
||||
|
||||
Each project is self-contained. Nova doesn't depend on Orchestrator being up. Future projects can be added without touching existing ones. The container is tiny (~260 MB), idle resource usage is near zero, and each container owns its own SQLite volume.
|
||||
|
||||
```
|
||||
Project A Project B Future projects
|
||||
───────── ───────── ────────────────
|
||||
app + frontend app + frontend app + frontend
|
||||
│ │ │
|
||||
↓ ↓ ↓
|
||||
shade-a container shade-b container shade-n container
|
||||
(port 3900) (port 3901) (port 390n)
|
||||
sqlite volume sqlite volume sqlite volume
|
||||
```
|
||||
|
||||
## Dokploy deployment
|
||||
|
||||
1. Go to Dokploy → Projects → New Project → Docker Compose
|
||||
2. Paste the `docker-compose.yml` from [`examples/05-dokploy-deployment`](../examples/05-dokploy-deployment/docker-compose.yml)
|
||||
3. Set env vars in the Dokploy UI:
|
||||
- `SHADE_OBSERVER_TOKEN` (generate a random 32+ char string)
|
||||
4. Set the container name unique per project (e.g., `nova-shade`, `orchestrator-shade`)
|
||||
5. Deploy
|
||||
|
||||
Dokploy will pull the image, create the volume, and health check the container automatically.
|
||||
|
||||
## Volumes and backup
|
||||
|
||||
The `/data` volume holds:
|
||||
- `shade-prekeys.db` — the SQLite database with all identities, prekeys, and activity timestamps
|
||||
- WAL journal files for crash safety
|
||||
|
||||
**Backup:** Copy the `.db` file while the container is stopped, or use SQLite's online backup API:
|
||||
```bash
|
||||
docker exec my-project-shade sqlite3 /data/shade-prekeys.db ".backup /data/backup.db"
|
||||
docker cp my-project-shade:/data/backup.db ./local-backup.db
|
||||
```
|
||||
|
||||
**Restore:** Stop the container, copy the `.db` file into the volume, restart.
|
||||
|
||||
## PostgreSQL instead of SQLite
|
||||
|
||||
If you want to share a Postgres instance (or need HA), set `SHADE_PREKEY_PG_URL`:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- SHADE_PREKEY_PG_URL=postgres://shade:shade@postgres:5432/shade
|
||||
```
|
||||
|
||||
Tables will be created automatically with the `shade_server_*` prefix, so they coexist cleanly with any other tables in the same database.
|
||||
|
||||
## Environment variable reference
|
||||
|
||||
| Var | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `PORT` | `3900` | HTTP port |
|
||||
| `SHADE_PREKEY_DB_PATH` | `/data/shade-prekeys.db` | SQLite file location |
|
||||
| `SHADE_PREKEY_PG_URL` | unset | Postgres URL (overrides SQLite) |
|
||||
| `SHADE_OBSERVER_TOKEN` | unset | Enables dashboard at `/shade-observer/dashboard/`. Min 16 chars. |
|
||||
| `SHADE_STALE_DAYS` | `30` | Purge identities with no activity in N days |
|
||||
| `SHADE_CLEANUP_INTERVAL_HOURS` | `24` | Cleanup cycle interval |
|
||||
| `SHADE_LOG_LEVEL` | `info` | `debug` / `info` / `warn` / `error` |
|
||||
|
||||
## Health and observability
|
||||
|
||||
- **Health:** `GET /health` — returns `{"status":"ok"}` when the storage backend is reachable. Docker's HEALTHCHECK uses this.
|
||||
- **Metrics:** `GET /metrics` — Prometheus format with counters, histograms, and gauges for all routes.
|
||||
- **OpenAPI:** `GET /openapi.yaml` — machine-readable API contract for any language.
|
||||
- **Redoc viewer:** `GET /docs` — human-readable API reference.
|
||||
- **Dashboard:** `GET /shade-observer/dashboard/` — live activity viewer (requires token).
|
||||
|
||||
## Stale cleanup
|
||||
|
||||
Identities with no activity (no bundle fetches, no replenishments, no registration refreshes) for more than `SHADE_STALE_DAYS` days are automatically purged from the database. This keeps the database bounded without manual housekeeping.
|
||||
|
||||
The cleanup task runs once at startup and then every `SHADE_CLEANUP_INTERVAL_HOURS` hours. Each cycle logs the number of purged identities.
|
||||
|
||||
## Multiple Shade instances on the same host
|
||||
|
||||
Run multiple projects side-by-side with different container names and ports:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
nova-shade:
|
||||
container_name: nova-shade
|
||||
image: gt.zyon.no/stian/shade-prekey:latest
|
||||
ports: ["3900:3900"]
|
||||
volumes: [nova-shade-data:/data]
|
||||
environment:
|
||||
- SHADE_OBSERVER_TOKEN=nova-token-32-chars-minimum-xxx
|
||||
|
||||
orch-shade:
|
||||
container_name: orch-shade
|
||||
image: gt.zyon.no/stian/shade-prekey:latest
|
||||
ports: ["3901:3900"]
|
||||
volumes: [orch-shade-data:/data]
|
||||
environment:
|
||||
- SHADE_OBSERVER_TOKEN=orch-token-32-chars-minimum-xxx
|
||||
|
||||
volumes:
|
||||
nova-shade-data:
|
||||
orch-shade-data:
|
||||
```
|
||||
|
||||
## CI publishing
|
||||
|
||||
Tagged releases auto-publish to the Gitea container registry via `.gitea/workflows/docker.yml`. To cut a release:
|
||||
|
||||
```bash
|
||||
bun run version 1.0.1
|
||||
git push --tags
|
||||
```
|
||||
|
||||
## Security notes
|
||||
|
||||
- **Never commit `SHADE_OBSERVER_TOKEN`.** Use Dokploy secrets or environment-specific `.env` files.
|
||||
- The prekey server stores **public keys only**. No private keys ever touch it.
|
||||
- Rate limiting is on by default (5 registrations per hour per IP, etc.). Tune via `createPrekeyRoutes` options if embedding, or configure at reverse-proxy level for the container.
|
||||
- Put the container behind a reverse proxy (Traefik, Caddy) for TLS termination.
|
||||
@@ -1,9 +1,22 @@
|
||||
# Shade Prekey Server — Dokploy / Docker Compose deployment
|
||||
#
|
||||
# Pulls the published image from Gitea's container registry. Change
|
||||
# `my-project-shade` to something project-specific so you can run multiple
|
||||
# Shade instances side-by-side (one per project).
|
||||
#
|
||||
# Usage:
|
||||
# docker compose up -d
|
||||
#
|
||||
# To build locally from source instead of pulling, uncomment the `build:`
|
||||
# section and comment out `image:`.
|
||||
|
||||
services:
|
||||
shade-prekey:
|
||||
image: shade-prekey-server:latest
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: packages/shade-server/Dockerfile
|
||||
container_name: my-project-shade
|
||||
image: gt.zyon.no/stian/shade-prekey:latest
|
||||
# build:
|
||||
# context: ../..
|
||||
# dockerfile: packages/shade-server/Dockerfile
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "3900:3900"
|
||||
@@ -13,6 +26,8 @@ services:
|
||||
- PORT=3900
|
||||
- SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
||||
- SHADE_LOG_LEVEL=info
|
||||
- SHADE_STALE_DAYS=30
|
||||
- SHADE_CLEANUP_INTERVAL_HOURS=24
|
||||
# Optional: enable the live observer dashboard at /shade-observer/dashboard/
|
||||
# Token must be at least 16 characters. Use a real secret in production.
|
||||
# - SHADE_OBSERVER_TOKEN=change-me-must-be-at-least-16-chars
|
||||
|
||||
@@ -13,7 +13,9 @@
|
||||
"test:cli": "cd packages/shade-cli && bun test",
|
||||
"version": "bun run scripts/bump-version.ts",
|
||||
"publish:dry": "DRY_RUN=1 bun run scripts/publish-all.ts",
|
||||
"publish:all": "bun run scripts/publish-all.ts"
|
||||
"publish:all": "bun run scripts/publish-all.ts",
|
||||
"build:docker": "bun run scripts/build-docker.ts",
|
||||
"publish:docker": "bun run scripts/build-docker.ts -- --push"
|
||||
},
|
||||
"devDependencies": {
|
||||
"bun-types": "^1.3.11"
|
||||
|
||||
@@ -1,7 +1,17 @@
|
||||
node_modules
|
||||
dist
|
||||
**/node_modules
|
||||
**/dist
|
||||
**/dist-build
|
||||
*.tsbuildinfo
|
||||
.git
|
||||
.gitea
|
||||
.github
|
||||
.DS_Store
|
||||
**/tests
|
||||
**/*.test.ts
|
||||
**/tmp
|
||||
**/tmp-data
|
||||
examples
|
||||
bench
|
||||
android
|
||||
docs
|
||||
.env*
|
||||
|
||||
@@ -3,28 +3,27 @@ FROM oven/bun:1 AS builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Copy workspace root
|
||||
COPY package.json bun.lock ./
|
||||
COPY tsconfig.json ./
|
||||
# Copy workspace root config
|
||||
COPY package.json bun.lock tsconfig.json ./
|
||||
|
||||
# Copy all packages we depend on
|
||||
COPY packages/shade-core ./packages/shade-core
|
||||
COPY packages/shade-crypto-web ./packages/shade-crypto-web
|
||||
COPY packages/shade-server ./packages/shade-server
|
||||
COPY packages/shade-storage-sqlite ./packages/shade-storage-sqlite
|
||||
COPY packages/shade-storage-postgres ./packages/shade-storage-postgres
|
||||
# Copy all packages the server + observer + dashboard need
|
||||
COPY packages ./packages
|
||||
|
||||
RUN bun install --frozen-lockfile
|
||||
|
||||
# Build the dashboard SPA → dist/, then copy to observer's dist/
|
||||
RUN cd packages/shade-dashboard && bun run build
|
||||
|
||||
# ─── Production stage ───────────────────────────────────────
|
||||
FROM oven/bun:1-alpine
|
||||
|
||||
LABEL org.opencontainers.image.title="Shade Prekey Server"
|
||||
LABEL org.opencontainers.image.description="E2EE prekey distribution server (Signal Protocol)"
|
||||
LABEL org.opencontainers.image.source="https://github.com/Sterister/Shade"
|
||||
LABEL org.opencontainers.image.description="E2EE prekey distribution server with live observer (Signal Protocol)"
|
||||
LABEL org.opencontainers.image.source="https://gt.zyon.no/Stian/Shade"
|
||||
LABEL org.opencontainers.image.licenses="MIT"
|
||||
LABEL org.opencontainers.image.vendor="Stian"
|
||||
|
||||
# Install curl for healthcheck
|
||||
# curl for healthcheck
|
||||
RUN apk add --no-cache curl
|
||||
|
||||
# Non-root user
|
||||
@@ -33,7 +32,7 @@ RUN addgroup -S shade && adduser -S shade -G shade
|
||||
WORKDIR /app
|
||||
COPY --from=builder --chown=shade:shade /build /app
|
||||
|
||||
# Persistent data directory
|
||||
# Persistent data directory (SQLite file or any sidecar state)
|
||||
RUN mkdir -p /data && chown shade:shade /data
|
||||
VOLUME ["/data"]
|
||||
|
||||
@@ -41,9 +40,12 @@ USER shade
|
||||
|
||||
EXPOSE 3900
|
||||
|
||||
# Default to SQLite on the persistent volume
|
||||
# Defaults — override via `docker run -e`
|
||||
ENV SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
||||
ENV PORT=3900
|
||||
ENV SHADE_LOG_LEVEL=info
|
||||
ENV SHADE_STALE_DAYS=30
|
||||
ENV SHADE_CLEANUP_INTERVAL_HOURS=24
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||
CMD curl -fsS http://localhost:${PORT}/health || exit 1
|
||||
|
||||
90
packages/shade-server/README.md
Normal file
90
packages/shade-server/README.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# @shade/server — Shade Prekey Server (standalone container)
|
||||
|
||||
A self-contained Docker image that provides the prekey server, OpenAPI contract, observer dashboard, and stale cleanup — **everything a project needs to adopt Shade**, with zero coupling to the consumer's stack.
|
||||
|
||||
## Deploy in 2 minutes
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name my-project-shade \
|
||||
-v my-project-shade:/data \
|
||||
-p 3900:3900 \
|
||||
-e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
|
||||
gt.zyon.no/stian/shade-prekey:latest
|
||||
```
|
||||
|
||||
Done. Your prekey server is live:
|
||||
- `http://localhost:3900/health` — health check
|
||||
- `http://localhost:3900/openapi.yaml` — API contract for any language
|
||||
- `http://localhost:3900/docs` — interactive API reference (Redoc)
|
||||
- `http://localhost:3900/shade-observer/dashboard/` — live debugger (token required)
|
||||
- `http://localhost:3900/v1/keys/*` — prekey REST API
|
||||
|
||||
Your consumer projects (Nova, Orchestrator, Python apps, anything) then point at `http://localhost:3900` as their `prekeyServer` URL.
|
||||
|
||||
## One container per project
|
||||
|
||||
The recommended architecture is **one Shade container per project**:
|
||||
|
||||
```
|
||||
nova-shade (Docker container, SQLite volume) ← Nova backend + Android app
|
||||
orchestrator-shade (Docker container, SQLite volume) ← Orchestrator hub + workstations
|
||||
future-project (Docker container, SQLite volume) ← Any future app
|
||||
```
|
||||
|
||||
Each project owns its own container, its own volume, its own observer token. Zero cross-project coupling. If one project's Shade is down, the others keep running.
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Var | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `PORT` | `3900` | HTTP port |
|
||||
| `SHADE_PREKEY_DB_PATH` | `/data/shade-prekeys.db` | SQLite file path |
|
||||
| `SHADE_PREKEY_PG_URL` | unset | Postgres connection string. If set, overrides SQLite. |
|
||||
| `SHADE_OBSERVER_TOKEN` | unset | Bearer token for the dashboard. Min 16 chars. Unset = observer disabled. |
|
||||
| `SHADE_STALE_DAYS` | `30` | Purge identities with no activity in N days |
|
||||
| `SHADE_CLEANUP_INTERVAL_HOURS` | `24` | How often the cleanup task runs |
|
||||
| `SHADE_LOG_LEVEL` | `info` | `debug` / `info` / `warn` / `error` |
|
||||
|
||||
## Persistence
|
||||
|
||||
The `/data` volume holds the SQLite database. Back it up by copying the `.db` file (use SQLite's online backup API or just stop the container briefly).
|
||||
|
||||
To switch to Postgres, set `SHADE_PREKEY_PG_URL=postgres://user:pass@host/db`. Tables will be created automatically with the `shade_server_*` prefix.
|
||||
|
||||
## Stale cleanup
|
||||
|
||||
Identities that have no activity (no bundle fetches, no replenishments, no registration updates) for more than `SHADE_STALE_DAYS` days are automatically purged. This keeps the database bounded even if users never unregister cleanly.
|
||||
|
||||
## Using from your project
|
||||
|
||||
Any language can speak to a Shade container — it's just HTTP. See [openapi.yaml](./openapi.yaml) for the full contract.
|
||||
|
||||
**TypeScript / Bun:**
|
||||
```ts
|
||||
import { createShade } from '@shade/sdk';
|
||||
const shade = await createShade({ prekeyServer: 'http://my-project-shade:3900' });
|
||||
```
|
||||
|
||||
**Python / Go / Rust:** generate a client from the OpenAPI spec with `openapi-generator`, or implement the wire protocol directly (8 endpoints, Ed25519 signatures documented in the spec).
|
||||
|
||||
**Android:** use the `shade-android` Kotlin module. Same wire protocol, verified by cross-platform test vectors.
|
||||
|
||||
## Building locally
|
||||
|
||||
```bash
|
||||
bun run build:docker # build shade-prekey:dev
|
||||
bun run build:docker -- --tag v1.0.0 # custom tag
|
||||
GITEA_TOKEN=... bun run publish:docker # build + push to registry
|
||||
```
|
||||
|
||||
## CI publishing
|
||||
|
||||
Tag a release and CI publishes automatically:
|
||||
|
||||
```bash
|
||||
git tag v1.0.0
|
||||
git push --tags
|
||||
```
|
||||
|
||||
`.gitea/workflows/docker.yml` runs tests, builds the image, and pushes both `v1.0.0` and `latest` tags to `gt.zyon.no/stian/shade-prekey`.
|
||||
377
packages/shade-server/openapi.yaml
Normal file
377
packages/shade-server/openapi.yaml
Normal file
@@ -0,0 +1,377 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: Shade Prekey Server
|
||||
description: |
|
||||
Signal Protocol prekey distribution server. Stores identity public keys
|
||||
and prekey bundles. Any language can implement a client — this spec
|
||||
documents the wire contract.
|
||||
|
||||
**Security model:** Write operations (register, replenish, delete) are
|
||||
authenticated by Ed25519 signatures over the request body. Bundle fetches
|
||||
are anonymous. See the `SignedPayload` schema for the signing format.
|
||||
version: "1.0.0"
|
||||
license:
|
||||
name: MIT
|
||||
url: https://gt.zyon.no/Stian/Shade/raw/branch/main/LICENSE
|
||||
|
||||
servers:
|
||||
- url: https://shade.example.com
|
||||
description: Replace with your project's prekey server URL
|
||||
|
||||
paths:
|
||||
/health:
|
||||
get:
|
||||
summary: Health check
|
||||
description: Returns 200 if the server and its storage backend are reachable.
|
||||
tags: [Infrastructure]
|
||||
responses:
|
||||
'200':
|
||||
description: Healthy
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/HealthResponse'
|
||||
'503':
|
||||
description: Unhealthy
|
||||
|
||||
/metrics:
|
||||
get:
|
||||
summary: Prometheus metrics
|
||||
description: Counter, histogram, and gauge metrics in Prometheus text format.
|
||||
tags: [Infrastructure]
|
||||
responses:
|
||||
'200':
|
||||
description: Metrics
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
example: |
|
||||
# HELP shade_requests_total Total HTTP requests
|
||||
# TYPE shade_requests_total counter
|
||||
shade_requests_total{route="/v1/keys/register",status="200"} 42
|
||||
|
||||
/openapi.yaml:
|
||||
get:
|
||||
summary: This OpenAPI spec
|
||||
description: Serves the YAML spec itself, so clients can auto-download it.
|
||||
tags: [Infrastructure]
|
||||
responses:
|
||||
'200':
|
||||
description: OpenAPI spec
|
||||
content:
|
||||
application/yaml:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/v1/keys/register:
|
||||
post:
|
||||
summary: Register identity and upload prekey bundle
|
||||
description: |
|
||||
Register a new identity with the prekey server, or update an existing
|
||||
one. The request body must be signed with the identity's own Ed25519
|
||||
signing key (TOFU — first signature establishes the identity).
|
||||
tags: [Keys]
|
||||
security:
|
||||
- Ed25519Signature: []
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
allOf:
|
||||
- $ref: '#/components/schemas/SignedPayload'
|
||||
- type: object
|
||||
required: [address, identitySigningKey, identityDHKey, signedPreKey]
|
||||
properties:
|
||||
address:
|
||||
$ref: '#/components/schemas/Address'
|
||||
identitySigningKey:
|
||||
type: string
|
||||
description: Ed25519 public key, base64
|
||||
identityDHKey:
|
||||
type: string
|
||||
description: X25519 public key, base64
|
||||
signedPreKey:
|
||||
$ref: '#/components/schemas/SignedPreKeyEntry'
|
||||
oneTimePreKeys:
|
||||
type: array
|
||||
items:
|
||||
$ref: '#/components/schemas/OneTimePreKeyEntry'
|
||||
responses:
|
||||
'200':
|
||||
description: Registered
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
ok: { type: boolean }
|
||||
'400':
|
||||
$ref: '#/components/responses/ValidationError'
|
||||
'401':
|
||||
$ref: '#/components/responses/Unauthorized'
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/v1/keys/bundle/{address}:
|
||||
get:
|
||||
summary: Fetch a prekey bundle (anonymous)
|
||||
description: |
|
||||
Fetch an identity's prekey bundle so Alice can start an X3DH session
|
||||
with Bob. Each call consumes one one-time prekey (FIFO) if any are
|
||||
available — if not, returns a bundle without the one-time prekey.
|
||||
tags: [Keys]
|
||||
parameters:
|
||||
- name: address
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: '#/components/schemas/Address'
|
||||
responses:
|
||||
'200':
|
||||
description: Prekey bundle
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/PreKeyBundle'
|
||||
'404':
|
||||
$ref: '#/components/responses/NotFound'
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/v1/keys/count/{address}:
|
||||
get:
|
||||
summary: Get remaining one-time prekey count (anonymous)
|
||||
tags: [Keys]
|
||||
parameters:
|
||||
- name: address
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: '#/components/schemas/Address'
|
||||
responses:
|
||||
'200':
|
||||
description: Count
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
count: { type: integer, minimum: 0 }
|
||||
|
||||
/v1/keys/replenish:
|
||||
post:
|
||||
summary: Upload more one-time prekeys (signed)
|
||||
tags: [Keys]
|
||||
security:
|
||||
- Ed25519Signature: []
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
allOf:
|
||||
- $ref: '#/components/schemas/SignedPayload'
|
||||
- type: object
|
||||
required: [address, oneTimePreKeys]
|
||||
properties:
|
||||
address:
|
||||
$ref: '#/components/schemas/Address'
|
||||
oneTimePreKeys:
|
||||
type: array
|
||||
items:
|
||||
$ref: '#/components/schemas/OneTimePreKeyEntry'
|
||||
responses:
|
||||
'200':
|
||||
description: Replenished
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
ok: { type: boolean }
|
||||
remaining: { type: integer, minimum: 0 }
|
||||
'401':
|
||||
$ref: '#/components/responses/Unauthorized'
|
||||
'404':
|
||||
$ref: '#/components/responses/NotFound'
|
||||
'429':
|
||||
$ref: '#/components/responses/RateLimited'
|
||||
|
||||
/v1/keys/{address}:
|
||||
delete:
|
||||
summary: Unregister an identity (signed)
|
||||
tags: [Keys]
|
||||
security:
|
||||
- Ed25519Signature: []
|
||||
parameters:
|
||||
- name: address
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: '#/components/schemas/Address'
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/SignedPayload'
|
||||
responses:
|
||||
'200':
|
||||
description: Deleted
|
||||
'401':
|
||||
$ref: '#/components/responses/Unauthorized'
|
||||
|
||||
/shade-observer/api/state:
|
||||
get:
|
||||
summary: Current observer snapshot (optional)
|
||||
description: |
|
||||
Returns the aggregated state visible to the dashboard. Only available
|
||||
if the server was started with `SHADE_OBSERVER_TOKEN` set.
|
||||
tags: [Observer]
|
||||
security:
|
||||
- ObserverBearerToken: []
|
||||
responses:
|
||||
'200':
|
||||
description: Observer state snapshot
|
||||
'401':
|
||||
description: Invalid or missing token
|
||||
|
||||
components:
|
||||
schemas:
|
||||
Address:
|
||||
type: string
|
||||
description: |
|
||||
An address on the prekey server. Alphanumeric plus `:_-.` characters,
|
||||
max 256 chars. NFKC-normalized. Format is typically
|
||||
`user@domain:deviceId` or `device:uuid`.
|
||||
pattern: '^[a-zA-Z0-9][a-zA-Z0-9:_\-.]{0,255}$'
|
||||
example: "alice@example.com:phone"
|
||||
|
||||
SignedPayload:
|
||||
type: object
|
||||
description: |
|
||||
Base type for all signed request bodies. The `signature` field is
|
||||
an Ed25519 signature over the canonical JSON of the payload
|
||||
(all fields sorted, `signature` omitted) using the identity's
|
||||
signing private key. `signedAt` must be within ±5 minutes of
|
||||
server time.
|
||||
required: [signedAt, signature]
|
||||
properties:
|
||||
signedAt:
|
||||
type: integer
|
||||
format: int64
|
||||
description: Unix epoch milliseconds
|
||||
signature:
|
||||
type: string
|
||||
description: Ed25519 signature, base64
|
||||
|
||||
SignedPreKeyEntry:
|
||||
type: object
|
||||
required: [keyId, publicKey, signature]
|
||||
properties:
|
||||
keyId:
|
||||
type: integer
|
||||
minimum: 0
|
||||
publicKey:
|
||||
type: string
|
||||
description: X25519 public key, base64
|
||||
signature:
|
||||
type: string
|
||||
description: Ed25519 signature over `publicKey`, base64
|
||||
|
||||
OneTimePreKeyEntry:
|
||||
type: object
|
||||
required: [keyId, publicKey]
|
||||
properties:
|
||||
keyId:
|
||||
type: integer
|
||||
minimum: 0
|
||||
publicKey:
|
||||
type: string
|
||||
description: X25519 public key, base64
|
||||
|
||||
PreKeyBundle:
|
||||
type: object
|
||||
required: [identitySigningKey, identityDHKey, signedPreKey]
|
||||
properties:
|
||||
identitySigningKey:
|
||||
type: string
|
||||
description: Ed25519 public key, base64
|
||||
identityDHKey:
|
||||
type: string
|
||||
description: X25519 public key, base64
|
||||
signedPreKey:
|
||||
$ref: '#/components/schemas/SignedPreKeyEntry'
|
||||
oneTimePreKey:
|
||||
$ref: '#/components/schemas/OneTimePreKeyEntry'
|
||||
|
||||
HealthResponse:
|
||||
type: object
|
||||
properties:
|
||||
status:
|
||||
type: string
|
||||
enum: [ok, error]
|
||||
service:
|
||||
type: string
|
||||
|
||||
ErrorResponse:
|
||||
type: object
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
code:
|
||||
type: string
|
||||
pattern: '^SHADE_'
|
||||
message:
|
||||
type: string
|
||||
|
||||
responses:
|
||||
ValidationError:
|
||||
description: Invalid request body
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
Unauthorized:
|
||||
description: Signature verification failed or replay window exceeded
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
NotFound:
|
||||
description: Address not registered
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
RateLimited:
|
||||
description: Rate limit exceeded
|
||||
headers:
|
||||
Retry-After:
|
||||
schema:
|
||||
type: integer
|
||||
description: Seconds until the client can retry
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ErrorResponse'
|
||||
|
||||
securitySchemes:
|
||||
Ed25519Signature:
|
||||
type: apiKey
|
||||
in: header
|
||||
name: X-Shade-Signature-Info
|
||||
description: |
|
||||
NOT an actual header. Signature is carried inside the JSON request
|
||||
body as the `signature` and `signedAt` fields (see SignedPayload).
|
||||
The canonical form signed is the request body with all keys sorted
|
||||
and the `signature` field omitted. Use your Ed25519 signing key.
|
||||
ObserverBearerToken:
|
||||
type: http
|
||||
scheme: bearer
|
||||
description: |
|
||||
`Authorization: Bearer <SHADE_OBSERVER_TOKEN>`. The observer also
|
||||
accepts the token via `?token=...` query string for SSE endpoints
|
||||
that can't set headers.
|
||||
@@ -8,6 +8,12 @@
|
||||
"@shade/core": "workspace:*",
|
||||
"hono": "^4.12.12"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@shade/observer": "workspace:*",
|
||||
"@shade/storage-sqlite": "workspace:*",
|
||||
"@shade/storage-postgres": "workspace:*",
|
||||
"@shade/crypto-web": "workspace:*"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@shade/crypto-web": "workspace:*"
|
||||
}
|
||||
|
||||
70
packages/shade-server/src/cleanup.ts
Normal file
70
packages/shade-server/src/cleanup.ts
Normal file
@@ -0,0 +1,70 @@
|
||||
import type { PrekeyStore } from './store.js';
|
||||
import { logger } from './logger.js';
|
||||
|
||||
/**
|
||||
* Background task that periodically purges stale identities from the store.
|
||||
*
|
||||
* "Stale" = no activity (register, fetch bundle, replenish, delete) for
|
||||
* more than `staleDays` days. The threshold and interval are configurable
|
||||
* via env vars:
|
||||
* SHADE_STALE_DAYS (default 30)
|
||||
* SHADE_CLEANUP_INTERVAL_HOURS (default 24)
|
||||
*/
|
||||
export class StaleCleanupTask {
|
||||
private timer: ReturnType<typeof setInterval> | null = null;
|
||||
private running = false;
|
||||
private readonly staleMs: number;
|
||||
private readonly intervalMs: number;
|
||||
|
||||
constructor(
|
||||
private readonly store: PrekeyStore,
|
||||
options: { staleDays?: number; intervalHours?: number } = {},
|
||||
) {
|
||||
const staleDays = options.staleDays
|
||||
?? Number(process.env.SHADE_STALE_DAYS ?? 30);
|
||||
const intervalHours = options.intervalHours
|
||||
?? Number(process.env.SHADE_CLEANUP_INTERVAL_HOURS ?? 24);
|
||||
this.staleMs = staleDays * 24 * 60 * 60 * 1000;
|
||||
this.intervalMs = intervalHours * 60 * 60 * 1000;
|
||||
}
|
||||
|
||||
start(): void {
|
||||
if (this.running) return;
|
||||
this.running = true;
|
||||
// Run once immediately (so operators see it in the logs at startup)
|
||||
this.runOnce().catch((err) => {
|
||||
logger.error('Initial stale cleanup failed', { error: String(err) });
|
||||
});
|
||||
this.timer = setInterval(() => {
|
||||
this.runOnce().catch((err) => {
|
||||
logger.error('Stale cleanup failed', { error: String(err) });
|
||||
});
|
||||
}, this.intervalMs);
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
this.running = false;
|
||||
if (this.timer) {
|
||||
clearInterval(this.timer);
|
||||
this.timer = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Run a single cleanup cycle. Exposed for tests and manual triggers. */
|
||||
async runOnce(): Promise<number> {
|
||||
const count = await this.store.purgeStaleIdentities(this.staleMs);
|
||||
if (count > 0) {
|
||||
logger.info('Stale cleanup purged identities', {
|
||||
count,
|
||||
staleDays: this.staleMs / (24 * 60 * 60 * 1000),
|
||||
});
|
||||
} else {
|
||||
logger.debug('Stale cleanup: nothing to purge');
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
get isRunning(): boolean {
|
||||
return this.running;
|
||||
}
|
||||
}
|
||||
@@ -23,6 +23,7 @@ export class MemoryPrekeyStore implements PrekeyStore {
|
||||
private identities = new Map<string, IdentityRecord>();
|
||||
private signedPreKeys = new Map<string, SignedPreKeyRecord>();
|
||||
private oneTimePreKeys = new Map<string, OneTimePreKeyRecord[]>();
|
||||
private lastActivity = new Map<string, number>();
|
||||
|
||||
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
||||
this.identities.set(address, { identitySigningKey, identityDHKey });
|
||||
@@ -60,5 +61,32 @@ export class MemoryPrekeyStore implements PrekeyStore {
|
||||
this.identities.delete(address);
|
||||
this.signedPreKeys.delete(address);
|
||||
this.oneTimePreKeys.delete(address);
|
||||
this.lastActivity.delete(address);
|
||||
}
|
||||
|
||||
// ─── Stale cleanup ────────────────────────────────────
|
||||
|
||||
async touchIdentity(address: string): Promise<void> {
|
||||
this.lastActivity.set(address, Date.now());
|
||||
}
|
||||
|
||||
async purgeStaleIdentities(olderThanMs: number): Promise<number> {
|
||||
const cutoff = Date.now() - olderThanMs;
|
||||
const stale: string[] = [];
|
||||
for (const [address, ts] of this.lastActivity) {
|
||||
if (ts < cutoff) stale.push(address);
|
||||
}
|
||||
// Also purge identities that have a row but were never touched
|
||||
// (last_activity = 0/undefined before first touch)
|
||||
for (const address of this.identities.keys()) {
|
||||
if (!this.lastActivity.has(address)) stale.push(address);
|
||||
}
|
||||
for (const address of stale) {
|
||||
this.identities.delete(address);
|
||||
this.signedPreKeys.delete(address);
|
||||
this.oneTimePreKeys.delete(address);
|
||||
this.lastActivity.delete(address);
|
||||
}
|
||||
return stale.length;
|
||||
}
|
||||
}
|
||||
|
||||
54
packages/shade-server/src/openapi.ts
Normal file
54
packages/shade-server/src/openapi.ts
Normal file
@@ -0,0 +1,54 @@
|
||||
import { Hono } from 'hono';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { dirname, join } from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
/**
|
||||
* Serves the OpenAPI spec at /openapi.yaml and a Redoc HTML viewer at /docs.
|
||||
*
|
||||
* Any language can fetch /openapi.yaml and generate a client with
|
||||
* openapi-generator. The /docs HTML viewer is a thin Redoc wrapper.
|
||||
*/
|
||||
export function createOpenApiRoutes(specPath?: string): Hono {
|
||||
const app = new Hono();
|
||||
|
||||
const defaultPath = join(
|
||||
dirname(fileURLToPath(import.meta.url)),
|
||||
'..',
|
||||
'openapi.yaml',
|
||||
);
|
||||
const path = specPath ?? defaultPath;
|
||||
|
||||
app.get('/openapi.yaml', (c) => {
|
||||
if (!existsSync(path)) {
|
||||
return c.text('OpenAPI spec not found', 404);
|
||||
}
|
||||
const content = readFileSync(path, 'utf-8');
|
||||
c.header('Content-Type', 'application/yaml; charset=utf-8');
|
||||
return c.body(content);
|
||||
});
|
||||
|
||||
app.get('/docs', (c) => {
|
||||
c.header('Content-Type', 'text/html; charset=utf-8');
|
||||
return c.body(redocHtml());
|
||||
});
|
||||
|
||||
return app;
|
||||
}
|
||||
|
||||
function redocHtml(): string {
|
||||
return `<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Shade API Reference</title>
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<link href="https://fonts.googleapis.com/css?family=Montserrat:300,400,700|Roboto:300,400,700" rel="stylesheet">
|
||||
<style>body { margin: 0; padding: 0; }</style>
|
||||
</head>
|
||||
<body>
|
||||
<redoc spec-url="/openapi.yaml"></redoc>
|
||||
<script src="https://cdn.redoc.ly/redoc/latest/bundles/redoc.standalone.js"></script>
|
||||
</body>
|
||||
</html>`;
|
||||
}
|
||||
@@ -157,6 +157,9 @@ export function createPrekeyRoutes(
|
||||
};
|
||||
}
|
||||
|
||||
// Update activity so stale cleanup doesn't purge active addresses
|
||||
await store.touchIdentity(address);
|
||||
|
||||
events?.emit('server.bundle_fetched', {
|
||||
address,
|
||||
hadOneTimePreKey: oneTimePreKey != null,
|
||||
@@ -192,6 +195,7 @@ export function createPrekeyRoutes(
|
||||
publicKey: b64ToBytes(k.publicKey),
|
||||
}));
|
||||
await store.saveOneTimePreKeys(addr, keys);
|
||||
await store.touchIdentity(addr);
|
||||
|
||||
const count = await store.getOneTimePreKeyCount(addr);
|
||||
events?.emit('server.prekeys_replenished', {
|
||||
|
||||
@@ -3,6 +3,9 @@ import { SubtleCryptoProvider } from '@shade/crypto-web';
|
||||
import { createPrekeyRoutes } from './routes.js';
|
||||
import { createHealthRoutes } from './health.js';
|
||||
import { createMetricsRoutes, metricsMiddleware } from './metrics.js';
|
||||
import { createOpenApiRoutes } from './openapi.js';
|
||||
import { PrekeyServerEvents } from './events.js';
|
||||
import { StaleCleanupTask } from './cleanup.js';
|
||||
import { logger } from './logger.js';
|
||||
import type { PrekeyStore } from './store.js';
|
||||
|
||||
@@ -41,13 +44,47 @@ function maskUrl(url: string): string {
|
||||
|
||||
const crypto = new SubtleCryptoProvider();
|
||||
const store = await createStore();
|
||||
const events = new PrekeyServerEvents();
|
||||
|
||||
// Compose the full app: metrics middleware + health + metrics + prekey routes
|
||||
const app = new Hono();
|
||||
app.use('*', metricsMiddleware());
|
||||
app.route('/', createHealthRoutes(store, VERSION));
|
||||
app.route('/', createMetricsRoutes());
|
||||
app.route('/', createPrekeyRoutes(store, crypto));
|
||||
app.route('/', createOpenApiRoutes());
|
||||
app.route('/', createPrekeyRoutes(store, crypto, { events }));
|
||||
|
||||
// ─── Optional: Observer + Dashboard ──────────────────────────
|
||||
|
||||
const observerToken = process.env.SHADE_OBSERVER_TOKEN;
|
||||
if (observerToken && observerToken.length >= 16) {
|
||||
try {
|
||||
const { createObserver } = await import('@shade/observer');
|
||||
const observer = createObserver({
|
||||
token: observerToken,
|
||||
serverEvents: events,
|
||||
});
|
||||
app.route('/shade-observer', observer);
|
||||
logger.info('Observer enabled', { path: '/shade-observer/dashboard/' });
|
||||
} catch (err) {
|
||||
logger.warn('Observer module not available, skipping', { error: String(err) });
|
||||
}
|
||||
} else if (observerToken) {
|
||||
logger.warn('SHADE_OBSERVER_TOKEN is set but too short (needs ≥16 chars), observer disabled');
|
||||
} else {
|
||||
logger.info('Observer disabled (SHADE_OBSERVER_TOKEN not set)');
|
||||
}
|
||||
|
||||
// ─── Stale cleanup task ──────────────────────────────────────
|
||||
|
||||
const cleanupTask = new StaleCleanupTask(store);
|
||||
cleanupTask.start();
|
||||
logger.info('Stale cleanup task started', {
|
||||
staleDays: Number(process.env.SHADE_STALE_DAYS ?? 30),
|
||||
intervalHours: Number(process.env.SHADE_CLEANUP_INTERVAL_HOURS ?? 24),
|
||||
});
|
||||
|
||||
// ─── Start HTTP server ───────────────────────────────────────
|
||||
|
||||
const port = Number(process.env.PORT ?? 3900);
|
||||
|
||||
@@ -64,6 +101,7 @@ async function shutdown(signal: string) {
|
||||
logger.info('Shutting down', { signal });
|
||||
|
||||
try {
|
||||
cleanupTask.stop();
|
||||
server.stop();
|
||||
if ('close' in store && typeof store.close === 'function') {
|
||||
await store.close();
|
||||
|
||||
@@ -45,4 +45,22 @@ export interface PrekeyStore {
|
||||
|
||||
/** Delete all keys for an address */
|
||||
deleteAll(address: string): Promise<void>;
|
||||
|
||||
// ─── Stale cleanup ─────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Update the last-activity timestamp for an address to the current time.
|
||||
* Called on every read or write that references the address, so stale
|
||||
* cleanup only removes addresses nobody has touched in a long time.
|
||||
*/
|
||||
touchIdentity(address: string): Promise<void>;
|
||||
|
||||
/**
|
||||
* Purge every identity whose last activity is older than `olderThanMs`
|
||||
* milliseconds ago. Cascades deletion to signed prekeys and one-time
|
||||
* prekeys for the affected addresses.
|
||||
*
|
||||
* @returns the number of addresses purged
|
||||
*/
|
||||
purgeStaleIdentities(olderThanMs: number): Promise<number>;
|
||||
}
|
||||
|
||||
65
packages/shade-server/tests/cleanup-api.test.ts
Normal file
65
packages/shade-server/tests/cleanup-api.test.ts
Normal file
@@ -0,0 +1,65 @@
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { MemoryPrekeyStore } from '../src/memory-store.js';
|
||||
|
||||
function rand(n: number): Uint8Array {
|
||||
const buf = new Uint8Array(n);
|
||||
globalThis.crypto.getRandomValues(buf);
|
||||
return buf;
|
||||
}
|
||||
|
||||
describe('PrekeyStore cleanup API', () => {
|
||||
test('touchIdentity updates last activity', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
await store.saveIdentity('alice', rand(32), rand(32));
|
||||
await store.touchIdentity('alice');
|
||||
// Verify identity is still there
|
||||
expect(await store.getIdentity('alice')).not.toBeNull();
|
||||
});
|
||||
|
||||
test('purgeStaleIdentities removes old addresses', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
|
||||
// Alice was active long ago
|
||||
await store.saveIdentity('alice', rand(32), rand(32));
|
||||
await store.saveSignedPreKey('alice', 1, rand(32), rand(64));
|
||||
await store.saveOneTimePreKeys('alice', [{ keyId: 100, publicKey: rand(32) }]);
|
||||
// Manually set alice's activity to 1 day ago
|
||||
await store.touchIdentity('alice');
|
||||
(store as any).lastActivity.set('alice', Date.now() - 2 * 24 * 60 * 60 * 1000);
|
||||
|
||||
// Bob is fresh
|
||||
await store.saveIdentity('bob', rand(32), rand(32));
|
||||
await store.touchIdentity('bob');
|
||||
|
||||
// Purge anything older than 1 day
|
||||
const count = await store.purgeStaleIdentities(24 * 60 * 60 * 1000);
|
||||
expect(count).toBe(1);
|
||||
|
||||
// Alice is gone
|
||||
expect(await store.getIdentity('alice')).toBeNull();
|
||||
expect(await store.getSignedPreKey('alice')).toBeNull();
|
||||
expect(await store.getOneTimePreKeyCount('alice')).toBe(0);
|
||||
|
||||
// Bob is still there
|
||||
expect(await store.getIdentity('bob')).not.toBeNull();
|
||||
});
|
||||
|
||||
test('purge returns 0 when nothing is stale', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
await store.saveIdentity('alice', rand(32), rand(32));
|
||||
await store.touchIdentity('alice');
|
||||
|
||||
const count = await store.purgeStaleIdentities(60 * 60 * 1000); // 1 hour
|
||||
expect(count).toBe(0);
|
||||
});
|
||||
|
||||
test('untouched identities are considered stale', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
// Save without touching — simulates an ancient identity from before cleanup API
|
||||
await store.saveIdentity('ancient', rand(32), rand(32));
|
||||
|
||||
const count = await store.purgeStaleIdentities(1000);
|
||||
expect(count).toBe(1);
|
||||
expect(await store.getIdentity('ancient')).toBeNull();
|
||||
});
|
||||
});
|
||||
74
packages/shade-server/tests/cleanup.test.ts
Normal file
74
packages/shade-server/tests/cleanup.test.ts
Normal file
@@ -0,0 +1,74 @@
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { StaleCleanupTask } from '../src/cleanup.js';
|
||||
import { MemoryPrekeyStore } from '../src/memory-store.js';
|
||||
|
||||
function rand(n: number): Uint8Array {
|
||||
const buf = new Uint8Array(n);
|
||||
globalThis.crypto.getRandomValues(buf);
|
||||
return buf;
|
||||
}
|
||||
|
||||
describe('StaleCleanupTask', () => {
|
||||
test('runs once immediately on start', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
// Save an untouched (so stale) identity
|
||||
await store.saveIdentity('ancient', rand(32), rand(32));
|
||||
|
||||
const task = new StaleCleanupTask(store, { staleDays: 1, intervalHours: 24 });
|
||||
task.start();
|
||||
// Give the immediate run a microtask to complete
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
|
||||
expect(await store.getIdentity('ancient')).toBeNull();
|
||||
task.stop();
|
||||
});
|
||||
|
||||
test('runOnce returns count of purged addresses', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
await store.saveIdentity('ancient1', rand(32), rand(32));
|
||||
await store.saveIdentity('ancient2', rand(32), rand(32));
|
||||
|
||||
const task = new StaleCleanupTask(store, { staleDays: 1 });
|
||||
const count = await task.runOnce();
|
||||
expect(count).toBe(2);
|
||||
});
|
||||
|
||||
test('leaves fresh identities alone', async () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
await store.saveIdentity('fresh', rand(32), rand(32));
|
||||
await store.touchIdentity('fresh');
|
||||
|
||||
const task = new StaleCleanupTask(store, { staleDays: 30 });
|
||||
const count = await task.runOnce();
|
||||
expect(count).toBe(0);
|
||||
expect(await store.getIdentity('fresh')).not.toBeNull();
|
||||
});
|
||||
|
||||
test('stop clears the interval', () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
const task = new StaleCleanupTask(store, { staleDays: 1, intervalHours: 1 });
|
||||
task.start();
|
||||
expect(task.isRunning).toBe(true);
|
||||
task.stop();
|
||||
expect(task.isRunning).toBe(false);
|
||||
});
|
||||
|
||||
test('reads defaults from env vars', () => {
|
||||
const store = new MemoryPrekeyStore();
|
||||
const oldDays = process.env.SHADE_STALE_DAYS;
|
||||
const oldHours = process.env.SHADE_CLEANUP_INTERVAL_HOURS;
|
||||
process.env.SHADE_STALE_DAYS = '7';
|
||||
process.env.SHADE_CLEANUP_INTERVAL_HOURS = '12';
|
||||
try {
|
||||
const task = new StaleCleanupTask(store);
|
||||
// staleMs = 7 days, intervalMs = 12 hours
|
||||
expect((task as any).staleMs).toBe(7 * 24 * 60 * 60 * 1000);
|
||||
expect((task as any).intervalMs).toBe(12 * 60 * 60 * 1000);
|
||||
} finally {
|
||||
if (oldDays !== undefined) process.env.SHADE_STALE_DAYS = oldDays;
|
||||
else delete process.env.SHADE_STALE_DAYS;
|
||||
if (oldHours !== undefined) process.env.SHADE_CLEANUP_INTERVAL_HOURS = oldHours;
|
||||
else delete process.env.SHADE_CLEANUP_INTERVAL_HOURS;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -66,9 +66,19 @@ export async function ensurePrekeyServerTables(sql: Sql): Promise<void> {
|
||||
CREATE TABLE IF NOT EXISTS shade_server_identities (
|
||||
address TEXT PRIMARY KEY,
|
||||
identity_signing_key TEXT NOT NULL,
|
||||
identity_dh_key TEXT NOT NULL
|
||||
identity_dh_key TEXT NOT NULL,
|
||||
last_activity_at BIGINT NOT NULL DEFAULT 0
|
||||
)
|
||||
`;
|
||||
// Migrate existing deployments (no-op if column exists)
|
||||
await sql`
|
||||
ALTER TABLE shade_server_identities
|
||||
ADD COLUMN IF NOT EXISTS last_activity_at BIGINT NOT NULL DEFAULT 0
|
||||
`;
|
||||
await sql`
|
||||
CREATE INDEX IF NOT EXISTS shade_server_identities_activity_idx
|
||||
ON shade_server_identities(last_activity_at)
|
||||
`;
|
||||
await sql`
|
||||
CREATE TABLE IF NOT EXISTS shade_server_signed_prekeys (
|
||||
address TEXT PRIMARY KEY,
|
||||
|
||||
@@ -34,11 +34,12 @@ export class PostgresPrekeyStore implements PrekeyStore {
|
||||
|
||||
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
||||
await this.sql`
|
||||
INSERT INTO shade_server_identities (address, identity_signing_key, identity_dh_key)
|
||||
VALUES (${address}, ${toBase64(identitySigningKey)}, ${toBase64(identityDHKey)})
|
||||
INSERT INTO shade_server_identities (address, identity_signing_key, identity_dh_key, last_activity_at)
|
||||
VALUES (${address}, ${toBase64(identitySigningKey)}, ${toBase64(identityDHKey)}, ${Date.now()})
|
||||
ON CONFLICT (address) DO UPDATE SET
|
||||
identity_signing_key = EXCLUDED.identity_signing_key,
|
||||
identity_dh_key = EXCLUDED.identity_dh_key
|
||||
identity_dh_key = EXCLUDED.identity_dh_key,
|
||||
last_activity_at = EXCLUDED.last_activity_at
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -122,4 +123,31 @@ export class PostgresPrekeyStore implements PrekeyStore {
|
||||
await sql`DELETE FROM shade_server_one_time_prekeys WHERE address = ${address}`;
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Stale cleanup ──────────────────────────────────────
|
||||
|
||||
async touchIdentity(address: string): Promise<void> {
|
||||
await this.sql`
|
||||
UPDATE shade_server_identities
|
||||
SET last_activity_at = ${Date.now()}
|
||||
WHERE address = ${address}
|
||||
`;
|
||||
}
|
||||
|
||||
async purgeStaleIdentities(olderThanMs: number): Promise<number> {
|
||||
const cutoff = Date.now() - olderThanMs;
|
||||
const rows = await this.sql<Array<{ address: string }>>`
|
||||
SELECT address FROM shade_server_identities
|
||||
WHERE last_activity_at < ${cutoff}
|
||||
`;
|
||||
if (rows.length === 0) return 0;
|
||||
|
||||
const addresses = rows.map((r) => r.address);
|
||||
await this.sql.begin(async (sql) => {
|
||||
await sql`DELETE FROM shade_server_identities WHERE address = ANY(${addresses})`;
|
||||
await sql`DELETE FROM shade_server_signed_prekeys WHERE address = ANY(${addresses})`;
|
||||
await sql`DELETE FROM shade_server_one_time_prekeys WHERE address = ANY(${addresses})`;
|
||||
});
|
||||
return addresses.length;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,6 +25,8 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
||||
deleteIdentity: ReturnType<Database['prepare']>;
|
||||
deleteSignedPreKey: ReturnType<Database['prepare']>;
|
||||
deleteOTPKs: ReturnType<Database['prepare']>;
|
||||
touchIdentity: ReturnType<Database['prepare']>;
|
||||
findStale: ReturnType<Database['prepare']>;
|
||||
};
|
||||
|
||||
constructor(dbPath?: string) {
|
||||
@@ -40,7 +42,8 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
||||
CREATE TABLE IF NOT EXISTS identities (
|
||||
address TEXT PRIMARY KEY,
|
||||
identity_signing_key TEXT NOT NULL,
|
||||
identity_dh_key TEXT NOT NULL
|
||||
identity_dh_key TEXT NOT NULL,
|
||||
last_activity_at INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS signed_prekeys (
|
||||
address TEXT PRIMARY KEY,
|
||||
@@ -55,12 +58,23 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
||||
public_key TEXT NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_otp_address ON one_time_prekeys(address);
|
||||
CREATE INDEX IF NOT EXISTS idx_identities_activity ON identities(last_activity_at);
|
||||
`);
|
||||
|
||||
// Migrate existing databases: add last_activity_at if missing
|
||||
try {
|
||||
const cols = this.db.prepare('PRAGMA table_info(identities)').all() as any[];
|
||||
const hasActivity = cols.some((c) => c.name === 'last_activity_at');
|
||||
if (!hasActivity) {
|
||||
this.db.exec('ALTER TABLE identities ADD COLUMN last_activity_at INTEGER NOT NULL DEFAULT 0');
|
||||
this.db.exec('CREATE INDEX IF NOT EXISTS idx_identities_activity ON identities(last_activity_at)');
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
|
||||
private prepareStatements() {
|
||||
this.stmts = {
|
||||
saveIdentity: this.db.prepare('INSERT OR REPLACE INTO identities (address, identity_signing_key, identity_dh_key) VALUES (?, ?, ?)'),
|
||||
saveIdentity: this.db.prepare('INSERT OR REPLACE INTO identities (address, identity_signing_key, identity_dh_key, last_activity_at) VALUES (?, ?, ?, ?)'),
|
||||
getIdentity: this.db.prepare('SELECT identity_signing_key, identity_dh_key FROM identities WHERE address = ?'),
|
||||
saveSignedPreKey: this.db.prepare('INSERT OR REPLACE INTO signed_prekeys (address, key_id, public_key, signature) VALUES (?, ?, ?, ?)'),
|
||||
getSignedPreKey: this.db.prepare('SELECT key_id, public_key, signature FROM signed_prekeys WHERE address = ?'),
|
||||
@@ -70,6 +84,8 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
||||
deleteIdentity: this.db.prepare('DELETE FROM identities WHERE address = ?'),
|
||||
deleteSignedPreKey: this.db.prepare('DELETE FROM signed_prekeys WHERE address = ?'),
|
||||
deleteOTPKs: this.db.prepare('DELETE FROM one_time_prekeys WHERE address = ?'),
|
||||
touchIdentity: this.db.prepare('UPDATE identities SET last_activity_at = ? WHERE address = ?'),
|
||||
findStale: this.db.prepare('SELECT address FROM identities WHERE last_activity_at < ?'),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -78,7 +94,7 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
||||
}
|
||||
|
||||
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
||||
this.stmts.saveIdentity.run(address, toBase64(identitySigningKey), toBase64(identityDHKey));
|
||||
this.stmts.saveIdentity.run(address, toBase64(identitySigningKey), toBase64(identityDHKey), Date.now());
|
||||
}
|
||||
|
||||
async getIdentity(address: string): Promise<{ identitySigningKey: Uint8Array; identityDHKey: Uint8Array } | null> {
|
||||
@@ -135,4 +151,26 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
||||
});
|
||||
deleteAllTx();
|
||||
}
|
||||
|
||||
// ─── Stale cleanup ──────────────────────────────────────
|
||||
|
||||
async touchIdentity(address: string): Promise<void> {
|
||||
this.stmts.touchIdentity.run(Date.now(), address);
|
||||
}
|
||||
|
||||
async purgeStaleIdentities(olderThanMs: number): Promise<number> {
|
||||
const cutoff = Date.now() - olderThanMs;
|
||||
const staleRows = this.stmts.findStale.all(cutoff) as Array<{ address: string }>;
|
||||
if (staleRows.length === 0) return 0;
|
||||
|
||||
const purgeTx = this.db.transaction(() => {
|
||||
for (const row of staleRows) {
|
||||
this.stmts.deleteIdentity.run(row.address);
|
||||
this.stmts.deleteSignedPreKey.run(row.address);
|
||||
this.stmts.deleteOTPKs.run(row.address);
|
||||
}
|
||||
});
|
||||
purgeTx();
|
||||
return staleRows.length;
|
||||
}
|
||||
}
|
||||
|
||||
65
scripts/build-docker.ts
Normal file
65
scripts/build-docker.ts
Normal file
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Build the Shade Prekey Server Docker image locally.
|
||||
*
|
||||
* Usage:
|
||||
* bun run build:docker # build as shade-prekey:dev
|
||||
* bun run build:docker -- --tag X # custom tag
|
||||
* bun run build:docker -- --push # build + push to Gitea registry
|
||||
*
|
||||
* Env:
|
||||
* GITEA_TOKEN — required for --push
|
||||
* GITEA_USER — default Stian
|
||||
*/
|
||||
import { $ } from 'bun';
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const tagIdx = args.indexOf('--tag');
|
||||
const tag = tagIdx >= 0 ? args[tagIdx + 1] : 'dev';
|
||||
const shouldPush = args.includes('--push');
|
||||
const user = process.env.GITEA_USER ?? 'Stian';
|
||||
const registry = 'gt.zyon.no';
|
||||
const image = `${registry.toLowerCase()}/${user.toLowerCase()}/shade-prekey`;
|
||||
const fullTag = `${image}:${tag}`;
|
||||
|
||||
async function main() {
|
||||
console.log(`Building ${fullTag}…`);
|
||||
await $`docker build -f packages/shade-server/Dockerfile -t ${fullTag} .`;
|
||||
|
||||
console.log(`✓ Built ${fullTag}`);
|
||||
const { stdout: size } = await $`docker images ${fullTag} --format {{.Size}}`.quiet();
|
||||
console.log(` Size: ${size.toString().trim()}`);
|
||||
|
||||
if (shouldPush) {
|
||||
const token = process.env.GITEA_TOKEN;
|
||||
if (!token) {
|
||||
console.error('GITEA_TOKEN env var required for --push');
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(`Pushing ${fullTag}…`);
|
||||
// Login via stdin to avoid leaking token in process table
|
||||
const loginProc = Bun.spawn(['docker', 'login', registry, '-u', user, '--password-stdin'], {
|
||||
stdin: 'pipe',
|
||||
stdout: 'inherit',
|
||||
stderr: 'inherit',
|
||||
});
|
||||
loginProc.stdin.write(token);
|
||||
await loginProc.stdin.end();
|
||||
await loginProc.exited;
|
||||
|
||||
await $`docker push ${fullTag}`;
|
||||
console.log(`✓ Pushed ${fullTag}`);
|
||||
|
||||
if (tag !== 'latest') {
|
||||
const latestTag = `${image}:latest`;
|
||||
await $`docker tag ${fullTag} ${latestTag}`;
|
||||
await $`docker push ${latestTag}`;
|
||||
console.log(`✓ Pushed ${latestTag}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err);
|
||||
process.exit(1);
|
||||
});
|
||||
Reference in New Issue
Block a user