feat(container): M-Box 1-8 — stack-agnostic standalone Docker container
Some checks failed
Test / test (push) Has been cancelled
Some checks failed
Test / test (push) Has been cancelled
Shade now ships as a self-contained Docker image. Deploy one container per project, any stack (Bun, Python, Go, Rust, Kotlin) can talk to it via plain HTTP. Zero coupling to consumer codebases. M-Box 1: Stale identity cleanup API - touchIdentity + purgeStaleIdentities on PrekeyStore interface - Implemented for Memory, SQLite, and Postgres backends - SQLite adds last_activity_at column with migration ALTER for existing DBs - Postgres adds the same via raw SQL with IF NOT EXISTS guards - Routes call touchIdentity on register, bundle fetch, replenish - 4 new tests for the cleanup API M-Box 2: Stale cleanup background task - StaleCleanupTask runs purge on startup + every 24h (configurable) - Reads SHADE_STALE_DAYS (default 30) and SHADE_CLEANUP_INTERVAL_HOURS - Wired into standalone.ts, stopped on graceful shutdown - 5 new tests for the task M-Box 3: Observer baked into the container - standalone.ts conditionally mounts @shade/observer at /shade-observer when SHADE_OBSERVER_TOKEN is set (and >= 16 chars) - Shared PrekeyServerEvents emitter feeds both routes and observer - @shade/observer added as optional dependency of @shade/server M-Box 4: Dockerfile with dashboard build - Multi-stage build: oven/bun:1 builder → oven/bun:1-alpine runtime - COPY packages/ wholesale so workspace lockfile resolves cleanly - RUN bun run build inside shade-dashboard → dist/ → observer/dist/ - Non-root shade user, /data volume, healthcheck, env defaults - Final image: 260 MB M-Box 5: OpenAPI spec for stack-agnostic clients - packages/shade-server/openapi.yaml documents all 9 endpoints with request/response schemas, security (Ed25519 signatures + bearer token) - createOpenApiRoutes serves /openapi.yaml and /docs (Redoc viewer) - Any language can generate a client with openapi-generator M-Box 6: Docker CI pipeline - .gitea/workflows/docker.yml builds + pushes on git tag v* - scripts/build-docker.ts for local builds, supports --push with GITEA_TOKEN - Root package.json: build:docker, publish:docker scripts M-Box 7: Deployment documentation - packages/shade-server/README rewritten: 5-line quickstart with the image - docs/DEPLOYMENT.md: full reference, env vars, backup, Dokploy, PG setup - examples/05-dokploy-deployment/docker-compose.yml updated to pull published image (gt.zyon.no/stian/shade-prekey:latest) - Root README deployment section rewritten M-Box 8: End-to-end verification - Image builds locally (bun run build:docker) - /health, /openapi.yaml, /docs, /metrics, /shade-observer all respond - 401 without observer token, 200 with - Real SDK client round-trip: Alice → container → Bob → reply → Alice - Persistence: identity + prekeys survive container restart (count 20→18 as expected from two bundle fetches) 285 tests passing, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
52
.gitea/workflows/docker.yml
Normal file
52
.gitea/workflows/docker.yml
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
name: Docker build and publish
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
tags:
|
||||||
|
- 'v*'
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
docker:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Install Bun
|
||||||
|
run: curl -fsSL https://bun.sh/install | bash
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: ~/.bun/bin/bun install --frozen-lockfile
|
||||||
|
|
||||||
|
- name: Run tests (gate)
|
||||||
|
run: ~/.bun/bin/bun test --recursive
|
||||||
|
|
||||||
|
- name: Extract version from tag
|
||||||
|
id: version
|
||||||
|
run: |
|
||||||
|
VERSION="${GITHUB_REF_NAME#v}"
|
||||||
|
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: Set up Docker Buildx
|
||||||
|
uses: docker/setup-buildx-action@v3
|
||||||
|
|
||||||
|
- name: Log in to Gitea container registry
|
||||||
|
uses: docker/login-action@v3
|
||||||
|
with:
|
||||||
|
registry: gt.zyon.no
|
||||||
|
username: Stian
|
||||||
|
password: ${{ secrets.GITEA_PUBLISH_TOKEN }}
|
||||||
|
|
||||||
|
- name: Build and push
|
||||||
|
uses: docker/build-push-action@v5
|
||||||
|
with:
|
||||||
|
context: .
|
||||||
|
file: packages/shade-server/Dockerfile
|
||||||
|
push: true
|
||||||
|
tags: |
|
||||||
|
gt.zyon.no/stian/shade-prekey:${{ steps.version.outputs.version }}
|
||||||
|
gt.zyon.no/stian/shade-prekey:latest
|
||||||
|
labels: |
|
||||||
|
org.opencontainers.image.version=${{ steps.version.outputs.version }}
|
||||||
|
org.opencontainers.image.source=https://gt.zyon.no/Stian/Shade
|
||||||
|
org.opencontainers.image.revision=${{ github.sha }}
|
||||||
33
README.md
33
README.md
@@ -154,25 +154,28 @@ bun run publish:all
|
|||||||
- [examples/](./examples/) — Runnable example applications
|
- [examples/](./examples/) — Runnable example applications
|
||||||
- [MIGRATION.md](./MIGRATION.md) — How to replace existing crypto with Shade
|
- [MIGRATION.md](./MIGRATION.md) — How to replace existing crypto with Shade
|
||||||
|
|
||||||
## Deployment
|
## Deployment — one container per project
|
||||||
|
|
||||||
For containerized deployment (Docker/Dokploy):
|
Shade ships as a self-contained Docker image. Deploy one container per project, point your app at it, done. Any stack (Bun, Python, Go, Rust, Kotlin) can use it — the container exposes a plain HTTP API documented in OpenAPI.
|
||||||
|
|
||||||
```yaml
|
```bash
|
||||||
services:
|
docker run -d \
|
||||||
shade-prekey:
|
--name my-project-shade \
|
||||||
image: shade-prekey-server:latest
|
-v my-project-shade:/data \
|
||||||
ports:
|
-p 3900:3900 \
|
||||||
- "3900:3900"
|
-e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
|
||||||
volumes:
|
gt.zyon.no/stian/shade-prekey:latest
|
||||||
- shade-data:/data
|
|
||||||
environment:
|
|
||||||
- SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
|
||||||
volumes:
|
|
||||||
shade-data:
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The SQLite database persists to a Docker volume so all keys and prekey bundles survive restarts.
|
The container includes:
|
||||||
|
- **Prekey server** — `/v1/keys/*` REST API
|
||||||
|
- **Observer dashboard** — `/shade-observer/dashboard/` (off unless token is set)
|
||||||
|
- **OpenAPI spec** — `/openapi.yaml` and interactive `/docs` viewer
|
||||||
|
- **Prometheus metrics** — `/metrics`
|
||||||
|
- **Health check** — `/health`
|
||||||
|
- **Stale cleanup** — purges inactive identities automatically
|
||||||
|
|
||||||
|
See [docs/DEPLOYMENT.md](./docs/DEPLOYMENT.md) for the full deployment guide, environment variables, PostgreSQL config, backup strategy, and Dokploy instructions.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
6
bun.lock
6
bun.lock
@@ -106,6 +106,12 @@
|
|||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
},
|
},
|
||||||
|
"optionalDependencies": {
|
||||||
|
"@shade/crypto-web": "workspace:*",
|
||||||
|
"@shade/observer": "workspace:*",
|
||||||
|
"@shade/storage-postgres": "workspace:*",
|
||||||
|
"@shade/storage-sqlite": "workspace:*",
|
||||||
|
},
|
||||||
},
|
},
|
||||||
"packages/shade-storage-postgres": {
|
"packages/shade-storage-postgres": {
|
||||||
"name": "@shade/storage-postgres",
|
"name": "@shade/storage-postgres",
|
||||||
|
|||||||
136
docs/DEPLOYMENT.md
Normal file
136
docs/DEPLOYMENT.md
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
# Deploying Shade
|
||||||
|
|
||||||
|
Shade ships as a single Docker image that contains the prekey server, observer dashboard, OpenAPI contract, and stale cleanup. You deploy one container per project.
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--name my-project-shade \
|
||||||
|
-v my-project-shade:/data \
|
||||||
|
-p 3900:3900 \
|
||||||
|
-e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
|
||||||
|
gt.zyon.no/stian/shade-prekey:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it. Your projects can now register identities and exchange prekey bundles via `http://localhost:3900`.
|
||||||
|
|
||||||
|
## Why one container per project
|
||||||
|
|
||||||
|
Each project is self-contained. Nova doesn't depend on Orchestrator being up. Future projects can be added without touching existing ones. The container is tiny (~260 MB), idle resource usage is near zero, and each container owns its own SQLite volume.
|
||||||
|
|
||||||
|
```
|
||||||
|
Project A Project B Future projects
|
||||||
|
───────── ───────── ────────────────
|
||||||
|
app + frontend app + frontend app + frontend
|
||||||
|
│ │ │
|
||||||
|
↓ ↓ ↓
|
||||||
|
shade-a container shade-b container shade-n container
|
||||||
|
(port 3900) (port 3901) (port 390n)
|
||||||
|
sqlite volume sqlite volume sqlite volume
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dokploy deployment
|
||||||
|
|
||||||
|
1. Go to Dokploy → Projects → New Project → Docker Compose
|
||||||
|
2. Paste the `docker-compose.yml` from [`examples/05-dokploy-deployment`](../examples/05-dokploy-deployment/docker-compose.yml)
|
||||||
|
3. Set env vars in the Dokploy UI:
|
||||||
|
- `SHADE_OBSERVER_TOKEN` (generate a random 32+ char string)
|
||||||
|
4. Set the container name unique per project (e.g., `nova-shade`, `orchestrator-shade`)
|
||||||
|
5. Deploy
|
||||||
|
|
||||||
|
Dokploy will pull the image, create the volume, and health check the container automatically.
|
||||||
|
|
||||||
|
## Volumes and backup
|
||||||
|
|
||||||
|
The `/data` volume holds:
|
||||||
|
- `shade-prekeys.db` — the SQLite database with all identities, prekeys, and activity timestamps
|
||||||
|
- WAL journal files for crash safety
|
||||||
|
|
||||||
|
**Backup:** Copy the `.db` file while the container is stopped, or use SQLite's online backup API:
|
||||||
|
```bash
|
||||||
|
docker exec my-project-shade sqlite3 /data/shade-prekeys.db ".backup /data/backup.db"
|
||||||
|
docker cp my-project-shade:/data/backup.db ./local-backup.db
|
||||||
|
```
|
||||||
|
|
||||||
|
**Restore:** Stop the container, copy the `.db` file into the volume, restart.
|
||||||
|
|
||||||
|
## PostgreSQL instead of SQLite
|
||||||
|
|
||||||
|
If you want to share a Postgres instance (or need HA), set `SHADE_PREKEY_PG_URL`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
environment:
|
||||||
|
- SHADE_PREKEY_PG_URL=postgres://shade:shade@postgres:5432/shade
|
||||||
|
```
|
||||||
|
|
||||||
|
Tables will be created automatically with the `shade_server_*` prefix, so they coexist cleanly with any other tables in the same database.
|
||||||
|
|
||||||
|
## Environment variable reference
|
||||||
|
|
||||||
|
| Var | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `PORT` | `3900` | HTTP port |
|
||||||
|
| `SHADE_PREKEY_DB_PATH` | `/data/shade-prekeys.db` | SQLite file location |
|
||||||
|
| `SHADE_PREKEY_PG_URL` | unset | Postgres URL (overrides SQLite) |
|
||||||
|
| `SHADE_OBSERVER_TOKEN` | unset | Enables dashboard at `/shade-observer/dashboard/`. Min 16 chars. |
|
||||||
|
| `SHADE_STALE_DAYS` | `30` | Purge identities with no activity in N days |
|
||||||
|
| `SHADE_CLEANUP_INTERVAL_HOURS` | `24` | Cleanup cycle interval |
|
||||||
|
| `SHADE_LOG_LEVEL` | `info` | `debug` / `info` / `warn` / `error` |
|
||||||
|
|
||||||
|
## Health and observability
|
||||||
|
|
||||||
|
- **Health:** `GET /health` — returns `{"status":"ok"}` when the storage backend is reachable. Docker's HEALTHCHECK uses this.
|
||||||
|
- **Metrics:** `GET /metrics` — Prometheus format with counters, histograms, and gauges for all routes.
|
||||||
|
- **OpenAPI:** `GET /openapi.yaml` — machine-readable API contract for any language.
|
||||||
|
- **Redoc viewer:** `GET /docs` — human-readable API reference.
|
||||||
|
- **Dashboard:** `GET /shade-observer/dashboard/` — live activity viewer (requires token).
|
||||||
|
|
||||||
|
## Stale cleanup
|
||||||
|
|
||||||
|
Identities with no activity (no bundle fetches, no replenishments, no registration refreshes) for more than `SHADE_STALE_DAYS` days are automatically purged from the database. This keeps the database bounded without manual housekeeping.
|
||||||
|
|
||||||
|
The cleanup task runs once at startup and then every `SHADE_CLEANUP_INTERVAL_HOURS` hours. Each cycle logs the number of purged identities.
|
||||||
|
|
||||||
|
## Multiple Shade instances on the same host
|
||||||
|
|
||||||
|
Run multiple projects side-by-side with different container names and ports:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
nova-shade:
|
||||||
|
container_name: nova-shade
|
||||||
|
image: gt.zyon.no/stian/shade-prekey:latest
|
||||||
|
ports: ["3900:3900"]
|
||||||
|
volumes: [nova-shade-data:/data]
|
||||||
|
environment:
|
||||||
|
- SHADE_OBSERVER_TOKEN=nova-token-32-chars-minimum-xxx
|
||||||
|
|
||||||
|
orch-shade:
|
||||||
|
container_name: orch-shade
|
||||||
|
image: gt.zyon.no/stian/shade-prekey:latest
|
||||||
|
ports: ["3901:3900"]
|
||||||
|
volumes: [orch-shade-data:/data]
|
||||||
|
environment:
|
||||||
|
- SHADE_OBSERVER_TOKEN=orch-token-32-chars-minimum-xxx
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
nova-shade-data:
|
||||||
|
orch-shade-data:
|
||||||
|
```
|
||||||
|
|
||||||
|
## CI publishing
|
||||||
|
|
||||||
|
Tagged releases auto-publish to the Gitea container registry via `.gitea/workflows/docker.yml`. To cut a release:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bun run version 1.0.1
|
||||||
|
git push --tags
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security notes
|
||||||
|
|
||||||
|
- **Never commit `SHADE_OBSERVER_TOKEN`.** Use Dokploy secrets or environment-specific `.env` files.
|
||||||
|
- The prekey server stores **public keys only**. No private keys ever touch it.
|
||||||
|
- Rate limiting is on by default (5 registrations per hour per IP, etc.). Tune via `createPrekeyRoutes` options if embedding, or configure at reverse-proxy level for the container.
|
||||||
|
- Put the container behind a reverse proxy (Traefik, Caddy) for TLS termination.
|
||||||
@@ -1,9 +1,22 @@
|
|||||||
|
# Shade Prekey Server — Dokploy / Docker Compose deployment
|
||||||
|
#
|
||||||
|
# Pulls the published image from Gitea's container registry. Change
|
||||||
|
# `my-project-shade` to something project-specific so you can run multiple
|
||||||
|
# Shade instances side-by-side (one per project).
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# docker compose up -d
|
||||||
|
#
|
||||||
|
# To build locally from source instead of pulling, uncomment the `build:`
|
||||||
|
# section and comment out `image:`.
|
||||||
|
|
||||||
services:
|
services:
|
||||||
shade-prekey:
|
shade-prekey:
|
||||||
image: shade-prekey-server:latest
|
container_name: my-project-shade
|
||||||
build:
|
image: gt.zyon.no/stian/shade-prekey:latest
|
||||||
context: ../..
|
# build:
|
||||||
dockerfile: packages/shade-server/Dockerfile
|
# context: ../..
|
||||||
|
# dockerfile: packages/shade-server/Dockerfile
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
ports:
|
ports:
|
||||||
- "3900:3900"
|
- "3900:3900"
|
||||||
@@ -13,6 +26,8 @@ services:
|
|||||||
- PORT=3900
|
- PORT=3900
|
||||||
- SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
- SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
||||||
- SHADE_LOG_LEVEL=info
|
- SHADE_LOG_LEVEL=info
|
||||||
|
- SHADE_STALE_DAYS=30
|
||||||
|
- SHADE_CLEANUP_INTERVAL_HOURS=24
|
||||||
# Optional: enable the live observer dashboard at /shade-observer/dashboard/
|
# Optional: enable the live observer dashboard at /shade-observer/dashboard/
|
||||||
# Token must be at least 16 characters. Use a real secret in production.
|
# Token must be at least 16 characters. Use a real secret in production.
|
||||||
# - SHADE_OBSERVER_TOKEN=change-me-must-be-at-least-16-chars
|
# - SHADE_OBSERVER_TOKEN=change-me-must-be-at-least-16-chars
|
||||||
|
|||||||
@@ -13,7 +13,9 @@
|
|||||||
"test:cli": "cd packages/shade-cli && bun test",
|
"test:cli": "cd packages/shade-cli && bun test",
|
||||||
"version": "bun run scripts/bump-version.ts",
|
"version": "bun run scripts/bump-version.ts",
|
||||||
"publish:dry": "DRY_RUN=1 bun run scripts/publish-all.ts",
|
"publish:dry": "DRY_RUN=1 bun run scripts/publish-all.ts",
|
||||||
"publish:all": "bun run scripts/publish-all.ts"
|
"publish:all": "bun run scripts/publish-all.ts",
|
||||||
|
"build:docker": "bun run scripts/build-docker.ts",
|
||||||
|
"publish:docker": "bun run scripts/build-docker.ts -- --push"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"bun-types": "^1.3.11"
|
"bun-types": "^1.3.11"
|
||||||
|
|||||||
@@ -1,7 +1,17 @@
|
|||||||
node_modules
|
**/node_modules
|
||||||
dist
|
**/dist
|
||||||
|
**/dist-build
|
||||||
*.tsbuildinfo
|
*.tsbuildinfo
|
||||||
.git
|
.git
|
||||||
|
.gitea
|
||||||
|
.github
|
||||||
.DS_Store
|
.DS_Store
|
||||||
**/tests
|
**/tests
|
||||||
**/*.test.ts
|
**/*.test.ts
|
||||||
|
**/tmp
|
||||||
|
**/tmp-data
|
||||||
|
examples
|
||||||
|
bench
|
||||||
|
android
|
||||||
|
docs
|
||||||
|
.env*
|
||||||
|
|||||||
@@ -3,28 +3,27 @@ FROM oven/bun:1 AS builder
|
|||||||
|
|
||||||
WORKDIR /build
|
WORKDIR /build
|
||||||
|
|
||||||
# Copy workspace root
|
# Copy workspace root config
|
||||||
COPY package.json bun.lock ./
|
COPY package.json bun.lock tsconfig.json ./
|
||||||
COPY tsconfig.json ./
|
|
||||||
|
|
||||||
# Copy all packages we depend on
|
# Copy all packages the server + observer + dashboard need
|
||||||
COPY packages/shade-core ./packages/shade-core
|
COPY packages ./packages
|
||||||
COPY packages/shade-crypto-web ./packages/shade-crypto-web
|
|
||||||
COPY packages/shade-server ./packages/shade-server
|
|
||||||
COPY packages/shade-storage-sqlite ./packages/shade-storage-sqlite
|
|
||||||
COPY packages/shade-storage-postgres ./packages/shade-storage-postgres
|
|
||||||
|
|
||||||
RUN bun install --frozen-lockfile
|
RUN bun install --frozen-lockfile
|
||||||
|
|
||||||
|
# Build the dashboard SPA → dist/, then copy to observer's dist/
|
||||||
|
RUN cd packages/shade-dashboard && bun run build
|
||||||
|
|
||||||
# ─── Production stage ───────────────────────────────────────
|
# ─── Production stage ───────────────────────────────────────
|
||||||
FROM oven/bun:1-alpine
|
FROM oven/bun:1-alpine
|
||||||
|
|
||||||
LABEL org.opencontainers.image.title="Shade Prekey Server"
|
LABEL org.opencontainers.image.title="Shade Prekey Server"
|
||||||
LABEL org.opencontainers.image.description="E2EE prekey distribution server (Signal Protocol)"
|
LABEL org.opencontainers.image.description="E2EE prekey distribution server with live observer (Signal Protocol)"
|
||||||
LABEL org.opencontainers.image.source="https://github.com/Sterister/Shade"
|
LABEL org.opencontainers.image.source="https://gt.zyon.no/Stian/Shade"
|
||||||
LABEL org.opencontainers.image.licenses="MIT"
|
LABEL org.opencontainers.image.licenses="MIT"
|
||||||
|
LABEL org.opencontainers.image.vendor="Stian"
|
||||||
|
|
||||||
# Install curl for healthcheck
|
# curl for healthcheck
|
||||||
RUN apk add --no-cache curl
|
RUN apk add --no-cache curl
|
||||||
|
|
||||||
# Non-root user
|
# Non-root user
|
||||||
@@ -33,7 +32,7 @@ RUN addgroup -S shade && adduser -S shade -G shade
|
|||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY --from=builder --chown=shade:shade /build /app
|
COPY --from=builder --chown=shade:shade /build /app
|
||||||
|
|
||||||
# Persistent data directory
|
# Persistent data directory (SQLite file or any sidecar state)
|
||||||
RUN mkdir -p /data && chown shade:shade /data
|
RUN mkdir -p /data && chown shade:shade /data
|
||||||
VOLUME ["/data"]
|
VOLUME ["/data"]
|
||||||
|
|
||||||
@@ -41,9 +40,12 @@ USER shade
|
|||||||
|
|
||||||
EXPOSE 3900
|
EXPOSE 3900
|
||||||
|
|
||||||
# Default to SQLite on the persistent volume
|
# Defaults — override via `docker run -e`
|
||||||
ENV SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
ENV SHADE_PREKEY_DB_PATH=/data/shade-prekeys.db
|
||||||
ENV PORT=3900
|
ENV PORT=3900
|
||||||
|
ENV SHADE_LOG_LEVEL=info
|
||||||
|
ENV SHADE_STALE_DAYS=30
|
||||||
|
ENV SHADE_CLEANUP_INTERVAL_HOURS=24
|
||||||
|
|
||||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||||
CMD curl -fsS http://localhost:${PORT}/health || exit 1
|
CMD curl -fsS http://localhost:${PORT}/health || exit 1
|
||||||
|
|||||||
90
packages/shade-server/README.md
Normal file
90
packages/shade-server/README.md
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
# @shade/server — Shade Prekey Server (standalone container)
|
||||||
|
|
||||||
|
A self-contained Docker image that provides the prekey server, OpenAPI contract, observer dashboard, and stale cleanup — **everything a project needs to adopt Shade**, with zero coupling to the consumer's stack.
|
||||||
|
|
||||||
|
## Deploy in 2 minutes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--name my-project-shade \
|
||||||
|
-v my-project-shade:/data \
|
||||||
|
-p 3900:3900 \
|
||||||
|
-e SHADE_OBSERVER_TOKEN=change-me-to-at-least-16-chars \
|
||||||
|
gt.zyon.no/stian/shade-prekey:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
Done. Your prekey server is live:
|
||||||
|
- `http://localhost:3900/health` — health check
|
||||||
|
- `http://localhost:3900/openapi.yaml` — API contract for any language
|
||||||
|
- `http://localhost:3900/docs` — interactive API reference (Redoc)
|
||||||
|
- `http://localhost:3900/shade-observer/dashboard/` — live debugger (token required)
|
||||||
|
- `http://localhost:3900/v1/keys/*` — prekey REST API
|
||||||
|
|
||||||
|
Your consumer projects (Nova, Orchestrator, Python apps, anything) then point at `http://localhost:3900` as their `prekeyServer` URL.
|
||||||
|
|
||||||
|
## One container per project
|
||||||
|
|
||||||
|
The recommended architecture is **one Shade container per project**:
|
||||||
|
|
||||||
|
```
|
||||||
|
nova-shade (Docker container, SQLite volume) ← Nova backend + Android app
|
||||||
|
orchestrator-shade (Docker container, SQLite volume) ← Orchestrator hub + workstations
|
||||||
|
future-project (Docker container, SQLite volume) ← Any future app
|
||||||
|
```
|
||||||
|
|
||||||
|
Each project owns its own container, its own volume, its own observer token. Zero cross-project coupling. If one project's Shade is down, the others keep running.
|
||||||
|
|
||||||
|
## Environment variables
|
||||||
|
|
||||||
|
| Var | Default | Description |
|
||||||
|
|-----|---------|-------------|
|
||||||
|
| `PORT` | `3900` | HTTP port |
|
||||||
|
| `SHADE_PREKEY_DB_PATH` | `/data/shade-prekeys.db` | SQLite file path |
|
||||||
|
| `SHADE_PREKEY_PG_URL` | unset | Postgres connection string. If set, overrides SQLite. |
|
||||||
|
| `SHADE_OBSERVER_TOKEN` | unset | Bearer token for the dashboard. Min 16 chars. Unset = observer disabled. |
|
||||||
|
| `SHADE_STALE_DAYS` | `30` | Purge identities with no activity in N days |
|
||||||
|
| `SHADE_CLEANUP_INTERVAL_HOURS` | `24` | How often the cleanup task runs |
|
||||||
|
| `SHADE_LOG_LEVEL` | `info` | `debug` / `info` / `warn` / `error` |
|
||||||
|
|
||||||
|
## Persistence
|
||||||
|
|
||||||
|
The `/data` volume holds the SQLite database. Back it up by copying the `.db` file (use SQLite's online backup API or just stop the container briefly).
|
||||||
|
|
||||||
|
To switch to Postgres, set `SHADE_PREKEY_PG_URL=postgres://user:pass@host/db`. Tables will be created automatically with the `shade_server_*` prefix.
|
||||||
|
|
||||||
|
## Stale cleanup
|
||||||
|
|
||||||
|
Identities that have no activity (no bundle fetches, no replenishments, no registration updates) for more than `SHADE_STALE_DAYS` days are automatically purged. This keeps the database bounded even if users never unregister cleanly.
|
||||||
|
|
||||||
|
## Using from your project
|
||||||
|
|
||||||
|
Any language can speak to a Shade container — it's just HTTP. See [openapi.yaml](./openapi.yaml) for the full contract.
|
||||||
|
|
||||||
|
**TypeScript / Bun:**
|
||||||
|
```ts
|
||||||
|
import { createShade } from '@shade/sdk';
|
||||||
|
const shade = await createShade({ prekeyServer: 'http://my-project-shade:3900' });
|
||||||
|
```
|
||||||
|
|
||||||
|
**Python / Go / Rust:** generate a client from the OpenAPI spec with `openapi-generator`, or implement the wire protocol directly (8 endpoints, Ed25519 signatures documented in the spec).
|
||||||
|
|
||||||
|
**Android:** use the `shade-android` Kotlin module. Same wire protocol, verified by cross-platform test vectors.
|
||||||
|
|
||||||
|
## Building locally
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bun run build:docker # build shade-prekey:dev
|
||||||
|
bun run build:docker -- --tag v1.0.0 # custom tag
|
||||||
|
GITEA_TOKEN=... bun run publish:docker # build + push to registry
|
||||||
|
```
|
||||||
|
|
||||||
|
## CI publishing
|
||||||
|
|
||||||
|
Tag a release and CI publishes automatically:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git tag v1.0.0
|
||||||
|
git push --tags
|
||||||
|
```
|
||||||
|
|
||||||
|
`.gitea/workflows/docker.yml` runs tests, builds the image, and pushes both `v1.0.0` and `latest` tags to `gt.zyon.no/stian/shade-prekey`.
|
||||||
377
packages/shade-server/openapi.yaml
Normal file
377
packages/shade-server/openapi.yaml
Normal file
@@ -0,0 +1,377 @@
|
|||||||
|
openapi: 3.1.0
|
||||||
|
info:
|
||||||
|
title: Shade Prekey Server
|
||||||
|
description: |
|
||||||
|
Signal Protocol prekey distribution server. Stores identity public keys
|
||||||
|
and prekey bundles. Any language can implement a client — this spec
|
||||||
|
documents the wire contract.
|
||||||
|
|
||||||
|
**Security model:** Write operations (register, replenish, delete) are
|
||||||
|
authenticated by Ed25519 signatures over the request body. Bundle fetches
|
||||||
|
are anonymous. See the `SignedPayload` schema for the signing format.
|
||||||
|
version: "1.0.0"
|
||||||
|
license:
|
||||||
|
name: MIT
|
||||||
|
url: https://gt.zyon.no/Stian/Shade/raw/branch/main/LICENSE
|
||||||
|
|
||||||
|
servers:
|
||||||
|
- url: https://shade.example.com
|
||||||
|
description: Replace with your project's prekey server URL
|
||||||
|
|
||||||
|
paths:
|
||||||
|
/health:
|
||||||
|
get:
|
||||||
|
summary: Health check
|
||||||
|
description: Returns 200 if the server and its storage backend are reachable.
|
||||||
|
tags: [Infrastructure]
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Healthy
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/HealthResponse'
|
||||||
|
'503':
|
||||||
|
description: Unhealthy
|
||||||
|
|
||||||
|
/metrics:
|
||||||
|
get:
|
||||||
|
summary: Prometheus metrics
|
||||||
|
description: Counter, histogram, and gauge metrics in Prometheus text format.
|
||||||
|
tags: [Infrastructure]
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Metrics
|
||||||
|
content:
|
||||||
|
text/plain:
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
example: |
|
||||||
|
# HELP shade_requests_total Total HTTP requests
|
||||||
|
# TYPE shade_requests_total counter
|
||||||
|
shade_requests_total{route="/v1/keys/register",status="200"} 42
|
||||||
|
|
||||||
|
/openapi.yaml:
|
||||||
|
get:
|
||||||
|
summary: This OpenAPI spec
|
||||||
|
description: Serves the YAML spec itself, so clients can auto-download it.
|
||||||
|
tags: [Infrastructure]
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: OpenAPI spec
|
||||||
|
content:
|
||||||
|
application/yaml:
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
|
||||||
|
/v1/keys/register:
|
||||||
|
post:
|
||||||
|
summary: Register identity and upload prekey bundle
|
||||||
|
description: |
|
||||||
|
Register a new identity with the prekey server, or update an existing
|
||||||
|
one. The request body must be signed with the identity's own Ed25519
|
||||||
|
signing key (TOFU — first signature establishes the identity).
|
||||||
|
tags: [Keys]
|
||||||
|
security:
|
||||||
|
- Ed25519Signature: []
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/components/schemas/SignedPayload'
|
||||||
|
- type: object
|
||||||
|
required: [address, identitySigningKey, identityDHKey, signedPreKey]
|
||||||
|
properties:
|
||||||
|
address:
|
||||||
|
$ref: '#/components/schemas/Address'
|
||||||
|
identitySigningKey:
|
||||||
|
type: string
|
||||||
|
description: Ed25519 public key, base64
|
||||||
|
identityDHKey:
|
||||||
|
type: string
|
||||||
|
description: X25519 public key, base64
|
||||||
|
signedPreKey:
|
||||||
|
$ref: '#/components/schemas/SignedPreKeyEntry'
|
||||||
|
oneTimePreKeys:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: '#/components/schemas/OneTimePreKeyEntry'
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Registered
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
ok: { type: boolean }
|
||||||
|
'400':
|
||||||
|
$ref: '#/components/responses/ValidationError'
|
||||||
|
'401':
|
||||||
|
$ref: '#/components/responses/Unauthorized'
|
||||||
|
'429':
|
||||||
|
$ref: '#/components/responses/RateLimited'
|
||||||
|
|
||||||
|
/v1/keys/bundle/{address}:
|
||||||
|
get:
|
||||||
|
summary: Fetch a prekey bundle (anonymous)
|
||||||
|
description: |
|
||||||
|
Fetch an identity's prekey bundle so Alice can start an X3DH session
|
||||||
|
with Bob. Each call consumes one one-time prekey (FIFO) if any are
|
||||||
|
available — if not, returns a bundle without the one-time prekey.
|
||||||
|
tags: [Keys]
|
||||||
|
parameters:
|
||||||
|
- name: address
|
||||||
|
in: path
|
||||||
|
required: true
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/Address'
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Prekey bundle
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/PreKeyBundle'
|
||||||
|
'404':
|
||||||
|
$ref: '#/components/responses/NotFound'
|
||||||
|
'429':
|
||||||
|
$ref: '#/components/responses/RateLimited'
|
||||||
|
|
||||||
|
/v1/keys/count/{address}:
|
||||||
|
get:
|
||||||
|
summary: Get remaining one-time prekey count (anonymous)
|
||||||
|
tags: [Keys]
|
||||||
|
parameters:
|
||||||
|
- name: address
|
||||||
|
in: path
|
||||||
|
required: true
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/Address'
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Count
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
count: { type: integer, minimum: 0 }
|
||||||
|
|
||||||
|
/v1/keys/replenish:
|
||||||
|
post:
|
||||||
|
summary: Upload more one-time prekeys (signed)
|
||||||
|
tags: [Keys]
|
||||||
|
security:
|
||||||
|
- Ed25519Signature: []
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/components/schemas/SignedPayload'
|
||||||
|
- type: object
|
||||||
|
required: [address, oneTimePreKeys]
|
||||||
|
properties:
|
||||||
|
address:
|
||||||
|
$ref: '#/components/schemas/Address'
|
||||||
|
oneTimePreKeys:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: '#/components/schemas/OneTimePreKeyEntry'
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Replenished
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
ok: { type: boolean }
|
||||||
|
remaining: { type: integer, minimum: 0 }
|
||||||
|
'401':
|
||||||
|
$ref: '#/components/responses/Unauthorized'
|
||||||
|
'404':
|
||||||
|
$ref: '#/components/responses/NotFound'
|
||||||
|
'429':
|
||||||
|
$ref: '#/components/responses/RateLimited'
|
||||||
|
|
||||||
|
/v1/keys/{address}:
|
||||||
|
delete:
|
||||||
|
summary: Unregister an identity (signed)
|
||||||
|
tags: [Keys]
|
||||||
|
security:
|
||||||
|
- Ed25519Signature: []
|
||||||
|
parameters:
|
||||||
|
- name: address
|
||||||
|
in: path
|
||||||
|
required: true
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/Address'
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/SignedPayload'
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Deleted
|
||||||
|
'401':
|
||||||
|
$ref: '#/components/responses/Unauthorized'
|
||||||
|
|
||||||
|
/shade-observer/api/state:
|
||||||
|
get:
|
||||||
|
summary: Current observer snapshot (optional)
|
||||||
|
description: |
|
||||||
|
Returns the aggregated state visible to the dashboard. Only available
|
||||||
|
if the server was started with `SHADE_OBSERVER_TOKEN` set.
|
||||||
|
tags: [Observer]
|
||||||
|
security:
|
||||||
|
- ObserverBearerToken: []
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: Observer state snapshot
|
||||||
|
'401':
|
||||||
|
description: Invalid or missing token
|
||||||
|
|
||||||
|
components:
|
||||||
|
schemas:
|
||||||
|
Address:
|
||||||
|
type: string
|
||||||
|
description: |
|
||||||
|
An address on the prekey server. Alphanumeric plus `:_-.` characters,
|
||||||
|
max 256 chars. NFKC-normalized. Format is typically
|
||||||
|
`user@domain:deviceId` or `device:uuid`.
|
||||||
|
pattern: '^[a-zA-Z0-9][a-zA-Z0-9:_\-.]{0,255}$'
|
||||||
|
example: "alice@example.com:phone"
|
||||||
|
|
||||||
|
SignedPayload:
|
||||||
|
type: object
|
||||||
|
description: |
|
||||||
|
Base type for all signed request bodies. The `signature` field is
|
||||||
|
an Ed25519 signature over the canonical JSON of the payload
|
||||||
|
(all fields sorted, `signature` omitted) using the identity's
|
||||||
|
signing private key. `signedAt` must be within ±5 minutes of
|
||||||
|
server time.
|
||||||
|
required: [signedAt, signature]
|
||||||
|
properties:
|
||||||
|
signedAt:
|
||||||
|
type: integer
|
||||||
|
format: int64
|
||||||
|
description: Unix epoch milliseconds
|
||||||
|
signature:
|
||||||
|
type: string
|
||||||
|
description: Ed25519 signature, base64
|
||||||
|
|
||||||
|
SignedPreKeyEntry:
|
||||||
|
type: object
|
||||||
|
required: [keyId, publicKey, signature]
|
||||||
|
properties:
|
||||||
|
keyId:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
publicKey:
|
||||||
|
type: string
|
||||||
|
description: X25519 public key, base64
|
||||||
|
signature:
|
||||||
|
type: string
|
||||||
|
description: Ed25519 signature over `publicKey`, base64
|
||||||
|
|
||||||
|
OneTimePreKeyEntry:
|
||||||
|
type: object
|
||||||
|
required: [keyId, publicKey]
|
||||||
|
properties:
|
||||||
|
keyId:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
publicKey:
|
||||||
|
type: string
|
||||||
|
description: X25519 public key, base64
|
||||||
|
|
||||||
|
PreKeyBundle:
|
||||||
|
type: object
|
||||||
|
required: [identitySigningKey, identityDHKey, signedPreKey]
|
||||||
|
properties:
|
||||||
|
identitySigningKey:
|
||||||
|
type: string
|
||||||
|
description: Ed25519 public key, base64
|
||||||
|
identityDHKey:
|
||||||
|
type: string
|
||||||
|
description: X25519 public key, base64
|
||||||
|
signedPreKey:
|
||||||
|
$ref: '#/components/schemas/SignedPreKeyEntry'
|
||||||
|
oneTimePreKey:
|
||||||
|
$ref: '#/components/schemas/OneTimePreKeyEntry'
|
||||||
|
|
||||||
|
HealthResponse:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
status:
|
||||||
|
type: string
|
||||||
|
enum: [ok, error]
|
||||||
|
service:
|
||||||
|
type: string
|
||||||
|
|
||||||
|
ErrorResponse:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
name:
|
||||||
|
type: string
|
||||||
|
code:
|
||||||
|
type: string
|
||||||
|
pattern: '^SHADE_'
|
||||||
|
message:
|
||||||
|
type: string
|
||||||
|
|
||||||
|
responses:
|
||||||
|
ValidationError:
|
||||||
|
description: Invalid request body
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/ErrorResponse'
|
||||||
|
Unauthorized:
|
||||||
|
description: Signature verification failed or replay window exceeded
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/ErrorResponse'
|
||||||
|
NotFound:
|
||||||
|
description: Address not registered
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/ErrorResponse'
|
||||||
|
RateLimited:
|
||||||
|
description: Rate limit exceeded
|
||||||
|
headers:
|
||||||
|
Retry-After:
|
||||||
|
schema:
|
||||||
|
type: integer
|
||||||
|
description: Seconds until the client can retry
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: '#/components/schemas/ErrorResponse'
|
||||||
|
|
||||||
|
securitySchemes:
|
||||||
|
Ed25519Signature:
|
||||||
|
type: apiKey
|
||||||
|
in: header
|
||||||
|
name: X-Shade-Signature-Info
|
||||||
|
description: |
|
||||||
|
NOT an actual header. Signature is carried inside the JSON request
|
||||||
|
body as the `signature` and `signedAt` fields (see SignedPayload).
|
||||||
|
The canonical form signed is the request body with all keys sorted
|
||||||
|
and the `signature` field omitted. Use your Ed25519 signing key.
|
||||||
|
ObserverBearerToken:
|
||||||
|
type: http
|
||||||
|
scheme: bearer
|
||||||
|
description: |
|
||||||
|
`Authorization: Bearer <SHADE_OBSERVER_TOKEN>`. The observer also
|
||||||
|
accepts the token via `?token=...` query string for SSE endpoints
|
||||||
|
that can't set headers.
|
||||||
@@ -8,6 +8,12 @@
|
|||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"hono": "^4.12.12"
|
"hono": "^4.12.12"
|
||||||
},
|
},
|
||||||
|
"optionalDependencies": {
|
||||||
|
"@shade/observer": "workspace:*",
|
||||||
|
"@shade/storage-sqlite": "workspace:*",
|
||||||
|
"@shade/storage-postgres": "workspace:*",
|
||||||
|
"@shade/crypto-web": "workspace:*"
|
||||||
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@shade/crypto-web": "workspace:*"
|
"@shade/crypto-web": "workspace:*"
|
||||||
}
|
}
|
||||||
|
|||||||
70
packages/shade-server/src/cleanup.ts
Normal file
70
packages/shade-server/src/cleanup.ts
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
import type { PrekeyStore } from './store.js';
|
||||||
|
import { logger } from './logger.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Background task that periodically purges stale identities from the store.
|
||||||
|
*
|
||||||
|
* "Stale" = no activity (register, fetch bundle, replenish, delete) for
|
||||||
|
* more than `staleDays` days. The threshold and interval are configurable
|
||||||
|
* via env vars:
|
||||||
|
* SHADE_STALE_DAYS (default 30)
|
||||||
|
* SHADE_CLEANUP_INTERVAL_HOURS (default 24)
|
||||||
|
*/
|
||||||
|
export class StaleCleanupTask {
|
||||||
|
private timer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
private running = false;
|
||||||
|
private readonly staleMs: number;
|
||||||
|
private readonly intervalMs: number;
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly store: PrekeyStore,
|
||||||
|
options: { staleDays?: number; intervalHours?: number } = {},
|
||||||
|
) {
|
||||||
|
const staleDays = options.staleDays
|
||||||
|
?? Number(process.env.SHADE_STALE_DAYS ?? 30);
|
||||||
|
const intervalHours = options.intervalHours
|
||||||
|
?? Number(process.env.SHADE_CLEANUP_INTERVAL_HOURS ?? 24);
|
||||||
|
this.staleMs = staleDays * 24 * 60 * 60 * 1000;
|
||||||
|
this.intervalMs = intervalHours * 60 * 60 * 1000;
|
||||||
|
}
|
||||||
|
|
||||||
|
start(): void {
|
||||||
|
if (this.running) return;
|
||||||
|
this.running = true;
|
||||||
|
// Run once immediately (so operators see it in the logs at startup)
|
||||||
|
this.runOnce().catch((err) => {
|
||||||
|
logger.error('Initial stale cleanup failed', { error: String(err) });
|
||||||
|
});
|
||||||
|
this.timer = setInterval(() => {
|
||||||
|
this.runOnce().catch((err) => {
|
||||||
|
logger.error('Stale cleanup failed', { error: String(err) });
|
||||||
|
});
|
||||||
|
}, this.intervalMs);
|
||||||
|
}
|
||||||
|
|
||||||
|
stop(): void {
|
||||||
|
this.running = false;
|
||||||
|
if (this.timer) {
|
||||||
|
clearInterval(this.timer);
|
||||||
|
this.timer = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Run a single cleanup cycle. Exposed for tests and manual triggers. */
|
||||||
|
async runOnce(): Promise<number> {
|
||||||
|
const count = await this.store.purgeStaleIdentities(this.staleMs);
|
||||||
|
if (count > 0) {
|
||||||
|
logger.info('Stale cleanup purged identities', {
|
||||||
|
count,
|
||||||
|
staleDays: this.staleMs / (24 * 60 * 60 * 1000),
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
logger.debug('Stale cleanup: nothing to purge');
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
get isRunning(): boolean {
|
||||||
|
return this.running;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -23,6 +23,7 @@ export class MemoryPrekeyStore implements PrekeyStore {
|
|||||||
private identities = new Map<string, IdentityRecord>();
|
private identities = new Map<string, IdentityRecord>();
|
||||||
private signedPreKeys = new Map<string, SignedPreKeyRecord>();
|
private signedPreKeys = new Map<string, SignedPreKeyRecord>();
|
||||||
private oneTimePreKeys = new Map<string, OneTimePreKeyRecord[]>();
|
private oneTimePreKeys = new Map<string, OneTimePreKeyRecord[]>();
|
||||||
|
private lastActivity = new Map<string, number>();
|
||||||
|
|
||||||
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
||||||
this.identities.set(address, { identitySigningKey, identityDHKey });
|
this.identities.set(address, { identitySigningKey, identityDHKey });
|
||||||
@@ -60,5 +61,32 @@ export class MemoryPrekeyStore implements PrekeyStore {
|
|||||||
this.identities.delete(address);
|
this.identities.delete(address);
|
||||||
this.signedPreKeys.delete(address);
|
this.signedPreKeys.delete(address);
|
||||||
this.oneTimePreKeys.delete(address);
|
this.oneTimePreKeys.delete(address);
|
||||||
|
this.lastActivity.delete(address);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Stale cleanup ────────────────────────────────────
|
||||||
|
|
||||||
|
async touchIdentity(address: string): Promise<void> {
|
||||||
|
this.lastActivity.set(address, Date.now());
|
||||||
|
}
|
||||||
|
|
||||||
|
async purgeStaleIdentities(olderThanMs: number): Promise<number> {
|
||||||
|
const cutoff = Date.now() - olderThanMs;
|
||||||
|
const stale: string[] = [];
|
||||||
|
for (const [address, ts] of this.lastActivity) {
|
||||||
|
if (ts < cutoff) stale.push(address);
|
||||||
|
}
|
||||||
|
// Also purge identities that have a row but were never touched
|
||||||
|
// (last_activity = 0/undefined before first touch)
|
||||||
|
for (const address of this.identities.keys()) {
|
||||||
|
if (!this.lastActivity.has(address)) stale.push(address);
|
||||||
|
}
|
||||||
|
for (const address of stale) {
|
||||||
|
this.identities.delete(address);
|
||||||
|
this.signedPreKeys.delete(address);
|
||||||
|
this.oneTimePreKeys.delete(address);
|
||||||
|
this.lastActivity.delete(address);
|
||||||
|
}
|
||||||
|
return stale.length;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
54
packages/shade-server/src/openapi.ts
Normal file
54
packages/shade-server/src/openapi.ts
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
import { Hono } from 'hono';
|
||||||
|
import { existsSync, readFileSync } from 'fs';
|
||||||
|
import { dirname, join } from 'path';
|
||||||
|
import { fileURLToPath } from 'url';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Serves the OpenAPI spec at /openapi.yaml and a Redoc HTML viewer at /docs.
|
||||||
|
*
|
||||||
|
* Any language can fetch /openapi.yaml and generate a client with
|
||||||
|
* openapi-generator. The /docs HTML viewer is a thin Redoc wrapper.
|
||||||
|
*/
|
||||||
|
export function createOpenApiRoutes(specPath?: string): Hono {
|
||||||
|
const app = new Hono();
|
||||||
|
|
||||||
|
const defaultPath = join(
|
||||||
|
dirname(fileURLToPath(import.meta.url)),
|
||||||
|
'..',
|
||||||
|
'openapi.yaml',
|
||||||
|
);
|
||||||
|
const path = specPath ?? defaultPath;
|
||||||
|
|
||||||
|
app.get('/openapi.yaml', (c) => {
|
||||||
|
if (!existsSync(path)) {
|
||||||
|
return c.text('OpenAPI spec not found', 404);
|
||||||
|
}
|
||||||
|
const content = readFileSync(path, 'utf-8');
|
||||||
|
c.header('Content-Type', 'application/yaml; charset=utf-8');
|
||||||
|
return c.body(content);
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get('/docs', (c) => {
|
||||||
|
c.header('Content-Type', 'text/html; charset=utf-8');
|
||||||
|
return c.body(redocHtml());
|
||||||
|
});
|
||||||
|
|
||||||
|
return app;
|
||||||
|
}
|
||||||
|
|
||||||
|
function redocHtml(): string {
|
||||||
|
return `<!DOCTYPE html>
|
||||||
|
<html>
|
||||||
|
<head>
|
||||||
|
<title>Shade API Reference</title>
|
||||||
|
<meta charset="utf-8"/>
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<link href="https://fonts.googleapis.com/css?family=Montserrat:300,400,700|Roboto:300,400,700" rel="stylesheet">
|
||||||
|
<style>body { margin: 0; padding: 0; }</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<redoc spec-url="/openapi.yaml"></redoc>
|
||||||
|
<script src="https://cdn.redoc.ly/redoc/latest/bundles/redoc.standalone.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>`;
|
||||||
|
}
|
||||||
@@ -157,6 +157,9 @@ export function createPrekeyRoutes(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Update activity so stale cleanup doesn't purge active addresses
|
||||||
|
await store.touchIdentity(address);
|
||||||
|
|
||||||
events?.emit('server.bundle_fetched', {
|
events?.emit('server.bundle_fetched', {
|
||||||
address,
|
address,
|
||||||
hadOneTimePreKey: oneTimePreKey != null,
|
hadOneTimePreKey: oneTimePreKey != null,
|
||||||
@@ -192,6 +195,7 @@ export function createPrekeyRoutes(
|
|||||||
publicKey: b64ToBytes(k.publicKey),
|
publicKey: b64ToBytes(k.publicKey),
|
||||||
}));
|
}));
|
||||||
await store.saveOneTimePreKeys(addr, keys);
|
await store.saveOneTimePreKeys(addr, keys);
|
||||||
|
await store.touchIdentity(addr);
|
||||||
|
|
||||||
const count = await store.getOneTimePreKeyCount(addr);
|
const count = await store.getOneTimePreKeyCount(addr);
|
||||||
events?.emit('server.prekeys_replenished', {
|
events?.emit('server.prekeys_replenished', {
|
||||||
|
|||||||
@@ -3,6 +3,9 @@ import { SubtleCryptoProvider } from '@shade/crypto-web';
|
|||||||
import { createPrekeyRoutes } from './routes.js';
|
import { createPrekeyRoutes } from './routes.js';
|
||||||
import { createHealthRoutes } from './health.js';
|
import { createHealthRoutes } from './health.js';
|
||||||
import { createMetricsRoutes, metricsMiddleware } from './metrics.js';
|
import { createMetricsRoutes, metricsMiddleware } from './metrics.js';
|
||||||
|
import { createOpenApiRoutes } from './openapi.js';
|
||||||
|
import { PrekeyServerEvents } from './events.js';
|
||||||
|
import { StaleCleanupTask } from './cleanup.js';
|
||||||
import { logger } from './logger.js';
|
import { logger } from './logger.js';
|
||||||
import type { PrekeyStore } from './store.js';
|
import type { PrekeyStore } from './store.js';
|
||||||
|
|
||||||
@@ -41,13 +44,47 @@ function maskUrl(url: string): string {
|
|||||||
|
|
||||||
const crypto = new SubtleCryptoProvider();
|
const crypto = new SubtleCryptoProvider();
|
||||||
const store = await createStore();
|
const store = await createStore();
|
||||||
|
const events = new PrekeyServerEvents();
|
||||||
|
|
||||||
// Compose the full app: metrics middleware + health + metrics + prekey routes
|
// Compose the full app: metrics middleware + health + metrics + prekey routes
|
||||||
const app = new Hono();
|
const app = new Hono();
|
||||||
app.use('*', metricsMiddleware());
|
app.use('*', metricsMiddleware());
|
||||||
app.route('/', createHealthRoutes(store, VERSION));
|
app.route('/', createHealthRoutes(store, VERSION));
|
||||||
app.route('/', createMetricsRoutes());
|
app.route('/', createMetricsRoutes());
|
||||||
app.route('/', createPrekeyRoutes(store, crypto));
|
app.route('/', createOpenApiRoutes());
|
||||||
|
app.route('/', createPrekeyRoutes(store, crypto, { events }));
|
||||||
|
|
||||||
|
// ─── Optional: Observer + Dashboard ──────────────────────────
|
||||||
|
|
||||||
|
const observerToken = process.env.SHADE_OBSERVER_TOKEN;
|
||||||
|
if (observerToken && observerToken.length >= 16) {
|
||||||
|
try {
|
||||||
|
const { createObserver } = await import('@shade/observer');
|
||||||
|
const observer = createObserver({
|
||||||
|
token: observerToken,
|
||||||
|
serverEvents: events,
|
||||||
|
});
|
||||||
|
app.route('/shade-observer', observer);
|
||||||
|
logger.info('Observer enabled', { path: '/shade-observer/dashboard/' });
|
||||||
|
} catch (err) {
|
||||||
|
logger.warn('Observer module not available, skipping', { error: String(err) });
|
||||||
|
}
|
||||||
|
} else if (observerToken) {
|
||||||
|
logger.warn('SHADE_OBSERVER_TOKEN is set but too short (needs ≥16 chars), observer disabled');
|
||||||
|
} else {
|
||||||
|
logger.info('Observer disabled (SHADE_OBSERVER_TOKEN not set)');
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Stale cleanup task ──────────────────────────────────────
|
||||||
|
|
||||||
|
const cleanupTask = new StaleCleanupTask(store);
|
||||||
|
cleanupTask.start();
|
||||||
|
logger.info('Stale cleanup task started', {
|
||||||
|
staleDays: Number(process.env.SHADE_STALE_DAYS ?? 30),
|
||||||
|
intervalHours: Number(process.env.SHADE_CLEANUP_INTERVAL_HOURS ?? 24),
|
||||||
|
});
|
||||||
|
|
||||||
|
// ─── Start HTTP server ───────────────────────────────────────
|
||||||
|
|
||||||
const port = Number(process.env.PORT ?? 3900);
|
const port = Number(process.env.PORT ?? 3900);
|
||||||
|
|
||||||
@@ -64,6 +101,7 @@ async function shutdown(signal: string) {
|
|||||||
logger.info('Shutting down', { signal });
|
logger.info('Shutting down', { signal });
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
cleanupTask.stop();
|
||||||
server.stop();
|
server.stop();
|
||||||
if ('close' in store && typeof store.close === 'function') {
|
if ('close' in store && typeof store.close === 'function') {
|
||||||
await store.close();
|
await store.close();
|
||||||
|
|||||||
@@ -45,4 +45,22 @@ export interface PrekeyStore {
|
|||||||
|
|
||||||
/** Delete all keys for an address */
|
/** Delete all keys for an address */
|
||||||
deleteAll(address: string): Promise<void>;
|
deleteAll(address: string): Promise<void>;
|
||||||
|
|
||||||
|
// ─── Stale cleanup ─────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Update the last-activity timestamp for an address to the current time.
|
||||||
|
* Called on every read or write that references the address, so stale
|
||||||
|
* cleanup only removes addresses nobody has touched in a long time.
|
||||||
|
*/
|
||||||
|
touchIdentity(address: string): Promise<void>;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Purge every identity whose last activity is older than `olderThanMs`
|
||||||
|
* milliseconds ago. Cascades deletion to signed prekeys and one-time
|
||||||
|
* prekeys for the affected addresses.
|
||||||
|
*
|
||||||
|
* @returns the number of addresses purged
|
||||||
|
*/
|
||||||
|
purgeStaleIdentities(olderThanMs: number): Promise<number>;
|
||||||
}
|
}
|
||||||
|
|||||||
65
packages/shade-server/tests/cleanup-api.test.ts
Normal file
65
packages/shade-server/tests/cleanup-api.test.ts
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
import { describe, test, expect } from 'bun:test';
|
||||||
|
import { MemoryPrekeyStore } from '../src/memory-store.js';
|
||||||
|
|
||||||
|
function rand(n: number): Uint8Array {
|
||||||
|
const buf = new Uint8Array(n);
|
||||||
|
globalThis.crypto.getRandomValues(buf);
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('PrekeyStore cleanup API', () => {
|
||||||
|
test('touchIdentity updates last activity', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
await store.saveIdentity('alice', rand(32), rand(32));
|
||||||
|
await store.touchIdentity('alice');
|
||||||
|
// Verify identity is still there
|
||||||
|
expect(await store.getIdentity('alice')).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('purgeStaleIdentities removes old addresses', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
|
||||||
|
// Alice was active long ago
|
||||||
|
await store.saveIdentity('alice', rand(32), rand(32));
|
||||||
|
await store.saveSignedPreKey('alice', 1, rand(32), rand(64));
|
||||||
|
await store.saveOneTimePreKeys('alice', [{ keyId: 100, publicKey: rand(32) }]);
|
||||||
|
// Manually set alice's activity to 1 day ago
|
||||||
|
await store.touchIdentity('alice');
|
||||||
|
(store as any).lastActivity.set('alice', Date.now() - 2 * 24 * 60 * 60 * 1000);
|
||||||
|
|
||||||
|
// Bob is fresh
|
||||||
|
await store.saveIdentity('bob', rand(32), rand(32));
|
||||||
|
await store.touchIdentity('bob');
|
||||||
|
|
||||||
|
// Purge anything older than 1 day
|
||||||
|
const count = await store.purgeStaleIdentities(24 * 60 * 60 * 1000);
|
||||||
|
expect(count).toBe(1);
|
||||||
|
|
||||||
|
// Alice is gone
|
||||||
|
expect(await store.getIdentity('alice')).toBeNull();
|
||||||
|
expect(await store.getSignedPreKey('alice')).toBeNull();
|
||||||
|
expect(await store.getOneTimePreKeyCount('alice')).toBe(0);
|
||||||
|
|
||||||
|
// Bob is still there
|
||||||
|
expect(await store.getIdentity('bob')).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('purge returns 0 when nothing is stale', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
await store.saveIdentity('alice', rand(32), rand(32));
|
||||||
|
await store.touchIdentity('alice');
|
||||||
|
|
||||||
|
const count = await store.purgeStaleIdentities(60 * 60 * 1000); // 1 hour
|
||||||
|
expect(count).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('untouched identities are considered stale', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
// Save without touching — simulates an ancient identity from before cleanup API
|
||||||
|
await store.saveIdentity('ancient', rand(32), rand(32));
|
||||||
|
|
||||||
|
const count = await store.purgeStaleIdentities(1000);
|
||||||
|
expect(count).toBe(1);
|
||||||
|
expect(await store.getIdentity('ancient')).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
74
packages/shade-server/tests/cleanup.test.ts
Normal file
74
packages/shade-server/tests/cleanup.test.ts
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
import { describe, test, expect } from 'bun:test';
|
||||||
|
import { StaleCleanupTask } from '../src/cleanup.js';
|
||||||
|
import { MemoryPrekeyStore } from '../src/memory-store.js';
|
||||||
|
|
||||||
|
function rand(n: number): Uint8Array {
|
||||||
|
const buf = new Uint8Array(n);
|
||||||
|
globalThis.crypto.getRandomValues(buf);
|
||||||
|
return buf;
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('StaleCleanupTask', () => {
|
||||||
|
test('runs once immediately on start', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
// Save an untouched (so stale) identity
|
||||||
|
await store.saveIdentity('ancient', rand(32), rand(32));
|
||||||
|
|
||||||
|
const task = new StaleCleanupTask(store, { staleDays: 1, intervalHours: 24 });
|
||||||
|
task.start();
|
||||||
|
// Give the immediate run a microtask to complete
|
||||||
|
await new Promise((r) => setTimeout(r, 50));
|
||||||
|
|
||||||
|
expect(await store.getIdentity('ancient')).toBeNull();
|
||||||
|
task.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('runOnce returns count of purged addresses', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
await store.saveIdentity('ancient1', rand(32), rand(32));
|
||||||
|
await store.saveIdentity('ancient2', rand(32), rand(32));
|
||||||
|
|
||||||
|
const task = new StaleCleanupTask(store, { staleDays: 1 });
|
||||||
|
const count = await task.runOnce();
|
||||||
|
expect(count).toBe(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('leaves fresh identities alone', async () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
await store.saveIdentity('fresh', rand(32), rand(32));
|
||||||
|
await store.touchIdentity('fresh');
|
||||||
|
|
||||||
|
const task = new StaleCleanupTask(store, { staleDays: 30 });
|
||||||
|
const count = await task.runOnce();
|
||||||
|
expect(count).toBe(0);
|
||||||
|
expect(await store.getIdentity('fresh')).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('stop clears the interval', () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
const task = new StaleCleanupTask(store, { staleDays: 1, intervalHours: 1 });
|
||||||
|
task.start();
|
||||||
|
expect(task.isRunning).toBe(true);
|
||||||
|
task.stop();
|
||||||
|
expect(task.isRunning).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('reads defaults from env vars', () => {
|
||||||
|
const store = new MemoryPrekeyStore();
|
||||||
|
const oldDays = process.env.SHADE_STALE_DAYS;
|
||||||
|
const oldHours = process.env.SHADE_CLEANUP_INTERVAL_HOURS;
|
||||||
|
process.env.SHADE_STALE_DAYS = '7';
|
||||||
|
process.env.SHADE_CLEANUP_INTERVAL_HOURS = '12';
|
||||||
|
try {
|
||||||
|
const task = new StaleCleanupTask(store);
|
||||||
|
// staleMs = 7 days, intervalMs = 12 hours
|
||||||
|
expect((task as any).staleMs).toBe(7 * 24 * 60 * 60 * 1000);
|
||||||
|
expect((task as any).intervalMs).toBe(12 * 60 * 60 * 1000);
|
||||||
|
} finally {
|
||||||
|
if (oldDays !== undefined) process.env.SHADE_STALE_DAYS = oldDays;
|
||||||
|
else delete process.env.SHADE_STALE_DAYS;
|
||||||
|
if (oldHours !== undefined) process.env.SHADE_CLEANUP_INTERVAL_HOURS = oldHours;
|
||||||
|
else delete process.env.SHADE_CLEANUP_INTERVAL_HOURS;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -66,9 +66,19 @@ export async function ensurePrekeyServerTables(sql: Sql): Promise<void> {
|
|||||||
CREATE TABLE IF NOT EXISTS shade_server_identities (
|
CREATE TABLE IF NOT EXISTS shade_server_identities (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
identity_signing_key TEXT NOT NULL,
|
identity_signing_key TEXT NOT NULL,
|
||||||
identity_dh_key TEXT NOT NULL
|
identity_dh_key TEXT NOT NULL,
|
||||||
|
last_activity_at BIGINT NOT NULL DEFAULT 0
|
||||||
)
|
)
|
||||||
`;
|
`;
|
||||||
|
// Migrate existing deployments (no-op if column exists)
|
||||||
|
await sql`
|
||||||
|
ALTER TABLE shade_server_identities
|
||||||
|
ADD COLUMN IF NOT EXISTS last_activity_at BIGINT NOT NULL DEFAULT 0
|
||||||
|
`;
|
||||||
|
await sql`
|
||||||
|
CREATE INDEX IF NOT EXISTS shade_server_identities_activity_idx
|
||||||
|
ON shade_server_identities(last_activity_at)
|
||||||
|
`;
|
||||||
await sql`
|
await sql`
|
||||||
CREATE TABLE IF NOT EXISTS shade_server_signed_prekeys (
|
CREATE TABLE IF NOT EXISTS shade_server_signed_prekeys (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
|
|||||||
@@ -34,11 +34,12 @@ export class PostgresPrekeyStore implements PrekeyStore {
|
|||||||
|
|
||||||
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
||||||
await this.sql`
|
await this.sql`
|
||||||
INSERT INTO shade_server_identities (address, identity_signing_key, identity_dh_key)
|
INSERT INTO shade_server_identities (address, identity_signing_key, identity_dh_key, last_activity_at)
|
||||||
VALUES (${address}, ${toBase64(identitySigningKey)}, ${toBase64(identityDHKey)})
|
VALUES (${address}, ${toBase64(identitySigningKey)}, ${toBase64(identityDHKey)}, ${Date.now()})
|
||||||
ON CONFLICT (address) DO UPDATE SET
|
ON CONFLICT (address) DO UPDATE SET
|
||||||
identity_signing_key = EXCLUDED.identity_signing_key,
|
identity_signing_key = EXCLUDED.identity_signing_key,
|
||||||
identity_dh_key = EXCLUDED.identity_dh_key
|
identity_dh_key = EXCLUDED.identity_dh_key,
|
||||||
|
last_activity_at = EXCLUDED.last_activity_at
|
||||||
`;
|
`;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -122,4 +123,31 @@ export class PostgresPrekeyStore implements PrekeyStore {
|
|||||||
await sql`DELETE FROM shade_server_one_time_prekeys WHERE address = ${address}`;
|
await sql`DELETE FROM shade_server_one_time_prekeys WHERE address = ${address}`;
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Stale cleanup ──────────────────────────────────────
|
||||||
|
|
||||||
|
async touchIdentity(address: string): Promise<void> {
|
||||||
|
await this.sql`
|
||||||
|
UPDATE shade_server_identities
|
||||||
|
SET last_activity_at = ${Date.now()}
|
||||||
|
WHERE address = ${address}
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async purgeStaleIdentities(olderThanMs: number): Promise<number> {
|
||||||
|
const cutoff = Date.now() - olderThanMs;
|
||||||
|
const rows = await this.sql<Array<{ address: string }>>`
|
||||||
|
SELECT address FROM shade_server_identities
|
||||||
|
WHERE last_activity_at < ${cutoff}
|
||||||
|
`;
|
||||||
|
if (rows.length === 0) return 0;
|
||||||
|
|
||||||
|
const addresses = rows.map((r) => r.address);
|
||||||
|
await this.sql.begin(async (sql) => {
|
||||||
|
await sql`DELETE FROM shade_server_identities WHERE address = ANY(${addresses})`;
|
||||||
|
await sql`DELETE FROM shade_server_signed_prekeys WHERE address = ANY(${addresses})`;
|
||||||
|
await sql`DELETE FROM shade_server_one_time_prekeys WHERE address = ANY(${addresses})`;
|
||||||
|
});
|
||||||
|
return addresses.length;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -25,6 +25,8 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
|||||||
deleteIdentity: ReturnType<Database['prepare']>;
|
deleteIdentity: ReturnType<Database['prepare']>;
|
||||||
deleteSignedPreKey: ReturnType<Database['prepare']>;
|
deleteSignedPreKey: ReturnType<Database['prepare']>;
|
||||||
deleteOTPKs: ReturnType<Database['prepare']>;
|
deleteOTPKs: ReturnType<Database['prepare']>;
|
||||||
|
touchIdentity: ReturnType<Database['prepare']>;
|
||||||
|
findStale: ReturnType<Database['prepare']>;
|
||||||
};
|
};
|
||||||
|
|
||||||
constructor(dbPath?: string) {
|
constructor(dbPath?: string) {
|
||||||
@@ -40,7 +42,8 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
|||||||
CREATE TABLE IF NOT EXISTS identities (
|
CREATE TABLE IF NOT EXISTS identities (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
identity_signing_key TEXT NOT NULL,
|
identity_signing_key TEXT NOT NULL,
|
||||||
identity_dh_key TEXT NOT NULL
|
identity_dh_key TEXT NOT NULL,
|
||||||
|
last_activity_at INTEGER NOT NULL DEFAULT 0
|
||||||
);
|
);
|
||||||
CREATE TABLE IF NOT EXISTS signed_prekeys (
|
CREATE TABLE IF NOT EXISTS signed_prekeys (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
@@ -55,12 +58,23 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
|||||||
public_key TEXT NOT NULL
|
public_key TEXT NOT NULL
|
||||||
);
|
);
|
||||||
CREATE INDEX IF NOT EXISTS idx_otp_address ON one_time_prekeys(address);
|
CREATE INDEX IF NOT EXISTS idx_otp_address ON one_time_prekeys(address);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_identities_activity ON identities(last_activity_at);
|
||||||
`);
|
`);
|
||||||
|
|
||||||
|
// Migrate existing databases: add last_activity_at if missing
|
||||||
|
try {
|
||||||
|
const cols = this.db.prepare('PRAGMA table_info(identities)').all() as any[];
|
||||||
|
const hasActivity = cols.some((c) => c.name === 'last_activity_at');
|
||||||
|
if (!hasActivity) {
|
||||||
|
this.db.exec('ALTER TABLE identities ADD COLUMN last_activity_at INTEGER NOT NULL DEFAULT 0');
|
||||||
|
this.db.exec('CREATE INDEX IF NOT EXISTS idx_identities_activity ON identities(last_activity_at)');
|
||||||
|
}
|
||||||
|
} catch {}
|
||||||
}
|
}
|
||||||
|
|
||||||
private prepareStatements() {
|
private prepareStatements() {
|
||||||
this.stmts = {
|
this.stmts = {
|
||||||
saveIdentity: this.db.prepare('INSERT OR REPLACE INTO identities (address, identity_signing_key, identity_dh_key) VALUES (?, ?, ?)'),
|
saveIdentity: this.db.prepare('INSERT OR REPLACE INTO identities (address, identity_signing_key, identity_dh_key, last_activity_at) VALUES (?, ?, ?, ?)'),
|
||||||
getIdentity: this.db.prepare('SELECT identity_signing_key, identity_dh_key FROM identities WHERE address = ?'),
|
getIdentity: this.db.prepare('SELECT identity_signing_key, identity_dh_key FROM identities WHERE address = ?'),
|
||||||
saveSignedPreKey: this.db.prepare('INSERT OR REPLACE INTO signed_prekeys (address, key_id, public_key, signature) VALUES (?, ?, ?, ?)'),
|
saveSignedPreKey: this.db.prepare('INSERT OR REPLACE INTO signed_prekeys (address, key_id, public_key, signature) VALUES (?, ?, ?, ?)'),
|
||||||
getSignedPreKey: this.db.prepare('SELECT key_id, public_key, signature FROM signed_prekeys WHERE address = ?'),
|
getSignedPreKey: this.db.prepare('SELECT key_id, public_key, signature FROM signed_prekeys WHERE address = ?'),
|
||||||
@@ -70,6 +84,8 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
|||||||
deleteIdentity: this.db.prepare('DELETE FROM identities WHERE address = ?'),
|
deleteIdentity: this.db.prepare('DELETE FROM identities WHERE address = ?'),
|
||||||
deleteSignedPreKey: this.db.prepare('DELETE FROM signed_prekeys WHERE address = ?'),
|
deleteSignedPreKey: this.db.prepare('DELETE FROM signed_prekeys WHERE address = ?'),
|
||||||
deleteOTPKs: this.db.prepare('DELETE FROM one_time_prekeys WHERE address = ?'),
|
deleteOTPKs: this.db.prepare('DELETE FROM one_time_prekeys WHERE address = ?'),
|
||||||
|
touchIdentity: this.db.prepare('UPDATE identities SET last_activity_at = ? WHERE address = ?'),
|
||||||
|
findStale: this.db.prepare('SELECT address FROM identities WHERE last_activity_at < ?'),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -78,7 +94,7 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
async saveIdentity(address: string, identitySigningKey: Uint8Array, identityDHKey: Uint8Array): Promise<void> {
|
||||||
this.stmts.saveIdentity.run(address, toBase64(identitySigningKey), toBase64(identityDHKey));
|
this.stmts.saveIdentity.run(address, toBase64(identitySigningKey), toBase64(identityDHKey), Date.now());
|
||||||
}
|
}
|
||||||
|
|
||||||
async getIdentity(address: string): Promise<{ identitySigningKey: Uint8Array; identityDHKey: Uint8Array } | null> {
|
async getIdentity(address: string): Promise<{ identitySigningKey: Uint8Array; identityDHKey: Uint8Array } | null> {
|
||||||
@@ -135,4 +151,26 @@ export class SqlitePrekeyStore implements PrekeyStore {
|
|||||||
});
|
});
|
||||||
deleteAllTx();
|
deleteAllTx();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Stale cleanup ──────────────────────────────────────
|
||||||
|
|
||||||
|
async touchIdentity(address: string): Promise<void> {
|
||||||
|
this.stmts.touchIdentity.run(Date.now(), address);
|
||||||
|
}
|
||||||
|
|
||||||
|
async purgeStaleIdentities(olderThanMs: number): Promise<number> {
|
||||||
|
const cutoff = Date.now() - olderThanMs;
|
||||||
|
const staleRows = this.stmts.findStale.all(cutoff) as Array<{ address: string }>;
|
||||||
|
if (staleRows.length === 0) return 0;
|
||||||
|
|
||||||
|
const purgeTx = this.db.transaction(() => {
|
||||||
|
for (const row of staleRows) {
|
||||||
|
this.stmts.deleteIdentity.run(row.address);
|
||||||
|
this.stmts.deleteSignedPreKey.run(row.address);
|
||||||
|
this.stmts.deleteOTPKs.run(row.address);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
purgeTx();
|
||||||
|
return staleRows.length;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
65
scripts/build-docker.ts
Normal file
65
scripts/build-docker.ts
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
#!/usr/bin/env bun
|
||||||
|
/**
|
||||||
|
* Build the Shade Prekey Server Docker image locally.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* bun run build:docker # build as shade-prekey:dev
|
||||||
|
* bun run build:docker -- --tag X # custom tag
|
||||||
|
* bun run build:docker -- --push # build + push to Gitea registry
|
||||||
|
*
|
||||||
|
* Env:
|
||||||
|
* GITEA_TOKEN — required for --push
|
||||||
|
* GITEA_USER — default Stian
|
||||||
|
*/
|
||||||
|
import { $ } from 'bun';
|
||||||
|
|
||||||
|
const args = process.argv.slice(2);
|
||||||
|
const tagIdx = args.indexOf('--tag');
|
||||||
|
const tag = tagIdx >= 0 ? args[tagIdx + 1] : 'dev';
|
||||||
|
const shouldPush = args.includes('--push');
|
||||||
|
const user = process.env.GITEA_USER ?? 'Stian';
|
||||||
|
const registry = 'gt.zyon.no';
|
||||||
|
const image = `${registry.toLowerCase()}/${user.toLowerCase()}/shade-prekey`;
|
||||||
|
const fullTag = `${image}:${tag}`;
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
console.log(`Building ${fullTag}…`);
|
||||||
|
await $`docker build -f packages/shade-server/Dockerfile -t ${fullTag} .`;
|
||||||
|
|
||||||
|
console.log(`✓ Built ${fullTag}`);
|
||||||
|
const { stdout: size } = await $`docker images ${fullTag} --format {{.Size}}`.quiet();
|
||||||
|
console.log(` Size: ${size.toString().trim()}`);
|
||||||
|
|
||||||
|
if (shouldPush) {
|
||||||
|
const token = process.env.GITEA_TOKEN;
|
||||||
|
if (!token) {
|
||||||
|
console.error('GITEA_TOKEN env var required for --push');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
console.log(`Pushing ${fullTag}…`);
|
||||||
|
// Login via stdin to avoid leaking token in process table
|
||||||
|
const loginProc = Bun.spawn(['docker', 'login', registry, '-u', user, '--password-stdin'], {
|
||||||
|
stdin: 'pipe',
|
||||||
|
stdout: 'inherit',
|
||||||
|
stderr: 'inherit',
|
||||||
|
});
|
||||||
|
loginProc.stdin.write(token);
|
||||||
|
await loginProc.stdin.end();
|
||||||
|
await loginProc.exited;
|
||||||
|
|
||||||
|
await $`docker push ${fullTag}`;
|
||||||
|
console.log(`✓ Pushed ${fullTag}`);
|
||||||
|
|
||||||
|
if (tag !== 'latest') {
|
||||||
|
const latestTag = `${image}:latest`;
|
||||||
|
await $`docker tag ${fullTag} ${latestTag}`;
|
||||||
|
await $`docker push ${latestTag}`;
|
||||||
|
console.log(`✓ Pushed ${latestTag}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error(err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user