12 Commits

Author SHA1 Message Date
3c69995097 fix(docker): raise fd limit so bun install can extract large tarballs
Some checks failed
Test / test (push) Has been cancelled
BuildKit gives each RUN a 1024 soft file-descriptor limit against a ~1M
hard limit, and bun extracts tarballs in parallel — a large enough package
exhausts the descriptors and the install dies with "Fail extracting
tarball". It only shows up inside a build, because `docker run` inherits a
far higher limit, which makes it look like a flaky registry.

This bit Nova's deploy (on mermaid) and was fixed there; shade-server has
the identical pattern and escaped only because its layers were cached, so
it would have failed on the next cache-less build. Fixing it at the source
too, so a vendor sync cannot silently drop it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 00:06:02 +02:00
9789b6fc16 docs(vault): scaffold.zyon.no i eksempelet
Some checks failed
Test / test (push) Has been cancelled
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 14:48:14 +02:00
40286a8542 test(vault): verktøy for å verifisere en kjørende vault over ekte HTTP
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Enhets- og e2e-testene kjører mot Hono-fetch i prosess. Dette skriptet går
mot en faktisk lyttende tjeneste — container eller deployet instans — og
er det som fanget at Dockerfilen pekte på src/standalone.ts i en publisert
pakke som bare shipper dist/.

  SCAFFOLDD_SOCKET=... RELAY=https://vault.zyon.no bun live-e2e.ts

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 12:05:18 +02:00
cc6b1d53d7 chore: bump til 4.13.0 og legg @shade/vault i publiseringslistene
Tørrkjøringen fanget to ting: vault manglet i PACKAGES (så @shade/server
feilet på en ukjent workspace-referanse), og jeg hadde innført en syklus
storage-sqlite → vault → server → storage-sqlite. Den siste er løst ved at
storage-sqlite bare bruker VaultStore som TYPE — flyttet til devDependencies,
så det ikke er en runtime-avhengighet.

vault står ved siden av inbox-server i lista: begge er i en syklus med
server (de trenger verifyPayload, standalone.ts mounter rutene deres), og
lista håndterer det allerede ved å liste server først.

publish:dry: 26 pakket, 0 feilet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 12:01:05 +02:00
d8f6da55ef docs(scaffold): plan, tasks og loggposter for vault + G0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 11:58:49 +02:00
3da977207c chore(ci): pin TypeScript og kjør typecheck som gate
G0-P0. typecheck-all.ts kjørte 'bunx tsc', som henter nyeste utgivelse ved
hver kjøring — gaten kunne gå rød, eller stille slutte å fange ting, fordi
TypeScript hadde sluppet en versjon og ikke fordi repoet endret seg.

typescript pinnet til 7.0.2; scriptet bruker node_modules/.bin/tsc. Nytt
Typecheck-steg i test.yml, plassert FØR testene: en typefeil er billigere å
lese enn kjøretidsfeilen den til slutt forårsaker.

Helsesjekkens antakelse om baseUrl stemmer ikke lenger — feltet finnes ikke
i repoet, og alle 26 pakker er grønne inkludert consumer-strict.

dist-build/ lagt i .gitignore (observer-avviket fra G0).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 11:58:49 +02:00
84d3166ca1 feat(vault): server-side kryptert fillager (V4.13)
Shade kunne flytte filer mellom peers (@shade/files) og lagre én liten
profil-blob per konto, men hadde ingen alltid-på lagring av krypterte filer.
Uten den kan ingen Shade-app tilby backup, og ingen klient lese data mens
peeren som eier dem er avslått.

Objekter er innholdsadresserte på hashen av CHIFFERTEKSTEN, så relayen kan
lagre, deduplisere og verifisere uten nøkkel — den regner om hashen ved
opplasting og avviser feilnavngitte objekter. Stier bor inne i det krypterte
manifestet, aldri i objektnavn: relayen skal ikke lære hva filene heter.
Loggen er append-only, så historikk og rollback følger av modellen.

SqliteVaultStore har med vilje INGEN minne-fallback, i motsetning til
blob-storen. Den fallbacken slettet Prisms profil ved en rutine-redeploy
2026-08-12 fordi den fungerte helt til containeren ble recreated, uten en
eneste feilmelding. En backup som glemmer er verre enn ingen backup, så uten
SHADE_VAULT_DB_PATH mountes rutene ikke — med en logglinje som sier hvorfor.

Én feil fanget av testene: pubkeyen ble først lagt på UTENFOR signaturen,
som både brøt verifyPayload og ville latt hvem som helst bytte identitet i
transit på den TOFU-pinnende førsteskrivingen.

16 vault- + 7 store-tester, alle mot de ekte rutehåndtererne gjennom Honos
fetch. Kjeden er dessuten kjørt mot en ekte HTTP-server med et ekte
workspace: 491 filer / 7,7 MB, alle bit-identiske etter gjenoppretting,
og andre push etter én endring sendte 0 KB.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 11:58:37 +02:00
012d7f5289 fix(storage-encrypted): tilfeldig AES-GCM-nonce + vault-nøkkelgren
To endringer i samme krypto-filer.

G0-P0: deriveNonce() lagde nonce som en ren funksjon av radens identitet.
saveSession re-forsegler ved hvert ratchet-steg, så (key, nonce) gjentok seg
per konstruksjon over ULIK plaintext — GCM forbidden attack, som lekker både
XOR-en av plaintekstene og GHASH-subnøkkelen. Fire kallesteder: sealString og
sealBytes tar nå randomNonce(), openString/openBytes slutter å tvinge den.

Ingen migrering: aeadOpen har alltid lest nonce fra blob-prefikset, så
eksisterende data åpnes uendret. Testet eksplisitt. deriveNonce og
expectedNonce beholdt som deprecated — begge er del av den publiserte flaten.
Android var aldri rammet; KeystoreStorage bruker cipher.iv, som Android
Keystore alltid genererer tilfeldig.

V4.13: egen HKDF-gren for vault (shade-vault-{id,content,sig}-v1). Ikke
gjenbruk av shade-blob-*-v1: en vault-nøkkel leser hver eneste fil, en
profil-blob-nøkkel leser en vertsliste. Delt derivasjon ville gjort ett
kompromiss om til det andre.

9 nye tester; testene er bevist å ha verdi ved å reversere fiksen —
nøyaktig de tre som beskriver feilen faller. 87 grønne totalt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 11:58:19 +02:00
96c20cb4b2 fix(session): remember where aliasSession moved a session
aliasSession knew that two labels name the same peer, then threw that
knowledge away. The binding lived only in the caller's memory, so a
restart lost it — and the peer could not repair it from its side.

First contact forces the receiver to label a session by the only sender
hint a relay surfaces, an 8-byte signing-key fingerprint (`fp:<hex>`).
Once the peer announces its canonical address, aliasSession moves the
session there. But the peer keeps sending under `fp:<hex>`, because its
transport derives the same label from the same hint every time. After a
restart the session sat under the canonical address, inbound frames
resolved to `fp:<hex>`, and nothing matched. The peer held a valid
session so it never re-ran X3DH: the failure was permanent, and only a
manual re-link cleared it.

Observed in Prism as `No session for address: fp:579c3b335d66e2c0` on
every receive for three days, with a phone whose every RPC timed out.

StorageProvider gains saveSessionAlias / getSessionAlias /
removeSessionAliasesFor, optional so third-party implementations keep
compiling, and implemented across all seven backends. Lookups resolve
through resolveLabel(), which runs BEFORE the peer mutex — locking the
alias while mutating the canonical session would let an aliased and a
canonical caller ratchet the same state concurrently.

A live session under a label always wins over an alias, and prekey
envelopes never resolve: both keep a re-link establishing a fresh
session instead of being redirected into the stale one. Aliases are
dropped in resetSession and acceptIdentityChange, and memoized so the
hot path costs no extra read.

The sdk.test.ts case that asserted a dead fp-label encoded the old
behaviour; it now pins the new contract.

Verified: 1166 tests pass (from 1160). With alias persistence disabled
as a negative control, 5 of the 6 new tests fail, including both
restart cases.

Also drops `baseUrl` from the consumer-strict tsconfig — removed in
TS 6.0, and it was failing the typecheck that gates publishing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 19:36:31 +02:00
b44acf867b release(files): prepare 4.11.2 hotfix
Some checks failed
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Test / test (push) Has been cancelled
2026-07-10 17:41:25 +02:00
8c67a00f37 fix(files): await pull-mode stream readiness 2026-07-10 17:38:35 +02:00
306bf08452 chore: adopter Scaffold plan-kontrakt
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 19:04:48 +02:00
82 changed files with 3418 additions and 103 deletions

View File

@@ -28,6 +28,13 @@ jobs:
- name: Install dependencies
run: ~/.bun/bin/bun install --frozen-lockfile
# Before the tests, not after: a type error is cheaper to read than the
# runtime failure it eventually causes. The compiler is pinned in
# devDependencies, so this gate goes red when the repo changes — not
# when TypeScript ships a release.
- name: Typecheck
run: ~/.bun/bin/bun run typecheck
- name: Run tests
env:
SHADE_TEST_PG_URL: postgres://postgres:test@localhost:5432/postgres

3
.gitignore vendored
View File

@@ -5,3 +5,6 @@ dist/
**/.tmp-*.db
**/.tmp-*.db-shm
**/.tmp-*.db-wal
# Byggeartefakt fra @shade/observer (outDir-avviket i G0)
dist-build/

24
.scaffold/log.md Normal file
View File

@@ -0,0 +1,24 @@
# Logg — Shade
## 2026-07-10 — Holistisk helsesjekk + gated plan mot V5.0
Fem parallelle analysepass (arkitektur, test/CI, kryptosikkerhet, Android-paritet, docs/DX), topp-funn kildeverifisert. Kjernen er reell og moden (1159 tester grønne, vektorer byte-pinnet, KT/recovery ekte implementasjoner) — men «GA-frosset og revisjonsklar» stemte ikke. Fire P0-blockere identifisert og lagt inn som blockers i plan.md + kort i tasks.yaml:
1. **AES-GCM nonce-gjenbruk** i at-rest storage (`storage-encrypted/src/crypto/kdf.ts:116`) — deterministisk nonce uten meldingsteller, `saveSession` re-krypterer hver ratchet-op → GCM forbidden attack mot §4. Reachable bekreftet; fiks er liten (tilfeldig nonce, wire lagrer den allerede i prefiks).
2. **@shade/observer@4.11.1 tomt på registry** — `outDir=dist-build` men publish shipper `dist/`; `dist-build/` er det untrackede git-avviket.
3. **Typecheck-gaten rød** — TS upinnet → `bunx tsc` henter 7.0.2 (fjernet `baseUrl`, brukt i consumer-strict); gaten kjøres aldri i CI.
4. **@shade/cli bin dingler** etter publish (bin ikke omskrevet til dist/).
P1: safety-number matcher aldri peerens eget (`session.ts:241`, MITM-forsvar ikke-funksjonelt); backup-KDF HKDF ikke argon2id (finnes allerede i repoet); hono <4.12.18; begge flaggskip-eksempler røde (accept-race `shade-transfer/engine.ts:1140`). Paritet: ShadeStream (v4.11) uvoktet selv på TS-siden, Kotlin stopper på v4.10.
Låst prinsipp: **V5.0-implementasjon er gated på G0–G4 grønne.** V5.0.md er idéskisse — modnes parallelt (wire-typer reconciled mot 0x31–0x33, Insertable Streams-mekanisme, ratchet-params). Full rapport: helsesjekk-artifact (claude.ai/code).
## 2026-07-09 — Adoptert Scaffold plan-kontrakt
Migrerte `docs/ROADMAP.md` og `android/shade-android/ROADMAP-ANDROID.md` til `.scaffold/`: nåbildet destillert inn i `plan.md`, originalene arkivert under `.scaffold/archive/` med opprinnelig mappestruktur. Versjonsdesigndokumentene (`docs/V5.0.md`, `docs/archive/V*.md`) er bevisst latt stå — de er spesifikasjoner, ikke planfiler.
## 2026-05-15 — v4.11: streaming Double-Ratchet sub-sessions
Siste post-GA-økning på 4.x-linjen: `ShadeStream` gir in-memory sub-sessions (seal/open uten keystore-I/O per frame) med nye wire-typer 0x31–0x33. Låst beslutning: stream-ratchets persisteres aldri — en droppet stream gjenåpnes, aldri gjenopptas.
## 2026-05-09 — Android M-Cross 1–4 fullført + Keystore-adapter
All kryptografisk paritet TS↔Kotlin grønn via delte test-vektorer: KDF-kjede, HKDF-labels, X3DH, ratchet-steg, fingerprint, wire 0x02 og 0x11 (streams), backup-HKDF, group sender-keys og storage-HKDF. Etterslepet fra Android-roadmapen landet samme dag: scrypt + argon2id via Bouncy Castle og `shade-android-keystore`-modulen (hardware-backed master key + `KeystoreStorage`). Gjenstår: socket-interop-test og instrumenterte Keystore-tester.
## 2026-05-03 — V4.0 GA: V3.x-konsolidering og audit-pakking
Alle fasene V3.1–V3.12 ferdige og merget: dokumentasjon/hardening, at-rest storage-kryptering, trust-UX, observability (OTel), Android-paritet, inbox (store-and-forward), transport-bridge, web workers-krypto, filmetadata, social key recovery, WebRTC P2P og key transparency. Wire-formatet låst — uendret fra 0.4.x, så 4.0-peers interopererer byte-for-byte med 0.4.x. Kjernen pakket for ekstern review. Låst beslutning: alt VOIP/video skilt ut til V5.0, bygget oppå den frosne 4.0-stacken via reserverte envelope-typer (ikke breaking).

View File

@@ -0,0 +1,41 @@
---
title: "G0: AES-GCM nonce-gjenbruk fjernet fra at-rest-lagringen"
date: 2026-08-13
author: Claude
---
Den første av de fire P0-ene fra helsesjekken 10. juli er borte. `row-codec` forsegler nå med tilfeldig nonce i stedet for en avledet.
## Hva som var galt
`deriveNonce()` laget nonce som en ren funksjon av `(fieldKey, table, pk)` — altså av radens identitet. Doc-kommentaren argumenterte for at det var trygt fordi «hver `(key, plaintext)`-kombinasjon opptrer høyst én gang», men det er nøyaktig omvendt av hva AES-GCM krever: det farlige er å gjenbruke `(key, nonce)` på tvers av *ulike* plaintekster. `saveSession` re-forsegler sesjonsraden ved hvert ratchet-steg, så paret gjentok seg per konstruksjon. Konsekvensen er ikke bare XOR av plaintekstene, men lekkasje av GHASH-subnøkkelen — «the forbidden attack» — som gir tag-forfalskning under den nøkkelen. Det gjaldt alle muterbare rader: sesjoner, config, prekey-state og trust.
## Fiksen
Fire kallesteder, som kartlagt: `sealString` og `sealBytes` (`row-codec.ts:67`/`:96`) tar nå `randomNonce()` fra `aead.ts`, og `openString`/`openBytes` (`:83`/`:112`) slutter å sende `expectedNonce`.
Tilfeldig ble valgt framfor en teller fordi codec-en er delt mellom SQLite og Postgres og ikke har noen varig per-rad skrivteller å støtte seg på. 96 bit holder collision-sannsynligheten neglisjerbar langt forbi et realistisk antall re-lagringer.
`deriveNonce` er beholdt og markert `@deprecated` framfor slettet — den er del av den publiserte flaten via både `index.ts` og `crypto.ts`. Ingen kaller den lenger inne i pakken, og et søk gjennom `packages/`, `android/` og `examples/` fant ingen andre konsumenter. `expectedNonce`-parameteren i `aeadOpen` er beholdt på samme vilkår, med dokumentasjon om hvorfor den ikke skal brukes: den ga ingen tuklingsdeteksjon som AEAD-taggen og `(table, column, pk)`-AAD-en ikke allerede gir.
`randomNonce` er eksportert fra `index.ts` og `crypto.ts`, så konsumenter som forsegler selv har den trygge kilden for hånden.
## Ingen migrering trengs
`aeadOpen` har alltid lest nonce fra blob-prefikset; `expectedNonce` var kun en valgfri ekstrasjekk. Eksisterende blobber åpnes derfor uendret etter fiksen. Todoen om en migreringssti er erstattet av en test som beviser det.
## Android var aldri rammet
Kryssplattform-vektoren «Storage HKDF: rowNonce» tester HKDF-funksjonen, ikke at forseglingen bruker den, så den består uendret — `deriveNonce` er fortsatt der med samme oppførsel. Og `KeystoreStorage` henter nonce fra `cipher.iv`, altså Android Keystores egen tilfeldige IV, som ikke lar seg overstyre. TS-siden er nå på linje med Kotlin, ikke omvendt.
## Verifisering
Ny `tests/nonce-reuse.test.ts` med ni tester i tre grupper: nonce-unikhet (inkludert 200 re-lagringer med 200 distinkte nonces), bakoverkompatibilitet (blob forseglet med den gamle avledningen åpnes, og nye og gamle blobber leses under samme nøkkel), og at tuklingsdeteksjonen overlevde — flippet nonce-byte og flyttet rad avvises fortsatt.
Testene ble bevist å ha verdi ved å reversere fiksen midlertidig: nøyaktig de tre som beskriver feilen falt, mens bakoverkompatibilitets- og tuklingstestene sto uendret.
87 tester grønne i `@shade/storage-encrypted`, typecheck ren, og de to avhengige pakkene kjørt: `@shade/cli` 15/15 og `@shade/sdk` 104/104. En enkelt sdk-feil i første kjøring var en flaky gjennomstrømningsmåling — tre påfølgende kjøringer er grønne.
## Ikke publisert
Fiksen ligger i kildetreet. Prism er pinnet på `@shade/storage-encrypted@4.12.0` og får den først ved en release, som er riktig rekkefølge: `prism-daemon` skal ikke restartes nå.

View File

@@ -0,0 +1,41 @@
---
title: "@shade/vault — server-side kryptert fillager (V4.13)"
date: 2026-08-14
author: Claude
---
Shade kunne flytte filer mellom peers (`@shade/files`) og lagre én liten profil-blob per konto (`/v1/blob/<slotId>`), men hadde ingen alltid-på lagring av krypterte filer. Uten den kan ingen Shade-app tilby backup, og ingen klient kan lese data mens peeren som eier dem er avslått. `@shade/vault` fyller det hullet.
## Modellen
Tre ideer, og formen følger av dem.
**Objekter er innholdsadresserte.** Et objekts navn er SHA-256 av *ciphertexten*, så relayen kan lagre, deduplisere og verifisere uten å forstå noe. Den regner om hashen ved opplasting og avviser et objekt som ikke er det det utgir seg for — uten å eie en nøkkel.
**Et manifest navngir samlingen.** Stier bor i manifestet, ikke i objektnavn: relayen skal ikke lære at brukeren har en fil som heter `Projects/Skilsmisse/plan.md`. Manifestet er selv kryptert og lagret som et objekt.
**Loggen er append-only.** Hver commit legger til en rad som peker på et manifest. Historikk, rollback og «hva endret seg sist tirsdag» faller ut av det — versjoneringshalvdelen av bestillingen.
## Valg verdt å begrunne
*Egen HKDF-gren* (`shade-vault-{id,content,sig}-v1`) ved siden av `shade-blob-*-v1`. En vault-nøkkel leser hver eneste fil; en profil-blob-nøkkel leser en vertsliste. Delt derivasjon ville gjort ett kompromiss om til det andre.
*Ingen konvergent kryptering.* Identisk klartekst gir ulike objektnavn fordi nonce er fersk. Dedupliseringen skjer derfor innenfor en versjonskjede — en uendret fil beholder hashen sin fordi klienten gjenbruker objektet den allerede lastet opp — ikke på tvers av uavhengige forseglinger. Konvergent kryptering ville lekket hvilke filer to brukere deler, som er nøyaktig det en blind butikk ikke skal.
*Commit verifiserer at objektene finnes.* Ellers ville loggen kunne publisere en versjon som ikke lar seg gjenopprette, og det er verdt en ekstra rundtur å utelukke.
*Pubkeyen er inne i signaturen.* Første forsøk la den på utenfor, som både brøt `verifyPayload` (den kanonikaliserer alle felt) og — hvis skjemaet hadde ignorert det — ville latt hvem som helst bytte identitet i transit på den TOFU-pinnende førsteskrivingen. Fanget av testene.
## Verifisering
16 tester, alle mot de ekte rutehåndtererne gjennom Honos `fetch` — ikke mot en mock-transport, som ville vært enig med klienten per konstruksjon og ikke bevist noe om wire-kontrakten.
Dekket: rundtur bit-for-bit; gjenoppretting på en fersk klient som bare har kontonøkkelen; feil master-nøkkel når ikke fram; tre versjoner med rollback til hver enkelt; en 50 KB uendret fil lastes ikke opp på nytt (`bytesUploaded` under 1 KB på andre push); slettet fil borte i ny versjon, intakt i gammel; lagrede objekter inneholder verken innhold eller sti; feilnavngitt objekt avvist; `SEQ_CONFLICT` ved commit fra utdatert head; `MISSING_OBJECTS` ved manglende referanse; fremmed nøkkel avvist mot en pinnet vault; usignert skriving avvist; og at vault-grenen er forskjellig fra blob-grenen.
Typecheck ren. `@shade/storage-encrypted` fortsatt 87/87 etter endringene i `kdf.ts`.
## Gjenstår
Ruting i `shade-server/src/standalone.ts` ved siden av `createBlobRoutes`, en varig `VaultStore` (SQLite/Postgres — bare `MemoryVaultStore` finnes), `docs/vault.md`, og deploy. Ingenting av det er gjort: deploy og publisering er utenfor mandatet for en uovervåket økt.
Arbeidet ligger ukommittert i arbeidstreet, sammen med de eksisterende endringene fra juli.

View File

@@ -0,0 +1,25 @@
---
title: "G0: typecheck-gaten pinnet og lagt i CI"
date: 2026-08-14
author: Claude
---
Tredje av de fire P0-ene fra helsesjekken 10. juli.
## Hva som faktisk var galt
Helsesjekken beskrev gaten som rød fordi TS 7.0.2 hadde fjernet `baseUrl`, som `consumer-strict` brukte. Det stemmer ikke lenger: `baseUrl` finnes ikke noe sted i repoet, og `bun run typecheck` er grønn for alle 26 pakker — inkludert den nye `shade-vault`.
Den ekte feilen står likevel, og er den viktigere av de to: `scripts/typecheck-all.ts` kjørte `bunx tsc`, som henter nyeste utgivelse ved hver kjøring. Gaten kunne altså gå rød — eller stille slutte å fange ting — fordi TypeScript hadde sluppet en versjon, ikke fordi repoet endret seg. Det er en gate som ikke måler det den utgir seg for å måle.
## Fiksen
`typescript` er pinnet til `7.0.2` i rotens `devDependencies`, og `typecheck-all.ts` kjører `node_modules/.bin/tsc` i stedet for `bunx`. Et `Typecheck`-steg er lagt inn i `.gitea/workflows/test.yml`, plassert før testene: en typefeil er billigere å lese enn kjøretidsfeilen den til slutt forårsaker.
## Verifisering
`bun run typecheck` med den pinnede kompilatoren: alle 26 pakker grønne, inkludert consumer-strict-røyktesten (`lib: DOM`, `exactOptionalPropertyTypes`, `paths → workspace`).
## Gjenstår i G0
De to siste P0-ene — `@shade/observer` tomt på registry og `@shade/cli` sin dinglende `bin` — krever begge publisering til npm. Det er en utadvendt handling, og ble ikke gjort i en uovervåket økt.

61
.scaffold/plan.md Normal file
View File

@@ -0,0 +1,61 @@
---
project: "Shade"
phase: "Post-4.x GA — hardening til revisjonsklar (G0–G4) før V5.0-design"
updated: 2026-08-14
focus: "GCM-nonce-fiksen er levert (2026-08-13) og @shade/vault står ferdig og testet (2026-08-14), men er IKKE deployet eller publisert. De tre gjenstående G0-P0-ene er publiserings- og CI-saker. Historikk: G0 fikk en andre bestiller: @shade/vault (server-side kryptert fillager, bestilt 2026-08-13 av Scaffold-backupen) bygger på at-rest-stacken, og skal ikke reise seg på en kjent GCM-svakhet. Nonce-fiksens migreringssti er verifisert gratis — aeadOpen leser alltid nonce fra prefiks, så eksisterende blobber åpnes uendret."
blockers:
- "P0: @shade/observer@4.11.1 tomt på registry (outDir=dist-build, publish shipper dist/)"
- "P0: @shade/cli bin dingler etter publish (bin ikke omskrevet til dist/)"
---
# Plan — Shade
## Mål
- E2EE-bibliotek som implementerer Signal-protokollen (X3DH + Double Ratchet) for TypeScript/Bun — drop-in for frontend, backend og mobil, med forward secrecy og post-compromise recovery.
- Byte-for-byte kryssplattform-paritet: Kotlin/Android-porten verifiseres kontinuerlig mot TS-referansen via delte test-vektorer i `test-vectors/`.
- Reelt revisjonsklar 4.x-kjerne som fundament (G0–G4 grønne); sanntid (V5.0) bygges oppå den låste stacken uten å røre kjernekrypto-revisjonen.
## Nå
Gated hardening — hard utgangsport per steg (full plan + funn: helsesjekk-artifact 2026-07-10, se log.md).
- [ ] **G0 — Stopp blødningen (P0):** ~~tilfeldig nonce per skriving i
storage-encrypted~~ (gjort 2026-08-13 — 9 nye tester, ingen migrering nødvendig,
Android var aldri rammet; se `logs/2026-08-13-gcm-nonce-reuse-fikset.md`.
**Ikke publisert** — Prism er pinnet på 4.12.0); observer `outDir→dist` +
republiser 4.11.2 + gitignore `dist-build/`; ~~pin `typescript` + typecheck i CI~~
(gjort 2026-08-14 — pinnet 7.0.2, `typecheck-all.ts` bruker lokal binær i
stedet for `bunx` latest, steg lagt i `test.yml`; `baseUrl` var allerede
borte og alle 26 pakker er grønne); fiks `bin`-omskriving + tarball-verifisering i publish.
Port: typecheck grønn i CI, nonce-negativtest grønn, `bun pm pack` inneholder
main-fil for alle pakker.
- [ ] **G1 — Revisjonsklar kjerne (P1):** peer-signeringsnøkkel per sesjon → symmetrisk safety-number; argon2id i backup-KDF (TS+Kotlin); hono ≥4.12.18; håndhevende §4 at-rest-tuklingstest; bestill ekstern krypto-review + pentest. Port: `alice.verify(bob, bob.getIdentityFingerprint())`===true, `bun audit` ren.
- [ ] **G2 — CI som mur:** biome + coverage-terskel + soak:smoke i CI; fjern/synk `.github/workflows`; publish kjører `prepublish:check`; bryt server↔inbox-server-syklus; rydd sdk/cli/files-deps; `dashboard: private`; pin bun. Port: PR blokkeres ved brudd, alle 8 examples grønne i CI.
## Neste
- **V4.13 — `@shade/vault`: server-side kryptert fillager.** Klient, ruter,
`MemoryVaultStore`, `SqliteVaultStore`, ruting i `standalone.ts` og
`docs/vault.md` er levert natt til 2026-08-14 (auto-mode); 16 + 7 tester
grønne, og kjeden er kjørt mot en ekte HTTP-server med et ekte workspace:
491 filer bit-identiske etter gjenoppretting, andre push sendte 0 KB.
Gjenstår: Postgres-store, rate limiting på vault-rutene, GC av objekter
ingen manifest lenger refererer — og deploy. Bestilt 2026-08-13 av
Scaffold-backupen, men er en generell mangel: Shade kan flytte filer mellom peers
(`@shade/files`) og lagre én liten profil-blob per konto (`/v1/blob/<slotId>`, V4.9),
men har ingen **alltid-på lagring av krypterte filer på relayen**. Hver konsument
docs/files.md nevner (Dispatch, Mail, Drive-style apps) trenger det samme, så det
hører i Shade og ikke i den enkelte appen.
- *Form:* innholdsadressert lager — klienten krypterer og navngir etter hash av
ciphertext, serveren ser aldri klartekst og trenger ingen forståelse av innholdet.
- *Historikk:* append-only oplog per samling gir versjonering og rollback gratis;
det er «mini github»-halvdelen av Scaffold-bestillingen.
- *Nøkler:* egen HKDF-gren ved siden av `shade-blob-*-v1`, så et vault-kompromiss
ikke rører profil-slotten. Recovery er kontocredentials, som blob-primitivet —
med det forbeholdet at gjenopprettingsstyrken da er passordstyrken (se G1s
argon2id-punkt, som løfter nettopp den).
- *Gated på G0:* lageret bygger på at-rest-stacken, og skal ikke reise seg på en
kjent GCM forbidden attack.
- **G3 — Paritetslås:** ShadeStream-vektorer (3-DH, wire 0x31–0x33) konsumert av begge sider; Kotlin-port av v4.11; utvid cross-vectors path-filter; `PARITY-VERSION`-sjekk; symmetriser alle 14 vektorfiler; Robolectric-tester for KeystoreStorage; E2E socket-interop TS↔Kotlin. Port: begge sider verifiserer alle vektorer, minor-bump uten Android-ack ⇒ rød CI. Vilkår for «production»-label på Android.
- **G4 — Docs = sannhet:** README/MIGRATION/CHANGELOG/CONTRIBUTING → 4.11.x; fiks brutte lenker + riktig remote; legg indexeddb/ShadeStream/broadcast i README+SHADE-BY-SCENARIO; Postgres at-rest quick-start + salt-livssyklus; per-pakke README for de 6 viktigste.
## Senere
- **G5 — V5.0 (Voice/Video):** implementasjon gated på G0–G4 grønne. `docs/V5.0.md` er i dag idéskisse (135 linjer) — modnes parallelt: wire-type-allokering reconciled mot 0x31–0x33 (nå tatt av ShadeStream), browser-SFrame-mekanisme (RTCRtpScriptTransform/Insertable Streams), ratchet-parametre, API-flate, milepæler, intern konsistens (60fps/1080p vs 30fps-test). Godkjennes i diskusjon (Stian + Fable) før koding.
- Versjonsdesigndokumentene bor fortsatt i `docs/V5.0.md` og `docs/archive/V*.md` — de er spesifikasjoner, ikke planfiler.

328
.scaffold/tasks.yaml Normal file
View File

@@ -0,0 +1,328 @@
# Scaffold kanban-kort. Skjema per kort:
# - id: kort-slug # unik i fila
# title: ""
# description: ""
# status: todo # todo | doing | done | blocked
# column: todo # speiler status (kanban-kolonne)
# priority: medium # low | medium | high
# assignee: human # cursor | claude | human | any
# context: "" # valgfri: hvorfor/hvor i koden
# acceptance: "" # valgfri: hva «ferdig» betyr
# todos:
# - text: "Underoppgave"
# done: false
# created: 2026-07-09
# updated: 2026-07-09
# completed_at: null
tasks:
# ─────────────── G0 — Stopp blødningen (P0-blockere) ───────────────
- id: g0-gcm-nonce-reuse
title: "Fjern AES-GCM nonce-gjenbruk i at-rest storage"
description: >
deriveNonce() lager deterministisk nonce fra (fieldKey, tabell, pk) uten
meldingsteller. saveSession re-krypterer sesjonsraden ved hvert ratchet-steg,
så to DB-snapshots deler (key, nonce) over ulik plaintext = GCM forbidden
attack. Systemisk: gjelder alle muterbare rader (config, prekey-state, trust).
status: done
column: done
priority: high
assignee: claude
context: >
Kallesteder kartlagt 2026-08-13: kun row-codec.ts:67 og :96 lager nonce,
og :83/:112 tvinger den ved lesing. deriveNonce() er eksportert fra både
index.ts og crypto.ts, så den beholdes (deprecated) framfor å brekke
konsumenter. VERIFISERT: aeadOpen leser alltid nonce fra blob-prefiks
(aead.ts:72) og expectedNonce er kun en valgfri ekstrasjekk — eksisterende
blobber åpnes derfor uendret, og ingen migreringssti trengs.
acceptance: "Tilfeldig nonce per skriving; aeadOpen leser nonce fra prefiks (dropp expectedNonce-tvang). Negativ test: to session-snapshots ved ulik message-count deler ikke (key, nonce). Test som åpner en blob forseglet med gammel deterministisk nonce. Alle eksisterende tester grønne."
todos:
- text: "row-codec.ts:67/96 → tilfeldig 12-byte nonce, lagret i blob-prefiks"
done: true
- text: "Dropp expectedNonce-argumentet i row-codec.ts:83/112"
done: true
- text: "Negativ regresjonstest for nonce-unikhet på tvers av re-saves"
done: true
- text: "Bakoverkompatibilitetstest: blob med gammel deterministisk nonce åpnes"
done: true
- text: "Marker deriveNonce deprecated (fortsatt eksportert fra index/crypto)"
done: true
- text: "IKKE PUBLISERT — Prism er pinnet på 4.12.0 og får fiksen først ved release"
done: false
created: 2026-07-10
updated: 2026-08-13
completed_at: 2026-08-13
- id: g0-observer-publish
title: "Fiks @shade/observer publiserings-artefakt (tomt på registry)"
description: >
outDir=dist-build avviker fra alle andre pakker (dist). Publish kjører
rm -rf dist og shipper files:[dist] → tarballen mangler JS, main peker på
manglende fil. dist-build/ er det untrackede avviket i git status.
status: todo
column: todo
priority: high
assignee: claude
context: "packages/shade-observer/tsconfig.json (outDir), scripts/publish-all.ts:109/186-227, dashboard copy-to-observer.ts-konflikten"
acceptance: "outDir=dist; dashboard-SPA-konflikt løst eksplisitt; 4.11.2 republisert med verifisert tarball; dist-build/ i .gitignore."
todos:
- text: "tsconfig outDir → dist"
done: false
- text: "Løs dashboard-SPA vs publish-eierskap til dist/"
done: false
- text: "gitignore dist-build/ + fjern restene"
done: false
- text: "Republiser observer 4.11.2"
done: false
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: g0-typecheck-gate
title: "Pin TypeScript, fiks baseUrl, kjør typecheck i CI"
description: >
TS ikke pinnet noe sted → bunx tsc henter 7.0.2 som fjernet baseUrl (brukt i
consumer-strict). typecheck exit 1 nå → prepublish:check rød. Gaten kjøres
heller aldri i CI.
status: done
column: done
priority: high
assignee: claude
context: "tests/consumer-strict/tsconfig.json:17 (baseUrl), scripts/typecheck-all.ts (bruker upinnet bunx tsc), .gitea/workflows/test.yml"
acceptance: "typescript pinnet som devDependency og brukt av typecheck-all.ts; baseUrl fjernet; bun run typecheck grønn og kjøres i .gitea/workflows/test.yml."
todos:
- text: "Pin typescript i root devDependencies"
done: true
- text: "typecheck-all.ts bruker lokal tsc, ikke bunx latest"
done: true
- text: "Fjern baseUrl fra consumer-strict tsconfig (var allerede borte)"
done: true
- text: "Legg typecheck-steg i test.yml"
done: true
created: 2026-07-10
updated: 2026-08-14
completed_at: 2026-08-14
- id: g0-cli-bin-tarball-verify
title: "Fiks cli bin-omskriving + tarball-verifisering i publish"
description: >
bin.shade=src/cli.ts omskrives ikke til dist/ (rewriteEntryPointsForDist
dekker kun main/types/exports). npm i -g @shade/cli gir brutt binær. Samme
rot som observer: publish mangler integritetssjekk.
status: todo
column: todo
priority: high
assignee: claude
context: "scripts/publish-all.ts:186-196 (rewriteEntryPointsForDist), packages/shade-cli/package.json (bin)"
acceptance: "bin omskrives til dist/; publish asserterer at bun pm pack inneholder main-filen for hver pakke; npm i -g @shade/cli kjørbar."
todos:
- text: "Utvid rewriteEntryPointsForDist til pkgJson.bin"
done: false
- text: "Legg tarball-innhold-assert (bun pm pack) i publish-flyten"
done: false
created: 2026-07-10
updated: 2026-07-10
completed_at: null
# ─────────────── G1 — Revisjonsklar kjerne (P1-sikkerhet) ───────────────
- id: g1-symmetric-safety-number
title: "Symmetrisk safety-number (lagre peer-signeringsnøkkel per sesjon)"
description: >
getRemoteFingerprint sender remoteIdentityKey (DH) som begge argumenter →
H(dh‖dh) ≠ peerens egen H(sig‖dh). OOB-verifisering kan aldri matche →
primær MITM-forsvar ikke-funksjonelt.
status: todo
column: todo
priority: high
assignee: claude
context: "packages/shade-core/src/session.ts:238-245, SDK verify()/markPeerVerified()/isPeerVerified() shade.ts:590/649/659"
acceptance: "Peer-signeringsnøkkel lagres per sesjon; kanonisk symmetrisk safety-number; kryssparts-test alice.verify(bob, bob.getIdentityFingerprint())===true uten MITM."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: g1-argon2id-backup
title: "Koble argon2id inn i backup-KDF (TS + Kotlin)"
description: >
Backup-passord-KDF er HKDF → offline brute-force i HKDF-hastighet.
argon2idAsync fra @noble/hashes er allerede dependency og brukes i
storage-encrypted. Kotlin har deriveMasterKeyArgon2id via Bouncy Castle.
status: todo
column: todo
priority: high
assignee: claude
context: "packages/shade-sdk/src/backup.ts:112-119, android/.../backup/BackupKdf.kt:27-36, referanse: storage-encrypted/src/crypto/kdf.ts:73-91"
acceptance: "Backup-KDF = argon2id på begge plattformer; backup-vektor re-pinnet; cross-platform-vektortest grønn."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: g1-hono-upgrade-audit
title: "Oppgrader hono ≥4.12.18 + bun audit-gate"
description: >
hono <4.12.18 har bodyLimit-bypass (svekker §7 64 KiB body-cap), path
traversal i serve-static, IP-restriction-bypass. Dev/test: happy-dom
(critical VM-escape), vite, postcss.
status: todo
column: todo
priority: high
assignee: claude
context: "hono i shade-server/inbox-server/transfer/observer/transport-bridge; bun.lock pinner nåværende"
acceptance: "hono ≥4.12.18 overalt; bun audit uten runtime-high/critical; §7 DoS-cap re-evaluert."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: g1-at-rest-tamper-test
title: "Håndhevende §4 at-rest-tuklingstest + bestill ekstern review"
description: >
SECURITY.md-cellen «§4 at-rest session DB» er «none yet» → ikke håndhevet.
Ekstern krypto-review + pentest står «Pending V4.0» og må ikke skli forbi V5.0.
status: todo
column: todo
priority: high
assignee: any
context: "SECURITY.md:18 (celler uten testlenke ikke håndhevet), THREAT-MODEL §4"
acceptance: "Negativ test som beviser at at-rest-tukling fanges (AAD/tag); SECURITY.md-celle lenket. Ekstern review + pentest bestilt."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
# ─────────────── G2/G3-hoder (nedbrytes når G0/G1 nærmer seg) ───────────────
- id: g2-ci-wall
title: "G2: CI som mur — lint/format/coverage/soak + rydd .github + arkitektur"
description: >
biome (lint+format), coverage-terskel (gulv=dagens), soak:smoke i CI;
fjern/synk stale .github/workflows (7 pakker→npmjs); publish kjører
prepublish:check; bryt server↔inbox-server-syklus; rydd sdk/cli/files ubrukte
deps; dashboard private; pin bun i workflows; kjør alle 8 examples (fiks
accept-race i shade-transfer/engine.ts:1140).
status: todo
column: todo
priority: medium
assignee: claude
context: "Se helsesjekk-artifact 2026-07-10 (P2-arkitektur + CI-hull)"
acceptance: "PR blokkeres ved brudd på typecheck·lint·format·test·coverage; alle 8 examples grønne i CI; ingen udeklarert/ubrukt hard dep."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: g3-parity-lock
title: "G3: Paritetslås — ShadeStream-vektorer + Kotlin v4.11 + interop"
description: >
ShadeStream (v4.11) har null vektorer selv på TS-siden. Kotlin stopper på
v4.10. Lag vektorer (3-DH deriveStreamRootKey, wire 0x31-0x33, frame
seal/open) konsumert av begge; port ShadeStream til Kotlin; utvid
cross-vectors path-filter; PARITY-VERSION-sjekk; symmetriser alle 14
vektorfiler; Robolectric-tester for KeystoreStorage; E2E socket-interop.
status: todo
column: todo
priority: medium
assignee: claude
context: "packages/shade-core/src/stream.ts, scripts/generate-vectors.ts, .gitea/workflows/cross-vectors.yml, android/shade-android-keystore (0 tester)"
acceptance: "Begge sider verifiserer alle 14 vektorer + ShadeStream; minor-bump uten Android-ack ⇒ rød CI; E2E TS-server↔Kotlin-klient over socket grønn."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: g4-docs-truth
title: "G4: Docs = sannhet — oppdater til 4.11.x, fiks lenker/remote/eksempler"
description: >
README/MIGRATION/CHANGELOG/CONTRIBUTING frosset på 4.0. Riktig remote er
Gitea (ikke GitHub). Legg indexeddb/ShadeStream/broadcast i README+
SHADE-BY-SCENARIO; Postgres at-rest quick-start + salt-livssyklus; per-pakke
README for sdk/core/files/streams/storage-encrypted/transport.
status: todo
column: todo
priority: medium
assignee: claude
context: "README.md, MIGRATION.md, CHANGELOG.md, CONTRIBUTING.md, docs/SHADE-BY-SCENARIO.md"
acceptance: "Én sannhetskilde for versjon; ingen brutte doc-lenker (CI link-check); hvert ECOSYSTEM-scenario har kjørende CI-testet eksempel-sti."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
# ─────────────── Åpne beslutninger (Stian avgjør i plandiskusjon) ───────────────
- id: decision-g3-vs-g4-order
title: "BESLUTNING: rekkefølge G3 (paritetslås) vs G4 (docs)"
description: >
Foreslått G3 før G4 fordi «production»-label på Android henger på paritet.
Men hvis Prism/Vyvern trenger docs-sannhet raskere, kan de bytte plass.
status: blocked
column: blocked
priority: medium
assignee: human
acceptance: "Rekkefølge bestemt; plan.md Neste-seksjon speiler valget."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: decision-external-review-timing
title: "BESLUTNING: bestille ekstern krypto-review/pentest nå eller etter G0+G1"
description: >
Parallelt nå (raskere kalender) vs etter G0+G1-fiksene (revisor ser korrekt
kjerne — nonce-fiks + symmetrisk safety-number inne). Fable heller mot fikse først.
status: blocked
column: blocked
priority: medium
assignee: human
context: "Kobles til g1-at-rest-tamper-test; SECURITY.md-celler «Pending V4.0»"
acceptance: "Tidspunkt bestemt; g1-at-rest-tamper-test oppdatert deretter."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
- id: decision-argon2id-migration
title: "BESLUTNING: argon2id backup-migreringsstrategi"
description: >
Re-pinning av backup-vektor betyr at eksisterende 4.x-backups må migreres.
Big-bang i 4.12 vs dobbelt-lesende overgangsvindu (les HKDF-legacy, skriv argon2id).
status: blocked
column: blocked
priority: medium
assignee: human
context: "Kobles til g1-argon2id-backup"
acceptance: "Migreringsstrategi valgt; g1-argon2id-backup-kortet får todos deretter."
created: 2026-07-10
updated: 2026-07-10
completed_at: null
# ─────────────── V4.13 — @shade/vault (server-side kryptert fillager) ───────────────
- id: vault-package
title: "@shade/vault — innholdsadressert kryptert fillager på relayen"
description: >
Shade kan flytte filer mellom peers (@shade/files) og lagre én liten
profil-blob per konto (/v1/blob/<slotId>), men har ingen alltid-på lagring
av krypterte filer. Uten den kan ingen Shade-app tilby backup, og ingen
klient kan lese data når peeren er avslått. Klienten krypterer og navngir
etter hash av ciphertext; serveren ser aldri klartekst.
status: doing
column: doing
priority: high
assignee: claude
context: >
Bestilt av Scaffold-backupen 2026-08-13, men generell: docs/files.md peker
selv på Dispatch/Mail/Drive-style apps som konsumenter. Egen HKDF-gren ved
siden av shade-blob-*-v1 i storage-encrypted/src/crypto/kdf.ts, så et
vault-kompromiss ikke rører profil-slotten. Rutes i shade-server/src/
standalone.ts ved siden av createBlobRoutes.
acceptance: >
Klient kan pushe/hente en samling krypterte objekter mot relayen uten at
serveren kan lese noe; oplog gir versjonsliste og henting av tidligere
versjon; ny enhet med kontocredentials rekonstruerer hele samlingen.
SHADE_VAULT_DB_PATH dokumentert med samme advarsel som SHADE_BLOB_DB_PATH.
todos:
- text: "HKDF-gren for vault-nøkler (slot, innhold, signatur)"
done: true
- text: "Innholdsadressert put/get med CAS-semantikk"
done: true
- text: "Append-only oplog per samling — versjonsliste og rollback"
done: true
- text: "Serverruter + persistenskonfigurasjon i standalone.ts"
done: true
- text: "docs/vault.md + README-rad i statustabellen"
done: false
- text: "IKKE DEPLOYET — krever SHADE_VAULT_DB_PATH i compose + Dokploy-runde"
done: false
created: 2026-08-13
updated: 2026-08-14
completed_at: null

View File

@@ -5,6 +5,20 @@ All notable changes to Shade are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [4.11.2] — 2026-07-10 — Files pull-mode startup hotfix
**`@shade/files`**
- Fix a construction race in HTTP pull-mode clients: an immediate streamed
read/write now awaits asynchronous stream-bridge registration instead of
treating the not-yet-created queue drainer as proof that the client is
inline-only.
- Add a deterministic integration test that holds incoming-transfer
registration, starts a 512 KiB write immediately, and verifies that it
remains pending until streaming is ready.
- Add `publish-all.ts --only <pkg>` so isolated package hotfixes do not publish
unrelated workspace packages.
## [4.11.0] — 2026-05-15 — Streaming Double-Ratchet sub-sessions
Answers Vyvern FR `shade-ws-streaming-ratchet.md` (the last Phase-2

4
CLAUDE.md Normal file
View File

@@ -0,0 +1,4 @@
## Planlegging
Planen for dette prosjektet bor i `.scaffold/`: `plan.md` er nåbildet (hold den kort,
bump `updated` i frontmatter ved endring), `log.md` er append-only historikk (nyeste øverst),
`tasks.yaml` er kanban. Ikke opprett planfiler andre steder.

123
bun.lock
View File

@@ -13,11 +13,12 @@
"devDependencies": {
"bun-types": "^1.3.11",
"fast-check": "^3.22.0",
"typescript": "7.0.2",
},
},
"packages/shade-cli": {
"name": "@shade/cli",
"version": "4.8.5",
"version": "4.13.0",
"bin": {
"shade": "src/cli.ts",
},
@@ -36,7 +37,7 @@
},
"packages/shade-core": {
"name": "@shade/core",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/observability": "workspace:*",
},
@@ -49,7 +50,7 @@
},
"packages/shade-crypto-web": {
"name": "@shade/crypto-web",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@noble/curves": "^2.0.1",
"@noble/hashes": "^2.0.1",
@@ -59,7 +60,7 @@
},
"packages/shade-dashboard": {
"name": "@shade/dashboard",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/widgets": "workspace:*",
"react": "^19.0.0",
@@ -74,7 +75,7 @@
},
"packages/shade-files": {
"name": "@shade/files",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
@@ -101,7 +102,7 @@
},
"packages/shade-inbox": {
"name": "@shade/inbox",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/proto": "workspace:*",
@@ -114,7 +115,7 @@
},
"packages/shade-inbox-server": {
"name": "@shade/inbox-server",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/observability": "workspace:*",
@@ -132,7 +133,7 @@
},
"packages/shade-key-transparency": {
"name": "@shade/key-transparency",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@noble/hashes": "^2.0.1",
"@shade/core": "workspace:*",
@@ -144,11 +145,11 @@
},
"packages/shade-keychain": {
"name": "@shade/keychain",
"version": "4.8.5",
"version": "4.13.0",
},
"packages/shade-observability": {
"name": "@shade/observability",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@noble/hashes": "^2.0.1",
},
@@ -166,7 +167,7 @@
},
"packages/shade-observer": {
"name": "@shade/observer",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/server": "workspace:*",
@@ -178,14 +179,14 @@
},
"packages/shade-proto": {
"name": "@shade/proto",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
},
},
"packages/shade-recovery": {
"name": "@shade/recovery",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
@@ -198,7 +199,7 @@
},
"packages/shade-sdk": {
"name": "@shade/sdk",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
@@ -228,12 +229,14 @@
},
"packages/shade-server": {
"name": "@shade/server",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/inbox-server": "workspace:*",
"@shade/key-transparency": "workspace:*",
"@shade/observability": "workspace:*",
"@shade/storage-sqlite": "workspace:*",
"@shade/vault": "workspace:*",
"hono": "^4.12.12",
},
"devDependencies": {
@@ -248,7 +251,7 @@
},
"packages/shade-storage-encrypted": {
"name": "@shade/storage-encrypted",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@noble/hashes": "^2.0.1",
"@shade/core": "workspace:*",
@@ -270,7 +273,7 @@
},
"packages/shade-storage-indexeddb": {
"name": "@shade/storage-indexeddb",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"idb": "^8.0.3",
@@ -282,7 +285,7 @@
},
"packages/shade-storage-postgres": {
"name": "@shade/storage-postgres",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/inbox-server": "workspace:*",
@@ -297,17 +300,20 @@
},
"packages/shade-storage-sqlite": {
"name": "@shade/storage-sqlite",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/inbox-server": "workspace:*",
"@shade/server": "workspace:*",
},
"devDependencies": {
"@shade/vault": "workspace:*",
},
},
"packages/shade-streams": {
"name": "@shade/streams",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@noble/hashes": "^2.0.1",
"@shade/core": "workspace:*",
@@ -319,7 +325,7 @@
},
"packages/shade-transfer": {
"name": "@shade/transfer",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
@@ -336,7 +342,7 @@
},
"packages/shade-transport": {
"name": "@shade/transport",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
@@ -347,7 +353,7 @@
},
"packages/shade-transport-bridge": {
"name": "@shade/transport-bridge",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/server": "workspace:*",
@@ -369,16 +375,29 @@
},
"packages/shade-transport-webrtc": {
"name": "@shade/transport-webrtc",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/core": "workspace:*",
"@shade/streams": "workspace:*",
"@shade/transfer": "workspace:*",
},
},
"packages/shade-vault": {
"name": "@shade/vault",
"version": "4.13.0",
"dependencies": {
"@noble/hashes": "^2.0.1",
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/observability": "workspace:*",
"@shade/server": "workspace:*",
"@shade/storage-encrypted": "workspace:*",
"hono": "^4.12.18",
},
},
"packages/shade-widgets": {
"name": "@shade/widgets",
"version": "4.8.5",
"version": "4.13.0",
"dependencies": {
"@shade/recovery": "workspace:*",
"@shade/sdk": "workspace:*",
@@ -603,6 +622,8 @@
"@shade/transport-webrtc": ["@shade/transport-webrtc@workspace:packages/shade-transport-webrtc"],
"@shade/vault": ["@shade/vault@workspace:packages/shade-vault"],
"@shade/widgets": ["@shade/widgets@workspace:packages/shade-widgets"],
"@types/babel__core": ["@types/babel__core@7.20.5", "", { "dependencies": { "@babel/parser": "^7.20.7", "@babel/types": "^7.20.7", "@types/babel__generator": "*", "@types/babel__template": "*", "@types/babel__traverse": "*" } }, "sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA=="],
@@ -621,6 +642,46 @@
"@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="],
"@typescript/typescript-aix-ppc64": ["@typescript/typescript-aix-ppc64@7.0.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ=="],
"@typescript/typescript-darwin-arm64": ["@typescript/typescript-darwin-arm64@7.0.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA=="],
"@typescript/typescript-darwin-x64": ["@typescript/typescript-darwin-x64@7.0.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA=="],
"@typescript/typescript-freebsd-arm64": ["@typescript/typescript-freebsd-arm64@7.0.2", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ=="],
"@typescript/typescript-freebsd-x64": ["@typescript/typescript-freebsd-x64@7.0.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw=="],
"@typescript/typescript-linux-arm": ["@typescript/typescript-linux-arm@7.0.2", "", { "os": "linux", "cpu": "arm" }, "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ=="],
"@typescript/typescript-linux-arm64": ["@typescript/typescript-linux-arm64@7.0.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ=="],
"@typescript/typescript-linux-loong64": ["@typescript/typescript-linux-loong64@7.0.2", "", { "os": "linux", "cpu": "none" }, "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ=="],
"@typescript/typescript-linux-mips64el": ["@typescript/typescript-linux-mips64el@7.0.2", "", { "os": "linux", "cpu": "none" }, "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA=="],
"@typescript/typescript-linux-ppc64": ["@typescript/typescript-linux-ppc64@7.0.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA=="],
"@typescript/typescript-linux-riscv64": ["@typescript/typescript-linux-riscv64@7.0.2", "", { "os": "linux", "cpu": "none" }, "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ=="],
"@typescript/typescript-linux-s390x": ["@typescript/typescript-linux-s390x@7.0.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw=="],
"@typescript/typescript-linux-x64": ["@typescript/typescript-linux-x64@7.0.2", "", { "os": "linux", "cpu": "x64" }, "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A=="],
"@typescript/typescript-netbsd-arm64": ["@typescript/typescript-netbsd-arm64@7.0.2", "", { "os": "none", "cpu": "arm64" }, "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA=="],
"@typescript/typescript-netbsd-x64": ["@typescript/typescript-netbsd-x64@7.0.2", "", { "os": "none", "cpu": "x64" }, "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA=="],
"@typescript/typescript-openbsd-arm64": ["@typescript/typescript-openbsd-arm64@7.0.2", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ=="],
"@typescript/typescript-openbsd-x64": ["@typescript/typescript-openbsd-x64@7.0.2", "", { "os": "openbsd", "cpu": "x64" }, "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg=="],
"@typescript/typescript-sunos-x64": ["@typescript/typescript-sunos-x64@7.0.2", "", { "os": "sunos", "cpu": "x64" }, "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g=="],
"@typescript/typescript-win32-arm64": ["@typescript/typescript-win32-arm64@7.0.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ=="],
"@typescript/typescript-win32-x64": ["@typescript/typescript-win32-x64@7.0.2", "", { "os": "win32", "cpu": "x64" }, "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g=="],
"@vitejs/plugin-react": ["@vitejs/plugin-react@4.7.0", "", { "dependencies": { "@babel/core": "^7.28.0", "@babel/plugin-transform-react-jsx-self": "^7.27.1", "@babel/plugin-transform-react-jsx-source": "^7.27.1", "@rolldown/pluginutils": "1.0.0-beta.27", "@types/babel__core": "^7.20.5", "react-refresh": "^0.17.0" }, "peerDependencies": { "vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0" } }, "sha512-gUu9hwfWvvEDBBmgtAowQCojwZmJ5mcLn3aufeCsitijs3+f2NsrPtlAWIR6OPiqljl96GVCUbLe0HyqIpVaoA=="],
"baseline-browser-mapping": ["baseline-browser-mapping@2.10.17", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-HdrkN8eVG2CXxeifv/VdJ4A4RSra1DTW8dc/hdxzhGHN8QePs6gKaWM9pHPcpCoxYZJuOZ8drHmbdpLHjCYjLA=="],
@@ -703,6 +764,8 @@
"tinyglobby": ["tinyglobby@0.2.16", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg=="],
"typescript": ["typescript@7.0.2", "", { "optionalDependencies": { "@typescript/typescript-aix-ppc64": "7.0.2", "@typescript/typescript-darwin-arm64": "7.0.2", "@typescript/typescript-darwin-x64": "7.0.2", "@typescript/typescript-freebsd-arm64": "7.0.2", "@typescript/typescript-freebsd-x64": "7.0.2", "@typescript/typescript-linux-arm": "7.0.2", "@typescript/typescript-linux-arm64": "7.0.2", "@typescript/typescript-linux-loong64": "7.0.2", "@typescript/typescript-linux-mips64el": "7.0.2", "@typescript/typescript-linux-ppc64": "7.0.2", "@typescript/typescript-linux-riscv64": "7.0.2", "@typescript/typescript-linux-s390x": "7.0.2", "@typescript/typescript-linux-x64": "7.0.2", "@typescript/typescript-netbsd-arm64": "7.0.2", "@typescript/typescript-netbsd-x64": "7.0.2", "@typescript/typescript-openbsd-arm64": "7.0.2", "@typescript/typescript-openbsd-x64": "7.0.2", "@typescript/typescript-sunos-x64": "7.0.2", "@typescript/typescript-win32-arm64": "7.0.2", "@typescript/typescript-win32-x64": "7.0.2" }, "bin": { "tsc": "bin/tsc" } }, "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA=="],
"undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
"update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="],
@@ -716,5 +779,15 @@
"yallist": ["yallist@3.1.1", "", {}, "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g=="],
"zod": ["zod@3.25.76", "", {}, "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ=="],
"@shade/inbox-server/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
"@shade/observer/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
"@shade/server/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
"@shade/transport-bridge/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
"@shade/vault/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
}
}

126
docs/vault.md Normal file
View File

@@ -0,0 +1,126 @@
# `@shade/vault` — server-side encrypted file store
V4.13. Where the blob primitive (V4.9) holds one small blob per account, a
vault holds a whole **collection** — a workspace, a mailbox, a drive folder —
as content-addressed objects plus an append-only log of manifests.
It is the piece Shade was missing to let an app offer backup, or to let a
client read data while the peer that owns it is switched off.
## The model
**Objects are content-addressed.** An object's name is the SHA-256 of its
*ciphertext*, so the relay can store, dedupe and verify without a key: it
recomputes the hash on upload and rejects anything mislabelled.
**A manifest names the collection.** Paths live inside the encrypted manifest,
never in object names — the relay must not learn that a user has a file called
`Projects/Divorce/plan.md`. The manifest is itself stored as an object.
**The log is append-only.** Each commit adds a row pointing at a manifest.
History, rollback and "what changed last Tuesday" all fall out of that.
## Keys
Three deterministic derivations off the account master key, in their own HKDF
branch:
```
vaultId = HKDF(masterKey, "shade-vault-id-v1:<app>")
contentKey = HKDF(masterKey, "shade-vault-content-v1:<app>")
sigSeed = HKDF(masterKey, "shade-vault-sig-v1:<app>")
```
Separate from `shade-blob-*-v1` on purpose: a vault key reads every file, a
profile-blob key reads a host list. Sharing a derivation would turn one
compromise into the other.
Because the derivation is deterministic, **recovery is credentials**. A fresh
device that can derive the profile master can derive these, and the relay hands
over ciphertext it has never been able to read. The flip side is that recovery
strength is password strength — see G1's argon2id item.
## Client
```ts
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { VaultClient, HttpVaultTransport, deriveVaultKeys } from '@shade/vault';
const crypto = new SubtleCryptoProvider();
const keys = await deriveVaultKeys(masterKey, 'scaffold');
const client = new VaultClient(crypto, keys, new HttpVaultTransport('https://vault.example'));
// Back up. Unchanged files are not re-uploaded.
const res = await client.push(files, Date.now(), 'nightly');
// → { seq: 4, uploaded: 2, reused: 489, bytesUploaded: 1_204 }
// Restore — the newest, or any earlier version.
const { manifest, files: restored } = await client.pull();
const old = await client.pull(2);
// History.
const log = await client.history();
```
`push` takes `at` rather than reading the clock, so a queued backup records
when it was *taken* rather than when it finally reached the relay.
## Server
```ts
import { createVaultRoutes } from '@shade/vault/server';
import { SqliteVaultStore } from '@shade/storage-sqlite';
app.route('/', createVaultRoutes(new SqliteVaultStore('/data/shade-vault.db'), crypto));
```
Routes:
| | |
|---|---|
| `GET /v1/vault/:id/log` | `{ entries, head }` |
| `HEAD /v1/vault/:id/object/:hash` | 200 / 404 — the have-check |
| `GET /v1/vault/:id/object/:hash` | raw ciphertext |
| `PUT /v1/vault/:id/object/:hash` | signed; verifies hash matches bytes |
| `POST /v1/vault/:id/commit` | signed; refuses a stale `seq` or a missing object |
Auth is TOFU-Ed25519, as for blobs: the first signed write pins a pubkey, and
every later write must be signed by it. The pubkey travels **inside** the
signed payload — appended afterwards it would not be covered by the signature,
and could be swapped in transit on the pinning write.
### Configuration
```
SHADE_VAULT_DB_PATH=/data/shade-vault.db # required — no in-memory fallback
SHADE_DISABLE_VAULT=1 # optional: off entirely
```
**There is deliberately no in-memory fallback.** The blob store has one, and on
2026-08-12 a routine redeploy destroyed every profile because the path was
never set in compose: the fallback worked perfectly right up until the
container was recreated, and nothing ever failed loudly. A vault that forgets
is worse than no vault, so an unset path means the routes are simply not
mounted, with a warning that says why.
### Limits
Per object 8 MiB, per vault 512 MiB, both overridable via `VaultRoutesOptions`.
## What the relay learns
Object sizes, how many objects there are, when commits happen, and that some
opaque 64-hex id is active. Not contents, not paths, not filenames, and not
which user a vaultId belongs to — the id is itself derived from a secret.
Deduplication is **within** a version chain, not across independent seals:
sealing uses a fresh nonce, so identical plaintext yields different names.
Convergent encryption would dedupe globally but leak which files two users
share, which is exactly what a blind store must not do.
## Status
Client, routes, `MemoryVaultStore` and `SqliteVaultStore` are implemented and
tested (16 unit + 7 store tests, plus an end-to-end suite against a real
workspace of 491 files). Not yet: a Postgres store, rate limiting on the vault
routes, and garbage collection of objects no manifest references any more.

View File

@@ -1,7 +1,9 @@
{
"name": "shade",
"private": true,
"workspaces": ["packages/*"],
"workspaces": [
"packages/*"
],
"scripts": {
"test": "bun test --recursive",
"test:core": "cd packages/shade-core && bun test",
@@ -25,7 +27,8 @@
},
"devDependencies": {
"bun-types": "^1.3.11",
"fast-check": "^3.22.0"
"fast-check": "^3.22.0",
"typescript": "7.0.2"
},
"dependencies": {
"@noble/curves": "^2.0.1",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/cli",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/cli.ts",
"bin": {

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/core",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -86,6 +86,14 @@ export class ShadeSessionManager {
* fully concurrent.
*/
private readonly peerOpChains = new Map<string, Promise<unknown>>();
/**
* Memoized `alias → canonical` lookups; `null` records "no alias" so a
* label without one costs a single storage read for the life of the
* process instead of one per encrypt/decrypt. Aliases change only via
* the mutators in this class, each of which clears the whole map — it
* holds at most one entry per peer, so rebuilding is cheap.
*/
private readonly aliasCache = new Map<string, string | null>();
constructor(
private readonly crypto: CryptoProvider,
@@ -151,6 +159,46 @@ export class ShadeSessionManager {
}
}
/**
* Map a session label onto the label its state actually lives under.
*
* `aliasSession` moves a session from a first-contact label (typically
* `fp:<hex>`) to the peer's canonical address, but the peer keeps
* sending under the old one. The persisted alias lets us follow that
* move across restarts — see the alias block in `StorageProvider`.
*
* A live session under `label` always wins: after a re-link the peer
* re-runs X3DH and a fresh session is established under the
* first-contact label again, and that new session — not the stale
* alias target — is the one that can decrypt what follows.
*
* Resolves one hop only. Aliases are always written pointing at a
* canonical label, so a chain would mean corrupt state; following it
* would risk a loop for no legitimate gain.
*
* MUST be called before taking the peer mutex: locking the alias while
* mutating the canonical session would let an aliased caller and a
* canonical caller ratchet the same state concurrently.
*/
private async resolveLabel(label: string): Promise<string> {
let canonical = this.aliasCache.get(label);
if (canonical === undefined) {
canonical = (await this.storage.getSessionAlias?.(label)) ?? null;
this.aliasCache.set(label, canonical);
}
if (canonical === null || canonical === label) return label;
if (await this.storage.getSession(label)) return label;
return canonical;
}
/**
* Public label resolution for callers that need to know where a
* peer's state lives (e.g. a transport routing an inbound frame).
*/
async resolveSessionLabel(label: string): Promise<string> {
return this.resolveLabel(label);
}
/** Get the event emitter (if observability is enabled) */
getEvents(): ShadeEventEmitter | undefined {
return this.events;
@@ -268,6 +316,11 @@ export class ShadeSessionManager {
*/
async resetSession(address: string): Promise<void> {
await this.storage.removeSession(address);
// Aliases pointing here are now dangling — drop them so the next
// first-contact frame resolves to its own label and establishes the
// fresh session this reset exists to force.
await this.storage.removeSessionAliasesFor?.(address);
this.aliasCache.clear();
this.events?.emit('session.removed', { address });
// Note: we keep the trusted identity; new session will verify against it.
}
@@ -336,6 +389,14 @@ export class ShadeSessionManager {
await this.storage.bumpPeerIdentityVersion(newLabel);
}
await this.storage.removeSession(oldLabel);
// Remember the move durably. The peer goes on sending under
// `oldLabel` — its transport derives the same first-contact label
// from our relay hint every time — so without this record every
// inbound frame after a restart resolves to a label whose session
// we just removed, and the peer (holding a valid session, never
// re-running X3DH) can never recover on its own.
await this.storage.saveSessionAlias?.(oldLabel, newLabel);
this.aliasCache.clear();
this.events?.emit('session.aliased', { oldLabel, newLabel });
}
@@ -349,6 +410,10 @@ export class ShadeSessionManager {
// because isTrustedIdentity() compares not retrieves; we just emit the new hash)
await this.storage.saveTrustedIdentity(address, newIdentityKey);
await this.storage.removeSession(address);
// The peer rotated identity — any alias into the old session is
// dangling and must not redirect frames meant for the new one.
await this.storage.removeSessionAliasesFor?.(address);
this.aliasCache.clear();
if (this.events) {
const newHash = await shortHash(this.crypto, newIdentityKey);
@@ -516,14 +581,19 @@ export class ShadeSessionManager {
* Subsequent messages are standard RatchetMessages.
*/
async encrypt(address: string, plaintext: string): Promise<ShadeEnvelope> {
return this.withSpan('encrypt', address, async () => {
const session = await this.storage.getSession(address);
if (!session) throw new NoSessionError(address);
// Follow a persisted alias so a caller still holding a first-contact
// label reaches the session that alias moved to — without this, a
// restarted host can decrypt a peer's frames but not reply to them.
// Resolved before the mutex so the lock lands on the canonical label.
const target = await this.resolveLabel(address);
return this.withSpan('encrypt', target, async () => {
const session = await this.storage.getSession(target);
if (!session) throw new NoSessionError(target);
const ratchetMsg = await ratchetEncrypt(this.crypto, session, enc.encode(plaintext));
this.events?.emit('message.encrypted', {
address,
address: target,
counter: ratchetMsg.counter,
ciphertextSize: ratchetMsg.ciphertext.length,
});
@@ -532,7 +602,7 @@ export class ShadeSessionManager {
const x3dh = (session as any).__x3dh;
if (x3dh) {
delete (session as any).__x3dh;
await this.storage.saveSession(address, session);
await this.storage.saveSession(target, session);
const preKeyMsg: PreKeyMessage = {
registrationId: x3dh.registrationId,
@@ -546,16 +616,16 @@ export class ShadeSessionManager {
type: 'prekey',
content: preKeyMsg,
timestamp: Date.now(),
senderAddress: address,
senderAddress: target,
};
}
await this.storage.saveSession(address, session);
await this.storage.saveSession(target, session);
return {
type: 'ratchet',
content: ratchetMsg,
timestamp: Date.now(),
senderAddress: address,
senderAddress: target,
};
});
}
@@ -564,11 +634,17 @@ export class ShadeSessionManager {
* Decrypt a message from a peer. Handles both PreKeyMessage and RatchetMessage.
*/
async decrypt(address: string, envelope: ShadeEnvelope): Promise<string> {
return this.withSpan('decrypt', address, async () => {
// A prekey envelope carries its own X3DH material and establishes a
// fresh session, which must land under the label it arrived on —
// that is exactly what a re-link looks like. Only ratchet envelopes,
// which need state that already exists, follow an alias.
const target =
envelope.type === 'prekey' ? address : await this.resolveLabel(address);
return this.withSpan('decrypt', target, async () => {
if (envelope.type === 'prekey') {
return this.decryptPreKeyMessage(address, envelope.content as PreKeyMessage);
return this.decryptPreKeyMessage(target, envelope.content as PreKeyMessage);
}
return this.decryptRatchetMessage(address, envelope.content as RatchetMessage);
return this.decryptRatchetMessage(target, envelope.content as RatchetMessage);
});
}

View File

@@ -165,6 +165,36 @@ export interface StorageProvider {
/** Remove session for a peer */
removeSession(address: string): Promise<void>;
// ─── Session label aliases (V4.12) ────────────────────────
//
// First contact forces the receiver to label a session by the only
// sender hint the relay surfaces — an 8-byte signing-key fingerprint
// (`fp:<hex>`). A later in-band announcement reveals the peer's
// canonical address and `aliasSession` moves the session there.
//
// The peer, however, keeps sending under whatever label its own
// transport derives — which for a fingerprint-hinted relay is still
// `fp:<hex>`. Before V4.12 that binding lived only in the consumer's
// memory: after a restart the alias was gone, inbound ratchet frames
// resolved to `fp:<hex>`, found no session there, and failed forever
// (the peer holds a valid session so it never re-runs X3DH).
//
// Persisting the alias makes the binding survive restarts, so
// `getSession` can follow it. Optional so third-party storage
// implementations keep compiling — they simply lose alias recovery.
/**
* Record that `alias` names the same peer session as `canonical`.
* Idempotent upsert on `alias`.
*/
saveSessionAlias?(alias: string, canonical: string): Promise<void>;
/** Resolve an alias to its canonical label (null when unaliased). */
getSessionAlias?(alias: string): Promise<string | null>;
/** Drop every alias pointing at `canonical` (session teardown). */
removeSessionAliasesFor?(canonical: string): Promise<void>;
/** Check if we trust a remote identity key (for TOFU or pinned keys) */
isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean>;

View File

@@ -0,0 +1,165 @@
import { describe, test, expect, beforeEach } from 'bun:test';
import { SubtleCryptoProvider, MemoryStorage } from '@shade/crypto-web';
import { ShadeSessionManager } from '../src/index.js';
const crypto = new SubtleCryptoProvider();
/**
* Durable session-label aliases (V4.12).
*
* THE BUG THIS FILE EXISTS TO KILL — diagnosed live in Prism:
*
* A phone pairs with a host. First contact forces the host to label
* the session by the only sender hint the relay surfaces, an 8-byte
* signing-key fingerprint (`fp:<hex>`). The pair handshake then
* announces the phone's canonical address and the host calls
* `aliasSession(fp:<hex> → device:<addr>)`, which moved the session
* on disk and dropped the binding.
*
* The phone, however, keeps sending under `fp:<hex>` — its transport
* derives the same label from the same relay hint every time. While
* the host process lived, an in-memory map papered over the gap.
* After a restart that map was empty, every inbound ratchet frame
* resolved to `fp:<hex>`, found no session, and failed. The phone
* held a perfectly valid session so it never re-ran X3DH — meaning
* the failure was permanent and self-inflicted, not transient.
*
* Observed as `No session for address: fp:579c3b335d66e2c0` on every
* receive for three days, with the phone's RPCs timing out forever.
*
* The fix: `aliasSession` persists the binding, and session lookup
* follows it. These tests pin the restart behaviour specifically —
* a same-process test cannot fail the way production did.
*/
describe('session label aliases', () => {
let alice: ShadeSessionManager;
let bob: ShadeSessionManager;
let aliceStorage: MemoryStorage;
let bobStorage: MemoryStorage;
/** The first-contact label Alice is forced to use for Bob. */
const FP = 'fp:579c3b335d66e2c0';
beforeEach(async () => {
aliceStorage = new MemoryStorage();
bobStorage = new MemoryStorage();
alice = new ShadeSessionManager(crypto, aliceStorage);
bob = new ShadeSessionManager(crypto, bobStorage);
await alice.initialize();
await bob.initialize();
});
/**
* Bob initiates X3DH against Alice, exactly like a phone reaching a
* host it just scanned. Returns Bob's first (prekey) envelope.
*/
async function bobInitiates(target: ShadeSessionManager, initiator: ShadeSessionManager) {
const otpks = await target.generateOneTimePreKeys(10);
const bundle = await target.createPreKeyBundle();
const otpk = otpks[0]!;
bundle.oneTimePreKey = { keyId: otpk.keyId, publicKey: otpk.keyPair.publicKey };
await initiator.initSessionFromBundle('alice', bundle);
}
/** Simulate a host restart: fresh manager, same durable storage. */
async function restartAlice(): Promise<ShadeSessionManager> {
const revived = new ShadeSessionManager(crypto, aliceStorage);
await revived.initialize();
return revived;
}
test('an aliased session still decrypts under the old label after a restart', async () => {
await bobInitiates(alice, bob);
// First contact lands under the fingerprint label.
const env1 = await bob.encrypt('alice', 'hello, my address is bob');
expect(await alice.decrypt(FP, env1)).toBe('hello, my address is bob');
// Alice canonicalizes to Bob's announced address.
await alice.aliasSession(FP, 'bob');
// The host restarts. Storage survives; every in-memory map does not.
const alice2 = await restartAlice();
// Bob has a valid session and keeps sending under the same label he
// always has. Before the fix this threw NoSessionError forever.
const env2 = await bob.encrypt('alice', 'still here after restart');
expect(await alice2.decrypt(FP, env2)).toBe('still here after restart');
});
test('the host can reply under the old label after a restart', async () => {
await bobInitiates(alice, bob);
const env1 = await bob.encrypt('alice', 'hi');
await alice.decrypt(FP, env1);
await alice.aliasSession(FP, 'bob');
const alice2 = await restartAlice();
// Decrypting is only half of it — a host that cannot encrypt back
// leaves every RPC hanging just the same.
const reply = await alice2.encrypt(FP, 'reply from the host');
expect(await bob.decrypt('alice', reply)).toBe('reply from the host');
});
test('a live session under the label wins over an alias (re-link)', async () => {
await bobInitiates(alice, bob);
const env1 = await bob.encrypt('alice', 'first pairing');
await alice.decrypt(FP, env1);
await alice.aliasSession(FP, 'bob');
const alice2 = await restartAlice();
// Bob reinstalls: brand-new identity, same relay fingerprint label.
const bob2Storage = new MemoryStorage();
const bob2 = new ShadeSessionManager(crypto, bob2Storage);
await bob2.initialize();
await bobInitiates(alice2, bob2);
// The prekey envelope must establish a FRESH session under FP rather
// than being redirected into the stale aliased one.
const fresh1 = await bob2.encrypt('alice', 'fresh contact');
expect(await alice2.decrypt(FP, fresh1)).toBe('fresh contact');
// And subsequent ratchet frames must keep using that new session.
const fresh2 = await bob2.encrypt('alice', 'second message');
expect(await alice2.decrypt(FP, fresh2)).toBe('second message');
});
test('resolveSessionLabel reports where the state actually lives', async () => {
await bobInitiates(alice, bob);
const env1 = await bob.encrypt('alice', 'hi');
await alice.decrypt(FP, env1);
expect(await alice.resolveSessionLabel(FP)).toBe(FP);
await alice.aliasSession(FP, 'bob');
const alice2 = await restartAlice();
expect(await alice2.resolveSessionLabel(FP)).toBe('bob');
// An unaliased label resolves to itself.
expect(await alice2.resolveSessionLabel('carol')).toBe('carol');
});
test('resetSession drops aliases pointing at the cleared session', async () => {
await bobInitiates(alice, bob);
const env1 = await bob.encrypt('alice', 'hi');
await alice.decrypt(FP, env1);
await alice.aliasSession(FP, 'bob');
expect(await aliceStorage.getSessionAlias(FP)).toBe('bob');
await alice.resetSession('bob');
// A dangling alias would redirect the next first-contact frame into
// a session that no longer exists, defeating the reset.
expect(await aliceStorage.getSessionAlias(FP)).toBeNull();
expect(await alice.resolveSessionLabel(FP)).toBe(FP);
});
test('aliasing persists the binding to storage', async () => {
await bobInitiates(alice, bob);
const env1 = await bob.encrypt('alice', 'hi');
await alice.decrypt(FP, env1);
expect(await aliceStorage.getSessionAlias(FP)).toBeNull();
await alice.aliasSession(FP, 'bob');
expect(await aliceStorage.getSessionAlias(FP)).toBe('bob');
});
});

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/crypto-web",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -15,6 +15,8 @@ export class MemoryStorage implements StorageProvider {
private signedPreKeys = new Map<number, SignedPreKey>();
private oneTimePreKeys = new Map<number, OneTimePreKey>();
private sessions = new Map<string, SessionState>();
/** alias → canonical session label (V4.12). */
private sessionAliases = new Map<string, string>();
private trustedIdentities = new Map<string, Uint8Array>();
private retiredIdentities: RetiredIdentity[] = [];
@@ -82,6 +84,22 @@ export class MemoryStorage implements StorageProvider {
this.sessions.delete(address);
}
// ─── Session label aliases ────────────────────────────────
async getSessionAlias(alias: string): Promise<string | null> {
return this.sessionAliases.get(alias) ?? null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
this.sessionAliases.set(alias, canonical);
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
for (const [alias, target] of this.sessionAliases) {
if (target === canonical) this.sessionAliases.delete(alias);
}
}
// ─── Trust ────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/dashboard",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"scripts": {
"dev": "vite",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/files",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -444,8 +444,12 @@ export function createFilesHttpClient(
};
return out;
}
// Streamed read — only supported when the queue drainer is wired.
if (drainer === null) {
// Streamed read — only supported when pull-mode was configured.
// `drainer` is assigned asynchronously after the streams bridge has
// subscribed, so checking it here races client construction. The
// promise itself is the synchronous configuration signal; awaiting it
// also guarantees the drainer has been installed by its `.then()`.
if (streamsBridgePromise === null) {
throw new InternalFileError(
`http RPC client received a streamed read (size ${wire.size}) but is in inline-only mode. Pass { outboundQueueUrl, transferBaseUrl } when constructing the client to enable streamed reads.`,
);
@@ -507,8 +511,11 @@ export function createFilesHttpClient(
);
}
// Streamed write — requires the queue drainer + streams-bridge.
if (drainer === null) {
// Streamed write — requires pull-mode configuration. Do not inspect
// `drainer` as a readiness flag: bridge registration is asynchronous,
// and an immediate large write must wait for it instead of being
// misclassified as an inline-only client.
if (streamsBridgePromise === null) {
throw new ConflictError(
`http RPC client supports inline writes only (≤ ${INLINE_THRESHOLD} bytes). The supplied input was promoted to streams (size ${decision.size ?? 'unknown'}). Pass { outboundQueueUrl, transferBaseUrl } to enable streamed writes.`,
);

View File

@@ -7,8 +7,10 @@ import {
} from '@shade/server';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { Hono } from 'hono';
import { createFilesHttpClient, type FileEntry } from '../../src/index.js';
const crypto = new SubtleCryptoProvider();
type ShadeInstance = Awaited<ReturnType<typeof createShade>>;
/**
* Stand up the full pull-mode rig:
@@ -20,7 +22,8 @@ const crypto = new SubtleCryptoProvider();
* no inbound listener, streams supported via long-poll.
*/
async function setupPullRig(opts: {
bobHandler: Parameters<NonNullable<Awaited<ReturnType<typeof createShade>>['files']>['serve']>[0];
bobHandler: Parameters<NonNullable<ShadeInstance['files']>['serve']>[0];
wrapClientShade?: (shade: ShadeInstance) => Parameters<typeof createFilesHttpClient>[0];
}) {
const prekey = createPrekeyServer({
crypto,
@@ -46,7 +49,7 @@ async function setupPullRig(opts: {
const bobServer = Bun.serve({ port: 0, fetch: app.fetch });
const baseUrl = `http://localhost:${bobServer.port}`;
const fs = alice.files.httpClient('bob', {
const fs = createFilesHttpClient(opts.wrapClientShade?.(alice) ?? alice, 'bob', {
rpcUrl: `${baseUrl}/rpc`,
outboundQueueUrl: `${baseUrl}/queue`,
transferBaseUrl: baseUrl,
@@ -70,6 +73,83 @@ async function setupPullRig(opts: {
}
describe('@shade/files HTTP RPC — pull-mode streams', () => {
test('immediate streamed write waits for asynchronous stream-bridge registration', async () => {
const payload = new Uint8Array(512 * 1024);
for (let i = 0; i < payload.length; i++) payload[i] = (i * 53) & 0xff;
let releaseRegistration!: () => void;
const registrationGate = new Promise<void>((resolve) => {
releaseRegistration = resolve;
});
let registrationStarted!: () => void;
const started = new Promise<void>((resolve) => {
registrationStarted = resolve;
});
const rig = await setupPullRig({
wrapClientShade: (alice) =>
new Proxy(alice, {
get(target, property) {
if (property === 'onIncomingTransfer') {
return async (handler: Parameters<typeof target.onIncomingTransfer>[0]) => {
registrationStarted();
await registrationGate;
return await target.onIncomingTransfer(handler);
};
}
const value = Reflect.get(target, property, target) as unknown;
return typeof value === 'function' ? value.bind(target) : value;
},
}),
bobHandler: {
write: async (ctx) => {
const content = ctx.args.content;
if (content.kind !== 'streams') throw new Error('expected streamed content');
const reader = content.stream.getReader();
let received = 0;
while (true) {
const { value, done } = await reader.read();
if (done) break;
received += value?.byteLength ?? 0;
}
reader.releaseLock();
await content.sha256;
const entry: FileEntry = {
name: 'immediate.bin',
kind: 'file',
size: received,
mtime: Date.now(),
metadata: {},
};
return { entry };
},
},
});
try {
let settled = false;
const write = rig.fs.write('/immediate.bin', payload);
void write.then(
() => {
settled = true;
},
() => {
settled = true;
},
);
await started;
await Bun.sleep(10);
expect(settled).toBe(false);
releaseRegistration();
const result = await write;
expect(result.entry.size).toBe(payload.byteLength);
} finally {
releaseRegistration();
await rig.teardown();
}
}, 15_000);
test('streamed read (4 MiB) via long-poll queue', async () => {
const payload = new Uint8Array(4 * 1024 * 1024);
for (let i = 0; i < payload.length; i++) payload[i] = (i * 97) & 0xff;

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/inbox-server",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/inbox",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/key-transparency",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/keychain",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/observability",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/observer",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/proto",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/recovery",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/sdk",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -701,6 +701,22 @@ export class Shade {
this.decryptChains.delete(oldLabel);
}
/**
* Resolve a session label to the label its state actually lives under,
* following any alias left behind by `aliasSession`.
*
* Transports need this to route by the canonical address after a
* restart, when the only sender hint they hold is the first-contact
* `fp:<hex>` label. `encrypt`/`decrypt` resolve internally — this is
* for callers that must know the address itself.
*
* V4.12 — durable session-label aliases.
*/
async resolveSessionLabel(label: string): Promise<string> {
if (!this.initialized) throw new Error('Not initialized');
return this.manager.resolveSessionLabel(label);
}
/**
* Accept a peer's rotated identity. Bumps the per-peer identity-version
* counter so any earlier verification automatically goes stale, then

View File

@@ -162,10 +162,15 @@ describe('createShade — happy path', () => {
const env3 = await alice.send('bob', 'reply 2');
expect(await bob.receive('alice', env3)).toBe('reply 2');
// The old fp-label has no session — receive under it would now
// fail. (We don't assert the error shape, only that the label is
// gone.)
await expect(alice.receive(fpLabel, env3)).rejects.toThrow();
// V4.12: the fp-label is no longer a dead end. `aliasSession` leaves
// a durable binding behind, so a peer that keeps sending under the
// first-contact label — which is exactly what a fingerprint-hinted
// relay makes it do — still reaches the canonicalized session.
// See `shade-core/tests/session-aliases.test.ts` for the restart
// behaviour this binding exists to protect.
expect(await alice.resolveSessionLabel(fpLabel)).toBe('bob');
const env4 = await bob.send('alice', 'reply 3');
expect(await alice.receive(fpLabel, env4)).toBe('reply 3');
});
test('aliasSession refuses to overwrite an existing session', async () => {

View File

@@ -9,10 +9,15 @@ COPY package.json bun.lock tsconfig.json ./
# Copy all packages the server + observer + dashboard need
COPY packages ./packages
RUN bun install --frozen-lockfile
# BuildKit gives each RUN a 1024 soft file-descriptor limit while the hard
# limit is ~1M. bun extracts tarballs in parallel, so a large package can
# exhaust the descriptors and the install dies with "Fail extracting tarball"
# — reproducible in a build, invisible outside one because `docker run`
# inherits a far higher limit. Raise the soft limit to what the host allows.
RUN ulimit -n "$(ulimit -Hn)" && bun install --frozen-lockfile
# Build the dashboard SPA → dist/, then copy to observer's dist/
RUN cd packages/shade-dashboard && bun run build
RUN ulimit -n "$(ulimit -Hn)" && cd packages/shade-dashboard && bun run build
# ─── Production stage ───────────────────────────────────────
FROM oven/bun:1-alpine

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/server",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
@@ -9,6 +9,8 @@
"@shade/inbox-server": "workspace:*",
"@shade/key-transparency": "workspace:*",
"@shade/observability": "workspace:*",
"@shade/storage-sqlite": "workspace:*",
"@shade/vault": "workspace:*",
"hono": "^4.12.12"
},
"optionalDependencies": {

View File

@@ -83,6 +83,30 @@ async function createInboxStore(): Promise<InboxStore & { close?: () => void | P
* also opt the blob store *off* entirely via `SHADE_DISABLE_BLOB=1` —
* useful for relays that only want the inbox surface.
*/
/**
* Build the vault store, or `null` when none is configured.
*
* Deliberately NOT falling back to an in-memory store the way the blob
* primitive does. A vault holds the user's actual files: an in-memory one
* accepts every write, serves every read, and loses the lot on the next
* container recreate — a backup that reports success and is not there. The
* blob store's silent fallback destroyed Prism's profile on 2026-08-12
* precisely because nothing failed loudly. Here, no path means no vault.
*/
async function createVaultStore(): Promise<import('@shade/vault').VaultStore | null> {
const sqlitePath = process.env.SHADE_VAULT_DB_PATH;
if (!sqlitePath) {
logger.warn(
'Vault not enabled — set SHADE_VAULT_DB_PATH to a path on a persistent volume. ' +
'There is no in-memory fallback on purpose: a vault that forgets is not a backup.',
);
return null;
}
const { SqliteVaultStore } = await import('@shade/storage-sqlite');
logger.info('Using SQLite vault store', { path: sqlitePath });
return new SqliteVaultStore(sqlitePath);
}
async function createBlobStore(): Promise<BlobStore & { close?: () => void | Promise<void> }> {
const sqlitePath = process.env.SHADE_BLOB_DB_PATH;
const pgUrl = process.env.SHADE_BLOB_PG_URL ?? process.env.SHADE_PREKEY_PG_URL;
@@ -255,6 +279,21 @@ if (blobDisabled) {
logger.info('Blob primitive enabled', { route: '/v1/blob/:slotId' });
}
// V4.13 — vault: server-side encrypted file store. Where the blob primitive
// holds one small blob per account, a vault holds a whole collection with an
// append-only version log. Off unless a store is configured, because a vault
// that silently lives in RAM is worse than no vault at all — see the warning
// on SHADE_VAULT_DB_PATH below.
const vaultDisabled = process.env.SHADE_DISABLE_VAULT === '1';
const vaultStore = vaultDisabled ? null : await createVaultStore();
if (vaultDisabled) {
logger.info('Vault disabled (SHADE_DISABLE_VAULT=1)');
} else if (vaultStore) {
const { createVaultRoutes } = await import('@shade/vault/server');
app.route('/', createVaultRoutes(vaultStore, crypto));
logger.info('Vault enabled', { route: '/v1/vault/:vaultId' });
}
// ─── Optional: Observer + Dashboard ──────────────────────────
const observerToken = process.env.SHADE_OBSERVER_TOKEN;

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/storage-encrypted",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -34,12 +34,16 @@ export {
deriveBlobSlotId,
deriveBlobKey,
deriveBlobSigningSeed,
deriveVaultId,
deriveVaultContentKey,
deriveVaultSigningSeed,
} from './crypto/kdf.js';
export {
AEAD_NONCE_LEN,
AEAD_TAG_LEN,
aeadSeal,
aeadOpen,
randomNonce,
} from './crypto/aead.js';
export {
COL,

View File

@@ -33,9 +33,32 @@ async function importKey(key: Uint8Array, usages: WebCryptoKeyUsage[]): Promise<
}
/**
* Encrypt a plaintext blob with the given key and a deterministic nonce.
* Returns `nonce || ct||tag` as a single Uint8Array suitable for direct
* BLOB storage.
* A fresh random 12-byte nonce.
*
* AES-GCM tolerates key reuse but not (key, nonce) reuse: two ciphertexts
* sealed under the same pair leak the XOR of their plaintexts and, worse,
* the GHASH authentication subkey — the "forbidden attack", which yields
* tag forgery for that key. Mutable rows are re-sealed on every change
* (`saveSession` runs on each ratchet step), so a nonce derived from row
* identity alone repeats by construction.
*
* Random beats a counter here because the codec is shared across SQLite
* and Postgres and has no durable per-row write counter to lean on. At
* 96 bits, collision probability stays negligible far past any realistic
* number of re-saves.
*/
export function randomNonce(): Uint8Array {
return globalThis.crypto.getRandomValues(new Uint8Array(NONCE_LEN));
}
/**
* Encrypt a plaintext blob with the given key and nonce. Returns
* `nonce || ct||tag` as a single Uint8Array suitable for direct BLOB
* storage.
*
* Callers must pass a nonce that is fresh for this key — see
* {@link randomNonce}. Never derive one deterministically from row
* identity for a row that can be written more than once.
*/
export async function aeadSeal(
key: Uint8Array,
@@ -58,9 +81,19 @@ export async function aeadSeal(
}
/**
* Decrypt a `nonce || ct||tag` blob. The expected nonce is verified against
* the prefix to detect tampering before we even reach the AEAD; if the
* caller passes a `expectedNonce`, mismatch throws before SubtleCrypto runs.
* Decrypt a `nonce || ct||tag` blob.
*
* The nonce always comes from the blob prefix, which is what makes the
* move to random nonces backward compatible: a blob sealed under the old
* deterministic scheme opens unchanged, because its nonce was already
* stored the same way.
*
* @param expectedNonce **Deprecated.** Only meaningful when the nonce is
* a pure function of row identity, which is exactly the pattern that made
* (key, nonce) repeat. It adds no tamper detection that the AEAD tag and
* the (table, column, pk) AAD do not already provide — a flipped bit in
* the nonce fails the tag. Retained so existing callers keep compiling;
* pass nothing.
*/
export async function aeadOpen(
key: Uint8Array,

View File

@@ -106,12 +106,20 @@ export function deriveFieldKey(storageKey: Uint8Array, table: string, column: st
}
/**
* Derive a deterministic 12-byte AEAD nonce from a row key (typically the
* field key) plus (table, pk) binding. With per-field keys, deterministic
* nonces are safe because each (key, plaintext) pair appears at most once
* — re-saving the same row reuses the (nonce, key) pair only because the
* plaintext also changes (chain ratchet, prekey state, etc.). The AAD
* also binds (table, column, pk) so swapping is rejected on decrypt.
* Derive a deterministic 12-byte AEAD nonce from a row key plus (table, pk).
*
* @deprecated Unsafe for anything that can be written twice, which is every
* mutable row. The original rationale held that re-saving a row was fine
* "because the plaintext also changes" — but that is precisely the case
* AES-GCM forbids: reusing (key, nonce) across *different* plaintexts leaks
* their XOR and the GHASH subkey (the forbidden attack), enabling tag
* forgery. `saveSession` re-seals on every ratchet step, so the pair
* repeated by construction. Use {@link randomNonce} from `./aead.js`.
*
* Kept exported (it is part of the published surface via `index.ts` and
* `crypto.ts`) so consumers keep compiling. No caller inside this package
* uses it any more. Reading old data needs no migration: `aeadOpen` has
* always taken the nonce from the blob prefix.
*/
export function deriveNonce(rowKey: Uint8Array, table: string, pk: string): Uint8Array {
const out = hkdfDerive(rowKey, `shade-row-nonce-v1:${table}:${pk}`, 12);
@@ -161,3 +169,42 @@ export function deriveBlobKey(masterKey: Uint8Array, app: string): Uint8Array {
export function deriveBlobSigningSeed(masterKey: Uint8Array, app: string): Uint8Array {
return hkdfDerive(masterKey, `shade-blob-sig-v1:${app}`, 32);
}
// ─── V4.13 — vault (server-side encrypted file store) ──────────
//
// The blob primitive above holds ONE small blob per slot. A vault holds
// a whole collection — a project workspace, a mailbox, a drive folder —
// as content-addressed objects plus an append-only log of manifests.
//
// Its own HKDF branch rather than a reuse of `shade-blob-*-v1`, for two
// reasons. A vault key is used on far more ciphertext than a profile
// blob ever is, and the two have different blast radii: whoever holds
// the vault key can read every file, while the profile key exposes only
// the host list. Separate labels mean compromising one cannot be turned
// into the other, even though both hang off the same master.
//
// vaultId = HKDF(masterKey, info=`shade-vault-id-v1:${app}`)
// contentKey = HKDF(masterKey, info=`shade-vault-content-v1:${app}`)
// sigSeed = HKDF(masterKey, info=`shade-vault-sig-v1:${app}`)
/** 32-byte vault identifier. Opaque to the relay, like a slotId. */
export function deriveVaultId(masterKey: Uint8Array, app: string): Uint8Array {
return hkdfDerive(masterKey, `shade-vault-id-v1:${app}`, 32);
}
/**
* AEAD key for the vault's objects and manifests.
*
* One key for the whole collection, not one per file: per-file keys would
* have to be stored somewhere, and that somewhere would be a manifest
* encrypted under a collection key anyway. Object AAD binds the content
* hash, so a ciphertext cannot be moved to another object's name.
*/
export function deriveVaultContentKey(masterKey: Uint8Array, app: string): Uint8Array {
return hkdfDerive(masterKey, `shade-vault-content-v1:${app}`, 32);
}
/** 32-byte Ed25519 signing seed; TOFU-pinned by the relay on first write. */
export function deriveVaultSigningSeed(masterKey: Uint8Array, app: string): Uint8Array {
return hkdfDerive(masterKey, `shade-vault-sig-v1:${app}`, 32);
}

View File

@@ -22,8 +22,8 @@ import {
serializeSignedPreKey, deserializeSignedPreKey,
toBase64, fromBase64,
} from '@shade/core';
import { aeadOpen, aeadSeal } from './aead.js';
import { buildAad, deriveNonce } from './kdf.js';
import { aeadOpen, aeadSeal, randomNonce } from './aead.js';
import { buildAad } from './kdf.js';
import type { KeyManager } from './key-manager.js';
const TEXT_ENCODER = new TextEncoder();
@@ -64,9 +64,8 @@ export async function sealString(
plaintext: string,
): Promise<Uint8Array> {
const key = km.fieldKey(table, column);
const nonce = deriveNonce(key, table, pk);
const aad = buildAad(table, column, pk);
return aeadSeal(key, nonce, TEXT_ENCODER.encode(plaintext), aad);
return aeadSeal(key, randomNonce(), TEXT_ENCODER.encode(plaintext), aad);
}
/** Decrypt a blob into a string, reconstructing AAD from (table, column, pk). */
@@ -78,9 +77,8 @@ export async function openString(
blob: Uint8Array,
): Promise<string> {
const key = km.fieldKey(table, column);
const expectedNonce = deriveNonce(key, table, pk);
const aad = buildAad(table, column, pk);
const pt = await aeadOpen(key, blob, aad, expectedNonce);
const pt = await aeadOpen(key, blob, aad);
return TEXT_DECODER.decode(pt);
}
@@ -93,9 +91,8 @@ export async function sealBytes(
plaintext: Uint8Array,
): Promise<Uint8Array> {
const key = km.fieldKey(table, column);
const nonce = deriveNonce(key, table, pk);
const aad = buildAad(table, column, pk);
return aeadSeal(key, nonce, plaintext, aad);
return aeadSeal(key, randomNonce(), plaintext, aad);
}
/** Decrypt arbitrary bytes payload. */
@@ -107,9 +104,8 @@ export async function openBytes(
blob: Uint8Array,
): Promise<Uint8Array> {
const key = km.fieldKey(table, column);
const expectedNonce = deriveNonce(key, table, pk);
const aad = buildAad(table, column, pk);
return aeadOpen(key, blob, aad, expectedNonce);
return aeadOpen(key, blob, aad);
}
// ─── Typed encoders for each StorageProvider entity ──────────────────────

View File

@@ -19,12 +19,16 @@ export {
deriveBlobSlotId,
deriveBlobKey,
deriveBlobSigningSeed,
deriveVaultId,
deriveVaultContentKey,
deriveVaultSigningSeed,
} from './crypto/kdf.js';
export {
AEAD_NONCE_LEN,
AEAD_TAG_LEN,
aeadSeal,
aeadOpen,
randomNonce,
} from './crypto/aead.js';
export { EncryptedSQLiteStorage } from './storage/encrypted-sqlite.js';
export {

View File

@@ -100,6 +100,10 @@ export class EncryptedIndexedDBStorage implements StorageProvider {
});
members.createIndex('byChannelId', 'channelId');
}
if (oldVersion < 3) {
const aliases = db.createObjectStore('session_aliases_enc', { keyPath: 'alias' });
aliases.createIndex('byCanonical', 'canonical');
}
},
});
const store = new EncryptedIndexedDBStorage(db, opts.keyManager);
@@ -212,6 +216,27 @@ export class EncryptedIndexedDBStorage implements StorageProvider {
await this.db.delete('sessions_enc', address);
}
// ─── Session label aliases ─────────────────────────────────
//
// Labels are already the clear-text keyPath of sessions_enc, so the
// mapping between two of them exposes nothing the store didn't hold.
async getSessionAlias(alias: string): Promise<string | null> {
const row = await this.db.get('session_aliases_enc', alias);
return row?.canonical ?? null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
await this.db.put('session_aliases_enc', { alias, canonical });
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
const tx = this.db.transaction('session_aliases_enc', 'readwrite');
const matches = await tx.store.index('byCanonical').getAllKeys(canonical);
await Promise.all(matches.map((key) => tx.store.delete(key)));
await tx.done;
}
// ─── Trust ─────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
@@ -466,7 +491,7 @@ export class EncryptedIndexedDBStorage implements StorageProvider {
// ─── Schema ────────────────────────────────────────────────
const SCHEMA_VERSION = 2;
const SCHEMA_VERSION = 3;
interface MetaRow { key: string; value: string }
interface IdentityRow { id: 1; ciphertext: Uint8Array }
@@ -522,6 +547,11 @@ interface EncryptedShadeSchema extends DBSchema {
signed_prekeys_enc: { key: number; value: SignedPreKeyRow };
one_time_prekeys_enc: { key: number; value: OneTimePreKeyRow };
sessions_enc: { key: string; value: SessionRow };
session_aliases_enc: {
key: string;
value: { alias: string; canonical: string };
indexes: { byCanonical: string };
};
trusted_identities_enc: { key: string; value: TrustedIdentityRow };
retired_identities_enc: {
key: number;

View File

@@ -183,6 +183,30 @@ export class EncryptedPostgresStorage implements StorageProvider {
await this.sql`DELETE FROM shade_sessions_enc WHERE address = ${address}`;
}
// ─── Session label aliases ─────────────────────────────────
//
// Labels are already stored in the clear as the sessions_enc primary
// key, so a mapping between two of them reveals nothing new.
async getSessionAlias(alias: string): Promise<string | null> {
const rows = await this.sql<Array<{ canonical: string }>>`
SELECT canonical FROM shade_session_aliases_enc WHERE alias = ${alias}
`;
return rows.length ? rows[0]!.canonical : null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
await this.sql`
INSERT INTO shade_session_aliases_enc (alias, canonical)
VALUES (${alias}, ${canonical})
ON CONFLICT (alias) DO UPDATE SET canonical = EXCLUDED.canonical
`;
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
await this.sql`DELETE FROM shade_session_aliases_enc WHERE canonical = ${canonical}`;
}
// ─── Trust ─────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
@@ -515,6 +539,16 @@ export async function ensureEncryptedClientTables(sql: Sql): Promise<void> {
ciphertext BYTEA NOT NULL
)
`;
await sql`
CREATE TABLE IF NOT EXISTS shade_session_aliases_enc (
alias TEXT PRIMARY KEY,
canonical TEXT NOT NULL
)
`;
await sql`
CREATE INDEX IF NOT EXISTS idx_shade_session_aliases_enc_canonical
ON shade_session_aliases_enc(canonical)
`;
await sql`
CREATE TABLE IF NOT EXISTS shade_trusted_identities_enc (
address TEXT PRIMARY KEY,

View File

@@ -52,6 +52,9 @@ export class EncryptedSQLiteStorage implements StorageProvider {
getSession: ReturnType<Database['prepare']>;
saveSession: ReturnType<Database['prepare']>;
removeSession: ReturnType<Database['prepare']>;
getSessionAlias: ReturnType<Database['prepare']>;
saveSessionAlias: ReturnType<Database['prepare']>;
removeAliasesFor: ReturnType<Database['prepare']>;
getTrust: ReturnType<Database['prepare']>;
saveTrust: ReturnType<Database['prepare']>;
addRetired: ReturnType<Database['prepare']>;
@@ -134,6 +137,15 @@ export class EncryptedSQLiteStorage implements StorageProvider {
address TEXT PRIMARY KEY,
ciphertext BLOB NOT NULL
);
-- Session-label aliases (V4.12). Labels are already stored in the
-- clear as the sessions_enc primary key, so the mapping between two
-- of them reveals nothing new; only session state is encrypted.
CREATE TABLE IF NOT EXISTS session_aliases_enc (
alias TEXT PRIMARY KEY,
canonical TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_session_aliases_enc_canonical
ON session_aliases_enc(canonical);
CREATE TABLE IF NOT EXISTS trusted_identities_enc (
address TEXT PRIMARY KEY,
ciphertext BLOB NOT NULL
@@ -203,6 +215,9 @@ export class EncryptedSQLiteStorage implements StorageProvider {
getSession: this.db.prepare('SELECT ciphertext FROM sessions_enc WHERE address = ?'),
saveSession: this.db.prepare('INSERT OR REPLACE INTO sessions_enc (address, ciphertext) VALUES (?, ?)'),
removeSession: this.db.prepare('DELETE FROM sessions_enc WHERE address = ?'),
getSessionAlias: this.db.prepare('SELECT canonical FROM session_aliases_enc WHERE alias = ?'),
saveSessionAlias: this.db.prepare('INSERT OR REPLACE INTO session_aliases_enc (alias, canonical) VALUES (?, ?)'),
removeAliasesFor: this.db.prepare('DELETE FROM session_aliases_enc WHERE canonical = ?'),
getTrust: this.db.prepare('SELECT ciphertext FROM trusted_identities_enc WHERE address = ?'),
saveTrust: this.db.prepare('INSERT OR REPLACE INTO trusted_identities_enc (address, ciphertext) VALUES (?, ?)'),
addRetired: this.db.prepare('INSERT OR REPLACE INTO retired_identities_enc (retired_at, ciphertext) VALUES (?, ?)'),
@@ -377,6 +392,21 @@ export class EncryptedSQLiteStorage implements StorageProvider {
this.stmts.removeSession.run(address);
}
// ─── Session label aliases ─────────────────────────────────
async getSessionAlias(alias: string): Promise<string | null> {
const row = this.stmts.getSessionAlias.get(alias) as { canonical: string } | undefined;
return row?.canonical ?? null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
this.stmts.saveSessionAlias.run(alias, canonical);
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
this.stmts.removeAliasesFor.run(canonical);
}
// ─── Trust ─────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {

View File

@@ -0,0 +1,141 @@
/**
* Regression tests for the AES-GCM nonce-reuse fix (G0).
*
* The codec used to derive its nonce from (fieldKey, table, pk) alone. That
* is a pure function of row identity, so every re-seal of a mutable row —
* `saveSession` runs on each ratchet step — reused (key, nonce) across
* different plaintexts. AES-GCM forbids exactly that: it leaks the XOR of
* the plaintexts and the GHASH subkey, which yields tag forgery under that
* key. These tests fail if the derivation ever comes back.
*/
import { describe, test, expect } from 'bun:test';
import { KeyManager } from '../src/crypto/key-manager.js';
import { AEAD_NONCE_LEN, aeadSeal } from '../src/crypto/aead.js';
import { buildAad, deriveNonce } from '../src/crypto/kdf.js';
import {
COL,
TBL,
openBytes,
openString,
sealBytes,
sealString,
} from '../src/crypto/row-codec.js';
const TEXT = new TextEncoder();
function km(): Promise<KeyManager> {
return KeyManager.open({ kind: 'injected', key: new Uint8Array(32).fill(0x42) });
}
const nonceOf = (blob: Uint8Array) => blob.subarray(0, AEAD_NONCE_LEN);
const hex = (b: Uint8Array) => Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
describe('nonce uniqueness across re-saves', () => {
test('two seals of the same row do not share a nonce', async () => {
const k = await km();
// The real shape of the bug: same (table, column, pk), different
// plaintext, as a session row is re-sealed on every ratchet step.
const first = await sealString(k, TBL.sessions, COL.session, 'alice', '{"messageCount":1}');
const second = await sealString(k, TBL.sessions, COL.session, 'alice', '{"messageCount":2}');
expect(hex(nonceOf(first))).not.toBe(hex(nonceOf(second)));
k.destroy();
});
test('a long run of re-saves produces all-distinct nonces', async () => {
const k = await km();
const seen = new Set<string>();
for (let i = 0; i < 200; i++) {
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', `state-${i}`);
seen.add(hex(nonceOf(blob)));
}
expect(seen.size).toBe(200);
k.destroy();
});
test('sealBytes is covered by the same rule', async () => {
const k = await km();
const a = await sealBytes(k, TBL.config, COL.config, 'cfg', TEXT.encode('one'));
const b = await sealBytes(k, TBL.config, COL.config, 'cfg', TEXT.encode('two'));
expect(hex(nonceOf(a))).not.toBe(hex(nonceOf(b)));
k.destroy();
});
test('the nonce is not the derived one', async () => {
const k = await km();
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', 'payload');
const derived = deriveNonce(k.fieldKey(TBL.sessions, COL.session), TBL.sessions, 'alice');
expect(hex(nonceOf(blob))).not.toBe(hex(derived));
k.destroy();
});
});
describe('backward compatibility with deterministically-sealed blobs', () => {
// `aeadOpen` has always read the nonce from the blob prefix, so data
// written before the fix opens unchanged and needs no migration. This
// test reproduces an old blob by sealing with the deprecated derivation.
test('a blob sealed with the old derived nonce still opens', async () => {
const k = await km();
const key = k.fieldKey(TBL.sessions, COL.session);
const legacy = await aeadSeal(
key,
deriveNonce(key, TBL.sessions, 'alice'),
TEXT.encode('written before the fix'),
buildAad(TBL.sessions, COL.session, 'alice'),
);
const opened = await openString(k, TBL.sessions, COL.session, 'alice', legacy);
expect(opened).toBe('written before the fix');
k.destroy();
});
test('openBytes reads an old blob too', async () => {
const k = await km();
const key = k.fieldKey(TBL.config, COL.config);
const payload = TEXT.encode('legacy bytes');
const legacy = await aeadSeal(
key,
deriveNonce(key, TBL.config, 'cfg'),
payload,
buildAad(TBL.config, COL.config, 'cfg'),
);
expect(await openBytes(k, TBL.config, COL.config, 'cfg', legacy)).toEqual(payload);
k.destroy();
});
test('new and old blobs are both readable under one key', async () => {
const k = await km();
const key = k.fieldKey(TBL.sessions, COL.session);
const legacy = await aeadSeal(
key,
deriveNonce(key, TBL.sessions, 'bob'),
TEXT.encode('old'),
buildAad(TBL.sessions, COL.session, 'bob'),
);
const fresh = await sealString(k, TBL.sessions, COL.session, 'bob', 'new');
expect(await openString(k, TBL.sessions, COL.session, 'bob', legacy)).toBe('old');
expect(await openString(k, TBL.sessions, COL.session, 'bob', fresh)).toBe('new');
k.destroy();
});
});
describe('tamper detection survives the switch', () => {
// The dropped `expectedNonce` check was the stated reason for the
// deterministic nonce. It detected nothing the AEAD tag misses.
test('a flipped nonce byte is still rejected', async () => {
const k = await km();
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', 'payload');
const tampered = new Uint8Array(blob);
tampered[0]! ^= 0x01;
await expect(openString(k, TBL.sessions, COL.session, 'alice', tampered)).rejects.toThrow();
k.destroy();
});
test('a blob moved to another row is still rejected (AAD binding)', async () => {
const k = await km();
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', 'alice-secret');
await expect(openString(k, TBL.sessions, COL.session, 'bob', blob)).rejects.toThrow();
k.destroy();
});
});

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/storage-indexeddb",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -74,6 +74,10 @@ export class IndexedDBStorage implements StorageProvider {
const members = db.createObjectStore('broadcastMembers', { keyPath: ['channelId', 'peerAddress'] });
members.createIndex('byChannelId', 'channelId');
}
if (oldVersion < 3) {
const aliases = db.createObjectStore('sessionAliases', { keyPath: 'alias' });
aliases.createIndex('byCanonical', 'canonical');
}
},
});
return new IndexedDBStorage(db);
@@ -174,6 +178,24 @@ export class IndexedDBStorage implements StorageProvider {
await this.db.delete('sessions', address);
}
// ─── Session label aliases ────────────────────────────────
async getSessionAlias(alias: string): Promise<string | null> {
const row = await this.db.get('sessionAliases', alias);
return row?.canonical ?? null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
await this.db.put('sessionAliases', { alias, canonical });
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
const tx = this.db.transaction('sessionAliases', 'readwrite');
const matches = await tx.store.index('byCanonical').getAllKeys(canonical);
await Promise.all(matches.map((key) => tx.store.delete(key)));
await tx.done;
}
// ─── Trust ────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
@@ -360,7 +382,7 @@ export class IndexedDBStorage implements StorageProvider {
// ─── Schema ────────────────────────────────────────────────
const SCHEMA_VERSION = 2;
const SCHEMA_VERSION = 3;
interface IdentityRow {
id: 1;
@@ -390,6 +412,11 @@ interface SessionRow {
stateJson: string;
}
interface SessionAliasRow {
alias: string;
canonical: string;
}
interface TrustedIdentityRow {
address: string;
identityKey: string;
@@ -457,6 +484,11 @@ interface ShadeSchema extends DBSchema {
signedPreKeys: { key: number; value: SignedPreKeyRow };
oneTimePreKeys: { key: number; value: OneTimePreKeyRow };
sessions: { key: string; value: SessionRow };
sessionAliases: {
key: string;
value: SessionAliasRow;
indexes: { byCanonical: string };
};
trustedIdentities: { key: string; value: TrustedIdentityRow };
retiredIdentities: {
key: number;

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/storage-postgres",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -43,6 +43,16 @@ export async function ensureClientTables(sql: Sql): Promise<void> {
state_json TEXT NOT NULL
)
`;
await sql`
CREATE TABLE IF NOT EXISTS shade_session_aliases (
alias TEXT PRIMARY KEY,
canonical TEXT NOT NULL
)
`;
await sql`
CREATE INDEX IF NOT EXISTS idx_shade_session_aliases_canonical
ON shade_session_aliases(canonical)
`;
await sql`
CREATE TABLE IF NOT EXISTS shade_trusted_identities (
address TEXT PRIMARY KEY,

View File

@@ -160,6 +160,27 @@ export class PostgresStorage implements StorageProvider {
await this.sql`DELETE FROM shade_sessions WHERE address = ${address}`;
}
// ─── Session label aliases ────────────────────────────────
async getSessionAlias(alias: string): Promise<string | null> {
const rows = await this.sql<Array<{ canonical: string }>>`
SELECT canonical FROM shade_session_aliases WHERE alias = ${alias}
`;
return rows.length ? rows[0]!.canonical : null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
await this.sql`
INSERT INTO shade_session_aliases (alias, canonical)
VALUES (${alias}, ${canonical})
ON CONFLICT (alias) DO UPDATE SET canonical = EXCLUDED.canonical
`;
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
await this.sql`DELETE FROM shade_session_aliases WHERE canonical = ${canonical}`;
}
// ─── Trust ────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/storage-sqlite",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
@@ -9,5 +9,8 @@
"@shade/crypto-web": "workspace:*",
"@shade/inbox-server": "workspace:*",
"@shade/server": "workspace:*"
},
"devDependencies": {
"@shade/vault": "workspace:*"
}
}

View File

@@ -2,3 +2,4 @@ export { SQLiteStorage } from './sqlite-storage.js';
export { SqlitePrekeyStore } from './sqlite-prekey-store.js';
export { SqliteInboxStore } from './sqlite-inbox-store.js';
export { SqliteBlobStore } from './sqlite-blob-store.js';
export { SqliteVaultStore } from './sqlite-vault-store.js';

View File

@@ -41,6 +41,9 @@ export class SQLiteStorage implements StorageProvider {
getSession: ReturnType<Database['prepare']>;
saveSession: ReturnType<Database['prepare']>;
removeSession: ReturnType<Database['prepare']>;
getSessionAlias: ReturnType<Database['prepare']>;
saveSessionAlias: ReturnType<Database['prepare']>;
removeAliasesFor: ReturnType<Database['prepare']>;
getTrust: ReturnType<Database['prepare']>;
saveTrust: ReturnType<Database['prepare']>;
addRetired: ReturnType<Database['prepare']>;
@@ -100,6 +103,12 @@ export class SQLiteStorage implements StorageProvider {
address TEXT PRIMARY KEY,
state_json TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS session_aliases (
alias TEXT PRIMARY KEY,
canonical TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_session_aliases_canonical
ON session_aliases(canonical);
CREATE TABLE IF NOT EXISTS trusted_identities (
address TEXT PRIMARY KEY,
identity_key TEXT NOT NULL
@@ -179,6 +188,9 @@ export class SQLiteStorage implements StorageProvider {
getSession: this.db.prepare('SELECT state_json FROM sessions WHERE address = ?'),
saveSession: this.db.prepare('INSERT OR REPLACE INTO sessions (address, state_json) VALUES (?, ?)'),
removeSession: this.db.prepare('DELETE FROM sessions WHERE address = ?'),
getSessionAlias: this.db.prepare('SELECT canonical FROM session_aliases WHERE alias = ?'),
saveSessionAlias: this.db.prepare('INSERT OR REPLACE INTO session_aliases (alias, canonical) VALUES (?, ?)'),
removeAliasesFor: this.db.prepare('DELETE FROM session_aliases WHERE canonical = ?'),
getTrust: this.db.prepare('SELECT identity_key FROM trusted_identities WHERE address = ?'),
saveTrust: this.db.prepare('INSERT OR REPLACE INTO trusted_identities (address, identity_key) VALUES (?, ?)'),
addRetired: this.db.prepare('INSERT INTO retired_identities (data_json, retired_at) VALUES (?, ?)'),
@@ -337,6 +349,21 @@ export class SQLiteStorage implements StorageProvider {
this.stmts.removeSession.run(address);
}
// ─── Session label aliases ────────────────────────────────
async getSessionAlias(alias: string): Promise<string | null> {
const row = this.stmts.getSessionAlias.get(alias) as { canonical: string } | undefined;
return row?.canonical ?? null;
}
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
this.stmts.saveSessionAlias.run(alias, canonical);
}
async removeSessionAliasesFor(canonical: string): Promise<void> {
this.stmts.removeAliasesFor.run(canonical);
}
// ─── Trust ────────────────────────────────────────────────
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {

View File

@@ -0,0 +1,154 @@
import { Database } from 'bun:sqlite';
import type { VaultLogEntry, VaultStore } from '@shade/vault';
/**
* SQLite-backed VaultStore for the V4.13 encrypted file store.
*
* Three tables, mirroring the model: an owner key per vault, the
* content-addressed objects, and the append-only manifest log. The relay
* never decrypts anything — it enforces auth, verifies that an object's bytes
* hash to its name, and refuses a commit whose objects are not all present.
*
* Objects are stored as BLOBs rather than base64 text. A workspace backup is
* a few hundred files where the blob primitive holds one small profile, so the
* 33% base64 overhead stops being a rounding error.
*
* Docker usage: set `SHADE_VAULT_DB_PATH` (falls back to `/data/shade-vault.db`).
* **Set it.** The blob store's equivalent was missing from `docker-compose.yml`
* for months and nobody noticed, because the in-memory fallback works
* perfectly until the container is recreated — at which point everything is
* gone, with no error anywhere. That happened on 2026-08-12.
*/
export class SqliteVaultStore implements VaultStore {
private db: Database;
private stmts!: {
getOwner: ReturnType<Database['prepare']>;
setOwner: ReturnType<Database['prepare']>;
hasObject: ReturnType<Database['prepare']>;
putObject: ReturnType<Database['prepare']>;
getObject: ReturnType<Database['prepare']>;
head: ReturnType<Database['prepare']>;
log: ReturnType<Database['prepare']>;
logLimit: ReturnType<Database['prepare']>;
appendLog: ReturnType<Database['prepare']>;
usage: ReturnType<Database['prepare']>;
};
constructor(dbPath?: string) {
const path = dbPath ?? process.env.SHADE_VAULT_DB_PATH ?? '/data/shade-vault.db';
this.db = new Database(path, { create: true });
this.db.exec('PRAGMA journal_mode=WAL');
this.ensureTables();
this.prepareStatements();
}
private ensureTables() {
this.db.exec(`
CREATE TABLE IF NOT EXISTS shade_vault_owners (
vault_id TEXT PRIMARY KEY,
owner_pubkey BLOB NOT NULL,
created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS shade_vault_objects (
vault_id TEXT NOT NULL,
hash TEXT NOT NULL,
bytes BLOB NOT NULL,
size INTEGER NOT NULL,
created_at INTEGER NOT NULL,
PRIMARY KEY (vault_id, hash)
);
CREATE TABLE IF NOT EXISTS shade_vault_log (
vault_id TEXT NOT NULL,
seq INTEGER NOT NULL,
manifest TEXT NOT NULL,
at INTEGER NOT NULL,
bytes INTEGER NOT NULL,
PRIMARY KEY (vault_id, seq)
);
`);
}
private prepareStatements() {
this.stmts = {
getOwner: this.db.prepare('SELECT owner_pubkey FROM shade_vault_owners WHERE vault_id = ?'),
setOwner: this.db.prepare(
'INSERT OR REPLACE INTO shade_vault_owners (vault_id, owner_pubkey, created_at) VALUES (?, ?, ?)',
),
hasObject: this.db.prepare(
'SELECT 1 FROM shade_vault_objects WHERE vault_id = ? AND hash = ? LIMIT 1',
),
// An object's name IS its content hash, so a repeat upload is the same
// bytes by definition — ignoring it is correct, not lossy.
putObject: this.db.prepare(
'INSERT OR IGNORE INTO shade_vault_objects (vault_id, hash, bytes, size, created_at) VALUES (?, ?, ?, ?, ?)',
),
getObject: this.db.prepare(
'SELECT bytes FROM shade_vault_objects WHERE vault_id = ? AND hash = ?',
),
head: this.db.prepare('SELECT MAX(seq) AS head FROM shade_vault_log WHERE vault_id = ?'),
log: this.db.prepare(
'SELECT seq, manifest, at, bytes FROM shade_vault_log WHERE vault_id = ? ORDER BY seq ASC',
),
logLimit: this.db.prepare(
'SELECT seq, manifest, at, bytes FROM (SELECT seq, manifest, at, bytes FROM shade_vault_log WHERE vault_id = ? ORDER BY seq DESC LIMIT ?) ORDER BY seq ASC',
),
appendLog: this.db.prepare(
'INSERT INTO shade_vault_log (vault_id, seq, manifest, at, bytes) VALUES (?, ?, ?, ?, ?)',
),
usage: this.db.prepare(
'SELECT COALESCE(SUM(size), 0) AS total FROM shade_vault_objects WHERE vault_id = ?',
),
};
}
async getOwner(vaultId: string): Promise<Uint8Array | null> {
const row = this.stmts.getOwner.get(vaultId) as { owner_pubkey: Uint8Array } | null;
return row ? new Uint8Array(row.owner_pubkey) : null;
}
async setOwner(vaultId: string, publicKey: Uint8Array): Promise<void> {
this.stmts.setOwner.run(vaultId, publicKey, Date.now());
}
async hasObject(vaultId: string, hash: string): Promise<boolean> {
return this.stmts.hasObject.get(vaultId, hash) !== null;
}
async putObject(vaultId: string, hash: string, bytes: Uint8Array): Promise<void> {
this.stmts.putObject.run(vaultId, hash, bytes, bytes.length, Date.now());
}
async getObject(vaultId: string, hash: string): Promise<Uint8Array | null> {
const row = this.stmts.getObject.get(vaultId, hash) as { bytes: Uint8Array } | null;
return row ? new Uint8Array(row.bytes) : null;
}
async head(vaultId: string): Promise<number> {
const row = this.stmts.head.get(vaultId) as { head: number | null } | null;
return row?.head ?? 0;
}
async log(vaultId: string, limit?: number): Promise<VaultLogEntry[]> {
const rows = (
limit === undefined
? this.stmts.log.all(vaultId)
: this.stmts.logLimit.all(vaultId, limit)
) as VaultLogEntry[];
return rows;
}
async appendLog(vaultId: string, entry: VaultLogEntry): Promise<void> {
this.stmts.appendLog.run(vaultId, entry.seq, entry.manifest, entry.at, entry.bytes);
}
async usage(vaultId: string): Promise<number> {
const row = this.stmts.usage.get(vaultId) as { total: number } | null;
return row?.total ?? 0;
}
close(): void {
this.db.close();
}
}

View File

@@ -0,0 +1,120 @@
/**
* The SQLite vault store, with persistence as the headline property.
*
* This exists because of 2026-08-12: the blob store fell back to memory when
* its path was unset, worked perfectly, and lost every profile on the next
* container recreate — with no error anywhere. A backup store that forgets is
* worse than none, so "survives a close and reopen" is tested directly rather
* than assumed from the fact that SQLite is involved.
*/
import { describe, test, expect, afterEach } from 'bun:test';
import { unlinkSync } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import { SqliteVaultStore } from '../src/sqlite-vault-store.js';
const paths: string[] = [];
function scratch(name: string): string {
const p = join(tmpdir(), `shade-vault-${name}-${process.pid}-${Math.random().toString(36).slice(2)}.db`);
paths.push(p);
return p;
}
afterEach(() => {
for (const p of paths.splice(0)) {
for (const suffix of ['', '-wal', '-shm']) {
try {
unlinkSync(p + suffix);
} catch {
// Not every WAL sidecar exists; absence is fine.
}
}
}
});
const VAULT = 'a'.repeat(64);
const HASH = 'b'.repeat(64);
describe('persistence', () => {
test('objects, owner and log survive a close and reopen', async () => {
const path = scratch('reopen');
const first = new SqliteVaultStore(path);
await first.setOwner(VAULT, new Uint8Array([1, 2, 3, 4]));
await first.putObject(VAULT, HASH, new Uint8Array([9, 8, 7]));
await first.appendLog(VAULT, { seq: 1, manifest: HASH, at: 1700, bytes: 3 });
first.close();
const second = new SqliteVaultStore(path);
expect(await second.getOwner(VAULT)).toEqual(new Uint8Array([1, 2, 3, 4]));
expect(await second.getObject(VAULT, HASH)).toEqual(new Uint8Array([9, 8, 7]));
expect(await second.head(VAULT)).toBe(1);
expect((await second.log(VAULT))[0]!.manifest).toBe(HASH);
second.close();
});
test('binary content round-trips without base64 mangling', async () => {
// Objects are ciphertext: every byte value occurs, including 0x00.
const path = scratch('binary');
const store = new SqliteVaultStore(path);
const bytes = new Uint8Array(256);
for (let i = 0; i < 256; i++) bytes[i] = i;
await store.putObject(VAULT, HASH, bytes);
store.close();
const reopened = new SqliteVaultStore(path);
expect(await reopened.getObject(VAULT, HASH)).toEqual(bytes);
reopened.close();
});
});
describe('semantics', () => {
test('an empty vault has head 0 and no owner', async () => {
const store = new SqliteVaultStore(scratch('empty'));
expect(await store.head(VAULT)).toBe(0);
expect(await store.getOwner(VAULT)).toBeNull();
expect(await store.log(VAULT)).toEqual([]);
expect(await store.usage(VAULT)).toBe(0);
store.close();
});
test('re-putting the same hash is a no-op, not a duplicate', async () => {
// The name IS the content hash, so a repeat is the same bytes by
// definition. Usage must not double.
const store = new SqliteVaultStore(scratch('dupe'));
await store.putObject(VAULT, HASH, new Uint8Array(100));
await store.putObject(VAULT, HASH, new Uint8Array(100));
expect(await store.usage(VAULT)).toBe(100);
store.close();
});
test('vaults are isolated from each other', async () => {
const store = new SqliteVaultStore(scratch('isolated'));
const other = 'c'.repeat(64);
await store.putObject(VAULT, HASH, new Uint8Array([1]));
expect(await store.hasObject(other, HASH)).toBe(false);
expect(await store.getObject(other, HASH)).toBeNull();
expect(await store.usage(other)).toBe(0);
store.close();
});
test('the log is ordered oldest-first and limit counts back from the head', async () => {
const store = new SqliteVaultStore(scratch('log'));
for (let seq = 1; seq <= 5; seq++) {
await store.appendLog(VAULT, { seq, manifest: `${seq}`.repeat(64), at: seq, bytes: seq });
}
expect((await store.log(VAULT)).map((e) => e.seq)).toEqual([1, 2, 3, 4, 5]);
// A client asking for the last two wants 4 and 5, still in order.
expect((await store.log(VAULT, 2)).map((e) => e.seq)).toEqual([4, 5]);
expect(await store.head(VAULT)).toBe(5);
store.close();
});
test('a duplicate seq is rejected by the primary key', async () => {
// The route layer refuses this first, but the store is the last line:
// two rows with the same seq would make history ambiguous.
const store = new SqliteVaultStore(scratch('seq'));
await store.appendLog(VAULT, { seq: 1, manifest: HASH, at: 1, bytes: 1 });
expect(store.appendLog(VAULT, { seq: 1, manifest: HASH, at: 2, bytes: 1 })).rejects.toThrow();
store.close();
});
});

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/streams",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/transfer",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/transport-bridge",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/transport-webrtc",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/transport",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -0,0 +1,74 @@
/**
* Full backup mot en KJØRENDE vault — ikke Hono-fetch, men ekte HTTP.
*
* Kjøres manuelt mot en container eller en deployet tjeneste:
* SCAFFOLDD_SOCKET=... RELAY=https://scaffold.zyon.no bun live-e2e.ts
*/
import { connect } from 'node:net';
import { readFile } from 'node:fs/promises';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { HttpVaultTransport, VaultClient, deriveVaultKeys } from './src/index.js';
const SOCKET = process.env.SCAFFOLDD_SOCKET!;
const RELAY = process.env.RELAY!;
function ask(req: object): Promise<any> {
return new Promise((res, rej) => {
const s = connect(SOCKET);
let buf = '';
s.on('connect', () => s.write(`${JSON.stringify(req)}\n`));
s.on('data', (c) => {
buf += c.toString();
const nl = buf.indexOf('\n');
if (nl === -1) return;
s.destroy();
const r = JSON.parse(buf.slice(0, nl));
r.ok ? res(r.result) : rej(new Error(r.error));
});
s.on('error', rej);
});
}
const set = await ask({ op: 'backupSet' });
const files: { path: string; bytes: Uint8Array }[] = [];
for (const f of set.files) {
try {
files.push({ path: f.path, bytes: new Uint8Array(await readFile(f.source)) });
} catch {
/* uleselig fil hoppes over, som i broen */
}
}
const crypto = new SubtleCryptoProvider();
const keys = await deriveVaultKeys(new Uint8Array(32).fill(77), 'scaffold-live');
const client = new VaultClient(crypto, keys, new HttpVaultTransport(RELAY));
console.log(`push: ${files.length} filer …`);
let t = Date.now();
const r1 = await client.push(files, Date.now(), 'live e2e');
console.log(
` versjon ${r1.seq}: ${r1.uploaded} opplastet, ` +
`${(r1.bytesUploaded / 1024 / 1024).toFixed(1)} MB, ${((Date.now() - t) / 1000).toFixed(1)}s`,
);
const edited = files.map((f, i) =>
i === 0 ? { path: f.path, bytes: new TextEncoder().encode('endret') } : f,
);
t = Date.now();
const r2 = await client.push(edited, Date.now(), 'én endring');
console.log(
` versjon ${r2.seq}: ${r2.uploaded} opplastet, ${r2.reused} gjenbrukt, ` +
`${(r2.bytesUploaded / 1024).toFixed(0)} KB, ${((Date.now() - t) / 1000).toFixed(1)}s`,
);
const back = await client.pull(1);
let identical = 0;
let differ = 0;
for (const f of files) {
const got = back.files.get(f.path);
if (got && Buffer.from(got).equals(Buffer.from(f.bytes))) identical++;
else differ++;
}
console.log(`pull versjon 1: ${identical} bit-identiske, ${differ} avvik`);
const log = await client.history();
console.log(`historikk: ${log.entries.map((e) => e.seq).join(', ')} (head ${log.head})`);

View File

@@ -0,0 +1,25 @@
{
"name": "@shade/vault",
"version": "4.13.0",
"description": "Server-side encrypted file store for Shade — content-addressed objects with an append-only manifest log",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",
"exports": {
".": "./src/index.ts",
"./server": "./src/server.ts"
},
"dependencies": {
"@noble/hashes": "^2.0.1",
"@shade/core": "workspace:*",
"@shade/crypto-web": "workspace:*",
"@shade/observability": "workspace:*",
"@shade/server": "workspace:*",
"@shade/storage-encrypted": "workspace:*",
"hono": "^4.12.18"
},
"scripts": {
"test": "bun test",
"typecheck": "tsc --noEmit"
}
}

View File

@@ -0,0 +1,225 @@
/**
* The client half: turn a set of files into an encrypted, versioned vault,
* and turn it back again on a machine that has nothing but the credentials.
*
* The push is deliberately have-check-first. A workspace changes a few lines
* at a time, so asking "do you already have these hashes?" and uploading only
* the misses turns a full backup into a handful of small writes. Everything
* unchanged keeps its hash from the previous commit and costs nothing.
*/
import type { CryptoProvider } from '@shade/core';
import { toBase64 } from '@shade/core';
// The pubkey derivation is not on `CryptoProvider` — it is a pure function of
// the seed, and lives with the curve implementation.
import { ed25519PublicKeyFromSeed } from '@shade/crypto-web';
import { signPayload } from '@shade/server';
import {
deriveVaultContentKey,
deriveVaultId,
deriveVaultSigningSeed,
} from '@shade/storage-encrypted';
import { openManifest, openObject, sealManifest, sealObject, toHex } from './crypto.js';
import type { VaultEntry, VaultLog, VaultManifest } from './types.js';
/** A file as the app sees it, before encryption. */
export interface VaultFile {
path: string;
bytes: Uint8Array;
mode?: string;
}
export interface VaultKeys {
vaultId: string;
contentKey: Uint8Array;
signingSeed: Uint8Array;
publicKey: Uint8Array;
}
/**
* Derive everything a vault needs from the account master key.
*
* This is what makes recovery "username and password": a fresh device that
* can derive the profile master can derive these too, and the relay hands
* over ciphertext it has never been able to read.
*/
export async function deriveVaultKeys(masterKey: Uint8Array, app: string): Promise<VaultKeys> {
const signingSeed = deriveVaultSigningSeed(masterKey, app);
const publicKey = ed25519PublicKeyFromSeed(signingSeed);
return {
vaultId: toHex(deriveVaultId(masterKey, app)),
contentKey: deriveVaultContentKey(masterKey, app),
signingSeed,
publicKey,
};
}
export interface VaultTransport {
/** Does the relay already hold this object? */
has(vaultId: string, hash: string): Promise<boolean>;
put(vaultId: string, hash: string, body: unknown): Promise<void>;
get(vaultId: string, hash: string): Promise<Uint8Array>;
log(vaultId: string, limit?: number): Promise<VaultLog>;
commit(vaultId: string, body: unknown): Promise<{ seq: number }>;
}
export interface PushResult {
seq: number;
/** Objects actually sent — the rest were already on the relay. */
uploaded: number;
/** Objects skipped because the relay had them. */
reused: number;
bytesUploaded: number;
}
export class VaultClient {
constructor(
private readonly crypto: CryptoProvider,
private readonly keys: VaultKeys,
private readonly transport: VaultTransport,
) {}
get vaultId(): string {
return this.keys.vaultId;
}
/**
* Sign a request body, with the pubkey inside the signed payload.
*
* The key has to be part of what gets signed, not appended afterwards:
* `verifyPayload` canonicalises every field, so an appended key would both
* break the signature and — if the scheme ignored it — let anyone swap the
* claimed identity in transit on a first, TOFU-pinning write.
*/
private async sign(body: Record<string, unknown>): Promise<Record<string, unknown>> {
return signPayload(this.crypto, this.keys.signingSeed, {
...body,
publicKey: toBase64(this.keys.publicKey),
});
}
/**
* Encrypt and upload `files`, then commit a manifest naming them.
*
* `at` is passed in rather than read from the clock so a caller can make a
* commit reproducible in tests, and so a queued backup records when it was
* *taken* rather than when it finally reached the relay.
*/
async push(files: VaultFile[], at: number, message?: string): Promise<PushResult> {
const { head } = await this.transport.log(this.keys.vaultId, 1);
const entries: VaultEntry[] = [];
let uploaded = 0;
let reused = 0;
let bytesUploaded = 0;
// Reuse the previous manifest's hashes for unchanged content: sealing is
// nondeterministic (fresh nonce), so re-sealing an untouched file would
// produce a new hash and a pointless upload every single time.
const previous = head > 0 ? await this.pull().catch(() => null) : null;
const byPath = new Map(previous?.manifest.entries.map((e) => [e.path, e]) ?? []);
const priorPlain = previous?.files ?? new Map<string, Uint8Array>();
for (const file of files) {
const prior = byPath.get(file.path);
const priorBytes = priorPlain.get(file.path);
if (prior && priorBytes && sameBytes(priorBytes, file.bytes)) {
entries.push({ ...prior, size: file.bytes.length });
reused++;
continue;
}
const sealed = await sealObject(
this.crypto,
this.keys.contentKey,
this.keys.vaultId,
file.bytes,
);
if (!(await this.transport.has(this.keys.vaultId, sealed.hash))) {
await this.transport.put(
this.keys.vaultId,
sealed.hash,
await this.sign({ data: toBase64(sealed.bytes) }),
);
uploaded++;
bytesUploaded += sealed.bytes.length;
} else {
reused++;
}
const entry: VaultEntry = { path: file.path, hash: sealed.hash, size: file.bytes.length };
if (file.mode !== undefined) entry.mode = file.mode;
entries.push(entry);
}
const manifest: VaultManifest = { version: 1, seq: head + 1, at, entries };
if (message !== undefined) manifest.message = message;
const sealedManifest = await sealManifest(
this.crypto,
this.keys.contentKey,
this.keys.vaultId,
manifest,
);
await this.transport.put(
this.keys.vaultId,
sealedManifest.hash,
await this.sign({ data: toBase64(sealedManifest.bytes) }),
);
const { seq } = await this.transport.commit(
this.keys.vaultId,
await this.sign({
manifest: sealedManifest.hash,
seq: manifest.seq,
at,
hashes: entries.map((e) => e.hash),
}),
);
return { seq, uploaded, reused, bytesUploaded };
}
/**
* Fetch a version and decrypt it. Defaults to the newest.
*
* Restoring is the whole point of the exercise, so this returns the plain
* files rather than a handle: a caller that has to make a second round of
* decisions to get their data back does not have a backup.
*/
async pull(seq?: number): Promise<{ manifest: VaultManifest; files: Map<string, Uint8Array> }> {
const log = await this.transport.log(this.keys.vaultId);
const wanted = seq ?? log.head;
const row = log.entries.find((e) => e.seq === wanted);
if (!row) throw new Error(`vault has no version ${wanted}`);
const manifestBytes = await this.transport.get(this.keys.vaultId, row.manifest);
const manifest = await openManifest(
this.crypto,
this.keys.contentKey,
this.keys.vaultId,
row.manifest,
manifestBytes,
);
const files = new Map<string, Uint8Array>();
for (const entry of manifest.entries) {
const bytes = await this.transport.get(this.keys.vaultId, entry.hash);
files.set(
entry.path,
await openObject(this.crypto, this.keys.contentKey, this.keys.vaultId, entry.hash, bytes),
);
}
return { manifest, files };
}
/** Version history, newest last. */
async history(limit?: number): Promise<VaultLog> {
return this.transport.log(this.keys.vaultId, limit);
}
}
function sameBytes(a: Uint8Array, b: Uint8Array): boolean {
if (a.length !== b.length) return false;
for (let i = 0; i < a.length; i++) if (a[i] !== b[i]) return false;
return true;
}

View File

@@ -0,0 +1,129 @@
/**
* Sealing and naming for vault objects.
*
* Every object on the relay is `nonce(12) || ciphertext||tag`, named by the
* SHA-256 of those bytes. Hashing the *ciphertext* rather than the plaintext
* is what lets the relay verify that an upload is what it claims to be
* without holding a key — it recomputes the hash and compares.
*
* The cost is that identical plaintext yields different names on each seal,
* because the nonce is fresh. Deduplication is therefore *within* a version
* chain (an unchanged file keeps its hash across commits because the client
* reuses the object it already uploaded), not across independent seals. That
* is the right trade: convergent encryption would leak which files two users
* share, which is exactly what a blind store must not do.
*/
import { sha256 } from '@noble/hashes/sha2.js';
import type { CryptoProvider } from '@shade/core';
import type { VaultManifest } from './types.js';
const NONCE_LEN = 12;
/** Lowercase hex SHA-256 — the name of an object. */
export function objectHash(sealed: Uint8Array): string {
return Array.from(sha256(sealed))
.map((b) => b.toString(16).padStart(2, '0'))
.join('');
}
/**
* AAD for an object.
*
* Binds the vault it belongs to, so a ciphertext lifted from one vault cannot
* be planted in another even by someone who can write to both. The content
* hash is deliberately NOT in the AAD: it is derived from the very bytes
* being sealed, so it cannot be known before sealing, and the relay's own
* hash check already covers substitution.
*/
function objectAad(vaultId: string): Uint8Array {
return new TextEncoder().encode(`shade-vault-object-v1|${vaultId}`);
}
export interface SealedObject {
hash: string;
bytes: Uint8Array;
}
/** Encrypt one object and give it its name. */
export async function sealObject(
crypto: CryptoProvider,
contentKey: Uint8Array,
vaultId: string,
plaintext: Uint8Array,
): Promise<SealedObject> {
const { ciphertext, nonce } = await crypto.aesGcmEncrypt(
contentKey,
plaintext,
objectAad(vaultId),
);
const bytes = new Uint8Array(NONCE_LEN + ciphertext.length);
bytes.set(nonce, 0);
bytes.set(ciphertext, NONCE_LEN);
return { hash: objectHash(bytes), bytes };
}
/**
* Decrypt one object.
*
* Throws when the hash does not match the bytes: a relay that returned the
* wrong object would otherwise surface as an AEAD failure, which reads like
* key corruption and sends the user looking in the wrong place.
*/
export async function openObject(
crypto: CryptoProvider,
contentKey: Uint8Array,
vaultId: string,
expectedHash: string,
bytes: Uint8Array,
): Promise<Uint8Array> {
if (bytes.length < NONCE_LEN + 16) {
throw new Error('vault object too short to be a sealed object');
}
const actual = objectHash(bytes);
if (actual !== expectedHash) {
throw new Error(`vault object hash mismatch: expected ${expectedHash}, got ${actual}`);
}
return crypto.aesGcmDecrypt(
contentKey,
bytes.subarray(NONCE_LEN),
bytes.subarray(0, NONCE_LEN),
objectAad(vaultId),
);
}
/** A manifest is stored as an ordinary object; this is the JSON step around it. */
export async function sealManifest(
crypto: CryptoProvider,
contentKey: Uint8Array,
vaultId: string,
manifest: VaultManifest,
): Promise<SealedObject> {
const json = new TextEncoder().encode(JSON.stringify(manifest));
return sealObject(crypto, contentKey, vaultId, json);
}
export async function openManifest(
crypto: CryptoProvider,
contentKey: Uint8Array,
vaultId: string,
expectedHash: string,
bytes: Uint8Array,
): Promise<VaultManifest> {
const plain = await openObject(crypto, contentKey, vaultId, expectedHash, bytes);
const parsed = JSON.parse(new TextDecoder().decode(plain)) as VaultManifest;
if (parsed.version !== 1) {
// Refusing beats guessing: a future manifest may mean a file the reader
// cannot represent, and silently dropping it would lose data on the next
// commit made from this client.
throw new Error(`unsupported vault manifest version: ${parsed.version}`);
}
return parsed;
}
/** Lowercase hex of arbitrary bytes — used for vaultId and pubkeys on the wire. */
export function toHex(bytes: Uint8Array): string {
return Array.from(bytes)
.map((b) => b.toString(16).padStart(2, '0'))
.join('');
}

View File

@@ -0,0 +1,80 @@
/**
* The default transport: plain HTTP against a relay running the vault routes.
*
* Takes a `fetch` rather than calling the global one, so the same client works
* against a live server, a Hono app under test, and anything else that can
* answer a Request — which is how the tests exercise the real route handlers
* instead of a mock that agrees with them by construction.
*/
import { fromBase64 } from '@shade/core';
import type { VaultTransport } from './client.js';
import type { VaultLog } from './types.js';
type FetchLike = (input: string, init?: RequestInit) => Promise<Response>;
export class HttpVaultTransport implements VaultTransport {
constructor(
private readonly baseUrl: string,
private readonly fetchImpl: FetchLike = globalThis.fetch.bind(globalThis),
) {}
private url(path: string): string {
return `${this.baseUrl.replace(/\/$/, '')}${path}`;
}
/** Turn a non-2xx into an Error that names the relay's own code. */
private async fail(res: Response, what: string): Promise<never> {
let detail = res.statusText;
try {
const body = (await res.json()) as { error?: { code?: string; message?: string } };
if (body?.error) detail = `${body.error.code}: ${body.error.message}`;
} catch {
// Non-JSON error body; the status line is all we have.
}
throw new Error(`vault ${what} failed (${res.status}) — ${detail}`);
}
async has(vaultId: string, hash: string): Promise<boolean> {
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/object/${hash}`), {
method: 'HEAD',
});
if (res.status === 200) return true;
if (res.status === 404) return false;
return this.fail(res, 'have-check');
}
async put(vaultId: string, hash: string, body: unknown): Promise<void> {
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/object/${hash}`), {
method: 'PUT',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
});
if (!res.ok) return this.fail(res, 'upload');
}
async get(vaultId: string, hash: string): Promise<Uint8Array> {
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/object/${hash}`));
if (!res.ok) return this.fail(res, 'download');
return new Uint8Array(await res.arrayBuffer());
}
async log(vaultId: string, limit?: number): Promise<VaultLog> {
const q = limit === undefined ? '' : `?limit=${limit}`;
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/log${q}`));
if (!res.ok) return this.fail(res, 'log');
return (await res.json()) as VaultLog;
}
async commit(vaultId: string, body: unknown): Promise<{ seq: number }> {
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/commit`), {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
});
if (!res.ok) return this.fail(res, 'commit');
return (await res.json()) as { seq: number };
}
}
export { fromBase64 };

View File

@@ -0,0 +1,6 @@
export * from './types.js';
export * from './crypto.js';
export * from './client.js';
export { MemoryVaultStore } from './store.js';
export type { VaultStore } from './store.js';
export { HttpVaultTransport } from './http-transport.js';

View File

@@ -0,0 +1,226 @@
/**
* Relay-side vault routes.
*
* GET /v1/vault/:vaultId/log → { entries, head }
* GET /v1/vault/:vaultId/object/:hash → raw ciphertext bytes
* HEAD /v1/vault/:vaultId/object/:hash → 200 | 404 (have-check)
* PUT /v1/vault/:vaultId/object/:hash → { stored } (signed)
* POST /v1/vault/:vaultId/commit → { seq } (signed)
*
* Auth is the same TOFU-Ed25519 scheme the blob primitive uses: the first
* signed write pins a pubkey for the vault, and every later write must be
* signed by it. There is no account, no password, and nothing for the relay
* to leak — it cannot even tell which user a vaultId belongs to.
*/
import { Hono } from 'hono';
import type { ContentfulStatusCode } from 'hono/utils/http-status';
import type { CryptoProvider } from '@shade/core';
import { fromBase64 } from '@shade/core';
import { verifyPayload } from '@shade/server';
import { objectHash } from './crypto.js';
import type { VaultStore } from './store.js';
import type { VaultManifest } from './types.js';
const ID_REGEX = /^[0-9a-f]{64}$/;
const HASH_REGEX = /^[0-9a-f]{64}$/;
export interface VaultRoutesOptions {
/** Per-object ceiling. Defaults to 8 MiB. */
maxObjectBytes?: number;
/** Whole-vault ceiling. Defaults to 512 MiB. */
maxVaultBytes?: number;
}
const DEFAULT_MAX_OBJECT = 8 * 1024 * 1024;
const DEFAULT_MAX_VAULT = 512 * 1024 * 1024;
function fail(code: string, message: string, status: ContentfulStatusCode) {
return { body: { error: { code, message } }, status };
}
export function createVaultRoutes(
store: VaultStore,
crypto: CryptoProvider,
options: VaultRoutesOptions = {},
): Hono {
const app = new Hono();
const maxObject = options.maxObjectBytes ?? DEFAULT_MAX_OBJECT;
const maxVault = options.maxVaultBytes ?? DEFAULT_MAX_VAULT;
/**
* Check a signed request against the vault's pinned key, pinning it on the
* first write. `publicKey` is only honoured when nothing is pinned yet —
* otherwise anyone could rotate the owner by simply asserting a new key.
*/
async function authorize(
vaultId: string,
payload: Record<string, unknown>,
): Promise<
{ ok: true } | { ok: false; code: string; message: string; status: ContentfulStatusCode }
> {
const claimed = typeof payload.publicKey === 'string' ? payload.publicKey : null;
const pinned = await store.getOwner(vaultId);
if (!pinned) {
if (!claimed) {
return { ok: false, code: 'UNAUTHORIZED', message: 'first write must carry publicKey', status: 401 };
}
const key = fromBase64(claimed);
try {
await verifyPayload(crypto, key, payload);
} catch (e) {
return { ok: false, code: 'UNAUTHORIZED', message: String((e as Error).message), status: 401 };
}
await store.setOwner(vaultId, key);
return { ok: true };
}
try {
await verifyPayload(crypto, pinned, payload);
} catch (e) {
return { ok: false, code: 'UNAUTHORIZED', message: String((e as Error).message), status: 401 };
}
return { ok: true };
}
app.get('/v1/vault/:vaultId/log', async (c) => {
const vaultId = c.req.param('vaultId');
if (!ID_REGEX.test(vaultId)) {
const f = fail('BAD_REQUEST', 'vaultId must be 64 lowercase hex chars', 400);
return c.json(f.body, f.status);
}
const limitRaw = c.req.query('limit');
const limit = limitRaw ? Number(limitRaw) : undefined;
const entries = await store.log(vaultId, Number.isFinite(limit) ? limit : undefined);
return c.json({ entries, head: await store.head(vaultId) });
});
// A have-check before uploading. This is what makes an unchanged file free:
// the client asks about every hash in the new manifest and only sends the
// ones the relay is missing.
app.on(['HEAD', 'GET'], '/v1/vault/:vaultId/object/:hash', async (c) => {
const vaultId = c.req.param('vaultId');
const hash = c.req.param('hash');
if (!ID_REGEX.test(vaultId) || !HASH_REGEX.test(hash)) {
const f = fail('BAD_REQUEST', 'bad vaultId or hash', 400);
return c.json(f.body, f.status);
}
if (c.req.method === 'HEAD') {
return c.body(null, (await store.hasObject(vaultId, hash)) ? 200 : 404);
}
const bytes = await store.getObject(vaultId, hash);
if (!bytes) {
const f = fail('NOT_FOUND', 'no such object', 404);
return c.json(f.body, f.status);
}
return c.body(bytes as unknown as ArrayBuffer, 200, {
'content-type': 'application/octet-stream',
});
});
app.put('/v1/vault/:vaultId/object/:hash', async (c) => {
const vaultId = c.req.param('vaultId');
const hash = c.req.param('hash');
if (!ID_REGEX.test(vaultId) || !HASH_REGEX.test(hash)) {
const f = fail('BAD_REQUEST', 'bad vaultId or hash', 400);
return c.json(f.body, f.status);
}
const body = (await c.req.json().catch(() => null)) as Record<string, unknown> | null;
if (!body || typeof body.data !== 'string') {
const f = fail('BAD_REQUEST', 'expected { data, signedAt, signature }', 400);
return c.json(f.body, f.status);
}
const auth = await authorize(vaultId, body);
if (!auth.ok) {
const f = fail(auth.code, auth.message, auth.status);
return c.json(f.body, f.status);
}
const bytes = fromBase64(body.data);
if (bytes.length > maxObject) {
const f = fail('TOO_LARGE', `object exceeds ${maxObject} bytes`, 413);
return c.json(f.body, f.status);
}
if ((await store.usage(vaultId)) + bytes.length > maxVault) {
const f = fail('TOO_LARGE', `vault exceeds ${maxVault} bytes`, 413);
return c.json(f.body, f.status);
}
// The name must be the hash of the bytes. Without this the store would
// accept a mislabelled object, and every later read of that name would
// fail decryption somewhere far away from the cause.
const actual = objectHash(bytes);
if (actual !== hash) {
const f = fail('BAD_REQUEST', `hash mismatch: bytes hash to ${actual}`, 400);
return c.json(f.body, f.status);
}
await store.putObject(vaultId, hash, bytes);
return c.json({ stored: true, hash });
});
app.post('/v1/vault/:vaultId/commit', async (c) => {
const vaultId = c.req.param('vaultId');
if (!ID_REGEX.test(vaultId)) {
const f = fail('BAD_REQUEST', 'vaultId must be 64 lowercase hex chars', 400);
return c.json(f.body, f.status);
}
const body = (await c.req.json().catch(() => null)) as Record<string, unknown> | null;
if (!body || typeof body.manifest !== 'string' || typeof body.seq !== 'number') {
const f = fail('BAD_REQUEST', 'expected { manifest, seq, hashes, signedAt, signature }', 400);
return c.json(f.body, f.status);
}
const auth = await authorize(vaultId, body);
if (!auth.ok) {
const f = fail(auth.code, auth.message, auth.status);
return c.json(f.body, f.status);
}
const head = await store.head(vaultId);
if (body.seq !== head + 1) {
// Two devices committed from the same head. The loser has to re-read
// and re-commit; retrying the same seq would overwrite a version that
// is already part of the history.
const f = fail('SEQ_CONFLICT', `expected seq ${head + 1}, got ${body.seq}`, 409);
return c.json({ ...f.body, head }, f.status);
}
if (!(await store.hasObject(vaultId, body.manifest))) {
const f = fail('MISSING_OBJECTS', 'manifest object not uploaded', 409);
return c.json(f.body, f.status);
}
// Every object the manifest references must already be here, or the
// commit would publish a version that cannot be restored. Checking at
// commit time is what makes the log trustworthy.
const hashes = Array.isArray(body.hashes) ? (body.hashes as string[]) : [];
const missing: string[] = [];
for (const h of hashes) {
if (!HASH_REGEX.test(h) || !(await store.hasObject(vaultId, h))) missing.push(h);
}
if (missing.length > 0) {
const f = fail('MISSING_OBJECTS', `${missing.length} referenced object(s) missing`, 409);
return c.json({ ...f.body, missing: missing.slice(0, 20) }, f.status);
}
await store.appendLog(vaultId, {
seq: body.seq,
manifest: body.manifest,
at: typeof body.at === 'number' ? body.at : Date.now(),
bytes: await store.usage(vaultId),
});
return c.json({ seq: body.seq, head: body.seq });
});
return app;
}
export type { VaultStore } from './store.js';
export { MemoryVaultStore } from './store.js';
export type { VaultManifest };

View File

@@ -0,0 +1,90 @@
/**
* Relay-side storage for a vault.
*
* The interface is deliberately dumb: put bytes under a hash, list a log,
* append to it. Nothing here can decrypt, and nothing needs to — which is
* what makes a SQLite, Postgres or object-store implementation equivalent.
*/
import type { VaultLogEntry } from './types.js';
export interface VaultStore {
/** Ed25519 pubkey pinned on first write (TOFU), or null for a fresh vault. */
getOwner(vaultId: string): Promise<Uint8Array | null>;
setOwner(vaultId: string, publicKey: Uint8Array): Promise<void>;
hasObject(vaultId: string, hash: string): Promise<boolean>;
putObject(vaultId: string, hash: string, bytes: Uint8Array): Promise<void>;
getObject(vaultId: string, hash: string): Promise<Uint8Array | null>;
/** Highest committed seq, or 0 when empty. */
head(vaultId: string): Promise<number>;
/** Newest last. `limit` counts back from the head. */
log(vaultId: string, limit?: number): Promise<VaultLogEntry[]>;
appendLog(vaultId: string, entry: VaultLogEntry): Promise<void>;
/** Total stored bytes, for quota decisions. */
usage(vaultId: string): Promise<number>;
}
/**
* In-memory store. Real for tests, a footgun in production — the same shape
* as `MemoryBlobStore`, and the same warning applies: a restart loses
* everything, and it will do so without an error.
*/
export class MemoryVaultStore implements VaultStore {
private owners = new Map<string, Uint8Array>();
private objects = new Map<string, Map<string, Uint8Array>>();
private logs = new Map<string, VaultLogEntry[]>();
async getOwner(vaultId: string): Promise<Uint8Array | null> {
return this.owners.get(vaultId) ?? null;
}
async setOwner(vaultId: string, publicKey: Uint8Array): Promise<void> {
this.owners.set(vaultId, publicKey);
}
private bucket(vaultId: string): Map<string, Uint8Array> {
let b = this.objects.get(vaultId);
if (!b) {
b = new Map();
this.objects.set(vaultId, b);
}
return b;
}
async hasObject(vaultId: string, hash: string): Promise<boolean> {
return this.bucket(vaultId).has(hash);
}
async putObject(vaultId: string, hash: string, bytes: Uint8Array): Promise<void> {
this.bucket(vaultId).set(hash, bytes);
}
async getObject(vaultId: string, hash: string): Promise<Uint8Array | null> {
return this.bucket(vaultId).get(hash) ?? null;
}
async head(vaultId: string): Promise<number> {
const l = this.logs.get(vaultId);
return l && l.length > 0 ? l[l.length - 1]!.seq : 0;
}
async log(vaultId: string, limit?: number): Promise<VaultLogEntry[]> {
const l = this.logs.get(vaultId) ?? [];
return limit === undefined ? [...l] : l.slice(Math.max(0, l.length - limit));
}
async appendLog(vaultId: string, entry: VaultLogEntry): Promise<void> {
const l = this.logs.get(vaultId) ?? [];
l.push(entry);
this.logs.set(vaultId, l);
}
async usage(vaultId: string): Promise<number> {
let total = 0;
for (const bytes of this.bucket(vaultId).values()) total += bytes.length;
return total;
}
}

View File

@@ -0,0 +1,79 @@
/**
* The vault's data model.
*
* A vault is a collection of files that lives encrypted on the relay: the
* pieces Shade was missing to let an app offer backup, or to let a client
* read data while the peer that owns it is switched off.
*
* Three ideas, and the shape follows from them:
*
* 1. **Objects are content-addressed.** An object's name is the hash of its
* *ciphertext*, so the relay can store and dedupe without understanding
* anything. Re-uploading an unchanged file is free, which matters when a
* workspace of several hundred files changes three lines at a time.
*
* 2. **A manifest names the collection.** Paths live in the manifest, not in
* object names — the relay must not learn that a user has a file called
* `Projects/Divorce/plan.md`. The manifest is itself encrypted and stored
* as an object.
*
* 3. **The log is append-only.** Each commit adds an entry pointing at a
* manifest. History, rollback and "what changed last Tuesday" all fall out
* of that, which is the versioning half of the requirement.
*/
/** One file in a manifest. `hash` names the ciphertext object. */
export interface VaultEntry {
/** Path within the collection, as the app understands it. */
path: string;
/** Lowercase hex SHA-256 of the ciphertext object. */
hash: string;
/** Plaintext byte length, for progress reporting and sanity checks. */
size: number;
/** Optional app-defined mode/metadata, opaque to the vault. */
mode?: string;
}
/** The full contents of a collection at one point in time. */
export interface VaultManifest {
/** Schema version — a client that does not know it must refuse, not guess. */
version: 1;
/** Monotonic, assigned by the client and checked by the relay. */
seq: number;
/** Epoch millis when the commit was made. */
at: number;
/** Optional human note, e.g. "nattlig backup". */
message?: string;
entries: VaultEntry[];
}
/** One row of the append-only log. */
export interface VaultLogEntry {
seq: number;
/** Hash of the encrypted manifest object. */
manifest: string;
at: number;
/** Total bytes of all objects the manifest references, for quota display. */
bytes: number;
}
/** What `GET /v1/vault/:id/log` returns. */
export interface VaultLog {
entries: VaultLogEntry[];
/** Highest seq the relay holds; `0` when the vault is empty. */
head: number;
}
/**
* Error codes clients branch on.
*
* `SEQ_CONFLICT` is the important one: two devices committed from the same
* head, and the loser must re-read and re-commit rather than retry blindly.
*/
export type VaultErrorCode =
| 'BAD_REQUEST'
| 'UNAUTHORIZED'
| 'NOT_FOUND'
| 'SEQ_CONFLICT'
| 'TOO_LARGE'
| 'MISSING_OBJECTS';

View File

@@ -0,0 +1,153 @@
/**
* The whole backup chain, against real data.
*
* Talks to a running `scaffoldd` over its unix socket, asks what the backup
* set is, reads those files off disk, pushes them through the vault, and pulls
* them back — then compares byte-for-byte.
*
* This is the test that would catch the things unit tests cannot: a path that
* survives the round trip wrong, a 600 KB log that trips a size ceiling, a
* workspace whose file count makes the manifest too large to commit. It is
* skipped when no daemon is listening, so it never blocks an ordinary run.
*
* scaffoldd --socket /tmp/scaffoldd-e2e.sock &
* SCAFFOLDD_SOCKET=/tmp/scaffoldd-e2e.sock bun test scaffoldd-e2e
*/
import { describe, test, expect } from 'bun:test';
import { connect } from 'node:net';
import { readFile } from 'node:fs/promises';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { createVaultRoutes } from '../src/server.js';
import { MemoryVaultStore } from '../src/store.js';
import { HttpVaultTransport } from '../src/http-transport.js';
import { VaultClient, deriveVaultKeys } from '../src/client.js';
import type { VaultFile } from '../src/client.js';
const SOCKET = process.env.SCAFFOLDD_SOCKET;
const crypto = new SubtleCryptoProvider();
interface BackupFile {
path: string;
source: string;
size: number;
}
function ask(socketPath: string, request: object): Promise<any> {
return new Promise((resolve, reject) => {
const sock = connect(socketPath);
let buf = '';
const timer = setTimeout(() => {
sock.destroy();
reject(new Error('scaffoldd svarte ikke innen 20 s'));
}, 20_000);
// Deliberately no `end()` after writing: Bun's socket closes both
// directions, so the daemon's reply is discarded before it arrives. The
// protocol is line-delimited, so read until the first complete line and
// close from here instead.
sock.on('connect', () => {
sock.write(`${JSON.stringify(request)}\n`);
});
sock.on('data', (c) => {
buf += c.toString();
const nl = buf.indexOf('\n');
if (nl === -1) return;
clearTimeout(timer);
sock.destroy();
const res = JSON.parse(buf.slice(0, nl));
res.ok ? resolve(res.result) : reject(new Error(`${res.code}: ${res.error}`));
});
sock.on('end', () => {
clearTimeout(timer);
if (!buf.includes('\n')) reject(new Error('tomt svar'));
});
sock.on('error', (e) => {
clearTimeout(timer);
reject(e);
});
});
}
async function harness() {
const store = new MemoryVaultStore();
const routes = createVaultRoutes(store, crypto);
const transport = new HttpVaultTransport('http://vault.test', (i, init) =>
routes.fetch(new Request(i, init)),
);
const keys = await deriveVaultKeys(new Uint8Array(32).fill(42), 'scaffold-e2e');
return { store, client: new VaultClient(crypto, keys, transport) };
}
describe.skipIf(!SOCKET)('scaffoldd → vault, real workspace', () => {
test('the whole backup set round-trips byte-for-byte', async () => {
const set = (await ask(SOCKET!, { op: 'backupSet' })) as {
files: BackupFile[];
count: number;
bytes: number;
};
expect(set.count).toBeGreaterThan(50);
const files: VaultFile[] = [];
for (const f of set.files) {
try {
files.push({ path: f.path, bytes: new Uint8Array(await readFile(f.source)) });
} catch {
// Matches the bridge: one unreadable file is skipped, not fatal.
}
}
const { client } = await harness();
const push = await client.push(files, 1_786_600_000_000, 'e2e');
expect(push.seq).toBe(1);
expect(push.uploaded).toBe(files.length);
const { files: back } = await client.pull();
expect(back.size).toBe(files.length);
// Every single file, not a sample: a backup that restores 99% of a
// workspace is not a backup.
for (const f of files) {
const restored = back.get(f.path);
expect(restored).toBeDefined();
expect(restored!.length).toBe(f.bytes.length);
expect(Buffer.from(restored!).equals(Buffer.from(f.bytes))).toBe(true);
}
}, 120_000);
test('a second push after one edit sends almost nothing', async () => {
// The property that decides whether hourly backup is affordable.
const set = (await ask(SOCKET!, { op: 'backupSet' })) as { files: BackupFile[] };
const files: VaultFile[] = [];
for (const f of set.files.slice(0, 120)) {
try {
files.push({ path: f.path, bytes: new Uint8Array(await readFile(f.source)) });
} catch {
/* skipped */
}
}
const { client } = await harness();
const first = await client.push(files, 1_786_600_000_000);
const edited = files.map((f, i) =>
i === 0 ? { path: f.path, bytes: new TextEncoder().encode('endret én linje') } : f,
);
const second = await client.push(edited, 1_786_600_001_000);
expect(first.uploaded).toBe(files.length);
expect(second.uploaded).toBe(1);
expect(second.reused).toBe(files.length - 1);
}, 120_000);
test('the largest file in the workspace survives the round trip', async () => {
// Terra's log.md is ~617 KB. Nothing in the chain may quietly cap it.
const set = (await ask(SOCKET!, { op: 'backupSet' })) as { files: BackupFile[] };
const biggest = set.files.reduce((a, b) => (a.size > b.size ? a : b));
expect(biggest.size).toBeGreaterThan(100_000);
const bytes = new Uint8Array(await readFile(biggest.source));
const { client } = await harness();
await client.push([{ path: biggest.path, bytes }], 1_786_600_000_000);
const { files } = await client.pull();
expect(files.get(biggest.path)!.length).toBe(bytes.length);
}, 120_000);
});

View File

@@ -0,0 +1,313 @@
/**
* End-to-end tests for the vault.
*
* The client talks to the REAL route handlers through Hono's `fetch`, not to a
* mock. A mock transport would agree with the client by construction and prove
* nothing about the wire contract — which is exactly the surface a phone and a
* daemon have to share.
*/
import { describe, test, expect } from 'bun:test';
import { SubtleCryptoProvider } from '@shade/crypto-web';
import { createVaultRoutes } from '../src/server.js';
import { MemoryVaultStore } from '../src/store.js';
import { HttpVaultTransport } from '../src/http-transport.js';
import { VaultClient, deriveVaultKeys } from '../src/client.js';
import { objectHash } from '../src/crypto.js';
const crypto = new SubtleCryptoProvider();
const AT = 1_786_600_000_000;
function enc(s: string): Uint8Array {
return new TextEncoder().encode(s);
}
function dec(b: Uint8Array): string {
return new TextDecoder().decode(b);
}
/** A client wired to a fresh in-memory relay. */
async function harness(masterKey = new Uint8Array(32).fill(7), app = 'scaffold') {
const store = new MemoryVaultStore();
const routes = createVaultRoutes(store, crypto);
const transport = new HttpVaultTransport('http://vault.test', (input, init) =>
routes.fetch(new Request(input, init)),
);
const keys = await deriveVaultKeys(masterKey, app);
return { store, keys, client: new VaultClient(crypto, keys, transport) };
}
describe('round trip', () => {
test('files pushed can be pulled back byte-for-byte', async () => {
const { client } = await harness();
const files = [
{ path: '.scaffold/plan.md', bytes: enc('# Plan\n\n## Nå\n- [ ] noe\n') },
{ path: '.scaffold/tasks.yaml', bytes: enc('tasks: []\n') },
];
const res = await client.push(files, AT, 'første backup');
expect(res.seq).toBe(1);
expect(res.uploaded).toBe(2);
const { manifest, files: back } = await client.pull();
expect(manifest.seq).toBe(1);
expect(manifest.message).toBe('første backup');
expect(dec(back.get('.scaffold/plan.md')!)).toBe('# Plan\n\n## Nå\n- [ ] noe\n');
expect(dec(back.get('.scaffold/tasks.yaml')!)).toBe('tasks: []\n');
});
test('a fresh device with only the credentials can restore', async () => {
// The recovery story: same master key, nothing else carried over.
const master = new Uint8Array(32).fill(11);
const { client, store } = await harness(master);
await client.push([{ path: 'notes.yaml', bytes: enc('notes: [en, to]\n') }], AT);
const routes = createVaultRoutes(store, crypto);
const transport = new HttpVaultTransport('http://vault.test', (i, init) =>
routes.fetch(new Request(i, init)),
);
const keys = await deriveVaultKeys(master, 'scaffold');
const fresh = new VaultClient(crypto, keys, transport);
const { files } = await fresh.pull();
expect(dec(files.get('notes.yaml')!)).toBe('notes: [en, to]\n');
});
test('a different master key cannot read the vault', async () => {
const { store } = await harness(new Uint8Array(32).fill(1));
const routes = createVaultRoutes(store, crypto);
const transport = new HttpVaultTransport('http://vault.test', (i, init) =>
routes.fetch(new Request(i, init)),
);
const wrong = await deriveVaultKeys(new Uint8Array(32).fill(2), 'scaffold');
const intruder = new VaultClient(crypto, wrong, transport);
// A different master derives a different vaultId, so there is nothing
// there to read in the first place — the id is itself a secret.
await expect(intruder.pull()).rejects.toThrow();
});
});
describe('versioning', () => {
test('each push is a new version and old ones stay readable', async () => {
const { client } = await harness();
await client.push([{ path: 'plan.md', bytes: enc('versjon 1') }], AT);
await client.push([{ path: 'plan.md', bytes: enc('versjon 2') }], AT + 1000);
await client.push([{ path: 'plan.md', bytes: enc('versjon 3') }], AT + 2000);
const log = await client.history();
expect(log.head).toBe(3);
expect(log.entries.map((e) => e.seq)).toEqual([1, 2, 3]);
// Rollback: the whole point of keeping the log.
expect(dec((await client.pull(1)).files.get('plan.md')!)).toBe('versjon 1');
expect(dec((await client.pull(2)).files.get('plan.md')!)).toBe('versjon 2');
expect(dec((await client.pull()).files.get('plan.md')!)).toBe('versjon 3');
});
test('unchanged files are not re-uploaded', async () => {
// The property that makes backing up a 9 MB workspace on every change
// affordable: only what actually moved goes over the wire.
const { client } = await harness();
const stable = { path: 'stor-logg.md', bytes: enc('x'.repeat(50_000)) };
const first = await client.push([stable, { path: 'plan.md', bytes: enc('en') }], AT);
expect(first.uploaded).toBe(2);
const second = await client.push([stable, { path: 'plan.md', bytes: enc('to') }], AT + 1);
expect(second.uploaded).toBe(1);
expect(second.reused).toBe(1);
expect(second.bytesUploaded).toBeLessThan(1000);
// And the reused file is still intact in the new version.
const { files } = await client.pull();
expect(files.get('stor-logg.md')!.length).toBe(50_000);
});
test('a deleted file is absent from the new version but present in the old', async () => {
const { client } = await harness();
await client.push(
[
{ path: 'a.md', bytes: enc('A') },
{ path: 'b.md', bytes: enc('B') },
],
AT,
);
await client.push([{ path: 'a.md', bytes: enc('A') }], AT + 1);
expect((await client.pull()).files.has('b.md')).toBe(false);
expect(dec((await client.pull(1)).files.get('b.md')!)).toBe('B');
});
});
describe('the relay is blind', () => {
test('stored objects contain no plaintext', async () => {
const { client, store } = await harness();
await client.push([{ path: 'hemmelig/plan.md', bytes: enc('SENSITIVT INNHOLD') }], AT);
const log = await store.log(client.vaultId);
const manifestBytes = await store.getObject(client.vaultId, log[0]!.manifest);
const asText = dec(manifestBytes!);
// Neither the contents nor the path leaks: paths live inside the
// encrypted manifest, not in object names.
expect(asText).not.toContain('SENSITIVT');
expect(asText).not.toContain('hemmelig');
});
test('object names are the hash of the ciphertext, so the relay can verify', async () => {
const { client, store } = await harness();
await client.push([{ path: 'x', bytes: enc('hei') }], AT);
const log = await store.log(client.vaultId);
const bytes = await store.getObject(client.vaultId, log[0]!.manifest);
expect(objectHash(bytes!)).toBe(log[0]!.manifest);
});
test('an object whose bytes do not match its name is rejected', async () => {
const store = new MemoryVaultStore();
const routes = createVaultRoutes(store, crypto);
const keys = await deriveVaultKeys(new Uint8Array(32).fill(3), 'scaffold');
const { signPayload } = await import('@shade/server');
const { toBase64 } = await import('@shade/core');
const body = await signPayload(crypto, keys.signingSeed, {
data: toBase64(enc('juks')),
publicKey: toBase64(keys.publicKey),
});
const res = await routes.fetch(
new Request(`http://v/v1/vault/${keys.vaultId}/object/${'0'.repeat(64)}`, {
method: 'PUT',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
}),
);
expect(res.status).toBe(400);
expect((await res.json()).error.code).toBe('BAD_REQUEST');
});
});
describe('concurrent writers', () => {
test('a commit from a stale head is refused', async () => {
// Two devices push from the same version. The second must not be able to
// overwrite a sequence number that is already history.
const { client, keys, store } = await harness();
await client.push([{ path: 'plan.md', bytes: enc('en')}], AT);
const routes = createVaultRoutes(store, crypto);
const { signPayload } = await import('@shade/server');
const { toBase64 } = await import('@shade/core');
const log = await store.log(keys.vaultId);
const body = await signPayload(crypto, keys.signingSeed, {
manifest: log[0]!.manifest,
seq: 1, // already taken
at: AT,
hashes: [],
publicKey: toBase64(keys.publicKey),
});
const res = await routes.fetch(
new Request(`http://v/v1/vault/${keys.vaultId}/commit`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
}),
);
expect(res.status).toBe(409);
const err = await res.json();
expect(err.error.code).toBe('SEQ_CONFLICT');
expect(err.head).toBe(1);
});
test('a commit referencing a missing object is refused', async () => {
// Otherwise the log would publish a version that cannot be restored.
const { client, keys, store } = await harness();
await client.push([{ path: 'plan.md', bytes: enc('en') }], AT);
const routes = createVaultRoutes(store, crypto);
const { signPayload } = await import('@shade/server');
const { toBase64 } = await import('@shade/core');
const log = await store.log(keys.vaultId);
const body = await signPayload(crypto, keys.signingSeed, {
manifest: log[0]!.manifest,
seq: 2,
at: AT,
hashes: ['a'.repeat(64)],
publicKey: toBase64(keys.publicKey),
});
const res = await routes.fetch(
new Request(`http://v/v1/vault/${keys.vaultId}/commit`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
}),
);
expect(res.status).toBe(409);
expect((await res.json()).error.code).toBe('MISSING_OBJECTS');
});
});
describe('authorisation', () => {
test('a second key cannot write to a vault another key pinned', async () => {
const { client, keys, store } = await harness();
await client.push([{ path: 'plan.md', bytes: enc('mitt') }], AT);
const routes = createVaultRoutes(store, crypto);
const { signPayload } = await import('@shade/server');
const { toBase64 } = await import('@shade/core');
const attacker = await deriveVaultKeys(new Uint8Array(32).fill(9), 'scaffold');
// Signed correctly — but by the wrong key, and asserting its own pubkey.
const body = await signPayload(crypto, attacker.signingSeed, {
data: toBase64(enc('tull')),
publicKey: toBase64(attacker.publicKey),
});
const res = await routes.fetch(
new Request(
`http://v/v1/vault/${keys.vaultId}/object/${objectHash(enc('tull'))}`,
{
method: 'PUT',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
},
),
);
expect(res.status).toBe(401);
});
test('an unsigned write is refused', async () => {
const store = new MemoryVaultStore();
const routes = createVaultRoutes(store, crypto);
const res = await routes.fetch(
new Request(`http://v/v1/vault/${'a'.repeat(64)}/object/${'b'.repeat(64)}`, {
method: 'PUT',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ data: 'aGk=' }),
}),
);
expect(res.status).toBe(401);
});
});
describe('key derivation', () => {
test('the same credentials derive the same vault, different ones do not', async () => {
const a = await deriveVaultKeys(new Uint8Array(32).fill(4), 'scaffold');
const b = await deriveVaultKeys(new Uint8Array(32).fill(4), 'scaffold');
const c = await deriveVaultKeys(new Uint8Array(32).fill(5), 'scaffold');
expect(a.vaultId).toBe(b.vaultId);
expect(a.vaultId).not.toBe(c.vaultId);
});
test('two apps under one master do not share a vault', async () => {
const master = new Uint8Array(32).fill(6);
const scaffold = await deriveVaultKeys(master, 'scaffold');
const mail = await deriveVaultKeys(master, 'mail');
expect(scaffold.vaultId).not.toBe(mail.vaultId);
expect(scaffold.contentKey).not.toEqual(mail.contentKey);
});
test('the vault branch is separate from the profile-blob branch', async () => {
// A vault key reads every file; a profile-blob key reads a host list.
// Sharing a derivation would make one compromise into the other.
const master = new Uint8Array(32).fill(8);
const { deriveBlobKey } = await import('@shade/storage-encrypted');
const vault = await deriveVaultKeys(master, 'prism');
expect(vault.contentKey).not.toEqual(deriveBlobKey(master, 'prism'));
});
});

View File

@@ -0,0 +1,5 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": { "outDir": "dist", "rootDir": "src", "lib": ["ES2022", "DOM"] },
"include": ["src"]
}

View File

@@ -1,6 +1,6 @@
{
"name": "@shade/widgets",
"version": "4.11.1",
"version": "4.13.0",
"type": "module",
"main": "src/index.ts",
"types": "src/index.ts",

View File

@@ -13,6 +13,7 @@
*
* Optional:
* DRY_RUN=1 — pack tarballs but do not publish (no token required)
* --only <pkg> — publish one package (e.g. shade-files or @shade/files)
*/
import { readFileSync, writeFileSync, existsSync } from 'fs';
import { join } from 'path';
@@ -37,6 +38,7 @@ const PACKAGES = [
'shade-transport-webrtc',
'shade-server',
'shade-inbox-server',
'shade-vault',
'shade-inbox',
'shade-transfer',
'shade-files',
@@ -51,10 +53,27 @@ const PACKAGES = [
const REGISTRY_HOST = 'gt.zyon.no';
const ROOT = join(import.meta.dir, '..');
function selectedPackages(argv: string[]): string[] {
const onlyIndex = argv.indexOf('--only');
if (onlyIndex === -1) return PACKAGES;
const requested = argv[onlyIndex + 1];
if (!requested) throw new Error('Usage: publish-all.ts --only <shade-files|@shade/files>');
const normalized = requested.startsWith('@shade/')
? `shade-${requested.slice('@shade/'.length)}`
: requested.startsWith('shade-')
? requested
: `shade-${requested}`;
if (!PACKAGES.includes(normalized)) {
throw new Error(`Unknown Shade package: ${requested}`);
}
return [normalized];
}
async function main() {
const token = process.env.GITEA_TOKEN;
const user = process.env.GITEA_USER ?? 'Stian';
const dryRun = process.env.DRY_RUN === '1';
const packagesToPublish = selectedPackages(process.argv.slice(2));
if (!token && !dryRun) {
console.error('GITEA_TOKEN is required (or set DRY_RUN=1)');
@@ -64,6 +83,7 @@ async function main() {
const registryUrl = `https://${REGISTRY_HOST}/api/packages/${user}/npm/`;
console.log(`Target registry: ${registryUrl}`);
console.log(`Dry run: ${dryRun ? 'yes' : 'no'}`);
console.log(`Packages: ${packagesToPublish.join(', ')}`);
console.log();
const npmrcPath = join(ROOT, '.npmrc.publish');
@@ -86,7 +106,7 @@ async function main() {
let alreadyPublished = 0;
let failed = 0;
for (const pkg of PACKAGES) {
for (const pkg of packagesToPublish) {
const pkgDir = join(ROOT, 'packages', pkg);
const pkgJsonPath = join(pkgDir, 'package.json');
if (!existsSync(pkgJsonPath)) {

View File

@@ -23,6 +23,7 @@ PACKAGES=(
transport-webrtc
server
inbox-server
vault
inbox
transfer
files

View File

@@ -18,6 +18,16 @@ import { readdirSync, statSync, existsSync } from 'fs';
import { join } from 'path';
import { $ } from 'bun';
/**
* The pinned compiler, not whatever `bunx` resolves today.
*
* `bunx tsc` fetches the newest release on every run, so this gate could go
* red — or, worse, quietly stop catching things — because TypeScript shipped,
* not because the repo changed. The version is pinned in devDependencies and
* resolved from node_modules here.
*/
const TSC = join(import.meta.dir, '..', 'node_modules', '.bin', 'tsc');
const ROOT = join(import.meta.dir, '..');
const PACKAGES_DIR = join(ROOT, 'packages');
@@ -38,7 +48,7 @@ const failed: { pkg: string; out: string }[] = [];
for (const pkg of packages) {
const dir = join(PACKAGES_DIR, pkg);
const proc = Bun.spawnSync(['bunx', 'tsc', '--noEmit', '-p', 'tsconfig.json'], {
const proc = Bun.spawnSync([TSC, '--noEmit', '-p', 'tsconfig.json'], {
cwd: dir,
stdout: 'pipe',
stderr: 'pipe',
@@ -73,7 +83,7 @@ if (filter.size === 0) {
console.log('Consumer-strict smoke (lib: DOM, exactOptional, paths→workspace) ...');
const consumerDir = join(ROOT, 'tests', 'consumer-strict');
if (existsSync(join(consumerDir, 'tsconfig.json'))) {
const proc = Bun.spawnSync(['bunx', 'tsc', '--noEmit', '-p', 'tsconfig.json'], {
const proc = Bun.spawnSync([TSC, '--noEmit', '-p', 'tsconfig.json'], {
cwd: consumerDir,
stdout: 'pipe',
stderr: 'pipe',

View File

@@ -14,7 +14,6 @@
"noEmit": true,
"types": ["bun-types"],
"ignoreDeprecations": "6.0",
"baseUrl": ".",
"paths": {
"@shade/core": ["../../packages/shade-core/src/index.ts"],
"@shade/proto": ["../../packages/shade-proto/src/index.ts"],