Compare commits
11 Commits
v4.11.0
...
40286a8542
| Author | SHA1 | Date | |
|---|---|---|---|
| 40286a8542 | |||
| cc6b1d53d7 | |||
| d8f6da55ef | |||
| 3da977207c | |||
| 84d3166ca1 | |||
| 012d7f5289 | |||
| 96c20cb4b2 | |||
| b44acf867b | |||
| 8c67a00f37 | |||
| 306bf08452 | |||
| 3243647aa1 |
@@ -28,6 +28,13 @@ jobs:
|
|||||||
- name: Install dependencies
|
- name: Install dependencies
|
||||||
run: ~/.bun/bin/bun install --frozen-lockfile
|
run: ~/.bun/bin/bun install --frozen-lockfile
|
||||||
|
|
||||||
|
# Before the tests, not after: a type error is cheaper to read than the
|
||||||
|
# runtime failure it eventually causes. The compiler is pinned in
|
||||||
|
# devDependencies, so this gate goes red when the repo changes — not
|
||||||
|
# when TypeScript ships a release.
|
||||||
|
- name: Typecheck
|
||||||
|
run: ~/.bun/bin/bun run typecheck
|
||||||
|
|
||||||
- name: Run tests
|
- name: Run tests
|
||||||
env:
|
env:
|
||||||
SHADE_TEST_PG_URL: postgres://postgres:test@localhost:5432/postgres
|
SHADE_TEST_PG_URL: postgres://postgres:test@localhost:5432/postgres
|
||||||
|
|||||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -5,3 +5,6 @@ dist/
|
|||||||
**/.tmp-*.db
|
**/.tmp-*.db
|
||||||
**/.tmp-*.db-shm
|
**/.tmp-*.db-shm
|
||||||
**/.tmp-*.db-wal
|
**/.tmp-*.db-wal
|
||||||
|
|
||||||
|
# Byggeartefakt fra @shade/observer (outDir-avviket i G0)
|
||||||
|
dist-build/
|
||||||
|
|||||||
24
.scaffold/log.md
Normal file
24
.scaffold/log.md
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
# Logg — Shade
|
||||||
|
|
||||||
|
## 2026-07-10 — Holistisk helsesjekk + gated plan mot V5.0
|
||||||
|
Fem parallelle analysepass (arkitektur, test/CI, kryptosikkerhet, Android-paritet, docs/DX), topp-funn kildeverifisert. Kjernen er reell og moden (1159 tester grønne, vektorer byte-pinnet, KT/recovery ekte implementasjoner) — men «GA-frosset og revisjonsklar» stemte ikke. Fire P0-blockere identifisert og lagt inn som blockers i plan.md + kort i tasks.yaml:
|
||||||
|
1. **AES-GCM nonce-gjenbruk** i at-rest storage (`storage-encrypted/src/crypto/kdf.ts:116`) — deterministisk nonce uten meldingsteller, `saveSession` re-krypterer hver ratchet-op → GCM forbidden attack mot §4. Reachable bekreftet; fiks er liten (tilfeldig nonce, wire lagrer den allerede i prefiks).
|
||||||
|
2. **@shade/observer@4.11.1 tomt på registry** — `outDir=dist-build` men publish shipper `dist/`; `dist-build/` er det untrackede git-avviket.
|
||||||
|
3. **Typecheck-gaten rød** — TS upinnet → `bunx tsc` henter 7.0.2 (fjernet `baseUrl`, brukt i consumer-strict); gaten kjøres aldri i CI.
|
||||||
|
4. **@shade/cli bin dingler** etter publish (bin ikke omskrevet til dist/).
|
||||||
|
|
||||||
|
P1: safety-number matcher aldri peerens eget (`session.ts:241`, MITM-forsvar ikke-funksjonelt); backup-KDF HKDF ikke argon2id (finnes allerede i repoet); hono <4.12.18; begge flaggskip-eksempler røde (accept-race `shade-transfer/engine.ts:1140`). Paritet: ShadeStream (v4.11) uvoktet selv på TS-siden, Kotlin stopper på v4.10.
|
||||||
|
|
||||||
|
Låst prinsipp: **V5.0-implementasjon er gated på G0–G4 grønne.** V5.0.md er idéskisse — modnes parallelt (wire-typer reconciled mot 0x31–0x33, Insertable Streams-mekanisme, ratchet-params). Full rapport: helsesjekk-artifact (claude.ai/code).
|
||||||
|
|
||||||
|
## 2026-07-09 — Adoptert Scaffold plan-kontrakt
|
||||||
|
Migrerte `docs/ROADMAP.md` og `android/shade-android/ROADMAP-ANDROID.md` til `.scaffold/`: nåbildet destillert inn i `plan.md`, originalene arkivert under `.scaffold/archive/` med opprinnelig mappestruktur. Versjonsdesigndokumentene (`docs/V5.0.md`, `docs/archive/V*.md`) er bevisst latt stå — de er spesifikasjoner, ikke planfiler.
|
||||||
|
|
||||||
|
## 2026-05-15 — v4.11: streaming Double-Ratchet sub-sessions
|
||||||
|
Siste post-GA-økning på 4.x-linjen: `ShadeStream` gir in-memory sub-sessions (seal/open uten keystore-I/O per frame) med nye wire-typer 0x31–0x33. Låst beslutning: stream-ratchets persisteres aldri — en droppet stream gjenåpnes, aldri gjenopptas.
|
||||||
|
|
||||||
|
## 2026-05-09 — Android M-Cross 1–4 fullført + Keystore-adapter
|
||||||
|
All kryptografisk paritet TS↔Kotlin grønn via delte test-vektorer: KDF-kjede, HKDF-labels, X3DH, ratchet-steg, fingerprint, wire 0x02 og 0x11 (streams), backup-HKDF, group sender-keys og storage-HKDF. Etterslepet fra Android-roadmapen landet samme dag: scrypt + argon2id via Bouncy Castle og `shade-android-keystore`-modulen (hardware-backed master key + `KeystoreStorage`). Gjenstår: socket-interop-test og instrumenterte Keystore-tester.
|
||||||
|
|
||||||
|
## 2026-05-03 — V4.0 GA: V3.x-konsolidering og audit-pakking
|
||||||
|
Alle fasene V3.1–V3.12 ferdige og merget: dokumentasjon/hardening, at-rest storage-kryptering, trust-UX, observability (OTel), Android-paritet, inbox (store-and-forward), transport-bridge, web workers-krypto, filmetadata, social key recovery, WebRTC P2P og key transparency. Wire-formatet låst — uendret fra 0.4.x, så 4.0-peers interopererer byte-for-byte med 0.4.x. Kjernen pakket for ekstern review. Låst beslutning: alt VOIP/video skilt ut til V5.0, bygget oppå den frosne 4.0-stacken via reserverte envelope-typer (ikke breaking).
|
||||||
41
.scaffold/logs/2026-08-13-gcm-nonce-reuse-fikset.md
Normal file
41
.scaffold/logs/2026-08-13-gcm-nonce-reuse-fikset.md
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
---
|
||||||
|
title: "G0: AES-GCM nonce-gjenbruk fjernet fra at-rest-lagringen"
|
||||||
|
date: 2026-08-13
|
||||||
|
author: Claude
|
||||||
|
---
|
||||||
|
|
||||||
|
Den første av de fire P0-ene fra helsesjekken 10. juli er borte. `row-codec` forsegler nå med tilfeldig nonce i stedet for en avledet.
|
||||||
|
|
||||||
|
## Hva som var galt
|
||||||
|
|
||||||
|
`deriveNonce()` laget nonce som en ren funksjon av `(fieldKey, table, pk)` — altså av radens identitet. Doc-kommentaren argumenterte for at det var trygt fordi «hver `(key, plaintext)`-kombinasjon opptrer høyst én gang», men det er nøyaktig omvendt av hva AES-GCM krever: det farlige er å gjenbruke `(key, nonce)` på tvers av *ulike* plaintekster. `saveSession` re-forsegler sesjonsraden ved hvert ratchet-steg, så paret gjentok seg per konstruksjon. Konsekvensen er ikke bare XOR av plaintekstene, men lekkasje av GHASH-subnøkkelen — «the forbidden attack» — som gir tag-forfalskning under den nøkkelen. Det gjaldt alle muterbare rader: sesjoner, config, prekey-state og trust.
|
||||||
|
|
||||||
|
## Fiksen
|
||||||
|
|
||||||
|
Fire kallesteder, som kartlagt: `sealString` og `sealBytes` (`row-codec.ts:67`/`:96`) tar nå `randomNonce()` fra `aead.ts`, og `openString`/`openBytes` (`:83`/`:112`) slutter å sende `expectedNonce`.
|
||||||
|
|
||||||
|
Tilfeldig ble valgt framfor en teller fordi codec-en er delt mellom SQLite og Postgres og ikke har noen varig per-rad skrivteller å støtte seg på. 96 bit holder collision-sannsynligheten neglisjerbar langt forbi et realistisk antall re-lagringer.
|
||||||
|
|
||||||
|
`deriveNonce` er beholdt og markert `@deprecated` framfor slettet — den er del av den publiserte flaten via både `index.ts` og `crypto.ts`. Ingen kaller den lenger inne i pakken, og et søk gjennom `packages/`, `android/` og `examples/` fant ingen andre konsumenter. `expectedNonce`-parameteren i `aeadOpen` er beholdt på samme vilkår, med dokumentasjon om hvorfor den ikke skal brukes: den ga ingen tuklingsdeteksjon som AEAD-taggen og `(table, column, pk)`-AAD-en ikke allerede gir.
|
||||||
|
|
||||||
|
`randomNonce` er eksportert fra `index.ts` og `crypto.ts`, så konsumenter som forsegler selv har den trygge kilden for hånden.
|
||||||
|
|
||||||
|
## Ingen migrering trengs
|
||||||
|
|
||||||
|
`aeadOpen` har alltid lest nonce fra blob-prefikset; `expectedNonce` var kun en valgfri ekstrasjekk. Eksisterende blobber åpnes derfor uendret etter fiksen. Todoen om en migreringssti er erstattet av en test som beviser det.
|
||||||
|
|
||||||
|
## Android var aldri rammet
|
||||||
|
|
||||||
|
Kryssplattform-vektoren «Storage HKDF: rowNonce» tester HKDF-funksjonen, ikke at forseglingen bruker den, så den består uendret — `deriveNonce` er fortsatt der med samme oppførsel. Og `KeystoreStorage` henter nonce fra `cipher.iv`, altså Android Keystores egen tilfeldige IV, som ikke lar seg overstyre. TS-siden er nå på linje med Kotlin, ikke omvendt.
|
||||||
|
|
||||||
|
## Verifisering
|
||||||
|
|
||||||
|
Ny `tests/nonce-reuse.test.ts` med ni tester i tre grupper: nonce-unikhet (inkludert 200 re-lagringer med 200 distinkte nonces), bakoverkompatibilitet (blob forseglet med den gamle avledningen åpnes, og nye og gamle blobber leses under samme nøkkel), og at tuklingsdeteksjonen overlevde — flippet nonce-byte og flyttet rad avvises fortsatt.
|
||||||
|
|
||||||
|
Testene ble bevist å ha verdi ved å reversere fiksen midlertidig: nøyaktig de tre som beskriver feilen falt, mens bakoverkompatibilitets- og tuklingstestene sto uendret.
|
||||||
|
|
||||||
|
87 tester grønne i `@shade/storage-encrypted`, typecheck ren, og de to avhengige pakkene kjørt: `@shade/cli` 15/15 og `@shade/sdk` 104/104. En enkelt sdk-feil i første kjøring var en flaky gjennomstrømningsmåling — tre påfølgende kjøringer er grønne.
|
||||||
|
|
||||||
|
## Ikke publisert
|
||||||
|
|
||||||
|
Fiksen ligger i kildetreet. Prism er pinnet på `@shade/storage-encrypted@4.12.0` og får den først ved en release, som er riktig rekkefølge: `prism-daemon` skal ikke restartes nå.
|
||||||
41
.scaffold/logs/2026-08-14-shade-vault.md
Normal file
41
.scaffold/logs/2026-08-14-shade-vault.md
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
---
|
||||||
|
title: "@shade/vault — server-side kryptert fillager (V4.13)"
|
||||||
|
date: 2026-08-14
|
||||||
|
author: Claude
|
||||||
|
---
|
||||||
|
|
||||||
|
Shade kunne flytte filer mellom peers (`@shade/files`) og lagre én liten profil-blob per konto (`/v1/blob/<slotId>`), men hadde ingen alltid-på lagring av krypterte filer. Uten den kan ingen Shade-app tilby backup, og ingen klient kan lese data mens peeren som eier dem er avslått. `@shade/vault` fyller det hullet.
|
||||||
|
|
||||||
|
## Modellen
|
||||||
|
|
||||||
|
Tre ideer, og formen følger av dem.
|
||||||
|
|
||||||
|
**Objekter er innholdsadresserte.** Et objekts navn er SHA-256 av *ciphertexten*, så relayen kan lagre, deduplisere og verifisere uten å forstå noe. Den regner om hashen ved opplasting og avviser et objekt som ikke er det det utgir seg for — uten å eie en nøkkel.
|
||||||
|
|
||||||
|
**Et manifest navngir samlingen.** Stier bor i manifestet, ikke i objektnavn: relayen skal ikke lære at brukeren har en fil som heter `Projects/Skilsmisse/plan.md`. Manifestet er selv kryptert og lagret som et objekt.
|
||||||
|
|
||||||
|
**Loggen er append-only.** Hver commit legger til en rad som peker på et manifest. Historikk, rollback og «hva endret seg sist tirsdag» faller ut av det — versjoneringshalvdelen av bestillingen.
|
||||||
|
|
||||||
|
## Valg verdt å begrunne
|
||||||
|
|
||||||
|
*Egen HKDF-gren* (`shade-vault-{id,content,sig}-v1`) ved siden av `shade-blob-*-v1`. En vault-nøkkel leser hver eneste fil; en profil-blob-nøkkel leser en vertsliste. Delt derivasjon ville gjort ett kompromiss om til det andre.
|
||||||
|
|
||||||
|
*Ingen konvergent kryptering.* Identisk klartekst gir ulike objektnavn fordi nonce er fersk. Dedupliseringen skjer derfor innenfor en versjonskjede — en uendret fil beholder hashen sin fordi klienten gjenbruker objektet den allerede lastet opp — ikke på tvers av uavhengige forseglinger. Konvergent kryptering ville lekket hvilke filer to brukere deler, som er nøyaktig det en blind butikk ikke skal.
|
||||||
|
|
||||||
|
*Commit verifiserer at objektene finnes.* Ellers ville loggen kunne publisere en versjon som ikke lar seg gjenopprette, og det er verdt en ekstra rundtur å utelukke.
|
||||||
|
|
||||||
|
*Pubkeyen er inne i signaturen.* Første forsøk la den på utenfor, som både brøt `verifyPayload` (den kanonikaliserer alle felt) og — hvis skjemaet hadde ignorert det — ville latt hvem som helst bytte identitet i transit på den TOFU-pinnende førsteskrivingen. Fanget av testene.
|
||||||
|
|
||||||
|
## Verifisering
|
||||||
|
|
||||||
|
16 tester, alle mot de ekte rutehåndtererne gjennom Honos `fetch` — ikke mot en mock-transport, som ville vært enig med klienten per konstruksjon og ikke bevist noe om wire-kontrakten.
|
||||||
|
|
||||||
|
Dekket: rundtur bit-for-bit; gjenoppretting på en fersk klient som bare har kontonøkkelen; feil master-nøkkel når ikke fram; tre versjoner med rollback til hver enkelt; en 50 KB uendret fil lastes ikke opp på nytt (`bytesUploaded` under 1 KB på andre push); slettet fil borte i ny versjon, intakt i gammel; lagrede objekter inneholder verken innhold eller sti; feilnavngitt objekt avvist; `SEQ_CONFLICT` ved commit fra utdatert head; `MISSING_OBJECTS` ved manglende referanse; fremmed nøkkel avvist mot en pinnet vault; usignert skriving avvist; og at vault-grenen er forskjellig fra blob-grenen.
|
||||||
|
|
||||||
|
Typecheck ren. `@shade/storage-encrypted` fortsatt 87/87 etter endringene i `kdf.ts`.
|
||||||
|
|
||||||
|
## Gjenstår
|
||||||
|
|
||||||
|
Ruting i `shade-server/src/standalone.ts` ved siden av `createBlobRoutes`, en varig `VaultStore` (SQLite/Postgres — bare `MemoryVaultStore` finnes), `docs/vault.md`, og deploy. Ingenting av det er gjort: deploy og publisering er utenfor mandatet for en uovervåket økt.
|
||||||
|
|
||||||
|
Arbeidet ligger ukommittert i arbeidstreet, sammen med de eksisterende endringene fra juli.
|
||||||
25
.scaffold/logs/2026-08-14-typecheck-gaten.md
Normal file
25
.scaffold/logs/2026-08-14-typecheck-gaten.md
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
---
|
||||||
|
title: "G0: typecheck-gaten pinnet og lagt i CI"
|
||||||
|
date: 2026-08-14
|
||||||
|
author: Claude
|
||||||
|
---
|
||||||
|
|
||||||
|
Tredje av de fire P0-ene fra helsesjekken 10. juli.
|
||||||
|
|
||||||
|
## Hva som faktisk var galt
|
||||||
|
|
||||||
|
Helsesjekken beskrev gaten som rød fordi TS 7.0.2 hadde fjernet `baseUrl`, som `consumer-strict` brukte. Det stemmer ikke lenger: `baseUrl` finnes ikke noe sted i repoet, og `bun run typecheck` er grønn for alle 26 pakker — inkludert den nye `shade-vault`.
|
||||||
|
|
||||||
|
Den ekte feilen står likevel, og er den viktigere av de to: `scripts/typecheck-all.ts` kjørte `bunx tsc`, som henter nyeste utgivelse ved hver kjøring. Gaten kunne altså gå rød — eller stille slutte å fange ting — fordi TypeScript hadde sluppet en versjon, ikke fordi repoet endret seg. Det er en gate som ikke måler det den utgir seg for å måle.
|
||||||
|
|
||||||
|
## Fiksen
|
||||||
|
|
||||||
|
`typescript` er pinnet til `7.0.2` i rotens `devDependencies`, og `typecheck-all.ts` kjører `node_modules/.bin/tsc` i stedet for `bunx`. Et `Typecheck`-steg er lagt inn i `.gitea/workflows/test.yml`, plassert før testene: en typefeil er billigere å lese enn kjøretidsfeilen den til slutt forårsaker.
|
||||||
|
|
||||||
|
## Verifisering
|
||||||
|
|
||||||
|
`bun run typecheck` med den pinnede kompilatoren: alle 26 pakker grønne, inkludert consumer-strict-røyktesten (`lib: DOM`, `exactOptionalPropertyTypes`, `paths → workspace`).
|
||||||
|
|
||||||
|
## Gjenstår i G0
|
||||||
|
|
||||||
|
De to siste P0-ene — `@shade/observer` tomt på registry og `@shade/cli` sin dinglende `bin` — krever begge publisering til npm. Det er en utadvendt handling, og ble ikke gjort i en uovervåket økt.
|
||||||
61
.scaffold/plan.md
Normal file
61
.scaffold/plan.md
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
---
|
||||||
|
project: "Shade"
|
||||||
|
phase: "Post-4.x GA — hardening til revisjonsklar (G0–G4) før V5.0-design"
|
||||||
|
updated: 2026-08-14
|
||||||
|
focus: "GCM-nonce-fiksen er levert (2026-08-13) og @shade/vault står ferdig og testet (2026-08-14), men er IKKE deployet eller publisert. De tre gjenstående G0-P0-ene er publiserings- og CI-saker. Historikk: G0 fikk en andre bestiller: @shade/vault (server-side kryptert fillager, bestilt 2026-08-13 av Scaffold-backupen) bygger på at-rest-stacken, og skal ikke reise seg på en kjent GCM-svakhet. Nonce-fiksens migreringssti er verifisert gratis — aeadOpen leser alltid nonce fra prefiks, så eksisterende blobber åpnes uendret."
|
||||||
|
blockers:
|
||||||
|
- "P0: @shade/observer@4.11.1 tomt på registry (outDir=dist-build, publish shipper dist/)"
|
||||||
|
- "P0: @shade/cli bin dingler etter publish (bin ikke omskrevet til dist/)"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Plan — Shade
|
||||||
|
|
||||||
|
## Mål
|
||||||
|
- E2EE-bibliotek som implementerer Signal-protokollen (X3DH + Double Ratchet) for TypeScript/Bun — drop-in for frontend, backend og mobil, med forward secrecy og post-compromise recovery.
|
||||||
|
- Byte-for-byte kryssplattform-paritet: Kotlin/Android-porten verifiseres kontinuerlig mot TS-referansen via delte test-vektorer i `test-vectors/`.
|
||||||
|
- Reelt revisjonsklar 4.x-kjerne som fundament (G0–G4 grønne); sanntid (V5.0) bygges oppå den låste stacken uten å røre kjernekrypto-revisjonen.
|
||||||
|
|
||||||
|
## Nå
|
||||||
|
Gated hardening — hard utgangsport per steg (full plan + funn: helsesjekk-artifact 2026-07-10, se log.md).
|
||||||
|
- [ ] **G0 — Stopp blødningen (P0):** ~~tilfeldig nonce per skriving i
|
||||||
|
storage-encrypted~~ (gjort 2026-08-13 — 9 nye tester, ingen migrering nødvendig,
|
||||||
|
Android var aldri rammet; se `logs/2026-08-13-gcm-nonce-reuse-fikset.md`.
|
||||||
|
**Ikke publisert** — Prism er pinnet på 4.12.0); observer `outDir→dist` +
|
||||||
|
republiser 4.11.2 + gitignore `dist-build/`; ~~pin `typescript` + typecheck i CI~~
|
||||||
|
(gjort 2026-08-14 — pinnet 7.0.2, `typecheck-all.ts` bruker lokal binær i
|
||||||
|
stedet for `bunx` latest, steg lagt i `test.yml`; `baseUrl` var allerede
|
||||||
|
borte og alle 26 pakker er grønne); fiks `bin`-omskriving + tarball-verifisering i publish.
|
||||||
|
Port: typecheck grønn i CI, nonce-negativtest grønn, `bun pm pack` inneholder
|
||||||
|
main-fil for alle pakker.
|
||||||
|
- [ ] **G1 — Revisjonsklar kjerne (P1):** peer-signeringsnøkkel per sesjon → symmetrisk safety-number; argon2id i backup-KDF (TS+Kotlin); hono ≥4.12.18; håndhevende §4 at-rest-tuklingstest; bestill ekstern krypto-review + pentest. Port: `alice.verify(bob, bob.getIdentityFingerprint())`===true, `bun audit` ren.
|
||||||
|
- [ ] **G2 — CI som mur:** biome + coverage-terskel + soak:smoke i CI; fjern/synk `.github/workflows`; publish kjører `prepublish:check`; bryt server↔inbox-server-syklus; rydd sdk/cli/files-deps; `dashboard: private`; pin bun. Port: PR blokkeres ved brudd, alle 8 examples grønne i CI.
|
||||||
|
|
||||||
|
## Neste
|
||||||
|
- **V4.13 — `@shade/vault`: server-side kryptert fillager.** Klient, ruter,
|
||||||
|
`MemoryVaultStore`, `SqliteVaultStore`, ruting i `standalone.ts` og
|
||||||
|
`docs/vault.md` er levert natt til 2026-08-14 (auto-mode); 16 + 7 tester
|
||||||
|
grønne, og kjeden er kjørt mot en ekte HTTP-server med et ekte workspace:
|
||||||
|
491 filer bit-identiske etter gjenoppretting, andre push sendte 0 KB.
|
||||||
|
Gjenstår: Postgres-store, rate limiting på vault-rutene, GC av objekter
|
||||||
|
ingen manifest lenger refererer — og deploy. Bestilt 2026-08-13 av
|
||||||
|
Scaffold-backupen, men er en generell mangel: Shade kan flytte filer mellom peers
|
||||||
|
(`@shade/files`) og lagre én liten profil-blob per konto (`/v1/blob/<slotId>`, V4.9),
|
||||||
|
men har ingen **alltid-på lagring av krypterte filer på relayen**. Hver konsument
|
||||||
|
docs/files.md nevner (Dispatch, Mail, Drive-style apps) trenger det samme, så det
|
||||||
|
hører i Shade og ikke i den enkelte appen.
|
||||||
|
- *Form:* innholdsadressert lager — klienten krypterer og navngir etter hash av
|
||||||
|
ciphertext, serveren ser aldri klartekst og trenger ingen forståelse av innholdet.
|
||||||
|
- *Historikk:* append-only oplog per samling gir versjonering og rollback gratis;
|
||||||
|
det er «mini github»-halvdelen av Scaffold-bestillingen.
|
||||||
|
- *Nøkler:* egen HKDF-gren ved siden av `shade-blob-*-v1`, så et vault-kompromiss
|
||||||
|
ikke rører profil-slotten. Recovery er kontocredentials, som blob-primitivet —
|
||||||
|
med det forbeholdet at gjenopprettingsstyrken da er passordstyrken (se G1s
|
||||||
|
argon2id-punkt, som løfter nettopp den).
|
||||||
|
- *Gated på G0:* lageret bygger på at-rest-stacken, og skal ikke reise seg på en
|
||||||
|
kjent GCM forbidden attack.
|
||||||
|
- **G3 — Paritetslås:** ShadeStream-vektorer (3-DH, wire 0x31–0x33) konsumert av begge sider; Kotlin-port av v4.11; utvid cross-vectors path-filter; `PARITY-VERSION`-sjekk; symmetriser alle 14 vektorfiler; Robolectric-tester for KeystoreStorage; E2E socket-interop TS↔Kotlin. Port: begge sider verifiserer alle vektorer, minor-bump uten Android-ack ⇒ rød CI. Vilkår for «production»-label på Android.
|
||||||
|
- **G4 — Docs = sannhet:** README/MIGRATION/CHANGELOG/CONTRIBUTING → 4.11.x; fiks brutte lenker + riktig remote; legg indexeddb/ShadeStream/broadcast i README+SHADE-BY-SCENARIO; Postgres at-rest quick-start + salt-livssyklus; per-pakke README for de 6 viktigste.
|
||||||
|
|
||||||
|
## Senere
|
||||||
|
- **G5 — V5.0 (Voice/Video):** implementasjon gated på G0–G4 grønne. `docs/V5.0.md` er i dag idéskisse (135 linjer) — modnes parallelt: wire-type-allokering reconciled mot 0x31–0x33 (nå tatt av ShadeStream), browser-SFrame-mekanisme (RTCRtpScriptTransform/Insertable Streams), ratchet-parametre, API-flate, milepæler, intern konsistens (60fps/1080p vs 30fps-test). Godkjennes i diskusjon (Stian + Fable) før koding.
|
||||||
|
- Versjonsdesigndokumentene bor fortsatt i `docs/V5.0.md` og `docs/archive/V*.md` — de er spesifikasjoner, ikke planfiler.
|
||||||
328
.scaffold/tasks.yaml
Normal file
328
.scaffold/tasks.yaml
Normal file
@@ -0,0 +1,328 @@
|
|||||||
|
# Scaffold kanban-kort. Skjema per kort:
|
||||||
|
# - id: kort-slug # unik i fila
|
||||||
|
# title: ""
|
||||||
|
# description: ""
|
||||||
|
# status: todo # todo | doing | done | blocked
|
||||||
|
# column: todo # speiler status (kanban-kolonne)
|
||||||
|
# priority: medium # low | medium | high
|
||||||
|
# assignee: human # cursor | claude | human | any
|
||||||
|
# context: "" # valgfri: hvorfor/hvor i koden
|
||||||
|
# acceptance: "" # valgfri: hva «ferdig» betyr
|
||||||
|
# todos:
|
||||||
|
# - text: "Underoppgave"
|
||||||
|
# done: false
|
||||||
|
# created: 2026-07-09
|
||||||
|
# updated: 2026-07-09
|
||||||
|
# completed_at: null
|
||||||
|
tasks:
|
||||||
|
# ─────────────── G0 — Stopp blødningen (P0-blockere) ───────────────
|
||||||
|
- id: g0-gcm-nonce-reuse
|
||||||
|
title: "Fjern AES-GCM nonce-gjenbruk i at-rest storage"
|
||||||
|
description: >
|
||||||
|
deriveNonce() lager deterministisk nonce fra (fieldKey, tabell, pk) uten
|
||||||
|
meldingsteller. saveSession re-krypterer sesjonsraden ved hvert ratchet-steg,
|
||||||
|
så to DB-snapshots deler (key, nonce) over ulik plaintext = GCM forbidden
|
||||||
|
attack. Systemisk: gjelder alle muterbare rader (config, prekey-state, trust).
|
||||||
|
status: done
|
||||||
|
column: done
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: >
|
||||||
|
Kallesteder kartlagt 2026-08-13: kun row-codec.ts:67 og :96 lager nonce,
|
||||||
|
og :83/:112 tvinger den ved lesing. deriveNonce() er eksportert fra både
|
||||||
|
index.ts og crypto.ts, så den beholdes (deprecated) framfor å brekke
|
||||||
|
konsumenter. VERIFISERT: aeadOpen leser alltid nonce fra blob-prefiks
|
||||||
|
(aead.ts:72) og expectedNonce er kun en valgfri ekstrasjekk — eksisterende
|
||||||
|
blobber åpnes derfor uendret, og ingen migreringssti trengs.
|
||||||
|
acceptance: "Tilfeldig nonce per skriving; aeadOpen leser nonce fra prefiks (dropp expectedNonce-tvang). Negativ test: to session-snapshots ved ulik message-count deler ikke (key, nonce). Test som åpner en blob forseglet med gammel deterministisk nonce. Alle eksisterende tester grønne."
|
||||||
|
todos:
|
||||||
|
- text: "row-codec.ts:67/96 → tilfeldig 12-byte nonce, lagret i blob-prefiks"
|
||||||
|
done: true
|
||||||
|
- text: "Dropp expectedNonce-argumentet i row-codec.ts:83/112"
|
||||||
|
done: true
|
||||||
|
- text: "Negativ regresjonstest for nonce-unikhet på tvers av re-saves"
|
||||||
|
done: true
|
||||||
|
- text: "Bakoverkompatibilitetstest: blob med gammel deterministisk nonce åpnes"
|
||||||
|
done: true
|
||||||
|
- text: "Marker deriveNonce deprecated (fortsatt eksportert fra index/crypto)"
|
||||||
|
done: true
|
||||||
|
- text: "IKKE PUBLISERT — Prism er pinnet på 4.12.0 og får fiksen først ved release"
|
||||||
|
done: false
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-08-13
|
||||||
|
completed_at: 2026-08-13
|
||||||
|
|
||||||
|
- id: g0-observer-publish
|
||||||
|
title: "Fiks @shade/observer publiserings-artefakt (tomt på registry)"
|
||||||
|
description: >
|
||||||
|
outDir=dist-build avviker fra alle andre pakker (dist). Publish kjører
|
||||||
|
rm -rf dist og shipper files:[dist] → tarballen mangler JS, main peker på
|
||||||
|
manglende fil. dist-build/ er det untrackede avviket i git status.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: "packages/shade-observer/tsconfig.json (outDir), scripts/publish-all.ts:109/186-227, dashboard copy-to-observer.ts-konflikten"
|
||||||
|
acceptance: "outDir=dist; dashboard-SPA-konflikt løst eksplisitt; 4.11.2 republisert med verifisert tarball; dist-build/ i .gitignore."
|
||||||
|
todos:
|
||||||
|
- text: "tsconfig outDir → dist"
|
||||||
|
done: false
|
||||||
|
- text: "Løs dashboard-SPA vs publish-eierskap til dist/"
|
||||||
|
done: false
|
||||||
|
- text: "gitignore dist-build/ + fjern restene"
|
||||||
|
done: false
|
||||||
|
- text: "Republiser observer 4.11.2"
|
||||||
|
done: false
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: g0-typecheck-gate
|
||||||
|
title: "Pin TypeScript, fiks baseUrl, kjør typecheck i CI"
|
||||||
|
description: >
|
||||||
|
TS ikke pinnet noe sted → bunx tsc henter 7.0.2 som fjernet baseUrl (brukt i
|
||||||
|
consumer-strict). typecheck exit 1 nå → prepublish:check rød. Gaten kjøres
|
||||||
|
heller aldri i CI.
|
||||||
|
status: done
|
||||||
|
column: done
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: "tests/consumer-strict/tsconfig.json:17 (baseUrl), scripts/typecheck-all.ts (bruker upinnet bunx tsc), .gitea/workflows/test.yml"
|
||||||
|
acceptance: "typescript pinnet som devDependency og brukt av typecheck-all.ts; baseUrl fjernet; bun run typecheck grønn og kjøres i .gitea/workflows/test.yml."
|
||||||
|
todos:
|
||||||
|
- text: "Pin typescript i root devDependencies"
|
||||||
|
done: true
|
||||||
|
- text: "typecheck-all.ts bruker lokal tsc, ikke bunx latest"
|
||||||
|
done: true
|
||||||
|
- text: "Fjern baseUrl fra consumer-strict tsconfig (var allerede borte)"
|
||||||
|
done: true
|
||||||
|
- text: "Legg typecheck-steg i test.yml"
|
||||||
|
done: true
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-08-14
|
||||||
|
completed_at: 2026-08-14
|
||||||
|
|
||||||
|
- id: g0-cli-bin-tarball-verify
|
||||||
|
title: "Fiks cli bin-omskriving + tarball-verifisering i publish"
|
||||||
|
description: >
|
||||||
|
bin.shade=src/cli.ts omskrives ikke til dist/ (rewriteEntryPointsForDist
|
||||||
|
dekker kun main/types/exports). npm i -g @shade/cli gir brutt binær. Samme
|
||||||
|
rot som observer: publish mangler integritetssjekk.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: "scripts/publish-all.ts:186-196 (rewriteEntryPointsForDist), packages/shade-cli/package.json (bin)"
|
||||||
|
acceptance: "bin omskrives til dist/; publish asserterer at bun pm pack inneholder main-filen for hver pakke; npm i -g @shade/cli kjørbar."
|
||||||
|
todos:
|
||||||
|
- text: "Utvid rewriteEntryPointsForDist til pkgJson.bin"
|
||||||
|
done: false
|
||||||
|
- text: "Legg tarball-innhold-assert (bun pm pack) i publish-flyten"
|
||||||
|
done: false
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
# ─────────────── G1 — Revisjonsklar kjerne (P1-sikkerhet) ───────────────
|
||||||
|
- id: g1-symmetric-safety-number
|
||||||
|
title: "Symmetrisk safety-number (lagre peer-signeringsnøkkel per sesjon)"
|
||||||
|
description: >
|
||||||
|
getRemoteFingerprint sender remoteIdentityKey (DH) som begge argumenter →
|
||||||
|
H(dh‖dh) ≠ peerens egen H(sig‖dh). OOB-verifisering kan aldri matche →
|
||||||
|
primær MITM-forsvar ikke-funksjonelt.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: "packages/shade-core/src/session.ts:238-245, SDK verify()/markPeerVerified()/isPeerVerified() shade.ts:590/649/659"
|
||||||
|
acceptance: "Peer-signeringsnøkkel lagres per sesjon; kanonisk symmetrisk safety-number; kryssparts-test alice.verify(bob, bob.getIdentityFingerprint())===true uten MITM."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: g1-argon2id-backup
|
||||||
|
title: "Koble argon2id inn i backup-KDF (TS + Kotlin)"
|
||||||
|
description: >
|
||||||
|
Backup-passord-KDF er HKDF → offline brute-force i HKDF-hastighet.
|
||||||
|
argon2idAsync fra @noble/hashes er allerede dependency og brukes i
|
||||||
|
storage-encrypted. Kotlin har deriveMasterKeyArgon2id via Bouncy Castle.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: "packages/shade-sdk/src/backup.ts:112-119, android/.../backup/BackupKdf.kt:27-36, referanse: storage-encrypted/src/crypto/kdf.ts:73-91"
|
||||||
|
acceptance: "Backup-KDF = argon2id på begge plattformer; backup-vektor re-pinnet; cross-platform-vektortest grønn."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: g1-hono-upgrade-audit
|
||||||
|
title: "Oppgrader hono ≥4.12.18 + bun audit-gate"
|
||||||
|
description: >
|
||||||
|
hono <4.12.18 har bodyLimit-bypass (svekker §7 64 KiB body-cap), path
|
||||||
|
traversal i serve-static, IP-restriction-bypass. Dev/test: happy-dom
|
||||||
|
(critical VM-escape), vite, postcss.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: "hono i shade-server/inbox-server/transfer/observer/transport-bridge; bun.lock pinner nåværende"
|
||||||
|
acceptance: "hono ≥4.12.18 overalt; bun audit uten runtime-high/critical; §7 DoS-cap re-evaluert."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: g1-at-rest-tamper-test
|
||||||
|
title: "Håndhevende §4 at-rest-tuklingstest + bestill ekstern review"
|
||||||
|
description: >
|
||||||
|
SECURITY.md-cellen «§4 at-rest session DB» er «none yet» → ikke håndhevet.
|
||||||
|
Ekstern krypto-review + pentest står «Pending V4.0» og må ikke skli forbi V5.0.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: high
|
||||||
|
assignee: any
|
||||||
|
context: "SECURITY.md:18 (celler uten testlenke ikke håndhevet), THREAT-MODEL §4"
|
||||||
|
acceptance: "Negativ test som beviser at at-rest-tukling fanges (AAD/tag); SECURITY.md-celle lenket. Ekstern review + pentest bestilt."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
# ─────────────── G2/G3-hoder (nedbrytes når G0/G1 nærmer seg) ───────────────
|
||||||
|
- id: g2-ci-wall
|
||||||
|
title: "G2: CI som mur — lint/format/coverage/soak + rydd .github + arkitektur"
|
||||||
|
description: >
|
||||||
|
biome (lint+format), coverage-terskel (gulv=dagens), soak:smoke i CI;
|
||||||
|
fjern/synk stale .github/workflows (7 pakker→npmjs); publish kjører
|
||||||
|
prepublish:check; bryt server↔inbox-server-syklus; rydd sdk/cli/files ubrukte
|
||||||
|
deps; dashboard private; pin bun i workflows; kjør alle 8 examples (fiks
|
||||||
|
accept-race i shade-transfer/engine.ts:1140).
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: medium
|
||||||
|
assignee: claude
|
||||||
|
context: "Se helsesjekk-artifact 2026-07-10 (P2-arkitektur + CI-hull)"
|
||||||
|
acceptance: "PR blokkeres ved brudd på typecheck·lint·format·test·coverage; alle 8 examples grønne i CI; ingen udeklarert/ubrukt hard dep."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: g3-parity-lock
|
||||||
|
title: "G3: Paritetslås — ShadeStream-vektorer + Kotlin v4.11 + interop"
|
||||||
|
description: >
|
||||||
|
ShadeStream (v4.11) har null vektorer selv på TS-siden. Kotlin stopper på
|
||||||
|
v4.10. Lag vektorer (3-DH deriveStreamRootKey, wire 0x31-0x33, frame
|
||||||
|
seal/open) konsumert av begge; port ShadeStream til Kotlin; utvid
|
||||||
|
cross-vectors path-filter; PARITY-VERSION-sjekk; symmetriser alle 14
|
||||||
|
vektorfiler; Robolectric-tester for KeystoreStorage; E2E socket-interop.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: medium
|
||||||
|
assignee: claude
|
||||||
|
context: "packages/shade-core/src/stream.ts, scripts/generate-vectors.ts, .gitea/workflows/cross-vectors.yml, android/shade-android-keystore (0 tester)"
|
||||||
|
acceptance: "Begge sider verifiserer alle 14 vektorer + ShadeStream; minor-bump uten Android-ack ⇒ rød CI; E2E TS-server↔Kotlin-klient over socket grønn."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: g4-docs-truth
|
||||||
|
title: "G4: Docs = sannhet — oppdater til 4.11.x, fiks lenker/remote/eksempler"
|
||||||
|
description: >
|
||||||
|
README/MIGRATION/CHANGELOG/CONTRIBUTING frosset på 4.0. Riktig remote er
|
||||||
|
Gitea (ikke GitHub). Legg indexeddb/ShadeStream/broadcast i README+
|
||||||
|
SHADE-BY-SCENARIO; Postgres at-rest quick-start + salt-livssyklus; per-pakke
|
||||||
|
README for sdk/core/files/streams/storage-encrypted/transport.
|
||||||
|
status: todo
|
||||||
|
column: todo
|
||||||
|
priority: medium
|
||||||
|
assignee: claude
|
||||||
|
context: "README.md, MIGRATION.md, CHANGELOG.md, CONTRIBUTING.md, docs/SHADE-BY-SCENARIO.md"
|
||||||
|
acceptance: "Én sannhetskilde for versjon; ingen brutte doc-lenker (CI link-check); hvert ECOSYSTEM-scenario har kjørende CI-testet eksempel-sti."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
# ─────────────── Åpne beslutninger (Stian avgjør i plandiskusjon) ───────────────
|
||||||
|
- id: decision-g3-vs-g4-order
|
||||||
|
title: "BESLUTNING: rekkefølge G3 (paritetslås) vs G4 (docs)"
|
||||||
|
description: >
|
||||||
|
Foreslått G3 før G4 fordi «production»-label på Android henger på paritet.
|
||||||
|
Men hvis Prism/Vyvern trenger docs-sannhet raskere, kan de bytte plass.
|
||||||
|
status: blocked
|
||||||
|
column: blocked
|
||||||
|
priority: medium
|
||||||
|
assignee: human
|
||||||
|
acceptance: "Rekkefølge bestemt; plan.md Neste-seksjon speiler valget."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: decision-external-review-timing
|
||||||
|
title: "BESLUTNING: bestille ekstern krypto-review/pentest nå eller etter G0+G1"
|
||||||
|
description: >
|
||||||
|
Parallelt nå (raskere kalender) vs etter G0+G1-fiksene (revisor ser korrekt
|
||||||
|
kjerne — nonce-fiks + symmetrisk safety-number inne). Fable heller mot fikse først.
|
||||||
|
status: blocked
|
||||||
|
column: blocked
|
||||||
|
priority: medium
|
||||||
|
assignee: human
|
||||||
|
context: "Kobles til g1-at-rest-tamper-test; SECURITY.md-celler «Pending V4.0»"
|
||||||
|
acceptance: "Tidspunkt bestemt; g1-at-rest-tamper-test oppdatert deretter."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
- id: decision-argon2id-migration
|
||||||
|
title: "BESLUTNING: argon2id backup-migreringsstrategi"
|
||||||
|
description: >
|
||||||
|
Re-pinning av backup-vektor betyr at eksisterende 4.x-backups må migreres.
|
||||||
|
Big-bang i 4.12 vs dobbelt-lesende overgangsvindu (les HKDF-legacy, skriv argon2id).
|
||||||
|
status: blocked
|
||||||
|
column: blocked
|
||||||
|
priority: medium
|
||||||
|
assignee: human
|
||||||
|
context: "Kobles til g1-argon2id-backup"
|
||||||
|
acceptance: "Migreringsstrategi valgt; g1-argon2id-backup-kortet får todos deretter."
|
||||||
|
created: 2026-07-10
|
||||||
|
updated: 2026-07-10
|
||||||
|
completed_at: null
|
||||||
|
|
||||||
|
# ─────────────── V4.13 — @shade/vault (server-side kryptert fillager) ───────────────
|
||||||
|
- id: vault-package
|
||||||
|
title: "@shade/vault — innholdsadressert kryptert fillager på relayen"
|
||||||
|
description: >
|
||||||
|
Shade kan flytte filer mellom peers (@shade/files) og lagre én liten
|
||||||
|
profil-blob per konto (/v1/blob/<slotId>), men har ingen alltid-på lagring
|
||||||
|
av krypterte filer. Uten den kan ingen Shade-app tilby backup, og ingen
|
||||||
|
klient kan lese data når peeren er avslått. Klienten krypterer og navngir
|
||||||
|
etter hash av ciphertext; serveren ser aldri klartekst.
|
||||||
|
status: doing
|
||||||
|
column: doing
|
||||||
|
priority: high
|
||||||
|
assignee: claude
|
||||||
|
context: >
|
||||||
|
Bestilt av Scaffold-backupen 2026-08-13, men generell: docs/files.md peker
|
||||||
|
selv på Dispatch/Mail/Drive-style apps som konsumenter. Egen HKDF-gren ved
|
||||||
|
siden av shade-blob-*-v1 i storage-encrypted/src/crypto/kdf.ts, så et
|
||||||
|
vault-kompromiss ikke rører profil-slotten. Rutes i shade-server/src/
|
||||||
|
standalone.ts ved siden av createBlobRoutes.
|
||||||
|
acceptance: >
|
||||||
|
Klient kan pushe/hente en samling krypterte objekter mot relayen uten at
|
||||||
|
serveren kan lese noe; oplog gir versjonsliste og henting av tidligere
|
||||||
|
versjon; ny enhet med kontocredentials rekonstruerer hele samlingen.
|
||||||
|
SHADE_VAULT_DB_PATH dokumentert med samme advarsel som SHADE_BLOB_DB_PATH.
|
||||||
|
todos:
|
||||||
|
- text: "HKDF-gren for vault-nøkler (slot, innhold, signatur)"
|
||||||
|
done: true
|
||||||
|
- text: "Innholdsadressert put/get med CAS-semantikk"
|
||||||
|
done: true
|
||||||
|
- text: "Append-only oplog per samling — versjonsliste og rollback"
|
||||||
|
done: true
|
||||||
|
- text: "Serverruter + persistenskonfigurasjon i standalone.ts"
|
||||||
|
done: true
|
||||||
|
- text: "docs/vault.md + README-rad i statustabellen"
|
||||||
|
done: false
|
||||||
|
- text: "IKKE DEPLOYET — krever SHADE_VAULT_DB_PATH i compose + Dokploy-runde"
|
||||||
|
done: false
|
||||||
|
created: 2026-08-13
|
||||||
|
updated: 2026-08-14
|
||||||
|
completed_at: null
|
||||||
14
CHANGELOG.md
14
CHANGELOG.md
@@ -5,6 +5,20 @@ All notable changes to Shade are documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [4.11.2] — 2026-07-10 — Files pull-mode startup hotfix
|
||||||
|
|
||||||
|
**`@shade/files`**
|
||||||
|
|
||||||
|
- Fix a construction race in HTTP pull-mode clients: an immediate streamed
|
||||||
|
read/write now awaits asynchronous stream-bridge registration instead of
|
||||||
|
treating the not-yet-created queue drainer as proof that the client is
|
||||||
|
inline-only.
|
||||||
|
- Add a deterministic integration test that holds incoming-transfer
|
||||||
|
registration, starts a 512 KiB write immediately, and verifies that it
|
||||||
|
remains pending until streaming is ready.
|
||||||
|
- Add `publish-all.ts --only <pkg>` so isolated package hotfixes do not publish
|
||||||
|
unrelated workspace packages.
|
||||||
|
|
||||||
## [4.11.0] — 2026-05-15 — Streaming Double-Ratchet sub-sessions
|
## [4.11.0] — 2026-05-15 — Streaming Double-Ratchet sub-sessions
|
||||||
|
|
||||||
Answers Vyvern FR `shade-ws-streaming-ratchet.md` (the last Phase-2
|
Answers Vyvern FR `shade-ws-streaming-ratchet.md` (the last Phase-2
|
||||||
|
|||||||
4
CLAUDE.md
Normal file
4
CLAUDE.md
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
## Planlegging
|
||||||
|
Planen for dette prosjektet bor i `.scaffold/`: `plan.md` er nåbildet (hold den kort,
|
||||||
|
bump `updated` i frontmatter ved endring), `log.md` er append-only historikk (nyeste øverst),
|
||||||
|
`tasks.yaml` er kanban. Ikke opprett planfiler andre steder.
|
||||||
123
bun.lock
123
bun.lock
@@ -13,11 +13,12 @@
|
|||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"bun-types": "^1.3.11",
|
"bun-types": "^1.3.11",
|
||||||
"fast-check": "^3.22.0",
|
"fast-check": "^3.22.0",
|
||||||
|
"typescript": "7.0.2",
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
"packages/shade-cli": {
|
"packages/shade-cli": {
|
||||||
"name": "@shade/cli",
|
"name": "@shade/cli",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"bin": {
|
"bin": {
|
||||||
"shade": "src/cli.ts",
|
"shade": "src/cli.ts",
|
||||||
},
|
},
|
||||||
@@ -36,7 +37,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-core": {
|
"packages/shade-core": {
|
||||||
"name": "@shade/core",
|
"name": "@shade/core",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/observability": "workspace:*",
|
"@shade/observability": "workspace:*",
|
||||||
},
|
},
|
||||||
@@ -49,7 +50,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-crypto-web": {
|
"packages/shade-crypto-web": {
|
||||||
"name": "@shade/crypto-web",
|
"name": "@shade/crypto-web",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@noble/curves": "^2.0.1",
|
"@noble/curves": "^2.0.1",
|
||||||
"@noble/hashes": "^2.0.1",
|
"@noble/hashes": "^2.0.1",
|
||||||
@@ -59,7 +60,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-dashboard": {
|
"packages/shade-dashboard": {
|
||||||
"name": "@shade/dashboard",
|
"name": "@shade/dashboard",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/widgets": "workspace:*",
|
"@shade/widgets": "workspace:*",
|
||||||
"react": "^19.0.0",
|
"react": "^19.0.0",
|
||||||
@@ -74,7 +75,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-files": {
|
"packages/shade-files": {
|
||||||
"name": "@shade/files",
|
"name": "@shade/files",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
@@ -101,7 +102,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-inbox": {
|
"packages/shade-inbox": {
|
||||||
"name": "@shade/inbox",
|
"name": "@shade/inbox",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/proto": "workspace:*",
|
"@shade/proto": "workspace:*",
|
||||||
@@ -114,7 +115,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-inbox-server": {
|
"packages/shade-inbox-server": {
|
||||||
"name": "@shade/inbox-server",
|
"name": "@shade/inbox-server",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/observability": "workspace:*",
|
"@shade/observability": "workspace:*",
|
||||||
@@ -132,7 +133,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-key-transparency": {
|
"packages/shade-key-transparency": {
|
||||||
"name": "@shade/key-transparency",
|
"name": "@shade/key-transparency",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@noble/hashes": "^2.0.1",
|
"@noble/hashes": "^2.0.1",
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
@@ -144,11 +145,11 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-keychain": {
|
"packages/shade-keychain": {
|
||||||
"name": "@shade/keychain",
|
"name": "@shade/keychain",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
},
|
},
|
||||||
"packages/shade-observability": {
|
"packages/shade-observability": {
|
||||||
"name": "@shade/observability",
|
"name": "@shade/observability",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@noble/hashes": "^2.0.1",
|
"@noble/hashes": "^2.0.1",
|
||||||
},
|
},
|
||||||
@@ -166,7 +167,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-observer": {
|
"packages/shade-observer": {
|
||||||
"name": "@shade/observer",
|
"name": "@shade/observer",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/server": "workspace:*",
|
"@shade/server": "workspace:*",
|
||||||
@@ -178,14 +179,14 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-proto": {
|
"packages/shade-proto": {
|
||||||
"name": "@shade/proto",
|
"name": "@shade/proto",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
"packages/shade-recovery": {
|
"packages/shade-recovery": {
|
||||||
"name": "@shade/recovery",
|
"name": "@shade/recovery",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
@@ -198,7 +199,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-sdk": {
|
"packages/shade-sdk": {
|
||||||
"name": "@shade/sdk",
|
"name": "@shade/sdk",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
@@ -228,12 +229,14 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-server": {
|
"packages/shade-server": {
|
||||||
"name": "@shade/server",
|
"name": "@shade/server",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/inbox-server": "workspace:*",
|
"@shade/inbox-server": "workspace:*",
|
||||||
"@shade/key-transparency": "workspace:*",
|
"@shade/key-transparency": "workspace:*",
|
||||||
"@shade/observability": "workspace:*",
|
"@shade/observability": "workspace:*",
|
||||||
|
"@shade/storage-sqlite": "workspace:*",
|
||||||
|
"@shade/vault": "workspace:*",
|
||||||
"hono": "^4.12.12",
|
"hono": "^4.12.12",
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
@@ -248,7 +251,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-storage-encrypted": {
|
"packages/shade-storage-encrypted": {
|
||||||
"name": "@shade/storage-encrypted",
|
"name": "@shade/storage-encrypted",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@noble/hashes": "^2.0.1",
|
"@noble/hashes": "^2.0.1",
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
@@ -270,7 +273,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-storage-indexeddb": {
|
"packages/shade-storage-indexeddb": {
|
||||||
"name": "@shade/storage-indexeddb",
|
"name": "@shade/storage-indexeddb",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"idb": "^8.0.3",
|
"idb": "^8.0.3",
|
||||||
@@ -282,7 +285,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-storage-postgres": {
|
"packages/shade-storage-postgres": {
|
||||||
"name": "@shade/storage-postgres",
|
"name": "@shade/storage-postgres",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/inbox-server": "workspace:*",
|
"@shade/inbox-server": "workspace:*",
|
||||||
@@ -297,17 +300,20 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-storage-sqlite": {
|
"packages/shade-storage-sqlite": {
|
||||||
"name": "@shade/storage-sqlite",
|
"name": "@shade/storage-sqlite",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
"@shade/inbox-server": "workspace:*",
|
"@shade/inbox-server": "workspace:*",
|
||||||
"@shade/server": "workspace:*",
|
"@shade/server": "workspace:*",
|
||||||
},
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@shade/vault": "workspace:*",
|
||||||
|
},
|
||||||
},
|
},
|
||||||
"packages/shade-streams": {
|
"packages/shade-streams": {
|
||||||
"name": "@shade/streams",
|
"name": "@shade/streams",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@noble/hashes": "^2.0.1",
|
"@noble/hashes": "^2.0.1",
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
@@ -319,7 +325,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-transfer": {
|
"packages/shade-transfer": {
|
||||||
"name": "@shade/transfer",
|
"name": "@shade/transfer",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
@@ -336,7 +342,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-transport": {
|
"packages/shade-transport": {
|
||||||
"name": "@shade/transport",
|
"name": "@shade/transport",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
@@ -347,7 +353,7 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-transport-bridge": {
|
"packages/shade-transport-bridge": {
|
||||||
"name": "@shade/transport-bridge",
|
"name": "@shade/transport-bridge",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/server": "workspace:*",
|
"@shade/server": "workspace:*",
|
||||||
@@ -369,16 +375,29 @@
|
|||||||
},
|
},
|
||||||
"packages/shade-transport-webrtc": {
|
"packages/shade-transport-webrtc": {
|
||||||
"name": "@shade/transport-webrtc",
|
"name": "@shade/transport-webrtc",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/core": "workspace:*",
|
"@shade/core": "workspace:*",
|
||||||
"@shade/streams": "workspace:*",
|
"@shade/streams": "workspace:*",
|
||||||
"@shade/transfer": "workspace:*",
|
"@shade/transfer": "workspace:*",
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
"packages/shade-vault": {
|
||||||
|
"name": "@shade/vault",
|
||||||
|
"version": "4.13.0",
|
||||||
|
"dependencies": {
|
||||||
|
"@noble/hashes": "^2.0.1",
|
||||||
|
"@shade/core": "workspace:*",
|
||||||
|
"@shade/crypto-web": "workspace:*",
|
||||||
|
"@shade/observability": "workspace:*",
|
||||||
|
"@shade/server": "workspace:*",
|
||||||
|
"@shade/storage-encrypted": "workspace:*",
|
||||||
|
"hono": "^4.12.18",
|
||||||
|
},
|
||||||
|
},
|
||||||
"packages/shade-widgets": {
|
"packages/shade-widgets": {
|
||||||
"name": "@shade/widgets",
|
"name": "@shade/widgets",
|
||||||
"version": "4.8.5",
|
"version": "4.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@shade/recovery": "workspace:*",
|
"@shade/recovery": "workspace:*",
|
||||||
"@shade/sdk": "workspace:*",
|
"@shade/sdk": "workspace:*",
|
||||||
@@ -603,6 +622,8 @@
|
|||||||
|
|
||||||
"@shade/transport-webrtc": ["@shade/transport-webrtc@workspace:packages/shade-transport-webrtc"],
|
"@shade/transport-webrtc": ["@shade/transport-webrtc@workspace:packages/shade-transport-webrtc"],
|
||||||
|
|
||||||
|
"@shade/vault": ["@shade/vault@workspace:packages/shade-vault"],
|
||||||
|
|
||||||
"@shade/widgets": ["@shade/widgets@workspace:packages/shade-widgets"],
|
"@shade/widgets": ["@shade/widgets@workspace:packages/shade-widgets"],
|
||||||
|
|
||||||
"@types/babel__core": ["@types/babel__core@7.20.5", "", { "dependencies": { "@babel/parser": "^7.20.7", "@babel/types": "^7.20.7", "@types/babel__generator": "*", "@types/babel__template": "*", "@types/babel__traverse": "*" } }, "sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA=="],
|
"@types/babel__core": ["@types/babel__core@7.20.5", "", { "dependencies": { "@babel/parser": "^7.20.7", "@babel/types": "^7.20.7", "@types/babel__generator": "*", "@types/babel__template": "*", "@types/babel__traverse": "*" } }, "sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA=="],
|
||||||
@@ -621,6 +642,46 @@
|
|||||||
|
|
||||||
"@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="],
|
"@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-aix-ppc64": ["@typescript/typescript-aix-ppc64@7.0.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-darwin-arm64": ["@typescript/typescript-darwin-arm64@7.0.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-darwin-x64": ["@typescript/typescript-darwin-x64@7.0.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-freebsd-arm64": ["@typescript/typescript-freebsd-arm64@7.0.2", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-freebsd-x64": ["@typescript/typescript-freebsd-x64@7.0.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-arm": ["@typescript/typescript-linux-arm@7.0.2", "", { "os": "linux", "cpu": "arm" }, "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-arm64": ["@typescript/typescript-linux-arm64@7.0.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-loong64": ["@typescript/typescript-linux-loong64@7.0.2", "", { "os": "linux", "cpu": "none" }, "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-mips64el": ["@typescript/typescript-linux-mips64el@7.0.2", "", { "os": "linux", "cpu": "none" }, "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-ppc64": ["@typescript/typescript-linux-ppc64@7.0.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-riscv64": ["@typescript/typescript-linux-riscv64@7.0.2", "", { "os": "linux", "cpu": "none" }, "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-s390x": ["@typescript/typescript-linux-s390x@7.0.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-linux-x64": ["@typescript/typescript-linux-x64@7.0.2", "", { "os": "linux", "cpu": "x64" }, "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-netbsd-arm64": ["@typescript/typescript-netbsd-arm64@7.0.2", "", { "os": "none", "cpu": "arm64" }, "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-netbsd-x64": ["@typescript/typescript-netbsd-x64@7.0.2", "", { "os": "none", "cpu": "x64" }, "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-openbsd-arm64": ["@typescript/typescript-openbsd-arm64@7.0.2", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-openbsd-x64": ["@typescript/typescript-openbsd-x64@7.0.2", "", { "os": "openbsd", "cpu": "x64" }, "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-sunos-x64": ["@typescript/typescript-sunos-x64@7.0.2", "", { "os": "sunos", "cpu": "x64" }, "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-win32-arm64": ["@typescript/typescript-win32-arm64@7.0.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ=="],
|
||||||
|
|
||||||
|
"@typescript/typescript-win32-x64": ["@typescript/typescript-win32-x64@7.0.2", "", { "os": "win32", "cpu": "x64" }, "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g=="],
|
||||||
|
|
||||||
"@vitejs/plugin-react": ["@vitejs/plugin-react@4.7.0", "", { "dependencies": { "@babel/core": "^7.28.0", "@babel/plugin-transform-react-jsx-self": "^7.27.1", "@babel/plugin-transform-react-jsx-source": "^7.27.1", "@rolldown/pluginutils": "1.0.0-beta.27", "@types/babel__core": "^7.20.5", "react-refresh": "^0.17.0" }, "peerDependencies": { "vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0" } }, "sha512-gUu9hwfWvvEDBBmgtAowQCojwZmJ5mcLn3aufeCsitijs3+f2NsrPtlAWIR6OPiqljl96GVCUbLe0HyqIpVaoA=="],
|
"@vitejs/plugin-react": ["@vitejs/plugin-react@4.7.0", "", { "dependencies": { "@babel/core": "^7.28.0", "@babel/plugin-transform-react-jsx-self": "^7.27.1", "@babel/plugin-transform-react-jsx-source": "^7.27.1", "@rolldown/pluginutils": "1.0.0-beta.27", "@types/babel__core": "^7.20.5", "react-refresh": "^0.17.0" }, "peerDependencies": { "vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0" } }, "sha512-gUu9hwfWvvEDBBmgtAowQCojwZmJ5mcLn3aufeCsitijs3+f2NsrPtlAWIR6OPiqljl96GVCUbLe0HyqIpVaoA=="],
|
||||||
|
|
||||||
"baseline-browser-mapping": ["baseline-browser-mapping@2.10.17", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-HdrkN8eVG2CXxeifv/VdJ4A4RSra1DTW8dc/hdxzhGHN8QePs6gKaWM9pHPcpCoxYZJuOZ8drHmbdpLHjCYjLA=="],
|
"baseline-browser-mapping": ["baseline-browser-mapping@2.10.17", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-HdrkN8eVG2CXxeifv/VdJ4A4RSra1DTW8dc/hdxzhGHN8QePs6gKaWM9pHPcpCoxYZJuOZ8drHmbdpLHjCYjLA=="],
|
||||||
@@ -703,6 +764,8 @@
|
|||||||
|
|
||||||
"tinyglobby": ["tinyglobby@0.2.16", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg=="],
|
"tinyglobby": ["tinyglobby@0.2.16", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg=="],
|
||||||
|
|
||||||
|
"typescript": ["typescript@7.0.2", "", { "optionalDependencies": { "@typescript/typescript-aix-ppc64": "7.0.2", "@typescript/typescript-darwin-arm64": "7.0.2", "@typescript/typescript-darwin-x64": "7.0.2", "@typescript/typescript-freebsd-arm64": "7.0.2", "@typescript/typescript-freebsd-x64": "7.0.2", "@typescript/typescript-linux-arm": "7.0.2", "@typescript/typescript-linux-arm64": "7.0.2", "@typescript/typescript-linux-loong64": "7.0.2", "@typescript/typescript-linux-mips64el": "7.0.2", "@typescript/typescript-linux-ppc64": "7.0.2", "@typescript/typescript-linux-riscv64": "7.0.2", "@typescript/typescript-linux-s390x": "7.0.2", "@typescript/typescript-linux-x64": "7.0.2", "@typescript/typescript-netbsd-arm64": "7.0.2", "@typescript/typescript-netbsd-x64": "7.0.2", "@typescript/typescript-openbsd-arm64": "7.0.2", "@typescript/typescript-openbsd-x64": "7.0.2", "@typescript/typescript-sunos-x64": "7.0.2", "@typescript/typescript-win32-arm64": "7.0.2", "@typescript/typescript-win32-x64": "7.0.2" }, "bin": { "tsc": "bin/tsc" } }, "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA=="],
|
||||||
|
|
||||||
"undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
|
"undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
|
||||||
|
|
||||||
"update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="],
|
"update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="],
|
||||||
@@ -716,5 +779,15 @@
|
|||||||
"yallist": ["yallist@3.1.1", "", {}, "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g=="],
|
"yallist": ["yallist@3.1.1", "", {}, "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g=="],
|
||||||
|
|
||||||
"zod": ["zod@3.25.76", "", {}, "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ=="],
|
"zod": ["zod@3.25.76", "", {}, "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ=="],
|
||||||
|
|
||||||
|
"@shade/inbox-server/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
|
||||||
|
|
||||||
|
"@shade/observer/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
|
||||||
|
|
||||||
|
"@shade/server/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
|
||||||
|
|
||||||
|
"@shade/transport-bridge/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
|
||||||
|
|
||||||
|
"@shade/vault/hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="],
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
126
docs/vault.md
Normal file
126
docs/vault.md
Normal file
@@ -0,0 +1,126 @@
|
|||||||
|
# `@shade/vault` — server-side encrypted file store
|
||||||
|
|
||||||
|
V4.13. Where the blob primitive (V4.9) holds one small blob per account, a
|
||||||
|
vault holds a whole **collection** — a workspace, a mailbox, a drive folder —
|
||||||
|
as content-addressed objects plus an append-only log of manifests.
|
||||||
|
|
||||||
|
It is the piece Shade was missing to let an app offer backup, or to let a
|
||||||
|
client read data while the peer that owns it is switched off.
|
||||||
|
|
||||||
|
## The model
|
||||||
|
|
||||||
|
**Objects are content-addressed.** An object's name is the SHA-256 of its
|
||||||
|
*ciphertext*, so the relay can store, dedupe and verify without a key: it
|
||||||
|
recomputes the hash on upload and rejects anything mislabelled.
|
||||||
|
|
||||||
|
**A manifest names the collection.** Paths live inside the encrypted manifest,
|
||||||
|
never in object names — the relay must not learn that a user has a file called
|
||||||
|
`Projects/Divorce/plan.md`. The manifest is itself stored as an object.
|
||||||
|
|
||||||
|
**The log is append-only.** Each commit adds a row pointing at a manifest.
|
||||||
|
History, rollback and "what changed last Tuesday" all fall out of that.
|
||||||
|
|
||||||
|
## Keys
|
||||||
|
|
||||||
|
Three deterministic derivations off the account master key, in their own HKDF
|
||||||
|
branch:
|
||||||
|
|
||||||
|
```
|
||||||
|
vaultId = HKDF(masterKey, "shade-vault-id-v1:<app>")
|
||||||
|
contentKey = HKDF(masterKey, "shade-vault-content-v1:<app>")
|
||||||
|
sigSeed = HKDF(masterKey, "shade-vault-sig-v1:<app>")
|
||||||
|
```
|
||||||
|
|
||||||
|
Separate from `shade-blob-*-v1` on purpose: a vault key reads every file, a
|
||||||
|
profile-blob key reads a host list. Sharing a derivation would turn one
|
||||||
|
compromise into the other.
|
||||||
|
|
||||||
|
Because the derivation is deterministic, **recovery is credentials**. A fresh
|
||||||
|
device that can derive the profile master can derive these, and the relay hands
|
||||||
|
over ciphertext it has never been able to read. The flip side is that recovery
|
||||||
|
strength is password strength — see G1's argon2id item.
|
||||||
|
|
||||||
|
## Client
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
||||||
|
import { VaultClient, HttpVaultTransport, deriveVaultKeys } from '@shade/vault';
|
||||||
|
|
||||||
|
const crypto = new SubtleCryptoProvider();
|
||||||
|
const keys = await deriveVaultKeys(masterKey, 'scaffold');
|
||||||
|
const client = new VaultClient(crypto, keys, new HttpVaultTransport('https://vault.example'));
|
||||||
|
|
||||||
|
// Back up. Unchanged files are not re-uploaded.
|
||||||
|
const res = await client.push(files, Date.now(), 'nightly');
|
||||||
|
// → { seq: 4, uploaded: 2, reused: 489, bytesUploaded: 1_204 }
|
||||||
|
|
||||||
|
// Restore — the newest, or any earlier version.
|
||||||
|
const { manifest, files: restored } = await client.pull();
|
||||||
|
const old = await client.pull(2);
|
||||||
|
|
||||||
|
// History.
|
||||||
|
const log = await client.history();
|
||||||
|
```
|
||||||
|
|
||||||
|
`push` takes `at` rather than reading the clock, so a queued backup records
|
||||||
|
when it was *taken* rather than when it finally reached the relay.
|
||||||
|
|
||||||
|
## Server
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { createVaultRoutes } from '@shade/vault/server';
|
||||||
|
import { SqliteVaultStore } from '@shade/storage-sqlite';
|
||||||
|
|
||||||
|
app.route('/', createVaultRoutes(new SqliteVaultStore('/data/shade-vault.db'), crypto));
|
||||||
|
```
|
||||||
|
|
||||||
|
Routes:
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| `GET /v1/vault/:id/log` | `{ entries, head }` |
|
||||||
|
| `HEAD /v1/vault/:id/object/:hash` | 200 / 404 — the have-check |
|
||||||
|
| `GET /v1/vault/:id/object/:hash` | raw ciphertext |
|
||||||
|
| `PUT /v1/vault/:id/object/:hash` | signed; verifies hash matches bytes |
|
||||||
|
| `POST /v1/vault/:id/commit` | signed; refuses a stale `seq` or a missing object |
|
||||||
|
|
||||||
|
Auth is TOFU-Ed25519, as for blobs: the first signed write pins a pubkey, and
|
||||||
|
every later write must be signed by it. The pubkey travels **inside** the
|
||||||
|
signed payload — appended afterwards it would not be covered by the signature,
|
||||||
|
and could be swapped in transit on the pinning write.
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
```
|
||||||
|
SHADE_VAULT_DB_PATH=/data/shade-vault.db # required — no in-memory fallback
|
||||||
|
SHADE_DISABLE_VAULT=1 # optional: off entirely
|
||||||
|
```
|
||||||
|
|
||||||
|
**There is deliberately no in-memory fallback.** The blob store has one, and on
|
||||||
|
2026-08-12 a routine redeploy destroyed every profile because the path was
|
||||||
|
never set in compose: the fallback worked perfectly right up until the
|
||||||
|
container was recreated, and nothing ever failed loudly. A vault that forgets
|
||||||
|
is worse than no vault, so an unset path means the routes are simply not
|
||||||
|
mounted, with a warning that says why.
|
||||||
|
|
||||||
|
### Limits
|
||||||
|
|
||||||
|
Per object 8 MiB, per vault 512 MiB, both overridable via `VaultRoutesOptions`.
|
||||||
|
|
||||||
|
## What the relay learns
|
||||||
|
|
||||||
|
Object sizes, how many objects there are, when commits happen, and that some
|
||||||
|
opaque 64-hex id is active. Not contents, not paths, not filenames, and not
|
||||||
|
which user a vaultId belongs to — the id is itself derived from a secret.
|
||||||
|
|
||||||
|
Deduplication is **within** a version chain, not across independent seals:
|
||||||
|
sealing uses a fresh nonce, so identical plaintext yields different names.
|
||||||
|
Convergent encryption would dedupe globally but leak which files two users
|
||||||
|
share, which is exactly what a blind store must not do.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Client, routes, `MemoryVaultStore` and `SqliteVaultStore` are implemented and
|
||||||
|
tested (16 unit + 7 store tests, plus an end-to-end suite against a real
|
||||||
|
workspace of 491 files). Not yet: a Postgres store, rate limiting on the vault
|
||||||
|
routes, and garbage collection of objects no manifest references any more.
|
||||||
@@ -1,7 +1,9 @@
|
|||||||
{
|
{
|
||||||
"name": "shade",
|
"name": "shade",
|
||||||
"private": true,
|
"private": true,
|
||||||
"workspaces": ["packages/*"],
|
"workspaces": [
|
||||||
|
"packages/*"
|
||||||
|
],
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"test": "bun test --recursive",
|
"test": "bun test --recursive",
|
||||||
"test:core": "cd packages/shade-core && bun test",
|
"test:core": "cd packages/shade-core && bun test",
|
||||||
@@ -25,7 +27,8 @@
|
|||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"bun-types": "^1.3.11",
|
"bun-types": "^1.3.11",
|
||||||
"fast-check": "^3.22.0"
|
"fast-check": "^3.22.0",
|
||||||
|
"typescript": "7.0.2"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@noble/curves": "^2.0.1",
|
"@noble/curves": "^2.0.1",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/cli",
|
"name": "@shade/cli",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/cli.ts",
|
"main": "src/cli.ts",
|
||||||
"bin": {
|
"bin": {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/core",
|
"name": "@shade/core",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -86,6 +86,14 @@ export class ShadeSessionManager {
|
|||||||
* fully concurrent.
|
* fully concurrent.
|
||||||
*/
|
*/
|
||||||
private readonly peerOpChains = new Map<string, Promise<unknown>>();
|
private readonly peerOpChains = new Map<string, Promise<unknown>>();
|
||||||
|
/**
|
||||||
|
* Memoized `alias → canonical` lookups; `null` records "no alias" so a
|
||||||
|
* label without one costs a single storage read for the life of the
|
||||||
|
* process instead of one per encrypt/decrypt. Aliases change only via
|
||||||
|
* the mutators in this class, each of which clears the whole map — it
|
||||||
|
* holds at most one entry per peer, so rebuilding is cheap.
|
||||||
|
*/
|
||||||
|
private readonly aliasCache = new Map<string, string | null>();
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
private readonly crypto: CryptoProvider,
|
private readonly crypto: CryptoProvider,
|
||||||
@@ -151,6 +159,46 @@ export class ShadeSessionManager {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Map a session label onto the label its state actually lives under.
|
||||||
|
*
|
||||||
|
* `aliasSession` moves a session from a first-contact label (typically
|
||||||
|
* `fp:<hex>`) to the peer's canonical address, but the peer keeps
|
||||||
|
* sending under the old one. The persisted alias lets us follow that
|
||||||
|
* move across restarts — see the alias block in `StorageProvider`.
|
||||||
|
*
|
||||||
|
* A live session under `label` always wins: after a re-link the peer
|
||||||
|
* re-runs X3DH and a fresh session is established under the
|
||||||
|
* first-contact label again, and that new session — not the stale
|
||||||
|
* alias target — is the one that can decrypt what follows.
|
||||||
|
*
|
||||||
|
* Resolves one hop only. Aliases are always written pointing at a
|
||||||
|
* canonical label, so a chain would mean corrupt state; following it
|
||||||
|
* would risk a loop for no legitimate gain.
|
||||||
|
*
|
||||||
|
* MUST be called before taking the peer mutex: locking the alias while
|
||||||
|
* mutating the canonical session would let an aliased caller and a
|
||||||
|
* canonical caller ratchet the same state concurrently.
|
||||||
|
*/
|
||||||
|
private async resolveLabel(label: string): Promise<string> {
|
||||||
|
let canonical = this.aliasCache.get(label);
|
||||||
|
if (canonical === undefined) {
|
||||||
|
canonical = (await this.storage.getSessionAlias?.(label)) ?? null;
|
||||||
|
this.aliasCache.set(label, canonical);
|
||||||
|
}
|
||||||
|
if (canonical === null || canonical === label) return label;
|
||||||
|
if (await this.storage.getSession(label)) return label;
|
||||||
|
return canonical;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Public label resolution for callers that need to know where a
|
||||||
|
* peer's state lives (e.g. a transport routing an inbound frame).
|
||||||
|
*/
|
||||||
|
async resolveSessionLabel(label: string): Promise<string> {
|
||||||
|
return this.resolveLabel(label);
|
||||||
|
}
|
||||||
|
|
||||||
/** Get the event emitter (if observability is enabled) */
|
/** Get the event emitter (if observability is enabled) */
|
||||||
getEvents(): ShadeEventEmitter | undefined {
|
getEvents(): ShadeEventEmitter | undefined {
|
||||||
return this.events;
|
return this.events;
|
||||||
@@ -268,6 +316,11 @@ export class ShadeSessionManager {
|
|||||||
*/
|
*/
|
||||||
async resetSession(address: string): Promise<void> {
|
async resetSession(address: string): Promise<void> {
|
||||||
await this.storage.removeSession(address);
|
await this.storage.removeSession(address);
|
||||||
|
// Aliases pointing here are now dangling — drop them so the next
|
||||||
|
// first-contact frame resolves to its own label and establishes the
|
||||||
|
// fresh session this reset exists to force.
|
||||||
|
await this.storage.removeSessionAliasesFor?.(address);
|
||||||
|
this.aliasCache.clear();
|
||||||
this.events?.emit('session.removed', { address });
|
this.events?.emit('session.removed', { address });
|
||||||
// Note: we keep the trusted identity; new session will verify against it.
|
// Note: we keep the trusted identity; new session will verify against it.
|
||||||
}
|
}
|
||||||
@@ -336,6 +389,14 @@ export class ShadeSessionManager {
|
|||||||
await this.storage.bumpPeerIdentityVersion(newLabel);
|
await this.storage.bumpPeerIdentityVersion(newLabel);
|
||||||
}
|
}
|
||||||
await this.storage.removeSession(oldLabel);
|
await this.storage.removeSession(oldLabel);
|
||||||
|
// Remember the move durably. The peer goes on sending under
|
||||||
|
// `oldLabel` — its transport derives the same first-contact label
|
||||||
|
// from our relay hint every time — so without this record every
|
||||||
|
// inbound frame after a restart resolves to a label whose session
|
||||||
|
// we just removed, and the peer (holding a valid session, never
|
||||||
|
// re-running X3DH) can never recover on its own.
|
||||||
|
await this.storage.saveSessionAlias?.(oldLabel, newLabel);
|
||||||
|
this.aliasCache.clear();
|
||||||
this.events?.emit('session.aliased', { oldLabel, newLabel });
|
this.events?.emit('session.aliased', { oldLabel, newLabel });
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -349,6 +410,10 @@ export class ShadeSessionManager {
|
|||||||
// because isTrustedIdentity() compares not retrieves; we just emit the new hash)
|
// because isTrustedIdentity() compares not retrieves; we just emit the new hash)
|
||||||
await this.storage.saveTrustedIdentity(address, newIdentityKey);
|
await this.storage.saveTrustedIdentity(address, newIdentityKey);
|
||||||
await this.storage.removeSession(address);
|
await this.storage.removeSession(address);
|
||||||
|
// The peer rotated identity — any alias into the old session is
|
||||||
|
// dangling and must not redirect frames meant for the new one.
|
||||||
|
await this.storage.removeSessionAliasesFor?.(address);
|
||||||
|
this.aliasCache.clear();
|
||||||
|
|
||||||
if (this.events) {
|
if (this.events) {
|
||||||
const newHash = await shortHash(this.crypto, newIdentityKey);
|
const newHash = await shortHash(this.crypto, newIdentityKey);
|
||||||
@@ -516,14 +581,19 @@ export class ShadeSessionManager {
|
|||||||
* Subsequent messages are standard RatchetMessages.
|
* Subsequent messages are standard RatchetMessages.
|
||||||
*/
|
*/
|
||||||
async encrypt(address: string, plaintext: string): Promise<ShadeEnvelope> {
|
async encrypt(address: string, plaintext: string): Promise<ShadeEnvelope> {
|
||||||
return this.withSpan('encrypt', address, async () => {
|
// Follow a persisted alias so a caller still holding a first-contact
|
||||||
const session = await this.storage.getSession(address);
|
// label reaches the session that alias moved to — without this, a
|
||||||
if (!session) throw new NoSessionError(address);
|
// restarted host can decrypt a peer's frames but not reply to them.
|
||||||
|
// Resolved before the mutex so the lock lands on the canonical label.
|
||||||
|
const target = await this.resolveLabel(address);
|
||||||
|
return this.withSpan('encrypt', target, async () => {
|
||||||
|
const session = await this.storage.getSession(target);
|
||||||
|
if (!session) throw new NoSessionError(target);
|
||||||
|
|
||||||
const ratchetMsg = await ratchetEncrypt(this.crypto, session, enc.encode(plaintext));
|
const ratchetMsg = await ratchetEncrypt(this.crypto, session, enc.encode(plaintext));
|
||||||
|
|
||||||
this.events?.emit('message.encrypted', {
|
this.events?.emit('message.encrypted', {
|
||||||
address,
|
address: target,
|
||||||
counter: ratchetMsg.counter,
|
counter: ratchetMsg.counter,
|
||||||
ciphertextSize: ratchetMsg.ciphertext.length,
|
ciphertextSize: ratchetMsg.ciphertext.length,
|
||||||
});
|
});
|
||||||
@@ -532,7 +602,7 @@ export class ShadeSessionManager {
|
|||||||
const x3dh = (session as any).__x3dh;
|
const x3dh = (session as any).__x3dh;
|
||||||
if (x3dh) {
|
if (x3dh) {
|
||||||
delete (session as any).__x3dh;
|
delete (session as any).__x3dh;
|
||||||
await this.storage.saveSession(address, session);
|
await this.storage.saveSession(target, session);
|
||||||
|
|
||||||
const preKeyMsg: PreKeyMessage = {
|
const preKeyMsg: PreKeyMessage = {
|
||||||
registrationId: x3dh.registrationId,
|
registrationId: x3dh.registrationId,
|
||||||
@@ -546,16 +616,16 @@ export class ShadeSessionManager {
|
|||||||
type: 'prekey',
|
type: 'prekey',
|
||||||
content: preKeyMsg,
|
content: preKeyMsg,
|
||||||
timestamp: Date.now(),
|
timestamp: Date.now(),
|
||||||
senderAddress: address,
|
senderAddress: target,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
await this.storage.saveSession(address, session);
|
await this.storage.saveSession(target, session);
|
||||||
return {
|
return {
|
||||||
type: 'ratchet',
|
type: 'ratchet',
|
||||||
content: ratchetMsg,
|
content: ratchetMsg,
|
||||||
timestamp: Date.now(),
|
timestamp: Date.now(),
|
||||||
senderAddress: address,
|
senderAddress: target,
|
||||||
};
|
};
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -564,11 +634,17 @@ export class ShadeSessionManager {
|
|||||||
* Decrypt a message from a peer. Handles both PreKeyMessage and RatchetMessage.
|
* Decrypt a message from a peer. Handles both PreKeyMessage and RatchetMessage.
|
||||||
*/
|
*/
|
||||||
async decrypt(address: string, envelope: ShadeEnvelope): Promise<string> {
|
async decrypt(address: string, envelope: ShadeEnvelope): Promise<string> {
|
||||||
return this.withSpan('decrypt', address, async () => {
|
// A prekey envelope carries its own X3DH material and establishes a
|
||||||
|
// fresh session, which must land under the label it arrived on —
|
||||||
|
// that is exactly what a re-link looks like. Only ratchet envelopes,
|
||||||
|
// which need state that already exists, follow an alias.
|
||||||
|
const target =
|
||||||
|
envelope.type === 'prekey' ? address : await this.resolveLabel(address);
|
||||||
|
return this.withSpan('decrypt', target, async () => {
|
||||||
if (envelope.type === 'prekey') {
|
if (envelope.type === 'prekey') {
|
||||||
return this.decryptPreKeyMessage(address, envelope.content as PreKeyMessage);
|
return this.decryptPreKeyMessage(target, envelope.content as PreKeyMessage);
|
||||||
}
|
}
|
||||||
return this.decryptRatchetMessage(address, envelope.content as RatchetMessage);
|
return this.decryptRatchetMessage(target, envelope.content as RatchetMessage);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -165,6 +165,36 @@ export interface StorageProvider {
|
|||||||
/** Remove session for a peer */
|
/** Remove session for a peer */
|
||||||
removeSession(address: string): Promise<void>;
|
removeSession(address: string): Promise<void>;
|
||||||
|
|
||||||
|
// ─── Session label aliases (V4.12) ────────────────────────
|
||||||
|
//
|
||||||
|
// First contact forces the receiver to label a session by the only
|
||||||
|
// sender hint the relay surfaces — an 8-byte signing-key fingerprint
|
||||||
|
// (`fp:<hex>`). A later in-band announcement reveals the peer's
|
||||||
|
// canonical address and `aliasSession` moves the session there.
|
||||||
|
//
|
||||||
|
// The peer, however, keeps sending under whatever label its own
|
||||||
|
// transport derives — which for a fingerprint-hinted relay is still
|
||||||
|
// `fp:<hex>`. Before V4.12 that binding lived only in the consumer's
|
||||||
|
// memory: after a restart the alias was gone, inbound ratchet frames
|
||||||
|
// resolved to `fp:<hex>`, found no session there, and failed forever
|
||||||
|
// (the peer holds a valid session so it never re-runs X3DH).
|
||||||
|
//
|
||||||
|
// Persisting the alias makes the binding survive restarts, so
|
||||||
|
// `getSession` can follow it. Optional so third-party storage
|
||||||
|
// implementations keep compiling — they simply lose alias recovery.
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Record that `alias` names the same peer session as `canonical`.
|
||||||
|
* Idempotent upsert on `alias`.
|
||||||
|
*/
|
||||||
|
saveSessionAlias?(alias: string, canonical: string): Promise<void>;
|
||||||
|
|
||||||
|
/** Resolve an alias to its canonical label (null when unaliased). */
|
||||||
|
getSessionAlias?(alias: string): Promise<string | null>;
|
||||||
|
|
||||||
|
/** Drop every alias pointing at `canonical` (session teardown). */
|
||||||
|
removeSessionAliasesFor?(canonical: string): Promise<void>;
|
||||||
|
|
||||||
/** Check if we trust a remote identity key (for TOFU or pinned keys) */
|
/** Check if we trust a remote identity key (for TOFU or pinned keys) */
|
||||||
isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean>;
|
isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean>;
|
||||||
|
|
||||||
|
|||||||
165
packages/shade-core/tests/session-aliases.test.ts
Normal file
165
packages/shade-core/tests/session-aliases.test.ts
Normal file
@@ -0,0 +1,165 @@
|
|||||||
|
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||||
|
import { SubtleCryptoProvider, MemoryStorage } from '@shade/crypto-web';
|
||||||
|
import { ShadeSessionManager } from '../src/index.js';
|
||||||
|
|
||||||
|
const crypto = new SubtleCryptoProvider();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Durable session-label aliases (V4.12).
|
||||||
|
*
|
||||||
|
* THE BUG THIS FILE EXISTS TO KILL — diagnosed live in Prism:
|
||||||
|
*
|
||||||
|
* A phone pairs with a host. First contact forces the host to label
|
||||||
|
* the session by the only sender hint the relay surfaces, an 8-byte
|
||||||
|
* signing-key fingerprint (`fp:<hex>`). The pair handshake then
|
||||||
|
* announces the phone's canonical address and the host calls
|
||||||
|
* `aliasSession(fp:<hex> → device:<addr>)`, which moved the session
|
||||||
|
* on disk and dropped the binding.
|
||||||
|
*
|
||||||
|
* The phone, however, keeps sending under `fp:<hex>` — its transport
|
||||||
|
* derives the same label from the same relay hint every time. While
|
||||||
|
* the host process lived, an in-memory map papered over the gap.
|
||||||
|
* After a restart that map was empty, every inbound ratchet frame
|
||||||
|
* resolved to `fp:<hex>`, found no session, and failed. The phone
|
||||||
|
* held a perfectly valid session so it never re-ran X3DH — meaning
|
||||||
|
* the failure was permanent and self-inflicted, not transient.
|
||||||
|
*
|
||||||
|
* Observed as `No session for address: fp:579c3b335d66e2c0` on every
|
||||||
|
* receive for three days, with the phone's RPCs timing out forever.
|
||||||
|
*
|
||||||
|
* The fix: `aliasSession` persists the binding, and session lookup
|
||||||
|
* follows it. These tests pin the restart behaviour specifically —
|
||||||
|
* a same-process test cannot fail the way production did.
|
||||||
|
*/
|
||||||
|
describe('session label aliases', () => {
|
||||||
|
let alice: ShadeSessionManager;
|
||||||
|
let bob: ShadeSessionManager;
|
||||||
|
let aliceStorage: MemoryStorage;
|
||||||
|
let bobStorage: MemoryStorage;
|
||||||
|
|
||||||
|
/** The first-contact label Alice is forced to use for Bob. */
|
||||||
|
const FP = 'fp:579c3b335d66e2c0';
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
aliceStorage = new MemoryStorage();
|
||||||
|
bobStorage = new MemoryStorage();
|
||||||
|
alice = new ShadeSessionManager(crypto, aliceStorage);
|
||||||
|
bob = new ShadeSessionManager(crypto, bobStorage);
|
||||||
|
await alice.initialize();
|
||||||
|
await bob.initialize();
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Bob initiates X3DH against Alice, exactly like a phone reaching a
|
||||||
|
* host it just scanned. Returns Bob's first (prekey) envelope.
|
||||||
|
*/
|
||||||
|
async function bobInitiates(target: ShadeSessionManager, initiator: ShadeSessionManager) {
|
||||||
|
const otpks = await target.generateOneTimePreKeys(10);
|
||||||
|
const bundle = await target.createPreKeyBundle();
|
||||||
|
const otpk = otpks[0]!;
|
||||||
|
bundle.oneTimePreKey = { keyId: otpk.keyId, publicKey: otpk.keyPair.publicKey };
|
||||||
|
await initiator.initSessionFromBundle('alice', bundle);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Simulate a host restart: fresh manager, same durable storage. */
|
||||||
|
async function restartAlice(): Promise<ShadeSessionManager> {
|
||||||
|
const revived = new ShadeSessionManager(crypto, aliceStorage);
|
||||||
|
await revived.initialize();
|
||||||
|
return revived;
|
||||||
|
}
|
||||||
|
|
||||||
|
test('an aliased session still decrypts under the old label after a restart', async () => {
|
||||||
|
await bobInitiates(alice, bob);
|
||||||
|
|
||||||
|
// First contact lands under the fingerprint label.
|
||||||
|
const env1 = await bob.encrypt('alice', 'hello, my address is bob');
|
||||||
|
expect(await alice.decrypt(FP, env1)).toBe('hello, my address is bob');
|
||||||
|
|
||||||
|
// Alice canonicalizes to Bob's announced address.
|
||||||
|
await alice.aliasSession(FP, 'bob');
|
||||||
|
|
||||||
|
// The host restarts. Storage survives; every in-memory map does not.
|
||||||
|
const alice2 = await restartAlice();
|
||||||
|
|
||||||
|
// Bob has a valid session and keeps sending under the same label he
|
||||||
|
// always has. Before the fix this threw NoSessionError forever.
|
||||||
|
const env2 = await bob.encrypt('alice', 'still here after restart');
|
||||||
|
expect(await alice2.decrypt(FP, env2)).toBe('still here after restart');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the host can reply under the old label after a restart', async () => {
|
||||||
|
await bobInitiates(alice, bob);
|
||||||
|
const env1 = await bob.encrypt('alice', 'hi');
|
||||||
|
await alice.decrypt(FP, env1);
|
||||||
|
await alice.aliasSession(FP, 'bob');
|
||||||
|
|
||||||
|
const alice2 = await restartAlice();
|
||||||
|
|
||||||
|
// Decrypting is only half of it — a host that cannot encrypt back
|
||||||
|
// leaves every RPC hanging just the same.
|
||||||
|
const reply = await alice2.encrypt(FP, 'reply from the host');
|
||||||
|
expect(await bob.decrypt('alice', reply)).toBe('reply from the host');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a live session under the label wins over an alias (re-link)', async () => {
|
||||||
|
await bobInitiates(alice, bob);
|
||||||
|
const env1 = await bob.encrypt('alice', 'first pairing');
|
||||||
|
await alice.decrypt(FP, env1);
|
||||||
|
await alice.aliasSession(FP, 'bob');
|
||||||
|
const alice2 = await restartAlice();
|
||||||
|
|
||||||
|
// Bob reinstalls: brand-new identity, same relay fingerprint label.
|
||||||
|
const bob2Storage = new MemoryStorage();
|
||||||
|
const bob2 = new ShadeSessionManager(crypto, bob2Storage);
|
||||||
|
await bob2.initialize();
|
||||||
|
await bobInitiates(alice2, bob2);
|
||||||
|
|
||||||
|
// The prekey envelope must establish a FRESH session under FP rather
|
||||||
|
// than being redirected into the stale aliased one.
|
||||||
|
const fresh1 = await bob2.encrypt('alice', 'fresh contact');
|
||||||
|
expect(await alice2.decrypt(FP, fresh1)).toBe('fresh contact');
|
||||||
|
|
||||||
|
// And subsequent ratchet frames must keep using that new session.
|
||||||
|
const fresh2 = await bob2.encrypt('alice', 'second message');
|
||||||
|
expect(await alice2.decrypt(FP, fresh2)).toBe('second message');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('resolveSessionLabel reports where the state actually lives', async () => {
|
||||||
|
await bobInitiates(alice, bob);
|
||||||
|
const env1 = await bob.encrypt('alice', 'hi');
|
||||||
|
await alice.decrypt(FP, env1);
|
||||||
|
|
||||||
|
expect(await alice.resolveSessionLabel(FP)).toBe(FP);
|
||||||
|
await alice.aliasSession(FP, 'bob');
|
||||||
|
|
||||||
|
const alice2 = await restartAlice();
|
||||||
|
expect(await alice2.resolveSessionLabel(FP)).toBe('bob');
|
||||||
|
// An unaliased label resolves to itself.
|
||||||
|
expect(await alice2.resolveSessionLabel('carol')).toBe('carol');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('resetSession drops aliases pointing at the cleared session', async () => {
|
||||||
|
await bobInitiates(alice, bob);
|
||||||
|
const env1 = await bob.encrypt('alice', 'hi');
|
||||||
|
await alice.decrypt(FP, env1);
|
||||||
|
await alice.aliasSession(FP, 'bob');
|
||||||
|
expect(await aliceStorage.getSessionAlias(FP)).toBe('bob');
|
||||||
|
|
||||||
|
await alice.resetSession('bob');
|
||||||
|
|
||||||
|
// A dangling alias would redirect the next first-contact frame into
|
||||||
|
// a session that no longer exists, defeating the reset.
|
||||||
|
expect(await aliceStorage.getSessionAlias(FP)).toBeNull();
|
||||||
|
expect(await alice.resolveSessionLabel(FP)).toBe(FP);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('aliasing persists the binding to storage', async () => {
|
||||||
|
await bobInitiates(alice, bob);
|
||||||
|
const env1 = await bob.encrypt('alice', 'hi');
|
||||||
|
await alice.decrypt(FP, env1);
|
||||||
|
|
||||||
|
expect(await aliceStorage.getSessionAlias(FP)).toBeNull();
|
||||||
|
await alice.aliasSession(FP, 'bob');
|
||||||
|
expect(await aliceStorage.getSessionAlias(FP)).toBe('bob');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/crypto-web",
|
"name": "@shade/crypto-web",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -15,6 +15,8 @@ export class MemoryStorage implements StorageProvider {
|
|||||||
private signedPreKeys = new Map<number, SignedPreKey>();
|
private signedPreKeys = new Map<number, SignedPreKey>();
|
||||||
private oneTimePreKeys = new Map<number, OneTimePreKey>();
|
private oneTimePreKeys = new Map<number, OneTimePreKey>();
|
||||||
private sessions = new Map<string, SessionState>();
|
private sessions = new Map<string, SessionState>();
|
||||||
|
/** alias → canonical session label (V4.12). */
|
||||||
|
private sessionAliases = new Map<string, string>();
|
||||||
private trustedIdentities = new Map<string, Uint8Array>();
|
private trustedIdentities = new Map<string, Uint8Array>();
|
||||||
private retiredIdentities: RetiredIdentity[] = [];
|
private retiredIdentities: RetiredIdentity[] = [];
|
||||||
|
|
||||||
@@ -82,6 +84,22 @@ export class MemoryStorage implements StorageProvider {
|
|||||||
this.sessions.delete(address);
|
this.sessions.delete(address);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ────────────────────────────────
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
return this.sessionAliases.get(alias) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
this.sessionAliases.set(alias, canonical);
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
for (const [alias, target] of this.sessionAliases) {
|
||||||
|
if (target === canonical) this.sessionAliases.delete(alias);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ────────────────────────────────────────────────
|
// ─── Trust ────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/dashboard",
|
"name": "@shade/dashboard",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "vite",
|
"dev": "vite",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/files",
|
"name": "@shade/files",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -444,8 +444,12 @@ export function createFilesHttpClient(
|
|||||||
};
|
};
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
// Streamed read — only supported when the queue drainer is wired.
|
// Streamed read — only supported when pull-mode was configured.
|
||||||
if (drainer === null) {
|
// `drainer` is assigned asynchronously after the streams bridge has
|
||||||
|
// subscribed, so checking it here races client construction. The
|
||||||
|
// promise itself is the synchronous configuration signal; awaiting it
|
||||||
|
// also guarantees the drainer has been installed by its `.then()`.
|
||||||
|
if (streamsBridgePromise === null) {
|
||||||
throw new InternalFileError(
|
throw new InternalFileError(
|
||||||
`http RPC client received a streamed read (size ${wire.size}) but is in inline-only mode. Pass { outboundQueueUrl, transferBaseUrl } when constructing the client to enable streamed reads.`,
|
`http RPC client received a streamed read (size ${wire.size}) but is in inline-only mode. Pass { outboundQueueUrl, transferBaseUrl } when constructing the client to enable streamed reads.`,
|
||||||
);
|
);
|
||||||
@@ -507,8 +511,11 @@ export function createFilesHttpClient(
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Streamed write — requires the queue drainer + streams-bridge.
|
// Streamed write — requires pull-mode configuration. Do not inspect
|
||||||
if (drainer === null) {
|
// `drainer` as a readiness flag: bridge registration is asynchronous,
|
||||||
|
// and an immediate large write must wait for it instead of being
|
||||||
|
// misclassified as an inline-only client.
|
||||||
|
if (streamsBridgePromise === null) {
|
||||||
throw new ConflictError(
|
throw new ConflictError(
|
||||||
`http RPC client supports inline writes only (≤ ${INLINE_THRESHOLD} bytes). The supplied input was promoted to streams (size ${decision.size ?? 'unknown'}). Pass { outboundQueueUrl, transferBaseUrl } to enable streamed writes.`,
|
`http RPC client supports inline writes only (≤ ${INLINE_THRESHOLD} bytes). The supplied input was promoted to streams (size ${decision.size ?? 'unknown'}). Pass { outboundQueueUrl, transferBaseUrl } to enable streamed writes.`,
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -7,8 +7,10 @@ import {
|
|||||||
} from '@shade/server';
|
} from '@shade/server';
|
||||||
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
||||||
import { Hono } from 'hono';
|
import { Hono } from 'hono';
|
||||||
|
import { createFilesHttpClient, type FileEntry } from '../../src/index.js';
|
||||||
|
|
||||||
const crypto = new SubtleCryptoProvider();
|
const crypto = new SubtleCryptoProvider();
|
||||||
|
type ShadeInstance = Awaited<ReturnType<typeof createShade>>;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Stand up the full pull-mode rig:
|
* Stand up the full pull-mode rig:
|
||||||
@@ -20,7 +22,8 @@ const crypto = new SubtleCryptoProvider();
|
|||||||
* no inbound listener, streams supported via long-poll.
|
* no inbound listener, streams supported via long-poll.
|
||||||
*/
|
*/
|
||||||
async function setupPullRig(opts: {
|
async function setupPullRig(opts: {
|
||||||
bobHandler: Parameters<NonNullable<Awaited<ReturnType<typeof createShade>>['files']>['serve']>[0];
|
bobHandler: Parameters<NonNullable<ShadeInstance['files']>['serve']>[0];
|
||||||
|
wrapClientShade?: (shade: ShadeInstance) => Parameters<typeof createFilesHttpClient>[0];
|
||||||
}) {
|
}) {
|
||||||
const prekey = createPrekeyServer({
|
const prekey = createPrekeyServer({
|
||||||
crypto,
|
crypto,
|
||||||
@@ -46,7 +49,7 @@ async function setupPullRig(opts: {
|
|||||||
const bobServer = Bun.serve({ port: 0, fetch: app.fetch });
|
const bobServer = Bun.serve({ port: 0, fetch: app.fetch });
|
||||||
const baseUrl = `http://localhost:${bobServer.port}`;
|
const baseUrl = `http://localhost:${bobServer.port}`;
|
||||||
|
|
||||||
const fs = alice.files.httpClient('bob', {
|
const fs = createFilesHttpClient(opts.wrapClientShade?.(alice) ?? alice, 'bob', {
|
||||||
rpcUrl: `${baseUrl}/rpc`,
|
rpcUrl: `${baseUrl}/rpc`,
|
||||||
outboundQueueUrl: `${baseUrl}/queue`,
|
outboundQueueUrl: `${baseUrl}/queue`,
|
||||||
transferBaseUrl: baseUrl,
|
transferBaseUrl: baseUrl,
|
||||||
@@ -70,6 +73,83 @@ async function setupPullRig(opts: {
|
|||||||
}
|
}
|
||||||
|
|
||||||
describe('@shade/files HTTP RPC — pull-mode streams', () => {
|
describe('@shade/files HTTP RPC — pull-mode streams', () => {
|
||||||
|
test('immediate streamed write waits for asynchronous stream-bridge registration', async () => {
|
||||||
|
const payload = new Uint8Array(512 * 1024);
|
||||||
|
for (let i = 0; i < payload.length; i++) payload[i] = (i * 53) & 0xff;
|
||||||
|
|
||||||
|
let releaseRegistration!: () => void;
|
||||||
|
const registrationGate = new Promise<void>((resolve) => {
|
||||||
|
releaseRegistration = resolve;
|
||||||
|
});
|
||||||
|
let registrationStarted!: () => void;
|
||||||
|
const started = new Promise<void>((resolve) => {
|
||||||
|
registrationStarted = resolve;
|
||||||
|
});
|
||||||
|
|
||||||
|
const rig = await setupPullRig({
|
||||||
|
wrapClientShade: (alice) =>
|
||||||
|
new Proxy(alice, {
|
||||||
|
get(target, property) {
|
||||||
|
if (property === 'onIncomingTransfer') {
|
||||||
|
return async (handler: Parameters<typeof target.onIncomingTransfer>[0]) => {
|
||||||
|
registrationStarted();
|
||||||
|
await registrationGate;
|
||||||
|
return await target.onIncomingTransfer(handler);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const value = Reflect.get(target, property, target) as unknown;
|
||||||
|
return typeof value === 'function' ? value.bind(target) : value;
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
bobHandler: {
|
||||||
|
write: async (ctx) => {
|
||||||
|
const content = ctx.args.content;
|
||||||
|
if (content.kind !== 'streams') throw new Error('expected streamed content');
|
||||||
|
const reader = content.stream.getReader();
|
||||||
|
let received = 0;
|
||||||
|
while (true) {
|
||||||
|
const { value, done } = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
received += value?.byteLength ?? 0;
|
||||||
|
}
|
||||||
|
reader.releaseLock();
|
||||||
|
await content.sha256;
|
||||||
|
const entry: FileEntry = {
|
||||||
|
name: 'immediate.bin',
|
||||||
|
kind: 'file',
|
||||||
|
size: received,
|
||||||
|
mtime: Date.now(),
|
||||||
|
metadata: {},
|
||||||
|
};
|
||||||
|
return { entry };
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
let settled = false;
|
||||||
|
const write = rig.fs.write('/immediate.bin', payload);
|
||||||
|
void write.then(
|
||||||
|
() => {
|
||||||
|
settled = true;
|
||||||
|
},
|
||||||
|
() => {
|
||||||
|
settled = true;
|
||||||
|
},
|
||||||
|
);
|
||||||
|
await started;
|
||||||
|
await Bun.sleep(10);
|
||||||
|
expect(settled).toBe(false);
|
||||||
|
|
||||||
|
releaseRegistration();
|
||||||
|
const result = await write;
|
||||||
|
expect(result.entry.size).toBe(payload.byteLength);
|
||||||
|
} finally {
|
||||||
|
releaseRegistration();
|
||||||
|
await rig.teardown();
|
||||||
|
}
|
||||||
|
}, 15_000);
|
||||||
|
|
||||||
test('streamed read (4 MiB) via long-poll queue', async () => {
|
test('streamed read (4 MiB) via long-poll queue', async () => {
|
||||||
const payload = new Uint8Array(4 * 1024 * 1024);
|
const payload = new Uint8Array(4 * 1024 * 1024);
|
||||||
for (let i = 0; i < payload.length; i++) payload[i] = (i * 97) & 0xff;
|
for (let i = 0; i < payload.length; i++) payload[i] = (i * 97) & 0xff;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/inbox-server",
|
"name": "@shade/inbox-server",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/inbox",
|
"name": "@shade/inbox",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/key-transparency",
|
"name": "@shade/key-transparency",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/keychain",
|
"name": "@shade/keychain",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/observability",
|
"name": "@shade/observability",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/observer",
|
"name": "@shade/observer",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/proto",
|
"name": "@shade/proto",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/recovery",
|
"name": "@shade/recovery",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/sdk",
|
"name": "@shade/sdk",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -701,6 +701,22 @@ export class Shade {
|
|||||||
this.decryptChains.delete(oldLabel);
|
this.decryptChains.delete(oldLabel);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolve a session label to the label its state actually lives under,
|
||||||
|
* following any alias left behind by `aliasSession`.
|
||||||
|
*
|
||||||
|
* Transports need this to route by the canonical address after a
|
||||||
|
* restart, when the only sender hint they hold is the first-contact
|
||||||
|
* `fp:<hex>` label. `encrypt`/`decrypt` resolve internally — this is
|
||||||
|
* for callers that must know the address itself.
|
||||||
|
*
|
||||||
|
* V4.12 — durable session-label aliases.
|
||||||
|
*/
|
||||||
|
async resolveSessionLabel(label: string): Promise<string> {
|
||||||
|
if (!this.initialized) throw new Error('Not initialized');
|
||||||
|
return this.manager.resolveSessionLabel(label);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Accept a peer's rotated identity. Bumps the per-peer identity-version
|
* Accept a peer's rotated identity. Bumps the per-peer identity-version
|
||||||
* counter so any earlier verification automatically goes stale, then
|
* counter so any earlier verification automatically goes stale, then
|
||||||
|
|||||||
@@ -162,10 +162,15 @@ describe('createShade — happy path', () => {
|
|||||||
const env3 = await alice.send('bob', 'reply 2');
|
const env3 = await alice.send('bob', 'reply 2');
|
||||||
expect(await bob.receive('alice', env3)).toBe('reply 2');
|
expect(await bob.receive('alice', env3)).toBe('reply 2');
|
||||||
|
|
||||||
// The old fp-label has no session — receive under it would now
|
// V4.12: the fp-label is no longer a dead end. `aliasSession` leaves
|
||||||
// fail. (We don't assert the error shape, only that the label is
|
// a durable binding behind, so a peer that keeps sending under the
|
||||||
// gone.)
|
// first-contact label — which is exactly what a fingerprint-hinted
|
||||||
await expect(alice.receive(fpLabel, env3)).rejects.toThrow();
|
// relay makes it do — still reaches the canonicalized session.
|
||||||
|
// See `shade-core/tests/session-aliases.test.ts` for the restart
|
||||||
|
// behaviour this binding exists to protect.
|
||||||
|
expect(await alice.resolveSessionLabel(fpLabel)).toBe('bob');
|
||||||
|
const env4 = await bob.send('alice', 'reply 3');
|
||||||
|
expect(await alice.receive(fpLabel, env4)).toBe('reply 3');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('aliasSession refuses to overwrite an existing session', async () => {
|
test('aliasSession refuses to overwrite an existing session', async () => {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/server",
|
"name": "@shade/server",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
@@ -9,6 +9,8 @@
|
|||||||
"@shade/inbox-server": "workspace:*",
|
"@shade/inbox-server": "workspace:*",
|
||||||
"@shade/key-transparency": "workspace:*",
|
"@shade/key-transparency": "workspace:*",
|
||||||
"@shade/observability": "workspace:*",
|
"@shade/observability": "workspace:*",
|
||||||
|
"@shade/storage-sqlite": "workspace:*",
|
||||||
|
"@shade/vault": "workspace:*",
|
||||||
"hono": "^4.12.12"
|
"hono": "^4.12.12"
|
||||||
},
|
},
|
||||||
"optionalDependencies": {
|
"optionalDependencies": {
|
||||||
|
|||||||
@@ -83,6 +83,30 @@ async function createInboxStore(): Promise<InboxStore & { close?: () => void | P
|
|||||||
* also opt the blob store *off* entirely via `SHADE_DISABLE_BLOB=1` —
|
* also opt the blob store *off* entirely via `SHADE_DISABLE_BLOB=1` —
|
||||||
* useful for relays that only want the inbox surface.
|
* useful for relays that only want the inbox surface.
|
||||||
*/
|
*/
|
||||||
|
/**
|
||||||
|
* Build the vault store, or `null` when none is configured.
|
||||||
|
*
|
||||||
|
* Deliberately NOT falling back to an in-memory store the way the blob
|
||||||
|
* primitive does. A vault holds the user's actual files: an in-memory one
|
||||||
|
* accepts every write, serves every read, and loses the lot on the next
|
||||||
|
* container recreate — a backup that reports success and is not there. The
|
||||||
|
* blob store's silent fallback destroyed Prism's profile on 2026-08-12
|
||||||
|
* precisely because nothing failed loudly. Here, no path means no vault.
|
||||||
|
*/
|
||||||
|
async function createVaultStore(): Promise<import('@shade/vault').VaultStore | null> {
|
||||||
|
const sqlitePath = process.env.SHADE_VAULT_DB_PATH;
|
||||||
|
if (!sqlitePath) {
|
||||||
|
logger.warn(
|
||||||
|
'Vault not enabled — set SHADE_VAULT_DB_PATH to a path on a persistent volume. ' +
|
||||||
|
'There is no in-memory fallback on purpose: a vault that forgets is not a backup.',
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const { SqliteVaultStore } = await import('@shade/storage-sqlite');
|
||||||
|
logger.info('Using SQLite vault store', { path: sqlitePath });
|
||||||
|
return new SqliteVaultStore(sqlitePath);
|
||||||
|
}
|
||||||
|
|
||||||
async function createBlobStore(): Promise<BlobStore & { close?: () => void | Promise<void> }> {
|
async function createBlobStore(): Promise<BlobStore & { close?: () => void | Promise<void> }> {
|
||||||
const sqlitePath = process.env.SHADE_BLOB_DB_PATH;
|
const sqlitePath = process.env.SHADE_BLOB_DB_PATH;
|
||||||
const pgUrl = process.env.SHADE_BLOB_PG_URL ?? process.env.SHADE_PREKEY_PG_URL;
|
const pgUrl = process.env.SHADE_BLOB_PG_URL ?? process.env.SHADE_PREKEY_PG_URL;
|
||||||
@@ -255,6 +279,21 @@ if (blobDisabled) {
|
|||||||
logger.info('Blob primitive enabled', { route: '/v1/blob/:slotId' });
|
logger.info('Blob primitive enabled', { route: '/v1/blob/:slotId' });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// V4.13 — vault: server-side encrypted file store. Where the blob primitive
|
||||||
|
// holds one small blob per account, a vault holds a whole collection with an
|
||||||
|
// append-only version log. Off unless a store is configured, because a vault
|
||||||
|
// that silently lives in RAM is worse than no vault at all — see the warning
|
||||||
|
// on SHADE_VAULT_DB_PATH below.
|
||||||
|
const vaultDisabled = process.env.SHADE_DISABLE_VAULT === '1';
|
||||||
|
const vaultStore = vaultDisabled ? null : await createVaultStore();
|
||||||
|
if (vaultDisabled) {
|
||||||
|
logger.info('Vault disabled (SHADE_DISABLE_VAULT=1)');
|
||||||
|
} else if (vaultStore) {
|
||||||
|
const { createVaultRoutes } = await import('@shade/vault/server');
|
||||||
|
app.route('/', createVaultRoutes(vaultStore, crypto));
|
||||||
|
logger.info('Vault enabled', { route: '/v1/vault/:vaultId' });
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Optional: Observer + Dashboard ──────────────────────────
|
// ─── Optional: Observer + Dashboard ──────────────────────────
|
||||||
|
|
||||||
const observerToken = process.env.SHADE_OBSERVER_TOKEN;
|
const observerToken = process.env.SHADE_OBSERVER_TOKEN;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-encrypted",
|
"name": "@shade/storage-encrypted",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -34,12 +34,16 @@ export {
|
|||||||
deriveBlobSlotId,
|
deriveBlobSlotId,
|
||||||
deriveBlobKey,
|
deriveBlobKey,
|
||||||
deriveBlobSigningSeed,
|
deriveBlobSigningSeed,
|
||||||
|
deriveVaultId,
|
||||||
|
deriveVaultContentKey,
|
||||||
|
deriveVaultSigningSeed,
|
||||||
} from './crypto/kdf.js';
|
} from './crypto/kdf.js';
|
||||||
export {
|
export {
|
||||||
AEAD_NONCE_LEN,
|
AEAD_NONCE_LEN,
|
||||||
AEAD_TAG_LEN,
|
AEAD_TAG_LEN,
|
||||||
aeadSeal,
|
aeadSeal,
|
||||||
aeadOpen,
|
aeadOpen,
|
||||||
|
randomNonce,
|
||||||
} from './crypto/aead.js';
|
} from './crypto/aead.js';
|
||||||
export {
|
export {
|
||||||
COL,
|
COL,
|
||||||
|
|||||||
@@ -33,9 +33,32 @@ async function importKey(key: Uint8Array, usages: WebCryptoKeyUsage[]): Promise<
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Encrypt a plaintext blob with the given key and a deterministic nonce.
|
* A fresh random 12-byte nonce.
|
||||||
* Returns `nonce || ct||tag` as a single Uint8Array suitable for direct
|
*
|
||||||
* BLOB storage.
|
* AES-GCM tolerates key reuse but not (key, nonce) reuse: two ciphertexts
|
||||||
|
* sealed under the same pair leak the XOR of their plaintexts and, worse,
|
||||||
|
* the GHASH authentication subkey — the "forbidden attack", which yields
|
||||||
|
* tag forgery for that key. Mutable rows are re-sealed on every change
|
||||||
|
* (`saveSession` runs on each ratchet step), so a nonce derived from row
|
||||||
|
* identity alone repeats by construction.
|
||||||
|
*
|
||||||
|
* Random beats a counter here because the codec is shared across SQLite
|
||||||
|
* and Postgres and has no durable per-row write counter to lean on. At
|
||||||
|
* 96 bits, collision probability stays negligible far past any realistic
|
||||||
|
* number of re-saves.
|
||||||
|
*/
|
||||||
|
export function randomNonce(): Uint8Array {
|
||||||
|
return globalThis.crypto.getRandomValues(new Uint8Array(NONCE_LEN));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Encrypt a plaintext blob with the given key and nonce. Returns
|
||||||
|
* `nonce || ct||tag` as a single Uint8Array suitable for direct BLOB
|
||||||
|
* storage.
|
||||||
|
*
|
||||||
|
* Callers must pass a nonce that is fresh for this key — see
|
||||||
|
* {@link randomNonce}. Never derive one deterministically from row
|
||||||
|
* identity for a row that can be written more than once.
|
||||||
*/
|
*/
|
||||||
export async function aeadSeal(
|
export async function aeadSeal(
|
||||||
key: Uint8Array,
|
key: Uint8Array,
|
||||||
@@ -58,9 +81,19 @@ export async function aeadSeal(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Decrypt a `nonce || ct||tag` blob. The expected nonce is verified against
|
* Decrypt a `nonce || ct||tag` blob.
|
||||||
* the prefix to detect tampering before we even reach the AEAD; if the
|
*
|
||||||
* caller passes a `expectedNonce`, mismatch throws before SubtleCrypto runs.
|
* The nonce always comes from the blob prefix, which is what makes the
|
||||||
|
* move to random nonces backward compatible: a blob sealed under the old
|
||||||
|
* deterministic scheme opens unchanged, because its nonce was already
|
||||||
|
* stored the same way.
|
||||||
|
*
|
||||||
|
* @param expectedNonce **Deprecated.** Only meaningful when the nonce is
|
||||||
|
* a pure function of row identity, which is exactly the pattern that made
|
||||||
|
* (key, nonce) repeat. It adds no tamper detection that the AEAD tag and
|
||||||
|
* the (table, column, pk) AAD do not already provide — a flipped bit in
|
||||||
|
* the nonce fails the tag. Retained so existing callers keep compiling;
|
||||||
|
* pass nothing.
|
||||||
*/
|
*/
|
||||||
export async function aeadOpen(
|
export async function aeadOpen(
|
||||||
key: Uint8Array,
|
key: Uint8Array,
|
||||||
|
|||||||
@@ -106,12 +106,20 @@ export function deriveFieldKey(storageKey: Uint8Array, table: string, column: st
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Derive a deterministic 12-byte AEAD nonce from a row key (typically the
|
* Derive a deterministic 12-byte AEAD nonce from a row key plus (table, pk).
|
||||||
* field key) plus (table, pk) binding. With per-field keys, deterministic
|
*
|
||||||
* nonces are safe because each (key, plaintext) pair appears at most once
|
* @deprecated Unsafe for anything that can be written twice, which is every
|
||||||
* — re-saving the same row reuses the (nonce, key) pair only because the
|
* mutable row. The original rationale held that re-saving a row was fine
|
||||||
* plaintext also changes (chain ratchet, prekey state, etc.). The AAD
|
* "because the plaintext also changes" — but that is precisely the case
|
||||||
* also binds (table, column, pk) so swapping is rejected on decrypt.
|
* AES-GCM forbids: reusing (key, nonce) across *different* plaintexts leaks
|
||||||
|
* their XOR and the GHASH subkey (the forbidden attack), enabling tag
|
||||||
|
* forgery. `saveSession` re-seals on every ratchet step, so the pair
|
||||||
|
* repeated by construction. Use {@link randomNonce} from `./aead.js`.
|
||||||
|
*
|
||||||
|
* Kept exported (it is part of the published surface via `index.ts` and
|
||||||
|
* `crypto.ts`) so consumers keep compiling. No caller inside this package
|
||||||
|
* uses it any more. Reading old data needs no migration: `aeadOpen` has
|
||||||
|
* always taken the nonce from the blob prefix.
|
||||||
*/
|
*/
|
||||||
export function deriveNonce(rowKey: Uint8Array, table: string, pk: string): Uint8Array {
|
export function deriveNonce(rowKey: Uint8Array, table: string, pk: string): Uint8Array {
|
||||||
const out = hkdfDerive(rowKey, `shade-row-nonce-v1:${table}:${pk}`, 12);
|
const out = hkdfDerive(rowKey, `shade-row-nonce-v1:${table}:${pk}`, 12);
|
||||||
@@ -161,3 +169,42 @@ export function deriveBlobKey(masterKey: Uint8Array, app: string): Uint8Array {
|
|||||||
export function deriveBlobSigningSeed(masterKey: Uint8Array, app: string): Uint8Array {
|
export function deriveBlobSigningSeed(masterKey: Uint8Array, app: string): Uint8Array {
|
||||||
return hkdfDerive(masterKey, `shade-blob-sig-v1:${app}`, 32);
|
return hkdfDerive(masterKey, `shade-blob-sig-v1:${app}`, 32);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── V4.13 — vault (server-side encrypted file store) ──────────
|
||||||
|
//
|
||||||
|
// The blob primitive above holds ONE small blob per slot. A vault holds
|
||||||
|
// a whole collection — a project workspace, a mailbox, a drive folder —
|
||||||
|
// as content-addressed objects plus an append-only log of manifests.
|
||||||
|
//
|
||||||
|
// Its own HKDF branch rather than a reuse of `shade-blob-*-v1`, for two
|
||||||
|
// reasons. A vault key is used on far more ciphertext than a profile
|
||||||
|
// blob ever is, and the two have different blast radii: whoever holds
|
||||||
|
// the vault key can read every file, while the profile key exposes only
|
||||||
|
// the host list. Separate labels mean compromising one cannot be turned
|
||||||
|
// into the other, even though both hang off the same master.
|
||||||
|
//
|
||||||
|
// vaultId = HKDF(masterKey, info=`shade-vault-id-v1:${app}`)
|
||||||
|
// contentKey = HKDF(masterKey, info=`shade-vault-content-v1:${app}`)
|
||||||
|
// sigSeed = HKDF(masterKey, info=`shade-vault-sig-v1:${app}`)
|
||||||
|
|
||||||
|
/** 32-byte vault identifier. Opaque to the relay, like a slotId. */
|
||||||
|
export function deriveVaultId(masterKey: Uint8Array, app: string): Uint8Array {
|
||||||
|
return hkdfDerive(masterKey, `shade-vault-id-v1:${app}`, 32);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* AEAD key for the vault's objects and manifests.
|
||||||
|
*
|
||||||
|
* One key for the whole collection, not one per file: per-file keys would
|
||||||
|
* have to be stored somewhere, and that somewhere would be a manifest
|
||||||
|
* encrypted under a collection key anyway. Object AAD binds the content
|
||||||
|
* hash, so a ciphertext cannot be moved to another object's name.
|
||||||
|
*/
|
||||||
|
export function deriveVaultContentKey(masterKey: Uint8Array, app: string): Uint8Array {
|
||||||
|
return hkdfDerive(masterKey, `shade-vault-content-v1:${app}`, 32);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 32-byte Ed25519 signing seed; TOFU-pinned by the relay on first write. */
|
||||||
|
export function deriveVaultSigningSeed(masterKey: Uint8Array, app: string): Uint8Array {
|
||||||
|
return hkdfDerive(masterKey, `shade-vault-sig-v1:${app}`, 32);
|
||||||
|
}
|
||||||
|
|||||||
@@ -22,8 +22,8 @@ import {
|
|||||||
serializeSignedPreKey, deserializeSignedPreKey,
|
serializeSignedPreKey, deserializeSignedPreKey,
|
||||||
toBase64, fromBase64,
|
toBase64, fromBase64,
|
||||||
} from '@shade/core';
|
} from '@shade/core';
|
||||||
import { aeadOpen, aeadSeal } from './aead.js';
|
import { aeadOpen, aeadSeal, randomNonce } from './aead.js';
|
||||||
import { buildAad, deriveNonce } from './kdf.js';
|
import { buildAad } from './kdf.js';
|
||||||
import type { KeyManager } from './key-manager.js';
|
import type { KeyManager } from './key-manager.js';
|
||||||
|
|
||||||
const TEXT_ENCODER = new TextEncoder();
|
const TEXT_ENCODER = new TextEncoder();
|
||||||
@@ -64,9 +64,8 @@ export async function sealString(
|
|||||||
plaintext: string,
|
plaintext: string,
|
||||||
): Promise<Uint8Array> {
|
): Promise<Uint8Array> {
|
||||||
const key = km.fieldKey(table, column);
|
const key = km.fieldKey(table, column);
|
||||||
const nonce = deriveNonce(key, table, pk);
|
|
||||||
const aad = buildAad(table, column, pk);
|
const aad = buildAad(table, column, pk);
|
||||||
return aeadSeal(key, nonce, TEXT_ENCODER.encode(plaintext), aad);
|
return aeadSeal(key, randomNonce(), TEXT_ENCODER.encode(plaintext), aad);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Decrypt a blob into a string, reconstructing AAD from (table, column, pk). */
|
/** Decrypt a blob into a string, reconstructing AAD from (table, column, pk). */
|
||||||
@@ -78,9 +77,8 @@ export async function openString(
|
|||||||
blob: Uint8Array,
|
blob: Uint8Array,
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
const key = km.fieldKey(table, column);
|
const key = km.fieldKey(table, column);
|
||||||
const expectedNonce = deriveNonce(key, table, pk);
|
|
||||||
const aad = buildAad(table, column, pk);
|
const aad = buildAad(table, column, pk);
|
||||||
const pt = await aeadOpen(key, blob, aad, expectedNonce);
|
const pt = await aeadOpen(key, blob, aad);
|
||||||
return TEXT_DECODER.decode(pt);
|
return TEXT_DECODER.decode(pt);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -93,9 +91,8 @@ export async function sealBytes(
|
|||||||
plaintext: Uint8Array,
|
plaintext: Uint8Array,
|
||||||
): Promise<Uint8Array> {
|
): Promise<Uint8Array> {
|
||||||
const key = km.fieldKey(table, column);
|
const key = km.fieldKey(table, column);
|
||||||
const nonce = deriveNonce(key, table, pk);
|
|
||||||
const aad = buildAad(table, column, pk);
|
const aad = buildAad(table, column, pk);
|
||||||
return aeadSeal(key, nonce, plaintext, aad);
|
return aeadSeal(key, randomNonce(), plaintext, aad);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Decrypt arbitrary bytes payload. */
|
/** Decrypt arbitrary bytes payload. */
|
||||||
@@ -107,9 +104,8 @@ export async function openBytes(
|
|||||||
blob: Uint8Array,
|
blob: Uint8Array,
|
||||||
): Promise<Uint8Array> {
|
): Promise<Uint8Array> {
|
||||||
const key = km.fieldKey(table, column);
|
const key = km.fieldKey(table, column);
|
||||||
const expectedNonce = deriveNonce(key, table, pk);
|
|
||||||
const aad = buildAad(table, column, pk);
|
const aad = buildAad(table, column, pk);
|
||||||
return aeadOpen(key, blob, aad, expectedNonce);
|
return aeadOpen(key, blob, aad);
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── Typed encoders for each StorageProvider entity ──────────────────────
|
// ─── Typed encoders for each StorageProvider entity ──────────────────────
|
||||||
|
|||||||
@@ -19,12 +19,16 @@ export {
|
|||||||
deriveBlobSlotId,
|
deriveBlobSlotId,
|
||||||
deriveBlobKey,
|
deriveBlobKey,
|
||||||
deriveBlobSigningSeed,
|
deriveBlobSigningSeed,
|
||||||
|
deriveVaultId,
|
||||||
|
deriveVaultContentKey,
|
||||||
|
deriveVaultSigningSeed,
|
||||||
} from './crypto/kdf.js';
|
} from './crypto/kdf.js';
|
||||||
export {
|
export {
|
||||||
AEAD_NONCE_LEN,
|
AEAD_NONCE_LEN,
|
||||||
AEAD_TAG_LEN,
|
AEAD_TAG_LEN,
|
||||||
aeadSeal,
|
aeadSeal,
|
||||||
aeadOpen,
|
aeadOpen,
|
||||||
|
randomNonce,
|
||||||
} from './crypto/aead.js';
|
} from './crypto/aead.js';
|
||||||
export { EncryptedSQLiteStorage } from './storage/encrypted-sqlite.js';
|
export { EncryptedSQLiteStorage } from './storage/encrypted-sqlite.js';
|
||||||
export {
|
export {
|
||||||
|
|||||||
@@ -100,6 +100,10 @@ export class EncryptedIndexedDBStorage implements StorageProvider {
|
|||||||
});
|
});
|
||||||
members.createIndex('byChannelId', 'channelId');
|
members.createIndex('byChannelId', 'channelId');
|
||||||
}
|
}
|
||||||
|
if (oldVersion < 3) {
|
||||||
|
const aliases = db.createObjectStore('session_aliases_enc', { keyPath: 'alias' });
|
||||||
|
aliases.createIndex('byCanonical', 'canonical');
|
||||||
|
}
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
const store = new EncryptedIndexedDBStorage(db, opts.keyManager);
|
const store = new EncryptedIndexedDBStorage(db, opts.keyManager);
|
||||||
@@ -212,6 +216,27 @@ export class EncryptedIndexedDBStorage implements StorageProvider {
|
|||||||
await this.db.delete('sessions_enc', address);
|
await this.db.delete('sessions_enc', address);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ─────────────────────────────────
|
||||||
|
//
|
||||||
|
// Labels are already the clear-text keyPath of sessions_enc, so the
|
||||||
|
// mapping between two of them exposes nothing the store didn't hold.
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
const row = await this.db.get('session_aliases_enc', alias);
|
||||||
|
return row?.canonical ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
await this.db.put('session_aliases_enc', { alias, canonical });
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
const tx = this.db.transaction('session_aliases_enc', 'readwrite');
|
||||||
|
const matches = await tx.store.index('byCanonical').getAllKeys(canonical);
|
||||||
|
await Promise.all(matches.map((key) => tx.store.delete(key)));
|
||||||
|
await tx.done;
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ─────────────────────────────────────────────────
|
// ─── Trust ─────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
@@ -466,7 +491,7 @@ export class EncryptedIndexedDBStorage implements StorageProvider {
|
|||||||
|
|
||||||
// ─── Schema ────────────────────────────────────────────────
|
// ─── Schema ────────────────────────────────────────────────
|
||||||
|
|
||||||
const SCHEMA_VERSION = 2;
|
const SCHEMA_VERSION = 3;
|
||||||
|
|
||||||
interface MetaRow { key: string; value: string }
|
interface MetaRow { key: string; value: string }
|
||||||
interface IdentityRow { id: 1; ciphertext: Uint8Array }
|
interface IdentityRow { id: 1; ciphertext: Uint8Array }
|
||||||
@@ -522,6 +547,11 @@ interface EncryptedShadeSchema extends DBSchema {
|
|||||||
signed_prekeys_enc: { key: number; value: SignedPreKeyRow };
|
signed_prekeys_enc: { key: number; value: SignedPreKeyRow };
|
||||||
one_time_prekeys_enc: { key: number; value: OneTimePreKeyRow };
|
one_time_prekeys_enc: { key: number; value: OneTimePreKeyRow };
|
||||||
sessions_enc: { key: string; value: SessionRow };
|
sessions_enc: { key: string; value: SessionRow };
|
||||||
|
session_aliases_enc: {
|
||||||
|
key: string;
|
||||||
|
value: { alias: string; canonical: string };
|
||||||
|
indexes: { byCanonical: string };
|
||||||
|
};
|
||||||
trusted_identities_enc: { key: string; value: TrustedIdentityRow };
|
trusted_identities_enc: { key: string; value: TrustedIdentityRow };
|
||||||
retired_identities_enc: {
|
retired_identities_enc: {
|
||||||
key: number;
|
key: number;
|
||||||
|
|||||||
@@ -183,6 +183,30 @@ export class EncryptedPostgresStorage implements StorageProvider {
|
|||||||
await this.sql`DELETE FROM shade_sessions_enc WHERE address = ${address}`;
|
await this.sql`DELETE FROM shade_sessions_enc WHERE address = ${address}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ─────────────────────────────────
|
||||||
|
//
|
||||||
|
// Labels are already stored in the clear as the sessions_enc primary
|
||||||
|
// key, so a mapping between two of them reveals nothing new.
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
const rows = await this.sql<Array<{ canonical: string }>>`
|
||||||
|
SELECT canonical FROM shade_session_aliases_enc WHERE alias = ${alias}
|
||||||
|
`;
|
||||||
|
return rows.length ? rows[0]!.canonical : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
await this.sql`
|
||||||
|
INSERT INTO shade_session_aliases_enc (alias, canonical)
|
||||||
|
VALUES (${alias}, ${canonical})
|
||||||
|
ON CONFLICT (alias) DO UPDATE SET canonical = EXCLUDED.canonical
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
await this.sql`DELETE FROM shade_session_aliases_enc WHERE canonical = ${canonical}`;
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ─────────────────────────────────────────────────
|
// ─── Trust ─────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
@@ -515,6 +539,16 @@ export async function ensureEncryptedClientTables(sql: Sql): Promise<void> {
|
|||||||
ciphertext BYTEA NOT NULL
|
ciphertext BYTEA NOT NULL
|
||||||
)
|
)
|
||||||
`;
|
`;
|
||||||
|
await sql`
|
||||||
|
CREATE TABLE IF NOT EXISTS shade_session_aliases_enc (
|
||||||
|
alias TEXT PRIMARY KEY,
|
||||||
|
canonical TEXT NOT NULL
|
||||||
|
)
|
||||||
|
`;
|
||||||
|
await sql`
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_shade_session_aliases_enc_canonical
|
||||||
|
ON shade_session_aliases_enc(canonical)
|
||||||
|
`;
|
||||||
await sql`
|
await sql`
|
||||||
CREATE TABLE IF NOT EXISTS shade_trusted_identities_enc (
|
CREATE TABLE IF NOT EXISTS shade_trusted_identities_enc (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
|
|||||||
@@ -52,6 +52,9 @@ export class EncryptedSQLiteStorage implements StorageProvider {
|
|||||||
getSession: ReturnType<Database['prepare']>;
|
getSession: ReturnType<Database['prepare']>;
|
||||||
saveSession: ReturnType<Database['prepare']>;
|
saveSession: ReturnType<Database['prepare']>;
|
||||||
removeSession: ReturnType<Database['prepare']>;
|
removeSession: ReturnType<Database['prepare']>;
|
||||||
|
getSessionAlias: ReturnType<Database['prepare']>;
|
||||||
|
saveSessionAlias: ReturnType<Database['prepare']>;
|
||||||
|
removeAliasesFor: ReturnType<Database['prepare']>;
|
||||||
getTrust: ReturnType<Database['prepare']>;
|
getTrust: ReturnType<Database['prepare']>;
|
||||||
saveTrust: ReturnType<Database['prepare']>;
|
saveTrust: ReturnType<Database['prepare']>;
|
||||||
addRetired: ReturnType<Database['prepare']>;
|
addRetired: ReturnType<Database['prepare']>;
|
||||||
@@ -134,6 +137,15 @@ export class EncryptedSQLiteStorage implements StorageProvider {
|
|||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
ciphertext BLOB NOT NULL
|
ciphertext BLOB NOT NULL
|
||||||
);
|
);
|
||||||
|
-- Session-label aliases (V4.12). Labels are already stored in the
|
||||||
|
-- clear as the sessions_enc primary key, so the mapping between two
|
||||||
|
-- of them reveals nothing new; only session state is encrypted.
|
||||||
|
CREATE TABLE IF NOT EXISTS session_aliases_enc (
|
||||||
|
alias TEXT PRIMARY KEY,
|
||||||
|
canonical TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_session_aliases_enc_canonical
|
||||||
|
ON session_aliases_enc(canonical);
|
||||||
CREATE TABLE IF NOT EXISTS trusted_identities_enc (
|
CREATE TABLE IF NOT EXISTS trusted_identities_enc (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
ciphertext BLOB NOT NULL
|
ciphertext BLOB NOT NULL
|
||||||
@@ -203,6 +215,9 @@ export class EncryptedSQLiteStorage implements StorageProvider {
|
|||||||
getSession: this.db.prepare('SELECT ciphertext FROM sessions_enc WHERE address = ?'),
|
getSession: this.db.prepare('SELECT ciphertext FROM sessions_enc WHERE address = ?'),
|
||||||
saveSession: this.db.prepare('INSERT OR REPLACE INTO sessions_enc (address, ciphertext) VALUES (?, ?)'),
|
saveSession: this.db.prepare('INSERT OR REPLACE INTO sessions_enc (address, ciphertext) VALUES (?, ?)'),
|
||||||
removeSession: this.db.prepare('DELETE FROM sessions_enc WHERE address = ?'),
|
removeSession: this.db.prepare('DELETE FROM sessions_enc WHERE address = ?'),
|
||||||
|
getSessionAlias: this.db.prepare('SELECT canonical FROM session_aliases_enc WHERE alias = ?'),
|
||||||
|
saveSessionAlias: this.db.prepare('INSERT OR REPLACE INTO session_aliases_enc (alias, canonical) VALUES (?, ?)'),
|
||||||
|
removeAliasesFor: this.db.prepare('DELETE FROM session_aliases_enc WHERE canonical = ?'),
|
||||||
getTrust: this.db.prepare('SELECT ciphertext FROM trusted_identities_enc WHERE address = ?'),
|
getTrust: this.db.prepare('SELECT ciphertext FROM trusted_identities_enc WHERE address = ?'),
|
||||||
saveTrust: this.db.prepare('INSERT OR REPLACE INTO trusted_identities_enc (address, ciphertext) VALUES (?, ?)'),
|
saveTrust: this.db.prepare('INSERT OR REPLACE INTO trusted_identities_enc (address, ciphertext) VALUES (?, ?)'),
|
||||||
addRetired: this.db.prepare('INSERT OR REPLACE INTO retired_identities_enc (retired_at, ciphertext) VALUES (?, ?)'),
|
addRetired: this.db.prepare('INSERT OR REPLACE INTO retired_identities_enc (retired_at, ciphertext) VALUES (?, ?)'),
|
||||||
@@ -377,6 +392,21 @@ export class EncryptedSQLiteStorage implements StorageProvider {
|
|||||||
this.stmts.removeSession.run(address);
|
this.stmts.removeSession.run(address);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ─────────────────────────────────
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
const row = this.stmts.getSessionAlias.get(alias) as { canonical: string } | undefined;
|
||||||
|
return row?.canonical ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
this.stmts.saveSessionAlias.run(alias, canonical);
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
this.stmts.removeAliasesFor.run(canonical);
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ─────────────────────────────────────────────────
|
// ─── Trust ─────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
|
|||||||
141
packages/shade-storage-encrypted/tests/nonce-reuse.test.ts
Normal file
141
packages/shade-storage-encrypted/tests/nonce-reuse.test.ts
Normal file
@@ -0,0 +1,141 @@
|
|||||||
|
/**
|
||||||
|
* Regression tests for the AES-GCM nonce-reuse fix (G0).
|
||||||
|
*
|
||||||
|
* The codec used to derive its nonce from (fieldKey, table, pk) alone. That
|
||||||
|
* is a pure function of row identity, so every re-seal of a mutable row —
|
||||||
|
* `saveSession` runs on each ratchet step — reused (key, nonce) across
|
||||||
|
* different plaintexts. AES-GCM forbids exactly that: it leaks the XOR of
|
||||||
|
* the plaintexts and the GHASH subkey, which yields tag forgery under that
|
||||||
|
* key. These tests fail if the derivation ever comes back.
|
||||||
|
*/
|
||||||
|
import { describe, test, expect } from 'bun:test';
|
||||||
|
import { KeyManager } from '../src/crypto/key-manager.js';
|
||||||
|
import { AEAD_NONCE_LEN, aeadSeal } from '../src/crypto/aead.js';
|
||||||
|
import { buildAad, deriveNonce } from '../src/crypto/kdf.js';
|
||||||
|
import {
|
||||||
|
COL,
|
||||||
|
TBL,
|
||||||
|
openBytes,
|
||||||
|
openString,
|
||||||
|
sealBytes,
|
||||||
|
sealString,
|
||||||
|
} from '../src/crypto/row-codec.js';
|
||||||
|
|
||||||
|
const TEXT = new TextEncoder();
|
||||||
|
|
||||||
|
function km(): Promise<KeyManager> {
|
||||||
|
return KeyManager.open({ kind: 'injected', key: new Uint8Array(32).fill(0x42) });
|
||||||
|
}
|
||||||
|
|
||||||
|
const nonceOf = (blob: Uint8Array) => blob.subarray(0, AEAD_NONCE_LEN);
|
||||||
|
const hex = (b: Uint8Array) => Array.from(b, (x) => x.toString(16).padStart(2, '0')).join('');
|
||||||
|
|
||||||
|
describe('nonce uniqueness across re-saves', () => {
|
||||||
|
test('two seals of the same row do not share a nonce', async () => {
|
||||||
|
const k = await km();
|
||||||
|
// The real shape of the bug: same (table, column, pk), different
|
||||||
|
// plaintext, as a session row is re-sealed on every ratchet step.
|
||||||
|
const first = await sealString(k, TBL.sessions, COL.session, 'alice', '{"messageCount":1}');
|
||||||
|
const second = await sealString(k, TBL.sessions, COL.session, 'alice', '{"messageCount":2}');
|
||||||
|
expect(hex(nonceOf(first))).not.toBe(hex(nonceOf(second)));
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a long run of re-saves produces all-distinct nonces', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const seen = new Set<string>();
|
||||||
|
for (let i = 0; i < 200; i++) {
|
||||||
|
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', `state-${i}`);
|
||||||
|
seen.add(hex(nonceOf(blob)));
|
||||||
|
}
|
||||||
|
expect(seen.size).toBe(200);
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('sealBytes is covered by the same rule', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const a = await sealBytes(k, TBL.config, COL.config, 'cfg', TEXT.encode('one'));
|
||||||
|
const b = await sealBytes(k, TBL.config, COL.config, 'cfg', TEXT.encode('two'));
|
||||||
|
expect(hex(nonceOf(a))).not.toBe(hex(nonceOf(b)));
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the nonce is not the derived one', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', 'payload');
|
||||||
|
const derived = deriveNonce(k.fieldKey(TBL.sessions, COL.session), TBL.sessions, 'alice');
|
||||||
|
expect(hex(nonceOf(blob))).not.toBe(hex(derived));
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('backward compatibility with deterministically-sealed blobs', () => {
|
||||||
|
// `aeadOpen` has always read the nonce from the blob prefix, so data
|
||||||
|
// written before the fix opens unchanged and needs no migration. This
|
||||||
|
// test reproduces an old blob by sealing with the deprecated derivation.
|
||||||
|
test('a blob sealed with the old derived nonce still opens', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const key = k.fieldKey(TBL.sessions, COL.session);
|
||||||
|
const legacy = await aeadSeal(
|
||||||
|
key,
|
||||||
|
deriveNonce(key, TBL.sessions, 'alice'),
|
||||||
|
TEXT.encode('written before the fix'),
|
||||||
|
buildAad(TBL.sessions, COL.session, 'alice'),
|
||||||
|
);
|
||||||
|
|
||||||
|
const opened = await openString(k, TBL.sessions, COL.session, 'alice', legacy);
|
||||||
|
expect(opened).toBe('written before the fix');
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('openBytes reads an old blob too', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const key = k.fieldKey(TBL.config, COL.config);
|
||||||
|
const payload = TEXT.encode('legacy bytes');
|
||||||
|
const legacy = await aeadSeal(
|
||||||
|
key,
|
||||||
|
deriveNonce(key, TBL.config, 'cfg'),
|
||||||
|
payload,
|
||||||
|
buildAad(TBL.config, COL.config, 'cfg'),
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(await openBytes(k, TBL.config, COL.config, 'cfg', legacy)).toEqual(payload);
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('new and old blobs are both readable under one key', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const key = k.fieldKey(TBL.sessions, COL.session);
|
||||||
|
const legacy = await aeadSeal(
|
||||||
|
key,
|
||||||
|
deriveNonce(key, TBL.sessions, 'bob'),
|
||||||
|
TEXT.encode('old'),
|
||||||
|
buildAad(TBL.sessions, COL.session, 'bob'),
|
||||||
|
);
|
||||||
|
const fresh = await sealString(k, TBL.sessions, COL.session, 'bob', 'new');
|
||||||
|
|
||||||
|
expect(await openString(k, TBL.sessions, COL.session, 'bob', legacy)).toBe('old');
|
||||||
|
expect(await openString(k, TBL.sessions, COL.session, 'bob', fresh)).toBe('new');
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('tamper detection survives the switch', () => {
|
||||||
|
// The dropped `expectedNonce` check was the stated reason for the
|
||||||
|
// deterministic nonce. It detected nothing the AEAD tag misses.
|
||||||
|
test('a flipped nonce byte is still rejected', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', 'payload');
|
||||||
|
const tampered = new Uint8Array(blob);
|
||||||
|
tampered[0]! ^= 0x01;
|
||||||
|
await expect(openString(k, TBL.sessions, COL.session, 'alice', tampered)).rejects.toThrow();
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a blob moved to another row is still rejected (AAD binding)', async () => {
|
||||||
|
const k = await km();
|
||||||
|
const blob = await sealString(k, TBL.sessions, COL.session, 'alice', 'alice-secret');
|
||||||
|
await expect(openString(k, TBL.sessions, COL.session, 'bob', blob)).rejects.toThrow();
|
||||||
|
k.destroy();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-indexeddb",
|
"name": "@shade/storage-indexeddb",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -74,6 +74,10 @@ export class IndexedDBStorage implements StorageProvider {
|
|||||||
const members = db.createObjectStore('broadcastMembers', { keyPath: ['channelId', 'peerAddress'] });
|
const members = db.createObjectStore('broadcastMembers', { keyPath: ['channelId', 'peerAddress'] });
|
||||||
members.createIndex('byChannelId', 'channelId');
|
members.createIndex('byChannelId', 'channelId');
|
||||||
}
|
}
|
||||||
|
if (oldVersion < 3) {
|
||||||
|
const aliases = db.createObjectStore('sessionAliases', { keyPath: 'alias' });
|
||||||
|
aliases.createIndex('byCanonical', 'canonical');
|
||||||
|
}
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
return new IndexedDBStorage(db);
|
return new IndexedDBStorage(db);
|
||||||
@@ -174,6 +178,24 @@ export class IndexedDBStorage implements StorageProvider {
|
|||||||
await this.db.delete('sessions', address);
|
await this.db.delete('sessions', address);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ────────────────────────────────
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
const row = await this.db.get('sessionAliases', alias);
|
||||||
|
return row?.canonical ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
await this.db.put('sessionAliases', { alias, canonical });
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
const tx = this.db.transaction('sessionAliases', 'readwrite');
|
||||||
|
const matches = await tx.store.index('byCanonical').getAllKeys(canonical);
|
||||||
|
await Promise.all(matches.map((key) => tx.store.delete(key)));
|
||||||
|
await tx.done;
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ────────────────────────────────────────────────
|
// ─── Trust ────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
@@ -360,7 +382,7 @@ export class IndexedDBStorage implements StorageProvider {
|
|||||||
|
|
||||||
// ─── Schema ────────────────────────────────────────────────
|
// ─── Schema ────────────────────────────────────────────────
|
||||||
|
|
||||||
const SCHEMA_VERSION = 2;
|
const SCHEMA_VERSION = 3;
|
||||||
|
|
||||||
interface IdentityRow {
|
interface IdentityRow {
|
||||||
id: 1;
|
id: 1;
|
||||||
@@ -390,6 +412,11 @@ interface SessionRow {
|
|||||||
stateJson: string;
|
stateJson: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface SessionAliasRow {
|
||||||
|
alias: string;
|
||||||
|
canonical: string;
|
||||||
|
}
|
||||||
|
|
||||||
interface TrustedIdentityRow {
|
interface TrustedIdentityRow {
|
||||||
address: string;
|
address: string;
|
||||||
identityKey: string;
|
identityKey: string;
|
||||||
@@ -457,6 +484,11 @@ interface ShadeSchema extends DBSchema {
|
|||||||
signedPreKeys: { key: number; value: SignedPreKeyRow };
|
signedPreKeys: { key: number; value: SignedPreKeyRow };
|
||||||
oneTimePreKeys: { key: number; value: OneTimePreKeyRow };
|
oneTimePreKeys: { key: number; value: OneTimePreKeyRow };
|
||||||
sessions: { key: string; value: SessionRow };
|
sessions: { key: string; value: SessionRow };
|
||||||
|
sessionAliases: {
|
||||||
|
key: string;
|
||||||
|
value: SessionAliasRow;
|
||||||
|
indexes: { byCanonical: string };
|
||||||
|
};
|
||||||
trustedIdentities: { key: string; value: TrustedIdentityRow };
|
trustedIdentities: { key: string; value: TrustedIdentityRow };
|
||||||
retiredIdentities: {
|
retiredIdentities: {
|
||||||
key: number;
|
key: number;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-postgres",
|
"name": "@shade/storage-postgres",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -43,6 +43,16 @@ export async function ensureClientTables(sql: Sql): Promise<void> {
|
|||||||
state_json TEXT NOT NULL
|
state_json TEXT NOT NULL
|
||||||
)
|
)
|
||||||
`;
|
`;
|
||||||
|
await sql`
|
||||||
|
CREATE TABLE IF NOT EXISTS shade_session_aliases (
|
||||||
|
alias TEXT PRIMARY KEY,
|
||||||
|
canonical TEXT NOT NULL
|
||||||
|
)
|
||||||
|
`;
|
||||||
|
await sql`
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_shade_session_aliases_canonical
|
||||||
|
ON shade_session_aliases(canonical)
|
||||||
|
`;
|
||||||
await sql`
|
await sql`
|
||||||
CREATE TABLE IF NOT EXISTS shade_trusted_identities (
|
CREATE TABLE IF NOT EXISTS shade_trusted_identities (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
|
|||||||
@@ -160,6 +160,27 @@ export class PostgresStorage implements StorageProvider {
|
|||||||
await this.sql`DELETE FROM shade_sessions WHERE address = ${address}`;
|
await this.sql`DELETE FROM shade_sessions WHERE address = ${address}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ────────────────────────────────
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
const rows = await this.sql<Array<{ canonical: string }>>`
|
||||||
|
SELECT canonical FROM shade_session_aliases WHERE alias = ${alias}
|
||||||
|
`;
|
||||||
|
return rows.length ? rows[0]!.canonical : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
await this.sql`
|
||||||
|
INSERT INTO shade_session_aliases (alias, canonical)
|
||||||
|
VALUES (${alias}, ${canonical})
|
||||||
|
ON CONFLICT (alias) DO UPDATE SET canonical = EXCLUDED.canonical
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
await this.sql`DELETE FROM shade_session_aliases WHERE canonical = ${canonical}`;
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ────────────────────────────────────────────────
|
// ─── Trust ────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/storage-sqlite",
|
"name": "@shade/storage-sqlite",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
@@ -9,5 +9,8 @@
|
|||||||
"@shade/crypto-web": "workspace:*",
|
"@shade/crypto-web": "workspace:*",
|
||||||
"@shade/inbox-server": "workspace:*",
|
"@shade/inbox-server": "workspace:*",
|
||||||
"@shade/server": "workspace:*"
|
"@shade/server": "workspace:*"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@shade/vault": "workspace:*"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,3 +2,4 @@ export { SQLiteStorage } from './sqlite-storage.js';
|
|||||||
export { SqlitePrekeyStore } from './sqlite-prekey-store.js';
|
export { SqlitePrekeyStore } from './sqlite-prekey-store.js';
|
||||||
export { SqliteInboxStore } from './sqlite-inbox-store.js';
|
export { SqliteInboxStore } from './sqlite-inbox-store.js';
|
||||||
export { SqliteBlobStore } from './sqlite-blob-store.js';
|
export { SqliteBlobStore } from './sqlite-blob-store.js';
|
||||||
|
export { SqliteVaultStore } from './sqlite-vault-store.js';
|
||||||
|
|||||||
@@ -41,6 +41,9 @@ export class SQLiteStorage implements StorageProvider {
|
|||||||
getSession: ReturnType<Database['prepare']>;
|
getSession: ReturnType<Database['prepare']>;
|
||||||
saveSession: ReturnType<Database['prepare']>;
|
saveSession: ReturnType<Database['prepare']>;
|
||||||
removeSession: ReturnType<Database['prepare']>;
|
removeSession: ReturnType<Database['prepare']>;
|
||||||
|
getSessionAlias: ReturnType<Database['prepare']>;
|
||||||
|
saveSessionAlias: ReturnType<Database['prepare']>;
|
||||||
|
removeAliasesFor: ReturnType<Database['prepare']>;
|
||||||
getTrust: ReturnType<Database['prepare']>;
|
getTrust: ReturnType<Database['prepare']>;
|
||||||
saveTrust: ReturnType<Database['prepare']>;
|
saveTrust: ReturnType<Database['prepare']>;
|
||||||
addRetired: ReturnType<Database['prepare']>;
|
addRetired: ReturnType<Database['prepare']>;
|
||||||
@@ -100,6 +103,12 @@ export class SQLiteStorage implements StorageProvider {
|
|||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
state_json TEXT NOT NULL
|
state_json TEXT NOT NULL
|
||||||
);
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS session_aliases (
|
||||||
|
alias TEXT PRIMARY KEY,
|
||||||
|
canonical TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_session_aliases_canonical
|
||||||
|
ON session_aliases(canonical);
|
||||||
CREATE TABLE IF NOT EXISTS trusted_identities (
|
CREATE TABLE IF NOT EXISTS trusted_identities (
|
||||||
address TEXT PRIMARY KEY,
|
address TEXT PRIMARY KEY,
|
||||||
identity_key TEXT NOT NULL
|
identity_key TEXT NOT NULL
|
||||||
@@ -179,6 +188,9 @@ export class SQLiteStorage implements StorageProvider {
|
|||||||
getSession: this.db.prepare('SELECT state_json FROM sessions WHERE address = ?'),
|
getSession: this.db.prepare('SELECT state_json FROM sessions WHERE address = ?'),
|
||||||
saveSession: this.db.prepare('INSERT OR REPLACE INTO sessions (address, state_json) VALUES (?, ?)'),
|
saveSession: this.db.prepare('INSERT OR REPLACE INTO sessions (address, state_json) VALUES (?, ?)'),
|
||||||
removeSession: this.db.prepare('DELETE FROM sessions WHERE address = ?'),
|
removeSession: this.db.prepare('DELETE FROM sessions WHERE address = ?'),
|
||||||
|
getSessionAlias: this.db.prepare('SELECT canonical FROM session_aliases WHERE alias = ?'),
|
||||||
|
saveSessionAlias: this.db.prepare('INSERT OR REPLACE INTO session_aliases (alias, canonical) VALUES (?, ?)'),
|
||||||
|
removeAliasesFor: this.db.prepare('DELETE FROM session_aliases WHERE canonical = ?'),
|
||||||
getTrust: this.db.prepare('SELECT identity_key FROM trusted_identities WHERE address = ?'),
|
getTrust: this.db.prepare('SELECT identity_key FROM trusted_identities WHERE address = ?'),
|
||||||
saveTrust: this.db.prepare('INSERT OR REPLACE INTO trusted_identities (address, identity_key) VALUES (?, ?)'),
|
saveTrust: this.db.prepare('INSERT OR REPLACE INTO trusted_identities (address, identity_key) VALUES (?, ?)'),
|
||||||
addRetired: this.db.prepare('INSERT INTO retired_identities (data_json, retired_at) VALUES (?, ?)'),
|
addRetired: this.db.prepare('INSERT INTO retired_identities (data_json, retired_at) VALUES (?, ?)'),
|
||||||
@@ -337,6 +349,21 @@ export class SQLiteStorage implements StorageProvider {
|
|||||||
this.stmts.removeSession.run(address);
|
this.stmts.removeSession.run(address);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── Session label aliases ────────────────────────────────
|
||||||
|
|
||||||
|
async getSessionAlias(alias: string): Promise<string | null> {
|
||||||
|
const row = this.stmts.getSessionAlias.get(alias) as { canonical: string } | undefined;
|
||||||
|
return row?.canonical ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async saveSessionAlias(alias: string, canonical: string): Promise<void> {
|
||||||
|
this.stmts.saveSessionAlias.run(alias, canonical);
|
||||||
|
}
|
||||||
|
|
||||||
|
async removeSessionAliasesFor(canonical: string): Promise<void> {
|
||||||
|
this.stmts.removeAliasesFor.run(canonical);
|
||||||
|
}
|
||||||
|
|
||||||
// ─── Trust ────────────────────────────────────────────────
|
// ─── Trust ────────────────────────────────────────────────
|
||||||
|
|
||||||
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
async isTrustedIdentity(address: string, identityKey: Uint8Array): Promise<boolean> {
|
||||||
|
|||||||
154
packages/shade-storage-sqlite/src/sqlite-vault-store.ts
Normal file
154
packages/shade-storage-sqlite/src/sqlite-vault-store.ts
Normal file
@@ -0,0 +1,154 @@
|
|||||||
|
import { Database } from 'bun:sqlite';
|
||||||
|
import type { VaultLogEntry, VaultStore } from '@shade/vault';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* SQLite-backed VaultStore for the V4.13 encrypted file store.
|
||||||
|
*
|
||||||
|
* Three tables, mirroring the model: an owner key per vault, the
|
||||||
|
* content-addressed objects, and the append-only manifest log. The relay
|
||||||
|
* never decrypts anything — it enforces auth, verifies that an object's bytes
|
||||||
|
* hash to its name, and refuses a commit whose objects are not all present.
|
||||||
|
*
|
||||||
|
* Objects are stored as BLOBs rather than base64 text. A workspace backup is
|
||||||
|
* a few hundred files where the blob primitive holds one small profile, so the
|
||||||
|
* 33% base64 overhead stops being a rounding error.
|
||||||
|
*
|
||||||
|
* Docker usage: set `SHADE_VAULT_DB_PATH` (falls back to `/data/shade-vault.db`).
|
||||||
|
* **Set it.** The blob store's equivalent was missing from `docker-compose.yml`
|
||||||
|
* for months and nobody noticed, because the in-memory fallback works
|
||||||
|
* perfectly until the container is recreated — at which point everything is
|
||||||
|
* gone, with no error anywhere. That happened on 2026-08-12.
|
||||||
|
*/
|
||||||
|
export class SqliteVaultStore implements VaultStore {
|
||||||
|
private db: Database;
|
||||||
|
private stmts!: {
|
||||||
|
getOwner: ReturnType<Database['prepare']>;
|
||||||
|
setOwner: ReturnType<Database['prepare']>;
|
||||||
|
hasObject: ReturnType<Database['prepare']>;
|
||||||
|
putObject: ReturnType<Database['prepare']>;
|
||||||
|
getObject: ReturnType<Database['prepare']>;
|
||||||
|
head: ReturnType<Database['prepare']>;
|
||||||
|
log: ReturnType<Database['prepare']>;
|
||||||
|
logLimit: ReturnType<Database['prepare']>;
|
||||||
|
appendLog: ReturnType<Database['prepare']>;
|
||||||
|
usage: ReturnType<Database['prepare']>;
|
||||||
|
};
|
||||||
|
|
||||||
|
constructor(dbPath?: string) {
|
||||||
|
const path = dbPath ?? process.env.SHADE_VAULT_DB_PATH ?? '/data/shade-vault.db';
|
||||||
|
this.db = new Database(path, { create: true });
|
||||||
|
this.db.exec('PRAGMA journal_mode=WAL');
|
||||||
|
this.ensureTables();
|
||||||
|
this.prepareStatements();
|
||||||
|
}
|
||||||
|
|
||||||
|
private ensureTables() {
|
||||||
|
this.db.exec(`
|
||||||
|
CREATE TABLE IF NOT EXISTS shade_vault_owners (
|
||||||
|
vault_id TEXT PRIMARY KEY,
|
||||||
|
owner_pubkey BLOB NOT NULL,
|
||||||
|
created_at INTEGER NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS shade_vault_objects (
|
||||||
|
vault_id TEXT NOT NULL,
|
||||||
|
hash TEXT NOT NULL,
|
||||||
|
bytes BLOB NOT NULL,
|
||||||
|
size INTEGER NOT NULL,
|
||||||
|
created_at INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY (vault_id, hash)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS shade_vault_log (
|
||||||
|
vault_id TEXT NOT NULL,
|
||||||
|
seq INTEGER NOT NULL,
|
||||||
|
manifest TEXT NOT NULL,
|
||||||
|
at INTEGER NOT NULL,
|
||||||
|
bytes INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY (vault_id, seq)
|
||||||
|
);
|
||||||
|
`);
|
||||||
|
}
|
||||||
|
|
||||||
|
private prepareStatements() {
|
||||||
|
this.stmts = {
|
||||||
|
getOwner: this.db.prepare('SELECT owner_pubkey FROM shade_vault_owners WHERE vault_id = ?'),
|
||||||
|
setOwner: this.db.prepare(
|
||||||
|
'INSERT OR REPLACE INTO shade_vault_owners (vault_id, owner_pubkey, created_at) VALUES (?, ?, ?)',
|
||||||
|
),
|
||||||
|
hasObject: this.db.prepare(
|
||||||
|
'SELECT 1 FROM shade_vault_objects WHERE vault_id = ? AND hash = ? LIMIT 1',
|
||||||
|
),
|
||||||
|
// An object's name IS its content hash, so a repeat upload is the same
|
||||||
|
// bytes by definition — ignoring it is correct, not lossy.
|
||||||
|
putObject: this.db.prepare(
|
||||||
|
'INSERT OR IGNORE INTO shade_vault_objects (vault_id, hash, bytes, size, created_at) VALUES (?, ?, ?, ?, ?)',
|
||||||
|
),
|
||||||
|
getObject: this.db.prepare(
|
||||||
|
'SELECT bytes FROM shade_vault_objects WHERE vault_id = ? AND hash = ?',
|
||||||
|
),
|
||||||
|
head: this.db.prepare('SELECT MAX(seq) AS head FROM shade_vault_log WHERE vault_id = ?'),
|
||||||
|
log: this.db.prepare(
|
||||||
|
'SELECT seq, manifest, at, bytes FROM shade_vault_log WHERE vault_id = ? ORDER BY seq ASC',
|
||||||
|
),
|
||||||
|
logLimit: this.db.prepare(
|
||||||
|
'SELECT seq, manifest, at, bytes FROM (SELECT seq, manifest, at, bytes FROM shade_vault_log WHERE vault_id = ? ORDER BY seq DESC LIMIT ?) ORDER BY seq ASC',
|
||||||
|
),
|
||||||
|
appendLog: this.db.prepare(
|
||||||
|
'INSERT INTO shade_vault_log (vault_id, seq, manifest, at, bytes) VALUES (?, ?, ?, ?, ?)',
|
||||||
|
),
|
||||||
|
usage: this.db.prepare(
|
||||||
|
'SELECT COALESCE(SUM(size), 0) AS total FROM shade_vault_objects WHERE vault_id = ?',
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async getOwner(vaultId: string): Promise<Uint8Array | null> {
|
||||||
|
const row = this.stmts.getOwner.get(vaultId) as { owner_pubkey: Uint8Array } | null;
|
||||||
|
return row ? new Uint8Array(row.owner_pubkey) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async setOwner(vaultId: string, publicKey: Uint8Array): Promise<void> {
|
||||||
|
this.stmts.setOwner.run(vaultId, publicKey, Date.now());
|
||||||
|
}
|
||||||
|
|
||||||
|
async hasObject(vaultId: string, hash: string): Promise<boolean> {
|
||||||
|
return this.stmts.hasObject.get(vaultId, hash) !== null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async putObject(vaultId: string, hash: string, bytes: Uint8Array): Promise<void> {
|
||||||
|
this.stmts.putObject.run(vaultId, hash, bytes, bytes.length, Date.now());
|
||||||
|
}
|
||||||
|
|
||||||
|
async getObject(vaultId: string, hash: string): Promise<Uint8Array | null> {
|
||||||
|
const row = this.stmts.getObject.get(vaultId, hash) as { bytes: Uint8Array } | null;
|
||||||
|
return row ? new Uint8Array(row.bytes) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async head(vaultId: string): Promise<number> {
|
||||||
|
const row = this.stmts.head.get(vaultId) as { head: number | null } | null;
|
||||||
|
return row?.head ?? 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
async log(vaultId: string, limit?: number): Promise<VaultLogEntry[]> {
|
||||||
|
const rows = (
|
||||||
|
limit === undefined
|
||||||
|
? this.stmts.log.all(vaultId)
|
||||||
|
: this.stmts.logLimit.all(vaultId, limit)
|
||||||
|
) as VaultLogEntry[];
|
||||||
|
return rows;
|
||||||
|
}
|
||||||
|
|
||||||
|
async appendLog(vaultId: string, entry: VaultLogEntry): Promise<void> {
|
||||||
|
this.stmts.appendLog.run(vaultId, entry.seq, entry.manifest, entry.at, entry.bytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
async usage(vaultId: string): Promise<number> {
|
||||||
|
const row = this.stmts.usage.get(vaultId) as { total: number } | null;
|
||||||
|
return row?.total ?? 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
close(): void {
|
||||||
|
this.db.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
120
packages/shade-storage-sqlite/tests/sqlite-vault-store.test.ts
Normal file
120
packages/shade-storage-sqlite/tests/sqlite-vault-store.test.ts
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
/**
|
||||||
|
* The SQLite vault store, with persistence as the headline property.
|
||||||
|
*
|
||||||
|
* This exists because of 2026-08-12: the blob store fell back to memory when
|
||||||
|
* its path was unset, worked perfectly, and lost every profile on the next
|
||||||
|
* container recreate — with no error anywhere. A backup store that forgets is
|
||||||
|
* worse than none, so "survives a close and reopen" is tested directly rather
|
||||||
|
* than assumed from the fact that SQLite is involved.
|
||||||
|
*/
|
||||||
|
import { describe, test, expect, afterEach } from 'bun:test';
|
||||||
|
import { unlinkSync } from 'node:fs';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { SqliteVaultStore } from '../src/sqlite-vault-store.js';
|
||||||
|
|
||||||
|
const paths: string[] = [];
|
||||||
|
function scratch(name: string): string {
|
||||||
|
const p = join(tmpdir(), `shade-vault-${name}-${process.pid}-${Math.random().toString(36).slice(2)}.db`);
|
||||||
|
paths.push(p);
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
for (const p of paths.splice(0)) {
|
||||||
|
for (const suffix of ['', '-wal', '-shm']) {
|
||||||
|
try {
|
||||||
|
unlinkSync(p + suffix);
|
||||||
|
} catch {
|
||||||
|
// Not every WAL sidecar exists; absence is fine.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const VAULT = 'a'.repeat(64);
|
||||||
|
const HASH = 'b'.repeat(64);
|
||||||
|
|
||||||
|
describe('persistence', () => {
|
||||||
|
test('objects, owner and log survive a close and reopen', async () => {
|
||||||
|
const path = scratch('reopen');
|
||||||
|
const first = new SqliteVaultStore(path);
|
||||||
|
await first.setOwner(VAULT, new Uint8Array([1, 2, 3, 4]));
|
||||||
|
await first.putObject(VAULT, HASH, new Uint8Array([9, 8, 7]));
|
||||||
|
await first.appendLog(VAULT, { seq: 1, manifest: HASH, at: 1700, bytes: 3 });
|
||||||
|
first.close();
|
||||||
|
|
||||||
|
const second = new SqliteVaultStore(path);
|
||||||
|
expect(await second.getOwner(VAULT)).toEqual(new Uint8Array([1, 2, 3, 4]));
|
||||||
|
expect(await second.getObject(VAULT, HASH)).toEqual(new Uint8Array([9, 8, 7]));
|
||||||
|
expect(await second.head(VAULT)).toBe(1);
|
||||||
|
expect((await second.log(VAULT))[0]!.manifest).toBe(HASH);
|
||||||
|
second.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('binary content round-trips without base64 mangling', async () => {
|
||||||
|
// Objects are ciphertext: every byte value occurs, including 0x00.
|
||||||
|
const path = scratch('binary');
|
||||||
|
const store = new SqliteVaultStore(path);
|
||||||
|
const bytes = new Uint8Array(256);
|
||||||
|
for (let i = 0; i < 256; i++) bytes[i] = i;
|
||||||
|
await store.putObject(VAULT, HASH, bytes);
|
||||||
|
store.close();
|
||||||
|
|
||||||
|
const reopened = new SqliteVaultStore(path);
|
||||||
|
expect(await reopened.getObject(VAULT, HASH)).toEqual(bytes);
|
||||||
|
reopened.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('semantics', () => {
|
||||||
|
test('an empty vault has head 0 and no owner', async () => {
|
||||||
|
const store = new SqliteVaultStore(scratch('empty'));
|
||||||
|
expect(await store.head(VAULT)).toBe(0);
|
||||||
|
expect(await store.getOwner(VAULT)).toBeNull();
|
||||||
|
expect(await store.log(VAULT)).toEqual([]);
|
||||||
|
expect(await store.usage(VAULT)).toBe(0);
|
||||||
|
store.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('re-putting the same hash is a no-op, not a duplicate', async () => {
|
||||||
|
// The name IS the content hash, so a repeat is the same bytes by
|
||||||
|
// definition. Usage must not double.
|
||||||
|
const store = new SqliteVaultStore(scratch('dupe'));
|
||||||
|
await store.putObject(VAULT, HASH, new Uint8Array(100));
|
||||||
|
await store.putObject(VAULT, HASH, new Uint8Array(100));
|
||||||
|
expect(await store.usage(VAULT)).toBe(100);
|
||||||
|
store.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('vaults are isolated from each other', async () => {
|
||||||
|
const store = new SqliteVaultStore(scratch('isolated'));
|
||||||
|
const other = 'c'.repeat(64);
|
||||||
|
await store.putObject(VAULT, HASH, new Uint8Array([1]));
|
||||||
|
expect(await store.hasObject(other, HASH)).toBe(false);
|
||||||
|
expect(await store.getObject(other, HASH)).toBeNull();
|
||||||
|
expect(await store.usage(other)).toBe(0);
|
||||||
|
store.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the log is ordered oldest-first and limit counts back from the head', async () => {
|
||||||
|
const store = new SqliteVaultStore(scratch('log'));
|
||||||
|
for (let seq = 1; seq <= 5; seq++) {
|
||||||
|
await store.appendLog(VAULT, { seq, manifest: `${seq}`.repeat(64), at: seq, bytes: seq });
|
||||||
|
}
|
||||||
|
expect((await store.log(VAULT)).map((e) => e.seq)).toEqual([1, 2, 3, 4, 5]);
|
||||||
|
// A client asking for the last two wants 4 and 5, still in order.
|
||||||
|
expect((await store.log(VAULT, 2)).map((e) => e.seq)).toEqual([4, 5]);
|
||||||
|
expect(await store.head(VAULT)).toBe(5);
|
||||||
|
store.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a duplicate seq is rejected by the primary key', async () => {
|
||||||
|
// The route layer refuses this first, but the store is the last line:
|
||||||
|
// two rows with the same seq would make history ambiguous.
|
||||||
|
const store = new SqliteVaultStore(scratch('seq'));
|
||||||
|
await store.appendLog(VAULT, { seq: 1, manifest: HASH, at: 1, bytes: 1 });
|
||||||
|
expect(store.appendLog(VAULT, { seq: 1, manifest: HASH, at: 2, bytes: 1 })).rejects.toThrow();
|
||||||
|
store.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/streams",
|
"name": "@shade/streams",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transfer",
|
"name": "@shade/transfer",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transport-bridge",
|
"name": "@shade/transport-bridge",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transport-webrtc",
|
"name": "@shade/transport-webrtc",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/transport",
|
"name": "@shade/transport",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
74
packages/shade-vault/live-e2e.ts
Normal file
74
packages/shade-vault/live-e2e.ts
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
/**
|
||||||
|
* Full backup mot en KJØRENDE vault — ikke Hono-fetch, men ekte HTTP.
|
||||||
|
*
|
||||||
|
* Kjøres manuelt mot en container eller en deployet tjeneste:
|
||||||
|
* SCAFFOLDD_SOCKET=... RELAY=https://vault.zyon.no bun live-e2e.ts
|
||||||
|
*/
|
||||||
|
import { connect } from 'node:net';
|
||||||
|
import { readFile } from 'node:fs/promises';
|
||||||
|
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
||||||
|
import { HttpVaultTransport, VaultClient, deriveVaultKeys } from './src/index.js';
|
||||||
|
|
||||||
|
const SOCKET = process.env.SCAFFOLDD_SOCKET!;
|
||||||
|
const RELAY = process.env.RELAY!;
|
||||||
|
|
||||||
|
function ask(req: object): Promise<any> {
|
||||||
|
return new Promise((res, rej) => {
|
||||||
|
const s = connect(SOCKET);
|
||||||
|
let buf = '';
|
||||||
|
s.on('connect', () => s.write(`${JSON.stringify(req)}\n`));
|
||||||
|
s.on('data', (c) => {
|
||||||
|
buf += c.toString();
|
||||||
|
const nl = buf.indexOf('\n');
|
||||||
|
if (nl === -1) return;
|
||||||
|
s.destroy();
|
||||||
|
const r = JSON.parse(buf.slice(0, nl));
|
||||||
|
r.ok ? res(r.result) : rej(new Error(r.error));
|
||||||
|
});
|
||||||
|
s.on('error', rej);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const set = await ask({ op: 'backupSet' });
|
||||||
|
const files: { path: string; bytes: Uint8Array }[] = [];
|
||||||
|
for (const f of set.files) {
|
||||||
|
try {
|
||||||
|
files.push({ path: f.path, bytes: new Uint8Array(await readFile(f.source)) });
|
||||||
|
} catch {
|
||||||
|
/* uleselig fil hoppes over, som i broen */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const crypto = new SubtleCryptoProvider();
|
||||||
|
const keys = await deriveVaultKeys(new Uint8Array(32).fill(77), 'scaffold-live');
|
||||||
|
const client = new VaultClient(crypto, keys, new HttpVaultTransport(RELAY));
|
||||||
|
|
||||||
|
console.log(`push: ${files.length} filer …`);
|
||||||
|
let t = Date.now();
|
||||||
|
const r1 = await client.push(files, Date.now(), 'live e2e');
|
||||||
|
console.log(
|
||||||
|
` versjon ${r1.seq}: ${r1.uploaded} opplastet, ` +
|
||||||
|
`${(r1.bytesUploaded / 1024 / 1024).toFixed(1)} MB, ${((Date.now() - t) / 1000).toFixed(1)}s`,
|
||||||
|
);
|
||||||
|
|
||||||
|
const edited = files.map((f, i) =>
|
||||||
|
i === 0 ? { path: f.path, bytes: new TextEncoder().encode('endret') } : f,
|
||||||
|
);
|
||||||
|
t = Date.now();
|
||||||
|
const r2 = await client.push(edited, Date.now(), 'én endring');
|
||||||
|
console.log(
|
||||||
|
` versjon ${r2.seq}: ${r2.uploaded} opplastet, ${r2.reused} gjenbrukt, ` +
|
||||||
|
`${(r2.bytesUploaded / 1024).toFixed(0)} KB, ${((Date.now() - t) / 1000).toFixed(1)}s`,
|
||||||
|
);
|
||||||
|
|
||||||
|
const back = await client.pull(1);
|
||||||
|
let identical = 0;
|
||||||
|
let differ = 0;
|
||||||
|
for (const f of files) {
|
||||||
|
const got = back.files.get(f.path);
|
||||||
|
if (got && Buffer.from(got).equals(Buffer.from(f.bytes))) identical++;
|
||||||
|
else differ++;
|
||||||
|
}
|
||||||
|
console.log(`pull versjon 1: ${identical} bit-identiske, ${differ} avvik`);
|
||||||
|
const log = await client.history();
|
||||||
|
console.log(`historikk: ${log.entries.map((e) => e.seq).join(', ')} (head ${log.head})`);
|
||||||
25
packages/shade-vault/package.json
Normal file
25
packages/shade-vault/package.json
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"name": "@shade/vault",
|
||||||
|
"version": "4.13.0",
|
||||||
|
"description": "Server-side encrypted file store for Shade — content-addressed objects with an append-only manifest log",
|
||||||
|
"type": "module",
|
||||||
|
"main": "src/index.ts",
|
||||||
|
"types": "src/index.ts",
|
||||||
|
"exports": {
|
||||||
|
".": "./src/index.ts",
|
||||||
|
"./server": "./src/server.ts"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@noble/hashes": "^2.0.1",
|
||||||
|
"@shade/core": "workspace:*",
|
||||||
|
"@shade/crypto-web": "workspace:*",
|
||||||
|
"@shade/observability": "workspace:*",
|
||||||
|
"@shade/server": "workspace:*",
|
||||||
|
"@shade/storage-encrypted": "workspace:*",
|
||||||
|
"hono": "^4.12.18"
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"test": "bun test",
|
||||||
|
"typecheck": "tsc --noEmit"
|
||||||
|
}
|
||||||
|
}
|
||||||
225
packages/shade-vault/src/client.ts
Normal file
225
packages/shade-vault/src/client.ts
Normal file
@@ -0,0 +1,225 @@
|
|||||||
|
/**
|
||||||
|
* The client half: turn a set of files into an encrypted, versioned vault,
|
||||||
|
* and turn it back again on a machine that has nothing but the credentials.
|
||||||
|
*
|
||||||
|
* The push is deliberately have-check-first. A workspace changes a few lines
|
||||||
|
* at a time, so asking "do you already have these hashes?" and uploading only
|
||||||
|
* the misses turns a full backup into a handful of small writes. Everything
|
||||||
|
* unchanged keeps its hash from the previous commit and costs nothing.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { CryptoProvider } from '@shade/core';
|
||||||
|
import { toBase64 } from '@shade/core';
|
||||||
|
// The pubkey derivation is not on `CryptoProvider` — it is a pure function of
|
||||||
|
// the seed, and lives with the curve implementation.
|
||||||
|
import { ed25519PublicKeyFromSeed } from '@shade/crypto-web';
|
||||||
|
import { signPayload } from '@shade/server';
|
||||||
|
import {
|
||||||
|
deriveVaultContentKey,
|
||||||
|
deriveVaultId,
|
||||||
|
deriveVaultSigningSeed,
|
||||||
|
} from '@shade/storage-encrypted';
|
||||||
|
import { openManifest, openObject, sealManifest, sealObject, toHex } from './crypto.js';
|
||||||
|
import type { VaultEntry, VaultLog, VaultManifest } from './types.js';
|
||||||
|
|
||||||
|
/** A file as the app sees it, before encryption. */
|
||||||
|
export interface VaultFile {
|
||||||
|
path: string;
|
||||||
|
bytes: Uint8Array;
|
||||||
|
mode?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VaultKeys {
|
||||||
|
vaultId: string;
|
||||||
|
contentKey: Uint8Array;
|
||||||
|
signingSeed: Uint8Array;
|
||||||
|
publicKey: Uint8Array;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Derive everything a vault needs from the account master key.
|
||||||
|
*
|
||||||
|
* This is what makes recovery "username and password": a fresh device that
|
||||||
|
* can derive the profile master can derive these too, and the relay hands
|
||||||
|
* over ciphertext it has never been able to read.
|
||||||
|
*/
|
||||||
|
export async function deriveVaultKeys(masterKey: Uint8Array, app: string): Promise<VaultKeys> {
|
||||||
|
const signingSeed = deriveVaultSigningSeed(masterKey, app);
|
||||||
|
const publicKey = ed25519PublicKeyFromSeed(signingSeed);
|
||||||
|
return {
|
||||||
|
vaultId: toHex(deriveVaultId(masterKey, app)),
|
||||||
|
contentKey: deriveVaultContentKey(masterKey, app),
|
||||||
|
signingSeed,
|
||||||
|
publicKey,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VaultTransport {
|
||||||
|
/** Does the relay already hold this object? */
|
||||||
|
has(vaultId: string, hash: string): Promise<boolean>;
|
||||||
|
put(vaultId: string, hash: string, body: unknown): Promise<void>;
|
||||||
|
get(vaultId: string, hash: string): Promise<Uint8Array>;
|
||||||
|
log(vaultId: string, limit?: number): Promise<VaultLog>;
|
||||||
|
commit(vaultId: string, body: unknown): Promise<{ seq: number }>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PushResult {
|
||||||
|
seq: number;
|
||||||
|
/** Objects actually sent — the rest were already on the relay. */
|
||||||
|
uploaded: number;
|
||||||
|
/** Objects skipped because the relay had them. */
|
||||||
|
reused: number;
|
||||||
|
bytesUploaded: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class VaultClient {
|
||||||
|
constructor(
|
||||||
|
private readonly crypto: CryptoProvider,
|
||||||
|
private readonly keys: VaultKeys,
|
||||||
|
private readonly transport: VaultTransport,
|
||||||
|
) {}
|
||||||
|
|
||||||
|
get vaultId(): string {
|
||||||
|
return this.keys.vaultId;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sign a request body, with the pubkey inside the signed payload.
|
||||||
|
*
|
||||||
|
* The key has to be part of what gets signed, not appended afterwards:
|
||||||
|
* `verifyPayload` canonicalises every field, so an appended key would both
|
||||||
|
* break the signature and — if the scheme ignored it — let anyone swap the
|
||||||
|
* claimed identity in transit on a first, TOFU-pinning write.
|
||||||
|
*/
|
||||||
|
private async sign(body: Record<string, unknown>): Promise<Record<string, unknown>> {
|
||||||
|
return signPayload(this.crypto, this.keys.signingSeed, {
|
||||||
|
...body,
|
||||||
|
publicKey: toBase64(this.keys.publicKey),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Encrypt and upload `files`, then commit a manifest naming them.
|
||||||
|
*
|
||||||
|
* `at` is passed in rather than read from the clock so a caller can make a
|
||||||
|
* commit reproducible in tests, and so a queued backup records when it was
|
||||||
|
* *taken* rather than when it finally reached the relay.
|
||||||
|
*/
|
||||||
|
async push(files: VaultFile[], at: number, message?: string): Promise<PushResult> {
|
||||||
|
const { head } = await this.transport.log(this.keys.vaultId, 1);
|
||||||
|
|
||||||
|
const entries: VaultEntry[] = [];
|
||||||
|
let uploaded = 0;
|
||||||
|
let reused = 0;
|
||||||
|
let bytesUploaded = 0;
|
||||||
|
|
||||||
|
// Reuse the previous manifest's hashes for unchanged content: sealing is
|
||||||
|
// nondeterministic (fresh nonce), so re-sealing an untouched file would
|
||||||
|
// produce a new hash and a pointless upload every single time.
|
||||||
|
const previous = head > 0 ? await this.pull().catch(() => null) : null;
|
||||||
|
const byPath = new Map(previous?.manifest.entries.map((e) => [e.path, e]) ?? []);
|
||||||
|
const priorPlain = previous?.files ?? new Map<string, Uint8Array>();
|
||||||
|
|
||||||
|
for (const file of files) {
|
||||||
|
const prior = byPath.get(file.path);
|
||||||
|
const priorBytes = priorPlain.get(file.path);
|
||||||
|
if (prior && priorBytes && sameBytes(priorBytes, file.bytes)) {
|
||||||
|
entries.push({ ...prior, size: file.bytes.length });
|
||||||
|
reused++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const sealed = await sealObject(
|
||||||
|
this.crypto,
|
||||||
|
this.keys.contentKey,
|
||||||
|
this.keys.vaultId,
|
||||||
|
file.bytes,
|
||||||
|
);
|
||||||
|
if (!(await this.transport.has(this.keys.vaultId, sealed.hash))) {
|
||||||
|
await this.transport.put(
|
||||||
|
this.keys.vaultId,
|
||||||
|
sealed.hash,
|
||||||
|
await this.sign({ data: toBase64(sealed.bytes) }),
|
||||||
|
);
|
||||||
|
uploaded++;
|
||||||
|
bytesUploaded += sealed.bytes.length;
|
||||||
|
} else {
|
||||||
|
reused++;
|
||||||
|
}
|
||||||
|
const entry: VaultEntry = { path: file.path, hash: sealed.hash, size: file.bytes.length };
|
||||||
|
if (file.mode !== undefined) entry.mode = file.mode;
|
||||||
|
entries.push(entry);
|
||||||
|
}
|
||||||
|
|
||||||
|
const manifest: VaultManifest = { version: 1, seq: head + 1, at, entries };
|
||||||
|
if (message !== undefined) manifest.message = message;
|
||||||
|
|
||||||
|
const sealedManifest = await sealManifest(
|
||||||
|
this.crypto,
|
||||||
|
this.keys.contentKey,
|
||||||
|
this.keys.vaultId,
|
||||||
|
manifest,
|
||||||
|
);
|
||||||
|
await this.transport.put(
|
||||||
|
this.keys.vaultId,
|
||||||
|
sealedManifest.hash,
|
||||||
|
await this.sign({ data: toBase64(sealedManifest.bytes) }),
|
||||||
|
);
|
||||||
|
|
||||||
|
const { seq } = await this.transport.commit(
|
||||||
|
this.keys.vaultId,
|
||||||
|
await this.sign({
|
||||||
|
manifest: sealedManifest.hash,
|
||||||
|
seq: manifest.seq,
|
||||||
|
at,
|
||||||
|
hashes: entries.map((e) => e.hash),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
return { seq, uploaded, reused, bytesUploaded };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch a version and decrypt it. Defaults to the newest.
|
||||||
|
*
|
||||||
|
* Restoring is the whole point of the exercise, so this returns the plain
|
||||||
|
* files rather than a handle: a caller that has to make a second round of
|
||||||
|
* decisions to get their data back does not have a backup.
|
||||||
|
*/
|
||||||
|
async pull(seq?: number): Promise<{ manifest: VaultManifest; files: Map<string, Uint8Array> }> {
|
||||||
|
const log = await this.transport.log(this.keys.vaultId);
|
||||||
|
const wanted = seq ?? log.head;
|
||||||
|
const row = log.entries.find((e) => e.seq === wanted);
|
||||||
|
if (!row) throw new Error(`vault has no version ${wanted}`);
|
||||||
|
|
||||||
|
const manifestBytes = await this.transport.get(this.keys.vaultId, row.manifest);
|
||||||
|
const manifest = await openManifest(
|
||||||
|
this.crypto,
|
||||||
|
this.keys.contentKey,
|
||||||
|
this.keys.vaultId,
|
||||||
|
row.manifest,
|
||||||
|
manifestBytes,
|
||||||
|
);
|
||||||
|
|
||||||
|
const files = new Map<string, Uint8Array>();
|
||||||
|
for (const entry of manifest.entries) {
|
||||||
|
const bytes = await this.transport.get(this.keys.vaultId, entry.hash);
|
||||||
|
files.set(
|
||||||
|
entry.path,
|
||||||
|
await openObject(this.crypto, this.keys.contentKey, this.keys.vaultId, entry.hash, bytes),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return { manifest, files };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Version history, newest last. */
|
||||||
|
async history(limit?: number): Promise<VaultLog> {
|
||||||
|
return this.transport.log(this.keys.vaultId, limit);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function sameBytes(a: Uint8Array, b: Uint8Array): boolean {
|
||||||
|
if (a.length !== b.length) return false;
|
||||||
|
for (let i = 0; i < a.length; i++) if (a[i] !== b[i]) return false;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
129
packages/shade-vault/src/crypto.ts
Normal file
129
packages/shade-vault/src/crypto.ts
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
/**
|
||||||
|
* Sealing and naming for vault objects.
|
||||||
|
*
|
||||||
|
* Every object on the relay is `nonce(12) || ciphertext||tag`, named by the
|
||||||
|
* SHA-256 of those bytes. Hashing the *ciphertext* rather than the plaintext
|
||||||
|
* is what lets the relay verify that an upload is what it claims to be
|
||||||
|
* without holding a key — it recomputes the hash and compares.
|
||||||
|
*
|
||||||
|
* The cost is that identical plaintext yields different names on each seal,
|
||||||
|
* because the nonce is fresh. Deduplication is therefore *within* a version
|
||||||
|
* chain (an unchanged file keeps its hash across commits because the client
|
||||||
|
* reuses the object it already uploaded), not across independent seals. That
|
||||||
|
* is the right trade: convergent encryption would leak which files two users
|
||||||
|
* share, which is exactly what a blind store must not do.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { sha256 } from '@noble/hashes/sha2.js';
|
||||||
|
import type { CryptoProvider } from '@shade/core';
|
||||||
|
import type { VaultManifest } from './types.js';
|
||||||
|
|
||||||
|
const NONCE_LEN = 12;
|
||||||
|
|
||||||
|
/** Lowercase hex SHA-256 — the name of an object. */
|
||||||
|
export function objectHash(sealed: Uint8Array): string {
|
||||||
|
return Array.from(sha256(sealed))
|
||||||
|
.map((b) => b.toString(16).padStart(2, '0'))
|
||||||
|
.join('');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* AAD for an object.
|
||||||
|
*
|
||||||
|
* Binds the vault it belongs to, so a ciphertext lifted from one vault cannot
|
||||||
|
* be planted in another even by someone who can write to both. The content
|
||||||
|
* hash is deliberately NOT in the AAD: it is derived from the very bytes
|
||||||
|
* being sealed, so it cannot be known before sealing, and the relay's own
|
||||||
|
* hash check already covers substitution.
|
||||||
|
*/
|
||||||
|
function objectAad(vaultId: string): Uint8Array {
|
||||||
|
return new TextEncoder().encode(`shade-vault-object-v1|${vaultId}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SealedObject {
|
||||||
|
hash: string;
|
||||||
|
bytes: Uint8Array;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Encrypt one object and give it its name. */
|
||||||
|
export async function sealObject(
|
||||||
|
crypto: CryptoProvider,
|
||||||
|
contentKey: Uint8Array,
|
||||||
|
vaultId: string,
|
||||||
|
plaintext: Uint8Array,
|
||||||
|
): Promise<SealedObject> {
|
||||||
|
const { ciphertext, nonce } = await crypto.aesGcmEncrypt(
|
||||||
|
contentKey,
|
||||||
|
plaintext,
|
||||||
|
objectAad(vaultId),
|
||||||
|
);
|
||||||
|
const bytes = new Uint8Array(NONCE_LEN + ciphertext.length);
|
||||||
|
bytes.set(nonce, 0);
|
||||||
|
bytes.set(ciphertext, NONCE_LEN);
|
||||||
|
return { hash: objectHash(bytes), bytes };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decrypt one object.
|
||||||
|
*
|
||||||
|
* Throws when the hash does not match the bytes: a relay that returned the
|
||||||
|
* wrong object would otherwise surface as an AEAD failure, which reads like
|
||||||
|
* key corruption and sends the user looking in the wrong place.
|
||||||
|
*/
|
||||||
|
export async function openObject(
|
||||||
|
crypto: CryptoProvider,
|
||||||
|
contentKey: Uint8Array,
|
||||||
|
vaultId: string,
|
||||||
|
expectedHash: string,
|
||||||
|
bytes: Uint8Array,
|
||||||
|
): Promise<Uint8Array> {
|
||||||
|
if (bytes.length < NONCE_LEN + 16) {
|
||||||
|
throw new Error('vault object too short to be a sealed object');
|
||||||
|
}
|
||||||
|
const actual = objectHash(bytes);
|
||||||
|
if (actual !== expectedHash) {
|
||||||
|
throw new Error(`vault object hash mismatch: expected ${expectedHash}, got ${actual}`);
|
||||||
|
}
|
||||||
|
return crypto.aesGcmDecrypt(
|
||||||
|
contentKey,
|
||||||
|
bytes.subarray(NONCE_LEN),
|
||||||
|
bytes.subarray(0, NONCE_LEN),
|
||||||
|
objectAad(vaultId),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A manifest is stored as an ordinary object; this is the JSON step around it. */
|
||||||
|
export async function sealManifest(
|
||||||
|
crypto: CryptoProvider,
|
||||||
|
contentKey: Uint8Array,
|
||||||
|
vaultId: string,
|
||||||
|
manifest: VaultManifest,
|
||||||
|
): Promise<SealedObject> {
|
||||||
|
const json = new TextEncoder().encode(JSON.stringify(manifest));
|
||||||
|
return sealObject(crypto, contentKey, vaultId, json);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function openManifest(
|
||||||
|
crypto: CryptoProvider,
|
||||||
|
contentKey: Uint8Array,
|
||||||
|
vaultId: string,
|
||||||
|
expectedHash: string,
|
||||||
|
bytes: Uint8Array,
|
||||||
|
): Promise<VaultManifest> {
|
||||||
|
const plain = await openObject(crypto, contentKey, vaultId, expectedHash, bytes);
|
||||||
|
const parsed = JSON.parse(new TextDecoder().decode(plain)) as VaultManifest;
|
||||||
|
if (parsed.version !== 1) {
|
||||||
|
// Refusing beats guessing: a future manifest may mean a file the reader
|
||||||
|
// cannot represent, and silently dropping it would lose data on the next
|
||||||
|
// commit made from this client.
|
||||||
|
throw new Error(`unsupported vault manifest version: ${parsed.version}`);
|
||||||
|
}
|
||||||
|
return parsed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Lowercase hex of arbitrary bytes — used for vaultId and pubkeys on the wire. */
|
||||||
|
export function toHex(bytes: Uint8Array): string {
|
||||||
|
return Array.from(bytes)
|
||||||
|
.map((b) => b.toString(16).padStart(2, '0'))
|
||||||
|
.join('');
|
||||||
|
}
|
||||||
80
packages/shade-vault/src/http-transport.ts
Normal file
80
packages/shade-vault/src/http-transport.ts
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
/**
|
||||||
|
* The default transport: plain HTTP against a relay running the vault routes.
|
||||||
|
*
|
||||||
|
* Takes a `fetch` rather than calling the global one, so the same client works
|
||||||
|
* against a live server, a Hono app under test, and anything else that can
|
||||||
|
* answer a Request — which is how the tests exercise the real route handlers
|
||||||
|
* instead of a mock that agrees with them by construction.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { fromBase64 } from '@shade/core';
|
||||||
|
import type { VaultTransport } from './client.js';
|
||||||
|
import type { VaultLog } from './types.js';
|
||||||
|
|
||||||
|
type FetchLike = (input: string, init?: RequestInit) => Promise<Response>;
|
||||||
|
|
||||||
|
export class HttpVaultTransport implements VaultTransport {
|
||||||
|
constructor(
|
||||||
|
private readonly baseUrl: string,
|
||||||
|
private readonly fetchImpl: FetchLike = globalThis.fetch.bind(globalThis),
|
||||||
|
) {}
|
||||||
|
|
||||||
|
private url(path: string): string {
|
||||||
|
return `${this.baseUrl.replace(/\/$/, '')}${path}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Turn a non-2xx into an Error that names the relay's own code. */
|
||||||
|
private async fail(res: Response, what: string): Promise<never> {
|
||||||
|
let detail = res.statusText;
|
||||||
|
try {
|
||||||
|
const body = (await res.json()) as { error?: { code?: string; message?: string } };
|
||||||
|
if (body?.error) detail = `${body.error.code}: ${body.error.message}`;
|
||||||
|
} catch {
|
||||||
|
// Non-JSON error body; the status line is all we have.
|
||||||
|
}
|
||||||
|
throw new Error(`vault ${what} failed (${res.status}) — ${detail}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async has(vaultId: string, hash: string): Promise<boolean> {
|
||||||
|
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/object/${hash}`), {
|
||||||
|
method: 'HEAD',
|
||||||
|
});
|
||||||
|
if (res.status === 200) return true;
|
||||||
|
if (res.status === 404) return false;
|
||||||
|
return this.fail(res, 'have-check');
|
||||||
|
}
|
||||||
|
|
||||||
|
async put(vaultId: string, hash: string, body: unknown): Promise<void> {
|
||||||
|
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/object/${hash}`), {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
});
|
||||||
|
if (!res.ok) return this.fail(res, 'upload');
|
||||||
|
}
|
||||||
|
|
||||||
|
async get(vaultId: string, hash: string): Promise<Uint8Array> {
|
||||||
|
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/object/${hash}`));
|
||||||
|
if (!res.ok) return this.fail(res, 'download');
|
||||||
|
return new Uint8Array(await res.arrayBuffer());
|
||||||
|
}
|
||||||
|
|
||||||
|
async log(vaultId: string, limit?: number): Promise<VaultLog> {
|
||||||
|
const q = limit === undefined ? '' : `?limit=${limit}`;
|
||||||
|
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/log${q}`));
|
||||||
|
if (!res.ok) return this.fail(res, 'log');
|
||||||
|
return (await res.json()) as VaultLog;
|
||||||
|
}
|
||||||
|
|
||||||
|
async commit(vaultId: string, body: unknown): Promise<{ seq: number }> {
|
||||||
|
const res = await this.fetchImpl(this.url(`/v1/vault/${vaultId}/commit`), {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
});
|
||||||
|
if (!res.ok) return this.fail(res, 'commit');
|
||||||
|
return (await res.json()) as { seq: number };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export { fromBase64 };
|
||||||
6
packages/shade-vault/src/index.ts
Normal file
6
packages/shade-vault/src/index.ts
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
export * from './types.js';
|
||||||
|
export * from './crypto.js';
|
||||||
|
export * from './client.js';
|
||||||
|
export { MemoryVaultStore } from './store.js';
|
||||||
|
export type { VaultStore } from './store.js';
|
||||||
|
export { HttpVaultTransport } from './http-transport.js';
|
||||||
226
packages/shade-vault/src/server.ts
Normal file
226
packages/shade-vault/src/server.ts
Normal file
@@ -0,0 +1,226 @@
|
|||||||
|
/**
|
||||||
|
* Relay-side vault routes.
|
||||||
|
*
|
||||||
|
* GET /v1/vault/:vaultId/log → { entries, head }
|
||||||
|
* GET /v1/vault/:vaultId/object/:hash → raw ciphertext bytes
|
||||||
|
* HEAD /v1/vault/:vaultId/object/:hash → 200 | 404 (have-check)
|
||||||
|
* PUT /v1/vault/:vaultId/object/:hash → { stored } (signed)
|
||||||
|
* POST /v1/vault/:vaultId/commit → { seq } (signed)
|
||||||
|
*
|
||||||
|
* Auth is the same TOFU-Ed25519 scheme the blob primitive uses: the first
|
||||||
|
* signed write pins a pubkey for the vault, and every later write must be
|
||||||
|
* signed by it. There is no account, no password, and nothing for the relay
|
||||||
|
* to leak — it cannot even tell which user a vaultId belongs to.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Hono } from 'hono';
|
||||||
|
import type { ContentfulStatusCode } from 'hono/utils/http-status';
|
||||||
|
import type { CryptoProvider } from '@shade/core';
|
||||||
|
import { fromBase64 } from '@shade/core';
|
||||||
|
import { verifyPayload } from '@shade/server';
|
||||||
|
import { objectHash } from './crypto.js';
|
||||||
|
import type { VaultStore } from './store.js';
|
||||||
|
import type { VaultManifest } from './types.js';
|
||||||
|
|
||||||
|
const ID_REGEX = /^[0-9a-f]{64}$/;
|
||||||
|
const HASH_REGEX = /^[0-9a-f]{64}$/;
|
||||||
|
|
||||||
|
export interface VaultRoutesOptions {
|
||||||
|
/** Per-object ceiling. Defaults to 8 MiB. */
|
||||||
|
maxObjectBytes?: number;
|
||||||
|
/** Whole-vault ceiling. Defaults to 512 MiB. */
|
||||||
|
maxVaultBytes?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DEFAULT_MAX_OBJECT = 8 * 1024 * 1024;
|
||||||
|
const DEFAULT_MAX_VAULT = 512 * 1024 * 1024;
|
||||||
|
|
||||||
|
function fail(code: string, message: string, status: ContentfulStatusCode) {
|
||||||
|
return { body: { error: { code, message } }, status };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createVaultRoutes(
|
||||||
|
store: VaultStore,
|
||||||
|
crypto: CryptoProvider,
|
||||||
|
options: VaultRoutesOptions = {},
|
||||||
|
): Hono {
|
||||||
|
const app = new Hono();
|
||||||
|
const maxObject = options.maxObjectBytes ?? DEFAULT_MAX_OBJECT;
|
||||||
|
const maxVault = options.maxVaultBytes ?? DEFAULT_MAX_VAULT;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check a signed request against the vault's pinned key, pinning it on the
|
||||||
|
* first write. `publicKey` is only honoured when nothing is pinned yet —
|
||||||
|
* otherwise anyone could rotate the owner by simply asserting a new key.
|
||||||
|
*/
|
||||||
|
async function authorize(
|
||||||
|
vaultId: string,
|
||||||
|
payload: Record<string, unknown>,
|
||||||
|
): Promise<
|
||||||
|
{ ok: true } | { ok: false; code: string; message: string; status: ContentfulStatusCode }
|
||||||
|
> {
|
||||||
|
const claimed = typeof payload.publicKey === 'string' ? payload.publicKey : null;
|
||||||
|
const pinned = await store.getOwner(vaultId);
|
||||||
|
|
||||||
|
if (!pinned) {
|
||||||
|
if (!claimed) {
|
||||||
|
return { ok: false, code: 'UNAUTHORIZED', message: 'first write must carry publicKey', status: 401 };
|
||||||
|
}
|
||||||
|
const key = fromBase64(claimed);
|
||||||
|
try {
|
||||||
|
await verifyPayload(crypto, key, payload);
|
||||||
|
} catch (e) {
|
||||||
|
return { ok: false, code: 'UNAUTHORIZED', message: String((e as Error).message), status: 401 };
|
||||||
|
}
|
||||||
|
await store.setOwner(vaultId, key);
|
||||||
|
return { ok: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await verifyPayload(crypto, pinned, payload);
|
||||||
|
} catch (e) {
|
||||||
|
return { ok: false, code: 'UNAUTHORIZED', message: String((e as Error).message), status: 401 };
|
||||||
|
}
|
||||||
|
return { ok: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
app.get('/v1/vault/:vaultId/log', async (c) => {
|
||||||
|
const vaultId = c.req.param('vaultId');
|
||||||
|
if (!ID_REGEX.test(vaultId)) {
|
||||||
|
const f = fail('BAD_REQUEST', 'vaultId must be 64 lowercase hex chars', 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
const limitRaw = c.req.query('limit');
|
||||||
|
const limit = limitRaw ? Number(limitRaw) : undefined;
|
||||||
|
const entries = await store.log(vaultId, Number.isFinite(limit) ? limit : undefined);
|
||||||
|
return c.json({ entries, head: await store.head(vaultId) });
|
||||||
|
});
|
||||||
|
|
||||||
|
// A have-check before uploading. This is what makes an unchanged file free:
|
||||||
|
// the client asks about every hash in the new manifest and only sends the
|
||||||
|
// ones the relay is missing.
|
||||||
|
app.on(['HEAD', 'GET'], '/v1/vault/:vaultId/object/:hash', async (c) => {
|
||||||
|
const vaultId = c.req.param('vaultId');
|
||||||
|
const hash = c.req.param('hash');
|
||||||
|
if (!ID_REGEX.test(vaultId) || !HASH_REGEX.test(hash)) {
|
||||||
|
const f = fail('BAD_REQUEST', 'bad vaultId or hash', 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
if (c.req.method === 'HEAD') {
|
||||||
|
return c.body(null, (await store.hasObject(vaultId, hash)) ? 200 : 404);
|
||||||
|
}
|
||||||
|
const bytes = await store.getObject(vaultId, hash);
|
||||||
|
if (!bytes) {
|
||||||
|
const f = fail('NOT_FOUND', 'no such object', 404);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
return c.body(bytes as unknown as ArrayBuffer, 200, {
|
||||||
|
'content-type': 'application/octet-stream',
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
app.put('/v1/vault/:vaultId/object/:hash', async (c) => {
|
||||||
|
const vaultId = c.req.param('vaultId');
|
||||||
|
const hash = c.req.param('hash');
|
||||||
|
if (!ID_REGEX.test(vaultId) || !HASH_REGEX.test(hash)) {
|
||||||
|
const f = fail('BAD_REQUEST', 'bad vaultId or hash', 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
const body = (await c.req.json().catch(() => null)) as Record<string, unknown> | null;
|
||||||
|
if (!body || typeof body.data !== 'string') {
|
||||||
|
const f = fail('BAD_REQUEST', 'expected { data, signedAt, signature }', 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
const auth = await authorize(vaultId, body);
|
||||||
|
if (!auth.ok) {
|
||||||
|
const f = fail(auth.code, auth.message, auth.status);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
const bytes = fromBase64(body.data);
|
||||||
|
if (bytes.length > maxObject) {
|
||||||
|
const f = fail('TOO_LARGE', `object exceeds ${maxObject} bytes`, 413);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
if ((await store.usage(vaultId)) + bytes.length > maxVault) {
|
||||||
|
const f = fail('TOO_LARGE', `vault exceeds ${maxVault} bytes`, 413);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The name must be the hash of the bytes. Without this the store would
|
||||||
|
// accept a mislabelled object, and every later read of that name would
|
||||||
|
// fail decryption somewhere far away from the cause.
|
||||||
|
const actual = objectHash(bytes);
|
||||||
|
if (actual !== hash) {
|
||||||
|
const f = fail('BAD_REQUEST', `hash mismatch: bytes hash to ${actual}`, 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
await store.putObject(vaultId, hash, bytes);
|
||||||
|
return c.json({ stored: true, hash });
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/v1/vault/:vaultId/commit', async (c) => {
|
||||||
|
const vaultId = c.req.param('vaultId');
|
||||||
|
if (!ID_REGEX.test(vaultId)) {
|
||||||
|
const f = fail('BAD_REQUEST', 'vaultId must be 64 lowercase hex chars', 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
const body = (await c.req.json().catch(() => null)) as Record<string, unknown> | null;
|
||||||
|
if (!body || typeof body.manifest !== 'string' || typeof body.seq !== 'number') {
|
||||||
|
const f = fail('BAD_REQUEST', 'expected { manifest, seq, hashes, signedAt, signature }', 400);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
const auth = await authorize(vaultId, body);
|
||||||
|
if (!auth.ok) {
|
||||||
|
const f = fail(auth.code, auth.message, auth.status);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
const head = await store.head(vaultId);
|
||||||
|
if (body.seq !== head + 1) {
|
||||||
|
// Two devices committed from the same head. The loser has to re-read
|
||||||
|
// and re-commit; retrying the same seq would overwrite a version that
|
||||||
|
// is already part of the history.
|
||||||
|
const f = fail('SEQ_CONFLICT', `expected seq ${head + 1}, got ${body.seq}`, 409);
|
||||||
|
return c.json({ ...f.body, head }, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(await store.hasObject(vaultId, body.manifest))) {
|
||||||
|
const f = fail('MISSING_OBJECTS', 'manifest object not uploaded', 409);
|
||||||
|
return c.json(f.body, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every object the manifest references must already be here, or the
|
||||||
|
// commit would publish a version that cannot be restored. Checking at
|
||||||
|
// commit time is what makes the log trustworthy.
|
||||||
|
const hashes = Array.isArray(body.hashes) ? (body.hashes as string[]) : [];
|
||||||
|
const missing: string[] = [];
|
||||||
|
for (const h of hashes) {
|
||||||
|
if (!HASH_REGEX.test(h) || !(await store.hasObject(vaultId, h))) missing.push(h);
|
||||||
|
}
|
||||||
|
if (missing.length > 0) {
|
||||||
|
const f = fail('MISSING_OBJECTS', `${missing.length} referenced object(s) missing`, 409);
|
||||||
|
return c.json({ ...f.body, missing: missing.slice(0, 20) }, f.status);
|
||||||
|
}
|
||||||
|
|
||||||
|
await store.appendLog(vaultId, {
|
||||||
|
seq: body.seq,
|
||||||
|
manifest: body.manifest,
|
||||||
|
at: typeof body.at === 'number' ? body.at : Date.now(),
|
||||||
|
bytes: await store.usage(vaultId),
|
||||||
|
});
|
||||||
|
|
||||||
|
return c.json({ seq: body.seq, head: body.seq });
|
||||||
|
});
|
||||||
|
|
||||||
|
return app;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type { VaultStore } from './store.js';
|
||||||
|
export { MemoryVaultStore } from './store.js';
|
||||||
|
export type { VaultManifest };
|
||||||
90
packages/shade-vault/src/store.ts
Normal file
90
packages/shade-vault/src/store.ts
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
/**
|
||||||
|
* Relay-side storage for a vault.
|
||||||
|
*
|
||||||
|
* The interface is deliberately dumb: put bytes under a hash, list a log,
|
||||||
|
* append to it. Nothing here can decrypt, and nothing needs to — which is
|
||||||
|
* what makes a SQLite, Postgres or object-store implementation equivalent.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { VaultLogEntry } from './types.js';
|
||||||
|
|
||||||
|
export interface VaultStore {
|
||||||
|
/** Ed25519 pubkey pinned on first write (TOFU), or null for a fresh vault. */
|
||||||
|
getOwner(vaultId: string): Promise<Uint8Array | null>;
|
||||||
|
setOwner(vaultId: string, publicKey: Uint8Array): Promise<void>;
|
||||||
|
|
||||||
|
hasObject(vaultId: string, hash: string): Promise<boolean>;
|
||||||
|
putObject(vaultId: string, hash: string, bytes: Uint8Array): Promise<void>;
|
||||||
|
getObject(vaultId: string, hash: string): Promise<Uint8Array | null>;
|
||||||
|
|
||||||
|
/** Highest committed seq, or 0 when empty. */
|
||||||
|
head(vaultId: string): Promise<number>;
|
||||||
|
/** Newest last. `limit` counts back from the head. */
|
||||||
|
log(vaultId: string, limit?: number): Promise<VaultLogEntry[]>;
|
||||||
|
appendLog(vaultId: string, entry: VaultLogEntry): Promise<void>;
|
||||||
|
|
||||||
|
/** Total stored bytes, for quota decisions. */
|
||||||
|
usage(vaultId: string): Promise<number>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* In-memory store. Real for tests, a footgun in production — the same shape
|
||||||
|
* as `MemoryBlobStore`, and the same warning applies: a restart loses
|
||||||
|
* everything, and it will do so without an error.
|
||||||
|
*/
|
||||||
|
export class MemoryVaultStore implements VaultStore {
|
||||||
|
private owners = new Map<string, Uint8Array>();
|
||||||
|
private objects = new Map<string, Map<string, Uint8Array>>();
|
||||||
|
private logs = new Map<string, VaultLogEntry[]>();
|
||||||
|
|
||||||
|
async getOwner(vaultId: string): Promise<Uint8Array | null> {
|
||||||
|
return this.owners.get(vaultId) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async setOwner(vaultId: string, publicKey: Uint8Array): Promise<void> {
|
||||||
|
this.owners.set(vaultId, publicKey);
|
||||||
|
}
|
||||||
|
|
||||||
|
private bucket(vaultId: string): Map<string, Uint8Array> {
|
||||||
|
let b = this.objects.get(vaultId);
|
||||||
|
if (!b) {
|
||||||
|
b = new Map();
|
||||||
|
this.objects.set(vaultId, b);
|
||||||
|
}
|
||||||
|
return b;
|
||||||
|
}
|
||||||
|
|
||||||
|
async hasObject(vaultId: string, hash: string): Promise<boolean> {
|
||||||
|
return this.bucket(vaultId).has(hash);
|
||||||
|
}
|
||||||
|
|
||||||
|
async putObject(vaultId: string, hash: string, bytes: Uint8Array): Promise<void> {
|
||||||
|
this.bucket(vaultId).set(hash, bytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
async getObject(vaultId: string, hash: string): Promise<Uint8Array | null> {
|
||||||
|
return this.bucket(vaultId).get(hash) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async head(vaultId: string): Promise<number> {
|
||||||
|
const l = this.logs.get(vaultId);
|
||||||
|
return l && l.length > 0 ? l[l.length - 1]!.seq : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
async log(vaultId: string, limit?: number): Promise<VaultLogEntry[]> {
|
||||||
|
const l = this.logs.get(vaultId) ?? [];
|
||||||
|
return limit === undefined ? [...l] : l.slice(Math.max(0, l.length - limit));
|
||||||
|
}
|
||||||
|
|
||||||
|
async appendLog(vaultId: string, entry: VaultLogEntry): Promise<void> {
|
||||||
|
const l = this.logs.get(vaultId) ?? [];
|
||||||
|
l.push(entry);
|
||||||
|
this.logs.set(vaultId, l);
|
||||||
|
}
|
||||||
|
|
||||||
|
async usage(vaultId: string): Promise<number> {
|
||||||
|
let total = 0;
|
||||||
|
for (const bytes of this.bucket(vaultId).values()) total += bytes.length;
|
||||||
|
return total;
|
||||||
|
}
|
||||||
|
}
|
||||||
79
packages/shade-vault/src/types.ts
Normal file
79
packages/shade-vault/src/types.ts
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
/**
|
||||||
|
* The vault's data model.
|
||||||
|
*
|
||||||
|
* A vault is a collection of files that lives encrypted on the relay: the
|
||||||
|
* pieces Shade was missing to let an app offer backup, or to let a client
|
||||||
|
* read data while the peer that owns it is switched off.
|
||||||
|
*
|
||||||
|
* Three ideas, and the shape follows from them:
|
||||||
|
*
|
||||||
|
* 1. **Objects are content-addressed.** An object's name is the hash of its
|
||||||
|
* *ciphertext*, so the relay can store and dedupe without understanding
|
||||||
|
* anything. Re-uploading an unchanged file is free, which matters when a
|
||||||
|
* workspace of several hundred files changes three lines at a time.
|
||||||
|
*
|
||||||
|
* 2. **A manifest names the collection.** Paths live in the manifest, not in
|
||||||
|
* object names — the relay must not learn that a user has a file called
|
||||||
|
* `Projects/Divorce/plan.md`. The manifest is itself encrypted and stored
|
||||||
|
* as an object.
|
||||||
|
*
|
||||||
|
* 3. **The log is append-only.** Each commit adds an entry pointing at a
|
||||||
|
* manifest. History, rollback and "what changed last Tuesday" all fall out
|
||||||
|
* of that, which is the versioning half of the requirement.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** One file in a manifest. `hash` names the ciphertext object. */
|
||||||
|
export interface VaultEntry {
|
||||||
|
/** Path within the collection, as the app understands it. */
|
||||||
|
path: string;
|
||||||
|
/** Lowercase hex SHA-256 of the ciphertext object. */
|
||||||
|
hash: string;
|
||||||
|
/** Plaintext byte length, for progress reporting and sanity checks. */
|
||||||
|
size: number;
|
||||||
|
/** Optional app-defined mode/metadata, opaque to the vault. */
|
||||||
|
mode?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The full contents of a collection at one point in time. */
|
||||||
|
export interface VaultManifest {
|
||||||
|
/** Schema version — a client that does not know it must refuse, not guess. */
|
||||||
|
version: 1;
|
||||||
|
/** Monotonic, assigned by the client and checked by the relay. */
|
||||||
|
seq: number;
|
||||||
|
/** Epoch millis when the commit was made. */
|
||||||
|
at: number;
|
||||||
|
/** Optional human note, e.g. "nattlig backup". */
|
||||||
|
message?: string;
|
||||||
|
entries: VaultEntry[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One row of the append-only log. */
|
||||||
|
export interface VaultLogEntry {
|
||||||
|
seq: number;
|
||||||
|
/** Hash of the encrypted manifest object. */
|
||||||
|
manifest: string;
|
||||||
|
at: number;
|
||||||
|
/** Total bytes of all objects the manifest references, for quota display. */
|
||||||
|
bytes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** What `GET /v1/vault/:id/log` returns. */
|
||||||
|
export interface VaultLog {
|
||||||
|
entries: VaultLogEntry[];
|
||||||
|
/** Highest seq the relay holds; `0` when the vault is empty. */
|
||||||
|
head: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Error codes clients branch on.
|
||||||
|
*
|
||||||
|
* `SEQ_CONFLICT` is the important one: two devices committed from the same
|
||||||
|
* head, and the loser must re-read and re-commit rather than retry blindly.
|
||||||
|
*/
|
||||||
|
export type VaultErrorCode =
|
||||||
|
| 'BAD_REQUEST'
|
||||||
|
| 'UNAUTHORIZED'
|
||||||
|
| 'NOT_FOUND'
|
||||||
|
| 'SEQ_CONFLICT'
|
||||||
|
| 'TOO_LARGE'
|
||||||
|
| 'MISSING_OBJECTS';
|
||||||
153
packages/shade-vault/tests/scaffoldd-e2e.test.ts
Normal file
153
packages/shade-vault/tests/scaffoldd-e2e.test.ts
Normal file
@@ -0,0 +1,153 @@
|
|||||||
|
/**
|
||||||
|
* The whole backup chain, against real data.
|
||||||
|
*
|
||||||
|
* Talks to a running `scaffoldd` over its unix socket, asks what the backup
|
||||||
|
* set is, reads those files off disk, pushes them through the vault, and pulls
|
||||||
|
* them back — then compares byte-for-byte.
|
||||||
|
*
|
||||||
|
* This is the test that would catch the things unit tests cannot: a path that
|
||||||
|
* survives the round trip wrong, a 600 KB log that trips a size ceiling, a
|
||||||
|
* workspace whose file count makes the manifest too large to commit. It is
|
||||||
|
* skipped when no daemon is listening, so it never blocks an ordinary run.
|
||||||
|
*
|
||||||
|
* scaffoldd --socket /tmp/scaffoldd-e2e.sock &
|
||||||
|
* SCAFFOLDD_SOCKET=/tmp/scaffoldd-e2e.sock bun test scaffoldd-e2e
|
||||||
|
*/
|
||||||
|
import { describe, test, expect } from 'bun:test';
|
||||||
|
import { connect } from 'node:net';
|
||||||
|
import { readFile } from 'node:fs/promises';
|
||||||
|
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
||||||
|
import { createVaultRoutes } from '../src/server.js';
|
||||||
|
import { MemoryVaultStore } from '../src/store.js';
|
||||||
|
import { HttpVaultTransport } from '../src/http-transport.js';
|
||||||
|
import { VaultClient, deriveVaultKeys } from '../src/client.js';
|
||||||
|
import type { VaultFile } from '../src/client.js';
|
||||||
|
|
||||||
|
const SOCKET = process.env.SCAFFOLDD_SOCKET;
|
||||||
|
const crypto = new SubtleCryptoProvider();
|
||||||
|
|
||||||
|
interface BackupFile {
|
||||||
|
path: string;
|
||||||
|
source: string;
|
||||||
|
size: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
function ask(socketPath: string, request: object): Promise<any> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const sock = connect(socketPath);
|
||||||
|
let buf = '';
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
sock.destroy();
|
||||||
|
reject(new Error('scaffoldd svarte ikke innen 20 s'));
|
||||||
|
}, 20_000);
|
||||||
|
// Deliberately no `end()` after writing: Bun's socket closes both
|
||||||
|
// directions, so the daemon's reply is discarded before it arrives. The
|
||||||
|
// protocol is line-delimited, so read until the first complete line and
|
||||||
|
// close from here instead.
|
||||||
|
sock.on('connect', () => {
|
||||||
|
sock.write(`${JSON.stringify(request)}\n`);
|
||||||
|
});
|
||||||
|
sock.on('data', (c) => {
|
||||||
|
buf += c.toString();
|
||||||
|
const nl = buf.indexOf('\n');
|
||||||
|
if (nl === -1) return;
|
||||||
|
clearTimeout(timer);
|
||||||
|
sock.destroy();
|
||||||
|
const res = JSON.parse(buf.slice(0, nl));
|
||||||
|
res.ok ? resolve(res.result) : reject(new Error(`${res.code}: ${res.error}`));
|
||||||
|
});
|
||||||
|
sock.on('end', () => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
if (!buf.includes('\n')) reject(new Error('tomt svar'));
|
||||||
|
});
|
||||||
|
sock.on('error', (e) => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
reject(e);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function harness() {
|
||||||
|
const store = new MemoryVaultStore();
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const transport = new HttpVaultTransport('http://vault.test', (i, init) =>
|
||||||
|
routes.fetch(new Request(i, init)),
|
||||||
|
);
|
||||||
|
const keys = await deriveVaultKeys(new Uint8Array(32).fill(42), 'scaffold-e2e');
|
||||||
|
return { store, client: new VaultClient(crypto, keys, transport) };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe.skipIf(!SOCKET)('scaffoldd → vault, real workspace', () => {
|
||||||
|
test('the whole backup set round-trips byte-for-byte', async () => {
|
||||||
|
const set = (await ask(SOCKET!, { op: 'backupSet' })) as {
|
||||||
|
files: BackupFile[];
|
||||||
|
count: number;
|
||||||
|
bytes: number;
|
||||||
|
};
|
||||||
|
expect(set.count).toBeGreaterThan(50);
|
||||||
|
|
||||||
|
const files: VaultFile[] = [];
|
||||||
|
for (const f of set.files) {
|
||||||
|
try {
|
||||||
|
files.push({ path: f.path, bytes: new Uint8Array(await readFile(f.source)) });
|
||||||
|
} catch {
|
||||||
|
// Matches the bridge: one unreadable file is skipped, not fatal.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const { client } = await harness();
|
||||||
|
const push = await client.push(files, 1_786_600_000_000, 'e2e');
|
||||||
|
expect(push.seq).toBe(1);
|
||||||
|
expect(push.uploaded).toBe(files.length);
|
||||||
|
|
||||||
|
const { files: back } = await client.pull();
|
||||||
|
expect(back.size).toBe(files.length);
|
||||||
|
|
||||||
|
// Every single file, not a sample: a backup that restores 99% of a
|
||||||
|
// workspace is not a backup.
|
||||||
|
for (const f of files) {
|
||||||
|
const restored = back.get(f.path);
|
||||||
|
expect(restored).toBeDefined();
|
||||||
|
expect(restored!.length).toBe(f.bytes.length);
|
||||||
|
expect(Buffer.from(restored!).equals(Buffer.from(f.bytes))).toBe(true);
|
||||||
|
}
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
test('a second push after one edit sends almost nothing', async () => {
|
||||||
|
// The property that decides whether hourly backup is affordable.
|
||||||
|
const set = (await ask(SOCKET!, { op: 'backupSet' })) as { files: BackupFile[] };
|
||||||
|
const files: VaultFile[] = [];
|
||||||
|
for (const f of set.files.slice(0, 120)) {
|
||||||
|
try {
|
||||||
|
files.push({ path: f.path, bytes: new Uint8Array(await readFile(f.source)) });
|
||||||
|
} catch {
|
||||||
|
/* skipped */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const { client } = await harness();
|
||||||
|
const first = await client.push(files, 1_786_600_000_000);
|
||||||
|
|
||||||
|
const edited = files.map((f, i) =>
|
||||||
|
i === 0 ? { path: f.path, bytes: new TextEncoder().encode('endret én linje') } : f,
|
||||||
|
);
|
||||||
|
const second = await client.push(edited, 1_786_600_001_000);
|
||||||
|
|
||||||
|
expect(first.uploaded).toBe(files.length);
|
||||||
|
expect(second.uploaded).toBe(1);
|
||||||
|
expect(second.reused).toBe(files.length - 1);
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
test('the largest file in the workspace survives the round trip', async () => {
|
||||||
|
// Terra's log.md is ~617 KB. Nothing in the chain may quietly cap it.
|
||||||
|
const set = (await ask(SOCKET!, { op: 'backupSet' })) as { files: BackupFile[] };
|
||||||
|
const biggest = set.files.reduce((a, b) => (a.size > b.size ? a : b));
|
||||||
|
expect(biggest.size).toBeGreaterThan(100_000);
|
||||||
|
|
||||||
|
const bytes = new Uint8Array(await readFile(biggest.source));
|
||||||
|
const { client } = await harness();
|
||||||
|
await client.push([{ path: biggest.path, bytes }], 1_786_600_000_000);
|
||||||
|
const { files } = await client.pull();
|
||||||
|
expect(files.get(biggest.path)!.length).toBe(bytes.length);
|
||||||
|
}, 120_000);
|
||||||
|
});
|
||||||
313
packages/shade-vault/tests/vault.test.ts
Normal file
313
packages/shade-vault/tests/vault.test.ts
Normal file
@@ -0,0 +1,313 @@
|
|||||||
|
/**
|
||||||
|
* End-to-end tests for the vault.
|
||||||
|
*
|
||||||
|
* The client talks to the REAL route handlers through Hono's `fetch`, not to a
|
||||||
|
* mock. A mock transport would agree with the client by construction and prove
|
||||||
|
* nothing about the wire contract — which is exactly the surface a phone and a
|
||||||
|
* daemon have to share.
|
||||||
|
*/
|
||||||
|
import { describe, test, expect } from 'bun:test';
|
||||||
|
import { SubtleCryptoProvider } from '@shade/crypto-web';
|
||||||
|
import { createVaultRoutes } from '../src/server.js';
|
||||||
|
import { MemoryVaultStore } from '../src/store.js';
|
||||||
|
import { HttpVaultTransport } from '../src/http-transport.js';
|
||||||
|
import { VaultClient, deriveVaultKeys } from '../src/client.js';
|
||||||
|
import { objectHash } from '../src/crypto.js';
|
||||||
|
|
||||||
|
const crypto = new SubtleCryptoProvider();
|
||||||
|
const AT = 1_786_600_000_000;
|
||||||
|
|
||||||
|
function enc(s: string): Uint8Array {
|
||||||
|
return new TextEncoder().encode(s);
|
||||||
|
}
|
||||||
|
function dec(b: Uint8Array): string {
|
||||||
|
return new TextDecoder().decode(b);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A client wired to a fresh in-memory relay. */
|
||||||
|
async function harness(masterKey = new Uint8Array(32).fill(7), app = 'scaffold') {
|
||||||
|
const store = new MemoryVaultStore();
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const transport = new HttpVaultTransport('http://vault.test', (input, init) =>
|
||||||
|
routes.fetch(new Request(input, init)),
|
||||||
|
);
|
||||||
|
const keys = await deriveVaultKeys(masterKey, app);
|
||||||
|
return { store, keys, client: new VaultClient(crypto, keys, transport) };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('round trip', () => {
|
||||||
|
test('files pushed can be pulled back byte-for-byte', async () => {
|
||||||
|
const { client } = await harness();
|
||||||
|
const files = [
|
||||||
|
{ path: '.scaffold/plan.md', bytes: enc('# Plan\n\n## Nå\n- [ ] noe\n') },
|
||||||
|
{ path: '.scaffold/tasks.yaml', bytes: enc('tasks: []\n') },
|
||||||
|
];
|
||||||
|
|
||||||
|
const res = await client.push(files, AT, 'første backup');
|
||||||
|
expect(res.seq).toBe(1);
|
||||||
|
expect(res.uploaded).toBe(2);
|
||||||
|
|
||||||
|
const { manifest, files: back } = await client.pull();
|
||||||
|
expect(manifest.seq).toBe(1);
|
||||||
|
expect(manifest.message).toBe('første backup');
|
||||||
|
expect(dec(back.get('.scaffold/plan.md')!)).toBe('# Plan\n\n## Nå\n- [ ] noe\n');
|
||||||
|
expect(dec(back.get('.scaffold/tasks.yaml')!)).toBe('tasks: []\n');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a fresh device with only the credentials can restore', async () => {
|
||||||
|
// The recovery story: same master key, nothing else carried over.
|
||||||
|
const master = new Uint8Array(32).fill(11);
|
||||||
|
const { client, store } = await harness(master);
|
||||||
|
await client.push([{ path: 'notes.yaml', bytes: enc('notes: [en, to]\n') }], AT);
|
||||||
|
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const transport = new HttpVaultTransport('http://vault.test', (i, init) =>
|
||||||
|
routes.fetch(new Request(i, init)),
|
||||||
|
);
|
||||||
|
const keys = await deriveVaultKeys(master, 'scaffold');
|
||||||
|
const fresh = new VaultClient(crypto, keys, transport);
|
||||||
|
|
||||||
|
const { files } = await fresh.pull();
|
||||||
|
expect(dec(files.get('notes.yaml')!)).toBe('notes: [en, to]\n');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a different master key cannot read the vault', async () => {
|
||||||
|
const { store } = await harness(new Uint8Array(32).fill(1));
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const transport = new HttpVaultTransport('http://vault.test', (i, init) =>
|
||||||
|
routes.fetch(new Request(i, init)),
|
||||||
|
);
|
||||||
|
const wrong = await deriveVaultKeys(new Uint8Array(32).fill(2), 'scaffold');
|
||||||
|
const intruder = new VaultClient(crypto, wrong, transport);
|
||||||
|
// A different master derives a different vaultId, so there is nothing
|
||||||
|
// there to read in the first place — the id is itself a secret.
|
||||||
|
await expect(intruder.pull()).rejects.toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('versioning', () => {
|
||||||
|
test('each push is a new version and old ones stay readable', async () => {
|
||||||
|
const { client } = await harness();
|
||||||
|
await client.push([{ path: 'plan.md', bytes: enc('versjon 1') }], AT);
|
||||||
|
await client.push([{ path: 'plan.md', bytes: enc('versjon 2') }], AT + 1000);
|
||||||
|
await client.push([{ path: 'plan.md', bytes: enc('versjon 3') }], AT + 2000);
|
||||||
|
|
||||||
|
const log = await client.history();
|
||||||
|
expect(log.head).toBe(3);
|
||||||
|
expect(log.entries.map((e) => e.seq)).toEqual([1, 2, 3]);
|
||||||
|
|
||||||
|
// Rollback: the whole point of keeping the log.
|
||||||
|
expect(dec((await client.pull(1)).files.get('plan.md')!)).toBe('versjon 1');
|
||||||
|
expect(dec((await client.pull(2)).files.get('plan.md')!)).toBe('versjon 2');
|
||||||
|
expect(dec((await client.pull()).files.get('plan.md')!)).toBe('versjon 3');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('unchanged files are not re-uploaded', async () => {
|
||||||
|
// The property that makes backing up a 9 MB workspace on every change
|
||||||
|
// affordable: only what actually moved goes over the wire.
|
||||||
|
const { client } = await harness();
|
||||||
|
const stable = { path: 'stor-logg.md', bytes: enc('x'.repeat(50_000)) };
|
||||||
|
|
||||||
|
const first = await client.push([stable, { path: 'plan.md', bytes: enc('en') }], AT);
|
||||||
|
expect(first.uploaded).toBe(2);
|
||||||
|
|
||||||
|
const second = await client.push([stable, { path: 'plan.md', bytes: enc('to') }], AT + 1);
|
||||||
|
expect(second.uploaded).toBe(1);
|
||||||
|
expect(second.reused).toBe(1);
|
||||||
|
expect(second.bytesUploaded).toBeLessThan(1000);
|
||||||
|
|
||||||
|
// And the reused file is still intact in the new version.
|
||||||
|
const { files } = await client.pull();
|
||||||
|
expect(files.get('stor-logg.md')!.length).toBe(50_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a deleted file is absent from the new version but present in the old', async () => {
|
||||||
|
const { client } = await harness();
|
||||||
|
await client.push(
|
||||||
|
[
|
||||||
|
{ path: 'a.md', bytes: enc('A') },
|
||||||
|
{ path: 'b.md', bytes: enc('B') },
|
||||||
|
],
|
||||||
|
AT,
|
||||||
|
);
|
||||||
|
await client.push([{ path: 'a.md', bytes: enc('A') }], AT + 1);
|
||||||
|
|
||||||
|
expect((await client.pull()).files.has('b.md')).toBe(false);
|
||||||
|
expect(dec((await client.pull(1)).files.get('b.md')!)).toBe('B');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('the relay is blind', () => {
|
||||||
|
test('stored objects contain no plaintext', async () => {
|
||||||
|
const { client, store } = await harness();
|
||||||
|
await client.push([{ path: 'hemmelig/plan.md', bytes: enc('SENSITIVT INNHOLD') }], AT);
|
||||||
|
|
||||||
|
const log = await store.log(client.vaultId);
|
||||||
|
const manifestBytes = await store.getObject(client.vaultId, log[0]!.manifest);
|
||||||
|
const asText = dec(manifestBytes!);
|
||||||
|
// Neither the contents nor the path leaks: paths live inside the
|
||||||
|
// encrypted manifest, not in object names.
|
||||||
|
expect(asText).not.toContain('SENSITIVT');
|
||||||
|
expect(asText).not.toContain('hemmelig');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('object names are the hash of the ciphertext, so the relay can verify', async () => {
|
||||||
|
const { client, store } = await harness();
|
||||||
|
await client.push([{ path: 'x', bytes: enc('hei') }], AT);
|
||||||
|
const log = await store.log(client.vaultId);
|
||||||
|
const bytes = await store.getObject(client.vaultId, log[0]!.manifest);
|
||||||
|
expect(objectHash(bytes!)).toBe(log[0]!.manifest);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an object whose bytes do not match its name is rejected', async () => {
|
||||||
|
const store = new MemoryVaultStore();
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const keys = await deriveVaultKeys(new Uint8Array(32).fill(3), 'scaffold');
|
||||||
|
const { signPayload } = await import('@shade/server');
|
||||||
|
const { toBase64 } = await import('@shade/core');
|
||||||
|
|
||||||
|
const body = await signPayload(crypto, keys.signingSeed, {
|
||||||
|
data: toBase64(enc('juks')),
|
||||||
|
publicKey: toBase64(keys.publicKey),
|
||||||
|
});
|
||||||
|
const res = await routes.fetch(
|
||||||
|
new Request(`http://v/v1/vault/${keys.vaultId}/object/${'0'.repeat(64)}`, {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(res.status).toBe(400);
|
||||||
|
expect((await res.json()).error.code).toBe('BAD_REQUEST');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('concurrent writers', () => {
|
||||||
|
test('a commit from a stale head is refused', async () => {
|
||||||
|
// Two devices push from the same version. The second must not be able to
|
||||||
|
// overwrite a sequence number that is already history.
|
||||||
|
const { client, keys, store } = await harness();
|
||||||
|
await client.push([{ path: 'plan.md', bytes: enc('en')}], AT);
|
||||||
|
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const { signPayload } = await import('@shade/server');
|
||||||
|
const { toBase64 } = await import('@shade/core');
|
||||||
|
const log = await store.log(keys.vaultId);
|
||||||
|
const body = await signPayload(crypto, keys.signingSeed, {
|
||||||
|
manifest: log[0]!.manifest,
|
||||||
|
seq: 1, // already taken
|
||||||
|
at: AT,
|
||||||
|
hashes: [],
|
||||||
|
publicKey: toBase64(keys.publicKey),
|
||||||
|
});
|
||||||
|
|
||||||
|
const res = await routes.fetch(
|
||||||
|
new Request(`http://v/v1/vault/${keys.vaultId}/commit`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(res.status).toBe(409);
|
||||||
|
const err = await res.json();
|
||||||
|
expect(err.error.code).toBe('SEQ_CONFLICT');
|
||||||
|
expect(err.head).toBe(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a commit referencing a missing object is refused', async () => {
|
||||||
|
// Otherwise the log would publish a version that cannot be restored.
|
||||||
|
const { client, keys, store } = await harness();
|
||||||
|
await client.push([{ path: 'plan.md', bytes: enc('en') }], AT);
|
||||||
|
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const { signPayload } = await import('@shade/server');
|
||||||
|
const { toBase64 } = await import('@shade/core');
|
||||||
|
const log = await store.log(keys.vaultId);
|
||||||
|
const body = await signPayload(crypto, keys.signingSeed, {
|
||||||
|
manifest: log[0]!.manifest,
|
||||||
|
seq: 2,
|
||||||
|
at: AT,
|
||||||
|
hashes: ['a'.repeat(64)],
|
||||||
|
publicKey: toBase64(keys.publicKey),
|
||||||
|
});
|
||||||
|
|
||||||
|
const res = await routes.fetch(
|
||||||
|
new Request(`http://v/v1/vault/${keys.vaultId}/commit`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(res.status).toBe(409);
|
||||||
|
expect((await res.json()).error.code).toBe('MISSING_OBJECTS');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('authorisation', () => {
|
||||||
|
test('a second key cannot write to a vault another key pinned', async () => {
|
||||||
|
const { client, keys, store } = await harness();
|
||||||
|
await client.push([{ path: 'plan.md', bytes: enc('mitt') }], AT);
|
||||||
|
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const { signPayload } = await import('@shade/server');
|
||||||
|
const { toBase64 } = await import('@shade/core');
|
||||||
|
const attacker = await deriveVaultKeys(new Uint8Array(32).fill(9), 'scaffold');
|
||||||
|
|
||||||
|
// Signed correctly — but by the wrong key, and asserting its own pubkey.
|
||||||
|
const body = await signPayload(crypto, attacker.signingSeed, {
|
||||||
|
data: toBase64(enc('tull')),
|
||||||
|
publicKey: toBase64(attacker.publicKey),
|
||||||
|
});
|
||||||
|
const res = await routes.fetch(
|
||||||
|
new Request(
|
||||||
|
`http://v/v1/vault/${keys.vaultId}/object/${objectHash(enc('tull'))}`,
|
||||||
|
{
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
},
|
||||||
|
),
|
||||||
|
);
|
||||||
|
expect(res.status).toBe(401);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('an unsigned write is refused', async () => {
|
||||||
|
const store = new MemoryVaultStore();
|
||||||
|
const routes = createVaultRoutes(store, crypto);
|
||||||
|
const res = await routes.fetch(
|
||||||
|
new Request(`http://v/v1/vault/${'a'.repeat(64)}/object/${'b'.repeat(64)}`, {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
body: JSON.stringify({ data: 'aGk=' }),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(res.status).toBe(401);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('key derivation', () => {
|
||||||
|
test('the same credentials derive the same vault, different ones do not', async () => {
|
||||||
|
const a = await deriveVaultKeys(new Uint8Array(32).fill(4), 'scaffold');
|
||||||
|
const b = await deriveVaultKeys(new Uint8Array(32).fill(4), 'scaffold');
|
||||||
|
const c = await deriveVaultKeys(new Uint8Array(32).fill(5), 'scaffold');
|
||||||
|
expect(a.vaultId).toBe(b.vaultId);
|
||||||
|
expect(a.vaultId).not.toBe(c.vaultId);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('two apps under one master do not share a vault', async () => {
|
||||||
|
const master = new Uint8Array(32).fill(6);
|
||||||
|
const scaffold = await deriveVaultKeys(master, 'scaffold');
|
||||||
|
const mail = await deriveVaultKeys(master, 'mail');
|
||||||
|
expect(scaffold.vaultId).not.toBe(mail.vaultId);
|
||||||
|
expect(scaffold.contentKey).not.toEqual(mail.contentKey);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the vault branch is separate from the profile-blob branch', async () => {
|
||||||
|
// A vault key reads every file; a profile-blob key reads a host list.
|
||||||
|
// Sharing a derivation would make one compromise into the other.
|
||||||
|
const master = new Uint8Array(32).fill(8);
|
||||||
|
const { deriveBlobKey } = await import('@shade/storage-encrypted');
|
||||||
|
const vault = await deriveVaultKeys(master, 'prism');
|
||||||
|
expect(vault.contentKey).not.toEqual(deriveBlobKey(master, 'prism'));
|
||||||
|
});
|
||||||
|
});
|
||||||
5
packages/shade-vault/tsconfig.json
Normal file
5
packages/shade-vault/tsconfig.json
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"extends": "../../tsconfig.json",
|
||||||
|
"compilerOptions": { "outDir": "dist", "rootDir": "src", "lib": ["ES2022", "DOM"] },
|
||||||
|
"include": ["src"]
|
||||||
|
}
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@shade/widgets",
|
"name": "@shade/widgets",
|
||||||
"version": "4.11.0",
|
"version": "4.13.0",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"types": "src/index.ts",
|
"types": "src/index.ts",
|
||||||
|
|||||||
@@ -13,6 +13,7 @@
|
|||||||
*
|
*
|
||||||
* Optional:
|
* Optional:
|
||||||
* DRY_RUN=1 — pack tarballs but do not publish (no token required)
|
* DRY_RUN=1 — pack tarballs but do not publish (no token required)
|
||||||
|
* --only <pkg> — publish one package (e.g. shade-files or @shade/files)
|
||||||
*/
|
*/
|
||||||
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
||||||
import { join } from 'path';
|
import { join } from 'path';
|
||||||
@@ -37,6 +38,7 @@ const PACKAGES = [
|
|||||||
'shade-transport-webrtc',
|
'shade-transport-webrtc',
|
||||||
'shade-server',
|
'shade-server',
|
||||||
'shade-inbox-server',
|
'shade-inbox-server',
|
||||||
|
'shade-vault',
|
||||||
'shade-inbox',
|
'shade-inbox',
|
||||||
'shade-transfer',
|
'shade-transfer',
|
||||||
'shade-files',
|
'shade-files',
|
||||||
@@ -51,10 +53,27 @@ const PACKAGES = [
|
|||||||
const REGISTRY_HOST = 'gt.zyon.no';
|
const REGISTRY_HOST = 'gt.zyon.no';
|
||||||
const ROOT = join(import.meta.dir, '..');
|
const ROOT = join(import.meta.dir, '..');
|
||||||
|
|
||||||
|
function selectedPackages(argv: string[]): string[] {
|
||||||
|
const onlyIndex = argv.indexOf('--only');
|
||||||
|
if (onlyIndex === -1) return PACKAGES;
|
||||||
|
const requested = argv[onlyIndex + 1];
|
||||||
|
if (!requested) throw new Error('Usage: publish-all.ts --only <shade-files|@shade/files>');
|
||||||
|
const normalized = requested.startsWith('@shade/')
|
||||||
|
? `shade-${requested.slice('@shade/'.length)}`
|
||||||
|
: requested.startsWith('shade-')
|
||||||
|
? requested
|
||||||
|
: `shade-${requested}`;
|
||||||
|
if (!PACKAGES.includes(normalized)) {
|
||||||
|
throw new Error(`Unknown Shade package: ${requested}`);
|
||||||
|
}
|
||||||
|
return [normalized];
|
||||||
|
}
|
||||||
|
|
||||||
async function main() {
|
async function main() {
|
||||||
const token = process.env.GITEA_TOKEN;
|
const token = process.env.GITEA_TOKEN;
|
||||||
const user = process.env.GITEA_USER ?? 'Stian';
|
const user = process.env.GITEA_USER ?? 'Stian';
|
||||||
const dryRun = process.env.DRY_RUN === '1';
|
const dryRun = process.env.DRY_RUN === '1';
|
||||||
|
const packagesToPublish = selectedPackages(process.argv.slice(2));
|
||||||
|
|
||||||
if (!token && !dryRun) {
|
if (!token && !dryRun) {
|
||||||
console.error('GITEA_TOKEN is required (or set DRY_RUN=1)');
|
console.error('GITEA_TOKEN is required (or set DRY_RUN=1)');
|
||||||
@@ -64,6 +83,7 @@ async function main() {
|
|||||||
const registryUrl = `https://${REGISTRY_HOST}/api/packages/${user}/npm/`;
|
const registryUrl = `https://${REGISTRY_HOST}/api/packages/${user}/npm/`;
|
||||||
console.log(`Target registry: ${registryUrl}`);
|
console.log(`Target registry: ${registryUrl}`);
|
||||||
console.log(`Dry run: ${dryRun ? 'yes' : 'no'}`);
|
console.log(`Dry run: ${dryRun ? 'yes' : 'no'}`);
|
||||||
|
console.log(`Packages: ${packagesToPublish.join(', ')}`);
|
||||||
console.log();
|
console.log();
|
||||||
|
|
||||||
const npmrcPath = join(ROOT, '.npmrc.publish');
|
const npmrcPath = join(ROOT, '.npmrc.publish');
|
||||||
@@ -86,7 +106,7 @@ async function main() {
|
|||||||
let alreadyPublished = 0;
|
let alreadyPublished = 0;
|
||||||
let failed = 0;
|
let failed = 0;
|
||||||
|
|
||||||
for (const pkg of PACKAGES) {
|
for (const pkg of packagesToPublish) {
|
||||||
const pkgDir = join(ROOT, 'packages', pkg);
|
const pkgDir = join(ROOT, 'packages', pkg);
|
||||||
const pkgJsonPath = join(pkgDir, 'package.json');
|
const pkgJsonPath = join(pkgDir, 'package.json');
|
||||||
if (!existsSync(pkgJsonPath)) {
|
if (!existsSync(pkgJsonPath)) {
|
||||||
@@ -99,10 +119,23 @@ async function main() {
|
|||||||
const pkgJson = JSON.parse(originalPkgJson);
|
const pkgJson = JSON.parse(originalPkgJson);
|
||||||
console.log(`→ ${pkgJson.name}@${pkgJson.version}`);
|
console.log(`→ ${pkgJson.name}@${pkgJson.version}`);
|
||||||
|
|
||||||
rewriteWorkspaceSpecs(pkgJson, versionByName);
|
|
||||||
writeFileSync(pkgJsonPath, `${JSON.stringify(pkgJson, null, 2)}\n`);
|
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
// Phase 1 — build. Leave `main`/`types`/`exports` pointing at `src/`
|
||||||
|
// so `tsc` can resolve cross-package `@shade/*` imports through
|
||||||
|
// workspace source (`dist/` doesn't exist yet for sibling packages
|
||||||
|
// when their package.json points there). We ship pre-built artefacts
|
||||||
|
// so strict-mode consumers (Cyndr et al.) don't recompile our source.
|
||||||
|
await $`cd ${pkgDir} && rm -rf dist && bunx tsc -p tsconfig.json`.quiet();
|
||||||
|
|
||||||
|
// Phase 2 — rewrite package.json for the publish surface. Swap entry
|
||||||
|
// points to `dist/`, drop `workspace:*` for real versions, ensure
|
||||||
|
// `files: ["dist"]` so npm pack ships only the built artefacts.
|
||||||
|
// Restored from `originalPkgJson` in the `finally` block at the end.
|
||||||
|
rewriteWorkspaceSpecs(pkgJson, versionByName);
|
||||||
|
rewriteEntryPointsForDist(pkgJson);
|
||||||
|
ensureFilesIncludesDist(pkgJson);
|
||||||
|
writeFileSync(pkgJsonPath, `${JSON.stringify(pkgJson, null, 2)}\n`);
|
||||||
|
|
||||||
if (dryRun) {
|
if (dryRun) {
|
||||||
await $`cd ${pkgDir} && bun pm pack --dry-run`.quiet();
|
await $`cd ${pkgDir} && bun pm pack --dry-run`.quiet();
|
||||||
} else {
|
} else {
|
||||||
@@ -159,6 +192,55 @@ function rewriteWorkspaceSpecs(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Swap `src/*.ts` references in `main`, `types`, and every `exports.<subpath>`
|
||||||
|
* entry over to `dist/*.{js,d.ts}` for the duration of the publish. The
|
||||||
|
* in-repo `package.json` is restored from `originalPkgJson` after pack so
|
||||||
|
* the source-pointing form survives in git.
|
||||||
|
*
|
||||||
|
* Why this lives at publish time instead of permanently: tooling that runs
|
||||||
|
* before any build (`tsc --noEmit`, IDE hover, the workspace dev loop) needs
|
||||||
|
* `main` to point at source so cross-package imports resolve without a
|
||||||
|
* forced build pass. Shipping dist-only is a consumer-facing concern.
|
||||||
|
*/
|
||||||
|
function rewriteEntryPointsForDist(pkgJson: Record<string, unknown>): void {
|
||||||
|
if (typeof pkgJson.main === 'string') {
|
||||||
|
pkgJson.main = pkgJson.main.replace(/^src\//, 'dist/').replace(/\.ts$/, '.js');
|
||||||
|
}
|
||||||
|
if (typeof pkgJson.types === 'string') {
|
||||||
|
pkgJson.types = pkgJson.types.replace(/^src\//, 'dist/').replace(/\.ts$/, '.d.ts');
|
||||||
|
}
|
||||||
|
if (pkgJson.exports !== undefined) {
|
||||||
|
pkgJson.exports = mapExportsForDist(pkgJson.exports);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function mapExportsForDist(node: unknown, isTypesCondition = false): unknown {
|
||||||
|
if (typeof node === 'string') {
|
||||||
|
if (!node.startsWith('./src/')) return node;
|
||||||
|
return node
|
||||||
|
.replace(/^\.\/src\//, './dist/')
|
||||||
|
.replace(/\.ts$/, isTypesCondition ? '.d.ts' : '.js');
|
||||||
|
}
|
||||||
|
if (node === null || typeof node !== 'object' || Array.isArray(node)) {
|
||||||
|
return node;
|
||||||
|
}
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const [key, value] of Object.entries(node as Record<string, unknown>)) {
|
||||||
|
out[key] = mapExportsForDist(value, key === 'types' || isTypesCondition);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureFilesIncludesDist(pkgJson: Record<string, unknown>): void {
|
||||||
|
const files = pkgJson['files'];
|
||||||
|
if (Array.isArray(files)) {
|
||||||
|
if (!files.includes('dist')) files.push('dist');
|
||||||
|
} else {
|
||||||
|
pkgJson['files'] = ['dist'];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
main().catch((err) => {
|
main().catch((err) => {
|
||||||
console.error(err);
|
console.error(err);
|
||||||
process.exit(1);
|
process.exit(1);
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ PACKAGES=(
|
|||||||
transport-webrtc
|
transport-webrtc
|
||||||
server
|
server
|
||||||
inbox-server
|
inbox-server
|
||||||
|
vault
|
||||||
inbox
|
inbox
|
||||||
transfer
|
transfer
|
||||||
files
|
files
|
||||||
|
|||||||
@@ -18,6 +18,16 @@ import { readdirSync, statSync, existsSync } from 'fs';
|
|||||||
import { join } from 'path';
|
import { join } from 'path';
|
||||||
import { $ } from 'bun';
|
import { $ } from 'bun';
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The pinned compiler, not whatever `bunx` resolves today.
|
||||||
|
*
|
||||||
|
* `bunx tsc` fetches the newest release on every run, so this gate could go
|
||||||
|
* red — or, worse, quietly stop catching things — because TypeScript shipped,
|
||||||
|
* not because the repo changed. The version is pinned in devDependencies and
|
||||||
|
* resolved from node_modules here.
|
||||||
|
*/
|
||||||
|
const TSC = join(import.meta.dir, '..', 'node_modules', '.bin', 'tsc');
|
||||||
const ROOT = join(import.meta.dir, '..');
|
const ROOT = join(import.meta.dir, '..');
|
||||||
const PACKAGES_DIR = join(ROOT, 'packages');
|
const PACKAGES_DIR = join(ROOT, 'packages');
|
||||||
|
|
||||||
@@ -38,7 +48,7 @@ const failed: { pkg: string; out: string }[] = [];
|
|||||||
|
|
||||||
for (const pkg of packages) {
|
for (const pkg of packages) {
|
||||||
const dir = join(PACKAGES_DIR, pkg);
|
const dir = join(PACKAGES_DIR, pkg);
|
||||||
const proc = Bun.spawnSync(['bunx', 'tsc', '--noEmit', '-p', 'tsconfig.json'], {
|
const proc = Bun.spawnSync([TSC, '--noEmit', '-p', 'tsconfig.json'], {
|
||||||
cwd: dir,
|
cwd: dir,
|
||||||
stdout: 'pipe',
|
stdout: 'pipe',
|
||||||
stderr: 'pipe',
|
stderr: 'pipe',
|
||||||
@@ -73,7 +83,7 @@ if (filter.size === 0) {
|
|||||||
console.log('Consumer-strict smoke (lib: DOM, exactOptional, paths→workspace) ...');
|
console.log('Consumer-strict smoke (lib: DOM, exactOptional, paths→workspace) ...');
|
||||||
const consumerDir = join(ROOT, 'tests', 'consumer-strict');
|
const consumerDir = join(ROOT, 'tests', 'consumer-strict');
|
||||||
if (existsSync(join(consumerDir, 'tsconfig.json'))) {
|
if (existsSync(join(consumerDir, 'tsconfig.json'))) {
|
||||||
const proc = Bun.spawnSync(['bunx', 'tsc', '--noEmit', '-p', 'tsconfig.json'], {
|
const proc = Bun.spawnSync([TSC, '--noEmit', '-p', 'tsconfig.json'], {
|
||||||
cwd: consumerDir,
|
cwd: consumerDir,
|
||||||
stdout: 'pipe',
|
stdout: 'pipe',
|
||||||
stderr: 'pipe',
|
stderr: 'pipe',
|
||||||
|
|||||||
@@ -14,7 +14,6 @@
|
|||||||
"noEmit": true,
|
"noEmit": true,
|
||||||
"types": ["bun-types"],
|
"types": ["bun-types"],
|
||||||
"ignoreDeprecations": "6.0",
|
"ignoreDeprecations": "6.0",
|
||||||
"baseUrl": ".",
|
|
||||||
"paths": {
|
"paths": {
|
||||||
"@shade/core": ["../../packages/shade-core/src/index.ts"],
|
"@shade/core": ["../../packages/shade-core/src/index.ts"],
|
||||||
"@shade/proto": ["../../packages/shade-proto/src/index.ts"],
|
"@shade/proto": ["../../packages/shade-proto/src/index.ts"],
|
||||||
|
|||||||
Reference in New Issue
Block a user