release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled

V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.

Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
  Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
  bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
  shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
  non-realtime stack.

Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
  version 4.0.0 on /health.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-03 18:35:35 +02:00
parent 8b055912b7
commit e6fdf31b49
298 changed files with 37909 additions and 256 deletions

View File

@@ -0,0 +1,137 @@
# Shade Android — Roadmap & Parity Status
This document tracks the M-Cross milestones from `docs/V3.5.md` and the
status of every cross-platform parity sjekkpunkt. The Kotlin port must be
**byte-for-byte compatible** with the TypeScript implementation; this is
verified continuously by `test-vectors/*.json` consumed by both runners.
> **No "production" label** is allowed on Android until M-Cross 2 is green
> (ratchet + wire 0x02 + storage encryption) and M-Cross 3 is green
> (streams 0x11). See `docs/V3.5.md` §Akseptansekriterier.
## Milestones
### M-Cross 1 — Scaffold ✅
Foundation primitives. All passing in CI.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| 1. KDF chain (root + chain ratchet) | `kdf-chain.json` | ✅ | ✅ |
| 2. HKDF labels | `hkdf.json` | ✅ | ✅ |
| 3. X3DH initial root key (3 + 4 DH outputs) | `x3dh.json` | ✅ | ✅ |
| 5. Fingerprint (60-digit safety number) | `fingerprint.json` | ✅ | ✅ |
### M-Cross 2 — Ratchet & Wire 0x02 ✅
Full ratchet step + binary envelope encoding for both message types.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| 4. Ratchet step (encrypt deterministic) | `ratchet-step.json` | ✅ | ✅ |
| 4. Ratchet step (decrypt roundtrip) | `ratchet-step.json` | ✅ | ✅ |
| 6. Wire 0x02 RatchetMessage | `wire-format.json` | ✅ | ✅ |
| 6. Wire 0x02 PreKeyMessage (with OTPK) | `wire-format.json` | ✅ | ✅ |
| 6. Wire 0x02 PreKeyMessage (no OTPK, 0xFFFFFFFF marker) | `wire-format.json` | ✅ | ✅ |
The ratchet-step vector exercises every layer that contributes to a
ratchet message's wire bytes: `kdfRootKey``kdfChainKey` → 40-byte header
AAD → AES-256-GCM with deterministic nonce. Both implementations recompute
each layer and compare against the recorded hex. The decrypt half feeds
the recorded ciphertext back through `aesGcmDecrypt(messageKey, nonce, aad)`
and checks the plaintext recovers — proving the AEAD agrees in both
directions.
### M-Cross 3 — Streams 0x11 ✅
Multi-lane chunk encryption (`@shade/streams`) ported. KDF labels with
embedded NULs match TS byte-for-byte; deterministic
`(laneId, seq)`-derived nonces and the 29-byte chunk AAD agree across
runners; wire 0x11 encode/decode is roundtrip-verified.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| `deriveStreamKey` (HKDF, info `shade-stream/v1\0master`) | `streams.json` | ✅ | ✅ |
| `deriveLaneKey` (HKDF, info `shade-stream/v1\0lane\0` ‖ u32_be laneId) — incl. laneId 0xFFFFFFFF | `streams.json` | ✅ | ✅ |
| `buildChunkNonce(laneId, seq)` — incl. seq = 2^64 - 2 | `streams.json` | ✅ | ✅ |
| `buildChunkAad(streamId, laneId, seq, isLast)` | `streams.json` | ✅ | ✅ |
| Chunk AES-256-GCM encrypt + decrypt (deterministic nonce + AAD) | `streams.json` | ✅ | ✅ |
| Wire 0x11 envelope encode + decode + type-tag inspector | `streams.json` | ✅ | ✅ |
Sequence numbers are unsigned u64 on the wire; the Kotlin port accepts
them as `Long` for the bit pattern (negative-signed-long for values past
2^63 - 1) — this matches the JVM `ByteBuffer.putLong` behavior and the
`java.lang.Long.parseUnsignedLong` JSON-decoder used in tests.
Pending end-to-end interop test (TS server → Kotlin client over an actual
socket) — not gated by vectors but recommended before flipping the
"production" label.
### M-Cross 4 — Backup, Group, Storage HKDF ✅ (cryptographic layer)
The cryptographic primitives that Kotlin needs to share with TS are now
covered. The remaining work is the high-level glue (BackupBlob JSON
schema, full SenderKey/GroupSession state-tracking, Android-Keystore
storage adapter, scrypt password-KDF) — all per-platform plumbing that
doesn't gate vector parity.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| 7. Backup v1 HKDF (`info="ShadeBackupKey"`) | `backup.json` | ✅ | ✅ |
| 7. Backup v1 AES-GCM roundtrip (no AAD) | `backup.json` | ✅ | ✅ |
| Group sender header AAD (u16/u16/u32 length prefixes) | `group.json` | ✅ | ✅ |
| Group sender-key step: `kdfChainKey` + AES-GCM + Ed25519 sign(aad ‖ ct) | `group.json` | ✅ | ✅ |
| Storage HKDF: `storageKey` (`info="shade-storage-v1"`) | `storage-hkdf.json` | ✅ | ✅ |
| Storage HKDF: `fieldKey` (`info="shade-field-v1:{table}:{column}"`) | `storage-hkdf.json` | ✅ | ✅ |
| Storage HKDF: `rowNonce` (`info="shade-row-nonce-v1:{table}:{pk}"`) | `storage-hkdf.json` | ✅ | ✅ |
Pending sub-tasks (don't gate vector parity):
- **scrypt master-key derivation**: `test-vectors/storage-encryption.json`
pins `scrypt(N=1024, r=8, p=1, dkLen=32)` for unit-test config; Tink
doesn't ship scrypt. Add Bouncy Castle (`org.bouncycastle:bcprov-jdk18on`)
to the Kotlin module, wrap as `CryptoProvider.scrypt(...)`, then a follow-up
vector consumes the full storage-encryption.json end to end.
- **argon2id**: Both backup.ts and the threat-model docs flag HKDF as a
placeholder for a real password KDF. When `argon2id` is added to
`CryptoProvider`, both ports swap together and the backup vector gets
re-pinned.
- **Android KeystoreStorage adapter**: lives in a sibling Android Library
Gradle module that depends on this JVM module. Binds Tink to the Android
Keystore + EncryptedSharedPreferences.
## Build & Test
This module compiles as a **pure-JVM** Kotlin library (`kotlin("jvm")`)
so the parity gate can run without an Android SDK installation in CI.
The protocol code uses `tink:1.15.0` (JVM JAR), `java.nio.ByteBuffer`,
and `javax.crypto` — no `android.*` imports.
The Android-specific storage adapter (KeystoreStorage,
EncryptedSharedPreferences) will land as a sibling Gradle module
(`shade-android-keystore`) in M-Cross 4 and depend on this one.
```bash
# From repo root
cd android
./gradlew :shade-android:test
```
Requires JDK 17. The Gradle wrapper downloads Gradle 8.10.2 on first run.
## Compatibility contract
The Kotlin implementation must produce byte-identical output to the TS
reference for:
- KDF chain derivations (root key ratchet, chain key ratchet)
- X3DH shared secrets (3- and 4-DH variants)
- Ratchet message keys + AES-GCM ciphertext (given the same key/plaintext/AAD/nonce)
- Header AAD encoding (40 bytes: `dhPublicKey(32) || u32_be(prevCounter) || u32_be(counter)`)
- Fingerprints (12 × 5-digit groups)
- Binary wire format 0x02 (RatchetMessage + PreKeyMessage)
- Binary wire format 0x11 (StreamChunk) — M-Cross 3
- Storage encryption KDF chain — M-Cross 4
Each is covered by a vector file in `/test-vectors/`. Adding a new
sjekkpunkt: see `docs/cross-platform.md`.