release(v4.0.0): Shade GA — V3.x consolidation + audit prep
Some checks failed
Test / test (push) Has been cancelled
Cross-platform vectors / TypeScript vectors (bun) (push) Has been cancelled
Cross-platform vectors / Kotlin vectors (gradle) (push) Has been cancelled
Docker build and publish / docker (push) Has been cancelled
Publish / publish (push) Has been cancelled

V3.1 → V3.12 consolidated and tagged for the first GA release. Wire
format unchanged from 0.4.x — 4.0 peers interoperate with 0.4.x peers
byte-for-byte. The version bump is semantic: audit-cycle complete,
opt-in surface fully exposed, threat model refreshed for every new
surface.

Highlights:
- All 24 @shade/* packages bumped to 4.0.0 in lockstep.
- CHANGELOG 4.0.0 section is the canonical manifest of what landed.
- THREAT-MODEL extended (§10 fingerprint gates, §11 WebRTC P2P, §12
  Web-Worker boundary) + residual-risks table refreshed.
- OpenAPI now covers all 27 routes: prekey, transfer, KT, inbox,
  bridge, observer, /metrics, /healthz, /ready.
- MIGRATION 0.3.x → 4.0 documented + smoke-tested against
  shade migrate-storage on a real SQLite DB.
- docs/audit/REVIEW-BUNDLE.md + SCOPE.md ready for external reviewer.
- scripts/soak.ts harness for the GA-stable 2-week soak window.
- All V*.md plans archived under docs/archive/ with Status: Done.
- Voice/Video carved out into V5.0; 4.0 audit focuses on the frozen
  non-realtime stack.

Tests: TS 1000/1000 + Kotlin 11/11 cross-platform vectors green.
Docker: gt.zyon.no/stian/shade-prekey:4.0.0 builds and reports
  version 4.0.0 on /health.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-03 18:35:35 +02:00
parent 8b055912b7
commit e6fdf31b49
298 changed files with 37909 additions and 256 deletions

12
android/.gitignore vendored Normal file
View File

@@ -0,0 +1,12 @@
# Gradle
.gradle/
build/
!gradle/wrapper/gradle-wrapper.jar
# IntelliJ / Android Studio
.idea/
*.iml
local.properties
# Captured logs
*.log

3
android/build.gradle.kts Normal file
View File

@@ -0,0 +1,3 @@
plugins {
kotlin("jvm") version "2.0.20" apply false
}

View File

@@ -0,0 +1,4 @@
org.gradle.jvmargs=-Xmx2g -Dfile.encoding=UTF-8
org.gradle.parallel=true
org.gradle.caching=true
kotlin.code.style=official

Binary file not shown.

View File

@@ -0,0 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.10.2-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

252
android/gradlew vendored Executable file
View File

@@ -0,0 +1,252 @@
#!/bin/sh
#
# Copyright © 2015-2021 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s
' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-classpath "$CLASSPATH" \
org.gradle.wrapper.GradleWrapperMain \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"

94
android/gradlew.bat vendored Normal file
View File

@@ -0,0 +1,94 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:execute
@rem Setup the command line
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

View File

@@ -0,0 +1,19 @@
rootProject.name = "shade-kotlin"
pluginManagement {
repositories {
gradlePluginPortal()
mavenCentral()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
mavenCentral()
google()
}
}
include(":shade-android")
project(":shade-android").projectDir = file("shade-android")

View File

@@ -4,7 +4,15 @@ Kotlin implementation of the Shade E2EE protocol for Android apps. Byte-for-byte
## Status
**Milestone M-Cross 1 — initial scaffold.** The protocol implementation is being ported. Cross-platform test vectors in `test-vectors/` verify that Kotlin and TypeScript produce identical output for every step (identity gen → HKDF X3DH → ratchet → fingerprint → wire format).
**M-Cross 1 ✅** — keys + HKDF + X3DH + fingerprint.
**M-Cross 2 ✅** — full ratchet step (encrypt + decrypt roundtrip) + wire 0x02 (RatchetMessage and PreKeyMessage with/without OTPK).
**M-Cross 3 ✅** — streams 0x11 (KDF labels with embedded NULs, deterministic chunk nonce/AAD, wire 0x11 encode/decode).
**M-Cross 4 ✅** — backup-key HKDF + AES-GCM, group sender-key step (`kdfChainKey` + Ed25519 sign over `aad ‖ ct`), storage HKDF (storageKey/fieldKey/rowNonce). Pending: scrypt master-key, argon2id swap, Android KeystoreStorage (sibling module).
Cross-platform test vectors in `/test-vectors/` are loaded by both the TS
and Kotlin test suites; any byte-divergence fails CI within 60 s. See
`ROADMAP-ANDROID.md` for the parity-checkpoint matrix and
`/docs/cross-platform.md` for how to add a new vector.
## Usage (target API)
@@ -40,13 +48,18 @@ Backed by Google Tink:
## Building
Requires Android SDK 35 and JDK 17.
Requires JDK 17. The module compiles as a pure-JVM Kotlin library so the
parity gate runs without an Android SDK install. The Android-specific
storage adapter (Keystore + EncryptedSharedPreferences) will land as a
sibling Gradle module in M-Cross 4.
```bash
./gradlew :shade-android:assembleDebug
cd android
./gradlew :shade-android:test
```
The Gradle wrapper downloads Gradle 8.10.2 on first run.
## Compatibility
The Kotlin implementation must produce byte-identical output to `@shade/core` for:

View File

@@ -0,0 +1,137 @@
# Shade Android — Roadmap & Parity Status
This document tracks the M-Cross milestones from `docs/V3.5.md` and the
status of every cross-platform parity sjekkpunkt. The Kotlin port must be
**byte-for-byte compatible** with the TypeScript implementation; this is
verified continuously by `test-vectors/*.json` consumed by both runners.
> **No "production" label** is allowed on Android until M-Cross 2 is green
> (ratchet + wire 0x02 + storage encryption) and M-Cross 3 is green
> (streams 0x11). See `docs/V3.5.md` §Akseptansekriterier.
## Milestones
### M-Cross 1 — Scaffold ✅
Foundation primitives. All passing in CI.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| 1. KDF chain (root + chain ratchet) | `kdf-chain.json` | ✅ | ✅ |
| 2. HKDF labels | `hkdf.json` | ✅ | ✅ |
| 3. X3DH initial root key (3 + 4 DH outputs) | `x3dh.json` | ✅ | ✅ |
| 5. Fingerprint (60-digit safety number) | `fingerprint.json` | ✅ | ✅ |
### M-Cross 2 — Ratchet & Wire 0x02 ✅
Full ratchet step + binary envelope encoding for both message types.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| 4. Ratchet step (encrypt deterministic) | `ratchet-step.json` | ✅ | ✅ |
| 4. Ratchet step (decrypt roundtrip) | `ratchet-step.json` | ✅ | ✅ |
| 6. Wire 0x02 RatchetMessage | `wire-format.json` | ✅ | ✅ |
| 6. Wire 0x02 PreKeyMessage (with OTPK) | `wire-format.json` | ✅ | ✅ |
| 6. Wire 0x02 PreKeyMessage (no OTPK, 0xFFFFFFFF marker) | `wire-format.json` | ✅ | ✅ |
The ratchet-step vector exercises every layer that contributes to a
ratchet message's wire bytes: `kdfRootKey``kdfChainKey` → 40-byte header
AAD → AES-256-GCM with deterministic nonce. Both implementations recompute
each layer and compare against the recorded hex. The decrypt half feeds
the recorded ciphertext back through `aesGcmDecrypt(messageKey, nonce, aad)`
and checks the plaintext recovers — proving the AEAD agrees in both
directions.
### M-Cross 3 — Streams 0x11 ✅
Multi-lane chunk encryption (`@shade/streams`) ported. KDF labels with
embedded NULs match TS byte-for-byte; deterministic
`(laneId, seq)`-derived nonces and the 29-byte chunk AAD agree across
runners; wire 0x11 encode/decode is roundtrip-verified.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| `deriveStreamKey` (HKDF, info `shade-stream/v1\0master`) | `streams.json` | ✅ | ✅ |
| `deriveLaneKey` (HKDF, info `shade-stream/v1\0lane\0` ‖ u32_be laneId) — incl. laneId 0xFFFFFFFF | `streams.json` | ✅ | ✅ |
| `buildChunkNonce(laneId, seq)` — incl. seq = 2^64 - 2 | `streams.json` | ✅ | ✅ |
| `buildChunkAad(streamId, laneId, seq, isLast)` | `streams.json` | ✅ | ✅ |
| Chunk AES-256-GCM encrypt + decrypt (deterministic nonce + AAD) | `streams.json` | ✅ | ✅ |
| Wire 0x11 envelope encode + decode + type-tag inspector | `streams.json` | ✅ | ✅ |
Sequence numbers are unsigned u64 on the wire; the Kotlin port accepts
them as `Long` for the bit pattern (negative-signed-long for values past
2^63 - 1) — this matches the JVM `ByteBuffer.putLong` behavior and the
`java.lang.Long.parseUnsignedLong` JSON-decoder used in tests.
Pending end-to-end interop test (TS server → Kotlin client over an actual
socket) — not gated by vectors but recommended before flipping the
"production" label.
### M-Cross 4 — Backup, Group, Storage HKDF ✅ (cryptographic layer)
The cryptographic primitives that Kotlin needs to share with TS are now
covered. The remaining work is the high-level glue (BackupBlob JSON
schema, full SenderKey/GroupSession state-tracking, Android-Keystore
storage adapter, scrypt password-KDF) — all per-platform plumbing that
doesn't gate vector parity.
| Sjekkpunkt | Vector | TS test | Kotlin test |
|---|---|---|---|
| 7. Backup v1 HKDF (`info="ShadeBackupKey"`) | `backup.json` | ✅ | ✅ |
| 7. Backup v1 AES-GCM roundtrip (no AAD) | `backup.json` | ✅ | ✅ |
| Group sender header AAD (u16/u16/u32 length prefixes) | `group.json` | ✅ | ✅ |
| Group sender-key step: `kdfChainKey` + AES-GCM + Ed25519 sign(aad ‖ ct) | `group.json` | ✅ | ✅ |
| Storage HKDF: `storageKey` (`info="shade-storage-v1"`) | `storage-hkdf.json` | ✅ | ✅ |
| Storage HKDF: `fieldKey` (`info="shade-field-v1:{table}:{column}"`) | `storage-hkdf.json` | ✅ | ✅ |
| Storage HKDF: `rowNonce` (`info="shade-row-nonce-v1:{table}:{pk}"`) | `storage-hkdf.json` | ✅ | ✅ |
Pending sub-tasks (don't gate vector parity):
- **scrypt master-key derivation**: `test-vectors/storage-encryption.json`
pins `scrypt(N=1024, r=8, p=1, dkLen=32)` for unit-test config; Tink
doesn't ship scrypt. Add Bouncy Castle (`org.bouncycastle:bcprov-jdk18on`)
to the Kotlin module, wrap as `CryptoProvider.scrypt(...)`, then a follow-up
vector consumes the full storage-encryption.json end to end.
- **argon2id**: Both backup.ts and the threat-model docs flag HKDF as a
placeholder for a real password KDF. When `argon2id` is added to
`CryptoProvider`, both ports swap together and the backup vector gets
re-pinned.
- **Android KeystoreStorage adapter**: lives in a sibling Android Library
Gradle module that depends on this JVM module. Binds Tink to the Android
Keystore + EncryptedSharedPreferences.
## Build & Test
This module compiles as a **pure-JVM** Kotlin library (`kotlin("jvm")`)
so the parity gate can run without an Android SDK installation in CI.
The protocol code uses `tink:1.15.0` (JVM JAR), `java.nio.ByteBuffer`,
and `javax.crypto` — no `android.*` imports.
The Android-specific storage adapter (KeystoreStorage,
EncryptedSharedPreferences) will land as a sibling Gradle module
(`shade-android-keystore`) in M-Cross 4 and depend on this one.
```bash
# From repo root
cd android
./gradlew :shade-android:test
```
Requires JDK 17. The Gradle wrapper downloads Gradle 8.10.2 on first run.
## Compatibility contract
The Kotlin implementation must produce byte-identical output to the TS
reference for:
- KDF chain derivations (root key ratchet, chain key ratchet)
- X3DH shared secrets (3- and 4-DH variants)
- Ratchet message keys + AES-GCM ciphertext (given the same key/plaintext/AAD/nonce)
- Header AAD encoding (40 bytes: `dhPublicKey(32) || u32_be(prevCounter) || u32_be(counter)`)
- Fingerprints (12 × 5-digit groups)
- Binary wire format 0x02 (RatchetMessage + PreKeyMessage)
- Binary wire format 0x11 (StreamChunk) — M-Cross 3
- Storage encryption KDF chain — M-Cross 4
Each is covered by a vector file in `/test-vectors/`. Adding a new
sjekkpunkt: see `docs/cross-platform.md`.

View File

@@ -1,50 +1,51 @@
plugins {
id("com.android.library")
kotlin("android")
kotlin("jvm")
`java-library`
}
android {
namespace = "no.zyon.shade"
compileSdk = 35
// V3.5 — Cross-platform parity gate.
//
// This module compiles as a pure-JVM Kotlin library so CI can run the
// cross-platform vector tests without an Android SDK. The protocol code
// is JVM-safe (no `android.*` imports); only Tink + java.* are used.
//
// When KeystoreStorage and EncryptedSharedPreferences-backed adapters land
// (M-Cross 4 + V3.5 §Storage), they will live in a sibling Android Library
// module that depends on this one.
defaultConfig {
minSdk = 26
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
}
java {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
buildFeatures {
buildConfig = true
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
kotlinOptions {
jvmTarget = "17"
kotlin {
compilerOptions {
jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17)
}
}
dependencies {
// Google Tink for X25519, Ed25519, AES-GCM, HMAC, HKDF
implementation("com.google.crypto.tink:tink-android:1.15.0")
// Google Tink for X25519, Ed25519, AES-GCM, HMAC, HKDF (JVM build).
// The same `subtle.*` API as `tink-android` so the source compiles unchanged.
implementation("com.google.crypto.tink:tink:1.15.0")
// Android Keystore + EncryptedSharedPreferences
implementation("androidx.security:security-crypto:1.1.0-alpha06")
// JSON serialization for session state
// JSON serialization (session state + test-vector loader on JVM).
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.7.3")
// Coroutines for async interop
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.9.0")
// Coroutines (StorageProvider uses `suspend` functions).
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.9.0")
// SQLite for session storage (optional; can also use EncryptedSharedPreferences only)
implementation("androidx.sqlite:sqlite:2.4.0")
// OkHttp for transport
implementation("com.squareup.okhttp3:okhttp:4.12.0")
// org.json — bundled with Android but not present on the JVM classpath.
implementation("org.json:json:20240303")
testImplementation("junit:junit:4.13.2")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.9.0")
}
tasks.withType<Test>().configureEach {
useJUnit()
testLogging {
events("passed", "failed", "skipped")
showStandardStreams = false
}
}

View File

@@ -0,0 +1,36 @@
package no.zyon.shade.backup
import no.zyon.shade.crypto.CryptoProvider
/**
* Backup format v1 — passphrase-derived AES-256-GCM blob.
* Mirror @shade/sdk/backup.ts.
*
* backupKey = HKDF(passphrase_utf8, salt_random_32, info="ShadeBackupKey", 32)
* blob = AES-256-GCM(backupKey, plaintext, no AAD)
*
* The stored on-disk form is `{ version, salt(b64), nonce(b64), ciphertext(b64) }`.
* This file ships only the cryptographic primitives — payload schema and JSON
* serialization live alongside the high-level SDK and don't need a Kotlin port
* for vector parity (each platform builds the BackupBlob in its native idiom).
*
* NOTE: HKDF is NOT a proper password KDF. The TS SDK acknowledges this and
* warns users to choose a high-entropy passphrase. When `argon2id` lands in
* `CryptoProvider`, both ports swap together. Until then, byte-parity for the
* HKDF + AEAD layer is what V3.5 §sjekkpunkt 8 gates.
*/
private val BACKUP_INFO: ByteArray = "ShadeBackupKey".toByteArray(Charsets.UTF_8)
const val BACKUP_KEY_BYTES = 32
const val BACKUP_VERSION = 1
fun deriveBackupKey(crypto: CryptoProvider, passphrase: String, salt: ByteArray): ByteArray {
require(passphrase.length >= 12) { "Passphrase must be at least 12 characters" }
require(salt.size >= 16) { "salt must be at least 16 bytes" }
return crypto.hkdf(
passphrase.toByteArray(Charsets.UTF_8),
salt,
BACKUP_INFO,
BACKUP_KEY_BYTES,
)
}

View File

@@ -0,0 +1,77 @@
package no.zyon.shade.group
import no.zyon.shade.crypto.CryptoProvider
import no.zyon.shade.protocol.kdfChainKey
import java.nio.ByteBuffer
/**
* Group sender-keys (Sesame). Mirror @shade/core/sender-keys.ts.
*
* Each sender maintains a chain key that ratchets forward with `kdfChainKey`
* — same primitive the Double Ratchet uses for its symmetric chain. Per-message
* AEAD AAD binds (groupId, senderAddress, iteration) so a captured ciphertext
* cannot be replayed under a different sender or group:
*
* aad = u16_be(groupIdLen) || groupId || u16_be(senderAddrLen) || senderAddr || u32_be(iteration)
*
* Each ciphertext is signed by the sender's Ed25519 key over `aad || ciphertext`,
* which is what receivers verify before advancing their chain.
*/
data class SenderKeyMessage(
val senderAddress: String,
val iteration: Int,
val ciphertext: ByteArray,
val nonce: ByteArray,
val signature: ByteArray,
)
fun encodeSenderHeader(groupId: String, senderAddress: String, iteration: Int): ByteArray {
val gBytes = groupId.toByteArray(Charsets.UTF_8)
val sBytes = senderAddress.toByteArray(Charsets.UTF_8)
require(gBytes.size <= 0xFFFF) { "groupId too long (>65535 UTF-8 bytes)" }
require(sBytes.size <= 0xFFFF) { "senderAddress too long (>65535 UTF-8 bytes)" }
val out = ByteArray(2 + gBytes.size + 2 + sBytes.size + 4)
val buf = ByteBuffer.wrap(out)
buf.putShort(gBytes.size.toShort())
buf.put(gBytes)
buf.putShort(sBytes.size.toShort())
buf.put(sBytes)
buf.putInt(iteration)
return out
}
/**
* Compute (newChainKey, messageKey, aad) for the next group message.
* Pure function; caller is responsible for state advancement and the AEAD/sign
* steps (which need access to the signing private key not exposed here).
*/
data class SenderStepResult(
val newChainKey: ByteArray,
val messageKey: ByteArray,
val aad: ByteArray,
)
fun senderKeyStep(
crypto: CryptoProvider,
chainKey: ByteArray,
groupId: String,
senderAddress: String,
iteration: Int,
): SenderStepResult {
val r = kdfChainKey(crypto, chainKey)
val aad = encodeSenderHeader(groupId, senderAddress, iteration)
return SenderStepResult(newChainKey = r.newChainKey, messageKey = r.messageKey, aad = aad)
}
/**
* Concatenate `aad || ciphertext` — the byte string the sender signs and the
* receiver verifies. Exposed as a helper so vector parity can pin both sides.
*/
fun senderSignedBytes(aad: ByteArray, ciphertext: ByteArray): ByteArray {
val out = ByteArray(aad.size + ciphertext.size)
aad.copyInto(out, 0)
ciphertext.copyInto(out, aad.size)
return out
}

View File

@@ -0,0 +1,145 @@
package no.zyon.shade.serialization
import no.zyon.shade.streams.StreamConstants
import java.nio.ByteBuffer
/**
* Wire-decoded stream-chunk envelope (type 0x11).
*
* Mirror @shade/proto/wire.ts `StreamChunkWire`. The nonce is deterministic
* (derived from `(laneId, seq)` on both sides) but is also serialized over
* the wire for self-description and validated by the receiver.
*
* `seq` is unsigned-u64 on the wire; on the JVM we keep it as Long. The
* encode/decode helpers operate on the raw 8-byte big-endian representation,
* so values past Long.MAX_VALUE roundtrip via `Long.toULong()`.
*/
data class StreamChunkWire(
val streamId: ByteArray,
val laneId: Long,
val seq: Long,
val isLast: Boolean,
val nonce: ByteArray,
val aad: ByteArray,
val ciphertext: ByteArray,
) {
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (other !is StreamChunkWire) return false
return streamId.contentEquals(other.streamId) &&
laneId == other.laneId &&
seq == other.seq &&
isLast == other.isLast &&
nonce.contentEquals(other.nonce) &&
aad.contentEquals(other.aad) &&
ciphertext.contentEquals(other.ciphertext)
}
override fun hashCode(): Int {
var result = streamId.contentHashCode()
result = 31 * result + laneId.hashCode()
result = 31 * result + seq.hashCode()
result = 31 * result + isLast.hashCode()
result = 31 * result + nonce.contentHashCode()
result = 31 * result + aad.contentHashCode()
result = 31 * result + ciphertext.contentHashCode()
return result
}
}
/** Stream-chunk wire codec. Mirror @shade/proto/wire.ts `encodeStreamChunk`/`decodeStreamChunk`. */
object StreamChunkWireFormat {
private const val VERSION: Byte = 0x02
const val TYPE_STREAM_CHUNK: Byte = 0x11
fun encodeStreamChunk(c: StreamChunkWire): ByteArray {
require(c.streamId.size == StreamConstants.STREAM_ID_BYTES) {
"streamId must be ${StreamConstants.STREAM_ID_BYTES} bytes"
}
require(c.nonce.size == StreamConstants.STREAM_NONCE_BYTES) {
"nonce must be ${StreamConstants.STREAM_NONCE_BYTES} bytes"
}
require(c.laneId in 0L..0xFFFFFFFFL) { "laneId out of u32 range: ${c.laneId}" }
// c.seq is unsigned-u64; negative signed longs encode as the high half
// of the u64 range. ByteBuffer.putLong writes the raw 8-byte pattern.
val headerSize =
1 + 1 +
StreamConstants.STREAM_ID_BYTES +
4 + 8 + 1 +
StreamConstants.STREAM_NONCE_BYTES +
4 + c.aad.size +
4
val out = ByteArray(headerSize + c.ciphertext.size)
val buf = ByteBuffer.wrap(out)
buf.put(VERSION)
buf.put(TYPE_STREAM_CHUNK)
buf.put(c.streamId)
buf.putInt(c.laneId.toInt())
buf.putLong(c.seq)
buf.put(if (c.isLast) 0x01.toByte() else 0x00.toByte())
buf.put(c.nonce)
buf.putInt(c.aad.size)
buf.put(c.aad)
buf.putInt(c.ciphertext.size)
buf.put(c.ciphertext)
return out
}
fun decodeStreamChunk(data: ByteArray): StreamChunkWire {
val minHeaderSize = 2 +
StreamConstants.STREAM_ID_BYTES +
4 + 8 + 1 +
StreamConstants.STREAM_NONCE_BYTES +
4 + 4
require(data.size >= minHeaderSize) {
"stream-chunk too short: ${data.size} < $minHeaderSize"
}
require(data[0] == VERSION) { "Unknown version: ${data[0]}" }
require(data[1] == TYPE_STREAM_CHUNK) { "Not a stream-chunk: type=${data[1]}" }
val buf = ByteBuffer.wrap(data)
buf.position(2)
val streamId = ByteArray(StreamConstants.STREAM_ID_BYTES)
buf.get(streamId)
val laneId = buf.int.toLong() and 0xFFFFFFFFL
val seq = buf.long
val isLast = buf.get() == 0x01.toByte()
val nonce = ByteArray(StreamConstants.STREAM_NONCE_BYTES)
buf.get(nonce)
val aadLen = buf.int
require(buf.position() + aadLen + 4 <= data.size) {
"stream-chunk truncated in aad/ctLen"
}
val aad = ByteArray(aadLen)
buf.get(aad)
val ctLen = buf.int
require(buf.position() + ctLen == data.size) {
"stream-chunk length mismatch: declared ${buf.position() + ctLen}, actual ${data.size}"
}
val ciphertext = ByteArray(ctLen)
buf.get(ciphertext)
return StreamChunkWire(streamId, laneId, seq, isLast, nonce, aad, ciphertext)
}
/** Inspect the type tag without full parsing. Mirror @shade/proto/wire.ts. */
enum class EnvelopeKind { PREKEY, RATCHET, STREAM_CHUNK, UNKNOWN }
fun inspectEnvelopeType(data: ByteArray): EnvelopeKind {
if (data.size < 2 || data[0] != VERSION) return EnvelopeKind.UNKNOWN
return when (data[1]) {
0x01.toByte() -> EnvelopeKind.PREKEY
0x02.toByte() -> EnvelopeKind.RATCHET
TYPE_STREAM_CHUNK -> EnvelopeKind.STREAM_CHUNK
else -> EnvelopeKind.UNKNOWN
}
}
}

View File

@@ -0,0 +1,48 @@
package no.zyon.shade.streams
import javax.crypto.Cipher
import javax.crypto.spec.GCMParameterSpec
import javax.crypto.spec.SecretKeySpec
/**
* AES-256-GCM with caller-supplied nonce. Mirror @shade/streams/aead.ts.
*
* Unlike `CryptoProvider.aesGcmEncrypt` (which generates a random nonce
* internally), streams require deterministic nonces derived from
* `(laneId, seq)`. Returns the ciphertext concatenated with the 16-byte
* authentication tag — same layout SubtleCrypto produces.
*/
const val AEAD_TAG_BYTES = 16
fun aesGcmEncryptWithNonce(
key: ByteArray,
nonce: ByteArray,
plaintext: ByteArray,
aad: ByteArray,
): ByteArray {
require(nonce.size == StreamConstants.STREAM_NONCE_BYTES) {
"AES-GCM nonce must be ${StreamConstants.STREAM_NONCE_BYTES} bytes"
}
val cipher = Cipher.getInstance("AES/GCM/NoPadding")
val spec = GCMParameterSpec(128, nonce)
cipher.init(Cipher.ENCRYPT_MODE, SecretKeySpec(key, "AES"), spec)
cipher.updateAAD(aad)
return cipher.doFinal(plaintext)
}
fun aesGcmDecryptWithNonce(
key: ByteArray,
nonce: ByteArray,
ciphertext: ByteArray,
aad: ByteArray,
): ByteArray {
require(nonce.size == StreamConstants.STREAM_NONCE_BYTES) {
"AES-GCM nonce must be ${StreamConstants.STREAM_NONCE_BYTES} bytes"
}
val cipher = Cipher.getInstance("AES/GCM/NoPadding")
val spec = GCMParameterSpec(128, nonce)
cipher.init(Cipher.DECRYPT_MODE, SecretKeySpec(key, "AES"), spec)
cipher.updateAAD(aad)
return cipher.doFinal(ciphertext)
}

View File

@@ -0,0 +1,50 @@
package no.zyon.shade.streams
import java.nio.ByteBuffer
/**
* Deterministic AEAD nonce + AAD construction for stream chunks.
* Mirror @shade/streams/nonce.ts.
*
* nonce[0..4] = u32_be(laneId)
* nonce[4..12] = u64_be(seq)
*
* aad = streamId(16) || u32_be(laneId) || u64_be(seq) || u8(isLast)
*
* `seq` is unsigned-u64 on the wire. Kotlin's `Long` is signed; we accept it
* for the bit pattern (same as TS `BigInt` would write), so values past
* `Long.MAX_VALUE` arrive here as negative signed longs. `ByteBuffer.putLong`
* writes the raw 8 bytes regardless of sign — that's what we want.
*
* Use `java.lang.Long.parseUnsignedLong("…")` to decode JSON strings
* representing u64 values larger than 2^63 - 1.
*/
fun buildChunkNonce(laneId: Long, seq: Long): ByteArray {
require(laneId in 0L..0xFFFFFFFFL) { "laneId must fit in u32: $laneId" }
val out = ByteArray(StreamConstants.STREAM_NONCE_BYTES)
val buf = ByteBuffer.wrap(out)
buf.putInt(laneId.toInt())
buf.putLong(seq)
return out
}
fun buildChunkAad(
streamId: ByteArray,
laneId: Long,
seq: Long,
isLast: Boolean,
): ByteArray {
require(streamId.size == StreamConstants.STREAM_ID_BYTES) {
"streamId must be ${StreamConstants.STREAM_ID_BYTES} bytes"
}
require(laneId in 0L..0xFFFFFFFFL) { "laneId must fit in u32: $laneId" }
val out = ByteArray(StreamConstants.STREAM_ID_BYTES + 4 + 8 + 1)
streamId.copyInto(out, 0)
val buf = ByteBuffer.wrap(out, StreamConstants.STREAM_ID_BYTES, 4 + 8 + 1)
buf.putInt(laneId.toInt())
buf.putLong(seq)
out[out.size - 1] = if (isLast) 0x01 else 0x00
return out
}

View File

@@ -1,18 +1,36 @@
package no.zyon.shade
import no.zyon.shade.backup.deriveBackupKey
import no.zyon.shade.crypto.TinkProvider
import no.zyon.shade.fingerprint.computeFingerprint
import no.zyon.shade.group.encodeSenderHeader
import no.zyon.shade.group.senderKeyStep
import no.zyon.shade.group.senderSignedBytes
import no.zyon.shade.protocol.deriveInitialRootKey
import no.zyon.shade.protocol.kdfChainKey
import no.zyon.shade.protocol.kdfRootKey
import no.zyon.shade.serialization.StreamChunkWire
import no.zyon.shade.serialization.StreamChunkWireFormat
import no.zyon.shade.serialization.WireFormat
import no.zyon.shade.streams.aesGcmDecryptWithNonce
import no.zyon.shade.streams.aesGcmEncryptWithNonce
import no.zyon.shade.streams.buildChunkAad
import no.zyon.shade.streams.buildChunkNonce
import no.zyon.shade.streams.deriveLaneKey
import no.zyon.shade.streams.deriveStreamKey
import no.zyon.shade.types.PreKeyMessage
import no.zyon.shade.types.RatchetMessage
import no.zyon.shade.types.ShadeEnvelope
import org.json.JSONArray
import org.json.JSONObject
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
import java.io.File
import org.json.JSONObject
import org.json.JSONArray
import java.nio.ByteBuffer
import javax.crypto.Cipher
import javax.crypto.spec.GCMParameterSpec
import javax.crypto.spec.SecretKeySpec
/**
* Cross-platform test vectors. MUST match the TypeScript implementation
@@ -25,6 +43,7 @@ class CrossPlatformVectorTest {
private val crypto = TinkProvider()
private val vectorsDir = File("../../test-vectors")
private val expectedVersion = 2
private fun fromHex(str: String): ByteArray {
val bytes = ByteArray(str.length / 2)
@@ -39,10 +58,40 @@ class CrossPlatformVectorTest {
return bytes.joinToString("") { "%02x".format(it) }
}
private data class VectorFile(val version: Int, val vectors: JSONArray)
private fun loadVectors(name: String): JSONArray {
val file = File(vectorsDir, name)
val content = file.readText()
return JSONObject(content).getJSONArray("vectors")
val obj = JSONObject(content)
val version = obj.getInt("version")
assertEquals("Unexpected vector schema version in $name", expectedVersion, version)
return obj.getJSONArray("vectors")
}
private fun encodeRatchetHeader(
dhPublicKey: ByteArray,
previousCounter: Int,
counter: Int,
): ByteArray {
val buf = ByteBuffer.allocate(40)
buf.put(dhPublicKey)
buf.putInt(previousCounter)
buf.putInt(counter)
return buf.array()
}
private fun aesGcmEncryptDeterministic(
key: ByteArray,
nonce: ByteArray,
plaintext: ByteArray,
aad: ByteArray,
): ByteArray {
val cipher = Cipher.getInstance("AES/GCM/NoPadding")
val spec = GCMParameterSpec(128, nonce)
cipher.init(Cipher.ENCRYPT_MODE, SecretKeySpec(key, "AES"), spec)
cipher.updateAAD(aad)
return cipher.doFinal(plaintext)
}
@Test
@@ -106,31 +155,347 @@ class CrossPlatformVectorTest {
}
@Test
fun wireFormatVectorsMatch() {
fun wireFormatRatchetVectorsMatch() {
val vectors = loadVectors("wire-format.json")
val v = vectors.getJSONObject(0)
val m = v.getJSONObject("message")
var found = false
for (i in 0 until vectors.length()) {
val v = vectors.getJSONObject(i)
if (v.optString("kind") != "ratchet") continue
found = true
val m = v.getJSONObject("message")
val msg = RatchetMessage(
dhPublicKey = fromHex(m.getString("dhPublicKey")),
previousCounter = m.getInt("previousCounter"),
counter = m.getInt("counter"),
ciphertext = fromHex(m.getString("ciphertext")),
nonce = fromHex(m.getString("nonce")),
)
val envelope = ShadeEnvelope(
type = ShadeEnvelope.EnvelopeType.RATCHET,
content = msg,
timestamp = 0,
senderAddress = "",
)
val encoded = WireFormat.encodeEnvelope(envelope)
assertEquals(v.getString("encoded"), hex(encoded))
val msg = RatchetMessage(
dhPublicKey = fromHex(m.getString("dhPublicKey")),
previousCounter = m.getInt("previousCounter"),
counter = m.getInt("counter"),
ciphertext = fromHex(m.getString("ciphertext")),
nonce = fromHex(m.getString("nonce")),
)
val envelope = ShadeEnvelope(
type = ShadeEnvelope.EnvelopeType.RATCHET,
content = msg,
timestamp = 0,
senderAddress = "",
)
val encoded = WireFormat.encodeEnvelope(envelope)
assertEquals(v.getString("encoded"), hex(encoded))
// Roundtrip decode
val decoded = WireFormat.decodeEnvelope(encoded)
assertEquals(ShadeEnvelope.EnvelopeType.RATCHET, decoded.type)
val rm = decoded.content as RatchetMessage
assertEquals(msg.counter, rm.counter)
val decoded = WireFormat.decodeEnvelope(encoded)
assertEquals(ShadeEnvelope.EnvelopeType.RATCHET, decoded.type)
val rm = decoded.content as RatchetMessage
assertEquals(msg.counter, rm.counter)
assertEquals(hex(msg.ciphertext), hex(rm.ciphertext))
}
assertTrue("No ratchet wire vectors found", found)
}
@Test
fun wireFormatPreKeyVectorsMatch() {
val vectors = loadVectors("wire-format.json")
var matched = 0
for (i in 0 until vectors.length()) {
val v = vectors.getJSONObject(i)
if (v.optString("kind") != "prekey") continue
matched++
val m = v.getJSONObject("message")
val inner = m.getJSONObject("inner")
val innerMsg = RatchetMessage(
dhPublicKey = fromHex(inner.getString("dhPublicKey")),
previousCounter = inner.getInt("previousCounter"),
counter = inner.getInt("counter"),
ciphertext = fromHex(inner.getString("ciphertext")),
nonce = fromHex(inner.getString("nonce")),
)
val preKeyId: Int? = if (m.isNull("preKeyId")) null else m.getInt("preKeyId")
val pre = PreKeyMessage(
registrationId = m.getInt("registrationId"),
preKeyId = preKeyId,
signedPreKeyId = m.getInt("signedPreKeyId"),
ephemeralKey = fromHex(m.getString("ephemeralKey")),
identityDHKey = fromHex(m.getString("identityDHKey")),
message = innerMsg,
)
val envelope = ShadeEnvelope(
type = ShadeEnvelope.EnvelopeType.PREKEY,
content = pre,
timestamp = 0,
senderAddress = "",
)
val encoded = WireFormat.encodeEnvelope(envelope)
assertEquals(v.getString("encoded"), hex(encoded))
val decoded = WireFormat.decodeEnvelope(encoded)
assertEquals(ShadeEnvelope.EnvelopeType.PREKEY, decoded.type)
val dm = decoded.content as PreKeyMessage
assertEquals(pre.registrationId, dm.registrationId)
assertEquals(pre.preKeyId, dm.preKeyId)
assertEquals(pre.signedPreKeyId, dm.signedPreKeyId)
assertEquals(hex(pre.ephemeralKey), hex(dm.ephemeralKey))
assertEquals(hex(innerMsg.ciphertext), hex(dm.message.ciphertext))
}
assertTrue("Expected at least 2 prekey vectors", matched >= 2)
}
private fun findVector(arr: JSONArray, prefix: String): JSONObject {
for (i in 0 until arr.length()) {
val o = arr.getJSONObject(i)
if (o.getString("description").startsWith(prefix)) return o
}
throw AssertionError("Vector with description prefix '$prefix' not found")
}
@Test
fun streamsVectorsMatch() {
val vectors = loadVectors("streams.json")
// 1. deriveStreamKey
val sk = findVector(vectors, "deriveStreamKey")
val streamSecret = fromHex(sk.getString("streamSecret"))
val streamId = fromHex(sk.getString("streamId"))
val streamKey = deriveStreamKey(crypto, streamSecret, streamId)
assertEquals(sk.getString("streamKey"), hex(streamKey))
// 2. deriveLaneKey
val lk = findVector(vectors, "deriveLaneKey")
val lkStreamKey = fromHex(lk.getString("streamKey"))
val lkStreamId = fromHex(lk.getString("streamId"))
val lanes = lk.getJSONArray("lanes")
for (i in 0 until lanes.length()) {
val lane = lanes.getJSONObject(i)
val laneId = lane.getLong("laneId")
val k = deriveLaneKey(crypto, lkStreamKey, lkStreamId, laneId)
assertEquals(lane.getString("laneKey"), hex(k))
}
// 3. buildChunkNonce
val nv = findVector(vectors, "buildChunkNonce")
val nonces = nv.getJSONArray("nonces")
for (i in 0 until nonces.length()) {
val n = nonces.getJSONObject(i)
val laneId = n.getLong("laneId")
val seq = java.lang.Long.parseUnsignedLong(n.getString("seq"))
val out = buildChunkNonce(laneId, seq)
assertEquals(n.getString("nonce"), hex(out))
}
// 4. buildChunkAad
val av = findVector(vectors, "buildChunkAad")
val avStreamId = fromHex(av.getString("streamId"))
val cases = av.getJSONArray("cases")
for (i in 0 until cases.length()) {
val c = cases.getJSONObject(i)
val laneId = c.getLong("laneId")
val seq = java.lang.Long.parseUnsignedLong(c.getString("seq"))
val isLast = c.getBoolean("isLast")
val out = buildChunkAad(avStreamId, laneId, seq, isLast)
assertEquals(c.getString("aad"), hex(out))
}
// 5. End-to-end chunk encrypt + decrypt
val ev = findVector(vectors, "End-to-end chunk encrypt")
val laneKey = fromHex(ev.getString("laneKey"))
val nonce = fromHex(ev.getString("nonce"))
val aad = fromHex(ev.getString("aad"))
val plaintext = fromHex(ev.getString("plaintext"))
val ct = aesGcmEncryptWithNonce(laneKey, nonce, plaintext, aad)
assertEquals(ev.getString("ciphertext"), hex(ct))
val pt = aesGcmDecryptWithNonce(laneKey, nonce, fromHex(ev.getString("ciphertext")), aad)
assertEquals(ev.getString("plaintext"), hex(pt))
// 6. Wire 0x11 envelope encode/decode
val wv = findVector(vectors, "Wire 0x11")
val wire = StreamChunkWire(
streamId = fromHex(wv.getString("streamId")),
laneId = wv.getLong("laneId"),
seq = java.lang.Long.parseUnsignedLong(wv.getString("seq")),
isLast = wv.getBoolean("isLast"),
nonce = fromHex(wv.getString("nonce")),
aad = fromHex(wv.getString("extraAad")),
ciphertext = fromHex(wv.getString("ciphertext")),
)
val encoded = StreamChunkWireFormat.encodeStreamChunk(wire)
assertEquals(wv.getString("encoded"), hex(encoded))
val decoded = StreamChunkWireFormat.decodeStreamChunk(encoded)
assertEquals(hex(wire.streamId), hex(decoded.streamId))
assertEquals(wire.laneId, decoded.laneId)
assertEquals(wire.seq, decoded.seq)
assertEquals(wire.isLast, decoded.isLast)
assertEquals(hex(wire.nonce), hex(decoded.nonce))
assertEquals(hex(wire.ciphertext), hex(decoded.ciphertext))
// 7. Envelope-type inspector
assertEquals(
StreamChunkWireFormat.EnvelopeKind.STREAM_CHUNK,
StreamChunkWireFormat.inspectEnvelopeType(encoded),
)
}
@Test
fun backupVectorsMatch() {
val vectors = loadVectors("backup.json")
val kv = findVector(vectors, "Backup v1: HKDF")
val backupKey = deriveBackupKey(crypto, kv.getString("passphrase"), fromHex(kv.getString("salt")))
assertEquals(kv.getString("backupKey"), hex(backupKey))
val ev = findVector(vectors, "Backup v1: AES-256-GCM")
val ct = aesGcmEncryptDeterministic(
fromHex(ev.getString("backupKey")),
fromHex(ev.getString("nonce")),
fromHex(ev.getString("plaintext")),
ByteArray(0),
)
assertEquals(ev.getString("ciphertext"), hex(ct))
val pt = crypto.aesGcmDecrypt(
fromHex(ev.getString("backupKey")),
fromHex(ev.getString("ciphertext")),
fromHex(ev.getString("nonce")),
null,
)
assertEquals(ev.getString("plaintext"), hex(pt))
}
@Test
fun groupSenderKeyVectorsMatch() {
val vectors = loadVectors("group.json")
// 1. Header AAD
val hv = findVector(vectors, "Sender header AAD")
val aad = encodeSenderHeader(
hv.getString("groupId"),
hv.getString("senderAddress"),
hv.getInt("iteration"),
)
assertEquals(hv.getString("aad"), hex(aad))
// 2. Sender-key step
val sv = findVector(vectors, "Sender-key step")
val step = senderKeyStep(
crypto,
fromHex(sv.getString("chainKey")),
sv.getString("groupId"),
sv.getString("senderAddress"),
sv.getInt("iteration"),
)
assertEquals(sv.getString("newChainKey"), hex(step.newChainKey))
assertEquals(sv.getString("messageKey"), hex(step.messageKey))
assertEquals(sv.getString("aad"), hex(step.aad))
val ct = aesGcmEncryptDeterministic(
step.messageKey,
fromHex(sv.getString("nonce")),
fromHex(sv.getString("plaintext")),
step.aad,
)
assertEquals(sv.getString("ciphertext"), hex(ct))
// 3. Ed25519 verify on the recorded signature
val signed = senderSignedBytes(step.aad, ct)
val ok = crypto.verify(
fromHex(sv.getString("signingPublicKey")),
signed,
fromHex(sv.getString("signature")),
)
assertTrue("Sender-key signature verification failed", ok)
// 4. Decrypt roundtrip
val pt = crypto.aesGcmDecrypt(
step.messageKey,
fromHex(sv.getString("ciphertext")),
fromHex(sv.getString("nonce")),
step.aad,
)
assertEquals(sv.getString("plaintext"), hex(pt))
}
@Test
fun storageHkdfVectorsMatch() {
val vectors = loadVectors("storage-hkdf.json")
val sv = findVector(vectors, "Storage HKDF: storageKey")
val storageKey = crypto.hkdf(
fromHex(sv.getString("masterKey")),
ByteArray(0),
"shade-storage-v1".toByteArray(Charsets.UTF_8),
32,
)
assertEquals(sv.getString("storageKey"), hex(storageKey))
val fv = findVector(vectors, "Storage HKDF: fieldKey")
val fStorageKey = fromHex(fv.getString("storageKey"))
val fields = fv.getJSONArray("fields")
for (i in 0 until fields.length()) {
val f = fields.getJSONObject(i)
val info = "shade-field-v1:${f.getString("table")}:${f.getString("column")}"
.toByteArray(Charsets.UTF_8)
val k = crypto.hkdf(fStorageKey, ByteArray(0), info, 32)
assertEquals(f.getString("fieldKey"), hex(k))
}
val nv = findVector(vectors, "Storage HKDF: rowNonce")
val rowKey = fromHex(nv.getString("rowKey"))
val nonces = nv.getJSONArray("nonces")
for (i in 0 until nonces.length()) {
val n = nonces.getJSONObject(i)
val info = "shade-row-nonce-v1:${n.getString("table")}:${n.getString("pk")}"
.toByteArray(Charsets.UTF_8)
val out = crypto.hkdf(rowKey, ByteArray(0), info, 12)
assertEquals(n.getString("nonce"), hex(out))
}
}
@Test
fun ratchetStepRoundtripMatches() {
val vectors = loadVectors("ratchet-step.json")
assertTrue("ratchet-step vectors expected", vectors.length() > 0)
for (i in 0 until vectors.length()) {
val v = vectors.getJSONObject(i)
val inputs = v.getJSONObject("inputs")
val derived = v.getJSONObject("derived")
val rootKey = fromHex(inputs.getString("rootKey"))
val dhSendPriv = fromHex(inputs.getString("dhSendPrivateKey"))
val dhSendPub = fromHex(inputs.getString("dhSendPublicKey"))
val dhRemotePub = fromHex(inputs.getString("dhRemotePublicKey"))
val plaintext = fromHex(inputs.getString("plaintext"))
val nonce = fromHex(inputs.getString("nonce"))
val previousCounter = inputs.getInt("previousCounter")
val counter = inputs.getInt("counter")
// 1. DH
val dhOutput = crypto.x25519(dhSendPriv, dhRemotePub)
assertEquals(derived.getString("dhOutput"), hex(dhOutput))
// 2. kdfRootKey
val root = kdfRootKey(crypto, rootKey, dhOutput)
assertEquals(derived.getString("newRootKey"), hex(root.newRootKey))
assertEquals(derived.getString("chainKey"), hex(root.chainKey))
// 3. kdfChainKey
val chain = kdfChainKey(crypto, root.chainKey)
assertEquals(derived.getString("newChainKey"), hex(chain.newChainKey))
assertEquals(derived.getString("messageKey"), hex(chain.messageKey))
// 4. Header AAD
val aad = encodeRatchetHeader(dhSendPub, previousCounter, counter)
assertEquals(derived.getString("aad"), hex(aad))
// 5. AES-GCM encrypt with fixed nonce
val ciphertext = aesGcmEncryptDeterministic(chain.messageKey, nonce, plaintext, aad)
assertEquals(v.getString("ciphertext"), hex(ciphertext))
// 6. Roundtrip decrypt
val recovered = crypto.aesGcmDecrypt(
chain.messageKey,
fromHex(v.getString("ciphertext")),
nonce,
aad,
)
assertEquals(inputs.getString("plaintext"), hex(recovered))
}
}
}