UAQ --:--:--

Documentation

Command-line verification

Summary. Three openly available tools verify a sealed hour against its manifest and against the hour before it. jq reads the manifest, b3sum recomputes BLAKE3 hashes, and zstd opens the raw file. The commands verify the format version, each listed file's size and hash, the raw file's line count, and the hash chain link to the previous hour. Each command prints its result and exits with a non-zero status when its check fails.

View as Markdown

What you need

1. Get two consecutive hours

The synthetic example hour and the hour before it:

curl -fsSO https://noxfze.com/docs/example-hour.tar
tar -xf example-hour.tar

Go to the channel's directory and name the hour to check and the hour before it:

cd example-hour/recorder=r1/venue=example/channel=trades
HOUR=date=2026-01-06/hour=00
PREV=date=2026-01-05/hour=23

For your own record, copy two consecutive hour directories of one channel from storage you own. Keep their date=YYYY-MM-DD/hour=HH paths, and set HOUR and PREV to them. The hour before 00:00 UTC is 23:00 UTC of the previous date.

2. Read the manifest

The manifest is format version 1. It lists each file of the hour with its size in bytes and its BLAKE3 hash:

jq -e '.format_version == 1' "$HOUR/manifest.json"
true
jq -r '.files[] | "\(.name)  \(.size)  \(.blake3)"' "$HOUR/manifest.json"
raw.jsonl.zst  318  d9aa031a1200254b0951863450ea62e506ecbe32f6d6b63106689b622238f96e
trades.parquet  2052  a693551b628a352b8303946729066efb66d32452d11a3ee0230055af9f81769e

3. Verify each file's size and hash

Each listed file has the listed size:

jq -r '.files[] | "\(.size) \(.name)"' "$HOUR/manifest.json" | while read -r size name; do
  test "$(wc -c < "$HOUR/$name")" -eq "$size" || { echo "$name: size differs"; exit 1; }
  echo "$name: $size bytes, as listed"
done
raw.jsonl.zst: 318 bytes, as listed
trades.parquet: 2052 bytes, as listed

Each listed file has the listed BLAKE3 hash. A file that is missing or differs by one byte fails here:

jq -r '.files[] | "\(.blake3)  \(.name)"' "$HOUR/manifest.json" | (cd "$HOUR" && b3sum --check)
raw.jsonl.zst: OK
trades.parquet: OK

4. Open the raw file

The raw file holds one line per frame, and its line count is the number of rows the manifest lists for it:

rows=$(jq '.files[] | select(.name == "raw.jsonl.zst") | .rows' "$HOUR/manifest.json")
test "$(zstd -dc "$HOUR/raw.jsonl.zst" | wc -l)" -eq "$rows" && echo "raw.jsonl.zst: $rows lines, as listed"
raw.jsonl.zst: 4 lines, as listed

The first frame of the hour, with the venue's payload as received and one receipt per replica that received it:

zstd -dc "$HOUR/raw.jsonl.zst" | head -n 1 | jq .
{
  "exchange_time": "2026-01-06T00:00:04.998000000Z",
  "payload": "{\"type\":\"trade\",\"instrument\":\"INST-1\",\"id\":\"1003\",\"side\":\"buy\",\"price\":\"100.5\",\"size\":\"0.25\",\"time\":1767657604998}",
  "payload_encoding": "text",
  "receipts": [
    {
      "clock_error_bound": 60000,
      "clock_offset": 300,
      "connection": "c1",
      "receive_time": "2026-01-06T00:00:05.000000000Z",
      "replica": "a"
    },
    {
      "connection": "c2",
      "receive_time": "2026-01-06T00:00:05.001000000Z",
      "replica": "b"
    }
  ]
}

The hour's manifest carries the BLAKE3 hash of the previous hour's manifest bytes. A change to any byte of the previous manifest, including a file hash it lists, changes that hash:

computed=$(b3sum --no-names "$PREV/manifest.json")
carried=$(jq -r .previous_manifest_blake3 "$HOUR/manifest.json")
echo "computed: $computed"
echo "carried:  $carried"
test "$computed" = "$carried" && echo "chain: OK"
computed: 05a9316fa40d8f7ea43cd1c6cbb27c88c979059466054d6cae45150c4aded1e8
carried:  05a9316fa40d8f7ea43cd1c6cbb27c88c979059466054d6cae45150c4aded1e8
chain: OK

The two hours are consecutive and belong to the same hash chain:

jq -e --slurpfile prev "$PREV/manifest.json" '.interval.start_time == $prev[0].interval.end_time
  and .recorder == $prev[0].recorder and .venue == $prev[0].venue and .channel == $prev[0].channel' \
  "$HOUR/manifest.json"
true

The first hour of a hash chain carries null instead of a hash. To verify a longer run of hours, repeat step 5 for each pair of consecutive hours.

What these commands do not verify

The commands make check 2 of the ten checks of record format, version 1. They also make the format version part of check 1, the line count of check 3, and the hourly links of check 5.

The other checks need an implementation of the format. They cover the canonical form of the manifests (check 1), the rest of the raw file and the tables (checks 3 and 4), and the daily manifests (check 5). They also cover signatures, on hours that carry them (check 6), and the clock fields, runs and conflicts (checks 7 to 10).

A pass does not show that these hours are the ones Nox FZE sealed: confirm that by matching an hour's root to a root you received earlier. An hour's root is the hash of its manifest. This root shows only that the hour is unchanged since Nox FZE sealed it, and which messages Nox FZE received from the venue's public interfaces and when. It gives no opinion on the venue or on its markets, volume, statements, controls or systems.

Nox works in writing. Email contact@noxfze.com with your venue and the instruments to record.

Open mail ↗

Copied to clipboard