# Gaps, reconnections and replicas

Summary: Every hour's manifest states what the record holds and what it misses. It lists each reconnection, each break Nox detects in your venue's message sequence, and what each replica received. A declared gap does not fail verification: the hour still verifies, and its status is partial with the reasons.

## The example with a declared gap

The [synthetic example](https://noxfze.com/docs/example) has a second pair of hours with a declared gap. In its second hour, both
replicas reconnect, and one trade is in no frame of the hour. The record format allows two replicas, `a` and `b`,
each with its own connections to the venue's public interfaces.

You need `jq`, `zstd`, `b3sum` and the DuckDB command-line client, and a POSIX shell.

```sh download
curl -fsSO https://noxfze.com/docs/example-gap-hour.tar
tar -xf example-gap-hour.tar
```

```sh setup
cd example-gap-hour/recorder=r1/venue=example/channel=trades
HOUR=date=2026-01-06/hour=00
PREV=date=2026-01-05/hour=23
```

## 1. Read the status

The status is `complete`, `partial` or `missing`, and the reasons say why an hour is not complete:

```sh check status
jq -c .completeness "$HOUR/manifest.json"
```

```text output status
{"reasons":["reconnect","sequence_gap"],"status":"partial"}
```

The [record format](https://noxfze.com/docs/format#hourly-manifest) lists every reason. An hour in which Nox received no messages
still has a manifest, so the chain never skips an hour.

## 2. Find the break in the sequence

Each entry names the instrument, the last identifier received before the break and the first one after it:

```sh check gaps
jq -c '.sequence_gaps[]' "$HOUR/manifest.json"
```

```text output gaps
{"after_id":"1003","before_id":"1005","instrument":"INST-1"}
```

The raw file holds trades 1003, 1005 and 1006. Trade 1004 is in no frame of the hour, and no value stands in for it:

```sh check frames
zstd -dc "$HOUR/raw.jsonl.zst" | jq -r '.payload | fromjson | .id'
```

```text output frames
1003
1005
1006
```

## 3. See each reconnection

Each replica lists its reconnect windows. A window carries the venue's close code when the venue closed the
connection with one:

```sh check windows
jq -r '.replicas[] | .replica as $r | .reconnect_windows[] |
  "\($r)  \(.start_time) to \(.end_time)  \(.close_code // "none")"' \
  "$HOUR/manifest.json"
```

```text output windows
a  2026-01-06T00:00:20.000000000Z to 2026-01-06T00:00:32.000000000Z  1001
b  2026-01-06T00:00:25.000000000Z to 2026-01-06T00:00:35.000000000Z  none
```

From 00:00:25 to 00:00:32 neither replica was connected.

## 4. Where each replica saw no gap

Each replica states the runs in which one connection was subscribed and it detected no gap in the venue's trade
sequence. A break lies between the end of one run and the start of the next:

```sh check runs
jq -r '.replicas[] | .replica as $r | .gap_free_runs[] |
  "\($r) \(.connection)  \(.start_time) to \(.end_time)"' \
  "$HOUR/manifest.json"
```

```text output runs
a c1  2026-01-06T00:00:00.000000000Z to 2026-01-06T00:00:05.000000001Z
a c3  2026-01-06T00:00:40.000000000Z to 2026-01-06T01:00:00.000000000Z
b c2  2026-01-06T00:00:00.000000000Z to 2026-01-06T00:00:05.001000001Z
b c4  2026-01-06T00:00:40.001000000Z to 2026-01-06T01:00:00.000000000Z
```

A run ends 1 ns after its connection's last frame before the break. The next run starts where the replica detected
the break. A run states that the replica detected no gap, not that none occurred.

## 5. Compare the replicas

Frames received by both replicas, by one only, and trade identifiers the replicas carry with different values:

```sh check agreement
jq -c .replica_agreement "$HOUR/manifest.json"
```

```text output agreement
{"both":3,"conflict":0,"one_only":0}
```

Each frame carries one receipt per replica that received it, with that replica's receive time:

```sh check receipts
zstd -dc "$HOUR/raw.jsonl.zst" |
  jq -r '[.receipts[] | "\(.replica) \(.receive_time)"] | join("  ")'
```

```text output receipts
a 2026-01-06T00:00:05.000000000Z  b 2026-01-06T00:00:05.001000000Z
a 2026-01-06T00:00:40.000000000Z  b 2026-01-06T00:00:40.001000000Z
a 2026-01-06T00:00:50.000000000Z  b 2026-01-06T00:00:50.001000000Z
```

## 6. The hour still verifies

A declared gap is part of the sealed hour. The checks of [command-line verification](https://noxfze.com/docs/verify) pass on it:

```sh check hashes
jq -r '.files[] | "\(.blake3)  \(.name)"' "$HOUR/manifest.json" |
  (cd "$HOUR" && b3sum --check)
```

```text output hashes
raw.jsonl.zst: OK
trades.parquet: OK
```

```sh check chain
test "$(b3sum --no-names "$PREV/manifest.json")" = \
  "$(jq -r .previous_manifest_blake3 "$HOUR/manifest.json")" &&
  echo "chain: OK"
```

```text output chain
chain: OK
```

## 7. Every hour that is not complete

Over a run of hours, DuckDB reads every hourly manifest as one table. This lists each hour with its status, its
reasons and its number of breaks:

```sh check hours
duckdb -csv <<SQL
SELECT interval.start_time AS hour,
  completeness.status AS status,
  completeness.reasons AS reasons,
  len(sequence_gaps) AS breaks
FROM read_json('date=*/hour=*/manifest.json')
ORDER BY hour;
SQL
```

```text output hours
hour,status,reasons,breaks
2026-01-05T23:00:00.000000000Z,complete,[],0
2026-01-06T00:00:00.000000000Z,partial,"[reconnect, sequence_gap]",1
```

Add `WHERE completeness.status <> 'complete'` to keep only the hours that are not complete. To read the trades of
those hours, see [reading a sealed hour](https://noxfze.com/docs/read).