Model monitor
Meta Llama 3.3 70B Instruct
meta-llama/llama-3.3-70b-instruct · via openrouter
As of 2026-08-05: reference route · 94 complete panels attest measurement across 5 task families.
Served route, trailing 30 days: median latency 3.2 s · p95 27.5 s across 1093 probe calls · served by DeepInfra (1093), unrecorded (27). Observed at panel time — a few calls a day, not continuous uptime.
Reference route: this model is the co-probed confound control. It never opens a public incident — its record attests measurement, not verdicts.
Certified claim, per family: H0: E[panel_mean_t | prior completed panels] ≥ frozen_null_floor. Fixed panel composition does not assume or establish temporal independence. Recent panels below link to their hash-committed evidence.
Task family
Structured output
Probing paused: nothing new lands on this family's record; everything already recorded below stands and stays citable.
- Frozen null floor
- 0.700
- Anytime level α
- 0.050
- Completed panels
- 2
- CS lower
- 0.000
- CS upper
- 1.000
Panel record — 2 completed panels
| Cycle | Completed | Member scores | Mean | Evidence |
|---|---|---|---|---|
| 44 | 2026-08-05 14:01 UTC | 1.001.001.000.381.00 | 0.875 | 41d713e42e7e… |
| 43 | 2026-08-05 12:58 UTC | 1.001.001.000.381.00 | 0.875 | 374ba772cdd5… |
Newest first. A failed member scores 0 and stays on the record. Each evidence link opens the panel’s full hash-committed record.
Epoch & design record
- Current epoch
6f065165-7fd0-4aff-bcd1-6c9b4c5a39aa- Opened
- 2026-08-05 12:54 UTC
- Previous epoch
e1c112ed-f9b3-46bd-9d9c-abe46e3fc5d1- Transition
- checker_refresh
- Design hash
8a4f5431d708d308ccd859fc31d097a5db0297535ace2795cab39b5f087eb318- Panel hash
a37ad21aa4b8c3e16dbae958c68a70064a9d95487e131d832f2c601f7fdacf14- Pool hash
8e9496011afbb949e38b3facc664e6dd560f1c5d2155fd424fcb720d084c69ee- Panel size
- 5
- Terminal failure
- zero — a failed member scores 0 and stays on the record
- Pool version
structured-output-v3- Checker version
structured_output-v2- Reference mean μ₀
- 0.700
- Minimum effect δ
- 0.000
- Evidence log e
- -0.14 · max 0.00 · alarm at 3.00 (= ln 1/α)
- Raw scores kept
- 10
- Provider spend, this epoch
- $0.0005278
The persisted monitor state is the authority a certificate replays byte-for-byte; the chart band above is a faithful replay of the same recorded panel means. Spend sums completed probe runs and includes reserved estimates where a run’s true cost was unrecoverable.
Task family
Extraction
Probing paused: nothing new lands on this family's record; everything already recorded below stands and stays citable.
- Frozen null floor
- 0.700
- Anytime level α
- 0.050
- Completed panels
- 1
- CS lower
- 0.000
- CS upper
- 1.000
Panel record — 1 completed panels
| Cycle | Completed | Member scores | Mean | Evidence |
|---|---|---|---|---|
| 44 | 2026-08-05 13:59 UTC | 0.000.000.000.001.00 | 0.200 | 8109f9c57ac3… |
Newest first. A failed member scores 0 and stays on the record. Each evidence link opens the panel’s full hash-committed record.
Epoch & design record
- Current epoch
a981fdd1-0fb2-48a0-8607-188573080370- Opened
- 2026-08-05 12:54 UTC
- Previous epoch
df11941f-8eda-4631-ad5f-0d6f37170316- Transition
- checker_refresh
- Design hash
a2749265eac9731c94d1b8cada6fedc0e453bbc919053c455268b3a7eb248a61- Panel hash
1703e9221c9ce6ce1a9f69974cf0fd77056ac1ee4009dec47e3e6f23be2d4351- Pool hash
51802fc8964f7d9927bcabfae23b4bd2f0905e8977c9f8fb4ce50884ab0fbba6- Panel size
- 5
- Terminal failure
- zero — a failed member scores 0 and stays on the record
- Pool version
extraction-v3- Checker version
extraction-v2- Reference mean μ₀
- 0.700
- Minimum effect δ
- 0.000
- Evidence log e
- 0.30 · max 0.30 · alarm at 3.00 (= ln 1/α)
- Raw scores kept
- 5
- Provider spend, this epoch
- $0.0000988
The persisted monitor state is the authority a certificate replays byte-for-byte; the chart band above is a faithful replay of the same recorded panel means. Spend sums completed probe runs and includes reserved estimates where a run’s true cost was unrecoverable.
Task family
Instruction following
Probing paused: nothing new lands on this family's record; everything already recorded below stands and stays citable.
- Frozen null floor
- 0.700
- Anytime level α
- 0.050
- Completed panels
- 45
- CS lower
- 0.771
- CS upper
- 1.000
Panel record — 45 completed panels
| Cycle | Completed | Member scores | Mean | Evidence |
|---|---|---|---|---|
| 44 | 2026-08-05 13:39 UTC | 1.001.000.830.861.00 | 0.938 | 7d29bceaf6a1… |
| 43 | 2026-08-05 12:36 UTC | 1.001.000.830.861.00 | 0.938 | 10ce7123e314… |
| 42 | 2026-08-05 11:35 UTC | 1.001.000.830.861.00 | 0.938 | 2b20efbff22a… |
| 41 | 2026-08-05 10:31 UTC | 1.001.000.830.861.00 | 0.938 | 1c0a53decc04… |
| 40 | 2026-08-05 09:31 UTC | 1.001.000.830.861.00 | 0.938 | 93287e309d6d… |
| 39 | 2026-08-05 08:26 UTC | 1.001.000.830.861.00 | 0.938 | f2832168059a… |
| 38 | 2026-08-05 07:21 UTC | 1.001.000.830.861.00 | 0.938 | 40bcfc51e3fa… |
| 37 | 2026-08-05 06:22 UTC | 1.001.000.830.861.00 | 0.938 | dc1e56848feb… |
| 36 | 2026-08-05 05:21 UTC | 1.001.000.830.861.00 | 0.938 | 3e7e648497c5… |
| 35 | 2026-08-05 04:16 UTC | 1.001.000.830.861.00 | 0.938 | cf5837cd64e1… |
| 34 | 2026-08-05 03:15 UTC | 1.001.000.830.861.00 | 0.938 | b9a2bb4a05b7… |
| 33 | 2026-08-05 02:14 UTC | 1.001.000.830.861.00 | 0.938 | de74c05949e6… |
| 32 | 2026-08-05 01:14 UTC | 1.001.000.830.861.00 | 0.938 | 7e1605e7b866… |
| 31 | 2026-08-05 00:09 UTC | 1.001.000.830.861.00 | 0.938 | 5f95417135e1… |
| 30 | 2026-08-04 23:09 UTC | 1.001.000.830.861.00 | 0.938 | 6140db2b4b77… |
| 29 | 2026-08-04 22:08 UTC | 1.001.000.830.861.00 | 0.938 | 7a87c2f7a0dd… |
| 28 | 2026-08-04 21:07 UTC | 1.001.000.830.861.00 | 0.938 | 1c7973135c2d… |
| 27 | 2026-08-04 20:06 UTC | 1.001.000.830.861.00 | 0.938 | 1b8cfe08fe2e… |
| 26 | 2026-08-04 19:03 UTC | 1.001.000.830.861.00 | 0.938 | fa9e98734d59… |
| 25 | 2026-08-04 17:59 UTC | 1.001.000.830.861.00 | 0.938 | 92117427b468… |
| 24 | 2026-08-04 16:55 UTC | 1.001.000.830.861.00 | 0.938 | 7895029acd14… |
| 23 | 2026-08-04 15:52 UTC | 1.001.000.830.861.00 | 0.938 | 1a114d1d39a4… |
| 22 | 2026-08-04 14:48 UTC | 1.001.000.830.861.00 | 0.938 | a60d438a040a… |
| 21 | 2026-08-04 13:43 UTC | 1.001.000.830.861.00 | 0.938 | 21c411f02494… |
| 20 | 2026-08-04 12:42 UTC | 1.001.000.830.861.00 | 0.938 | d5094fec6939… |
| 19 | 2026-08-04 11:41 UTC | 1.001.000.830.861.00 | 0.938 | 3c083c3602ec… |
| 18 | 2026-08-04 10:41 UTC | 1.001.000.830.861.00 | 0.938 | 711155790563… |
| 17 | 2026-08-04 09:36 UTC | 1.001.000.830.861.00 | 0.938 | ca1ec7d3069b… |
| 16 | 2026-08-04 08:33 UTC | 1.001.000.830.861.00 | 0.938 | c30a7fe14f2a… |
| 15 | 2026-08-04 07:30 UTC | 1.001.000.830.861.00 | 0.938 | e277af06d294… |
Newest first; older panels are retained and replayable even where this table truncates. A failed member scores 0 and stays on the record. Each evidence link opens the panel’s full hash-committed record.
Epoch & design record
- Current epoch
89818968-8251-40bc-a1e2-e9e22e05ba22- Opened
- 2026-08-03 16:27 UTC
- Design hash
4f5cc6b9ff4ce026ccab93f5a3c8241f78eb005bc8b8068e4c701e9236504412- Panel hash
e7dfea1f343f7bf37e066a1117ca21f3438e804ad61437bf72d810640cef7e71- Pool hash
9c5d065322394a6fb18c6d7f90c5bb63f00781f3e4c4aaa663536f76bb8ba7f9- Panel size
- 5
- Terminal failure
- zero — a failed member scores 0 and stays on the record
- Pool version
instruction_following-v3- Checker version
instruction_following-v1- Reference mean μ₀
- 0.700
- Minimum effect δ
- 0.000
- Evidence log e
- -0.18 · max 0.00 · alarm at 3.00 (= ln 1/α)
- Raw scores kept
- 225
- Provider spend, this epoch
- $0.0021852
The persisted monitor state is the authority a certificate replays byte-for-byte; the chart band above is a faithful replay of the same recorded panel means. Spend sums completed probe runs and includes reserved estimates where a run’s true cost was unrecoverable.
Task family
Code reasoning
Probing paused: nothing new lands on this family's record; everything already recorded below stands and stays citable.
- Frozen null floor
- 0.700
- Anytime level α
- 0.050
- Completed panels
- 1
- CS lower
- 0.000
- CS upper
- 1.000
Panel record — 1 completed panels
| Cycle | Completed | Member scores | Mean | Evidence |
|---|---|---|---|---|
| 43 | 2026-08-05 13:28 UTC | 0.000.000.000.000.00 | 0.000 | 25353d276e4b… |
Newest first. A failed member scores 0 and stays on the record. Each evidence link opens the panel’s full hash-committed record.
Epoch & design record
- Current epoch
d023180f-6535-4cc4-8b73-c6060ce2b87a- Opened
- 2026-08-05 12:54 UTC
- Previous epoch
b1990d63-611b-4ef7-bc55-bcdde3700d10- Transition
- checker_refresh
- Design hash
bc3668bd52d9d48803b744c2c90f4ed816d7e394525b32b4219f6bab5c9c6df2- Panel hash
827de544c7641d140ac37e321bdb2578a41e291915f519183abbc5bd68153e81- Pool hash
cbfba66601ff51f43c83017d6de9e49988afcee72235f5dca89d705f4f8bb4cc- Panel size
- 5
- Terminal failure
- zero — a failed member scores 0 and stays on the record
- Pool version
code_reasoning-v4- Checker version
code_reasoning-v2- Reference mean μ₀
- 0.700
- Minimum effect δ
- 0.000
- Evidence log e
- 0.39 · max 0.39 · alarm at 3.00 (= ln 1/α)
- Raw scores kept
- 5
- Provider spend, this epoch
- $0.0001405
The persisted monitor state is the authority a certificate replays byte-for-byte; the chart band above is a faithful replay of the same recorded panel means. Spend sums completed probe runs and includes reserved estimates where a run’s true cost was unrecoverable.
Task family
Math
Probing paused: nothing new lands on this family's record; everything already recorded below stands and stays citable.
- Frozen null floor
- 0.700
- Anytime level α
- 0.050
- Completed panels
- 45
- CS lower
- 0.313
- CS upper
- 0.665
Panel record — 45 completed panels
| Cycle | Completed | Member scores | Mean | Evidence |
|---|---|---|---|---|
| 44 | 2026-08-05 13:34 UTC | 0.001.001.000.001.00 | 0.600 | cf81b11b576b… |
| 43 | 2026-08-05 12:31 UTC | 1.001.001.000.000.00 | 0.600 | 41b3d242b64b… |
| 42 | 2026-08-05 11:33 UTC | 0.001.001.000.000.00 | 0.400 | 58355bdcbf86… |
| 41 | 2026-08-05 10:26 UTC | 0.001.001.000.000.00 | 0.400 | 015e7cad5d3a… |
| 40 | 2026-08-05 09:24 UTC | 1.001.001.000.000.00 | 0.600 | 4c2cdbca08b1… |
| 39 | 2026-08-05 08:18 UTC | 0.001.001.000.000.00 | 0.400 | 3ce395388f9c… |
| 38 | 2026-08-05 07:17 UTC | 1.001.001.000.000.00 | 0.600 | ba77f6387fe9… |
| 37 | 2026-08-05 06:09 UTC | 0.001.001.000.001.00 | 0.600 | 91d7273da096… |
| 36 | 2026-08-05 05:08 UTC | 0.001.001.000.001.00 | 0.600 | fd557daaa93e… |
| 35 | 2026-08-05 04:05 UTC | 0.001.001.000.000.00 | 0.400 | d566d81018c8… |
| 34 | 2026-08-05 03:09 UTC | 0.001.001.000.000.00 | 0.400 | 7b41923e3d5c… |
| 33 | 2026-08-05 02:04 UTC | 1.001.001.000.000.00 | 0.600 | fa07c2b275cb… |
| 32 | 2026-08-05 01:00 UTC | 0.001.001.000.000.00 | 0.400 | dfb6ce0ade30… |
| 31 | 2026-08-04 23:55 UTC | 0.001.001.000.001.00 | 0.600 | 56a153a80bd8… |
| 30 | 2026-08-04 22:57 UTC | 0.001.001.000.000.00 | 0.400 | 33fceb601499… |
| 29 | 2026-08-04 21:55 UTC | 0.001.001.000.000.00 | 0.400 | be32a878eab5… |
| 28 | 2026-08-04 20:53 UTC | 0.001.001.000.001.00 | 0.600 | cfc075909d3e… |
| 27 | 2026-08-04 19:53 UTC | 0.001.001.000.000.00 | 0.400 | 8252afb2bb61… |
| 26 | 2026-08-04 18:50 UTC | 0.001.001.000.000.00 | 0.400 | 71008a9fe829… |
| 25 | 2026-08-04 17:47 UTC | 0.001.001.000.001.00 | 0.600 | 58c0e9c783b1… |
| 24 | 2026-08-04 16:47 UTC | 1.001.001.000.000.00 | 0.600 | 9a73a223ebf5… |
| 23 | 2026-08-04 15:44 UTC | 1.001.001.000.000.00 | 0.600 | 678bf19f920f… |
| 22 | 2026-08-04 14:40 UTC | 0.001.001.000.001.00 | 0.600 | 29043431f148… |
| 21 | 2026-08-04 13:39 UTC | 0.001.001.000.000.00 | 0.400 | cc84605e9f8d… |
| 20 | 2026-08-04 12:37 UTC | 0.001.001.000.000.00 | 0.400 | 79916c164139… |
| 19 | 2026-08-04 11:36 UTC | 1.001.001.000.001.00 | 0.800 | 32a02df21468… |
| 18 | 2026-08-04 10:31 UTC | 0.001.001.000.000.00 | 0.400 | b110912623d6… |
| 17 | 2026-08-04 09:27 UTC | 0.001.001.000.000.00 | 0.400 | 362bf9bb0cbb… |
| 16 | 2026-08-04 08:23 UTC | 0.00 rate limited0.00 rate limited0.00 rate limited0.00 rate limited0.00 rate limited | 0.000 | 3dd96fdaeda5… |
| 15 | 2026-08-04 07:21 UTC | 1.001.001.000.000.00 | 0.600 | 997b1a13aae2… |
Newest first; older panels are retained and replayable even where this table truncates. A failed member scores 0 and stays on the record. Each evidence link opens the panel’s full hash-committed record.
Epoch & design record
- Current epoch
5047ee24-a758-4ea2-94a2-7c08c42c4ae2- Opened
- 2026-08-03 16:27 UTC
- Design hash
fdf94065c55c2ee7d86701efc50be0f5abdc5f02a236f6df52c2ac6f7db801eb- Panel hash
882f3932d07aeb85c4f6406a550d8ad14512e70d3df2432fe7aab03d0508bc75- Pool hash
7ed49fa9a63ba0e0907ff8326de00a843a926618f6d0dec40aa8e491d041e99b- Panel size
- 5
- Terminal failure
- zero — a failed member scores 0 and stays on the record
- Pool version
math-v4- Checker version
math-v1- Reference mean μ₀
- 0.700
- Minimum effect δ
- 0.000
- Evidence log e
- 12.27 · max 12.27 · alarm at 3.00 (= ln 1/α)
- Raw scores kept
- 213
- Provider spend, this epoch
- $0.0295865
The persisted monitor state is the authority a certificate replays byte-for-byte; the chart band above is a faithful replay of the same recorded panel means. Spend sums completed probe runs and includes reserved estimates where a run’s true cost was unrecoverable.