Model behaviour, on the record
The public watchtower.
Actuary’s own monitors, pointed at hosted models, published unedited — alarms, quiet spells and certificates alike.
Live monitor state
Model status grid
Each verdict reflects the model family’s current epoch.
| Model | Structured output | Extraction | Instruction following | Code reasoning | Math |
|---|---|---|---|---|---|
| Meta Llama 3.3 70B Instruct meta-llama/llama-3.3-70b-instruct | No alarm | No alarm | No alarm | No alarm | No alarm |
| Moonshot Kimi K2.5 moonshotai/kimi-k2.5 | No alarm | No alarm | No alarm | No alarm | No alarm |
“No alarm” means the current epoch holds a certified design and no confirmed alarm — a statement about the instrument, not a promise. “On watch” means the epoch is still learning its reference mean. Nothing is backfilled: a model added tomorrow starts accumulating tomorrow.
Pre-registered designs
Frozen before the first observation.
The unit here is a route — a model reached through a specific upstream at a specific quantisation — not a model.
| Route | Record | Panel | Floor | α | State |
|---|---|---|---|---|---|
| Meta Llama 3.3 70B Instruct meta-llama/llama-3.3-70b-instruct · upstream unpinned | 5 | 0.70 | 0.05 | Not started | |
| Mistral Small 3.2 24B Instruct mistralai/mistral-small-3.2-24b-instruct · upstream unpinned | 5 | 0.70 | 0.05 | Not started | |
| Qwen 2.5 72B Instruct qwen/qwen-2.5-72b-instruct · upstream unpinned | 5 | 0.70 | 0.05 | Not started |
Three routes across five task families, fifteen monitors in total. Each carries a frozen panel hash, checker version, floor and α; changing any of them opens a new epoch with a recorded reason. Upstream and quantisation are recorded but not yet pinned — until they are, an alarm describes the route, not the model.
Register
Entry 001 — the validation study.
The only detection so far, made deliberately: one model, two lanes — served normally, and degraded on purpose — to check the alarm fires when it should and stays quiet when it should not.
Study 001 · 2026-07-24 · certificate v4
| Lane | Panels | Outcome | Evidence | Bound |
|---|---|---|---|---|
| Stable | 20 | No alarm | 0.79 | — |
| Degraded | 12 | Alarm at block 12 | 29.96 | ≤ 19 |
Estimator: one-sided betting e-process on complete-panel means.
Subject: openai/gpt-4.1-nano, one frozen 5-item math
panel, floor 0.70, α 0.05.
n = 20 and n = 12 complete panels. The alarm bound was
registered before the run. Provider spend $0.0060477.
Deterministically replayable; the signed certificate verifies offline on
all three checks — recompute it yourself. One
trajectory, not a rate — see methodology for
the measured operating characteristics and their limits.
Standing commitments
What we have promised in advance.
Quiet is published too
A watchtower that only publishes when it finds something is a watchtower with an incentive to find something. Quiet spells appear here with the same prominence as alarms.
Two tiers, never merged
Automatically detected anomalies and reviewed incidents are separate lists. An anomaly is promoted only after an independent replication check agrees with it.
Funding is disclosed
If a provider we measure ever pays us for anything, it will be stated on the entry itself. We will not sign an agreement that prevents us disclosing the relationship.
Request a model
What should the record watch next?
Coverage today is three open-weight routes, stated plainly. Tell us which model matters to you and we will tell you what watching it would take.