nightly loop · last read 2026-08-14 21:55:00 UTC · cached 3m 41s
The nightly loop.
Every night a candidate is registered, run against a fixed suite, canaried against the incumbent, then promoted or rolled back. Every artifact of every night is committed to a public repository. This page reads those files.
2 last 10 days
0 candidates accepted
2 candidates rejected
0.70 min pass rate
$0 actual, public repo CPU minutes
last night
The most recent decision
2026-08-14 · rolled back
gate failed on pass_rate (pass_rate = 0)
gate 0 of 20 cases passed, pass rate 0.00, threshold 0.70 · p50 1,786 ms · p95 5,722 ms · verdict fail on pass_rate
canary 6 fixed cases, candidate quality 0.00 against incumbent 0.50 · verdict fail on quality
serving after 8a7af0c7b319
artifacts runs/2026-08-14
the record
Every night in the window
| date | candidate variant | gate | canary | action | candidate | serving after |
|---|---|---|---|---|---|---|
| 2026-08-14 | drift-truncate scheduled to degrade |
✕ fail | ✕ fail | rollback | b024ae0f4ce5 | 8a7af0c7b319 |
| 2026-08-13 | candidate not scheduled to degrade |
✕ fail | ✓ pass | rollback | 0867e8775c40 | 8a7af0c7b319 |
Counted from runs/<date>/summary.json, one file per night. The variant column is read from registry/versions/<hash>.json for the 6 most recent nights, so a night older than that shows "not read" rather than a guess.
Serving now: 8a7af0c7b319
variant base
model qwen2.5-0.5b-instruct-q4_k_m
No candidate has been promoted in the last 10 days. This version is what the registry serves because nothing has beaten it, not because it won a promotion in this window.
registry history as committed
| from | version | action |
|---|---|---|
| 2026-08-13 | 0867e8775c40 | rollback |
| 2026-08-13 | 0867e8775c40 | rollback |
| 2026-08-14 | b024ae0f4ce5 | rollback |
This table is printed exactly as registry/serving.json holds it, including a duplicated entry. The counts above are taken from the per night summaries instead, which have one file per night.
rollback events
2 rollbacks in the window
2026-08-14 · b024ae0f4ce5
Gate fail, canary fail. Variant drift-truncate: this candidate was scheduled to degrade, on a schedule published in the repository. The loop caught it. Serving stayed on 8a7af0c7b319.
2026-08-13 · 0867e8775c40
Gate fail, canary pass. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 8a7af0c7b319.
disclosure
What this does not show
The claim under test is the loop, not the model
- The agent is qwen2.5-0.5b-instruct-q4_k_m on llama.cpp, on CPU, inside GitHub Actions. It is small on purpose. Nothing here says this model is good.
- Scoring is mechanical: exact match and schema validity. No model grades another model on this page.
- The drift schedule degrades the candidate on purpose, on a schedule published in the repository. Day of month mod 7 equal to 5 raises temperature to 1.4. Day of month mod 3 equal to 2 cuts the prompt template to 60 percent of its length. Every other day gets a benign paraphrase. The variant column above is read from the committed version artifact, so a night that was scheduled to degrade and a night that was not are labelled differently.
- A rollback is a successful run of the loop. The pipeline exits zero on a caught failure, because catching it is the job. A nonzero exit means infrastructure broke, not that a policy fired.
- The cost estimate in each gate file counts generated tokens only against a reference price. It is not a bill. Actual spend is zero because the repository is public and the minutes are free.
Where this runs
The loop runs in GitHub Actions on a schedule and commits every artifact of every night to a public repository. This page reads those files from the raw file host and caches them for 15 minutes. Nothing is copied by hand. Every number here has a URL you can open yourself.
Machine readable: /live/status.json. Source of truth: https://github.com/coconut-labs/atlas-pipeline.