gallery unit · data core
Columnar scan / bytes-read guardrail
The query returns exactly the right rows. To get two columns it quietly read the whole table, and every check that watches correctness stays green.
scenario sandbox
You are the analytics engineer. The same 200,000-row, 12-column table sits under every query. Pick a query and a storage layout; the guardrail counts the bytes the read actually touches.
pick a query and a layout
within budget
Only the projected columns were read: 800 KB against a budget of 800 KB. Same right rows, a fraction of the bytes.
needed by the query · read but not needed · skipped
projection needs 800 KB · this read is 1.0x that
the answer is identical whichever layout you pick, so this check passes even the full scan.
Bytes here are the deterministic model. On a real Parquet file the Python harness measured the same effect for real: the projected read touched 193 KB versus 8.6 MB for the full scan of the same file, 44.7x, and 459.8x against a row-oriented CSV.
what you are looking at
01 · data in
A 200,000-row, 12-column table with a fixed byte width per column, modeled deterministically in your browser.
02 · check runs
Your query and layout decide which column chunks get read. The guardrail compares bytes touched to what the projection needs; a correctness check watches the rows.
03 · verdict out
The meter shows bytes read, and the banner says whether the read stayed inside budget.
Go deeper
Watch the walkthrough (45 s, silent)
The same query reading 2 columns or all 12 · recorded from this page, unedited.
The measured result: 44.7x over-read
The query returns the right rows. It just read the whole table to get two columns, and nothing that checks correctness can see it.
Same file, one query, read two ways
200k rows · 2 of 12 columns · guardrail integer-exact
| Check | Full scan | Projected | Verdict |
|---|---|---|---|
| Bytes-read guardrail | FLAGGED | passes | catches it |
| Result-correctness check | right rows | right rows | misses it |
| Bytes physically read | 8.6 MB | 192 KB | 44.7x more |
Both reads return the identical 18 groups. The full scan pulled 8.6 MB through the reader to do it; the projected read pulled 192 KB, 44.7x less, measured on a real Parquet file. Against a row-oriented CSV of the same data it is 460x.
Where the problem statement comes from
“A columnar database … allows having many columns in a table just in case, but to not pay the cost for unused columns on read query execution time (a traditional OLTP database reads all of the data during queries as the data is stored in rows and not columns).”
ClickHouse documentation · “What is a columnar database?” · updated 2026-05-22
The full mechanism: why the usual check fails
- Correctness checks watch the output. “Did the query return the right rows?” is true whether the engine read two columns or all twelve. The waste is entirely on the read side, where the result never looks.
- The cost is invisible until it scales. On a laptop file it is a warm-cache blip; on a wide table in object storage it is the bill and the latency. A row layout (or a columnar file read without projection pushdown) reads every column to answer a two-column question.
The guardrail ignores the rows and asserts a property of the read: the bytes touched must be within tolerance of what the projection needs. It flags the scan that a correctness check waves through.
query: SELECT country, device ... (2 of 12 columns) row layout : read every column → 88.5 MB ✗ (460x) columnar, full read : read every chunk → 8.6 MB ✗ ( 45x) columnar, projected : read 2 column chunks → 192 KB ✓ guardrail: flag when bytes_read > tolerance x projection_budget
Evidence + how to reproduce
A real off-the-shelf control. The harness writes a real Parquet file and a real CSV, then counts the bytes physically pulled through the reader (pre-buffering off) for the same query three ways. The instrumented projected read (192 KB) is cross-checked against the file’s own column-chunk metadata (130 KB floor), so the number is I/O, not buffer coalescing. Synthetic table (read-amplification needs a controlled row width); stated on the page, not hidden.
make setup && make test && make run # $0, laptop, no GPU, no network
Honest gaps
The dataset is synthetic and deterministic (seed=7): showing read amplification needs a controlled row width, so the numbers are on a designed table, but the failure mode is real and cited. The 44.7x is projection isolated (same file, same compression on both sides) and reflects projecting two low-cardinality dict-encoded columns; project a fat string column like url and the ratio shrinks. The 460x CSV figure additionally includes compression, so it is a second axis, not a like-for-like projection number. The guardrail covers projection (columns read); predicate pushdown / row-group skipping is a second mechanism with its own failure modes and is deliberately out of scope. The live browser demo uses the integer read model; the real compression and encoding effects live only in the measured Parquet numbers, never in the model, so Python and the TypeScript port agree to the byte.