coconutlabs

the proof page

Benchmarks

published from committed artifacts · kvwarden v0.1.6

One rule holds everywhere on this page: a number without its hardware, sample count, and methodology is not a result. Every figure below carries all three and links the raw artifact that produced it.

quiet tenant p99 TTFT

61.5ms

▲ 1.14× of solo · post-warmup · n=311

solo baseline p99

53.9ms

same box, no contention · n=320

tail vs FIFO

26×

1,585 ms → 61.5 ms

quiet requests

321ok

0 errors, both arms

1× A100 SXM4 · Llama-3.1-8B · vLLM 0.19.1 · 300 s · flooder 32 RPS / quiet 1 RPS

Quiet-tenant TTFT under flood

kvwarden token bucket · full 300 s bench

p50
37.9 ms
p95
51.4 ms
p99 (post-warmup)
61.5 ms
p99 · FIFO
1,585 ms

FIFO bar is off scale: the axis ends at 80 ms and the FIFO tail is 20× past it. That gap is the product. p50 and p95 are the full bench; p99 excludes the first 10 s warmup transient, methodology in CORRECTIONS C7.

what this does not show

  • The starvation regime is A100-at-saturation. On H100 at the same offered load, FIFO does not starve, so the mechanism runs clean but is not stressed.
  • An earlier run on vLLM 0.8.5 measured 523× starvation. The v1 batcher absorbs cold-start backpressure better; 29× is the honest steady-state number and the only one we cite.
  • Full-bench p99 for the kvwarden arm is 1,230 ms including the warmup transient. The 61.5 ms hero is post-warmup by stated method, not by trimming inconvenient data silently.
  • Flooder rejections are the mechanism working: 6,488 requests got 429, by design.

raw artifacts

reproducepip install kvwarden==0.1.6

The bench we most want does not exist yet: yours. Run it against your own traffic and tell us where it breaks. That is worth more to this lab than a star.