Ears teach evals
Mixing engineers evolved a complete discipline for working with untrustworthy perception, and eval builders are reinventing it piece by piece. Reference tracks are golden sets: known-quality exemplars consulted on every change so standards cannot drift with fatigue. The instant A/B is pairwise judgment, the most reliable shape there is; comparisons hold where absolute scores wander. Louder always wins unfairly, so you volume-match first: verbosity bias with a gain knob. Late-night ears flatter everything, so you bounce it and listen tomorrow: offline evaluation on a fresh judge. A studio mix must survive the car and the phone speaker: production monitoring.
Evals teach ears
The transfer runs the other way too. Building syn1, my Audio Unit synth, I wanted the discipline my ears cannot provide, because ears cannot hear a two percent regression. So the DSP core carries a harness the way a data platform carries one:
| Check | Result | | --- | --- | | 5-note pad, peak 15 voices | 0.31% of one core, against a 25% budget | | Sub-oscillator aliasing, worst case | -106.2 dB | | Pink noise slope | -2.89 dB/octave, ideal -3.01 | | Same audio at block 16 through 4096 | bit-exact | | Determinism and null tests | same input, same bytes out | | The suite | 141 cases, 3,050,113 assertions, auval green as the gate |
Read the table as an eval engineer and every row is familiar. The CPU budget is a latency SLO. The aliasing floor is a threshold gate. The pink-noise slope is a measurement against a known ideal, which is what a golden set is. Block-size invariance is the determinism law from stream processing applied to audio buffers. The null test, render twice and subtract to silence, is the cleanest regression check I know in any field.
The division that matters
The harness does not decide whether a patch sounds good. It decides whether the machine did what the math promised, so that when a human auditions a sound, the only variable left is taste. Automated gates for the floor, human judges for the ceiling, and never confuse which layer you are standing in. That sentence is equally true of a mix bus and a model deployment, which is why it earns a place on this shelf.