The instrument as a systems project
I am building the synthesizer I make music with: a C++ Audio Unit for Logic, codename syn1. The DSP core deliberately has no framework dependency, so the whole test suite, 330 cases, runs against a small test library in about ten seconds instead of compiling against 190 MB of JUCE. That one structural choice is why the project can be built with a model in the loop at all: fast, honest feedback beats clever suggestions.
Two build problems taught me more about AI-assisted engineering than the code did. The build has a switch to not install a freshly compiled plugin, because swapping a component under a live Logic session can take the session down. And each build directory fetches its own dependencies, because parallel workers racing to fill one shared cache corrupt it. Neither lesson is musical. Both decide whether the instrument ships.
Three jobs, one fence
In this project the model has three jobs.
Bench partner. DSP math, test design, build systems. The unglamorous majority of instrument building, where a well-read pair is worth real hours.
Menu widener, inside the instrument. The plugin carries a harmony assistant, a mood pad, and a generate panel. All three propose: candidate chords under a melody, parameter neighborhoods that match a mood, phrases to audition. Proposals arrive as options with your hands still on the keys. Nothing lands in the track without a human committing it.
Borrowed ears. A localhost tool turns a reference track into six separated stems and a scrubbable beat grid. On Apple Silicon the separation runs on the GPU, roughly 30x faster than CPU in my testing, with no audible difference in the stems. Reference listening with everything separated out, and nothing leaves the machine.
One prototype sat downstream of the decision, writing parts instead of proposing them. What came out was competent and dead, the average of everything, which is the one thing a track must never be. That failure set the fence: the useful place for a model in a creative tool is upstream of the decision, widening what you consider. The fence is structural, not aspirational.
What this has to do with inference systems
The fence generalizes. Proposal-only interfaces with a human commit point, measured locally and fast, are the same shape I argue for in inference systems: the machine expands the option space and the human owns the write path. The synth is where I test that argument against my own ears.