The default way to tie visuals to music is to analyse the incoming audio and drive parameters from it: amplitude to scale, bass band to brightness, onset detection to a trigger. It works immediately, which is why almost every first audio-reactive patch is built that way.
It also produces visuals that know nothing about the song. Dom Davis, writing for the Interactive & Immersive HQ on 26 October, makes the argument for the other approach, and gives it a name: music math.
The distinction
Davis’s framing: “knowing where I am in the song and using music math means knowing what’s coming next, so I can create better content with the larger show in mind.”
That is the whole difference. Audio reactivity is retrospective — it responds to sound that has already happened. Phrase counting is predictive: if you know you are on bar 25 of a 32-bar section, you know a change arrives in eight bars, and you can build toward it. One gives you a visual that flinches; the other gives you a visual that anticipates.
He is explicit that this is not new theory, just unnamed in this context: “you might know parts of it as phrasing or phrase counting.”
Montage, before any of the technical parts
The piece opens on montage, which is a notable choice for a TouchDesigner blog.
Davis works through A + B = C, the Kuleshov effect, and what Sergei Eisenstein called intellectual montage: “‘A’ may represent a blue sky, and ‘B’ may represent a happy dog; ‘C’ may then be taken as the by-product of both, which, for many, means the dog is happy because the sky is blue.”
The relevance to live visuals is that a VJ is cutting between images in time, which is editing, and editing has a century of theory about what two adjacent images do to each other. A patch that cross-fades on a kick drum is making montage decisions whether or not anyone intends them.
The toolkit he names
The post walks through, in order:
- Montage — meaning produced by adjacency.
- Metric and rhythmic editing — Eisenstein’s categories, where metric editing cuts on fixed measure lengths regardless of content, and rhythmic editing lets the content’s own movement set the cut.
- Genre and song structure — different genres have different section lengths and different conventions about where a drop or a turnaround sits.
- Musical frequency — the band content of the music, used as information rather than as a direct driver.
- A 64-beat counting method — his own practice for always knowing his position.
- When to break the rules — framed as the final section, and as intentional rather than accidental.
Why 64
He does not present 64 as arcane. Four beats to a bar, four bars to a phrase, four phrases to a 64-beat section is the skeleton of a great deal of electronic dance music, and it nests cleanly: a 64-count is 16 bars, half of a conventional 32-bar section, two of an 8-bar phrase.
Counting in 64s gives you a position that is meaningful at several scales at once, which is exactly what a visual needs — a 16-bar build wants a slow ramp, the 4-bar phrase inside it wants a repeating gesture, and the single bar wants the fast motion.
The practical consequence for a patch
Nothing here argues against audio analysis; it argues for a second, structural clock alongside it.
Concretely, that means a patch needs a notion of beat count since the start of the section, not just a beat trigger. The usual implementations are a tap-tempo or a clock from the DJ’s own gear — MIDI clock, Ableton Link, or OSC from a controller — counted into bars and phrases, with a manual reset so the operator can re-anchor when a track changes or the count slips.
Once a patch has that number, a lot of the hand-operated work disappears. A ramp over 64 beats is one expression. A cut every 16 is another. The transition that currently requires the VJ to hit a key at the right moment becomes something the patch can be told to expect.
Sources
- Music Math for Beginners: Turning Rhythm into Visuals — The Interactive & Immersive HQ
- TouchDesigner documentation — Derivative
- Ableton Link