Letting an audience influence a live show has always had a built-in tension: give them real control and you risk chaos; keep the control token and symbolic and the participation feels hollow. A new study, CrowdForge, tests that tension directly by handing a live concert audience real, substantive control over both the generative music and the mixed-reality visuals during an actual performance — and comes back with a specific, counterintuitive answer about what makes participation feel meaningful.
What the audience actually controlled
Researchers Torin Hopkins, Shih-Yu Ma, Zander Trafton-Hancock, Ellen Yi-Luen Do, and Luca Turchet built CrowdForge as what they call an “AI- and MR-mediated paradigm,” tested during a live performance featuring a guitarist and a DJ. Audiences collectively steered generative music parameters and the mixed-reality visual environment in real time, and the researchers ran a mixed-methods study across three separate interaction conditions: audio control alone, visual control alone, and both combined.
Observability, not control, drove the feeling of agency
The paper’s central finding is that observability — how clearly and immediately an audience member could see the effect of their own input — was the primary driver of felt agency, more than the raw amount of control on offer. Visual interaction gave easily observable, immediate feedback; audio parameters, by contrast, offered more macro-structural control over the music’s overall shape, but that control was harder for any individual audience member to directly attribute to their own input in the moment.
The adversarial twist nobody designed for
The most interesting result wasn’t something the researchers set out to build: audiences, given real control over AI-generated audio, tended to use it to test the musicians’ limits — deliberately pushing the generative system toward boundaries to see how the human performers would respond. The paper frames this as “a productive adversarial tension between the crowd and the musicians” that, rather than undermining the performance, deepened engagement and produced genuine moments of musical coherence precisely when the performers successfully resolved a challenge the audience had thrown at them. That’s a meaningfully different outcome than most participatory-art systems report, where audience control is usually designed to feel collaborative rather than oppositional — here, a bit of adversarial friction turned out to be the thing that made the performance feel alive.
What this means for designing the next one
The paper’s own recommendation for future technology-mediated audience participation (TMAP) systems is specific: prioritize human-centered, highly observable interactions to build a real sense of agency, and combine the expressive range of audio control with the immediate legibility of visual feedback rather than leaning on either alone. For anyone building participatory AI/MR performance work, that’s a genuinely actionable design principle, not just a research footnote — observability is a tunable, designable property of an interface, not an inherent fact about whatever medium you’re working in.
Related Reading
- CrowdForge: Effects of Audio and Visual Audience Participation Modalities on Agency, Collaboration, and Engagement in AI-Mediated MR Performances — arXiv abstract (2611.03389)
- The TMAP Framework (Turchet & Hödl, IEEE Access, 2025) — University of Trento research repository
- The Human Music Show: Crowd-Prompting as a Playable Social Interface for Generative Live Performance — NIME 2026 proceedings
- DruMR: A Feedback Driven Mixed Reality Air Drumming System