XARA — a Multimodal Agentic System for eXtended Reality Experiment Generation — comes from Tareq Alsaleh, Thomas Zhao, Luke Guardino, Keagan Valentine, Isabel Kim and Bilal Farooq, posted to arXiv on 15 October. It addresses a bottleneck that anyone who has tried to run an XR study will recognise immediately.
Controlled experiments are how transportation research gets done. XR is an excellent medium for them: you can put a participant at a crossing that does not exist and vary one thing about it. But building that crossing requires Unity or Unreal proficiency, and the paper is blunt about the consequence — the workflow is “technically demanding and inaccessible to many domain researchers.”
So the person who knows which variable matters is not the person who can change it. Every iteration becomes a request to a developer, and the cost of a request is what stops researchers from iterating.
The architecture
XARA pairs a dual-LLM architecture with an immersive XR environment. Spoken commands are first transcribed, then interpreted into structured commands that manipulate a Unity-based scene in real time.
Splitting those into two models rather than one is the design decision doing the work. Transcription is a solved, well-benchmarked problem with a clear correctness criterion. Mapping “move the bus stop closer to the corner” onto a structured scene operation is neither — it needs the scene’s current state, the available operations, and a notion of what “the corner” refers to. Giving each stage its own model means the second one receives clean text and emits something checkable against a schema, rather than a single model being asked to do speech recognition and spatial reasoning in one pass.
The supported operations, per the paper, are object creation, spatial manipulation, animation control, and scene persistence, through a modular architecture. Scene persistence is the one that converts this from a demo into a tool: without it you have a party trick, and with it you have an experiment you can save, reload and hand to the next participant unchanged.
Interaction combines voice commands and hand gestures inside the immersive 3D environment — so you author from where the participant will be standing, which is the correct vantage for deciding whether a sightline works.
Why “from inside the headset” is the real claim
The obvious framing of XARA is natural-language scene authoring, and that is the least interesting part. LLM-to-scene-graph has been attempted many times.
The substantive claim is about where the authoring happens. The traditional pipeline separates the place you build the scene — a desktop editor, a monitor, a mouse — from the place you experience it. Every iteration is an export, a headset, a look, a note, back to the desk. That loop is slow enough that researchers make fewer passes than the design deserves, and the specific judgements that XR studies live or die on — whether an occlusion reads, whether a distance feels right, whether a participant’s gaze lands where you assumed — are exactly the judgements that cannot be made from a monitor.
Authoring in situ collapses the loop. That is the same insight behind AdapTest’s in-headset authoring of environmentally responsive interfaces, published to arXiv the same week, and the convergence is probably not coincidental: the field appears to have decided simultaneously that the desktop editor is the problem.
Where it applies, and where to be careful
The paper describes applications across behavioural travel experiments, safety studies, human factors research, and policy testing, with prototyping of driving, pedestrian and urban design scenarios.
Two cautions, both of which the paper’s framing invites rather than hides.
Lowering the barrier does not lower it uniformly. A voice interface exposes whatever operations it was built to expose. A researcher who can only say things XARA understands is working inside a constrained design space, and the shape of that space is now an uncontrolled variable in their study design. The honest version of “no Unity proficiency required” is “no Unity proficiency required for the scenarios we anticipated.”
An LLM that misinterprets a command produces a scene that is wrong rather than broken. A developer who mis-codes a crossing usually gets an error or something visibly absurd. A model that places an object two metres off, or applies a change to the wrong instance, produces a plausible scene with an undocumented difference from the intended one. For experimental work that is a validity problem, and it argues for a review step — a diff of the scene state, not just a look at it — before data collection.
This is a systems paper: it describes the architecture, the voice-to-command pipeline, and discusses potential applications. It is not a user study, and it does not yet report how often the pipeline produces the scene the researcher meant.
Sources
- XARA: A Multimodal Agentic System for eXtended Reality Experiment Generation — arXiv:2610.21067
- arXiv listing, cs.HC
- Laboratory of Innovations in Transportation (LiTrans), Toronto Metropolitan University — the arXiv listing carries no affiliations, but four of the six authors, Bilal Farooq among them, appear on this lab’s own roster
- Unity XR documentation