Almost every XR project begins with a decision nobody revisits: is this AR or VR? It gets made early, for practical reasons — the hardware, the budget, the venue — and then the entire piece is built inside it.
A study posted 25 September 2026 treats that decision as a variable instead. The authors are Carl Tornberg, Alicia Torck, Lotfi El Hafi and Tadahiro Taniguchi — overlapping with the XR pen team whose work we covered this morning, which tells you this group is systematically interrogating XR input and modality rather than assuming either.
The framing
Their context is Cybernetic Avatars — robotic embodiments proposed to extend human capabilities as populations age and labour shortages intensify — which require effective human-robot interaction frameworks. XR offers those interfaces.
The gap they identify is the interesting part: prior research typically fixes the XR modality without evaluating its effect on task outcomes.
So the study asks two questions:
- Does the XR modality affect HRI performance at all?
- Does an adaptive interface, adjusting the level of virtuality along the Reality-Virtuality Continuum at runtime, improve it?
What the Reality-Virtuality Continuum is, and why it’s underused
Milgram and Kishino’s Reality-Virtuality Continuum (1994) is one of those frameworks everyone cites and few people use as intended. It places experiences on a spectrum from entirely real through Augmented Reality, then Augmented Virtuality (a virtual environment with real elements brought in), to entirely Virtual.
The useful implication is that these aren’t three product categories — they’re positions on a dial. In practice the industry has hardened them into categories, because a device is either an AR headset or a VR headset and an app is built for one.
Passthrough headsets have quietly dissolved that. A Quest or Vision Pro can be anywhere on the continuum, moment to moment, under software control. The hardware stopped forcing the choice, and most software hasn’t noticed.
Why an adaptive interface is a plausible idea
Because different sub-tasks want different amounts of reality, and a single setting compromises all of them.
Consider operating a robot in a real room. Navigating and avoiding obstacles wants a high proportion of reality — you need to see the actual floor, the actual furniture, the actual person walking through. Inspecting the robot’s plan or its sensor data wants virtuality — you want a clean abstract view uncluttered by the room. Precise manipulation may want AR with heavy overlay.
A fixed modality means one of those is always badly served. An interface that slides toward reality when you’re navigating and toward virtuality when you’re planning could serve both — if the transitions don’t themselves cost more than they save, which is the empirical question worth running.
What to take from it for creative work
The modality is a design variable, and you should test it rather than inherit it. If you’re building a spatial piece and you chose AR because the headset does passthrough, that’s a hardware decision masquerading as an artistic one.
Transitions are content. Moving a viewer along the continuum during a piece — starting in their real room and gradually replacing it, or pulling the room back in at a climactic moment — is a dramatic device available on current hardware and very rarely used. Immersive work overwhelmingly picks a level and stays there.
The transition cost is real and worth respecting. This is where the two papers from this group connect usefully. We covered research last week finding that video passthrough suppresses head movement four-fold, and that what a headset captures isn’t what the wearer can see. Changing modality mid-task means changing the perceptual conditions mid-task, and users re-adapt each time. Adaptive isn’t free, which is precisely why someone should measure it.
Related Reading
- Evaluating the Impact of Adaptive Extended Reality on Human-Robot Interaction Across the Reality-Virtuality Continuum — arXiv:2609.31138
- Comparative Evaluation of an XR Pen-based Control Interface — arXiv:2609.31117
- Passthrough Rigidity: The Behavioral and Visuomotor Costs of Mediated Perception — arXiv:2609.25002
- Milgram & Kishino, A Taxonomy of Mixed Reality Visual Displays (1994)
- arXiv cs.HC — Human-Computer Interaction listings