Everything about 3D capture is optimised for the eye. Photogrammetry, Gaussian splatting, NeRFs — they all produce objects that look right and have nothing else. Reach for one in VR and there is no texture under your fingers and no temperature at all.
TouchTherm: Building Multimodal Digital Twins of Objects for Tactile and Thermal Rendering, posted 1 October 2026 and revised the 2nd, captures both. The authors are Yitao Zhang, Hong Ying, Haoran Guo, Xiaoying Zhou, Guanyu Chen and Chenxi Xiao.
Their diagnosis of the gap is precise: existing 3D datasets and reconstruction methods “primarily represent object-scale geometry and visual appearance, overlooking microscale surface structure” — and thermal dynamics entirely.
The tactile half
Structured-light scanning combined with photometric stereo to obtain multiview normal maps. Those are registered to the scanned geometry and converted into tangent-space micro-height fields for optical tactile rendering, while a coarse mesh handles collision detection.
The two-representation split is the engineering insight here, and it mirrors how graphics has always done surfaces.
Structured light gives you the shape — millimetre-scale geometry, the object’s form. It cannot resolve the texture of brushed metal or the weave of fabric.
Photometric stereo gives you the microstructure. Light the object from several known directions, measure how each pixel’s brightness changes, and you can recover the surface normal at that pixel — at camera resolution, far finer than any depth sensor. It is the same principle as a normal map in a game engine, measured rather than authored.
So: coarse mesh for where your finger collides, micro-height field for what it feels like when it gets there. Exactly the division between a collision mesh and a normal map, applied to touch.
And “optical tactile rendering” points at the display technology these are for — vision-based tactile sensors and displays in the GelSight lineage, where a deformable membrane is imaged to read or present fine surface detail.
The thermal half is the more unusual contribution
Synchronised multiview infrared videos captured during natural cooling after controlled heating, from which they reconstruct a physics-regularised dynamic thermal field.
This is a genuinely clever experimental design, and it is worth seeing why.
You cannot directly measure the properties you want. What makes an object feel cold is its thermal conductivity, its heat capacity, and its effusivity — the rate at which it draws heat out of your skin. A steel rail and a wooden bench at the same room temperature feel wildly different because steel pulls heat from your hand far faster. These are material properties and you cannot photograph them.
But you can watch their consequences. Heat the object, then film it cooling in infrared from several angles. The cooling curve encodes the material properties — a high-conductivity, high-capacity object cools differently from a low one, and spatial variation in the cooling pattern reveals spatial variation in the material.
“Physics-regularised” means the reconstruction is constrained by the heat equation rather than being a free fit to the video. Which matters enormously: it means the resulting field can predict behaviour the camera never saw — how the object responds to being touched, to a different ambient temperature, to being heated somewhere else — instead of merely replaying a recording.
Validated on 20 objects, with thermal prediction errors of roughly 0.465–0.592 °C. For reference, human thermal discrimination on the skin is around 0.2–1 °C depending on the body site and rate of change, so half a degree is in the right neighbourhood — close enough to be useful, not so precise that there is nothing left to improve.
Delivered through a glove-based VR system providing spatially and temporally varying thermal feedback.
Why temperature is the underrated channel
Haptics research is overwhelmingly about force and vibration — what something feels like when you push it. Thermal feedback gets much less attention and it may be the better investment, for three reasons:
It identifies materials almost immediately. Close your eyes and touch metal, wood, stone and plastic at room temperature. You will name all four, and you will do it on thermal cues before texture resolves. Thermal effusivity is the single most diagnostic property of a material to a bare hand.
It is cheap and quiet to actuate. A Peltier element is a small, silent, solid-state device with no moving parts. Force feedback requires motors, linkages, and either a exoskeleton or a grounded armature — all heavy, noisy and expensive.
And it is persistent rather than momentary. A vibration is an event. A temperature is a state that continues as long as you are in contact, which means it contributes to presence over time rather than only at the moment of contact.
The tradeoff is latency: thermal sensation is slow, with perception lagging the stimulus by hundreds of milliseconds, and a Peltier takes time to change temperature. So thermal feedback cannot be used for anything requiring timing — but material identity is not a timing-critical signal.
What to do with it
For anyone building XR or installation work with physical contact, the transferable idea is capture the channel you intend to render. If a piece involves touching something, the data you need is not in a photogrammetry scan, and no amount of visual fidelity substitutes.
And the Peltier point is actionable at a much lower budget than this paper: a Peltier element, a driver board and a thermistor is a forty-dollar experiment, and a surface that is genuinely cold or warm when an audience touches it is a startlingly effective and under-used material. It works in a dark room, it needs no screen, and nobody expects it.