A point cloud is the least glamorous-sounding file format in computer graphics. It is just that: a cloud of points, each one a tiny measurement in space, with no surface, no color unless you add it, and no obvious shape until enough of them pile up. Surveyors have used point clouds since the 1990s to measure buildings and terrain. Nobody designed the format with artists in mind.
And yet it’s become one of the more interesting raw materials in creative technology — the thing LiDAR rigs, phone cameras, and now AI models all converge on producing, each from a different angle. This week’s “What are we talking about?” traces how that happened, and where it’s going next.
From Michelangelo’s David to a phone in your pocket
The clearest origin story for point clouds as an artistic tool, rather than a purely technical one, is the Digital Michelangelo Project, which Stanford’s Marc Levoy began in 1998. A team of roughly thirty people spent the 1998–99 academic year in Italy, running a motorized laser rangefinder over Michelangelo’s statues — including David — and 1,163 fragments of the ancient Roman city map known as the Forma Urbis Romae. The resulting models ran from 100 million to 2 billion polygons. The full-resolution David scan wasn’t finished until 2009, a full decade after the scanning started, and the data later fed into the statue’s own conservation. It was never conceived as an art piece — it was conservation science — but it proved that a point cloud could capture a masterpiece at a resolution no photograph could match, and that idea didn’t stay in the lab.
What kept the format out of most artists’ hands for another decade was cost. A laser scanner capable of Stanford’s resolution was (and still is) expensive, specialized equipment. The thing that actually put point clouds within reach of anyone with a camera was photogrammetry — reconstructing 3D geometry from a stack of ordinary photos, rather than a laser at all. Autodesk’s 123D Catch (originally shown under the codename “Photofly” in 2009) made this free and cloud-based: upload 20–40 phone photos, get a rough 3D model back. It shut down in 2016–17, but by then Agisoft’s PhotoScan (now Metashape) and a wave of open-source alternatives had already shown that the barrier to entry wasn’t really coming back up. We wrote a from-scratch guide to doing this with just a phone, and a roundup of the open-source tools worth wrapping into one GUI — the same democratization story, playing out a second time, in software this time rather than hardware.
Gaussian splatting: point clouds that paint themselves in
The newest chapter started with a single 2023 paper. 3D Gaussian Splatting for Real-Time Radiance Field Rendering, by Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler and George Drettakis — published in ACM Transactions on Graphics and presented at SIGGRAPH 2023 — starts from exactly the kind of point cloud that Structure-from-Motion photogrammetry already produces, then does something different with each point: instead of rendering it as a hard dot, it turns it into a soft, blurry “splat” with its own position, shape, color and transparency. fxguide’s technical rundown describes the practical effect well — a raw point cloud looks like a pointillist painting that falls apart up close, while a field of overlapping Gaussian splats reads more like Impressionist brushwork, because the splats blend into each other instead of leaving gaps. Magnopus’s explainer adds the practical angle: splats render in real time at a speed neural radiance fields (NeRFs) never managed, which is why splatting displaced NeRFs as the default technique within about a year of that first paper, in plugins for Unity, Unreal, Three.js, and a growing list of VR/AR toolkits.
We’ve tracked that adoption curve closely:
- Where to actually start with 3D Gaussian Splatting — our own from-zero guide
- Three.js r186 turns Gaussian splats into first-class scene objects
- A WebXR tutorial for putting a splat capture into a headset, in a browser
- SplatXR wants splats to be first-class citizens in mixed reality, not an afterthought
- ChronoFuseGS merges splats captured on different days and tracks what changed between them
- A factoring-tree method that lets you set a memory budget and trains a splat to fit it
- Insta360 put splat capture directly into a consumer 360 camera app
- Getting started with Nerfstudio, for turning a phone video into a navigable 3D scene
The fact that half of that list is tooling and tutorials, not research papers, is itself the story: two years after the SIGGRAPH paper, splatting isn’t a lab technique anymore. It’s a format artists are expected to know how to open.
Artists scanning the world around them
The most interesting work in this space treats scanning itself as the creative act, not just a step before “real” modeling begins. Simon De Diesbach laser-scanned a forest he used to run through as a teenager, then built a piece around running through the scan instead — the point cloud isn’t a means to an end, it is the artwork’s memory of a specific place. Artur Artinian sculpted a single basalt pillar, then scanned it to stand in for an entire canyon of columns, using the scan to multiply one physical object into a landscape. And when a VR studio scanned Thailand’s Tham Luang cave — the site of the 2018 youth-football-team rescue — they deliberately refused to turn the capture into a game, treating the point cloud as closer to a document than an asset.
Outside our own coverage, two public sculptures literally named Point Cloud bookend this idea nicely. In 2020, Leo Villareal used mobile 3D scanning to map the inside and outside of the ASU Art Museum’s 1987 building, then shaped over 200 million resulting data points into Point Cloud (ASU), an LED sculpture built from the scan of the very building it hangs in. Six years later and half a world away, French artist Vincent Leroy’s Point Cloud is a suspended field of translucent discs — not a literal scan, but a physical field of individually-moving “points” responding to wind and light — on display at Beijing’s Taikoo Li Sanlitun through October 2026. Neither artist is in dialogue with the other, as far as we can tell; they just arrived at the same name for the same underlying idea, a decade of hardware apart.
Closer to data-as-art in the literal sense, ScanLAB Projects’ Desert Pulse turns months of LiDAR documentation of the Sonoran Desert, captured by architects Matt Shaw and Will Trossell, into a piece for Phoenix’s Desert Botanical Garden built specifically to make climate-driven habitat shift visible — using the scan’s precision as the argument, not just its texture.
What a point cloud lets you build next
A raw point cloud is also increasingly a starting material rather than a finished look, something other tools reach into and reorganize. Open3D running inside TouchDesigner gives installation artists registration, meshing and segmentation on a scanned cloud for free, and TouchDesigner’s own new POP operator family — its first new operator family in over a decade — is built to push large point sets through the GPU in real time. On the reconstruction side, PrimitiveCAD reconstructs an actual parametric CAD model from a point cloud, rather than the undifferentiated mesh most scanning pipelines hand you — useful the moment you want to edit what you scanned, not just display it.
The newest twist skips scanning altogether. World Labs’ Marble generates an explorable 3D scene from nothing but a text sentence, and the earlier version of the same research turned a single still image into a walkable 3D scene — both effectively hallucinating the kind of spatial data a scan would have produced, skipping the capture step entirely. Niantic has bet its whole post-Pokémon-GO identity on the opposite approach: real, crowdsourced capture at scale. When Niantic sold its gaming portfolio — Pokémon GO included — to Scopely for $3.5 billion in March 2025, it deliberately kept Scaniverse, its free LiDAR and Gaussian-splatting scanning app, spinning the geospatial side out as Niantic Spatial with a reported initial investment in the hundreds of millions. The pitch, as one recent industry writeup put it, is that Scaniverse’s hundreds of thousands of monthly scanners are quietly building the training data for a “Large Geospatial Model” — the same crowdsourced-point-cloud logic that built Street View, now aimed at a format AI systems can actually read.
Scanning your own body, not just the world, is also an active research thread: TAP3D reconstructs a dense 3D point cloud of a person using nothing but a single cheap thermal camera array, no LiDAR or RGB camera required.
The open research questions
Point clouds and splats look finished, but how we actually use them in 3D and XR is still an open problem. A recent study found VR was no more accurate than a plain mouse at picking out individual points in a 3D point cloud — it was just 23% faster, and scored far higher on usability, which complicates the assumption that headsets are automatically the better interface for this data. Another found that a Gaussian splat that looks great on a flat screen can fall apart the moment you view it in stereo — a reminder that splatting’s photorealism is often judged on a 2D render, not the headset it’s increasingly destined for. Our mid-October research roundup rounded up several more papers chipping away at these same open edges — how you judge a generated world, how you fake physics inside one, how you control noise over time within a scene.
Where this leaves things
Point clouds spent most of their history as a measurement format nobody outside surveying and engineering thought about. What changed is that three separate paths — cheaper capture hardware, phone-based photogrammetry, and now Gaussian splatting plus AI world models — all converge on producing, or faking, the same kind of data: a cloud of points standing in for a place. Artists picked up on that convergence faster than the engineering world expected, and the tooling (TouchDesigner’s new operators, browser-based splat viewers, point-cloud-to-CAD reconstruction) is increasingly built with them in mind rather than retrofitted for them after the fact. The next edition of “What are we talking about?” lands next Monday, 9 November — if you have a topic you want us to dig into, it goes straight into the queue.