Research & Innovation

Getting Started With Nerfstudio: Turn a Phone Video Into a Navigable 3D Scene

The open-source, collaboration-friendly framework for NeRFs and radiance fields makes neural 3D capture approachable — here's the mental model, the capture-to-render pipeline, and the realistic hardware you'll need.

Nerfstudio is an open-source, collaboration-friendly framework — started by Berkeley students in 2022 — that makes NeRFs (Neural Radiance Fields) and related radiance-field methods approachable to people who aren’t graphics researchers. The payoff is striking: from an ordinary set of photos or a phone video, you get a photorealistic, view-dependent 3D scene you can fly a virtual camera through, complete with reflections and fine detail that traditional photogrammetry meshes struggle to capture. This is a getting-started orientation to the workflow and what it realistically takes to run.

Watch: Nerfstudio Viewer Tutorial (YouTube)

The mental model: capture, process, train, view

A NeRF pipeline has four stages, and understanding them prevents most beginner confusion. Capture: shoot many overlapping images of a subject from lots of angles — a slow, steady video orbit works well. Process: software estimates where each photo’s camera was in space (Nerfstudio uses tools like COLMAP for this camera-pose step). Train: the model learns a continuous representation of the scene’s color and density from those posed images. View: you explore the trained scene in Nerfstudio’s interactive viewer, moving a camera freely through it. Good captures are the whole game — blurry, sparse, or same-angle footage produces mush no amount of training will fix.

The honest hardware reality

The catch newcomers hit fastest: Nerfstudio effectively needs an NVIDIA GPU with CUDA to train in reasonable time, and installation involves a Python environment plus dependencies. This is the least beginner-friendly part, and it’s worth being clear-eyed about before you start. If you don’t have a suitable local GPU, running it in a cloud notebook (a Colab-style environment with a GPU) is the common workaround. Budget your first session for setup rather than results — getting the environment working is the real hurdle; once it runs, retraining on new captures is straightforward.

Capture tips that make or break it

Since capture quality dominates the outcome, a few rules pay off immediately: move slowly and keep the subject well and evenly lit; get lots of overlap and many viewpoints, including different heights; avoid moving objects, reflective or transparent surfaces, and changing light during the shoot. Start with a small, static, textured object on a table rather than a whole room — a contained first capture that trains quickly teaches the pipeline far better than an ambitious scene that fails for reasons you can’t yet diagnose.

Where it leads

NeRFs sit alongside Gaussian splatting and photogrammetry as complementary routes to 3D capture, and Nerfstudio increasingly supports splatting methods too, making it a good single home for experimenting across techniques. For the research-minded creators this site follows, it’s the most accessible doorway into neural 3D capture — a field moving fast enough that the friendly, open framework is often the best place to keep up. Get one clean capture working end to end first; everything more ambitious builds on that same loop.