Here is a bug that is in an enormous amount of creative code, including probably some of yours: treating a 360° image as a rectangle.
An equirectangular panorama is stored as a flat image, so every tool treats it as one. You blur it, you convolve it, you run a neural network on it, you resize it. And all of those operations are wrong, in a specific and predictable way: the top and bottom rows of an equirectangular image represent a tiny area of actual sphere stretched across the full width. A 3×3 convolution kernel near the pole covers a vastly different solid angle than the same kernel at the equator. The left and right edges are the same meridian and your kernel does not know it.
torch-harmonics — paper posted 30 September 2026 — is a PyTorch library for doing this properly.
What it provides
Efficient, differentiable implementations of advanced signal processing and machine learning methods for spherical data.
Specifically:
- Spherical harmonic transforms — the sphere’s equivalent of the Fourier transform
- Vector spherical harmonics — for fields that have direction, not just magnitude, at each point
- Discrete-continuous and spectral convolutions — convolution defined on the sphere rather than on a grid
- Spherical attention mechanisms
enabling “scalable, rotationally-aware learning and inference.”
Application domains listed include geophysics, planetary science, geodesy, atmospheric physics, quantum chemistry, cosmology — and virtual reality.
What spherical harmonics actually are, briefly
The Fourier transform decomposes a signal on a line or a plane into sines and cosines. Spherical harmonics do the same job on the surface of a sphere — they are the natural orthogonal basis functions there, indexed by a degree ℓ and an order m.
Low-degree harmonics are smooth and large-scale: ℓ=0 is a constant, ℓ=1 is a simple directional gradient, and so on. High degrees capture fine detail. Truncate the series and you get a smooth, band-limited version of your spherical function — which is exactly the property that makes them useful.
Rotationally aware is the critical phrase. Rotate a function on the sphere and its spherical-harmonic coefficients transform in a clean, predictable way within each degree. Rotate an equirectangular image and the pixel grid becomes nonsense. An operation built on harmonics behaves consistently regardless of where “up” is; an operation built on the image grid does not.
Where you have already met them without noticing
In 3D Gaussian splatting. Each splat’s view-dependent colour is stored as spherical harmonic coefficients — usually degree 3, which is 16 coefficients per colour channel. That is why splat files are large and why “SH degree” is a quality knob in every splat trainer.
In ambisonic audio. First-order ambisonics is literally four channels corresponding to the ℓ=0 and ℓ=1 spherical harmonics (W, X, Y, Z). Higher-order ambisonics adds higher degrees. Rotating an ambisonic soundfield to follow a listener’s head is a spherical harmonic rotation, and it is why head-tracked VR audio works at all.
In real-time graphics lighting. Irradiance environment maps are commonly stored as nine SH coefficients — a trick from Ramamoorthi and Hanrahan’s 2001 paper that is in essentially every game engine. It is how you get soft ambient lighting from an environment map for almost no cost.
In climate and weather models, which is where the heavy engineering in libraries like this comes from, and why they are fast.
Why a differentiable implementation matters
Because differentiable means you can train through it.
Everything above is normally a fixed preprocessing step: encode to SH, do something, decode. A differentiable spherical transform means the sphere’s geometry can sit inside a learned pipeline, with gradients flowing through it.
Concretely, for creative work:
360° video and image models that are not broken at the poles. A spherical convolution treats the top of the frame correctly. Any network trained on equirectangular images with ordinary 2D convolutions has learned to compensate for a distortion that didn’t need to exist, and it still fails on content near the poles.
Learned ambisonic processing. Spatial audio upmixing, soundfield denoising, or learned head-related transfer functions, with the soundfield’s rotational structure built into the architecture rather than hoped for.
Splat-adjacent work. If your pipeline touches SH coefficients — and splatting pipelines do — having a fast differentiable implementation of the transforms and rotations is directly useful.
Environment map and lighting work where you want to learn a mapping between environments, or predict lighting from an image, without the distortion of a flat representation.
Anything with a dome. Planetarium and fulldome work is spherical by definition, and it is a domain where people routinely fight equirectangular artifacts by hand.
The practical note
It is PyTorch, it is open (the paper is CC BY 4.0), and the torch- naming convention means it behaves like a normal set of modules you drop into a model. NVIDIA has maintained a torch-harmonics package for a while in the weather-modelling context; this paper appears to be the formal write-up of the library’s current, broader scope.
The honest caveat: spherical harmonics are band-limited by construction, so they are smooth and they are poor at representing sharp discontinuities — an edge on a sphere needs very high degree to resolve, which is expensive. If your spherical data is mostly hard edges, harmonics are the wrong basis and a different spherical representation (HEALPix, icosahedral meshes, cubemaps) may serve better. Know which problem you have.