expectedwrong hindsight

NVIDIA Did Text-to-4D and I Had This on My Roadmap for June

Align Your Gaussians takes a text prompt and returns a dynamic 3D scene, and December was apparently the right time for that.

2 min read 282 words #3d #diffusion #gaussian-splatting #nvidia #generative-ai
hindsight — nailed it

gaussian splatting became the dominant 3D representation. text-to-4D kept advancing. the timeline panic was warranted — everything arrived earlier than expected, and that pattern continued through 2025.

NVIDIA's Toronto AI Lab dropped Align Your Gaussians this week — text-to-4D, meaning you type a description and get back a dynamic 3D scene you can orbit and scrub through time.

Six months ago I would have said this was a 2024 H2 problem. Then again I said the same thing about photorealistic text-to-3D, and here we are.

The approach is not magic, it's just several non-obvious things stacked cleanly on top of each other. They use 3D Gaussian Splatting as the representation — which itself only went mainstream in August — and then drive it with score distillation from both image diffusion models and video diffusion models simultaneously. The video model handles the temporal dimension, the image model handles the spatial fidelity, and the Gaussians are the substrate that holds the whole thing together across frames. "Align" in the title refers to aligning those priors so they don't fight each other.

The output is what you'd expect if you'd been tracking this trajectory: a panda dancing, a fire burning, a dog shaking water off its coat — rendered as full volumetric scenes with genuine viewpoint freedom, not a flattened video playing on a billboard.

What gets me is the timing. Gaussian splatting as a research object is four months old. Video diffusion models that are good enough to guide 4D geometry are maybe six months old. Someone at NVIDIA looked at both of those things and said "those go together" fast enough to have a project page up before the year ended.

I had this in my head as a thing that would require more infrastructure to exist first. It didn't. That's the whole lesson and I keep not learning it.