expectedwrong hindsight

Frogs Can't Walk on Water

Dream Machine dropped and the benchmark that matters is immediate.

2 min read 372 words #ai-video #dream-machine #luma #generative-ai #benchmarks
hindsight — still happening

physics failures in AI video remain the tell. world models still don't fully understand physics — frogs still walk on water, objects still phase through each other. the 'eight fingers of AI video' framing was perfect. world models are getting closer but aren't there yet.

Luma dropped Dream Machine today and the first thing anyone did was generate frogs.

This is correct behavior.

The video quality is genuinely A+. The motion is fluid, the light is beautiful, the texture on things is better than it has any right to be. And then a frog walks across the surface of a pond like it's Jesus Christ of the lily pad, and you remember where we are.

Frogs walking on water is the eight fingers of AI video. It's the tell. The model knows "frog" and it knows "pond" and it has seen enough footage to render something that looks, at a glance, like a nature documentary — and then the physics just opts out. The frog strolls. The water holds. Nobody told the model that this is not how frogs work, or how water works, or how anything works.

Sora is apparently dropping any minute now, according to the general vibe. It will also generate frogs that can walk on water, probably with better lighting.

Someone tried "a small frog tries to jump out of a pond, but a fly knocks him out of the air" and it almost got it. Almost. There's something in there about the model being better at stories with a clear physical premise — give it a protagonist with a goal and an obstacle and it has something to organize around. "Frog in pond" is just vibes. "Frog vs. fly" is a narrative.

The other one that's haunting me: a raindrop falling toward a flower, and somewhere in the model's dream of what this should look like, the raindrop is trying to water the flower on the way down. Intentionally. The raindrop has a mission.

The model doesn't know what a raindrop is. It knows what a raindrop looks like in motion, it knows what watering a flower looks like, and when you put them together it outputs a tiny water droplet with apparent purpose. Cheese walking around on chicken legs. Hyperreal VFX of a thing that has never existed and cannot exist.

This is where we are in June 2024. The images are stunning. The physics are a polite suggestion. The frogs walk on water, and the raindrops are trying their best.