The 1M Context Window Is Already a Museum Piece
Google announced a million tokens like it was a finish line, and they're already sprinting past it.
gemini hit 2M tokens. the 'museum piece' framing was exactly right — 1M became the baseline, not the ceiling. the observation about metrics reading like '16MB of RAM' within a year was, if anything, conservative.
There's a video going around right now that is genuinely the best take I've seen on how to build new technology — what to optimize for, how to think about constraints, which metrics actually matter. Sharp. Grounded. The guy clearly thought hard about this.
The metrics he cites, as of this week, are correct.
Give it a year and they're going to read the way a spec sheet bragging about 16MB of RAM reads now. Not wrong, exactly. Just a snapshot of a world that no longer exists.
This is not a knock on the video. It's more of a structural observation about what's happening right now with context windows specifically — a number that felt like science fiction six months ago is already being lapped.
Here's the thing I keep turning over: Google announced Gemini 1.5 with a one-million token context window, experimental, not yet broadly in production. Everyone is waiting for the production rollout. I've been digging into the signals and I don't think that rollout is coming — not in the form people expect.
I think they skip it entirely.
Not because one million tokens is hard. Because ten million isn't, and there's no reason to celebrate a waypoint on the way to somewhere else.
The pattern with Google is to announce the demo, sit on the production release long enough that everyone gets frustrated, and then drop something that makes the original announcement look like a proof of concept. Which is, to be fair, exactly what it was.
A million tokens felt like the horizon. It was a rest stop.
Counterpoints
Push back, extend the argument, or sharpen it. New counterpoints go through review before they show up here.
No approved counterpoints yet.