{"version":"v1","site":{"name":"expectedwrong","url":"https://expectedwrong.com"},"links":{"collection":"https://expectedwrong.com/api/public/posts","rss":"https://expectedwrong.com/rss.xml","llms":"https://expectedwrong.com/llms.txt"},"post":{"slug":"1m-context-window-museum-piece","title":"The 1M Context Window Is Already a Museum Piece","subtitle":"Google announced a million tokens like it was a finish line, and they're already sprinting past it.","url":"https://expectedwrong.com/1m-context-window-museum-piece","api_url":"https://expectedwrong.com/api/public/posts/1m-context-window-museum-piece","published_at":1708430400,"published_at_iso":"2024-02-20T12:00:00.000Z","updated_at":1771537840,"updated_at_iso":"2026-02-19T21:50:40.000Z","tags":["ai","google","gemini","context-windows","predictions"],"excerpt":"Google announced a million tokens like it was a finish line, and they're already sprinting past it.","meta_description":"Google announced a million tokens like it was a finish line, and they're already sprinting past it.","reading_time_minutes":2,"word_count":264,"engagement":{"signals":0,"counterpoints":0},"body_markdown":"There's a video going around right now that is genuinely the best take I've seen on how to build new technology — what to optimize for, how to think about constraints, which metrics actually matter. Sharp. Grounded. The guy clearly thought hard about this.\n\nThe metrics he cites, as of this week, are correct.\n\nGive it a year and they're going to read the way a spec sheet bragging about 16MB of RAM reads now. Not wrong, exactly. Just a snapshot of a world that no longer exists.\n\nThis is not a knock on the video. It's more of a structural observation about what's happening right now with context windows specifically — a number that felt like science fiction six months ago is already being lapped.\n\nHere's the thing I keep turning over: Google announced Gemini 1.5 with a one-million token context window, experimental, not yet broadly in production. Everyone is waiting for the production rollout. I've been digging into the signals and I don't think that rollout is coming — not in the form people expect.\n\nI think they skip it entirely.\n\nNot because one million tokens is hard. Because ten million isn't, and there's no reason to celebrate a waypoint on the way to somewhere else.\n\nThe pattern with Google is to announce the demo, sit on the production release long enough that everyone gets frustrated, and then drop something that makes the original announcement look like a proof of concept. Which is, to be fair, exactly what it was.\n\nA million tokens felt like the horizon. It was a rest stop.","body_text":"There's a video going around right now that is genuinely the best take I've seen on how to build new technology — what to optimize for, how to think about constraints, which metrics actually matter. Sharp. Grounded. The guy clearly thought hard about this. The metrics he cites, as of this week, are correct. Give it a year and they're going to read the way a spec sheet bragging about 16MB of RAM reads now. Not wrong, exactly. Just a snapshot of a world that no longer exists. This is not a knock on the video. It's more of a structural observation about what's happening right now with context windows specifically — a number that felt like science fiction six months ago is already being lapped. Here's the thing I keep turning over: Google announced Gemini 1.5 with a one-million token context window, experimental, not yet broadly in production. Everyone is waiting for the production rollout. I've been digging into the signals and I don't think that rollout is coming — not in the form people expect. I think they skip it entirely. Not because one million tokens is hard. Because ten million isn't, and there's no reason to celebrate a waypoint on the way to somewhere else. The pattern with Google is to announce the demo, sit on the production release long enough that everyone gets frustrated, and then drop something that makes the original announcement look like a proof of concept. Which is, to be fair, exactly what it was. A million tokens felt like the horizon. It was a rest stop.","hindsight":{"verdict":"right","note":"gemini hit 2M tokens. the 'museum piece' framing was exactly right — 1M became the baseline, not the ceiling. the observation about metrics reading like '16MB of RAM' within a year was, if anything, conservative.","links":[],"at":1739980800,"at_iso":"2025-02-19T16:00:00.000Z"}}}