The Wall Everyone Pretended Wasn't There

Reuters ran a piece today — citing industry sources, the usual anonymous tipsters — confirming that the current approach to making AI smarter is hitting limitations, and that OpenAI and its competitors are now looking for a new path forward.

This is the kind of sentence that would have gotten you dismissed as a doomer eighteen months ago.

The specific claim isn't that transformers are broken or that scaling is fake. It's narrower and more damning: training runs that were supposed to produce the next leap are coming back disappointing. You pour in the compute, you pour in the data, and the curve that used to bend steeply upward starts to flatten — not dramatically, not catastrophically, just enough to matter, just enough that the models you're training for the future don't quite look like the future anymore.

The industry's response has been to pivot to inference-time compute, which is the current euphemism for "think harder at runtime." OpenAI released o1 in September, which is essentially a model trained to reason through problems step by step before answering — spending more compute per query rather than more compute per training run. It works. On certain benchmarks it works impressively. The question nobody has fully answered is whether "spend ten times more compute to answer one question" is a business model.

The data wall is the other half of this. The internet has a finite amount of high-quality text in it, and the models have mostly eaten it. What's left is either low-quality, paywalled, or already in the training set. Synthetic data is the proposed solution — have the model generate its own training data — which has a certain elegant circularity to it, and also the faint smell of a perpetual motion machine.

Nobody is saying the project is over. They're saying the specific tactic of "larger model, more data, more GPUs, smarter model" is producing diminishing returns, and that the next few years will be about finding what comes after that. New architectures, inference scaling, RL from human feedback pushed further than it's been pushed before.

What's quietly remarkable is that this was always predictable. The scaling hypothesis was always an empirical observation, not a physical law — it described what had happened, not what must happen. The surprise isn't that it's hitting limits. The surprise is that it held as long as it did, and that the industry bet so heavily on it continuing indefinitely that "it might not continue" became a thing you said quietly, to people you trusted.

Now it's in Reuters.