expectedwrong hindsight

A Council of Models to Fix What One Model Broke

Perplexity's Model Council accidentally documents the core failure mode of AI-powered web research.

2 min read 271 words #ai #perplexity #search #reliability #llm

Perplexity shipped a feature called Model Council — several frontier models deliberate over a query and vote toward a consensus answer, the idea being that what GPT-4 gets wrong, Claude might catch, and what Claude gets wrong, Gemini might catch, and so on until you've assembled enough disagreement that the truth accidentally surfaces.

Sounds familiar.

The thing is, the feature description reads less like a product announcement and more like a deposition. You are being told, in upbeat marketing language, that a single model giving you a single answer about the real world is not reliable enough to ship. The council is the hedge. The hedge is the product now.

Which means the previous product — one model, one answer, cite three blue links — was always the hedge's precondition. The thing you were using before Model Council was, structurally, a prototype for Model Council. You just weren't told that.

There is something clarifying about this if you let it land. The entire value proposition of AI-powered search has been "faster, smarter, synthesized" — not "we're going to run it by a committee and see if they agree." Committees are what you do when you don't trust the witness. Perplexity is now building a formal process for not trusting its own witnesses.

The voting mechanism is clever. The irony is that the vote's existence is the most informative data point in the whole system — more informative than any individual answer — because it tells you the individual answers cannot be trusted to settle the question on their own.

You would think this would be embarrassing. It's not. It's just shipped.