expectedwrong hindsight

The Blurry JPEG Fits in Your Pocket Now

Mixtral dropped, Mistral 7B runs on an iPhone at 6 tokens per second, and the genie is not going back in the bottle.

2 min read 288 words #ai #mistral #local-inference #llm #open-source
hindsight — nailed it

The blurry JPEG fits in your pocket and it got sharper. By 2025, quantized models on phones and laptops are routine. Ollama runs Llama 3.1 70B on an M-series Mac. The "this is the one" energy was justified — the moment local AI became normal for regular developers was exactly this moment.

I am running Mixtral — a mixture-of-experts model that trades blows with GPT-3.5-turbo — on my laptop, quantized to 4-bit, for free, with no usage limits and no content policy breathing down my neck.

I am also running Mistral 7B on my iPhone. Six tokens per second. In my pocket. Right now.

This is the one. Not a milestone on a roadmap — the actual moment. The thing that people will point to later when they explain how it all became irreversible.

Ted Chiang wrote that ChatGPT is a blurry JPEG of the web — a lossy compression of everything humans have written, capable of interpolating between ideas but not truly reasoning, not truly knowing. He meant it as a critique. It landed more like a description of an astonishing artifact. A blurry JPEG of the entire web is still the entire web, more or less, smeared and averaged and occasionally hallucinating captions for images that don't exist.

And now the blurry JPEG runs on an iPhone.

The scenario I keep thinking about: the internet implodes. The data centers go dark. Every API goes cold. You still have your phone, and on your phone you have something that knows, imperfectly, most of what mattered — medicine, history, code, recipes, the rough shape of how things work.

We are one quantization pass away from the trivial stuff too.

The genie is not large. It is not expensive. It is not locked behind a rate limit or a terms of service or a credit card. It fits in four bits times however many parameters, and right now it fits in my hand, and I genuinely don't know what comes next — only that the direction is more, smaller, faster, free.