expectedwrong hindsight

Microsoft MIT-Licensed a Model That Runs on Your Phone and Beats GPT-3.5

Phi-3-mini is 3.8 billion parameters, fits on a device, and you can do whatever you want with it.

2 min read 279 words #ai #microsoft #open-source #small-models #phi-3
hindsight — nailed it

the observation about microsoft hedging between openai investment and open-source was prescient. phi-3, phi-4 became important small models. MIT licensing a capable model while paying billions to own the frontier was a deliberate two-lane strategy.

Microsoft dropped Phi-3-mini today — 3.8 billion parameters, MIT license, reportedly competitive with GPT-3.5 on benchmarks — and the MIT license is the part worth sitting with for a second.

This is the company that made a multi-billion dollar bet on OpenAI specifically so it could own the frontier. It embedded Copilot into every product it sells. It has been, more than almost any other company, invested in the idea that capable AI is a thing you access through a corporate API, on their terms, on their infrastructure. And then they shipped a model you can run on a phone and do literally anything with.

The 3.8B size is real. Fits in memory most people have. No server, no API key, no rate limits, no terms of service lawyer.

The performance claims — MMLU scores competitive with Mixtral 8x7B, MT-bench near GPT-3.5 — are benchmark numbers and should be held accordingly. Benchmarks are a story a model tells about itself in controlled conditions. But even discounting heavily, a model this small getting anywhere near that neighborhood is the result of the training data approach: filtered web data plus synthetic data, same lineage as Phi-1 and Phi-2, the idea being that the quality of what you train on matters more than the volume.

Which is either obviously true and the whole industry just forgot, or it's a trick that stops working at the frontier and everything above a certain capability threshold still requires drowning the model in the entire internet. Nobody knows which one it is yet.

What they did ship, today, is MIT-licensed. Build on it. Fine-tune it. Ship products. Microsoft does not get a cut. That's not nothing.