expectedwrong hindsight

Snowflake Just Dropped a 480B Model and I Wasn't Ready

The data warehouse company apparently builds frontier LLMs now, and they gave it away.

2 min read 343 words #llms #open-source #snowflake #arctic #enterprise-ai
hindsight — evolved

snowflake arctic didn't achieve lasting impact as a model. snowflake pivoted away from frontier model training. but the observation that any company with enough data and compute could train a large model was directionally correct — it just turned out not to be worth it for a data warehouse company.

I was barely tracking that Snowflake was building a model.

That's the thing about the current moment — you look away for a week and a company whose entire identity is "we store your cold data cheaply" has quietly trained and released what may be the largest open-source LLM that exists. Snowflake Arctic. 480 billion parameters. Mixture of experts, so only 17B active at inference time, but the number on the box still says 480B and the box just landed on the table.

Apache 2.0 license. Commercial use, no strings.

That last part is the part that should actually register. We've had big open models before — but the licensing landscape has been a minefield of "open weights, closed vibes" arrangements where you could run the thing but not build a business on it without reading sixteen pages of acceptable use policy. Apache 2.0 is just: here is a model, it's yours, go make money. The model ships as freely as a numpy release.

The framing Snowflake went with is "enterprise intelligence" — SQL generation, coding, instruction following, the stuff their existing customers already pay them for. Which is a reasonable angle. They're not trying to beat GPT-4 at creative writing. They're trying to be the thing your data pipeline calls when it needs to turn a natural language question into a query. They have the distribution to make that matter.

What's strange is that they got here at all. Snowflake is a database company. A very successful, very unsexy database company that became inexplicably worth $70 billion by convincing enterprises to move their data warehouses to the cloud. The fact that this company now has a research team capable of training and releasing a 480B parameter model — and apparently did it efficiently enough to be proud of the training cost — is one of those data points that suggests the "AI is only for the big four" narrative is already outdated.

The model is out. The weights are downloadable. The license is clean.

I missed the announcement by almost a day.