Karpathy Finished It

Andrej Karpathy put nanochat on GitHub today and the running joke is that there is simply no other resource on the internet — not a textbook, not a course, not a research lab's internal onboarding doc — that gets you closer to complete understanding of how a large language model actually works.

nanoGPT was pretraining. That was already generously complete. nanochat is the full thing — instruction tuning, RLHF, the whole chat stack — implemented end-to-end in a codebase small enough to hold in your head.

Pair it with the ten-video playlist and you have something that shouldn't exist at this price, which is zero.

He calls nanochat the capstone project for his LLM101 course, which implies there's a course, which implies there's a curriculum, which implies he's been quietly assembling the most dangerous free education in machine learning while everyone else was charging $2,000 for a bootcamp that covers the same ground worse.

The Eureka angle is sitting right there, unspoken. He's not doing this purely for the love of pedagogy — or maybe he is, it's genuinely unclear with him — but whatever he's building at Eureka, he's now published the exact intellectual foundation you'd need to follow along. The moat isn't the knowledge anymore. He gave the knowledge away. The moat is something else.

What I keep coming back to is the specificity of the thing. Not "here's how transformers work conceptually." Not an Illustrated Guide with color-coded arrows. Actual code. Actual decisions made. Actual tradeoffs explained in the commit history and the comments and the shape of the thing itself.

Most explanations of LLMs are explanations of explanations. nanochat is the object itself, made small enough to see.