nanochat

Andrej Karpathy’s open-source reference for the full LLM training pipeline. The repo covers tokenization, pretraining, supervised fine-tuning, RLHF, evaluation, inference, and a minimal chat UI in roughly 8,000 lines of Python.

Released October 2025. MIT licensed. Past 50k GitHub stars by early 2026.

System Verdict

Pick nanochat if the goal is understanding how a ChatGPT-class system is actually built. The codebase reads end-to-end in a day. Every stage from tokenizer to RLHF is visible without wrappers hiding the mechanics.

Skip it for production anything. It is not a serving framework, not a multi-node distributed trainer, not a chatbot. Use a hosted API (Claude, ChatGPT) for deployment. Use Megatron-LM, NeMo, or Axolotl for real training workloads.

The natural companion is nanoGPT, which predates nanochat and covers pretraining only. Pick nanoGPT if the transformer loop is all that matters. Pick nanochat for the complete loop including RLHF and chat serving.

Key Facts

What it actually is

A single-repo walk-through of the LLM stack. The core library ships the tokenizer, transformer, training loop, and inference. Scripts handle each pipeline stage: pretraining on Fineweb/ClimbMix data, SFT on instruction data, RLHF, and a chat-interface demo.

The design dial is --depth. That one flag sets transformer layer count and auto-derives the rest for compute-optimal training. No hundred-parameter config files.

The GPT-2 speedrun is the headline benchmark. Reproducing 2019’s $43,000 training result costs roughly $100 in 2026 rental GPU time. That gap is the seven-year compounding of algorithmic and hardware efficiency gains.

When to pick nanochat

• Learning how language models are built. The codebase does not hide mechanics behind abstractions.
• Teaching LLM internals. Educators get a complete, citable, modern reference implementation in one repo.
• Research ablations on a small budget. Minimal baseline makes architecture experiments fast to iterate.
• Understanding what pretraining actually costs in 2026. The $100 speedrun is the clearest number in the literature.
• Companion reading to a theory course. Hugging Face and Stanford CS224N cover the math; nanochat is the working code.

When to pick something else

• Production LLM training at scale: Megatron-LM, NeMo, or Axolotl for fine-tuning. nanochat is not a distributed trainer.
• Deploying a chatbot: Claude or ChatGPT APIs. nanochat’s chat UI is a demo, not a product.
• Pretraining-only study: nanoGPT is Karpathy’s earlier repo. Smaller scope, fewer moving parts.
• Tiny LLM research with a ready-made checkpoint: TinyLlama (1.1B, fully trained). nanochat gives training code, not a usable model.
• Multimodal or MoE work: Out of scope. nanochat sticks to one well-defined text-only path.

Pricing

Prices verified 2026-04-17 via the nanochat GitHub README.

Against the alternatives

Failure modes

• Not a deployable chatbot. Models trained here are GPT-2-scale research artifacts. Quality is nowhere near a production assistant.
• Not a production training framework. No multi-node distribution, no production data pipelines, no inference safety rails.
• Hardware requirement for meaningful runs. The $100 speedrun needs an 8xH100 node. CPU and MPS paths exist but produce toy models.
• Scope is intentionally narrow. Multimodal, mixture-of-experts, and vision-language models are out of the design remit.
• Pedagogical value depends on the author. Karpathy’s commentary in release notes and videos is part of the learning loop. Without that context the code alone teaches less.
• Speedrun leaderboard implies competition the code was not built for. Community entries favor efficiency tricks that can obscure the teaching value of the default path.

Methodology

This page was produced by the aipedia.wiki editorial pipeline, an automated system that ingests vendor documentation, verifies claims against primary sources, and generates the editorial analysis shown here. No individual human wrote this review. Scoring follows the four-dimension rubric at /about/scoring/ (Utility × Value × Moat × Longevity, unweighted average). Last verified 2026-04-17 against the nanochat GitHub repo and Karpathy’s release thread.

FAQ

Is nanochat a chatbot I can use? No. The repo includes a minimal chat interface as an inference demo. Models trained with it are GPT-2-scale, not production assistants. For a real chatbot, use Claude or ChatGPT.

How many lines of code is nanochat? About 8,000 across the core library and scripts (GitHub). The design goal is a codebase a competent reader can walk end-to-end in a day.

What hardware is needed? For learning and small experiments, a laptop with CPU or Apple MPS runs the code at toy scale. For the headline GPT-2 speedrun, an 8xH100 rented node costs roughly $100 for two hours of compute.

What changed vs nanoGPT? nanoGPT covers pretraining only. nanochat adds the tokenizer, SFT, RLHF, eval suite, inference, and a chat UI in the same repo. Pick nanoGPT for pretraining theory, nanochat for the complete pipeline.

Can nanochat produce a usable model? Not in the modern assistant sense. The speedrun output is a GPT-2-grade model suitable for research and teaching, not for production chat. Use it to understand how capability scales with compute, not to deploy.

Sources

• nanochat GitHub (karpathy/nanochat): README, architecture, speedrun leaderboard
• Karpathy’s nanochat release thread: design intent and scope
• nanoGPT reference: companion repo for pretraining-only study

Related

• Category: AI Research · AI Coding
• Comparisons: Connected Papers vs nanochat · Consensus vs nanochat · Elicit vs nanochat

Embed this score on your site Free. Links back.

HTML Copy

<a href="https://aipedia.wiki/tools/nanochat/" target="_blank" rel="noopener"><img src="https://aipedia.wiki/badges/nanochat.svg" alt="nanochat on aipedia.wiki" width="260" height="72" /></a>

Markdown Copy

[![nanochat on aipedia.wiki](https://aipedia.wiki/badges/nanochat.svg)](https://aipedia.wiki/tools/nanochat/)

Badge value auto-updates if the editorial score changes. Attribution via the link is required.