Why Small Models Are Eating Their Teachers

The number that should have been bigger

DeepSeek-R1 was released in January 2025 with reasoning performance comparable to OpenAI's o1, at roughly 1/30th the inference cost. The number that mattered most in the technical report wasn't the benchmark — it was the size of the distilled variants. A 7B model that could reason at the level of GPT-4. A 1.5B model that could clear most undergraduate math.

The implication, which the AI industry spent the next twelve months absorbing, was simple: the things big models do that we care about are distillable. You don't need 700 billion parameters to do them. You need 700 billion parameters to discover them, then a much smaller model can execute them.

The mechanism, in plain terms

Distillation works because most of what a large model does is unnecessary. Inside a 70B parameter model, the parameters that fire for any given problem are a small subset. The rest are storing breadth — capabilities the model has but rarely uses for a given task. A focused 7B model trained to imitate the large model on a specific distribution of tasks can match or exceed it on those tasks, because the small model's parameters all work on the problem.

The 2024-era distillation was crude — train the small model to predict the large model's outputs. The 2025-2026 generation is much more refined: distill the reasoning traces, not just the final answers. Have the small model learn how the large model thought, then learn to think the same way faster.

The result, quietly, is that the gap between frontier-model performance and 7B-class performance has collapsed for the tasks that have plenty of training data. Coding. Math. Translation. Summarization. Customer support. Form classification. The boring, valuable middle of the AI economy.

What this does to the deployment story

The cost difference between running a 70B model and a 7B model is roughly 10-30x in inference time and roughly 50-100x in dollars per million tokens. That ratio is now decisive in a way it wasn't two years ago, because the quality difference no longer covers it.

For a high-traffic application — the back end of a customer support tool, an internal AI for a 50K-person company, a code-review bot running on every PR — the math now strongly favors small models. Spend the engineering effort on distilling for your domain rather than on calling the frontier model.

The exception, which is real, is the long tail. Anything genuinely novel — a question the model has never seen anything like — still benefits from frontier-model breadth. The 7B model knows the playbook for the cases it was trained on; the 70B model has options for cases it wasn't.

What this does to the model economy

The frontier-model providers (OpenAI, Anthropic, Google) all noticed. Every one of them now ships a "small" tier alongside the flagship — Haiku, Mini, Flash, Nano. The small models are no longer a feature concession. They're the volume product, and they're getting better faster than the flagship is.

This is reshaping the business in subtle ways. Frontier model R&D is still where the breakthroughs come from, but it's increasingly subsidized by smaller-tier inference revenue. The economics of "build the best model and charge for it" are giving way to "build the best model so you can distill the cheapest model and charge for that."

The open-source landscape moved even faster. Llama 4, Mistral Medium 3, Qwen 3, Phi-4 — all are 7B-to-30B class models that match closed-source frontier from 12-18 months prior. (DeepSeek-V3 is a separate 671B-param frontier-class teacher, often confused with the smaller distills it spawned.) The lag has narrowed to under a year. For most production applications, "we run our own model" is now a real option.

What this means for your stack

If you're building AI features in 2026 and you're calling Opus or GPT-5 for everything, you're paying for capability you're not using. The pattern that's working:

Use frontier for the hard 5%. The genuinely ambiguous queries, the edge cases, the unfamiliar domains. The model picks itself, or you route based on a small classifier.

Use small for the routine 95%. Distill if you have proprietary data; use an off-the-shelf small model otherwise. The savings are real and they compound.

Maintain a baseline eval. The thing that determines whether the small model is good enough isn't its parameter count — it's how it scores on your eval. Build the eval first, pick the model second.

The teacher-student inversion

The original story of AI was: we'll build bigger models, and they'll be smarter. The actual story turning out to be: we'll build bigger models, learn what good thinking looks like, then teach smaller models to do it. The frontier exists to be distilled. The 70B model's job is to make the 7B model good.

That's a strange place to land, but it's where we are. The students are eating the teachers. The teachers are mostly fine with it.