Skip to content

Tag

Google Colab

2 posts

Fine-Tune a Small LLM: Add a New Character and Measure the Forgetting
e=mc²9 min read

Fine-Tune a Small LLM: Add a New Character and Measure the Forgetting

Lesson 04: fine-tune the 5.9M-parameter Tiny Storyteller on 300 stories about a new character. Its old quiz perplexity jumps from 9.09 to 156. Mixing two old windows into every batch of twenty holds it at 11.75. Real numbers, and what the model got wrong.

2 views
Read
Build an LLM From Scratch on a Free GPU in 5 Minutes: Tiny Storyteller
e=mc²9 min read

Build an LLM From Scratch on a Free GPU in 5 Minutes: Tiny Storyteller

Lesson 01: a 5.9-million-parameter GPT trained from random weights on children's stories: 1,500 steps in 1.2 minutes of training on a free Colab GPU (the whole notebook is about 5 minutes). Every piece explained through Momo the monkey and Ella the elephant, with the real numbers and an honest look at what it wrote.

2 views
Read