e=mc²9 min read
Fine-Tune a Small LLM: Add a New Character and Measure the Forgetting
Lesson 04: fine-tune the 5.9M-parameter Tiny Storyteller on 300 stories about a new character. Its old quiz perplexity jumps from 9.09 to 156. Mixing two old windows into every batch of twenty holds it at 11.75. Real numbers, and what the model got wrong.
2 views
Read