Skip to content

Tag

LLM Routing

2 posts

Does It Earn a Lane? Quantize, Measure, and Decide
e=mc²9 min read

Does It Earn a Lane? Quantize, Measure, and Decide

Lesson 10: int8 quantization of the Stage C deciders, CPU latency and size, a stdlib server speaking pankhllm's decision shape, and a real run through pankhllm's own binary. The default int8 broke Qwen; the small decider got smaller and slower. And the contract, applied honestly: neither lane earns its place yet.

2 views
Read
A Language Model as a Decision Model: Train a Tiny Transformer to Pick the Operation
e=mc²9 min read

A Language Model as a Decision Model: Train a Tiny Transformer to Pick the Operation

Lesson 07: the Lesson 01 GPT, shrunk to 0.83M parameters, writes one token after a question: the operation that answers it. Beside it, pankhllm's hashed logistic regression. On phrasings neither saw, a threshold alone holds 98% precision for neither; with pankhllm's known-words guard, the transformer does, at 41.2% coverage.

2 views
Read