Skip to content

Tag

inference

2 posts

Speculative Decoding Is the Most Underrated LLM Inference Trick
Research8 min read

Speculative Decoding Is the Most Underrated LLM Inference Trick

Speculative decoding can double or quadruple LLM throughput with zero quality loss — yet most production stacks ignore it. Here's why, and when it's worth the complexity.

89 views
Read
Speculative Decoding: The Inference Trick Hiding in Plain Sight
Research7 min read

Speculative Decoding: The Inference Trick Hiding in Plain Sight

Speculative decoding promises faster LLM inference without touching model weights. The math holds up — but the production story is messier than the papers admit.

258 views
Read