A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply

A 1.7B model claims native million-token context, on-device

On September 1, 2026, iFLYTEK's subsidiary Ciyuan Xinghuo open-sourced two edge-side language models, Xinghuo (Spark) X2.5-4B and X2.5-1.7B, claiming they are the first on-device models to natively support context windows up to 1,000,000 tokens. Weights, code, and deployment docs went up on Hugging Face and GitHub, with support promised for llama.cpp, vLLM, SGLang, Ollama, and LM Studio, according to the team's own dev.to post and a fuller writeup on AIbase.

The 1M-token figure is self-reported, and neither post shows a phone or SBC actually holding that much context in memory — the practical RAM and latency cost of a million tokens on a 1.7B model, even quantized, is the real question nobody has answered yet with numbers.

If the claim holds up under independent testing, it matters: most small on-device models top out at 32K–128K tokens, which is fine for chat but not for a model reading a whole codebase or a stack of documents locally.

A 14MB model that only does one thing well

Cactus Compute has released Needle 2, a 45-million-parameter model built solely for tool calling, packaged with its tokenizer and a C++ inference engine into a single 14MB binary that runs a session in roughly 28MB of RAM, per a detailed writeup on PyShine. The developer's own benchmarks show it trading wins with models 5x to 70x larger, including FunctionGemma 270M and LFM2.5 230M, while running at 2 bits per weight.

Those comparisons are self-reported by the model's own team and cover a narrow benchmark set focused specifically on function calling, not general reasoning or chat — this is a specialist, not a small chatbot.

The target list is the interesting part: watchOS, tvOS, WebAssembly, and even RISC-V and MIPS, meaning the pitch is tool-calling on a smartwatch without phone tethering, a use case most LLM releases don't bother targeting.

An open voice-agent LLM you can run yourself

The team behind Pipecat, the open-source voice-agent framework maintained by Daily.co, has released PhoneLLM Alpha 1, an open-weights small model trained specifically for the low-latency, multi-turn dialogue that phone-call agents need. The announcement says it performs on par with a much larger commercial model at 94% lower cost and a faster time-to-first-token, ships under a BSD license, and can be self-hosted with no commercial restrictions.

Those latency and cost numbers come from Daily's own benchmarking against their comparison model, not a third party, and the release is meant to sit behind a transcription and text-to-speech stack rather than run standalone on a phone.

It still matters for the edge beat: an open, self-hostable brain for voice agents means builders of local intercoms, kiosks, and offline call-handling gear don't have to route every conversation through someone else's API.

A fully offline voice-and-RAG tool climbs GitHub's trending list

RunanywhereAI's RCLI, described by its author as letting you "talk to your Mac, query your docs, no cloud required," surfaced on GitHub's trending list on September 1, 2026 with over 1,500 stars, per the project's repo. It bundles on-device voice input with retrieval-augmented Q&A over local files, aimed at Mac users who want an assistant that never leaves the machine.

There's no independent benchmark of its speech recognition accuracy or retrieval quality yet, and it leans on existing local model backends rather than shipping a new model of its own.

Still, the traction is a small data point in a bigger pattern: fully offline voice-plus-RAG stacks are no longer a niche llama.cpp hobby project, they're something ordinary Mac users are starring by the thousand.

None of these four claims come with independent verification yet — a million-token context window, a 14MB tool-caller, and a discount voice-agent brain are all still self-reported. But taken together they sketch where the small-model race is actually headed: longer context, narrower specialization, and fewer excuses to phone home.