A 4GB NVIDIA Model Learns to Chat With Robots While Raspberry Pi Gets a Local-LLM NPU

NVIDIA's 4GB Nemotron Bolt Learns to Run the Robot, Not Just Chat

NVIDIA released Nemotron 3.5 Bolt E4B on October 23, 2026, a small language model built for voice assistants, game characters and robots rather than chatbots. It is post-trained from Google's Gemma 4 E4B with 8 billion total weights, described by NVIDIA as 4 billion "effective" parameters, and trained with reinforcement learning only — NVIDIA says supervised fine-tuning hurt conversational quality, so it skipped it entirely.

In NVIDIA's own demo, a Reachy Mini robot runs the model on a Jetson Orin Nano with 8GB of RAM, generating spoken replies and calling scripts from the robot's animation library, with a separate Cosmos 3 Edge model handling vision, according to RobotsBeat's writeup. NVIDIA reports the Q4_K_M build hits 272.6 tokens per second on an RTX 5070, per coverage citing NVIDIA's own benchmarks — self-reported numbers, not independently verified.

The caveat worth keeping: "effective parameters" is NVIDIA's own framing for an 8-billion-weight model, not a neutral measurement. Still, pairing a sub-4GB model with on-robot tool calling is a real step away from cloud-dependent voice interfaces for small bots.

A One-Click Pipeline Claims Big Speedups Getting Models Onto Jetson

Embedl published a post this week describing a pipeline that takes any Hugging Face model and optimizes it for a target Jetson board automatically. As a showcase, the company ran NVIDIA's Cosmos 3 Edge world model — which reasons over images and video and outputs robot actions — on a Jetson AGX Thor and clocked 208 tokens per second, which Embedl says is three times the 68 tokens per second NVIDIA's own Jetson AI Lab reports for the unoptimized BF16 model on the same chip.

On the cheaper 8GB Jetson Orin Nano Super, Embedl reports 152 tokens per second, 5.7 times the original FP16 model, with accuracy staying within 0.4 points of the baseline across five vision-language benchmarks.

These are Embedl's own numbers, measured against NVIDIA's own published baseline — not a third-party audit. Still, the gap between raw Hugging Face weights and what actually fits a Jetson's memory and power budget is exactly where most edge robotics projects stall, so tooling that automates that step matters even before independent verification.

llama.cpp Trims Gigabytes of VRAM Off a Big MoE Model

A pull request merged into llama.cpp this week by contributor sanmai reworks how the project selects experts for Qwen's sparse mixture-of-experts models, removing the kq_mask step from the Query-Sparse-Attention path. Contributor jacek2023 posted on r/LocalLLaMA that the change saves gigabytes of VRAM specifically on Qwen Flash Next by squeezing more out of the expert cache.

The pull request itself is a narrow engineering fix, not a new model or benchmark suite, and the VRAM savings are reported anecdotally in the thread rather than with a formal before/after table.

That said, this is the kind of unglamorous plumbing that decides whether a mixture-of-experts model runs on a 12GB card or doesn't run at all — the gap between a model existing and a model fitting on hardware someone actually owns.

Raspberry Pi's AI HAT+ 2 Adds Its Own RAM So LLMs Run Standalone

Raspberry Pi launched the AI HAT+ 2 this week at $130, pairing a Hailo 10H NPU with 8GB of its own LPDDR4X memory, according to Jeff Geerling's hands-on post. Because the HAT carries its own RAM, the Hailo 10H can run a small language model entirely on the accessory, leaving the Pi's own CPU and system memory free for everything else.

Geerling notes this is a meaningful shift from the original AI HAT+, which only accelerated vision models — this version is explicitly built to host LLM inference standalone, not just image classification.

No independent token-per-second numbers are in yet, but putting dedicated memory on a $130 accessory board is a cheap way to get a Pi 5 running a real local model without starving the rest of the system.

A $153 Pi-Sized Board Brings an RK3588S NPU and 32GB of RAM

ArmSoM's new Sige7s packs the RK3588S SoC — with a 6 TOPS NPU, Wi-Fi 6 and a Raspberry Pi-compatible 40-pin header — into a board starting at $153, with configurations running up to 32GB of RAM, per It's FOSS News' coverage. A compute-module version is also planned for custom carrier boards.

The 6 TOPS NPU is modest next to Jetson-class silicon, but the Pi-compatible footprint and header mean existing Pi HATs and cases should largely work unmodified, lowering the barrier for swapping NPU silicon into an already-built project.

No independent benchmarks of local model performance have surfaced yet for this particular board — the RK3588S itself is a known quantity from other SBCs, so expect similar quantized small-model throughput once someone runs the numbers.

Today's thread runs from a 4-gigabyte model learning to operate a tabletop robot's limbs down to an unglamorous llama.cpp patch trimming gigabytes off a mixture-of-experts cache — the usual story of edge AI progress arriving in small, mostly unverified increments.