Needle 2: 14MB On-Device LLM Brings Agentic AI to Budget Hardware

Needle 2: A 14MB Agentic LLM for the Devices Nobody's Talking About

Edge AI has a marketing problem. For most of the industry, "on-device" means running a 7B parameter model on a MacBook or a flagship smartphone. But Cactus Compute is making a different bet: that the real edge is the 21 billion connected IoT devices—budget phones, Raspberry Pis, microcontrollers, and wearables—that ship for under $200 and have no GPU, no NPU, and only a few hundred megabytes of RAM.

Today, the company released Needle 2, a 45M-parameter language model that compresses to a single 14MB binary and runs a full session in just 28MB of RAM. It's designed for one specific task: agentic tool calling. Turning on a light, sending a message, or controlling a robot doesn't require world knowledge or open-ended prose—it requires mapping a messy sentence onto a function with the right arguments. That narrow formulation is why 45M parameters can compete with models 5 to 70 times larger.

Performance That Punches Above Its Weight Class

On benchmarks like Google's Mobile Actions and Seal-Tools, Needle 2 trades wins with models like FunctionGemma 270M, LFM2.5 230M, and Apple's Foundation Model—all running at f16 precision while Needle runs at 2 bits. On Mobile Actions (961 rows), Needle 2 hits 63.7% accuracy, nearly matching LFM2.5's 69.1% while being 5x smaller. On Seal-Tools out-of-domain, Needle actually leads with 28.7% vs. LFM2.5's 17.0%.

The speed numbers are equally striking. Needle decodes at 500+ tokens/sec on a Raspberry Pi 5, between 400–1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and 300–700 tokens/sec on sub-$200 phones like the Samsung A-Series. It even runs on microcontrollers with external RAM, like the ESP32-P4.

The Architecture: Simple Attention Networks and Lossless 2-Bit Quantization

Needle 2 is built on the Simple Attention Network (SAN), a novel architecture detailed in Cactus's research paper. The key insight is replacing dense MLP projections with a fixed Walsh-Hadamard transform and learned diagonals, cutting channel mixing costs dramatically. A conventional transformer of Needle's size spends 164 MFLOPs per token; Needle spends just 70.

The model also uses a hashed n-gram memory system ("engrams") that stores world knowledge in tables read a few rows per token—capacity that's nearly free at decode time. This matters on devices where every megabyte read from flash costs battery life.

But the most interesting innovation is lossless 2-bit quantization. Most small models break under post-hoc quantization, so Cactus trained Needle against their Cactus Quants from pretrain through post-train—weights, activations, and KV cache alike. The 2-bit model you deploy is the model that was trained, not a degraded approximation.

continue reading below...

A Pragmatic Approach to Edge-Cloud Collaboration

No small model covers everything, and Needle doesn't pretend otherwise. Every response carries a learned confidence score; off-topic requests return an empty call. Above your threshold, act locally. Below it, escalate to the cloud. Most device requests are routine control, so escalation stays rare and the default path stays private, instant, and free.

This is a refreshingly honest design. Instead of hallucinating answers, Needle says "I don't know" and hands off to a larger model. It's the kind of pragmatic engineering that makes small models viable in production.

Production-Ready: Pebble's Index Ring Already Uses It

Needle isn't just a research project. Pebble, the pioneer of modern wearables, runs Needle 2 locally in its Index 01 app to turn spoken requests into actions without depending on a network connection. The Index Ring has no screen, so the action has to happen every time, with or without internet.

"We run Cactus Needle locally in the app, instead of relying on the cloud," a Pebble representative said. "The model's footprint is tiny and the performance never lets us down."

Why This Matters

The AI industry is obsessed with scaling laws and trillion-parameter models. But the vast majority of computing devices in the world are small, cheap, and power-constrained. Needle 2 demonstrates that agentic AI doesn't require frontier hardware—it just requires the right architecture and a willingness to specialize.

With Apache 2.0 licensing, weights on Hugging Face, and a GitHub repo with a single dependency-free C++ binary that runs from Cortex-M to x86 to WebAssembly, Needle 2 is positioned to become the default tool-calling model for the IoT era. The company is already talking to hardware makers about custom tool schemas and post-training.

For developers building smart home devices, wearables, robots, or automotive systems, Needle 2 offers a rare combination: tiny footprint, low latency, offline reliability, and genuine capability. It's a significant step toward making on-device AI truly ubiquitous.