Bonsai 27B: First 27B-Parameter AI Model Runs on a Phone

A New Milestone for On-Device AI

PrismML has officially launched Bonsai 27B, a 27-billion-parameter AI model that can run directly on a smartphone—a first for models of this size. The startup, which emerged from Caltech and is backed by Khosla Ventures, Cerberus, and Google, claims the model retains 90% to 95% of its full-precision intelligence while occupying as little as 3.9GB of memory. This breakthrough could fundamentally shift how developers deploy agentic AI workloads, moving them from cloud-dependent architectures to fully local execution.

The announcement, made on July 14, 2026, comes amid reports that Apple has held meetings with PrismML about potential integration. According to a report by The Information, Apple is exploring ways to use the technology for on-device AI features. Coincidentally, Apple's iOS 27 public beta, released the same day, includes new system-wide enhanced dictation and customizable Siri voices powered by a 27B-class model—widely believed to be Bonsai 27B—available on iPhone 17 Pro, iPhone 17 Pro Max, and iPhone Air.

How Bonsai 27B Works

Bonsai 27B is based on Alibaba's open-source Qwen 3.6 27B model. PrismML compresses it into two variants using extreme quantization. The Ternary Bonsai 27B uses ternary weights ({-1, 0, +1}) with FP16 group-wise scaling, achieving 1.71 effective bits per weight and a total size of 5.9GB. The 1-bit Bonsai 27B uses binary weights ({-1, +1}) with 1.125 effective bits per weight, reducing the footprint to just 3.9GB—small enough to fit within the memory constraints of an iPhone 17 Pro, which offers roughly 6GB of usable memory for apps after system overhead.

Both variants run the entire model end-to-end in low-bit precision, including embeddings, attention, MLPs, and the LM head, with no higher-precision escape hatches. They support a full 262K-token context, multimodal vision (via a compact 4-bit vision tower), speculative decoding for speed, and are released under the Apache 2.0 License.

Performance and Benchmark Results

Across a 15-benchmark suite covering knowledge, reasoning, math, coding, instruction following, tool calling, and vision, the Ternary Bonsai 27B retains 95% of the full-precision Qwen 3.6 27B baseline, while the 1-bit variant retains 90%. More importantly, the model excels in areas critical for agentic workloads: math and coding scores are nearly untouched, and tool-calling performance stays within a few points of full precision.

For context, a standard 4-bit quantized version of the same base model would require about 18GB and achieve lower scores than the 1-bit Bonsai 27B. The model's intelligence density—a metric PrismML introduced—is 0.53 per GB, which is more than 10x the full-precision baseline and roughly 2.7x the best low-bit alternative available.

Apple's Interest and iOS 27 Integration

Apple's reported interest in PrismML aligns with its broader push for on-device AI. The iOS 27 public beta, released on July 14, includes two new features that leverage a 27B-class model: a system-wide enhanced dictation experience that is more accurate and context-aware, and the ability to customize Siri's voice with varying levels of pace and expressivity. These features are exclusive to the iPhone 17 Pro, iPhone 17 Pro Max, and iPhone Air—devices with the necessary neural engine and memory bandwidth.

While Apple has not officially confirmed the use of Bonsai 27B, the timing and technical requirements strongly suggest a partnership. Apple has a history of acquiring AI startups, including the $2 billion purchase of Q.ai, and PrismML's technology could be key to unlocking more powerful on-device models without relying on cloud servers.

Why This Matters for Agentic AI

The shift from single-response models to sustained, multi-step agentic workloads—where an AI assistant makes hundreds of tool calls, processes documents, and interacts with software—creates new demands. Cloud-only execution introduces latency, per-token costs, and privacy risks, as every intermediate result and user data crosses the network. Local execution eliminates these constraints, enabling persistent on-device agents that work offline and keep data private.

Bonsai 27B reaches up to 163 tokens per second in 1-bit mode and 134 tok/s in ternary mode on an NVIDIA GeForce RTX 5090. On an Apple M5 Max, it achieves up to 87 tok/s and 58 tok/s respectively. This performance, combined with the small footprint, makes it feasible to run advanced AI directly on consumer devices, opening up new categories of applications.

Industry Context and Comparisons

The launch comes as other tech giants push the boundaries of on-device AI. Google recently updated its Android Bench ranking system for AI models coding for Android, with Anthropic's Claude Fable 5 leading at 84.5 points. Meanwhile, AMD's Ryzen AI Halo chips are promising 128GB of unified memory for local AI workloads, but Bonsai 27B achieves comparable capability at a fraction of the hardware cost.

For developers, Bonsai 27B offers a practical path to hybrid deployments: route privacy-sensitive and non-frontier tasks to the local model, and reserve cloud APIs for the hardest steps. This could dramatically reduce the per-task cost of agentic systems while improving responsiveness and privacy.

Availability and Next Steps

Bonsai 27B is available today under the Apache 2.0 License. It runs natively on Apple devices (Mac, iPhone, iPad) via MLX and on NVIDIA GPUs via CUDA, with custom low-bit kernels optimized for its hybrid-attention architecture. PrismML is also offering a free, limited-time developer preview API. The full technical whitepaper is available on GitHub.

As PrismML notes, the methodology behind Bonsai is architecture-agnostic, and the company is already working on larger models and new architectures. The frontier of intelligence density continues to move leftward, and Bonsai 27B represents its largest step yet toward making advanced AI truly ubiquitous.