Google's EmbeddingGemma 2 Repurposed as Blazing-Fast On-Device Speech-to-Text

TL;DR: A Multimodal Embedding Model That Transcribes Speech

In a striking demonstration of model repurposing, developers have transformed Google DeepMind's EmbeddingGemma 2—a 740-million-parameter multimodal embedding model—into a highly efficient on-device speech-to-text engine. The result: it processes an hour of audio in roughly 13 seconds on a base M3 MacBook Air, achieving 288x real-time speed. This is not a fine-tuned ASR system but a clever exploitation of the model's shared audio-text embedding space.

The project, dubbed "Embedding Gemma Transcribe," is available as a free Mac app. It runs entirely on the Neural Engine, keeping audio data on-device. While its word error rate (about 5%) trails leading dedicated models (2-3%), the approach underscores the versatility of modern multimodal embeddings and opens new avenues for efficient local transcription.

The Core Idea: Reading Words Out of an Embedding Space

EmbeddingGemma 2, released by Google DeepMind in October 2026, maps text, code, images, audio, and video into a single 768-dimensional vector space. This allows direct comparison between different modalities—for instance, a spoken word and its written form land close together. The developers behind this project hypothesized that this proximity could be leveraged to reconstruct text from audio.

The methodology is surprisingly simple:

• Unchanged Audio Encoder: EmbeddingGemma 2's audio encoder is used exactly as is, without any modification.
• Trained Reader Network: A small, trainable "reader" network is added to translate each short audio slice's embedding into the text embedding space.
• Nearest Word Selection: The system picks the closest English word from a vocabulary of ~229,000 words for each slice, generating a raw transcript.
• Punctuation/Capitalization Model: A tiny auxiliary model adds punctuation and proper casing to the raw output.

Only the reader is trained, and that took under two hours on eight NVIDIA H100 GPUs. The training data consisted of openly licensed English speech. Notably, the base EmbeddingGemma 2 model was never designed or trained for speech-to-text—this is a pure repurposing.

Performance Benchmarks: A New Speed Champion

The project's published benchmarks place it among the fastest on-device transcription systems, especially when considering the hardware used. The tests were conducted on a base M3 MacBook Air with 8 GB of RAM, a common consumer device.

It's important to note that the last three rows are from FluidAudio's published benchmarks on newer, more powerful Macs, so they are not directly comparable. However, the Embedding Gemma Transcribe's performance on a base model MacBook Air is impressive, outpacing IBM's Granite Speech on the same hardware by nearly 2x.

Context: EmbeddingGemma 2's Design and Capabilities

EmbeddingGemma 2 is a significant release from Google DeepMind. Built on the Gemma 4 foundation, it features a 740M parameter form factor with modular encoders: a 270M text model, a 170M vision encoder, and a 300M audio encoder. It supports flexible embedding dimensions (768 to 128) via Matryoshka Representation Learning (MRL), allowing developers to trade off precision for storage efficiency—up to a 6x reduction in vector storage.

The model is distributed under the permissive Apache 2.0 license, making it commercially viable. It runs on a variety of frameworks including transformers, sentence-transformers, MLX, vLLM, llama.cpp, SGLang, Ollama, and LM Studio. Google also provides a browser-based implementation via transformers.js with WebGPU support. The quantized text-only version uses roughly 191 MB of active memory on a Pixel 11 Pro, while the full model uses about 567 MB—making it well-suited for on-device applications.

Google's original intent for EmbeddingGemma 2 was to enable efficient cross-modal search without the need for separate transcription or captioning pipelines. For example, a text query could find a relevant voice memo or video clip by comparing embeddings directly. This project takes that concept one step further by actually decoding the audio embeddings back into text.

Why This Matters: The Power of Repurposing Foundation Models

This project is a compelling proof point that large foundation models can be adapted for tasks far outside their original scope. The developers didn't need to train a massive ASR model from scratch—they leveraged the rich, aligned representations already learned by EmbeddingGemma 2.

The implications are significant:

• Efficiency: The system runs at 288x real-time on a consumer laptop, making it ideal for batch transcription of large audio archives.
• Privacy: All processing is on-device, ensuring sensitive audio never leaves the machine.
• Cost: Training the reader network took less than 2 hours on 8 H100s, a fraction of the cost of training a dedicated ASR model.
• Innovation: It demonstrates that the embedding space contains latent information that can be extracted with minimal additional machinery.

However, the current accuracy (5% WER) is a limitation. The developers believe a better-trained reader could close the gap with leading models (2-3% WER). This is a plausible claim, given that the reader is a tiny network and the approach is still in its early stages.

How to Use It

The app, available for free download for macOS 14+ on Apple Silicon, offers two primary modes:

• File Transcription: Drop in an audio file and get a transcript.
• Dictation: Hold the Right Option key, speak, and release to dictate into any application.

Models download on first launch (715 MB). The app leverages the Mac's Neural Engine, ensuring all processing stays on-device.

Bottom Line

Embedding Gemma Transcribe is a remarkable hack that showcases the untapped potential of multimodal embedding models. While it may not replace dedicated ASR systems for accuracy-critical applications yet, its speed and efficiency make it a compelling option for many use cases. More importantly, it opens the door to further experimentation—if an embedding model can be turned into a speech recognizer, what other capabilities might be hiding in these shared spaces?