Block's Dessa: AI Voice Model Trained Only on Speech

Block's Dessa: A Conversation Model Trained Only on Speech

Block, the financial technology company led by Jack Dorsey, has unveiled Dessa, a new artificial intelligence model that promises to redefine human-AI voice interaction. Developed by a small research lab within Block, Dessa is claimed to be the world's first AI capable of holding a realistic live conversation after being trained exclusively on human speech. This departure from the text-centric paradigm of most large language models could mark a significant shift in how machines understand and generate spoken language.

The announcement, made via Block's website and a post on X by Dessa lead Ragavan Thurairatnam, positions the model as a direct challenge to the limitations of current voice assistants. These systems typically convert speech to text, process it, and then synthesize a spoken response, a pipeline that discards the rich emotional and paralinguistic information embedded in the human voice. Dessa, by contrast, operates as a true speech-to-speech model, processing audio directly and responding in real time with natural timing, emotion, and nuance.

Audio In, Audio Out: A New Architecture

Dessa is described as a full-duplex model, meaning it can listen and speak simultaneously, much like a human in conversation. This allows it to interrupt, backchannel with sounds like “yeah,” “got it,” or “ooh,” and respond to cues such as tone, pauses, and laughter. The model was trained from scratch on nothing but human conversation—no text, no pretrained models. This approach, according to Block, allows Dessa to natively hear and speak emotion, sarcasm, laughter, a frustrated sigh, or a voice cracking with sadness, elements that are typically lost when speech is flattened into text.

The training process was computationally intensive but efficient in terms of data. Block notes that Dessa learned language from hundreds of times fewer words than typical LLMs, though it also had to learn far more than language—accents, breathing, inflection, and conversational timing. The company states that the model was trained on a single NVIDIA GB200 NVL72, a powerful but not unprecedented hardware configuration. Inference runs faster than real time, and the model can operate locally on a MacBook Pro M4 Max or better, making it accessible for consumer-grade hardware.

Why This Matters: The Limits of Text-Based Voice AI

The motivation behind Dessa is threefold: curiosity, better human-AI understanding, and safety. Block argues that current voice AI is not only stilted and annoying but also potentially dangerous. By ignoring the emotional content of speech, these systems can fail to recognize distress or pain in a user's voice, leading to inappropriate or harmful responses. Dessa, by learning directly from speech, aims to bridge this gap, providing a more empathetic and context-aware interaction.

The company also highlights the inefficiency of text-based training. Spoken words carry thousands of times more bytes than their written counterparts, and speech data is inherently variable—no two utterances are identical. This makes the learning task brutal for a machine, but Block's team, composed of former members of the Toronto deep learning startup Dessa, has successfully navigated this complexity. The lab, which reports directly to Jack Dorsey, has been building generative speech models for over eight years, giving them a unique perspective on the field.

Demonstrations and Early Access

Block has released several live recordings of people interacting with Dessa, showcasing its ability to laugh mid-sentence, adjust volume when asked, and respond with genuine-sounding empathy. In one demo, Dessa even finishes a person's sentence when they blank on a movie title, demonstrating its conversational memory and predictive capabilities. Another clip shows Dessa talking with GPT-Live-1, a third-party AI model, with both voices being AI-generated.

These demonstrations are compelling, but they are Block-produced and not yet subject to independent evaluation. To address potential risks, the lab is offering early access to a select group of AI safety researchers. This move acknowledges the novelty of the model and the need for external scrutiny to identify unforeseen issues. The invitation is a deliberate effort to find risks the team hasn't anticipated, marking a responsible approach to deploying a new AI category.

Research Questions and Future Directions

Dessa opens up numerous research questions. How efficiently do speech-only models learn language and facts compared to text-based systems? How well can they acquire new languages? What kinds of in-context learning can they perform? These are open questions that Block is keen to explore, and the release of Dessa is intended to spur further investigation. The model's ability to learn from raw sound challenges the assumption that text is the most efficient medium for AI training, potentially leading to more natural and data-efficient AI systems.

Thurairatnam has called the release “a GPT3 moment for speech,” a bold claim that suggests Dessa could be a foundational technology for a new wave of voice AI. While it's too early to know if Dessa will live up to this billing, its approach is undeniably innovative. By prioritizing the full richness of human speech, Block is betting that the most natural interface to intelligence is, in fact, the human voice itself.

As Dessa moves from a research project to a more widely accessible tool, its impact on fields ranging from customer service to mental health support could be profound. The model's ability to perceive and respond to emotion could make AI interactions more human and, crucially, more safe. For now, Dessa remains a glimpse into a future where machines don't just hear what we say, but truly understand how we say it.