- Published on
Interfaze Ships diffusion-gemma-asr-small, an Open-Source Diffusion ASR Model Transcribing Six Languages via DiffusionGemma’s Parallel Denoising Decoder
- Authors
- Name
- DeepHorus
Interfaze, a Y Combinator-backed startup, has open-sourced what it claims is the first multilingual diffusion-based automatic speech recognition (ASR) model. Named diffusion-gemma-asr-small, the model transcribes audio in six languages using a diffusion decoder rather than the traditional autoregressive approach. This development marks a significant shift in speech recognition technology, offering potential efficiency gains in transcription speed and resource usage.
Traditional autoregressive models generate text one token at a time, sequentially predicting each word. In contrast, diffusion models refine all tokens in parallel through a denoising process. Diffusion-gemma-asr-small employs this diffusion method, leveraging Google's DiffusionGemma, a 26-billion-parameter mixture-of-experts model that activates 4 billion parameters per inference. The diffusion decoder uses uniform random-token diffusion instead of the more common absorbing
scheme, filling a fixed-length canvas with random vocabulary tokens and iteratively denoising them into coherent text.
Key to the model's efficiency is its small trainable adapter—only about 42 million parameters trained on top of the frozen 26-billion-parameter backbone, representing roughly 0.16% of total weights. This adapter handles all six languages from a single model. The system uses a frozen Whisper-small encoder as a feature extractor, converting 30 seconds of audio into 1500 frames of 768-dimensional features, which are then projected to 188 audio tokens of 2816 dimensions. These tokens are scattered into the prompt's reserved audio slots, allowing DiffusionGemma's decoder to denoise a 192-token transcript canvas in about 16 steps.
Performance metrics show the model leading among diffusion-based peers on LibriSpeech with a word error rate (WER) of 6.6%, compared to Whisfusion's 8.3%. However, it still trails the autoregressive Whisper model. The transcription cost scales with the number of denoising steps, not transcript length, offering potential advantages for longer audio segments. The adapter is released under the Apache-2.0 license, while DiffusionGemma uses Google's Gemma terms, and Whisper-small remains under MIT.
The open-source release addresses a key challenge in training diffusion ASR models: the initial failure of the projector to learn due to lack of gradient flow. The team applied a CTC loss directly on the audio embeddings to supervise the projector, breaking the circular deadlock. This approach allowed the model to effectively map audio features to text.
Industry reactions highlight the model's potential for democratizing ASR technology. By open-sourcing the adapter, Interfaze enables researchers and developers to build on a state-of-the-art diffusion framework with minimal additional training. The model's multilingual capability from a small adapter could reduce the computational overhead of deploying separate models for different languages.
Looking ahead, diffusion-gemma-asr-small may spur further innovation in non-autoregressive speech recognition. Its architecture, which separates the encoder, projector, and diffusion decoder, allows for modular upgrades. Future developments could include support for more languages, lower latency for real-time applications, and integration with other diffusion-based language models. As open-source diffusion ASR matures, it may challenge the dominance of autoregressive models in both performance and efficiency.
