- Published on
Alibaba Qwen Releases Qwen-Audio-3.1-Realtime: A Full-Duplex Voice Model Trained to Think, Act, and Decide When to Speak
- Authors
- Name
- DeepHorus
Alibaba’s Qwen team has released Qwen-Audio-3.1, a five-model audio stack covering automatic speech recognition, text-to-speech and realtime voice interaction. Its flagship, Qwen-Audio-3.1-Realtime, is a full-duplex model designed for voice agents that can call tools, search the web and decide independently when to listen, speak, pause or resume. The release matters because it combines conversational timing, reasoning and executable actions in one managed voice system. Qwen-Audio-3.1-Realtime-Plus is available through QwenCloud over WebSocket, although no open weights were announced.
The realtime model accepts text and audio and returns both formats. It provides a 262,000-token context window, with up to 245,000 input tokens and 16,000 output tokens, while default limits are 60 requests and 100,000 tokens per minute. Pricing is $6.40 per million audio-input tokens, $0.80 per million text-input tokens and $24 per million audio-output tokens; text output is not charged. Supported features include function calling, web search, structured outputs, context caching and fine-tuning.
Operationally, two models share an audio encoder and large-language-model design. A full-duplex decision model determines whether the system should continue listening, speak, stop or resume, while a speech-to-text model generates response content. A context-aware voice renderer then converts that text into streaming speech using conversation history, vocal cues and acoustic conditions.
Training is organized around four capabilities: Think, Act, Speak and Coordinate. The Think stage uses million-hour-scale paired data, on-policy distillation and expert reinforcement learning to improve empathy, pragmatic intent and acoustic understanding. During this process, a text teacher and frozen audio reference score tokens generated along the student model’s own trajectory rather than imitating fixed answers. Multiple expert policies are subsequently merged into one deployable model.
The Act stage trains tool use in executable environments containing tools, stateful JSON databases and natural-language business policies. Tasks require a permitted write, a justified refusal or recognition that a request is unsupported, and scoring prioritizes database outcomes over fluent language. Search training reduced average queries per call from 4.37 to 1.05, although trigger F1 declined from 60.87% to 58.61%.
Duplex training produced mixed results. On Full-Duplex-Bench v1.5, responses to people addressing someone else fell from 13% to 3%, while the v3.0 filler rate dropped from 0.7590 to 0.2960. However, the unwanted resume rate after interruptions rose from 0.035 to 0.130. Interruption-stop latency was 1.116 seconds, compared with 0.383 seconds for GPT-Realtime-2.
Against Qwen-Audio 3.0, Audio MultiChallenge rose from 47.12 to 52.21, the 14-language BBA average improved from 81.7% to 88.1%, and FLEURS word-error rate fell from 9.01 to 3.98. A 50-session human red-team study nevertheless placed GPT-Realtime-2 ahead, 96% to 92%. With prices reduced by roughly 85% for Realtime, 70% for TTS and up to 95% for ASR, Qwen is positioning its API as a lower-cost option, but continued adoption will depend on latency, duplex reliability and real-world agent performance.
