- Published on
Kyutai Releases Voice of Reason: A Speech-Native Model that Solves Spoken Math with Reinforcement Learning
- Authors
- Name
- DeepHorus
Kyutai, the French non-profit AI research lab, has released Voice of Reason, a pair of open-weight speech-to-speech models that solve mathematical word problems aloud. Both checkpoints start from GLM-4-Voice-9B and add supervised fine-tuning followed by reinforcement learning, an approach the team describes as the first application of RL to math reasoning in speech-native models. On the spoken GSM8K benchmark, accuracy climbs from 27.3% for the base model to 77.1%. Notably, there is no transcription step and no separate text LLM anywhere in the pipeline.
The work targets a known weakness of voice systems. Cascaded pipelines — speech-to-text, then a text LLM, then text-to-speech — still lead on reasoning, but each stage adds latency and discards paralinguistic cues such as tone. Speech-native models face a different ceiling: they must emit audio at regular intervals to remain interactive, which limits how many hidden reasoning tokens they can afford. Before this release, the earlier STITCH method had pushed the base score from 27.3% to 58.7% by inserting reasoning chunks.
Training proceeded in two stages. Supervised fine-tuning used 150,616 problems from Orca-Math, each rewritten for spoken delivery by Qwen3-235B and voiced in multiple accents by Kyutai's own DSM TTS system, lifting GSM8K accuracy to 61.7%. In the reinforcement learning stage, the model sampled four replies per spoken question at temperature 0.9, scored by judge model Qwen3-235B-A22B-2507 with a binary reward on the decoded text — the judge never sees the reference answer and agreed with human raters 88% of the time across 100 hand-checked cases. Rewards were centered within each group to form a group-relative REINFORCE objective related to GRPO but without PPO clipping or KL regularization, trained over 1,500 updates on 16 H100 GPUs.
Two design choices proved decisive. Dividing logits by the sampling temperature before the log-softmax was essential: without that correction, GSM8K performance collapsed from 65.5% to 12.3%. The second change sums all audio-vocabulary probabilities into one abstract token at each audio position, so the loss asks only whether audio comes next rather than which token. The authors prove this estimator is unbiased and lower-variance under a value-invariance assumption.
Deployment is lightweight: both BF16 checkpoints run on a single H100, provided users also pull the GLM-4-Voice repository for its speech tokenizer and decoder. The weights inherit the GLM-4-Voice license, and no Hugging Face inference provider hosts them yet. Two variants ship — glm-4-voice-of-reason-9b answers directly with any working spoken aloud, while glm-4-voice-of-reason-stitch-9b writes silent 100-token reasoning chunks that are generated while earlier speech plays, adding no extra latency.
Results hold up under scrutiny. Paper scores use top-k 50 decoding averaged over three seeds, reaching 70.3% and 77.1%; transcribed through Qwen3-ASR-1.7B, the Stitch model still scores 72.0 ± 1.9%, and UTMOSv2 naturalness ratings barely move. Replies actually shortened from 41.9 to 36.4 seconds, ruling out verbosity as the cause, while RL proved most valuable with scarce data, hitting 58.5% on just 10% of the SFT set. Spoken TriviaQA did dip from 40.6% to 34.0%, which the authors attribute mainly to full-data SFT rather than RL.
The team also dropped AddSub, MultiArith, SingleEQ and SVAMP from evaluation after finding 54.0% paraphrase-level overlap with Orca-Math among 678 checked questions. That transparency signals a field growing more careful about contamination. The release suggests reasoning may increasingly live inside the voice layer itself rather than being delegated to cascaded text systems, though whether it scales beyond arithmetic remains the open question.
