- Published on
An Open 295B Mixture-of-Experts (MoE) Model with 21B Active Parameters and 256K Context
- Authors
- Name
- DeepHorus
Tencent’s Hy team has released Hy3, a 295-billion-parameter Mixture-of-Experts (MoE) model that activates only 21 billion parameters per token, making it efficient for reasoning, agentic workflows, and long-context tasks. The model weights are available under the Apache License 2.0, marking a significant contribution to open-source AI development.
Hy3’s architecture features a sparse MoE design with 192 experts and top-8 routing, ensuring that only eight experts are activated per token to keep computational costs low. It also incorporates a Multi-Token Prediction (MTP) layer, which predicts multiple tokens at once to accelerate decoding, supported by vLLM and SGLang through speculative decoding. Additionally, a separate FP8 checkpoint reduces memory footprint for cheaper serving.
On benchmarks, Hy3 demonstrates strong performance across coding, STEM, and agent tasks. It achieves 78.0 on SWE-Bench Verified, 57.9 on SWE-Bench Pro, and 75.8 on SWE-Bench Multilingual. In STEM reasoning, scores include 90.4 on GPQA Diamond and 72.0 on USAMO 2026. A blind test with 270 experts on real workflows placed Hy3 at 2.67 out of 4, outperforming GLM-5.1’s 2.51, particularly in frontend development, CI/CD, and data storage.
The research team prioritized production reliability by addressing three key failure modes. Invalid tool calls that cause infinite loops have been reduced, and accuracy variance across agent scaffoldings stays within 4%. Hallucination rates dropped from 12.5% to 5.4%, and commonsense errors fell from 25.4% to 12.7%. Multi-turn intent tracking improved, with internal issues declining from 17.4% to 7.9% and MRCR long-dialogue scores rising to 75.1%.
Hy3 exposes an OpenAI-compatible API, deployable with vLLM or SGLang. Users can control reasoning depth via the reasoning_effort parameter—setting it to 'high' enables deep chain-of-thought for complex tasks. The recommended settings are temperature 0.9 and top_p 1.0. A free tier is available on OpenRouter as tencent/hy3:free but will end on July 21, 2026.
The model is designed for agent-style, long-context applications. Use cases include feeding entire code repositories into its 256K context window for bug fixing, processing long documents with reduced fabrication, and combining tables and prose for grounded financial analysis. The blind test also highlighted advantages in frontend and game development.
Looking ahead, Hy3 positions itself as a competitive open-source alternative for production AI workloads. Its focus on reliability and efficiency in agentic tasks suggests growing demand for models that can handle complex workflows with minimal hallucination and stable tool use. Tencent’s release may spur further innovation in MoE architectures for real-world applications.
