- Published on
Huawei Releases Xing4.0-29B, a Lightweight Code Agent LLM Trained on Ascend Super Nodes
- Authors
- Name
- DeepHorus
On September 17, China Telecom AI Technology Co., Ltd. officially released the new-generation Xingchen large model Xing4.0-29B-A4B. The model focuses on capabilities such as complex task understanding, task planning, tool invocation, and autonomous execution, supports long-context processing, and can be applied to scenarios including code development, data analysis, office automation, and life services. On the evening of September 19, Huawei China officially announced that this model is the first domestic large model with tens of billions of parameters to complete training based on the Ascend Atlas 900 A3 SuperPoD liquid-cooled supernode and the MindSpore framework.
Xing4.0-29B-A4B has a total of 29B parameters with only 4B activated parameters. It adopts a new-generation architecture design to further enhance long-horizon task processing and multi-step execution capabilities. Huawei officially stated that the model can autonomously plan task paths, invoke tools, and complete complex tasks based on user-provided goals. Mid-to-high-level code generation and agent execution are comprehensive tests of long-context understanding, complex reasoning and planning, and coordinated tool invocation.
Regarding the data system, the team built a data foundation covering tens of trillions of tokens. During the pre-training phase, PB-level cleaning of multi-source heterogeneous data was completed, incorporating massive amounts of agent behavior trajectories and structured knowledge graphs. In the post-training phase, a high-concurrency sandbox container was established to construct end-to-end long-horizon task synthetic data, and an automated verification mechanism was set up to ensure data correctness.
In terms of architecture design, the model is specifically designed for long-horizon agent tasks. It uses MLA (Multi-head Latent Attention) to compress KV cache, MT (Multi-Token) prediction to enhance generation coherence, mHC (manifold-constrained hyper-connections) to solve training numerical instability, and employs Muon and QK-Clip optimizers to ensure training stability. The training adopts a staged progressive strategy, gradually expanding sequence lengths from 4K to 32K, 128K, and 256K, systematically building long-horizon capabilities.
In terms of engineering optimization, the model achieved a 96% improvement in training performance on the Ascend supernode cluster through multi-level software-hardware co-optimization. Based on MindSpore's full-stack adaptation of mHC multi-residual connections and DSA long-sequence mechanisms, for fine-grained MoE (64 routers / Top4 activation), throughput was improved by 32% through expert load balancing, Grouped GEMM, communication-computation overlap, and DVM graph-operator fusion. Layer-level and operator-level selective recomputation further improved throughput by 19%. The self-developed Ascend C mHC fused operator resolved Sinkhorn fragmentation, improving computational efficiency by 30% and overall network throughput by another 25%.
In the reinforcement learning phase, the team completed adaptation of the Slime framework based on the Ascend ecosystem, conducting multi-expert reinforcement learning training for Agent, Coding, and Reasoning tasks. Through MOPD technology, efficient fusion of multi-expert capabilities was achieved, further improving performance on complex reasoning and agent tasks. Currently, Ascend has supported more than 40 industry-leading models in completing native training, and is also the only domestic vendor that supports commercial large-scale pre-training and has trained and deployed SOTA large models.
Xing4.0-29B-A4B has been open-sourced on platforms such as GitHub, HuggingFace, ModelScope (魔搭), Gitee, and Modelers (魔乐). The release of this model provides a new technical path for domestic large models in complex engineering tasks, and is expected to continue playing a role in scenarios such as code development and data analysis in the future.
