- Published on
NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure
- Authors
- Name
- DeepHorus
NVIDIA has unveiled NVLink Fusion, a new connective technology designed to help hyperscalers and AI-native companies build custom AI accelerators and integrate them into NVIDIA's AI infrastructure platform. This initiative addresses the growing demands of next-generation AI workloads, which require significantly more compute power and high-bandwidth memory to train and run increasingly complex models.
The announcement comes as AI factories face insatiable compute demands. Companies are developing custom chips, known as XPUs, to keep up. Deploying these at scale requires advanced memory, efficient packaging, and a resilient supply chain, all integrated into a rack-scale architecture. NVIDIA's platform aims to reduce the complexity and time to market for these semi-custom AI factories.
A key component of this ecosystem is NVHBM, a custom high-bandwidth memory base-die technology co-developed with leading memory vendors. NVHBM is designed to deliver up to 30% more memory bandwidth per stack compared to standard HBM4e. This improvement helps custom XPUs support larger models and process data faster, directly benefiting both training and large-scale inference tasks.
The technology tackles a critical bottleneck in AI accelerator design: ensuring compute engines are consistently fed with data. Higher memory bandwidth improves accelerator utilization and throughput. For memory-intensive AI workloads, NVHBM's increased bandwidth can translate into higher per-user token generation speeds during inference by moving data between memory and compute cores more efficiently.
NVIDIA emphasizes that while NVHBM boosts performance within a single accelerator, NVLink Fusion enables seamless connectivity across multiple chips in a rack. This scale-up capability is vital for advanced techniques like expert parallelism, where different AI model components reside on different GPUs and must synchronize at high speed across the entire system.
The technology also offers crucial design flexibility. By providing validated memory base dies, NVLink Fusion reduces integration bottlenecks and frees up valuable silicon area in custom chips. This allows accelerator designers to allocate more space for specialized compute logic, enhancing performance for specific AI workloads.
As AI models grow more complex, the underlying infrastructure must evolve. NVIDIA's NVLink Fusion and NVHBM provide a standardized yet flexible foundation for building next-generation AI accelerators. This could accelerate innovation in custom silicon while ensuring compatibility and performance within a mature ecosystem, shaping the future of AI factory design.
