- Published on
Performance 'unbelievable'
- Authors
- Name
- DeepHorus
OpenAI researcher Jeffrey Wang recently praised the AI inference solution jointly developed by AMD and Cerebras, calling its performance 'incredible.' The solution is designed for ultra-low latency AI inference scenarios, integrating AMD's Helios rack-level solution with Cerebras' wafer-scale engine, aiming to challenge the combination of Nvidia and Groq.
According to tech media Wccftech, the joint solution was disclosed on July 28, primarily targeting AI inference tasks that require extremely fast responses. Per the roadmap, the Helios rack serves as a high-performance, scalable throughput engine, mainly responsible for processing prompts and large context windows. Cerebras' Wafer-Scale Engine handles the token generation and decoding steps that demand higher memory bandwidth, emphasizing ultra-low latency performance. Combined, they are estimated to increase the number of tokens per watt per second by five times.
OpenAI researcher Jeffrey Wang verified the solution's performance in internal tests. He said: 'We have some OpenAI models running internally on Cerebras chips, and they perform very well, with impressive inference speed.' He further illustrated: 'Previously, I might have to wait minutes to complete a task, but now, the moment I switch tasks, that task is already completed, greatly improving our work efficiency.'
The launch of this collaborative solution comes at a time of growing demand for low latency and high efficiency in the AI inference market. Traditional GPU solutions face energy efficiency bottlenecks in inference scenarios, while the joint AMD-Cerebras solution, through a computation architecture optimized by division of labor, is expected to significantly reduce inference costs. Cerebras' wafer-scale engine is known for massive parallel processing, and combined with AMD's rack-level system, it can achieve better resource allocation in prompt processing and token generation.
In terms of industry impact, this solution may reshape the competitive landscape of the AI inference chip market. Nvidia has long dominated the inference market with its GPU ecosystem, but emerging companies like Groq have captured low-latency scenarios through specialized architectures. The AMD-Cerebras joint effort provides a third option, especially suitable for applications requiring real-time responses such as conversational AI and code generation.
Looking ahead, AMD and Cerebras plan to further improve system integration and energy efficiency. OpenAI's endorsement may accelerate the validation of this solution in larger-scale deployments. As AI model sizes continue to grow, inference efficiency will determine the speed of technology implementation, and this joint solution could be a key turning point.
