- Published on
Running Kimi K3 on MI355X at Better Performance per Dollar Than B300
- Authors
- Name
- DeepHorus
AMD's MI355X GPU has emerged as a surprising leader in serving Kimi K3, a massive open-source model, achieving roughly 952 tokens per second per node. This performance, reported by AI infrastructure company Wafer, positions the MI355X as a better value than Nvidia's B300 on performance per dollar. The announcement signals a shift in the economics of serving frontier-scale AI models, where memory capacity and cost are becoming as critical as raw compute.
The context is the rapid growth of open-source models, including DeepSeek V4-Pro and GLM5.2, which have reached near-frontier intelligence levels. Kimi K3, however, stands out with a staggering 2.8 trillion parameters, requiring over 1.5 terabytes of VRAM just for weights before allocating a KV cache for a 1-million-token context. A single node of eight Nvidia B200 GPUs cannot fit this model, forcing operators to use either a B300 node or two B200 nodes, both expensive options. AMD's MI355X, with 288GB of HBM per GPU, matches the B300's memory capacity at a significantly lower cost.
Wafer's benchmarks show that the MI355X achieves 952 tokens per second per node on a 1,024-token input and 400-token output test, with a single-stream rate of 118 tokens per second. This is over 3.8 times the aggregate throughput per node of a TP16 B200 deployment, which delivers a total of 498 tokens per second across two nodes. The B300 node still wins on raw aggregate throughput by about 1.65 times, but at 2.4 times the price per GPU, the MI355X delivers better performance per dollar.
Cost analysis at $2.50 per GPU-hour for the MI355X, $6.00 for the B300, and $4.25 for the B200 reinforces this advantage. Wafer notes that the B200 numbers are somewhat deflated by cross-node communication overhead on the decode path, as Kimi K3 requires two B200 nodes. This practical limitation highlights a key point: the MI355X's focus on high HBM capacity gives it a measurable edge for serving enormous models like Kimi K3.
Software support has historically been AMD's Achilles' heel, but AMD provided day-0 support for Kimi K3, reducing engineering effort. Still, Wafer encountered a critical issue on ROCm with speculative decoding, specifically a NameError in the SGLang scheduler due to a missing top_k_renorm_prob function. The fix required a simple PyTorch implementation of top-k renorm, which restored functionality and allowed the high throughput to be achieved.
This development suggests that memory capacity is becoming a key differentiator in AI hardware, challenging Nvidia's dominance in inference. The MI355X's success with Kimi K3 indicates that AMD can compete on performance-per-dollar in frontier deployments, especially for models with massive memory footprints. Wafer's claim that AI agents are improving kernel and model optimization suggests the software gap will continue to narrow.
Looking ahead, the ability to serve a 2.8T-parameter model on a single node at a lower cost could accelerate adoption of open-source models. As the industry moves toward larger and more capable models, the battle between Nvidia and AMD will hinge on memory capacity, pricing, and software maturity. The MI355X's strong showing in this test is a clear signal that AMD is a serious contender in the AI inference market.
