- Published on
Microsoft Commits Maia to Multi-Gigawatt Inference, and Its Efficiency Metric Draws Questions
- Authors
- Name
- DeepHorus
Microsoft will scale its Azure Maia accelerator family to multi-gigawatt deployments beginning in 2027, positioning Maia as "the primary inference system for Microsoft products and services." The commitment, laid out in a technical post accompanying Satya Nadella's announcement, moves the in-house silicon from a supporting role to the center of the company's AI infrastructure. Maia 300 begins production deployments in 2027.
Maia 200 is already in production, serving Microsoft Copilot and the company's MAI team, with GitHub Copilot and Microsoft Foundry to follow by the end of the year. It is deployed across the United States, Europe and Australia, and Microsoft says it delivers up to 1.7 times more tokens per watt for built-in AI in Teams and Copilot than leading GPU infrastructure.
Maia 300 is inference-first, aimed at long-context, memory-intensive and decode-heavy work rather than training, which the announcement does not mention. Each accelerator carries eight HBM4 stacks, providing 288 GB and 18 TB/s within a 950-watt TDP. A rack holds 96 accelerators across 24 four-accelerator trays, totaling 167 kW, 27.6 TB of HBM and 1.7 PB/s of aggregate bandwidth.
Compute chiplets are built on TSMC's N2P process, with memory-controller and I/O functions on separate dies, a split Microsoft says raises compute FLOPs by 36 percent and contributes to 2.5 times the memory bandwidth and roughly twice the HBM bandwidth per watt of Maia 200. Memory organization also shifts from uniform access to a NUMA layout across chiplets, letting software place data near the compute using it.
Networking is the most opinionated element. Maia's AI network controller implements a custom AI transport over Ethernet that Microsoft describes as "one fabric rather than separate scale-up and scale-out domains," with accelerators initiating communication to any peer in a superpod without involving the host or a discrete scale-out NIC. Optical second-tier switches connect up to 128 racks into a 12,288-accelerator superpod, and the in-rack versus cross-rack bandwidth split is set by job packing rather than fixed at design time.
Microsoft's headline claim is that Maia 300 will deliver 40 percent more tokens per megawatt than leading GPU-based systems, alongside up to seven times the rack-scale token throughput and more than 2.5 times the performance per watt of Maia 200. The unit, not the number, drew the sharpest reaction. Rick Bullotta wrote on Nadella's post: "Tokens per watt. What an absurd metric. As if there's a uniform, consistent definition of a token or a generic correlation to compute." Commenter Michel Floyd asked whether it should be tokens per second per watt or tokens per kilowatt-hour, while Vinay Bansal, founder of UBIQS, questioned the context length the figure was measured at, arguing memory becomes the binding constraint for long-running agents.
With production deployments set for 2027, Microsoft is betting that an inference-first, fabric-centric design can carry its own products before it faces outside workloads.
