The Next Platform은 Microsoft의 'Clea' Maia 300 추론 XPU를 분석하며, 이를 대역폭 괴물이라고 설명했다.

If everyone has an AI inference problem, with tokens still too expensive to generate even on state-of-the-art GPU and XPU accelerators, Microsoft probably has the biggest inference problem in the world.

For one thing, running inference for OpenAI's ChatGPT models, as well as helping the AI model builder train its models, is a big job. Meanwhile, all of Microsoft's worker productivity and back-office applications are being infused with AI routines, which is driving usage up the hockey stick handle at a rapid pace.

That is why Microsoft is putting the pedal to the metal with its Maia family of custom, homegrown XPU accelerators. Announcing that it is working on the problem, the company rolled out the feeds and speeds of its future "Clea" Maia 300 accelerator as the Open Compute Project Summit 2026 kicked off this week.

The Maia 300 is a much-improved device with a number of architectural changes designed to let GenAI inference run not only faster, but also scale further, so that larger agentic workloads can run. These workloads consist of dozens to hundreds of models, whether implicitly through a mixture-of-experts foundation model, actually physically distinct independent models, or a mix of the two, working in concert and needing to share the same network and memory space to work efficiently and with low latency.

Microsoft is no slouch when it comes to designing chips, and it often gets things right on the second or third try. The software and cloud giant dipped its toe in the water with the "Athena" Maia 100 XPU in November 2023. At the time, Microsoft was also unveiling its Cobalt 100 custom Arm server CPU, and it said very little about the architecture of either custom chip.

The second-generation "Braga" Maia 200, unveiled only in January of this year, is a much more serious XPU and is the one that has been ramping into production on the Azure cloud this year.

In a separate announcement, Scott Guthrie, executive vice president in charge of Cloud and AI at Microsoft, said that the "multigenerational Azure Maia program has become a foundational element of Microsoft's AI infrastructure strategy." He said that Maia 200 chips are doing inference for Copilot and helping the Microsoft AI team develop new models, and that GitHub Copilot and Microsoft Foundry will shift to Maia 200s by the end of the year. (GitHub Copilot was previously supported by Maia 100s.) The Maia 200s will run OpenAI's various models, Microsoft's own models, and various open-source models through Microsoft Foundry. It is not clear how much of OpenAI's GPT API service is running on Maia 100 or Maia 200 iron.

The rumor mill says Microsoft wants to get at least 1 million Maia sockets into the field, and Guthrie said in his statement that Maia would have "multi-gigawatt deployments" on the Azure cloud. A gigawatt of Maia 300s is around 575,000 devices, each drawing 1.74 kilowatts for the accelerator and its portion of the system host, storage, and scale-up networking. (Microsoft said a rack of Maia 300s burns about 167 kilowatts and has 96 XPUs interlinked.)

The Braga chip had much more HBM memory capacity and bandwidth, even as it trimmed back on-chip SRAM cache. It also sported a much-improved AI Transport Layer (ATL), a custom RoCE Ethernet implementation with 2.3X more bandwidth. This scaled the cluster coherence domain of the Maia cluster by 2.7X, to 6,144 devices, while increasing power consumption only 1.5X. By shifting from 6-bit MX6 data formats to 4-bit MXFP4, and by boosting the number of compute cores 1.5X while also raising the clock speed, the Braga chip delivered 3.4X more floating-point performance than the Athena chip it replaced.

*Racks of systems using the Maia 300 "prototype" XPU*

The Clea Maia 300 chip is really a chip complex in a socket, perhaps a system in a socket, and not, strictly speaking, a system on a chip as Microsoft refers to it in its technical deep dive. It is actually a collection of seven chips and eight HBM stacks on an interposer. It is pushing the envelope on all spec fronts, but it is clearly focused on effective performance on real-world GenAI inference workloads rather than simply cramming flops into a socket for spec-sheet superiority. Moreover, Microsoft has changed its ATL Ethernet network so it can cram more XPUs into a rack and more racks into a coherent domain to take on bigger agentic workflows.

The Maia 300 "system in socket," and the racks of machines that make up pod and superpod clusters, have a hierarchy of compute, memory, and networking that glues together aggregations of these capacities to bring them to bear on GenAI work.

The Clea compute tiles differ a little from those in the Athena and Braga predecessors. All three XPUs have cores with a tile tensor unit (TTU), a tile vector processor (TVP), a tile data movement engine (TDMA), and a tile control processor (TCP), plus L1 SRAM cache memory for these units to share. (How much was not divulged.) The Athena XPU organized four of these cores on a tile and sixteen tiles into a cluster, an in-socket cluster, for 64 cores in total. Microsoft did not divulge the organization of the Braga tiles and tile clusters last January, but a paper published on arXiv in August did, which allowed us to update our salient characteristics table for the Maia lineup.

The Braga XPU, as it turns out, has tensor units with matrices twice as large as the Athena unit. As a result, it needed a little less than half as many cores to deliver the throughput Microsoft pegged for the device than we had thought back in January. The Braga XPU has ten tiles per on-chip cluster and four clusters, for a total of 40 cores. They run at 2 GHz, and only nine cores are active per cluster to improve yield on the 3-nanometer N3P process Taiwan Semiconductor Manufacturing Co. uses to etch the chip. The Athena and Braga chips were monolithic designs, close to the reticle limit of lithography machines.

With the Clea XPU, Microsoft is moving to TSMC's 2-nanometer N2P process and to a chiplet architecture that separates the I/O and HBM controllers from the compute and SRAM cache elements. The Maia 300 has four tile clusters, just like the Maia 200, but each on-chip cluster now has twelve tiles. Two elements, a reshaper and a tile synchronizer, were also added with the Maia 200, though we did not know about them earlier this year. They help with the dataflow that spans the entire system, from the network down to the server node and all the way down to the tile.

The SRAM capacities per tile (L1) and per cluster (L2) have not been revealed. If I had to guess, and I do, there is around 400 MB of SRAM cache spread across the compute tile at the center of the Clea chip package, and another 200 MB embedded in the I/O chiplets to help cache network data as it enters and compute data as it exits the package.

There could be logic as well as SRAM embedded in the HBM controller dies, which are relatively large, as you can see. I am guessing there is 200 MB of SRAM in the two I/O dies on the Clea package.

Across the Clea complex, I reckon there is about 166.2 TB/sec of aggregate SRAM bandwidth, which counterbalances the 18 TB/sec of HBM memory bandwidth for the eight HBM4 stacks on the device. The Clea XPU has 288 GB of HBM4, which is presumably from SK Hynix, as was the case with the Braga XPU.

For the area and transistor count of the Clea package, you can make a rough estimate, since HBM memory sizes are known and we have images of the chip packages for all three Maia chips, as well as their process nodes. I assume the HBM memory controller and I/O dies in Clea use 5-nanometer rather than 2-nanometer processes, which would make them perform better and also be more cheaply made. (Shrinking I/O circuits does not necessarily improve their performance, and it most definitely raises their costs and shrinks their yield.) The Clea package appears to be about 1,980 mm² in area, which is 2.4X as large as I think the Braga package was, at 810 mm². Playing around with some numbers, I peg the transistor count at around 280 billion, give or take perhaps 10 percent, not including the HBM4 memory.

Take a look for yourself and make your own guesses. Here are the feeds and speeds for all three Maia XPUs, with my estimates shown in bright red bold italics as usual.

Microsoft has not said who its chip shepherd is for the Maia 300. Global UniChip (GUC) of Taiwan, a close packaging partner of TSMC (which owns 35 percent of the company), was said to be the shepherd for the Maia 100 a few years back. Everyone presumes GUC is shepherding the Maia 200 and Maia 300, although there were rumors two yea…