Microsoft's Maia 300 Is a Million-Chip Bet Against Nvidia's AI Crown

Microsoft wants out from under the Nvidia tax. On Monday, the company unveiled the Maia 300, the third generation of its custom AI chip, and said it plans to deploy potentially more than a million of them starting next year in what it called "multi-gigawatt deployments." The announcement is the most aggressive move yet in Silicon Valley's quiet war to loosen Nvidia's hold on the AI computing market.

The timing matters because of where the money is shifting. Training frontier models gets the headlines, but the daily economics of AI live in inference: the millions of queries that coding assistants, business agents, and chatbots run every day. Microsoft said in its announcement that Maia will be "the primary AI inference system for Microsoft products and services." The chip's predecessor, Maia 200, already runs Microsoft's in-house AI models as well as OpenAI models served through Azure. The new hardware is the bet that the future is made of inference, and that the company should own the silicon that performs it.

A different technical path

To build Maia 300, Microsoft took a deliberately different route from Nvidia. Nvidia's AI accelerators center on one massive computing die surrounded by high-bandwidth memory chips. Microsoft's approach is chiplet-based: the central processor is composed of multiple specialized chips bonded together, with compute chiplets built on TSMC's 2-nanometer N2P process node and supporting chiplets, for moving data on and off the processor, on cheaper 3-nanometer technology. The logic is straightforward: use the most advanced and most expensive process only where the math happens, and spend less everywhere else.

The design philosophy extends through the whole system. Maia 300 is built around low-precision number formats, including an enhanced MXFP4 format, that cut the memory and bandwidth needed to store model weights and the KV-cache behind long conversations. The payoff Microsoft claims is larger context windows, higher throughput, and more efficient models inside the same power envelope. Networking is treated as part of the same system: high-bandwidth interconnects are tuned for tensor parallelism, and a custom AI transport layer runs on Ethernet-based infrastructure so the architecture scales from one rack to datacenter scale.

96 chips, 167 kilowatts

At rack scale the numbers get serious. Each Maia 300 rack packs 96 accelerators into a 167-kilowatt power envelope, delivering 27.6 terabytes of HBM capacity and 1.7 petabytes per second of aggregate memory bandwidth in a single scale-up domain. For comparison, Nvidia's most recent GB300 AI server racks put 72 GPUs in a 142-kilowatt envelope. Microsoft claims the Maia 300 system delivers about 40 percent more useful computing per megawatt than "leading GPU-based systems," though it did not name the systems it measured against.

A million chips is not a product launch. It is Microsoft declaring that the economics of AI will be settled in silicon, not software.

"It's basically delivering more intelligence for every watt, every dollar, and every byte of memory by optimizing across the entire system," Rani Borkar, president of Azure hardware systems and infrastructure, told Reuters in an interview. Production deployments begin in 2027, and Microsoft said it expects Maia 300 to run multiple gigawatts of AI workloads starting next year.

Why the hurry

The economics are hard to argue with. Microsoft has said it expects to spend more than $175 billion in capital expenditures this calendar year, much of it on AI data centers. Every inference query that runs on Microsoft-designed silicon instead of Nvidia's GPUs is a query the company keeps more margin on. As agents and copilots push query volumes ever higher, the chip bill becomes one of the largest line items in the AI business. Owning the chip is owning the cost curve.

Microsoft is not early to this realization, and it knows it. The company introduced its first Maia chip in November 2023 and the second-generation Maia 200 in January of this year, but it has lagged rivals in scaling in-house silicon. Google is on its eighth generation of TPUs and began recognizing revenue from direct TPU sales in the quarter ended June. Amazon's Trainium line is seeing growing adoption. Meta and OpenAI have only recently started their own chip programs. The Maia 300 announcement is, in part, an admission that Microsoft needed to move faster, and a plan for doing so.

The Nvidia question

None of this means Nvidia is about to become irrelevant. The company remains the default for training and for much of inference, and it is adapting: rather than fighting every custom-silicon startup, Nvidia has begun investing in them, including a recent investment in inference chip startup d-Matrix. But the Maia announcement lands amid signs that Nvidia's total grip is loosening. Former Groq engineers are suing over the structure of Nvidia's $20 billion acquisition of Groq, alleging the deal moved billions away from minority shareholders. The AI chip market is consolidating and fracturing at the same time.

The real contest may not be Maia versus the GPU at all. It may be cloud versus cloud. Custom silicon is becoming a selling point for the cloud providers themselves: Microsoft's Azure and its Foundry AI platform compete with Amazon Web Services and Google Cloud to host companies' AI workloads, and a cheaper, more efficient inference stack is a direct weapon in that fight. Google sells its TPUs directly. Amazon courts customers with Trainium. Microsoft is now, finally, playing the same game at scale.

What could go wrong

Scale is the operative word, and it is also the risk. Reports earlier this fall said Microsoft was in talks with TSMC to secure capacity for more than 300,000 Maia 300 chips for delivery in 2027, with ambitions above one million over time, numbers the company has not confirmed. Chip supply, advanced packaging capacity, and power availability are all real constraints on a multi-gigawatt rollout. A 167-kilowatt rack does not plug into yesterday's data center; it demands liquid cooling and serious grid capacity.

Then there is the software moat. Nvidia's CUDA ecosystem is a decade of developer investment that no chiplet can dissolve overnight. Microsoft's bet is that inference, where models are increasingly standardized, is less CUDA-dependent than training. If that holds, Maia 300 could do for AI inference what Google's TPUs did for the company's own workloads: disappear into the infrastructure and quietly change the economics. The 40 percent efficiency claim deserves the usual vendor-benchmark skepticism, since the comparison systems were unnamed. But the direction is unmistakable: the companies spending hundreds of billions on AI infrastructure have decided they would rather own the silicon than rent it forever.