Microsoft Published the Training Receipts for MAI-Base-2

Microsoft AI announced MAI-Base-2 on Thursday, a 2.8-trillion-parameter base model it trained in-house and plans to license to businesses, according to a Microsoft AI blog post. A base model is the raw result of pre-training, before the tuning that turns it into a chatbot.

The post is unusually specific about how the model got built. It names the hardware, the length of the run, how busy the chips were and how often training stalled.

My read is that those figures matter more than the benchmark chart. A company selling model weights needs buyers to believe the next model will come quickly and cheaply, and the run log is Microsoft’s evidence.

The run, in numbers

The training ran on 16,000 Nvidia GB200 chips for 38 days. That works out to about 608,000 GPU-days, roughly 14.6 million chip-hours.

Average MFU was 21%. MFU, or model FLOPs utilization, measures how much of a chip’s theoretical peak math actually went into training the model.

Goodput, the share of time training was actually moving forward rather than stalled or recovering, was 89%. The model learned from a 50-trillion-token training mix, with tokens being the word fragments models read. Microsoft says its pool of unique tokens grew about 36% from MAI-Base-1, and that it processed every training token in-house.

Large training runs rarely get close to a chip’s peak, and Microsoft normalized its figure against the GB200’s theoretical maximum of 2.5 petaflops, excluding some overhead math. The disclosure gives outsiders a rare look at what a frontier-size pre-training run costs in chip time.

Where the gains came from

Microsoft attributes much of the improvement to engineering on its own software. Fixing inefficiencies in its mixture-of-experts layers raised their MFU from 24% to 35%, the post says. Mixture-of-experts is a design where only a slice of the model handles each word.

That fix let the team use mixture-of-experts in every layer, which MAI-Base-1 couldn’t do efficiently. Microsoft says MAI-Base-2 is 6.3 to 12.7 times more compute-efficient than MAI-Base-1, which it released in June.

That is about four months between models. The post says preparations for MAI-Base-3 are already underway while MAI-Base-2 goes through post-training, the tuning stage that follows pre-training.

The partner with a substitute

Microsoft has invested billions in OpenAI and holds a stake “likely worth more than $100 billion,” Business Insider estimates. The public blog post compares MAI-Base-2 only to open-weight models and does not mention OpenAI.

The comparisons Microsoft does publish are its own held-out evaluations, test sets kept out of training. It scored models with bits-per-byte, a measure of how well a model predicts unseen text that works the same regardless of how each model splits words. For code, it used its own codebase, and it commissioned STEM, math and Q&A tests from vendors to reduce the risk that the test material leaked into training.

The post also says it compared against several models where only tuned versions were available, and it flags that tuning may affect the results.

What Foundry customers get

Microsoft says owning the chips, software and model together lets it tune them for “the lowest cost, first in our own products and then for Foundry customers.” Foundry is Microsoft’s service that sells AI models to businesses.

Sources

• Microsoft AI: Frontier Pre-training at Microsoft AI
• Business Insider: Microsoft AI chief says the company beat OpenAI at its own game, using a quarter of the computing power