Microsoft’s Mustafa Suleyman says its AI outperforms doctors on hard cases as he chases ‘medical superintelligence’
Microsoft unveiled MAI‑DxO, an orchestrated suite of AI agents that interview patients, order tests, and debate diagnoses. In a Science‑published simulation using New England Journal of Medicine cases, DxO hit 83% correct diagnoses, while solo primary‑care doctors and internists managed only 36% with reference tools. The boost stems largely from the underlying large language models, especially OpenAI’s GPT‑5 and the newer GPT‑5.5, with DxO’s coordination layer shaving roughly $3,000 off the $8,400 per‑case cost of a bare GPT‑5 run while nudging accuracy up a few points. Gains were most pronounced on complex, specialist‑level cases; emergency‑department scenarios showed little improvement, indicating DxO excels when baseline models are weaker or tasks are highly intricate. Microsoft tested a cross‑section of models—from Anthropic to Meta’s DeepSeek—demonstrating that even modest models like Gemini 2.5 Flash rose from 46% to 69% accuracy when paired with DxO, though the gap between top and bottom models remained larger than DxO’s incremental lift.
This result lands Microsoft squarely in the escalating AI‑in‑healthcare arms race. Google recently released AMIE, evaluated in a Lancet study with Beth Israel Deaconess, while OpenAI’s own diagnostic tools are being piloted in hospitals. By openly benchmarking against generalist physicians rather than specialist teams, Microsoft sidesteps a direct head‑to‑head with existing expert panels but still stakes a claim of superiority over the average clinician. The partnership with the Mayo Clinic to build a dedicated healthcare foundation model and the upcoming DxO field trial at Beth Israel Deaconess suggest Microsoft is moving from lab‑only metrics to deployment pipelines, echoing the “flywheel” growth model Suleyman cites from software coding to clinical decision support.
If DxO’s simulated edge translates into real‑world settings, it could reshape triage, specialist referral, and cost structures, but several risks loom. The study’s exclusion of specialist physicians raises questions about comparative performance in actual care pathways. High per‑case compute costs—even after reductions—may limit scalability without further optimization. Regulatory scrutiny will intensify as AI suggestions influence treatment decisions, and any misdiagnosis could expose Microsoft to liability. Watch for the outcomes of the Beth Israel Deaconess trial, the rollout timeline of the Mayo‑clinic‑trained foundation model, and how cost‑efficiency evolves as newer models like GPT‑5.5 become mainstream.
Key Takeaways
MAI‑DxO achieved 83% diagnostic accuracy on complex cases, more than twice the 36% rate of solo generalist physicians in a controlled simulation.
The system’s biggest performance lifts appear on difficult, specialist‑level cases and when paired with less capable base models.
Microsoft’s strategy combines model orchestration with bespoke healthcare foundation models, aiming to move from benchmark studies to live hospital deployments.
Real‑world validation at Beth Israel Deaconess and cost‑reduction pressures will determine whether DxO can overcome regulatory, liability, and scalability hurdles.
About the Source
This analysis is based on reporting by TechCrunch. Here is a short excerpt for context:
Microsoft’s new AI diagnostic system beat generalist doctors on hard cases, according to a study in Science. But most of the improvement seems to come from the AI models underneath, not from Microsoft’s system itself.Read the original at TechCrunch
