Objective:

To evaluate whether Diagnostic Orchestrator (DxO), an AI system that selects tests and updates diagnoses as evidence emerges, can improve diagnostic accuracy and balance accuracy with estimated testing costs.

Approach:

• System: Microsoft researchers developed DxO using large language models to simulate a panel of clinical experts. It begins with limited information, asks questions, requests investigations, and revises its diagnosis.
• Evaluation: Researchers used SDBench, comprising 304 New England Journal of Medicine clinicopathological conference cases and 325 emergency department encounters. They assessed diagnostic accuracy and estimated investigation costs.

Key Findings:

• Using GPT-5, DxO achieved 83% accuracy on clinicopathological cases, compared with 36% for physicians with reference materials.
• For emergency department cases, DxO achieved 80% accuracy versus 47% for physicians.
• On clinicopathological cases, DxO achieved 81% accuracy at an average estimated testing cost of $2,241; physicians achieved 36% accuracy at an estimated cost of $4,725.
• DxO generally asked more questions than clinicians before ordering investigations.

Interpretation:

The researchers report that DxO may support diagnostic test selection and help determine when sufficient evidence has been gathered. The system can request examination findings but cannot perform physical examinations or directly observe patients.

Limitations:

• The evaluation used controlled, text-based conditions and excluded pathology slides, imaging data, incomplete records, and multidisciplinary decision-making.
• Testing costs were estimates, not observed expenditures.
• Further evaluation is needed to establish performance in real-world practice and effects on patient outcomes.

Conclusion:

The study suggests that AI could contribute to diagnostic stewardship, but the reported results do not establish effectiveness in routine clinical practice.

Sources:

• Article context