Clinical Scorecard: AI Diagnostic Tool Shows Higher Accuracy Than Physicians in Simulated Cases

At a Glance

Key Highlights

• The SDBench framework included 304 clinicopathological conference cases and 325 emergency department encounters.
• Using GPT-5, DxO achieved 83% accuracy on clinicopathological cases versus 36% for physicians with reference materials, and 80% versus 47% on emergency department cases.
• For clinicopathological cases, DxO achieved 81% accuracy at an estimated average testing cost of $2,241; physicians achieved 36% accuracy at an estimated cost of $4,725.
• DxO generally asked more questions than clinicians before ordering investigations.
• The evaluation was text-based and excluded physical examination, pathology slides, imaging data, incomplete records, and multidisciplinary decision-making.

Guideline-Based Recommendations

Diagnosis

• The study reports that DxO asks questions, requests investigations, and updates its diagnosis as evidence becomes available.
• The researchers state that the evaluation was conducted in simulated, text-based cases and did not establish performance in routine clinical practice.

Management

• The researchers describe potential use of AI to assist clinicians with diagnostic test selection and diagnostic stewardship.
• The researchers state that further evaluation is needed to establish effectiveness in real-world practice and effects on patient outcomes.

Monitoring & Follow-up

• The study assessed diagnostic accuracy and estimated investigation costs.
• Testing costs were estimates, not observed expenditure.

Risks

• DxO can request examination findings but cannot perform physical examinations or observe patients directly.
• The controlled text-based evaluation excluded pathology slides, imaging data, incomplete records, and multidisciplinary decision-making.

Patient & Prescribing Data

No patient treatment or prescribing population was evaluated; the study used simulated clinicopathological cases and emergency department encounters.

The article reports diagnostic performance and estimated testing costs, not treatment efficacy, medication use, or prescribing outcomes.

Clinical Best Practices

• The researchers note that accuracy and cost comparisons were obtained in a controlled, text-based evaluation.
• The researchers state that further evaluation is needed to establish real-world effectiveness and effects on patient outcomes.
• The article reports that DxO can request examination findings but cannot perform physical examinations or observe patients directly.

Related Resources & Content

• Study published in Science; evaluation using SDBench