Objective:

To evaluate the reliability of AI models in predicting cancer biomarkers from histology images and to distinguish between ideal models, which predict based solely on biological effects, and confounded models that rely on unrelated variables.

Key Findings:

• AI models often rely on related biomarkers and tumor characteristics rather than the target biomarker itself.
• Performance of models decreased significantly when stratified by related molecular features or histologic grade, highlighting the importance of subgroup analysis.
• Simple models using pathologist-assigned grade sometimes matched the performance of complex AI systems.

Interpretation:

AI models may capture correlated features rather than specific biomarker patterns, leading to misleading accuracy metrics that do not reflect true predictive power.

Limitations:

• Current AI approaches may not be robust enough to replace molecular assays.
• Aggregate accuracy metrics can overstate clinical utility without subgroup analysis.
• There is a need for more robust AI models to improve clinical utility.

Conclusion:

Bias-aware validation strategies are essential for deploying AI in clinical practice, and confirmatory molecular testing remains crucial for treatment decisions to ensure accurate patient care.