Pathologists and AI – Working Together
To enhance the work of pathologists, artificial intelligence must be responsive to their needs
Pathologists across the world frequently share patient cases on social media to get input from peers. James Nix and colleagues found that pathologists in developing countries often use social media, and 22 percent of the posts analyzed were seeking opinions on diagnosis (1). But social media does more than bring subspecialty expertise to hospitals where it may not otherwise be available. Social media increases access to pathologists and other physicians worldwide, potentially improving patient care – in line with the United Nations’ Sustainable Development Goal #3: good health and wellbeing (2).
But how can we make social media work better for pathologists? We’ve developed a new tool, called “pathobot,” to help (3). Given H&E photomicrographs of a patient case and a brief text description, pathobot finds similar patient cases on social media and PubMed and predicts the disease state of a case (nontumor/infection, benign/low-grade, or malignant) (4). The synergistic relationship between artificial intelligence (AI) and responsive pathologists is crucial for pathobot’s success. When a pathologist on Twitter mentions “@pathobot,” other pathologists can see the tool work – and, in some cases, they are inspired to share and collaborate with us (5,6). With more case data, pathobot’s AI improves and its search database expands to cover more disease entities. Even better, pathobot can be used repeatedly as pathologists further characterize a specific case – including differentials, immunohistochemistry, additional H&E photomicrographs, and more (4).
In AI, we often regard data as a static corpus – but we must remember that these data come from pathologists who are living, breathing, caring people. Pathologists absolutely want to share and discuss cases on social media – and that’s precisely where AI can help. Specifically, AI can do the “grunt work” to find similar cases, find key pathologists worldwide with similar cases, bring pathologists together to discuss the next steps in a patient’s care, adapt search results as more information becomes available, and grow as more pathologists collaborate with us. Our approach has been so well-received that we won the #PathVisions2019 Poster Award for Best Image Analysis (5,7).
But it’s not enough to simply offer a tool for pathologists who are already engaged to use. To reach the greatest possible number of pathologists and patients, let’s get a smartphone on every microscope. Why a smartphone? With the advent of COVID-19 and the concomitant increase in telepathology, sharing photomicrographs and teleconferencing via smartphone is more common than ever. Additionally, residents may not have access to microscope-mounted cameras, but still wish to share cases, so an inexpensive way to mount a smartphone to a microscope may help with education.
To this end, we have begun to address two main challenges:
- How can one cheaply mount a smartphone to a microscope? To reduce costs, we 3D print a system of parts called #pathobox (8), which we ship for free through our organization, @pathobotology (9). (We appreciate donations, particularly from well-resourced institutions or crowdfunding. Currently, I fund these efforts personally, with support from Mariam Aly and family.)
- How can one cheaply approximate a whole-slide image from the limited field of view a smartphone and microscope offer? Whole-slide images are important in digital pathology, at least in part because these images provide more context than a single field of view at a microscope. Unfortunately, whole-slide image scanners may be prohibitively expensive for low-resource regions or hospitals, and whole-slide images are far too large to post on social media (10,11). Therefore, we freely provide the #pathopan tool (12), which stitches together overlapping photomicrographs to form a small, low-quality whole-slide image – similar to the way a smartphone stitches together overlapping photos to form a panorama.
To improve the pathobot AI and ask increasingly sophisticated questions of our data, we are always looking to collaborate more widely with pathologists. Our goal? To leave tedious work – like manufacturing smartphone mounts and matching new cases to similar ones in our archives – to the robots and devote ourselves to the challenges only we can take on. It takes a human to collaborate with colleagues, to address evolving health challenges, and – most important of all – to care for a patient.
- J Nix et al., “Neuropathology education using social media”, J Neuropathol Exp Neurol, 77, 454 (2018). PMID: 29788115.
- BX Lee et al., “Transforming our world: implementing the 2030 agenda through sustainable development goal indicators”, J Public Health Pol, 37, 13 (2016). PMID: 27638240.
- AJ Schaumberg et al., “Interpretable multimodal deep learning for real-time pan-tissue pan-disease pathology search on social media”, Mod Pathol, [Epub ahead of print] (2020). PMID: 32467650.
- AJ Schaumberg et al., “Finding similar pathology cases on social media and PubMed via a helpful @pathobot, with an eye towards COVID-19” (2020). Available at: bit.ly/2N2P7fC.
- AJ Schaumberg et al., “Machine learning for real-time search and prediction of disease state to aid pathologist collaboration on social media”, J Pathol Inform, 11, 1 (2020).
- Pathobot (2020). Available at: twitter.com/pathobot.
- AJ Schaumberg (2020). Available at: bit.ly/3cM0ymq.
- AJ Schaumberg, “Pathobox” (2020). Available at: bit.ly/3dKcXJ0.
- AJ Schaumberg, “Pathobotology” (2020). Available at: bit.ly/3h7FJ8D.
- MC Montalto, “An industry perspective: an update on the adoption of whole slide imaging”, J Pathol Infom, 7, 18 (2016). PMID: 27141323.
- MD Zarella et al., “A practical guide to whole slide imaging: a white paper from the Digital Pathology Association”, Arch Pathol Lab Med, 143, 222 (2018). PMID: 30307746.
- AJ Schaumberg, “Pathopan” (2020). Available at: bit.ly/2MDbqIC.
Research Fellow at Harvard Medical School, Boston, Massachusetts, USA.