Pathologists and AI – Working Together

Pathologists across the world frequently share patient cases on social media to get input from peers. James Nix and colleagues found that pathologists in developing countries often use social media, and 22 percent of the posts analyzed were seeking opinions on diagnosis (1). But social media does more than bring subspecialty expertise to hospitals where it may not otherwise be available. Social media increases access to pathologists and other physicians worldwide, potentially improving patient care – in line with the United Nations’ Sustainable Development Goal #3: good health and wellbeing (2).

But how can we make social media work better for pathologists? We’ve developed a new tool, called “pathobot,” to help (3). Given H&E photomicrographs of a patient case and a brief text description, pathobot finds similar patient cases on social media and PubMed and predicts the disease state of a case (nontumor/infection, benign/low-grade, or malignant) (4). The synergistic relationship between artificial intelligence (AI) and responsive pathologists is crucial for pathobot’s success. When a pathologist on Twitter mentions “@pathobot,” other pathologists can see the tool work – and, in some cases, they are inspired to share and collaborate with us (5,6). With more case data, pathobot’s AI improves and its search database expands to cover more disease entities. Even better, pathobot can be used repeatedly as pathologists further characterize a specific case – including differentials, immunohistochemistry, additional H&E photomicrographs, and more (4).

In AI, we often regard data as a static corpus – but we must remember that these data come from pathologists who are living, breathing, caring people. Pathologists absolutely want to share and discuss cases on social media – and that’s precisely where AI can help. Specifically, AI can do the “grunt work” to find similar cases, find key pathologists worldwide with similar cases, bring pathologists together to discuss the next steps in a patient’s care, adapt search results as more information becomes available, and grow as more pathologists collaborate with us. Our approach has been so well-received that we won the #PathVisions2019 Poster Award for Best Image Analysis (5,7).

But it’s not enough to simply offer a tool for pathologists who are already engaged to use. To reach the greatest possible number of pathologists and patients, let’s get a smartphone on every microscope. Why a smartphone? With the advent of COVID-19 and the concomitant increase in telepathology, sharing photomicrographs and teleconferencing via smartphone is more common than ever. Additionally, residents may not have access to microscope-mounted cameras, but still wish to share cases, so an inexpensive way to mount a smartphone to a microscope may help with education.

To this end, we have begun to address two main challenges:

1. How can one cheaply mount a smartphone to a microscope? To reduce costs, we 3D print a system of parts called #pathobox (8), which we ship for free through our organization, @pathobotology (9). (We appreciate donations, particularly from well-resourced institutions or crowdfunding. Currently, I fund these efforts personally, with support from Mariam Aly and family.)
2. How can one cheaply approximate a whole-slide image from the limited field of view a smartphone and microscope offer? Whole-slide images are important in digital pathology, at least in part because these images provide more context than a single field of view at a microscope. Unfortunately, whole-slide image scanners may be prohibitively expensive for low-resource regions or hospitals, and whole-slide images are far too large to post on social media (10,11). Therefore, we freely provide the #pathopan tool (12), which stitches together overlapping photomicrographs to form a small, low-quality whole-slide image – similar to the way a smartphone stitches together overlapping photos to form a panorama.

To improve the pathobot AI and ask increasingly sophisticated questions of our data, we are always looking to collaborate more widely with pathologists. Our goal? To leave tedious work – like manufacturing smartphone mounts and matching new cases to similar ones in our archives – to the robots and devote ourselves to the challenges only we can take on. It takes a human to collaborate with colleagues, to address evolving health challenges, and – most important of all – to care for a patient.