Pathobot: Deep Learning for Humans and Machines

When you’d appreciate a quick second opinion, what do you do? You might check with on-site colleagues or reach out to others via your smartphone. But, as COVID-19 continues to spur the rise of telepathology, social media is an increasingly popular platform for sharing and discussing deidentified cases. “Social media makes my life more exciting,” says Celina Stayerman, a pathologist at Laboratorio TechniPath in Honduras. “And though I’m the only pathologist where I work, I’m never alone.” Olaleke Folaranmi, of Nigeria’s University of Ilorin Teaching Hospital, adds, “Some challenging cases I have posted to social media have posed great learning points for us. For example, the Nikolsky sign is negative in toxic epidermal necrolysis, but positive in disseminated varicella.”

Within this vibrant international community of pathologists on social media, hierarchies are flattened and a new kind of organizational structure has emerged – around hashtags. Jerad M. Gardner, a pathologist at Geisinger Medical Center in Pennsylvania, explains, “After organizing the first live Tweet group at the United States and Canadian Academy of Pathology (USCAP) meeting in 2015, we realized we needed a formal list of subspecialty hashtags, e.g., #breastpath, #dermpath, and #gipath (1). The USCAP Social Media Subcommittee compiled this ontology (2), submitted it to Symplur, and we’ve all been using it since.”

Years later, this structure attracted attention from computational fields. “We wondered if the hashtag-labeled photomicrographs on social media were data that could teach an artificial intelligence (AI) simple histopathology tasks,” says Andrew Schaumberg, a postdoctoral fellow at Brigham and Women’s Hospital in Massachusetts. “It turned out to be more complicated than the three-month summer project we anticipated…”

Fast forward to 2020 and this international group of 30 pathologists, computational scientists, and neuroscientists have published their study (3). Their work produced “pathobot,” an AI-driven bot that searches social media to connect pathologists with similar cases. From pathology AI to 3D-printed smartphone-to-microscope mounts, we catch up with the group’s endeavors to increase access to pathologists worldwide.

How it all began
Schaumberg cites a colleague as his inspiration: “Mariam Aly introduced me to Twitter, and I noticed pathologists posted photomicrographs. What a great source of data to download!”

Aly, who is an assistant professor at New York’s Columbia University, says, “I’ve long thought that Twitter is useful for keeping up with – and sharing – science. Andrew didn’t believe me at first – but he finally caved!”

After discussing their idea, the two decided to begin by consulting an Institutional Review Board and obtaining informed consent before downloading the (anonymized) data. But, despite their eagerness to begin, their need for help went beyond ethical implications. “I had a good experience mentoring a high school student the prior year,” says Schaumberg. “I figured that, if I mentored two students at once, we’d start and finish this project in the summer of 2018!”

“Naturally, projects take time, but the global scale of the effort was very enticing,” explains Thomas Fuchs, Co-Director of the Hasso Plattner Institute for Digital Health at the Icahn School of Medicine at Mount Sinai in New York. “It is a pristine example of how AI can help to democratize knowledge and be helpful worldwide – so we decided to proceed with this project in my laboratory.”

The first pathologist to consent was Mario Prieto Pozuelo – a pathologist at Hospital Universitario HM Sanchinarro, Spain, who not only provided his data, but also wrote a three-page introduction to fluorescence in situ hybridization (FISH) to explain his cases. “It was a simple thing,” he says. “There were many good questions. I’m happy to teach.” Schaumberg highlights their good luck in finding many approachable pathologists early in the project, citing both Pozuelo and Laura G. Pastrián – a pathologist at Spain’s Hospital Universitario La Paz – whom he says helped build his confidence in asking questions until he began recognizing slides himself. “Path Twitter is a fun place to share educational cases like these,” says Pastrián, highlighting one of social media’s greatest strengths.

An eye to AI
Schaumberg says, “Training an AI to predict whether or not an image was H&E was the low-hanging fruit we did first. Beach photos are not H&E (surprise!), and neither are chest X-rays.” But even this basic stain presents a challenge for a computer.

Aurélien Morini, a fifth-year resident at Université Paris Est Créteil, explains, “In France and elsewhere, H&E may include saffron to highlight collagen. Phyloxin may be included instead of eosin – but these are all still essentially H&E.”

Schaumberg adds, “Diff-quik, PAS, CISH, trichrome, and even some red-variant IHC stains may be easily confused with H&E, both to an untrained eye and to AI (see Figure 1). My mentees and I had a lot to learn!”

Next, the team took on harder tasks, such as training an AI to distinguish tissue types in the various subspecialties. Such tasks are simple for human pathologists, meaning an AI that struggles with them may not add much value. Distinguishing between benign and malignant tissue, on the other hand, might save significant time that pathologists could then devote to more complex problems. Unfortunately, that distinction varies from one tissue to the next – and the algorithm didn’t have a lot of data. To learn the difference, the creators first had to rigidly define both extremes.

“All disease is on a continuum,” says Pastrián. “There is no hard line between ‘benign’ and ‘malignant’ – and some things, such as infectious disease, are neither.” Colleagues add that the distinction often determines whether or not a patient will undergo surgery – and what the patient’s outlook is over the next six months or longer. Stephen Yip, a pathologist at BC Cancer in Canada, offers an example: “The acknowledged definition of ‘malignant’ in epithelial cancers is the ability to breach the basement membrane to invade into the adjacent tissue, lymphatics, and blood vessels. Extensive invasion can mean this is no longer treatable with surgical resection.” He explains that, although cytological appearance is typically associated with malignancy, the infiltrative nature of some tumors (such as primary diffuse CNS glioma or chordoma) means they are considered malignant even with “benign” cytology.

In pathobot’s case, “containment” largely defined malignancy, but a number of other factors help define what the AI can – and cannot – do. For instance, it is not designed to predict whether or not a patient should get surgery. “It can also be helpful that AI learns on a case-by-case basis in a data-driven manner,” says Schaumberg. With enough cases that carry a consensus opinion (benign, malignant, infectious, and so on), the AI can generalize a definition of each concept. “Unfortunately, hard lines are a necessary evil for an AI to learn distinctions like ‘benign versus malignant’ – even though disease in general is on a continuum. Perhaps, with more data, the AI will need fewer hard lines and assumptions to accurately learn.”

He goes on to explain how pathobot works. “Given a photomicrograph, the AI is basically trained to answer a multiple-choice quiz question about what the photomicrograph depicts: a) nontumor/infection, b) benign, or c) malignant disease. However, ‘benign’ was a grey area, especially for disease that may become malignant soon.” Collaborator S. Joseph Sirintrapun, Director of Pathology Informatics and a pathologist at Memorial Sloan Kettering Cancer Center, says, “We agreed to call this grey area ‘benign/low-grade malignant potential.’”

Gardner adds, “Some prior work of ours similarly classified all disease as one of three categories: non-neoplastic, benign, or malignant (4).” But did that consensus definition of disease work for pathobot?

Schaumberg says no. “The AI’s disease prediction performance was horrible at first!”

Defining the details
To improve pathobot’s performance, Schaumberg implored his pathologist collaborators to give him clues – and they stepped up. Morini says, “I reviewed all my cases shared with Andrew, and how they were annotated. A pathologist posts one to four images in a tweet to begin to describe a case. Some images show only benign tissue, whereas others show the malignancy. The photomicrographs in a tweet are not necessarily all benign – or all malignant.”

Sanjay Mukhopadhyay, Director of Pulmonary Pathology at the Cleveland Clinic in Ohio, says, “We thought it would be fair for the AI to be given both the photomicrograph and the tissue type, because pathologists know this, too. Tissue type matters because infectious disease is more common in pulmonary pathology than hematological pathology, for example.”

Folaranmi agrees. “Speaking of tissue types,” he says, “pathological processes, like Langerhans cell histocytosis, may be a daunting task. For instance, in lung, Langerhans cell histocytosis is considered a smoking-related reactive/non-neoplastic disease. However, in other tissues, Langerhans cell histocytosis may be considered neoplastic instead. Context matters.”

Once the AI (and its creators) had been fully trained, it was deployed as “pathobot” on Twitter – where pathologists liked the bot so much, they contributed their own data to help improve its performance. But how did pathobot transition from predicting disease to seeking out and connecting pathologists with similar cases? “Many AIs learn to predict in a way that also gives a similarity metric,” explains Schaumberg. “To such an AI, it’s as though some diseases are ‘closer together’ than others. So our AI that has learned to make accurate predictions gives us search capability ‘for free.’”

Networking skills
“If you type a search query into a search engine, you are the only one typing. In contrast, pathobot uses context from discussion threads surrounding a case on social media, so many pathologists are typing, thinking, and searching together,” says Schaumberg. “This is one way pathobot tries to leverage ‘more brains’ to search for similar cases. The notifications it sends when its search results link to their similar cases are another way to bring in more pathologist brains.”

But the most important question is – does it work? Pathologists agree that it does.

Stayerman says, “In my experience, pathobot finds similar cases (see Figure 2). These tend to be a mix of recent cases and others from a few years ago. The older cases, and the older discussions for those cases, are otherwise difficult to find in Twitter history.” Like Stayerman, many pathologists use social media to “check their work” – for example, by comparing their diagnostic impressions or differential diagnoses to those of colleagues. But when a case is unusual or there’s no time to spare, pathobot serves the same purpose. “Searching for appropriate older cases to review can be a prohibitively time-consuming task when there is no time to spare. Pathobot can help find these cases, uncovering helpful colleague discussions from the past. Reviewing them is definitely another useful check for me!”

Mukhopadhyay says, “I have tested pathobot occasionally for over a year. For the cases I’ve tested, I am impressed that pathobot’s histopathology search results are similar to my test cases and that they are quickly produced. Lately, I’ve found that pathobot’s horizons have expanded, with pathologists who have not been a part of its development using it.”

And Mukhopadhyay’s cases have been useful to others as well – Sofopoulos Michail, a consultant histopathologist at St. Savvas Anticancer Hospital in Greece who has also conducted occasional pathobot tests, says, “I was glad to get access to pathobot for a challenging mediastinal mass. Pathobot identified several cases similar to mine, including a case from Sanjay Mukhopadhyay.”

Folaranmi has also conducted pathobot tests since 2019. “I remember sharing a case of intravascular papillary endothelial hyperplasia and mentioning pathobot to trigger a search. Pathobot’s social media case database was smaller in 2019; however, it managed to find an intravascular papillary endothelial hyperplasia on PubMed and correctly predicted the case as benign. I think there is educational value in the search results pathobot finds.”

That was a bumper year for social media interactions – and for pathobot. Daliah A. Hafeez, a pathologist at Saudi Arabia’s King Fahad Armed Forces Hospital, says, “There were rounded structures in a liver subcapsular collection case of mine, and it wasn’t clear if these structures were helminth eggs, nematodes, or lentils (see Figure 3).”

At this time, in early 2019, pathobot’s PubMed search did not yet exist, but Schaumberg had internal tools that could search PubMed. Hafeez says, “Pathobot found a similar case of lentil on social media (1,5) and his PubMed searches found a similar case of food residue mimicking disease in a patient with a history of emergency surgery (6). We’re all working together here. I found the searches and literature reference helpful.”

Stayerman adds, “I’ve used pathobot’s keywords and requires commands to focus search results on a specific entity, for instance papillary lung adenocarcinoma. This is helpful if I have a diagnosis in mind already, especially a rare entity.”

Pastrián, who is internationally renowned for her expertise in seeds, has a specific rare entity in mind. “I’m waiting for pathobot to make a seed atlas of all the beautiful tomato seeds and lentils on social media!” A pipe dream? Perhaps not. Pathobot’s ability to search for cases of a more vegetarian nature has already been tested with some success – though the algorithm still has difficulty identifying seed species, such as soy. Schaumberg would like to address that in a future iteration of pathobot. “I think fixing that would be ‘soy’ much fun!”

Data in, data out
It’s clear that pathobot has found favor with diagnostic professionals on Twitter – but what of computational colleagues in the lab? Schaumberg recalls presenting pathobot in a recent lab meeting and getting his audience excited about the possibilities. “Pathobot sounded like a fun project,” says Richard Chen, a PhD candidate at Brigham and Women’s Hospital. “I wondered if I could query for similar cases in real-time during his presentation.” But Chen lacked one key thing – authorization for pathobot searches. Immediately after the presentation ended, he requested it. (“And he didn’t let me forget!” adds Schaumberg.)

“I queried with a whole-slide image region-of-interest in glioblastoma (no IDH mutation or 1p19q codeletion), and Pathobot retrieved cases of glioblastoma from Twitter and PubMed!” says Chen. “Some of these had similar molecular alterations. Impressive!”

“It was probably luck that pathobot handled all that negation correctly,” Schaumberg admits. “I was just saying in the lab meeting that ‘no,’ ‘not,’ and ‘absence’ can be challenging. For pathobot, we chose Twitter specifically because tweets are limited to 280 characters – so people tend to keep their language simple.”

Pathobot’s creators are now starting to do more with published cases from PubMed – for instance, sharing PubMed-based pathology quizzes daily on Twitter and inviting users to post their diagnoses and receive feedback. Schaumberg hopes that, in the future, this kind of data-gathering will power a smarter and more well-rounded pathobot. “There are a lot of whole slide images at The Cancer Genome Atlas that we’d like pathobot to search as well,” he adds. “This is still at a preliminary stage, but it’s important as more hospitals consider whole-slide image-based digital pathology.”

Introducing pathobox
Schaumberg also hopes to get as many students, residents, fellows, and pathologists as possible contributing to case discussions and helping each other, so he gives away 3D-printed “pathoboxes.” A pathobox mounts a smartphone, camera, or iPad to a microscope eyepiece for photos or video conferencing (see Figure 4) – and the devices, which Schaumberg produces at home, have begun to draw in new colleagues. “They fit in well with a number of our efforts in the lab, from 3D-printing to computational pathology,” says Faisal Mahmood, an assistant professor at Brigham and Women’s Hospital. “There are exciting applications of such methods in low-resource settings and elsewhere.”

And pathobox is slowly making its way from one pathologist to the next. S. Jeremy Minkowitz, a first-year resident at SUNY Downstate Pathology, recommends pathobox for sharing photos and videos with colleagues. “I used a pathobox from home as COVID-19 hit New York City. My father and I also used it at his private pathology practice. The pictures were pretty good – and video conferencing worked with it too. Although whole-slide images have their place in pathology, pathobox can be a nice, cheap, fast alternative that assists with ‘quick questions.’”

Minkowitz and Schaumberg collaborated to improve the device’s design – but some problems remain insurmountable. “I left it in my car during the summer and the pathobox warped from the heat,” confesses Minkowitz. “Wouldn’t recommend leaving it on a car seat in the sun!” 

Some pathoboxes, however, made their way to colder climates (see Figure 5). “My pathobox arrived in the middle of the pandemic in Saskatchewan,” says Henrike Rees, a pathologist at Saskatchewan Health Authority in Canada. “I had started to work from home in mid-March of 2020, so teaching pathology residents remotely became a new challenge. The pathobox has helped me connect with my residents and teach them from my home office. Because it is a modular system, it can be easily adjusted for a variety of smartphones or tablets.”

Second-year anatomical pathology resident Ariel Gershon agrees. “The pathobox is a great device. Here in Toronto, I use it to share images for informal consultation with friends or to save slides for viewing at home. When taking pictures by hand, I always moved slightly at the last moment, obscuring the picture. I had some trouble putting pathobox together initially, but Andrew kindly and quickly put together instructions on a YouTube video addressing my concerns (7).”

In April 2020, Schaumberg also tried shipping pathoboxes from the US to colleagues in Nigeria – but COVID-19 eliminated all shipping transport between the two countries, so the devices returned. Fortunately, the delay gave creators the opportunity to improve the design – and these updated pathoboxes were shipped in February 2021. The highly anticipated shipment arrived in March, but there was bad news. “Unfortunately, the pathobox did not fit my microscope. The eyepiece is too wide,” says Dauda Suleiman, a pathologist at the Abubakar Tafawa Balewa University Teaching Hospital in Bauchi, Nigeria.

“It’s a shame the pathobox design isn’t truly universal yet. No amount of heat or acetone could reshape the pathobox on-site to accommodate the eyepiece. So there’s room to improve the design, make things right, and keep testing as broadly as we can,” admits Schaumberg.  Unperturbed, Suleiman shipped the pathobox to a colleague, also in Nigeria – and, in May 2021, their luck improved.

“The pathobox was not hard to set up once I’d watched Andrew’s video,” says Nnamdi Orah, a pathologist at the College of Medicine, University of Lagos, Nigeria. “It needed a bit of tinkering to get the best position for my phone (see Figure 6). Once positioned, it worked very well. I expect the pathobox will be of great use to me in sharing pictures with my colleagues. Thank you very much!”

Pathobot and diversity
“For me, diversity protects against bias, both at the human and algorithmic levels,” says Schaumberg. “Diversity means we’re aware that H&E isn’t only H&E in every country – we know there may be saffron in the stain in France as Aurélien says, for instance. We have to handle the fact that H&E looks different across different institutions, countries, and photomicrographs; for instance, if the H&E appears more red (see Figures 8C and 11B) or pink (see Figures 1C1 and 8G) or brown (see Figure 8D) in enough cases, then the AI may be able to learn that the stain’s color is less important than its location on the slide. With diversity, we also know that the same disease might get surgery at one institution, but not another – as Joseph pointed out to us for the differential diagnosis of atypical lobular hyperplasia (ALH) or lobular carcinoma in situ (LCIS) – which is one reason we avoid predicting whether or not a patient needs surgery. Diversity forces us to reach consensus and do better.” In his view, that involves defining disease in a way that works for all users – and then working on an AI that works toward that consensus. “Sometimes, we’re defined more by our weaknesses than by our strengths,” Schaumberg observes. “Diversity means that the group as a whole is less likely to have people who all share the same weaknesses.”

But what does diversity mean to an AI? At an algorithmic level, a mechanical sort of diversity is built in. Pathobot, for instance, uses an approach known as random forest. “Part of our AI builds a committee of slightly randomized predictors, which can be thought of as a collection of individual ‘test takers’ that disagree with one another (see Figure 7). The overall AI’s prediction is a majority vote of these varied test takers,” explains Schaumberg. “We also use deep learning and ensembles to take majority votes for a mechanical sort of diversity that empirically works better (3). That said, these mechanical sorts of diversity won’t prevent AI mistakes if the data aren’t diverse enough, and there is a lot of great current work about AI methods to mitigate biases in non-diverse data. For us, though, we wanted to focus on collecting data from a diverse cohort of pathologists and patients to do the best we can. Diversity really comes from people, from data, and from inclusion.”

And because patient cases come from all over the world, diversity is baked into the data AIs such as pathobot study. Real-world data make it more difficult for AI to perform well, but provide a realistic look at remarkable pathology cases across many tissues and diseases. In machine learning, one often wants to know how an AI might be expected to perform in general on data the AI hasn’t seen before –sometimes called “generalization error.” Diverse international datasets can provide a good measure of how an AI can perform some tasks in general.

Of course, a single institution’s whole-slide library would not have this level of diversity. Rather, the data might be biased toward institution-specific protocols or the average socioeconomic status of its patients. If the institution specializes in a specific disease, there may be further biases in the data. For instance, a cancer center might not have infectious disease represented in its data at all – yet infectious disease is common in developing countries, whereas cancer is underrepresented. Diversity is fundamental to establishing the most general data with which to train the most general AI.

Answers in interpretability
“Interpretability in AI leads us to many interesting places!” says Schaumberg. “We revisited our definitions of nontumor/infection, benign/low-grade-malignant-potential, and malignant due to our AI’s interpretability.”

The AI had highlighted some hyperplastic cells as benign/low-grade-malignant-potential (8). “Some may argue hyperplastic cells are more closely ‘nontumor,’” says Gardner. Schaumberg agrees – so pathobot’s benign grey-area definition now explicitly includes pre-neoplastic disease.

“These conversations are a valuable consequence of interpretability,” Schaumberg says. “We take better advantage of our diverse expertise this way.”

Interpretability offers other benefits as well. For instance, Schaumberg and his colleagues observed that pathobot’s deep learning approach clustered cases together by disease state – so malignant cases were considered similar to other malignant cases and nontumor cases more similar to other nontumor cases. That’s a good thing – but, when they used features hand-engineered to represent color, texture, or edges, the team saw loose clusters of cases that were all from the same pathologist. “It’s a bad sign if the AI ‘thinks’ one pathologist’s cases are all similar to each other,” explains Schaumberg. “Hypothetically, if a pathologist tends to share malignant cases, but also uses a specific smartphone camera with a specific microscope and saves photos with specific JPEG compression artifacts… an AI may incorrectly learn that that camera, microscope, or JPEG artifact predicts malignancy.” Although these predictions of malignancy may be accurate, they will be accurate for the wrong reasons – meaning that the AI might also predict malignancy where none exists. Fortunately, the issue proved insignificant for pathobot. “In the end, we only saw a trace amount of potentially pathological behavior in the AI, so we were able to convince ourselves that the AI’s learning made sense.”

AI has a reputation as a “black box” because many users don’t understand its inner workings. Could interpretability “open the box” for pathologists? Schaumberg says, “Interpretability lets us see things that we haven’t seen before – such as what the AI ‘thinks’ are the core signatures of disease.” What has interpretability revealed about pathobot so far? “Loosely speaking, it tells us that the deep learning part of the AI learns to represent color and edges to accurately predict disease state. It also tells us that visual textures are important to the AI to accurately predict disease state – as, of course, is knowing the tissue type.”

Edges (continuous areas of bright pixels next to continuous areas of dark pixels) are easy for deep learning algorithms; they can arise from glands, nuclei, vasculature, and other structures. Colors are similarly easy to discern, although they can change for reasons ranging from hyperchromatic nuclei to understaining. (It’s important to keep in mind that over- and understaining can reduce AI accuracy.) Visual textures, in contrast, present a challenge. These occur where there is a specific pattern of pixels – for instance, a ring of dark pixels around a lighter pixel. Schaumberg says, “I’ve seen important visual textures around chromatin, dirty necrotic debris, in the tight spaces between cells where a few cells meet, and other places. I don’t see important visual texture so much in relatively homogenous cytoplasm, or in flat white background, or in vast swathes of pink acellular goop (see Figure 8C).”

He cautions that blur greatly reduces visual texture and therefore AI accuracy. Ultimately, pathobot’s deep learning algorithm did not effectively learn to represent visual textures, but the AI also had access to hand-engineered visual texture features called local binary patterns that assisted with disease state prediction. “Hopefully in the future we can help deep learning represent important pathology-related visual textures,” says Schaumberg. “But we’re not there yet.”

Why pathobot tweets

“Activities that seem trivial or playful to us, such as sharing interesting cases on social networks, can have interesting spinoffs, such as the creation of algorithms that predict the nature of a tumor or the presence of a somatic mutation, or tools such as pathobot that allow us to find similar cases or differential diagnoses,” says Morini. “When people are motivated and share their skills, the result is an enriching experience for everyone.”

Stayerman adds, “If someone had told me before I joined Twitter in 2018 that there was a vibrant worldwide community of pathologists eager to share their expertise, discuss cases, and respond to calls in a matter of seconds… I would have thought it was pure science fiction!” But for her, the Pathobot community stands out in the AI field because it grounds itself in a deep appreciation of pathologists’ ability to make a diagnosis – whether for a simple case with just a microscope, eyes, and brainpower or a more complex diagnosis involving ancillary studies. “I think pathobot paves the way for an easy connection between pathologists and the ever-growing data available in the pathology Twitter community.”

For most pathologists, asking for a “quick” expert opinion means physically visiting the offices of nearby colleagues with pertinent subspecialty expertise – or, if none are available, potentially mailing glass slides to more distant experts. However, pathologists in developing countries may not have the same opportunities – and they may have to make diagnoses without the benefits of ancillary studies or collaboration. Stayerman’s favorite thing about Twitter? “It allows every pathologist in every corner of the globe to be exposed to new entities. We also encourage active discussion by sharing up-to-date information and expert consensus on established entities as brief, enjoyable tweets. The constant exchange of knowledge, thinking processes, diagnostic criteria, work-ups, and great images – all this will ultimately help us standardize good practices worldwide.” In fact, she and other members of the #PathTwitter community have begun using the hashtag #PathTwitterFellowship – both because it describes explicitly what such tweets are about and because it highlights Twitter’s ability to contribute to pathologist education. “This platform serves dual purposes: acquiring the greatest repository of diverse pathology images and aiding the global standardization of good practices that will ultimately improve patient care,” says Stayerman.

Not a pathologist himself, Schaumberg has found himself impressed by Twitter’s pathology community. “There is so much to learn, and I’m grateful to the many colleagues who have taken the time to help me! After getting some generic familiarity with various stains and tissues, I still don’t have the expertise to really appreciate some of the challenging cases pathologists share. I’ll never have that expertise, but sometimes there’s a certain attention to detail that I find really striking.” Discussion of a particularly tricky case prompts him to add, “AI will never be as good as a pathologist, certainly within my lifetime, and probably many more. For me, it’s staggering that, from so much visual information, pathologists have the power to find a few cells that decisively change the diagnosis (see Figure 9). That’s why we use AI to connect pathologists – because a pathologist’s power to investigate and accurately diagnose a vast universe of possible diseases isn’t well-approximated by an AI that excels at some number of narrow benchmarks in isolation for controlled scientific study.”

So how can the pathology community maintain pathobot’s – and social media’s – momentum?  “Whether you’re a pathologist with cases to share or a data scientist interested in analysis, get in touch,” says Schaumberg. “I fund many of these efforts privately, with additional support from Mariam Aly and family. When I graduated in the summer of 2020, my grandmother gave me a monetary gift, so I decided to treat it like a microgrant for constructive purposes. This led to our 3D-printed pathobox giveaways, allowing us to help pathologists mount their smartphones to their microscopes for all kinds of telepathology – from photomicrographs to video conferences. Thanks, Grandma!”

His next goal? World domination – after a fashion. “Let’s mount a smartphone on every microscope and help each other out!”

If you’d like to get involved with future pathobot efforts, find out more at pathobotology.org