One Step Beyond: Artificial Intelligence for Image-Based Prognostication

It can be disheartening to hear pathologist colleagues say, “With artificial intelligence, I’ll be out of a job in 10 years.” It is reminiscent of 1999, when doomsayers expected Y2K to unleash technology Armageddon. The reality, of course, is more nuanced, but there is no denying that AI-related innovations will have a transformative effect on our discipline. Though many of its perceived benefits focus on improving diagnostic services, there is also scope to harness these innovations to bridge the gap between pathology and oncology.

When digital pathology and whole slide image (WSI) analysis transitioned from the research space into the clinical environment, they received a lukewarm welcome. Reluctance to engage with this technology is traceable to a lack of familiarity or training; belief that digital diagnosis is inefficient; higher levels of confidence in light microscopy; and individuals’ thresholds for embracing new technology (1,2,3). However, as scanners, algorithms, and perspectives have matured, WSI-based diagnosis has proven comparable to that with light microscopy (4,5,6) and numerous large centers across the globe now run partially or fully digitized pathology workflows (3,7).

This forward motion is key to the current state of affairs in histopathology. Globally, the discipline is under growing pressure – our increasing, aging population will yield a 60 percent increase in diagnostic demand by 2029. The increasing complexity of patient-tailored investigations will further exacerbate this burden (8), and the diagnostic backlog in the wake of the COVID-19 pandemic will have a compounding effect. Ironically, this rise in service demand is expected to be paralleled by a 30 percent fall in active pathologists relative to 2010 staffing levels (9). This dismal scenario is exemplified by figures from the Royal College of Pathologists’ workforce census that reveal that only three percent of departments are adequately staffed to meet diagnostic demand – a situation echoed in the US by analogous workforce shortages (10,11,12) that will only be worsened by an incipient retirement crisis and a shortfall in trainee recruitment (10). The impact of the status quo is not insignificant on National Health Service coffers, addressing the shortfall by using locum doctors and outsourcing services costs an estimated £27 million per year in the UK alone.

AI steps in

AI in digital pathology offers a range of potential diagnostic solutions with clear merits – yet it has received a cold shoulder in some quarters, something of a disappointment for the “third revolution in pathology” (13). Despite early teething problems, many AI-based solutions have shown potential clinical utility, albeit in an academic setting. One key benefit is that AI could shorten pathologists’ reporting time; algorithms capable of tumor detection, cell counting, and mitosis detection are now increasingly available (8). Others highlight areas of interest for review within WSIs (1), reducing the time needed for a pathologist to scan a case at low power. This is particularly useful in biopsies and resection specimens that are known to be time consuming to report (for example, nodal [micro]metastases) or where multifocality is important (for example, breast and prostate specimens) (1, 14). 

Busy clinical pathology departments, such as those dealing with high-volume primary care skin excisions, could also make use of simple algorithms (for example, cancer present or absent) to screen and prioritize malignant cases for review (7). They could also standardize diagnostic performance to offset inter- and intra-pathologist diagnostic discordance (15) – in essence, providing a second opinion. The possibility of bypassing ancillary testing and its allied delays is also welcome, highlighted by algorithms that resolve immunohistochemically stained HER2-equivocal cases without the need for fluorescence in situ hybridization (16). Notionally, the aim is to adopt such solutions clinically to accelerate case turnaround, minimize ancillary testing analytical time, improve diagnostic accuracy, and reduce costs. However, the disappointing reality is that none of these technologies have transitioned into routine reporting practice; their use remains largely confined to the academic setting.

Offering prognostically meaningful information that could guide patient management would be a further boon. And within the research environment, we now have tools at our disposal that increasingly attempt to link tumor morphology to the underlying biology and subsequent clinical outcome. However, this is much more than a simple exercise in satisfying academic curiosity; algorithms have been developed to appreciate the spatial distribution of tumor-infiltrating lymphocytes (9) and to capture tumor-associated stromal features (17) or microvascular proliferation (18) – all of which have demonstrated prognostic significance. Indeed, we feel the greatest promise can be drawn from these tantalizing early results suggesting that AI may be able to identify prognostic features that elude visual inspection.

Reaching far and wide

Other deep learning-based (and thus inherently less defined) methods in the research environment have delivered prognostic information that outperforms existing molecular and morphological prognostic markers in colorectal cancer (19) and refines prognosis in hepatocellular carcinoma (20). Similar approaches have extended this reach across multiple cancer types within and across pathologic stages (21). However, many have also attempted to improve the robustness of their algorithms using circuitous, often non-validated strategies, such as combining histology data with ancillary test results and clinico-demographic metadata. However, emphasis is increasingly on developing clinically ready solutions that can achieve regulatory body compliance. One promising platform uses a combination of statistical physics, computer vision, and tumor biology to harmonize diagnosis (including grading and molecular subtyping) and prognosis in breast cancer (22,23,24). Perhaps most importantly, it overcomes many of the thorny issues surrounding dependency on large training data sets, working with pre-annotated WSIs, generalizability across histological backgrounds, and scanning platform agnosia. Accordingly, it offers a clinically more attractive and accessible “white box” solution for both pathologists and oncologists.

The implications of consolidating the relationship between pathology and oncology are manifold. Getting a better handle on prognosis and molecular profiles offers a unique opportunity to reduce diagnostic costs and turnaround times, as well as inform the selection of optimal therapeutic modalities. Importantly, prognostic AI algorithms would give pathologists the opportunity to lead more strongly and holistically on both specimen evaluation and therapeutic management based on predicted outcomes. A single operator controlling all these facets of specimen reporting should also mitigate diagnostic or treatment delays and improve patient management across the cancer care pathway. This tailored approach could also minimize the risk and sequelae of under- or over-treating patients, predict chemotherapy and immunotherapy response, improve clinical outcomes, offer personalized prognostication, and contribute to pathology’s role in making precision oncology a reality.

Although it may be difficult to appreciate the scale of the benefits ensuing from explainable, automated diagnostic solutions, they would undoubtedly have a global reach. Because most of these technologies can be readily deployed in cloud-based environments, enhanced pathology services could be accessed from developing economies or those where diagnostic histopathology services may be inaccessible or unaffordable. In these environments, the threshold for acceptance may be lower. Indeed, although we have a legal, regulatory, and ethical duty to provide robust, reliable, and fully clinically validated diagnostic and prognostic solutions, the use of AI in digital pathology does not have to be perfect to be useful. Consider that Rwanda has six pathologists for its 12 million inhabitants, whereas Burundi has only two to serve a similar population – a stark contrast to the United States’ 60+ pathologists per million people. Once a scarce commodity, digital pathology is now reaching remote corners of the globe. Digital histopathology “packages” – from tissue processors to slide scanners – have even been deployed in remote sub-Saharan areas where they support a three-day diagnostic turnaround service (25). As such, it is pathologists and not digital pathology services that are lacking. If such countries had access to AI, the potential benefits would be immeasurable – from timely diagnosis to management and prognostication.

Staying on the bench

Frustratingly, many of the promising new AI advances discussed above have yet to translate into clinically meaningful solutions, largely due to real-world generalizability and large-scale validation problems that continue to dog widely used deep learning-based approaches (8). A noteworthy stumbling block is that any AI algorithmic solution that is not fixed (that is, prone to further learning-related instability and vulnerable to diagnostic drift) is unlikely to achieve regulatory clearance.

Despite independent, robust review, some of the residual unease among pathologists in embracing AI may lie with the naïve clinico-legal framework of its implementation. Given the adjunctive nature of AI-based technologies, diagnostic responsibility will still lie with the reporting clinician. One of the principal concerns surrounds areas of pathologist diagnostic error in the face of a discordant opinion relative to the supporting AI. One path out of the woods could be to limit early-stage AI technologies to the diagnostic (and not prognostic) interpretation of “lower risk” cases, such as basal cell carcinomas, in which diagnostic inaccuracy would be unlikely to cause significant harm, or to simple dichotomous diagnoses, such as the identification of metastases in lymph nodes. These reports could further include disclaimers acknowledging the potential limitations of partly AI-reported cases. Finally, the answer could also come from AI-based solutions themselves – a measure of diagnostic discordance with or without AI support would inform the associated level of diagnostic bias, confidence, and safety, thus gaining clinical credibility.

It remains difficult to escape the mysticism of the “black box” phenomenon; few algorithms to date offer an intelligible rationale to underpin their inner workings, which can impact their clinical acceptability. However, many in the industry argue that explainability may be a moot point if rigorous validation and failsafes are in place and patient benefit is apparent. Interestingly, this same level of scrutiny is not applied to individual histopathologists. As we progress through training, gain experience, and hone our skills, diagnoses rely progressively less on the detailed scouring of slides or WSIs. Instead, we begin to undertake overall assessments that conform to a mental diagnostic blueprint and enable us to home in on important key features – a process notoriously challenging to impart to others. This doubtlessly contributes to the verbosity of free text reports that can leave other specialty colleagues bemused by our elaborate morphological descriptions, some of which may seemingly have no bearing on the bottom-line diagnosis, but which may carry an as-yet ill-defined diagnostic or prognostic relevance for AI to identify.

Looking to the future

AI is stirring a sea-change that we can no longer ignore – so how can we reach the nonbelievers? A key issue is the idea of being replaced. Why would anyone endorse a technology that might one day put them out of a job? To relieve these concerns, we must not lose perspective. Though AI and digital pathology can transform our specialty, machines cannot simply replace a histopathologist’s multifaceted role – from intelligent specimen cutting to understanding a disease’s pathophysiology and clinico-radiological correlations. But that doesn’t mean that practicing pathologists cannot benefit from AI, particularly in environments where subspecialty expertise may be lacking. This fact is echoed by ongoing discussions we have had with patient groups internationally. Although patients acknowledge the value of AI, particularly in circumstances where it outperforms histopathologists, they still believe that it should be in the hands of a clinician – a striking endorsement for a profession that is conspicuous by its lack of patient interaction. Thus, with the comforting knowledge that AI can be used as an adjunct to enhance diagnostic performance, increase efficiency, reduce cost, and consolidate pathology-oncology cohesion, we may be able to change more minds. After all, as we embarked on our clinical careers, we committed to a path of lifelong learning, acutely aware of the shelf life of “current” medical knowledge. This is widely accepted across all branches of medicine and has underpinned the acceptance of new surgical approaches and therapeutic strategies – so pathology should be no stranger to this notion.

The path to bring AI from our imagination to the laboratory has clear milestones. AI diagnostic solutions could be affordably deployed even in partly digitized environments (20,21,22), where cost savings related to reduced dependency on ancillary tests could be diverted to budgets supporting the adoption of new technology. More specifically, a viable end-to-end diagnostic platform should be fully automated, improve diagnostic accuracy by providing a second opinion and a diagnostic confidence score, offer a range of safety features and, above all, be explainable in terms of the underlying tumor morphology and biology. Developing a computationally light device that integrates seamlessly into existing clinical workflows and healthcare systems’ IT infrastructure and produces pre-annotated, interactive synoptic reports pertaining to the original WSI could offer appropriate quality control while both reducing pathologists’ workload and increasing their throughput.

Although AI may appear to be a panacea for histopathology, pathologists’ current distrust is entirely natural. The addition of AI solutions to our diagnostic toolbox should be embraced and encouraged, but a significant burden of proof remains for real-life clinical utility. If, as a profession, we can actively engage with and maintain the open mindset with which we adopted immunohistochemistry and genomics during previous revolutions, we are well on our way to harnessing the potential of this next wave of disruptive technologies. We have a responsibility to not only embrace, but also drive, shape, and take ownership of AI to oversee its safe introduction into diagnostic practice. The impact of our flexibility and ambition will naturally cascade down the cancer care pathway to our oncology colleagues and, most importantly, to our patients. If AI’s inherent potential can overcome our initial skepticism and resistance to change, its transition and successful deployment in clinical histopathology will be with us before we know it.