Interrogating the TME with AI

The tumor microenvironment (TME) is the network of cells and extracellular matrix structures that surrounds, infiltrates, and interacts with tumor cells. The TME can include stromal components, such as fibroblasts, endothelial cells, and various immune cell subtypes, as well complex structures, such as fibrosis and/or lymphoid aggregates of varying degrees of maturity among many other elements (see Figure 1; 1). The TME plays a crucial role in promoting tumor growth, invasion, and metastasis by exerting its effects on the dysregulation of both the tumor-induced immune response as well as various cell growth and differentiation factors.

The TME is also important in the tumor’s response and resistance to oncology therapeutics. And that’s why gaining a deep understanding of the complex biological processes within the TME is essential to the development of effective oncology drugs.

Modern oncologic drug development requires effective measurement methods to interrogate the TME. Such methods should provide a high degree of resolution of biological entities at the level of an individual patient and scale across large patient cohorts – within randomized control settings as well as real world conditions. The measurements should also be standardized with quantitative and reproducible outputs. Importantly, the native state of the tumor (representing the spatial dimension of cells and tissue) should be preserved such that these spatial relationships can be interrogated and related directly to the natural pathophysiology that defines the disease.

For example, a multinational group of researchers led by S Loi demonstrated the prognostic role of stromally-located tumor-infiltrating lymphocytes (TILs) in predicting patient survival following adjuvant chemotherapy (see Figure 2; 2). This and similar examples point to the opportunity for clinical integration of TILs as a biomarker for patients with early-stage cancers.

Current methods and their limitations
 

For over 100 years, the clinical diagnosis of solid tumors has been defined using hematoxylin and eosin (H&E) stained tissue; however, because of the subjective and labor-intensive nature of human interpretations, H&E based analysis is not scalable, quantitative, or reproducible enough to glean an in-depth understanding and novel insights about the TME. For example, though a pathologist can differentiate a lymphocyte from a tumor cell, they cannot accurately count the amount of lymphocytes or tumor cells, which could be well beyond hundreds of thousands of cells. Equally, they cannot reproducibly identify subtle differences in the maturity gradient of lymphoid aggregates or measure the amount of macrophages within 30 µm of the epi-stromal interface, and so on. Any one of these tasks could be an important biomarker to understand disease pathophysiology and link to other important markers. In short, it has not been possible to fully study the TME using standard manual microscopy-based pathology methods. 

Other analytical methods such as proteomics, single cell genomics, and transcriptomics have recently been applied to understanding the TME with great success. However, these methods are costly and often require the disaggregation of the tumor, which prevents the spatial analysis inherent to fully understanding the TME. Methods that preserve spatial dimensions, such as multiplexed tissue-based immunofluorescence or nucleic acid-based spatial transcriptomics, are very powerful but also expensive, time consuming, and not easily scalable across large patient populations; thus, they cannot be applied to most real-world settings. These challenges mean that the generation of population level statistics is difficult, if not impossible. And because manual interpretation of H&E are neither quantitative nor reproducible to the same degree as contemporary “-omics” platforms, it’s extremely difficult to apply sophisticated analytics that can more clearly and routinely define and diagnose diseases. 

Unlocking the power of AI
 

The combination of deep learning and computer vision has ushered in a new era of AI-based analysis methods for medical imaging including pathology images. This is possible by imaging an H&E slide and having pathologists manually and exhaustively annotating cells and tissues. These annotations are used as a source of “truth” to train neural networks, which can subsequently be deployed to identify the same aspects of the TME on previously unseen slides. Thus, deep learning models can identify each and every cell and their exact locations on a single H&E slide (see Figure 3).

Once the cells and tissue features are accurately predicted, they can be used as a foundation to expand further into entirely novel measurements of pathological features otherwise impossible to calculate by humans. For example, the ratio of lymphocytes to fibroblasts within the area of 30 µm from the tumor boundary could be an important biological manifestation of immune dysregulation of which we were previously unaware  (4). We can refer to this entirely new set of AI-based pathological features as human interpretable features (HIFs) because they are rooted in the foundation of human annotations and thus provide a degree of biological “explainability” (see Figure 4). 

A single slide could then output hundreds or thousands of HIFs in a scalable, reproducible and standardized manner, lending themselves to analysis for both “within patient” and across large clinical trials or real-world cohorts. Importantly, HIFs can be directly related to genomic or transcriptomic data, providing a link between molecular underpinnings and pathological manifestations of disease (see Figure 5). For example, aneuploidy, which can be measured by molecular methods, is universally linked to cancer progression. A collaboration between researchers at the Cleveland Clinic and Path AI has shown that AI-derived nuclear HIFs (for example, variations in the nuclear minor axis length), which would be impossible to measure manually, are a manifestation of aneuploidy that can be further used as a prognostic factor(5).

A new tool for drug development – with potential for clinical use
 

HIF panels are a new and powerful tool that can augment existing analysis methods of the TME. HIFs can be used to predict the genomic profile of a tumor from H&E as a prescreening measure, allowing for confirmatory testing only for patients with a higher probability of genomic alterations. This approach has the potential to prevent unnecessary testing – making significant savings in the health care system – and more rapidly triage patients into appropriate treatments. In drug development, HIFs can be used to interrogate mechanisms of action or resistance and determine if changes pre- and post-experimental treatment are impacting the TME in a way that is consistent with a drug’s hypothesized mechanism of action. With such information, pharmaceutical researchers and manufacturers could make quicker, more informed development decisions; for example, halting a drug development program where proof of concept is not demonstrated in early clinical trials.

The generation of HIFs using modern AI-based methods is a new but important biological measurement that can bridge the gap in spatial analysis with scalability, reproductivity and quantification of features. HIFs provide an important degree of biological explainability that can be analyzed within individual patients, as well as across large clinical trials or real-world cohorts. I believe this powerful tool can augment existing analysis methods, offering great potential to unlock novel insights and expedite drug discovery and development.