Precisely What You’ve Been Looking For

According to a study from the Massachusetts Institute of Technology, 90 percent of information transmitted to the human brain is visual and we can identify images we see for as little as 13 milliseconds (1). No other profession understands this more intuitively or puts it into practice more than pathologists. Yet even as progressive hospitals adopt digital workflows in pathology, we are hobbling along with text-based search to find the data we need. What if we could search visually based on the content of the image? How much would we improve the quality and speed of diagnosis? How much could we accelerate discovery?

Imagine, for example, that a pathologist is reviewing a difficult case. What if they could instantly search and retrieve multiple results of similar-looking tissue along with the associated pathology reports from trusted colleagues? They could locate information from others at their hospital or hospital network, or even from experts around the world. How much could that help inform their diagnosis? And the benefits aren’t limited to the clinic. Researchers could use image search to discover previously unknown connections between cancer subtypes – or, if we think big, throughout the entire genome.

At the Pathology Visions conference in Orlando in October, Hamid Tizhoosh from the Kimia Lab at the University of Waterloo reported on a recent validation of image search, in which we indexed ~30,000 whole slides from 11,000 patients (2). The search encompassed 25 organs and 33 cancer subtypes from the NIH/NCI public dataset (3). From the project, we learned that it is possible to build diagnostic consensus with high confidence. The search engine itself uses a “majority vote” system by which it compares new, undiagnosed cases against all existing diagnosed cases in its dataset – a system with great success. In frozen sections and diagnostic slides, accuracy for certain cancer types approached 100 percent. We identified a positive correlation of 80 percent between the number of patients and the accuracy of majority consensus – that is, the more data the better.

Over the next five years, hospitals and labs will produce hundreds of millions of digital slides, exabytes of unstructured image data, and tens of millions of pathology reports. Image search will become the “must-have” functionality to bring intelligence to the huge quantity of unstructured image data, with the far-reaching potential to connect pathologists to the collective knowledge of their colleagues. Keep your eye on this technology – because, without it, we risk simply bypassing huge amounts of valuable information.