SMILE, It’s Easy

At a Glance

• Report preparation is important, but consumes a significant amount of the pathologist’s time and effort
  
• Secretary-mimicking artificial intelligence (SMILE) can help prepare reports quickly and precisely
  
• SMILE has helped me to both improve efficiency and reduce errors
  
• Instead of focusing on voice recognition technology, we should be welcoming artificial intelligence into our daily practice

As a dermatopathologist and general pathologist in a busy practice, I used to devote roughly half of my time to preparing reports. But the situation has changed dramatically since Ms. Smile became my assistant two years ago. Incredibly capable, she types 1,000 words per minute, and has an eidetic memory – she forgets nothing. She can read case-associated information and preliminary reports in seconds. She is attentive to detail, frequently catching errors in section codes and specimen dimensions, as well as transcription errors in clinical histories. When I dictate diagnoses in multi-specimen cases, she knows which specimen the diagnosis belongs to without the need to tell her explicitly. Some specimens require either no slide viewing (i.e., gross examination only specimens) or only confirmatory slide viewing (e.g. when diagnoses are almost always the same, such as with vasectomy or tubal ligation), and Ms. Smile will type the diagnoses for these without being told specifically what to type. Most of the time, I agree, and she finalizes the case electronically.

With the help of Ms. Smile, my report preparation time has been reduced by around 80 percent. With her vigilance, errors have decreased, too. You’re probably thinking “no way! It’s not humanly possible to read and transcribe so quickly, and know at all times which slide you’re viewing”, and you’re right. Because Ms. SMILE (Secretary-Mimicking Artificial Intelligence) isn’t human…

Prelude to a SMILE

Why did I see the need to develop an artificial intelligence (AI) approach to reports? Traditionally, report preparation is assisted by secretaries transcribing what the pathologist dictates. More recently, voice recognition (VR) technology has started to gain popularity (1–4) and replace human transcriptionists. This tends to decrease turnaround time and reduce staff costs. But VR technology has drawbacks in my view: secretarial tasks are shifted to the pathologist, taking up our time, and imperfect VR systems can lead to a potential increase in errors.

Along with VR, barcode scanning technology has become more widely available. These systems track the specimens from arrival, to placement in processor cassettes, to embedding in paraffin blocks, and finally to microscope slide production. Scanning the barcode of a glass slide in a new case brings the case information up on the screen.

Prior to the creation of SMILE, I used Dragon voice recognition and barcode scanning, driven by voice activated Windows automation scripts written by a colleague of mine. It was a remarkable system, which rivals any commercially available VR technology in my opinion, and was better than traditional human transcription. But despite these improvements, I was frustrated with the fact that for many straightforward cases, I was spending more time preparing reports than I was rendering interpretations. Additionally, some reports were finalized with nonsensical errors caused by imperfect VR – it was against this backdrop that SMILE was created.

The architecture of intelligence

The underlying implementation of SMILE is a collection of over 20 thousand lines of code, and the associated data files. In order to be intelligent, SMILE has to have knowledge. This knowledge resides in both short- and long-term memory. Short-term memory contains the slide-, specimen-, and case-level information; SMILE obtains that information via non-auditory means by reading the information while the pathologist is reviewing the case. This information is short-lived, as it changes from slide to slide, specimen to specimen, and case to case. The core components of the long-term memory are integral parts of the computer program; the user-specific long-term memory resides in the data files. The voice commands are used in the context of this both timely (short-term) and timeless (long-term) knowledge, and in a logical fashion. This is the foundation of SMILE’s intelligence, and allows SMILE to type reports and check for errors, communicating with the user via text to speech and on-screen messages (Figure 1).

This use of both long- and short-term memory allows SMILE to modify and even refuse some voice commands given by the pathologist – for example, SMILE will not allow a case to be signed out using the command “release case” if all the slides have not been reviewed, or the gross description contains a section code designation error. The reason for refusal will then be conveyed by voice. User input via dialog boxes continues to increase the long-term memory of SMILE, allowing it to become both smarter, and more user-specific, as time goes on.

Pathologist-SMILE interaction

Using its inbuilt knowledge, SMILE can respond to both verbal and non-verbal (slide scanning) actions by the user (Figure 2).

It should also be noted that while non-verbal actions are generally not skippable, some verbal actions can be skipped (or, to be more accurate, combined into the underlying implementation) in simple cases, such as “begin dictation” and “release case”, which can further save time.

To better explain how SMILE functions, I will use the Microsoft Word rendition of a series of screenshots from a real case (Figure 3). After reviewing the entire case in this example, I scanned the barcode of the slide of specimen #1, and dictated “chronic calculous cholecystitis” (Figure 3A). SMILE opened up a Word document and typed the headers for both specimens, as well as the diagnosis for the first specimen (Figure 3B). I then scanned the slide from the second specimen. SMILE judiciously put a period at the end of the diagnosis for the first specimen and then moved the cursor down, ready for dictation of the second specimen (Figure 3C). Once I had dictated the second diagnosis, the case was ready to be signed out electronically using a voice command (Figure 3D).

While working on this case, SMILE assigned the procedures (laparoscopic cholecystectomy and excision) correctly to the specimens, and for #2, SMILE changed “R lower eyelid BBC” to “Skin, right lower eyelid” before assigning the procedure “excision” to the specimen. No navigational command such as “next field” was needed, and I could also have scanned specimen #2 first and dictated the diagnosis for that specimen first, without navigational commands being required. This means the only thing I had to focus on was the interpretation – there was no need to touch the keyboard or the mouse throughout the entire process.

Functioning at an even higher level

In addition to performing in a prototypical fashion as described above, SMILE can execute many higher level requests; “understanding” the intent of the user and performing linked and complex actions, and making report preparation even more effortless.

For example, if I have a skin shave biopsy containing superficial type basal cell carcinoma, I would say “release case superficial BCC”. SMILE would type “Basal cell carcinoma, superficial type”, with an appropriate header such as “skin, right cheek, shave biopsy”, and electronically sign out the case. These actions are completed in 11 seconds following my voice command. If the case contained slides from two blocks, and I had reviewed only one, SMILE would type up the case, but instead of finalizing it, would tell me that a slide had not been reviewed. If there are certain mistakes in the clinical information or gross description, these would be automatically corrected for me: such as changing “seborrheic hyperplasia” to “sebaceous hyperplasia”, or adding the missing unit (cm) to a gross measurement

SMILE can further simplify the report preparation task in cases with multiple specimens. Perhaps I have a five specimen GI biopsy case – the first three specimens contain tubular adenomas, while the last two contain hyperplastic polyps. I would be able to scan the barcode of the slide of the first specimen, then say “tubular adenoma times three”, and the diagnosis for the first three specimens will be typed within seconds. Scanning the slide of the fourth specimen and instructing SMILE “hyperplastic polyp times two” means the entire case is typed. Similarly, in a prostate biopsy case where every specimen is benign, SMILE only requires me to say “benign prostatic tissue times six” to transcribe the entire report.

These are just a few examples of the many ways in which SMILE can work with the user to make reporting simple, accurate and fast – a more complete description of the system has been published in the July issue of the Archives of Pathology and Laboratory Medicine (5).

Expanding AI

I believe that the introduction of SMILE to pathology practice is a quantum leap; changing from a VR-centered, pathologist-controlled process of report writing, to an AI-centered, collaborative process between human intelligence and artificial intelligence. This would allow us to focus on the tasks that require a pathologist, and reduce our time spent on the tasks that don’t.

I have spent two years developing SMILE to assist me in creating pathology reports, and have been thoroughly impressed with the results – I can’t imagine going back to working without it. The most important aspect of SMILE is the underlying principle it represents: it removes the mundane secretarial tasks, and lets pathologists be pathologists.

Many challenges lie ahead; the first is the current lack of awareness of both the power and the feasibility of AI for this particular niche. The second is that commercially available technology is yet to be developed – it is probably unrealistic to expect every practice to have passionate pathologist programmers ready to write their own SMILE, but both pathology information system vendors and standalone entities can potentially make this technology available. The final issue is that, with past experience of human transcription as a basis for comparison, many users are probably happy with VR systems (1, 2). But despite these considerations, I believe Ms. SMILE could soon be joining a pathology practice near you.