Use of mobile high-resolution device for remote frozen section evaluation of whole slide images

Abstract

Introduction:

With recent advances, it is now possible to view whole slide images (WSI) on mobile, high-resolution, viewing devices (MVD). This creates a new paradigm in which MVDs may be used for consultation and/or diagnosis. Validation of the results with devices is important for practitioners and regulators. We evaluated the use of MVDs in frozen section (FS) interpretation.

Methods:

A series of 72 consecutive FS cases were selected for potential inclusion in the study. A 67 case subset of these were successfully scanned at 20x magnification. Scan times were recorded. A sample of WSI FS cases, with gross and clinical information, was presented to six pathologists on an iPad MVD using the Interpath application. Times to diagnosis were recorded. Results were compared with the original reported and final diagnosis. Participants also completed a survey assessing image quality, interface, and diagnostic comfort level.

Results:

Scan times averaged two minutes and 46 seconds per slide, (standard deviation [SD] 2 minutes 46 seconds). Evaluation times averaged 4 minutes and 59 seconds per case, range to 13 minutes and 50 seconds, SD 3 minutes 48 seconds. Concordance between initial FS diagnosis and rendered through the MVD was 89%. Minor discrepancies made up 8% and major disagreements 3%. The kappa statistic for this series is 0.85. Participants rated the experience at 5 on a 10-point scale, range 3 to 7. Two-thirds found the image quality to be adequate, half were satisfied with image resolution, and 33% would be willing to make a diagnosis on the iPad, plus one only for special cases. Five of six respondents (83%) found the navigation with the study software difficult.

Conclusion:

Image fidelity and resolution makes the iPad potentially suitable for WSI evaluation of FS. Acceptable accuracy is attainable for FS interpretation. But, although possible to obtain acceptable results, use of the iPad with Interpath to view WSI is not easy and meets user resistance. The obstacle of slide navigation at high magnification could introduce frustrations, delays, or errors.

INTRODUCTION

The transformation in work patterns from a fixed site with close proximities of work and workers to a distributed, dispersed array with wide geographic range has happened at a variable rate in different industries. Within medicine, a traditionally intimate relationship between caregivers has been stretched through various telemedicine applications to open doors of access to specialists at a distance from the patient. The adoption of digital technology in radiology has been the poster child for this change, allowing access to well-trained and alert specialists for patients seen in a host of settings, and at virtually any hour of the day or night. In pathology, the challenges of data density, scan times, storage space, and transmission times have been slower to succumb to technical solutions, but are now becoming more routine. The adoption of faster wireless networks over broader geographic areas and the introduction of mobile, high-resolution devices such as tablets and smart phones make the feasibility of interpretation of pathology images (slides) a legitimate question.

Interpretation of frozen sections (FS) performed in locations remote from the pathologist providing the interpretation has been shown to be possible for 20 years or more, and validated using a wide variety of technical solutions that have included robotic stage telepathology,[1,2] fixed image capture and forward, streaming video microscopy, and more recently, scanned whole slide images (WSI).[2–7] The choice of FS as the service of study grows out of the urgency of a diagnosis, which does not allow for even overnight transit of diagnostic materials. But the lessons learned from this application have been easily expanded to include other consultative activities in surgical pathology such as second opinions, quality assurance review, etc.

The introduction of high-resolution, mobile devices such as the iPad coupled with high-speed data networks opens the possibility of interpretation of medical images from locations not tied to a fixed workstation in a hospital or physician office. Recently, the Food and Drug Administration has approved such an application for use with radiologic images.[7] We sought to assess the capability of first-generation applications of these technologies for use in interpretation of pathology images, specifically whole slide scanned images of FS slides.

METHOD

Case files from our institution were searched from the fourth quarter of 2009 to identify all cases submitted for FS analysis and a consecutive set of 72 cases was identified for possible inclusion in the study. FS slides from 67 of these cases with approximately 210 FS slides were retrieved from the files and successfully scanned at 20× with an Aperio Scanscope. Scanning times were recorded, as well as scan failure rates. Clinical information available at the time of FS, and gross description or gross images if available, were included in the linked files on each case in the Spectrum software database housing the WSI. Nearly 100% of FS slides are cut and stained by trained full-time histotechnologists or pathology assistants at our institution. Cases were also categorized according to case type, tissue of origin, and nature of question sought by FS.

Six pathologists with varying levels of experience in pathology, and varying aptitudes toward technology participated in the study. See Table 1 for breakdown of participant details. Selected cases from the dataset were presented to each pathologist using an iPad and the Interpath (Institute for Medical Informatics, Oslo, version 1.3) application accessing the Spectrum secure database server. Wireless connectivity was through the campus wireless network. Evaluation times for each case were captured by an observer. Each pathologist interpreted between 4 and 9 FS cases, for a total of 58 WSI. Their answers were recorded and compared with both the original FS diagnosis and the final diagnosis. Discrepant cases were resolved for “truth” by a consensus review of both frozen and final diagnostic materials.

Table 1.

Breakdown of participants’ age and experience

Statistical analysis of results was performed with the free-marginal multirater kappa.[8]

Each pathologist observer was also asked to complete a survey regarding their experience interpreting the cases using the iPad using a scale of 1-10, with 10 being superb and 1 being poor. The users gave the iPad with the Interpath software a rating of 5/10 with a standard deviation of 1.4, with values ranging from 3 to 7.

RESULTS

Scan times for our selection of FS slides, generally with just one-level section per slide, averaged 2 minutes 46 seconds per slide (+/- 2:46 SD). The largest slide scan required 8 minutes 51 seconds. In our population of cases, the average case included three FS slides, with a range up to 14/case. No cytologic preparations were included in our study set, though we routinely perform these for neurologic cases and in other settings. The initial scan failure rate was less than 2%. After image adjustments, all slides were successfully scanned.

Case examination times using the iPad averaged 4 minutes 59 seconds/case, or an average of 1 minute 20 seconds per slide. Specific time details ranked by case are reported in Table 2.

Table 2.

Time per case with standard deviation

Diagnostic accuracy using the iPad for interpretation of FS slides was 89%. Minor discrepancies in diagnosis, defined as those not significantly impacting patient care or management intraoperatively, were identified for 8% of the cases; see Table 3 for details of discrepancies. Only 3% of cases had clinically significant discrepancies in diagnosis between the mobile, high-resolution, viewing devices (MVD) and traditional glass slide interpretation. The overall kappa statistic for this study set was 0.85.

Table 3.

Details of discrepant frozen section impressions from iPad diagnosis and original FS diagnosis

Study participants responded generally favorably to the overall experience of digital slide interpretation using the iPad, but most felt reluctance under the study parameters to place this into regular routine service. Two participants found the resolution to be inadequate for making an interpretation, three found it adequate, and one thought that it was adequate for margins, but not for making a specific diagnosis. Three participants would be willing to make a diagnosis with the platform, one only for select cases, and the remaining three would not due to the resolution of the iPad and problems with the software being able to reliably focus on areas of interest. One pathologist did not find any problems navigating the software of the iPad, the other users had problems with focusing on regions of interest, being able to scroll and look at margins at higher powers, loading times being too long and frustrating, and lack of ability to quantify how much the image was magnified. The most common problem noted was the slide automatically re-centering when scrolling took place at high magnification.

DISCUSSION

Our results demonstrate that successful interpretation of WSI of FS slides is feasible with an MVD such as the iPad. Image quality did not appear to be an issue for the majority (4 of 6) of our study participants. These results are perhaps the more remarkable given that neither the study participants were regular iPad users for other applications, nor were they routinely involved in interpretation of WSI for diagnostic purposes in other capacities. Our concordance statistics are comparable with other studies of WSI performance in the FS arena[1–6] in addition to our own internal studies. Diagnostic accuracy is also comparable with literature ranges for FS interpretation using routine measures[9] and also compare favorably with other studies using WSI for FS interpretation.[2,6] In some ways, these results attest to the relative robustness of the diagnostic process itself, since our participants were neither experienced users of the technologies under consideration, nor did they receive any significant advance training or practice in the device or software.

The times required for scanning and interpretation in our study are of note because they point out one challenge with routine use of this model, for FS interpretation could be failure to meet acceptable FS turn around expectations of surgeons and accrediting bodies.[10] Although average times might fall within the arbitrarily designated 20 minute standard used by many laboratories for reporting of FS results on uncomplicated cases (estimated 10 minutes for grossing, freezing, cutting and staining, plus average scan [5.5 minutes per tissue block] and evaluation times), the standard deviation for microscopic examination time is sufficiently large as to expect that a number of cases would regularly exceed these limits. As illustrated in Table 2, some of the single block cases had image evaluation times exceeding five minutes. Familiarity with the technology and further practice would probably reduce these times somewhat however. It is noteworthy, however, that one case, a challenging interpretation of a head and neck lesion, is responsible for two of the major discrepancies in our small study [Table 3]. Recognition of such circumstances may further foster adoption of WSI use for FS interpretation, either by MVD or other means, since such images could be used for rapid real-time, remote consultation with more experienced colleagues, or those with sub-specialty expertise. Additionally, our study did not address any potential additional delays that might be encountered with remote supervision of gross evaluation, or corresponding delay in navigating a reporting process from a non-routine site.

Judging from the participant survey results, existing software for evaluation of WSI on the iPad is a part of the obstacle to smoother use of the high-resolution potential of this device. Several users reported difficulty in navigating the slide, since the traditional intuitive gestures and manipulative tools common in other iPad settings did not function smoothly in the Interpath application. Alternative viewing applications or web-browser-based viewers are beginning to appear which may address this issue.[11] Although our evidence indicates that the system currently available to us poses some problems in routine application, these do not appear to be insurmountable problems. Scanning times continue to diminish, and are now faster than the older scanner used in our study. Additionally, new software for display and manipulation on mobile devices is being developed, along with several competing devices using alternative operating systems that may offer more ease in manipulating images such as WSI. We believe that the other issues inherent with this transition to mobile-capable interpretation, such as more continuous availability of key consultants, will make use of WSI in the mobile environment routine for FS and other uses in the near future.