Pathology Imaging Informatics for Clinical Practice and Investigative and Translational Research

Abstract

Pathologists routinely interpret gross and microscopic specimens to render diagnoses and to engage in a broad spectrum of investigative research. Multiple studies have demonstrated that imaging technologies have progressed to a level at which properly digitized specimens provide sufficient quality comparable to the traditional glass slides examinations. Continued advancements in this area will have a profound impact on the manner in which pathology is conducted from this point on. Several leading institutions have already undertaken ambitious projects directed toward digitally imaging, archiving, and sharing pathology specimens. As a result of these advances, the use of informatics in diagnostic and investigative pathology applications is expanding rapidly. In addition, the advent of novel technologies such as multispectral imaging makes it possible to visualize and analyze imaged specimens using multiple wavelengths simultaneously. As these powerful technologies become increasingly accepted and adopted, the opportunities for gaining new insight into the underlying mechanisms of diseases as well as the potential for discriminating among subtypes of pathologies are growing accordingly.

INTRODUCTION

Informatics is increasingly becoming recognized as an essential supporting component for routine activities throughout modern pathology laboratories. Evidence of this change is already prominent in many laboratory information systems where software is utilized routinely for specimen tracking, processing and reporting for both clinical and anatomical pathology operations. Soon, imaging informatics along with a wide range of associated technologies will become intricately interwoven and integrated with the pathology workflow (Figure 1). Evidence of these changes can already be found in several leading institutions, as the arsenal of tools that facilitate and assist pathologists in diagnosis and investigative research continues to gain acceptance.

Figure 1.

Anatomical pathology workflow incorporating imaging informatics into clinical practice, investigative research and education.

Recent advances and the steady cost reduction of high-performance computational power, faster network connections, and wider availability of mobile devices have opened the door for a wide array of additional opportunities and possible applications. Concurrent with these technological advances, data storage is becoming less expensive and more readily available for use in a range of emerging imaging applications. In the field of radiology, physicians have a long history of standing shoulder to shoulder with their computer science and engineering counterparts. This relationship has given rise to a broad spectrum of new imaging and computational technologies, culminating in the approval by the Food and Drug Administration for the use of software to assist in quality control and clinical decisions, as well as the use of mobile devices to facilitate remote consultations. With the advent of the next generation of imaging devices, it is likely that the field of pathology will follow a parallel path of adoption and implementation.

DIGITIZING PATHOLOGY SPECIMENS

Digital pathology images differ vastly from radiology images with regard to their spatial resolution, color depth, and the means by which they are stored, shared and transmitted. In radiology, much of the diagnostic material is created directly in digital format. In pathology, however, specimens are received as biological tissues, which are then processed into cassettes, embedded into paraffin blocks, and then subsequently sectioned into slides. In contrast to radiology, the act of digitizing pathology specimens does not eliminate the need to carry out tissue processing steps. Unfortunately, this requirement results in a more complex and tortuous workflow. For example, with regard to record-keeping, after having generated digitized pathology images, there still remains the need to store the actual glass slides. Furthermore, in order to maintain a complete data set, any slides prepared with the immunohistochemistry or any specialized preparation must be scanned separately and added to the corresponding case. All of these factors require proponents of these new technologies to present a compelling case for adoption. In the next few sections of this manuscript, we present several “value-added” advantages of introducing these relatively new approaches into the workflow.

There are several important considerations that must be taken into account when digitizing glass slides. The most important decision is the careful selection of the magnification at which to scan the slide, as it would be impractical to scan each slide using multiple magnifications. Doing so would quickly increase the size of data for each slide and case significantly. Since a scanned image can be digitally interpolated to approximate changes in magnification, it is only necessary to choose a single magnification with which to scan the slide. For optimal fidelity, however, this choice is highly related to the specific application at hand, since scanning a slide using low power and then later magnifying it digitally may create pixelated image or an image that appears to be out of focus.¹

Similarly, even slight color variations may also lead to suboptimal image quality. One group of investigators recently studied the effect of changing the contrast, red-green-blue color balance, and brightness of cytology images.² The investigators found that by changing these components, some diagnoses were affected significantly. It is clear that color calibration and monitors with the capacity to accurately render and display colors are required. One study has shown the benefit of using optical filters to improve resolution and color balance in viewing these images.³ The use of automatic histology staining systems to produce high definition slides from prepackaged stains that do not need human intervention has also been shown to result in slides exhibiting more consistent color, improved contrast and spatial detail, which in turn, leads to better images when scanned digitally.⁴

Unlike surgical pathology specimens, the bulk of the material in cytopathology exists only on the slide itself. In some cases, there may be additional material contained within the cell block, which can be utilized to delineate the differentiation and origin of the cells. However, there is no other tissue block from which a cytopathology slide can be reproduced. For these reasons, one must take special care when handling these specimens because there is no fail-safe solution if the slide is lost or compromised. Another challenge in analyzing cytopathology specimens is the importance of preserving the three-dimensional aspects of the constituent cells when digitizing them in order to maximize the utility of the sample. Studies that have used only static images at one plane of focus in cytology have shown major discrepancies between the diagnosis produced by reading a scanned image compared to using a traditional microscope.⁵ Similarly, reading these specimens at only a limited number focal planes is not as informative as doing so at a plurality of focal planes.⁶

While there are already some commercial scanners that feature the capability of scanning specimens at multiple planes, such strategies quickly lead to the generation of very large image files, which can cause significant data management challenges even during the course of fairly limited studies. Several investigators have attempted to resolve this issue by comparing different methods, such as multilayer stacking and extended focus. Both approaches present their own advantages, merits and limitations.⁷ One team of investigators experimented with the option of utilizing video capture to view a single field, in an attempt to mimic the manner in which an experienced pathologist would view the slide under the microscope. The primary advantage of this strategy is that the file produced for this single field is much smaller than that produced by taking multiplane images.⁸ The limitation of this approach, however, is that it is only feasible when looking at a single field under high magnification. In addition, it is completely impractical to utilize this methodology for viewing the slide in its entirety. Finally, this approach introduces a sampling error when the field of interest is selected by any individual other than the reading pathologist.

WHOLE SLIDE IMAGES

Despite the cited limitations and challenges associated with the driving technologies, there are many advantages that result from the effort of digitizing pathology specimens. Whole slide images (WSI) enable pathologists to reliably archive, systematically interrogate, transmit, visualize, enhance, and analyze a given specimen in its entirety without requiring access to the physical glass slide. Digitized specimens do not undergo degradation and are readily shared among colleagues. Retrieval of digitized slides is typically faster than physically searching for traditional glass slides. Furthermore, multiple, digitized slides can be viewed and compared with one another, side-by-side, e.g. a Hematoxylin & Eosin (H&E) stained slide and an immunohistochemically stained slide of similar regions. In addition, once a specimen has been digitized, it is possible to apply objective, reproducible quantitative methods in order to detect, track, and chronicle morphometric differences in the underlying pathology. The digitization procedure lends itself to computer-assisted analysis and interpretation.

Unfortunately, there currently remain two significant drawbacks to utilizing whole slide imaging strategies for routine pathology applications. Specifically, there are a number of direct (hardware and software) and indirect costs (staffing and storage) associated with digitizing specimens. Another challenge introduced by these technologies relates to difficulties experienced by end-users when navigating about a WSI dataset. This process requires a different set of skills from those required in viewing specimens using a traditional microscope, and the process is inherently much slower than reviewing glass slides. These facts tend to dissuade pathologists from using WSI in many scenarios, despite the comparable diagnostic accuracy for WSI compared to traditional approaches.^6,9 Such challenges are likely to be mitigated over time as the computational and imaging technologies utilized in WSI continue to advance and as the hardware and software components become increasingly available and affordable.

Evidence of some of the most recent advances in WSI technology relate to smart algorithms that have been integrated into many commercial systems to adjust for artifacts such as tissue folds and air bubbles. While it may not be a problem under the microscope itself, the presence of tissue folds can affect the quality of the WSI due to inadequate focusing. Recent software developments address this issue by specifically detecting folded areas and adjusting the focus of the entire slide accordingly.^10,11 Another advance that has been made to reduce overall scan time is the addition of a pilot scan to identify the location of tissue on the slide and to identify focus points for auto-focusing prior to executing the actual scan.¹⁰ These improvements to WSI technology facilitate the capture of high-resolution scans in a reliable and automatic fashion within a fraction of time needed just a few years ago.

In order to measure the performance of the emerging technologies used in WSI, several studies have been conducted to determine the capacity of WSI to support diagnostic applications. These studies have shown that the diagnoses given by pathologists via reading WSI were comparable to reading glass slides by standard microscope technique.^12–16 However, it is clear from these experiments that looking at images with poor color fidelity can result in failure to detect a small area of significant diagnostic importance within the specimen.¹² In addition, diagnoses that require high power observation such as in resolving inflammatory entities were shown to be more difficult when analyzed digitally due to the current limitations of the digital magnification of the device utilized to acquire the images as well as some of the challenges of navigation that were previously mentioned.¹³

One clear application area for WSI is in education, particularly for the instruction of histology and pathology for medical students and residents.^9,17 Digital teaching sets, unlike their glass slide counterparts, do not become degraded, broken or lost while being passed on from one person to another.18 These sets may also be reviewed anywhere and at any time instead of within the confines of a specific room in a pathology department. There are no limits to the number of individuals that can view the slides at once.¹⁹ WSI can be easily annotated, which is very useful in educational setting. Board examinations and proficiency testing have also largely switched to using digital pathology images. Furthermore, in residency programs, WSI sharing between institutions can benefit residents who have uneven exposures to different types of specimens. While it is impractical to transfer glass slides from one institution to another, WSI can easily be published on the web for access by the participants.²⁰

TELEPATHOLOGY

Telepathology is the practice of pathology among individuals located at two or more distant sites. It has been used to serve different purposes. Using telepathology to obtain second opinion from an expert generally improves turn-around time since there is no transportation involved. It also reduces costs and eliminates barriers, e.g. those encountered when transporting slides from Hawaii to the continental United States.²¹ Similarly, since there is a world-wide shortage of trained pathologists, pathology services from other countries that lack the expertise to render certain diagnoses may benefit from using telepathology to communicate with trained experts from other countries.²² This also provides a solution in delivering pathology services and consultations to underserved areas where a pathologist would not be readily available.²³

For more immediate use, telepathology has also been used to render diagnoses on frozen sections for after-hours emergency surgeries when a pathologist is not readily available. However, frozen sections, more so than permanent sections, may have areas of varying thickness on the slide, which will translate to areas of the telepathology image that are out of focus. Methods of tiling and focusing have been proposed to solve this issue, without excessively sacrificing the amount of time required to scan the slides. Generally, these images are obtained as several smaller images in grids or tiles and then stitched together to form one WSI.²⁴ The time it takes to scan, send and display the image remotely through telepathology may still be faster than the time it takes for the pathologist on-call to come into the hospital, thereby avoiding prolonged anesthesia time for the patient.

There are still many questions and concerns in regulations associated with incorporating telepathology in daily practice. For example, what are the legal ramifications when consultations are conducted across state or country borders? Will a pathologist that is licensed in one state be able to sign out specimens from a hospital in that state if he is physically in a different state? Does it make sense for a pathologist to have a national medical license?²⁵ What happens when the pathologist travels out of the country?

In neuropathology, where there often is a scarcity of available practicing specialists on-site to review intraoperative frozen sections, telepathology provides a viable solution. The frozen sections are sent using telepathology to neuropathologists that usually cover several different hospitals. In most hospitals where this technology is not available, those sections are typically read by the general surgical pathologists on site, who in many cases are not comfortable with reading neuropathology specimens. This practice does bring into question the matter of state licensing as well as sign-out privileges at the specific hospitals where the specimens originated. One study proposed that the neuropathologists who read the slides through telepathology serve only as consultants to the local pathologists who have the final jurisdiction over the patient care and are ultimately responsible for the diagnosis that is rendered.²⁶

For pathologists who prefer real-time image viewing, telepathology can be combined with a remote robotic microscopy, enabling the pathologist to control microscopic positioning of the glass slide from a remote location, therefore choosing his/her own plane of focus and areas to be viewed under high power. Several studies have found good diagnostic concordance with this method.⁵ However, there is often a significant delay involved in sending instruction to the microscope and receiving the image. Another downside to this practice is that the images that are reviewed are not typically stored.

Since there is a widely accepted use of mobile devices to view radiologic images, it is only fitting to explore this option for digital pathology imaging applications as well. A recent study utilizing a mobile device to view WSI demonstrated that the image quality is acceptable to allow pathologists to render diagnosis.²⁷ Provided the spatial resolution of the device is appropriate, the diagnostic accuracy using mobile devices is comparable with that rendered using routine methods. Nonetheless, transmission time, scanning failures, hardware malfunction, and network difficulties remain obstacles when utilizing these technologies under real-case scenarios.²⁸

DATA STORAGE

The capacity to digitize pathology images has opened a spectrum of new possibilities for pathologists to store large amounts of information which can be accessed by other colleagues and collaborators for consultation and research. Database models have been proposed to store this information along with any results, such as annotations and segmentations that may have been generated during the course of image analysis.²⁹ The information obtained from digitized pathology images can be integrated with information obtained from radiology to form radiologic-pathologic correlations, which may lead to better understanding of disease progression.³⁰ Several investigators have suggested making WSI versions of published images in the literature available for review by the community, so that other pathologists and scientists that are interested in the field can verify the information before conducting their own research.³¹

It has been a challenge to standardize a method for acquiring and storing these images. Digital Imaging and Communications in Medicine (DICOM) successfully standardized images in radiology, which can be accessed and archived by Picture Archiving and Communication Systems (PACS). Despite its success in radiology, DICOM has not yet been successfully extended to accommodate pathology whole slide images.³² In addition, most imaging systems are not integrated with the laboratory information system (LIS) standard. Consequently, patient’s data, orders, clinical information, images and annotations are spread out over different systems.³³ A standardized system would be highly valuable in enabling pathologists and researchers to sort through archives of images from different institutions while making it possible to create image microarrays that are conceptually similar to physical tissue microarrays.³⁴

COMPUTER ASSISTED INTERPRETATIONS AND DIAGNOSTICS

The task of quantification is something that is becoming increasingly important in pathology. It is no longer sufficient to simply determine whether a sample is positive or negative; instead it is also necessary to specify the degree or grade of disease severity. Unfortunately, this process is somewhat subjective, and the results may differ from one pathologist to another. Computer aided diagnosis (CAD) systems can be useful in this aspect. CAD is meant to assist, not replace, pathologists, as any interpretation by machines will require proper use and clinical interpretation by pathologists.^35,36 These systems are developed to perform measurements with great reproducibility and to discern subtle cues that human eyes may not be capable of observing.³⁶ A tremendous amount of work has been done in this field.³⁷

The first, critical step leading to CAD is accurate delineation of cellular and /or subcellular boundaries, a process known as segmentation. Inaccuracy of segmentation can lead to oversegmentations or undersegmentations, either of which will erroneously quantify the number of cells. Oversegmentations may happen when cells are larger or longer than expected, or when cells are multinucleated. In contrast, undersegmentations may occur when cells are clustered together with no clear visible cell membrane between them. These inaccuracies are sometimes brought about by the nature of the organization of tissues examined under the microscope, which include overlapping cells,^38,39 dense cytoplasm and high number of background cells.⁴⁰

Because of the inherent challenges that most pathology applications present, CAD systems are often developed separately for each individual organ and specific disease related to that organ, since criteria that apply for one entity in one organ do not necessarily extend well to other entities in the same organ and may not be appropriate for even the same entity in a different organ. Additionally, certain distinctive characteristics may be more relevant in diagnosing one disease but not in others. The majority of work done in disease-specific CAD systems relates to the two most prevalent cancer types in the United States: prostate and breast.⁴¹ For prostate cancer, most CAD development has focused on optimizing segmentation and detection of adenocarcinoma^42–45 to assist pathologists in reviewing a large number of biopsy slides that may only have small isolated foci of cancer. Detecting these foci is important in determining a plan of treatment for each individual patient. In contrast, for breast cancer, the focus is not so much on detecting small foci of cancer but rather in assessing and quantifying certain global characteristics of the cancer,^46–48 to suggest the appropriate type of chemotherapy and also to assess the prognosis.

MULTISPECTRAL IMAGING

One of the latest available technologies in pathology imaging is multispectral imaging. A multispectral image dataset called a spectral cube consists of a number of images representing brightness at each pixel as a function of wavelength. A grayscale image is then taken at each desired wavelength and the resulting images are stacked together to form the spectral cube. A multispectral microscope acquires images at specific wavelengths, rather than the typical 3 data points (red, green and blue) found in an RGB image. This technique allows much more quantitative and sensitive measurements to detect subtle differences in spectral emissions. This leads to the ability to use multiple markers, each with individual biological targets, in a single slide. Each marker will have different emission wavelengths, which can be detected and separated by the multispectral camera. In turn, the staining characteristics of each cell can easily be analyzed, allowing the detection of molecular events on a cell-by-cell basis.⁴⁹

In bright-field images, some tissue properties that are very hard to distinguish by eyes on H&E and that usually require immunohistochemical staining could potentially be visualized without such special stains by using multispectral microscopy. For example, it has been shown that multispectral microscopy is able to detect fibrosis in tissue, which can then be digitally stained with results comparable if not superior to those obtained using Trichrome stained slides.⁵⁰ Remarkably, multispectral imaging has also been shown to correctly distinguish between cytology cells with high morphological similarity, as shown by one group that analyzed parathyroid adenoma versus thyroid follicular adenoma.⁵¹ In practice, differentiation of these two entities typically requires immunohistochemical staining of the cell block.

When there is very little tissue available and multiple immunohistochemical stains are necessary, multispectral imaging techniques have the potential to be very beneficial for preserving tissues. Since the technique allows for multiple staining on a single slide, there is little concern about not having enough tissues to make several slides. Even if the stains overlap each other, multispectral imaging allows the separation of multiple stains.⁵²

Naturally, multispectral imaging is valuable when used with fluorescence slides. There are many commercially available fluorophores in different wavelengths that can each be separated and visualized by the multispectral camera (Figure 2). The major advantage of using multispectral imaging for fluorescence slides lies in the ability to remove autofluorescence and background noise. This will also improve accuracy in subsequent image analysis.^50,53 Multispectral fluorescence imaging allows for accurate quantitative evaluation of certain markers that have clinical meaning, such as the estrogen receptor (ER), progesterone receptor (PR) and the proliferation marker Ki67. In addition to quantitative evaluation, this type of imaging can be explored to reveal the type, distribution, intrinsic characteristics and biomarker state of each cell.⁵⁴

Figure 2.

Combined CD20 immunofluorescence (membrane staining) and (11;14) FISH of a germinal center in a frozen tonsil. The wavelength of each component detected by the multispectral camera is shown as follows: CD20 (top inset) - emission: 680nm, green FISH (middle inset) - emission: 540nm, and red FISH (bottom inset) - emission: 610nm. The emission for CD20 is near infrared and is not readily visible to human eyes but is easily detected by the camera. To provide the best color contrast, the CD20 shown on the image has been pseudo-colored cyan. (The figure is available at http://pleiad.umdnj.edu/CBII/images/NAJMS_image_2.png in higher resolution.)

Another convenience of multispectral imaging is to easily view dual-stained slides when looking for specific information that is harder to distinguish by light microscopy. One example is to visualize melanoma cells within the lymphatics among cells with similar brown pigmentation.⁵⁵ Cells with inherent brown pigmentation are typically hard to distinguish from the brown pigment of the applied immunohistochemical stain. Multispectral cameras can detect and separate metamers, which are two different spectral compositions of an image that are perceived by the human visual system as representing the same color.⁵⁶

CONCLUSION

In summary, significant progress has been made in the field of imaging informatics as it pertains to clinical practice and investigative research in anatomical pathology, starting with the capacity of digitizing pathology specimens for whole-slide imaging and telepathology applications and extending to computer-assisted interpretations and diagnostics. Several recent advances and technologies such as multispectral imaging make it possible to detect and track subtle changes in measurable parameters, which may lead to the discovery of novel diagnostic clues that are not apparent by human visual inspection alone. While there are still significant limitations and concerns associated with the integration of imaging technologies into the current clinical practice workflow, the continued evolution of computer power and increased level of acceptance among pathologists makes it likely that these approaches will become part of routine clinical methods and procedures in the very near future. In the meantime, the field of imaging informatics has already demonstrated many valuable tools in wide use today throughout the research and educational communities.