Multiscale nonlinear microscopy and widefield white light imaging enables rapid histological imaging of surgical specimen margins

Abstract

The ability to histologically assess surgical specimens in real-time is a long-standing challenge in cancer surgery, including applications such as breast conserving therapy (BCT). Up to 40% of women treated with BCT for breast cancer require a repeat surgery due to postoperative histological findings of close or positive surgical margins using conventional formalin fixed paraffin embedded histology. Imaging technologies such as nonlinear microscopy (NLM), combined with exogenous fluorophores can rapidly provide virtual H&E imaging of surgical specimens without requiring microtome sectioning, facilitating intraoperative assessment of margin status. However, the large volume of typical surgical excisions combined with the need for rapid assessment, make comprehensive cellular resolution margin assessment during surgery challenging. To address this limitation, we developed a multiscale, real-time microscope with variable magnification NLM and real-time, co-registered position display using a widefield white light imaging system. Margin assessment can be performed rapidly under operator guidance to image specific regions of interest located using widefield imaging. Using simulated surgical margins dissected from human breast excisions, we demonstrate that multi-centimeter margins can be comprehensively imaged at cellular resolution, enabling intraoperative margin assessment. These methods are consistent with pathology assessment performed using frozen section analysis (FSA), however NLM enables faster and more comprehensive assessment of surgical specimens because imaging can be performed without freezing and cryo-sectioning. Therefore, NLM methods have the potential to be applied to a wide range of intra-operative applications.

1. Introduction

Rapid histological assessment of surgical specimen margin status would have applications in many types of cancer surgery, including cancers of the breast, prostate, kidney, pulmonary tract, gastrointestinal tract, and biliary system. Breast cancer, which is responsible for almost 30% of all cancers in American women, is a classic example where intraoperative assessment would be beneficial for patient care and reduce healthcare costs [1]. For early stage breast cancer, breast-conserving therapy (BCT) and radiation is the standard of care because it achieves survival and local recurrence rates equivalent to those of a mastectomy while providing superior cosmetic outcomes, reduced morbidity, and improved post-operative quality of life [2–7]. To minimize morbidity and improve cosmetic outcomes, it is important to minimize the resection of normal tissue around the tumor margin [7]. However, local recurrence following BCT is strongly correlated with margin status as determined by post-operative histopathology, making complete resection of cancerous tissue essential [8,9]. Similar challenges occur in surgical scenarios such as prostate cancer where positive surgical margins on postoperative histopathology are associated with an increased risk of biochemical recurrence (increasing prostate specific antigen) as well as the need for postoperative adjuvant radiation therapy [10].

Standard post-operative formalin fixed paraffin embedded (FFPE) histology requires fixation, dehydration, embedding and microtome sectioning into thin layers. This process is time-and-labor-intensive which precludes its use for intraoperative evaluation. Owing to the limited sensitivity of gross examination, residual tumor can be present near the surgical margin in up to 40% of BCT procedures, typically necessitating a second surgery to eliminate residual tumor [11–13]. Frozen sections analysis (FSA) is an alternative to FFPE histology which allows intraoperative evaluation by freezing tissue to enable cryosectioning without fixation and paraffin embedding, but has limited acceptance due low throughput as well as poor sensitivity and specificity as compared to conventional histology of the breast [14–16]. Consequently, a majority of surgeons do not utilize any form of intraoperative histology to evaluate breast excisions and instead depend on post-operative assessment [17].

An alternative to FFPE and FSA histology is to image tissue that has not been physically sectioned using optical sectioning microscopy to evaluate two dimensional planes or three dimensional volumes within a larger tissue specimen. Because physical sectioning accounts for the vast majority of processing delay for both FFPE and FSA histology, optical sectioning can dramatically reduce processing times and enable intraoperative histological examination in scenarios where FFPE and FSA histology are too time consuming. Various methods have been proposed for imaging breast, prostate and other surgical margins without physical sectioning, including optical coherence tomography (OCT) [18–22], reflectance confocal microscopy (RCM) [23,24], confocal fluorescence microscopy (CFM) [25–29], structured illumination microscopy (SIM) [30], light sheet microscopy [31], microscopy with ultraviolet surface excitation (MUSE) [32,33], and nonlinear microscopy (NLM) [34–38]. Stimulated Raman scattering (SRS) has also been demonstrated for surgical histology [39], but typically has appreciably lower imaging speed or signal to noise ratio when performed without physical sectioning in reflectance mode. Similar to SRS, mid-IR spectroscopy can provide histological imaging based on intrinsic molecular contrast, but has limited ability to perform reflectance-mode imaging of live tissue, and therefore typically requires time-consuming dehydration and/or physical sectioning of tissue [40].

Diffuse optical techniques such as diffuse reflectance spectroscopy can quantitatively measure tissue optical properties associated with malignancy, but do not enable direct histological assessment of pathology [41]. Extremely rapid molecular identification of breast margin composition has been recently demonstrated using depth-resolved Raman scattering combined with rotational specimen scanning, although the sensitivity of this technique to different pathologies is still under investigation [42]. Techniques such as OCT and RCM can perform label-free, structural imaging at extremely high speed but accurate histological evaluation is challenging because there is low nuclear contrast from scattering [26]. These and other label free techniques are complementary to methods that perform histological imaging of the surgical specimen using exogenous labels and have the advantage that they can be used in vivo to directly assess the surgical cavity.

In contrast to optical modalities using scattering or intrinsic contrast to infer tissue properties, techniques such as CFM, SIM, light sheet, MUSE, and NLM can use exogenous labels to generate images that can be evaluated analogously to conventional histology. Techniques that produce histology-like images of surgical specimens involve fewer changes to clinical standards and can also be applied in multiple intraoperative applications, analogously to FSA. By reproducing classic histological features, these techniques enable direct histological evaluation of tissue, but without the delay that makes paraffin embedding or cryosectioning prohibitively time-consuming in many surgeries. Furthermore, several groups have demonstrated extraction of exogenous labels following fluorescent imaging, greatly reducing the possible risk of interference with postoperative immunohistochemistry or DNA assays, while retaining the high contrast enabled by molecularly specific labels [32,43]. Of these techniques, NLM is particularly attractive because of its very high imaging speed and ability to image below the tissue surface through blood and surgical debris. NLM has been evaluated in a blinded study of breast surgical specimens [38] using a fluorescent nuclear stain and second harmonic generation with virtual H&E rendering (VHE) to evaluate 179 surgical specimens from 50 women, obtaining 95.4% sensitivity and 93.3% specificity for NLM assessment of benign vs malignant pathologies compared with standard FFPE H&E histology. These results suggest that NLM can achieve sufficient performance to evaluate breast surgical pathology.

In spite of the large number of optical imaging modalities that have been demonstrated in fresh human surgical specimens, translation of advanced imaging techniques into BCT has been limited due to the extremely large size of typical surgical excisions and the limited time available for assessment of margin status [44]. In BCT, a volume of breast tissue is removed which may be more than 100 cm³. Following excision, the specimen is inked for orientation with a unique color per aspect, transected transverse to the tissue face into ‘bread loaf’ specimens, fixed and embedded, with typically a single paraffin section evaluated per bread loaf. For patients with ductal carcinoma in situ (DCIS), guidelines recommend histological processing of bread loaf specimens at 3-5 mm intervals [45]. For example, in a 3 x 2 x 5 cm lumpectomy specimen (30 cm³) this corresponds to evaluating approximately 10-15 bread loaf specimen with an average surface area of 3-6 cm². The total transected tissue area is therefore on the order of 50 cm² to more than 200 cm² depending on excision size and section thickness. The large area and limited imaging time during surgery pose a substantial barrier for clinical translation, and in the case of FSA, typically limits evaluation of the excised tissue to a small subset of what would normally be evaluated with post-operative FFPE histology [46]. Previous work developing high speed strip scanning confocal microscopy has demonstrated microscopic resolution and nuclear contrast with imaging rates as fast as 1.5-3 minutes per cm² [47,48]. However, even at this rapid imaging rate, it would still take several hours to comprehensively image the face of a large lumpectomy cut into bread loaf specimens, which would represent an unacceptable delay of surgery.

Examination times can be dramatically reduced by limiting imaging to areas of the specimen that are relevant for surgical management. This requires high frame rates and operator guidance but is analogous to the way that pathologists examine histology under light microscopy. For breast cancer evaluation, the Society of Surgical Oncology, American Society for Radiation Oncology and American Society of Clinical Oncology Consensus Guidelines for Breast Conserving Therapy (SSO-ASTRO-ASCO) recommend obtaining a negative inked margin during excision for invasive breast cancer [49], and a negative margin width of 2 mm from ink for DCIS [50]. Therefore, comprehensive intraoperative imaging of the entire bread loaf specimen surface is inefficient because most of the surface contains tissue far from an inked margin and would not affect surgical management. Furthermore, in standard histological evaluation, tissue is first evaluated at low magnification, and areas suspicious for pathology are imaged at high magnification. However, with mosaic imaging, the process of imaging and evaluation are performed in reverse order, and so the entire specimen must be imaged at high magnification, resulting in an inefficient use of imaging time.

To address these limitations, we have developed an NLM system for assessment of surgical specimens that incorporates real-time multiscale imaging using both white light imaging for gross evaluation and variable magnification (5x-40x) NLM imaging. In contrast to previous work using mosaic imaging, real-time operator control is used to reproduce the standard process of histological evaluation, where suspicious regions are located using low magnification and then evaluated at higher magnification. We demonstrate that real-time, multiscale imaging reduces the time required to image inked margins by more than an order of magnitude as compared to high-speed comprehensive mosaic imaging, facilitating intraoperative assessment of margin status without excessive delay of surgical procedures.

2. Methods

Human breast tissue not required for diagnostic purposes that would otherwise have been discarded was acquired during specimen grossing at Beth Israel Deaconess Medical Center (BIDMC). All tissue selection and imaging was performed under protocols approved by the Massachusetts Institute of Technology Committee on the Use of Humans as Experimental Subjects and BIDMC Committee on Clinical Investigations and Institutional Review Board. Tissue specimens were inked to simulate surgical margins and then grossed into ‘bread loaf’ specimens. Following grossing, specimens were fluorescently labeled using a protocol described previously [43] that consists of 2 minutes immersion in 40 μg/ml acridine orange (AO) combined with 40 μg/ml sulforhodamine 101 (SR101) dissolved in 50% ethanol/water solution followed by 20 seconds rinse in buffered saline.

Specimens were imaged at 1030 nm excitation wavelength using a custom, multiscale NLM system (described below) deployed at BIDMC in the Department of Pathology, typically within 3 hours of resection. Bread loaf specimens were mounted in a custom specimen tray on a translating stage and imaged using a line scan camera, generating a widefield white light image with a field of view equivalent to a 0.33x objective, enabling immediate identification of diagnostic features such as inked surgical margins. Next, low magnification 5x NLM was used to rapidly survey inked margins at centimeter-scale, excluding the vast majority of tissue which is typically normal adipose tissue or stroma. When regions of interest were identified at low magnification, a rapid objective translator was used to perform high magnification, cellular resolution imaging. This workflow closely resembles that used to read histology slides on a histology microscope, with the addition of the widefield image that allows navigation to regions of interest on specimens.

Following real-time imaging, mosaic images of each specimen were recorded for comparison to FFPE histology using either a 10x/0.45 NA objective with a pixel pitch of 490 nm or a 5x/0.25 NA objective with a pixel pitch of 830 nm. After mosaic imaging, 10% neutral buffered formalin was added to the specimen imaging tray (described below) and the specimen fixed for a minimum of 12 hours. Following fixation, specimens were submitted for standard histological processing. Because specimens were fixed before removal from the specimen tray, the orientation and shape of soft tissue was preserved until after fixation is completed, facilitating registration of FFPE histology and NLM imaging.

3. Multiscale imaging system design and construction

3.1 Multiscale nonlinear microscopy system

The multiscale NLM system was constructed on a wheeled, vibration isolated optical table using a 36”x30” optical breadboard (Fig. 1). The largest component, a Ti:Sapphire laser (Chameleon Ultra, Coherent, Inc.) occupies the rear (away from operator) half of the breadboard. The laser output power is controlled using a Glan-Thompson polarizer in a USB-controlled motorized rotation mount followed by an interlock shutter. A diffraction limited, low dispersion Galilean beam expander expands the beam 3.33x to fill the 4 mm scanner aperture. Beam scanning was performed using a modified commercial scanner (OPX1100; Thorlabs, Inc.) with a resonant mirror scanner (16 kHz bidirectional rate) and a galvanometer mirror scanner for the fast and slow imaging axes, respectively. The OPX1100 includes scanners, relay optics, and a fold mirror for an eye piece which was not used in this application. The scan assembly was mounted upside down from the intended orientation, enabling inverted operation. A dichroic mirror and FFV2001 filter cube and photomultiplier tube (PMT) assembly are used to relay the excitation light from the OPX1100 to the back aperture of the imaging objective while collecting emitted fluorescent light.

Fig. 1.

Multiscale NLM system. Diagram of the NLM beam path including laser power controller, beam expander, laser interlock shutter, galvanometer-resonant scanner, scan/tube lens, filter cube and PMTs (left). Photograph of the instrument deployed at BIDMC (right).

Assessment of centimeter-scale pathology specimens requires both low magnification imaging for identifying regions of interest and higher magnification imaging for detailed assessment of potential pathological findings. A motorized translation stage (X-LSM050B, Zaber, Inc.) and a custom objective tray (Fig. 2) were integrated into the imaging path to rapidly (< 1 second) switch between high magnification imaging using a CFI Plan Apo Lambda 10x 0.45 NA objective (Nikon Instruments, Inc.) with a maximum field size of 2 mm and low magnification using a Fluar 5x/0.25 NA objective (Carl Zeiss Microscopy, LLC) with a 3.4 mm field size. Higher magnifications (20x/40x) were implemented in software using the 0.45 NA objective with a reduced scan angle to more densely sample the field. Z-actuation of the imaging plane was performed using a motorized translator (ZFM2020, Thorlabs, Inc.) to vertically actuate the objective assembly. Excited fluorescence emission was collected through the active objective and relayed via a 680 nm longpass dichroic beam splitter into the FFV2001 filter cube in a non-descanned geometry. A second, 45 degree dichroic filter (T588lpxr, Chroma Technology, Inc.) was used to split light into AO (DNA) and SR101 (stroma) channels. Each channel was further filtered with an additional emission filter for acridine orange (ET540/40m, Chroma Technology) and sulforhodamine 101 (FF01-650/60, Semrock, Inc.). Filtered fluorescence was detected using H7422PA-40 PMTs (Hamamatsu Photonics, Inc.), amplified using TIA-60 transimpedance amplifiers (Thorlabs, Inc.) and digitized using an ATS9440 PCIe A/D (Alazar Technologies, Inc.).

Fig. 2.

Objective translator assembly shown integrated into the microscope (top left), separate from the microscope (bottom left), as assembled and installed (middle) and a ray diagram of the NLM system and objective translator (right).

3.2 Widefield white light imaging

Widefield white light imaging was performed using a line scan camera with specimen translation over a fixed imaging location (Fig. 3). Images were acquired using an UNiiQA + (e2v technologies) color line scan camera with 2048 RGB pixels per line and Asahi Macro-Takumar 50mm macro. The macro lens was configured to give a 62 mm wide field of view along the line scan axis at a 0.3 NA and a projected pixel size of 30 microns. In conventional bright-field microscopy, this is an equivalent field of view to a 0.33x magnification objective. The quadrature encoder output of the translation stage was used to generate a camera line trigger every 30 microns displacement of the specimen. Both the stage and camera are capable of high line scan imaging rates and the imaging time in this system was limited by illumination power to 1 millisecond per line, or 3 seconds for the entire specimen tray.

Fig. 3.

Diagram of the widefield imaging system. Line illumination is provided by an LED array projected onto the specimen tray. A line scan camera and fold mirror are used to image the illuminated line as the specimen tray is translated through the camera focus.

3.3 Interchangeable specimen tray

Interchangeable specimen trays (Fig. 4) were designed using a 3x4 inch, 400 μm glass plate that serves as both the mechanical support for the tissue specimens and the imaging window for the microscope objective. A 3D printed specimen divider insert, subdivided the glass window into six cells, each the size of a conventional histology cassette, enabling straightforward histological processing of specimens after imaging. The specimen tray had an aluminum lid that served both to exclude room light from the imaging system and to provide compression of the tissue specimens against the imaging surface. Open cell foam similar to conventional histology cassette foam inserts was used on top of the specimens so they are compressed by the lid. Additionally, the lid had a magnetic interlock sensor used to disable the laser via the interlock shutter when the lid is removed, preventing accidental exposure to laser light. Finally, the lid had a rubber gasket and filling port, to enable optionally adding fixative after imaging without having to remove specimens from the tray.

Fig. 4.

Specimen tray. Rendering of the interchangeable specimen tray (left) and photograph of the tray with 6 inked, bread loaf breast specimens (right). The tray incorporates a lid with fill port as well as inset dividers that can be used to separate multiple small specimens.

3.4 Software implementation

Both imaging systems were interfaced with a custom microscopy application written in C + + using the .NET 4.5 APIs (Microsoft, Inc.). Rendering was performed in OpenGL using the OpenTK .NET bindings to embed GPU-accelerated OpenGL surfaces into a conventional Windows 10 application (Fig. 5). The application integrated multiple functions, including rendering VHE NLM images, rendering widefield imaging data, initializing and controlling NLM scanner hardware, controlling laser power, switching objectives, recording imaging data for offline analysis or review and optionally performing mosaic mode imaging for comparison to conventional histology.

Fig. 5.

Software user interface and controls used to image freshly excised, inked bread loafed surgical specimens. The virtual H&E display shows a terminal ductal lobule unit (TDLU) imaged at 20x magnification, while the white light microscopy shows a widefield image with the current imaging location adjacent to a simulated inked margin. The user can rapidly (< 1 second) switch the magnification, while a joystick and digital focus knob are used for X/Y and Z control, respectively. An LED indicator shows the laser interlock status, enabling the operator to see when the specimen is loaded and the laser armed.

Rendering of VHE images was performed using an open source virtual transillumination microscopy (VTM) algorithm published previously [35]. The implementation of VTM used in this study performs physically realistic per-pixel absorption calculations through a simulated slide to produce transmission images from epifluorescence data. The algorithm rendered epifluorescence from AO and SR101 as virtual hematoxylin and eosin absorption, respectively. The algorithm was implemented in OpenGL and then rendered on a 4K resolution monitor calibrated to 100% of the sRGB colorspace. OpenGL rendering is essential at 4K resolution because real-time rendering without GPU acceleration was not possible at the data rate required. OpenGL was also used to process the widefield white light images using a flat field correction to compensate for illumination uniformity and a gamma correction for optimal contrast. Finally, a custom vertex shader was used to draw the current NLM stage position on top of the widefield image. On each screen update (16.67 ms period), the current microscope stage position is retrieved from the quadrature encoders, and the vertex shader updated with the current location. NLM field sizes are also polled and the NLM imaging location icon size adjusted for the magnification setting.

Figure 6 depicts the logical flow of signals during three distinct phases of system operation: (1) prior to NLM imaging when the widefield white light image is acquired (blue), (2) during real-time NLM imaging (red), and (3) in post-processing when the recorded data is analyzed (green). The first phase, prior to the start of real-time NLM imaging lasts 3 seconds, during which the specimen tray is translated over the line scan camera imaging plane, the white light illuminator is activated, and the translation stage quadrature encoders are used to generate uniformly spaced line scan camera triggers at 30 μm intervals. Finally, the line scan image is rendered to the screen and the stage is returned to the initial position.

Fig. 6.

Analog and digital data flow diagram separated into three phases of system operation.

Following white light image acquisition, the system enters the continuous phase of imaging where NLM images are acquired at 16 frames per second as the operator assesses the surgical margins on the specimen. During this phase, the resonant scanner unidirectional rate (8 kHz) serves as the master clock, synchronizing both the read out of NLM frames at 1 frame every 528 cycles (512 forward lines and 512 backward lines followed by 16 fly back cycles while the slow axis resets between frames) and the translation stage position is read out by the PCIe-6323 DAQ (National Instruments, Inc.) once per cycle (8 kHz). NLM frames and position encoder data are then sent to the VHE renderer, which compares the stage position to the previous frame’s position, and if the stage is stationary, opportunistically averages frames to improve image quality. If the stage is moving, the previous frame is discarded and no averaging is performed. Following any averaging, the renderer performs vignette correction and absorption calculations for VHE rendering. Opportunistic averaging enables the user to view averaged data when the stage is stationary without distortion or blurring while translating the specimen. During this phase, all acquired data, including the white light image, NLM frames, and position data is recorded, enabling replay of procedures.

After the conclusion of real-time imaging, logged data is optionally processed to generate a permanent record of the procedure. During this phase, the motion data is combined with the stored NLM frames to perform averaging/data reduction of any stationary frames, and then motion artifacts are removed from frames acquired during motion. Because the fast axis scanner is rapid (62.5 μs per line), a specimen translation speed of 1 cm/s results in an intra-line displacement of 625 nm, less than one resolvable spot. Consequently, individual fast axis lines have no intra-line motion artifacts and stage motion results only in an additional displacement added between scanlines along the slow axis. Using the position encoder data, specimen motion can be added to the slow axis displacement to reposition individual fast axis lines back to their true, undistorted position, effectively removing the effects of motion. Finally, the undistorted frames are stitched into seamless mosaics as described previously [35] (Fig. 7). This enables review of any real-time assessment procedures in a format analogous to a permanent paraffin histological section.

Fig. 7.

Dewarping and stitching of live data to generate mosaics. By synchronously recording position data with each fast axis cycle, the true geometric position of all pixels in each image can be calculated and the resulting frames stitched into an undistorted mosaic.

4. Results

4.1 NLM resolution

To measure the system resolution, 200 nm diameter fluorescent beads (FSDG002, Bangs Laboratories) were embedded in PDMS (n = 1.42), deposited onto the specimen tray and imaged at both 780 nm and 1030 nm to test the system resolution at two commonly used femtosecond slaser excitation wavelengths (Fig. 8). To compensate for the limited precision of the axial objective translator, the position encoder was read out synchronously with frame acquisition and z-stacks were resampled to reduce the effects of drift and backlash. The 5x objective lateral resolution was 1.45 μm and 2.10 μm for 780 nm and 1030 nm, respectively while the axial resolution was 20.0 and 27.0 μm, respectively. The 10x objective lateral resolution was 0.93 μm and 1.2 μm for 780 nm and 1030 nm, respectively, while the axial resolution was 7.1 μm and 8.7 μm, respectively. Notably, the 5x objective resolution was more strongly affected by longer wavelength operation than the 10x, likely reflecting the superior NIR achromatization of the 10x objective. The system lateral and axial resolution was degraded because of a substantially thicker imaging glass (400 μm) than the objectives’ design thickness (180 μm), although the effect is modest at 0.45 NA and insignificant at 0.25 NA. In a previous study, we evaluated attenuation at both wavelengths in normal human breast tissue and obtained an e⁻¹ depth of ~34 μm at 780 nm and ~52 μm at 1030 nm [43].

Fig. 8.

Lateral (left) and axial (right) point spread function plots for the 5x (top) and 10x (bottom) objectives measured using sub-resolution fluorescent beads.

NLM resolution was further assessed by imaging discarded human breast tissue taken from the superficial aspect of a mastectomy specimen using both 5x/0.25 and 10x/0.45 objectives. Laser power was adjusted so that both objectives produced equal peak fluorescence intensity and 8 cm² mosaic images were acquired using each objective (Fig. 9, top row). Both objectives readily resolved areas of fat, stroma, and aggregations of cell nuclei, but cellular details were difficult to discern due to the much thicker axial sectioning of the 5x/0.25 objective (Fig. 9, bottom row). In contrast, individual nuclei were well-resolved with the 10x/0.45 objective.

Fig. 9.

Comparison between 5x/0.25 NA and 10x/0.45 NA objectives at 1030 nm excitation wavelength using discarded superficial breast tissue excised during surgery for invasive ductal carcinoma. The biopsy site and skin are present along with large areas of fat and stroma (top left). Mosaic images viewed at low magnification show relatively limited difference between the two objectives (top center – 5x, top right – 10x), however zoomed views of the boxed regions show that cellular features are poorly resolved with the 5x/0.25 objective. In contrast, the structure of a blood vessel is readily identifiable with the 10x/0.45 objective. Scale bars: 1 cm (top) and 400 μm (bottom). Total area: 7.9 cm². Full resolution image: http://imstore.mit.edu/system/Fig9.html.

4.2 Comparison to histology

Mosaics were also acquired to compare NLM imaging to FFPE histology, although it is important to note this would not be performed during clinical procedures due to the longer acquisition times. The high speed of the translation stage (~100 ms) and the very high imaging rate of the NLM system enables relatively rapid mosaic mode imaging. Using a 3x image averaging, a 1 mm² field size, and 490 nm sampling density, acquisition of one frame followed by translation and a software delay of ~500 ms requires less than 1 second. Accounting for overlapping of adjacent frames to enable seamless stitching, this is a net equivalent pixel rate of 4 MP/s and a time per area of 2.3 minutes per cm², approaching some of the fastest area imaging rates reported in the literature [48].

To validate the NLM imaging and staining protocol, we performed mosaic imaging of 50 discarded breast specimens from 21 patients. Representative NLM VHE images and scanned FFPE H&E slides of a specimen with extensive DCIS are shown in Fig. 10. Low magnification images from NLM and the H&E scanned slide show an epithelial proliferation expanding ducts and lobules. Zoomed views of the same images show that the proliferation is composed of monomorphic atypical epithelial cells with low to intermediate grade nuclei consistent with a diagnosis of ductal carcinoma in situ.

Fig. 10.

Comparison between an NLM mosaic image of the specimen surface using the 10x / 0.45 NA objective and FFPE histology from approximately 300 μm below the surface of a breast surgical specimen with ductal carcinoma in situ. Scale bar: 2 mm. Total area: 2.2 cm². Full resolution image: http://imstore.mit.edu/system/Fig10.html.

4.3 Comparison between real-time assessment and mosaic imaging

To demonstrate the advantage of real-time margin assessment, inked bread loaf specimens were imaged using both comprehensive mosaic NLM and using real-time assessment where inked margins were identified using widefield white light microscopy and then imaged under operator control at video rate (Fig. 11). Although most surgical specimens have inked margins on only part of the circumference, we tested the most conservative case and imaged the entire circumference. In this example, real-time assessment of the margin using panning at 10x magnification required only 44.5 seconds including the time required to interpret images, compared with 487 seconds to acquire a mosaicked image of the entire specimen. Real-time imaging of the margin at 5x magnification required only 24 seconds due to the larger field of view, although detailed evaluation of pathology identified at low magnification would require changing to 10x magnification in regions of interest. If inked margins were present on only part of the circumference, the assessment time would be proportionally faster.

Fig. 11.

Comparison of mosaic and real-time imaging using normal human breast tissue inked to simulate a surgical margin. The widefield white light image (left) enables visualization of the tissue and regions of purple and blue ink on the simulated margins. Mosaic NLM (center) images the entire specimen face but required 487 seconds. In contrast, real-time assessment using the 10x objective required less than 45 seconds because the operator imaged only the margins using opportunistic averaging and did not image non-diagnostic regions such as adipose tissue. The lower scattering and absorption of 1030 nm excitation wavelength enabled imaging below thin layers of ink on the simulated margin. Scale bars: 2 mm. Total area: 3.9 cm². Full resolution image: http://imstore.mit.edu/system/Fig11.html.

We further evaluated real-time imaging using mastectomy specimens dissected to expose both invasive ductal carcinoma as well as normal tissue. Figure 12 shows a simulated positive margin inked yellow as well as two simulated negative margins inked green and orange on an approximately 2 cm² area specimen. Simulated inked margins can be visualized using both the widefield white light image (bottom left frame) as well as at high magnification using NLM, where the inked tissue shows as regions of distinctive fluorescence in the nuclear channel (green ink), stroma channel (orange ink) or both (yellow ink). NLM imaging using the 5x objective (top left frame) enabled all 3 margins to be assessed in less than 30 seconds, while imaging using only the 10x objective required approximately twice as long to survey due to the smaller field of view (top right frame). While cellular details are less well visualized using the 5x objective, normal adipose tissue or stroma and invasive ductal carcinoma can still be differentiated even at low magnification due to the higher density of strongly staining cell nuclei. To confirm this finding, real-time imaging with the 10x objective was used to visualize a region of invasive ductal carcinoma in contact with the yellow ink (bottom right frame), revealing both a poorly differentiated tumor and an adjacent region of necrosis with few nuclei.

Fig. 12.

Comparison of real-time imaging using 5x/0.25 NA and 10x/0.45 NA objectives to assess an inked tissue specimen containing both normal tissue and invasive ductal carcinoma. Orange, yellow and green ink are grossly visible in the widefield image (bottom left), while each color is visible as fluorescence in one or both spectral channels in the NLM images. Both 10x and 5x objectives enable surveying the entire simulated margin in less than 1 minute, however the 5x objective is approximately 2x faster and can assess tissue further from the inked margins. Scale bars: 2 mm (top, left) and 400 μm (bottom right). Total specimen area: 4.0 cm². Full resolution image: http://imstore.mit.edu/system/Fig12.html.

5. Discussion

Inking, transection and evaluation of bread loaf specimens is a standard protocol for postoperative surgical margin evaluation in breast, prostate and other surgeries. In this procedure, specimens are inked to indicate the true surgical margin location, and then cut perpendicular to the margin surface. A pathologist evaluates the minimum distance a pathology is present from each of the inked margins on the bread loaf specimens. In contrast to en face margin evaluation, as used in applications such as Mohs surgery, where most or all of the histologically processed tissue surface or aspect is histologically evaluated, in a bread loaf specimen, only the inked regions and an additional zone of tissue adjacent to the ink is relevant to determine margin status. The multiscale imaging approach we have developed is well suited for assessing bread loaf margin status because it follows the standard pathology practice of evaluating tissue grossly followed by evaluating it with increasing magnification while avoiding unnecessary high magnification imaging of non-diagnostic regions. To validate this approach, we used simulated surgical margins dissected from larger breast surgical specimens, and compared the time required to image and assess inked margins using real-time, video rate imaging versus the time required to acquire mosaic images.

We demonstrated that even for relatively small (2 cm²) bread loaf specimens, comprehensive mosaic imaging of a single depth of an entire tissue aspect was an order of magnitude slower than the combined time required to image and evaluate under real-time operator control. We further found that using a 5x objective for inspection followed by higher 10x magnification for areas of focal pathology was a further factor of ~2 faster. These results were demonstrated using specimens with both simulated negative margins and positive margins with invasive ductal carcinoma. While margins with pathologies such as invasive lobular carcinoma or DCIS could be more time consuming to assess than positive margins with uniform invasive ductal carcinoma, we note that the overwhelming majority of surgical margins are ultimately found to contain only normal tissue. Consequently, the time to exclude regions of normal tissue is expected to occupy most of the procedure time, making the ability to rapidly exclude normal tissue critical. Further studies will be required to determine the time required to evaluate true surgical margins with complex pathologies such as DCIS.

The multiscale NLM system integrates a number of features that facilitate rapid histological assessment of unsectioned tissue analogously to a conventional histology microscope viewing FFPE histology slides. First, the maximum field of view is adjustable between 3.4 mm using the 5x / 0.25 NA and 2.0 mm using the 10x / 0.45 NA objective, while higher magnification (20x/40x) is implemented through digital control of the scanner angles. At maximum scan angle, this corresponds to an imaging rate of 145 mm²/s and 50 mm²/s for the 5x and 10x objectives, respectively. The rapid imaging rate, low latency OpenGL VHE rendering, and adjustable magnification reproduce the conventional histological workflow where FFPE H&E slides are evaluated at low magnification to identify suspected areas of pathology prior to high magnification. Changing the objectives takes less than 1 second using the custom-built objective translator, a similar time to changing objectives in a conventional bright-field microscope. Second, to rapidly identify inked margins and grossly apparent features, the system incorporates a widefield white light view that tracks the location of the high magnification NLM image in real-time, enabling the operator to rapidly locate margins on large specimens while avoiding areas that are not relevant to assessment such regions far from any margins. Finally, by synchronously reading out the stage position with each fast axis scan, we implemented opportunistic averaging, a novel feature that enables averaging without motion blur or other artifacts when translating the stage. This data was post-processed to generate mosaic images from data acquired by real-time user panning, enabling efficient review of procedures in a fraction of the time required to replay the data at real-time speed.

As an alternative to real-time imaging, the time required to image margins on bread loaf specimens could be reduced using machine vision algorithms to segment inked margins identified on widefield white light imaging and then guide tissue mosaicking to these regions. However, it is challenging to determine which imaging depths are likely to be diagnostic when mosaicking without the time-consuming process of imaging each depth, as well as what objective and sampling density is optimum to assess a region. In contrast, an experienced pathologist can rapidly identify areas of suspected pathology and perform comprehensive imaging of all depths in seconds while selecting the lowest magnification required. Furthermore, when imaging in mosaic mode, the number of frames to average must be decided prior to imaging, consequently all areas of the tissue receive the same degree of averaging. In contrast, real-time assessment using the opportunistic averaging method developed here enables less relevant areas of tissue to be imaged without averaging, while performing a high degree of averaging over regions that the operator identifies as diagnostically relevant. Finally, while more efficient sampling strategies may enable more rapid acquisition of mosaic images, they do not overcome the fundamental disadvantage posed by the need to sequentially image, stitch frames and then assess histological images.

The emphasis of this study was to develop a technique that is consistent with existing clinical workflows based on bread loaf evaluation of surgical margins, the current standard for postoperative histological evaluation of invasive breast cancer [49] and DCIS [50]. As of 2018, a large majority of pathologists exclusively use bread loafing for breast margin assessment, a minority use a combination of bread loaf and en face histology, while en face evaluation alone is rarely utilized due to the difficulty in sectioning the surface aspect of breast specimens [51]. Consequently, the approach presented in this manuscript has the advantage of maintaining the current standard of care. However, bread loaf evaluation images only a very small fraction of the total surgical margin, typically less than 1% [52]. Further studies of the correlation between en face and bread loaf margin assessment may be required to understand if en face evaluation using fluorescent imaging modalities can improve the evaluation of breast surgical margins.

Intraoperative imaging of surgical specimens using NLM reproduces some aspects of existing FSA workflows [14–16] but with dramatically reduced processing times, enabling evaluation of a much larger number of bread loaf specimens in a shorter amount of time. Turnaround times for FSA processing are typically about 20 minutes to section a single specimen [53], necessitating selective sampling of a subset of the tissue that would be evaluated postoperatively. In contrast, using NLM an unlimited number of bread loaf specimens can be stained in less than 3 minutes. This improvement in throughput suggests that the entirety of the tissue postoperatively evaluated with FFPE under the SSO-ASTRO-ASCO guidelines could be evaluated in the time required to process a single FSA section. However, it is important to note that it is possible to perform FSA in parallel on multiple specimens to more comprehensively evaluate margins but this requires additional pathology infrastructure which may not be available or cost-effective. One study at the Mayo Clinic using FSA during BCT demonstrated a reduction of repeat surgeries from 55% to 19.3% with an average of 6 frozen sections per patient, and 27 minutes average processing time with 2 minutes additional time per specimen using multiple histotechs to process specimens in parallel [46]. However, times increased appreciably for larger number of specimens and in one extreme case, evaluation of 15 specimens required 1 hour. This study further used thicker frozen sections (20 µm, analogous to 5x data in Fig. 9), resulting in a tradeoff between processing time and axial section thickness. In contrast, no such tradeoff is required with the multiscale NLM approach presented here because NLM enables thin axial sectioning of fresh tissue. Furthermore, a large study of FSA margin evaluation in nerve sparing prostatectomy reported substantial improvements in rates of nerve sparing surgeries, but required 5 cryostats and 4 histotechs working in parallel to evaluate 10 to 24 specimens per patient in an average of 35 minutes [54]. In contrast, multiscale NLM scales much more favorably with increasing numbers of bread loaf specimens, because after inking and grossing, multiple bread loaf specimens can be stained simultaneously and then examined without physical sectioning as shown in Figs. 11 and 12. Furthermore, although bread loaf specimens were dissected to fit standard histological processing cassettes, in future applications NLM could image a large several centimeter bread loaf specimen without dissection, further reducing imaging times and enabling improved coverage.

While we imaged at 1030 nm, typical for ytterbium fiber lasers, we also evaluated the resolution of the NLM system using 780 nm excitation wavelength, typical for titanium sapphire or frequency doubled erbium fiber lasers (Fig. 8). Both 780 and 1030 nm efficiently excite AO/SR101 labeled tissue, but 1030 nm enables >50% deeper penetration through tissue. At both wavelengths, the maximum lateral resolution was comparable to a typical 20x/0.5 NA bright-field histology microscope, while axial sectioning was ~1.5 to ~2x lower than a typical paraffin section. The 5x/0.25 NA objective had reduced lateral and axial resolution as compared to the 10x objective, however discrimination of adipose tissue and normal stroma from regions of abnormal stroma or tumor was still possible because of the high contrast of the fluorescent labels used in this study. Notably, the autofluorescence of surgical marking inks as well as the attenuation from thin layers of ink and other surgical debris deposited onto tissue specimens during excision or grossing were reduced at longer wavelengths, enabling more comprehensive assessment of specimen margins. While evaluation of multiple depth serial sections per bread loaf is not typically performed for breast margin assessment, the ability to image through surgical debris is likely to be important. Combined with our recent demonstration that inexpensive and compact ytterbium fiber lasers are effective for NLM imaging of surgical specimens [43], the ability to operate at long wavelengths may be important for clinical translation of compact, affordable real-time surgical equipment that can image through surgical debris and tissue marking ink.

The multiscale imaging approach demonstrated here may be applicable to other less costly imaging modalities such as CFM, SIM, MUSE or OCT that can perform high-speed imaging using multiple objectives or variable magnification with a single objective. However, the ability to image through surgical debris, blood and tissue marking ink is an important advantage of NLM that enables imaging of tissue aspects that have surface contamination typical of breast surgical specimens. In a previous manuscript, we evaluated CFM and found its ability to image subsurface features to be significantly more limited than NLM [28], while MUSE can only image a single plane on the tissue surface and so cannot perform subsurface assessment [32]. OCT provides excellent subsurface imaging but has limited nuclear contrast. The extent to which these modalities could be used to image specimens with ink surfaces and other contamination requires further investigation.

6. Conclusion

We have demonstrated an instrument enabling the assessment of surgical specimens at rates sufficient to comprehensively assess surgical margins during breast conserving therapy and other intraoperative imaging scenarios. In contrast to previous work using image mosaicking, we introduce a multiscale imaging approach combining widefield white light imaging for gross specimen evaluation and identification of inked margins, low magnification NLM for rapid tissue surveying, and high magnification NLM for detailed evaluation at cellular resolution. GPU-accelerated processing is used to provide video-rate rendering of fluorescent data as virtual H&E slides. We demonstrate that this approach enables rapid assessment of centimeter-scale surgical specimens in a workflow analogous to FSA processing of tissue, but with dramatically faster tissue processing times, no freezing artifacts, and compatibility with subsequent histological processing. Clinical studies are necessary to demonstrate sensitivity and specificity of intraoperative NLM compared with the clinical standard of post-operative FFPE histology. However, the technology and methods described here could enable comprehensive intraoperative margin assessment in breast as well as a wide range of other cancer surgeries.

Funding

National Institutes of Health (R01-CA178636-04, R01-CA075289-20, F32-CA183400-03); Air Force Office of Scientific Research AFOSR contracts (FA9550-12-1-0551 and FA9550-15-1-0473).

Disclosures

The authors have filed intellectual property on portions of the methods and apparatus described in this article.