All-digital histopathology by infrared-optical hybrid microscopy

Significance

This study reports the ability to provide label-free molecular information from infrared (IR) spectroscopy via the ubiquitous optical microscope. Modeling the thermal-mechanical coupling of samples, we design, build, and validate an IR-optical hybrid (IR-OH) microscope that uses optical interferometry to measure the dimensional change in materials arising from spectral absorption. We show that the seamless compatibility of IR-OH with routine optical microscopy and with emerging computational ubiquity enables all-digital pathology with applications across the spectrum of biomedical science. IR-OH microscopy provides a means to retain the ease of use and universal availability of optical microscopy, add a wide palette of IR molecular contrast, and utilize emerging computational capabilities to change how we routinely handle, image, and understand microscopic tissue structure.

Abstract

Optical microscopy for biomedical samples requires expertise in staining to visualize structure and composition. Midinfrared (mid-IR) spectroscopic imaging offers label-free molecular recording and virtual staining by probing fundamental vibrational modes of molecular components. This quantitative signal can be combined with machine learning to enable microscopy in diverse fields from cancer diagnoses to forensics. However, absorption of IR light by common optical imaging components makes mid-IR light incompatible with modern optical microscopy and almost all biomedical research and clinical workflows. Here we conceptualize an IR-optical hybrid (IR-OH) approach that sensitively measures molecular composition based on an optical microscope with wide-field interferometric detection of absorption-induced sample expansion. We demonstrate that IR-OH exceeds state-of-the-art IR microscopy in coverage (10-fold), spatial resolution (fourfold), and spectral consistency (by mitigating the effects of scattering). The combined impact of these advances allows full slide infrared absorption images of unstained breast tissue sections on a visible microscope platform. We further show that automated histopathologic segmentation and generation of computationally stained (stainless) images is possible, resolving morphological features in both color and spatial detail comparable to current pathology protocols but without stains or human interpretation. IR-OH is compatible with clinical and research pathology practice and could make for a cost-effective alternative to conventional stain-based protocols for stainless, all-digital pathology.

Optical microscopy is ubiquitous in biomedical research for the examination of microscopic structure of tissues and forms a cornerstone of all development and disease studies and much medical decision-making (1). Visualizing structure and chemical composition in biomedical samples, however, requires the use of stains or labels. Label-free optical techniques have been proposed to visualize molecular content without dyes or labels to monitor processes unperturbed, to observe composition that is not easily amenable to staining (2). Molecular spectroscopy through more than 125 y of progress (3) enables the absorption of midinfrared (mid-IR) light to be used as an identifying molecular signature (4). Extant applications of mid-IR microscopy span biomedical tissue diagnostics (5) and cell biology (6) to polymeric (7), plant (8), and forensic samples (9) and interstellar analyses (10) over nearly 70 y (11). IR imaging, however, is incompatible with visible microscopy. Detectors used in visible and near-IR imaging are nonresponsive, and most glasses are highly absorbing over the IR spectral band (2 to 12 μm), precluding the use of commonly available imaging components that have advanced other spectroscopic approaches (12). Despite exciting recent advances in quantum cascade lasers (QCL) (13, 14) providing a powerful impetus for rapid, high-fidelity imaging in both point-scanning (15) and wide-field (16–19) modalities, IR pixel sizes are ∼100-fold larger than those easily achieved in visible microscopy, and data acquisition speed is still three to four orders of magnitude slower than in visible imaging (20). The abundance of applications, ease of use, and ubiquity of visible microscopy points to the significant potential of an integrated approach with IR spectroscopy.

Several transformative applications of a hybrid of visible microscopy and IR spectroscopy are apparent. For example, the microscopic examination of stained tissue using visible microscopy has been the standard method for detecting and grading most forms of human cancer in research and clinical care (21, 22). Combinations of staining and microscopy with advanced artificial intelligence (23, 24) now provide new capabilities, but this avenue is ultimately limited by available exogenous labels. Complementarily, IR imaging (25) offers detailed molecular contrast without the need to stain tissue, and its combination with machine learning is being used to augment and automate histopathology (26–28), discover cell physiology (29), inform therapeutic decisions (30), and aid discovery of new biomarkers (31). Making visible microscopy and IR imaging compatible could thus yield tremendous benefits. Using visible imaging components for recording IR absorption has recently been the focus of several approaches using up-conversion of light or secondary sample effects. Strong absorption causes secondary effects such as refractive index changes for ultrasensitive measurements (32) or local photothermal expansion for photoacoustic imaging (33), which provide a means to overcome the extant limitations of IR imaging. The concept of dynamic photothermal changes in morphology (34, 35), force (36, 37), or near-field coupling (38–40), using an atomic force microscope cantilever as local probe, has been reported for point-by-point IR measurements. Noncontact optical photothermal microscopy is more recent (41–50) and typically utilizes a local IR illumination coincident with a highly focused visible probe beam to measure local refractive index change by beam scattered out of the angular acceptance of the objective lens. This method has been applied to chemical imaging of tissue and live cells (44, 46, 47), bacteria (48), and pharmaceutical tablets (49). These point-scanning approaches, just as for point scanning FT-IR spectroscopy, involve long scan times (51). The need for sample scanning renders them impractical for the rapid acquisition of spectrally resolved photothermal datasets in cases of large samples such as tissue sections for pathology. Recent studies have demonstrated wide-field measures of photothermal absorption and scattering (52, 53). Despite providing optical compatibility, however, these methods provide fields of view and pixel counts that are much smaller than direct absorption IR microscopy (54). More importantly, data exhibit relatively poor sensitivity compared to direct absorption IR microscopy as the beam deflection is a measure of the photothermal change in the refractive index. Data quality is a primary driver of accuracy in obtaining biomedical information (55), thus limiting the utility of these methods and precluding any reports of sensitive histopathologic imaging. High-quality, high-speed, and high-resolution imaging of absorption of large areas of tissue remains elusive, but is needed for histopathology and the effective application of artificial intelligence.

Here we report a solution to these challenges by combining wide-field visible microscopy with IR sample illumination from a QCL, a modality we term IR-optical hybrid (IR-OH) imaging. We demonstrate that infrared absorption in thin films and tissue sections can be measured by detecting the IR absorption-induced sample expansion in a wide-field interferometric arrangement. Particularly, our design derives a synergy from both visible and IR microscopy, rather than merely combining them in a single unit, to provide an architecture for high-speed, superresolution IR imaging. One of the central tasks in all of biomedical sciences is the recognition of tissue structure as a measure of function, disease, or development associated changes. We acquire and segment multihundred megapixel IR datasets on a breast tissue microarray and a breast biopsy at high accuracy, which provides experimental evidence of the potential of our method to enable all-digital pathology based on visible microscopy and at a level of detail near current pathology practice.

In IR-OH, we directly measure sample expansion in an interferometric arrangement. We first developed a simple model to predict photothermal expansion in thin film samples to mimic those typical in histopathology (see SI Appendix, Note 1, for details). Our simulation involves the analysis of a uniform film placed on a semiinfinite substrate to understand the coupled thermal-mechanical response of this assembly. We solved heat conduction to obtain the temperature modulation profile, $\left|u_T\left(z,\mathrm{\Omega }\right)\right|$, in response to pulsed heating from the mid-IR laser at pulse frequency Ω. For a 5-$\mu \text{m}$ thin SU-8 polymer film and laser pulse frequency $\mathrm{\Omega }=600\hspace{0.5em}\text{Hz}$ (Fig. 1A), the model predicts a temperature modulation profile within the film on the order of 1 K and a resulting expansion of the film between 0.1 and 1 nm, which is readily measurable with wide-field interferometry (56–60). Based on this prediction, we conceptualized, built, and tested a wide-field IR-OH microscope as shown schematically in Fig. 1C. Detailed in Materials and Methods, our instrument consists of a pulsed mid-IR QCL that is used to illuminate a large area on the sample at the repetition rate, $\mathrm{\Omega }$. We probe the induced sample deformations by illuminating with visible light from a narrowband LED $\left({\lambda }_0=660\hspace{0.5em}\text{nm}\right)$ and collecting backscattered light from the sample with a Mirau interference objective. While the sample is vertically (z) translated, we capture interferograms, $\hspace{0.17em}{I}_{xy}\left(z\right)$, at a high camera frame rate of $F=500\hspace{0.5em}\text{Hz}$ (Fig. 1 D and E). Demodulation at frequency $\mathrm{\Omega }-S$ of the interferograms, $I_{xy}$, yields a signal proportional to the first-order harmonic of the surface deformation, $\mathit{ϵ}\left(\text{Ω}\right)$, where S is the fringe frequency as determined by the rate at which interference fringes pass by as a function of the vertical sample position z (61). Subsequent normalization of the demodulated signal to the incident IR intensity, $I_{\text{IR}}$, constructs the IR photothermal image, $s_{\text{p}}^{\left(\mathrm{\Omega }\right)}$. High SNR is provided by a high-intensity QCL and a million-electron, full-well camera that offers significantly reduced shot noise over conventional camera technology. The 1D model from Fig. 1A connects photothermal expansion, $s_{\text{p}}^{\left(\mathrm{\Omega }\right)}$, to IR absorption through the locally deposited heat $Q$, surface reflectivity $R_S$, and thermomechanical properties $M$ of the sample:

$$s_p^{\left(\mathrm{\Omega }\right)}\left(\omega \right)\propto \frac{4\pi }{{\lambda }_0}{A}_0^2\sqrt{R_S}\cdot \frac{Q\left(\omega \right)}{I_{IR}\left(\omega \right)}\cdot M\left(\mathrm{\Omega }\right),$$ [1]

which correlates well with conventional IR spectroscopy (Fig. 2). Intriguingly, experimental surface vibration amplitudes obtained with a thin SU-8 film for a variety of pulse frequencies, Ω, show good agreement with the model without requiring any fitting (Fig. 1B), which provides further confidence in our design. In addition to providing IR absorption, demodulation at frequency $S$ constructs an optical image of the sample akin to en face scattering maps of full-field optical coherence tomography (59, 60). Perfectly coregistered to the IR absorption image, this provides opportunity for IR-optical synergistic approaches as described later. Our IR-OH design leverages technological advances in visible imaging technology (refractive optics and modern sensors), while preserving compatibility with the existing knowledgebase of IR imaging of tissues (62). IR-OH allows us to access both the spatial content and the spectral information needed for advanced algorithms to enable all-digital pathology.

Fig. 1.

Concept of infrared-optical hybrid (IR-OH) microscopy. (A) Temperature modulation profile, $\left|u_T\left(z,\mathrm{\Omega }\right)\right|$, obtained with a 1D model of an absorbing thin film placed on a semiinfinite substrate. (B) Predicted and experimentally measured surface deformation amplitude, $\mathit{ϵ}\left(\text{Ω}\right)$, as a function of the laser pulse rate, Ω. The horizontal dashed line marks the noise floor of our instrument. (C) IR-OH microscope setup. QCL, single-frequency, midinfrared QCL tunable to $k=\mathrm{900...1},900{\text{cm}}^{-1}$; MO, Mirau interference objective; PM, parabolic mirror; CM, cylindrical mirror; CMOS, camera; CHOP, optical chopper. The QCL beam for IR sample illumination is shown in yellow. The optical probe beam is shown in red. The sample is mounted on a piezo stage (not shown) for z translation. (D) Interferogram $I_{xy}$ at a single pixel on SU-8 and (E) its Fourier transform, showing fringe frequency peak $S$ and surface deformation signals $\mathrm{\Omega }\pm S$.

Fig. 2.

Validation of IR-OH against FT-IR. (A and B) Spatial comparison. (A) Single field of view of IR-OH imaging, showing absorption at 1,502 cm⁻¹ (Left) and digital zoom on group 8 (Right; indicated by yellow arrow in Left) of an SU-8 test target wherein all elements from groups 5 to 8 were clearly resolved. (B) Composite high-definition FT-IR image of the same area, obtained by mosaicking due to the small field of view (tiles are indicated by white lines), did not resolve any element of group 8. (C) Spectral validation: comparison of IR-OH spectrum, taken from the square of group 6, against FT-IR spectrum of a bulk SU-8 film. (D) The improved spatial detail in IR-OH is quantitatively compared to FT-IR imaging via the contrast as a function of spatial frequency derived from the USAF standard. (E and F) IR-OH and FT-IR spectral profiles recorded along a line across an SU-8 film edge (elements 1 to 4 and 5 to 8 are on the SU-8 and substrate, respectively) showing greater consistency in spectra due to reduced scattering contributions in IR-OH compared to the measured beam attenuation in FT-IR imaging.

We validated IR-OH against the established standards of both Fourier transform infrared (FT-IR) spectroscopic imaging and optical microscopy using a United States Air Force (USAF) 1951 standard pattern synthesized for IR microscopy. Absorbance contrast (at 1,502 ${\text{cm}}^{-1})$ reveals that IR-OH improves spatial resolution by a factor of ∼4 (380 line pairs per mm vs. 100 line pairs per mm) compared to high-definition FT-IR imaging (Fig. 2 A and B), as quantified by contrast analysis (Fig. 2D). To validate IR spectral fidelity, we performed spectroscopic imaging in the range of 950 and 1,700 ${\text{cm}}^{-1}$ and observed spectral peak frequencies and line shapes consistent with FT-IR (Fig. 2C) spectra. Spectral noise of the IR-OH data are analyzed in SI Appendix, Fig. S1. Observed differences between spectra from IR-OH and FT-IR microscopy may be attributed to differences in optical configurations that cause well-known changes in recorded FT-IR spectra (63, 64) and the possibility of inhomogeneous heating in a thin film absorber rather than a uniform film on which our model is based. More accurate reconstruction of IR absorption might be achieved with more sophisticated thermal models, just as for scattering models in FT-IR imaging (65, 66). Importantly, IR-OH imaging with an optical camera delivered ∼2 megapixels (MP) of measurements at a time (Fig. 2A), while typical mid-IR uncooled (0.3-MP) or cooled (16-kP) cameras do not offer this large multichannel advantage nor the possibility of rapidly increasing sensor sizes or quality that visible cameras have experienced over the past decades. Together with the high sensitivity afforded by interferometric detection, our approach allows for 10-fold larger area coverage than state-of-the-art wide-field FT-IR imaging (460 × 460 μm² vs. 140 × 140 μm²) at a smaller pixel size (0.32 × 0.32 μm² vs. ∼1.1 × 1.1 μm²), equating to an ∼125-fold increase in coverage. Finally, by directly measuring absorption, we mitigate scattering and its effects on the recorded data (67). As illustrated by a spectral line scan taken across an edge in the SU-8 film, IR-OH data yielded consistent spectra on the SU-8 film (spectra 1 to 4 in Fig. 2E) and no signal on the substrate (spectra 5 to 8). The approach improves considerably over wide-field FT-IR imaging, whose large baseline fluctuations and peak distortion (Fig. 2F) present a challenge to accurate analyses, prompting the quest for many approaches to corrections, slowing data processing and triggering considerable debate over the interpretation of data (68–70). The suppression of the effects of scattering in the data in IR-OH presents a new opportunity for improved quantitative analyses, obviating spectral corrections such as baselining or Mie corrections and simplifying data processing for histopathology. While progress in classical IR microscopy is hindered by fundamental physical limitations (e.g., diffraction limit), our results reveal the potential of IR-OH for superresolution IR imaging, largely improved coverage, and suppression of IR scattering that can be employed for unforeseen, transformative applications in histopathologic imaging. Further, the interferometric detection in IR-OH provides a sensitivity advantage over previous photothermal techniques that relied on beam deflection from IR absorption-induced refractive index change (41, 42, 44, 45, 48, 53). While our method measures the physical response of expansion directly, deflection methods measure the refractive index change that occurs upon deflection and need higher probe intensity than we have employed. The direct expansion measurement is challenging but rewarding. It is notable that our method provides both phase and amplitude measurements that allow us to conduct further analyses. For example, we identified that the contribution of surface expansion to the optical signal is one magnitude larger than that contributed by a refractive index change, as measured by sample reflectivity (SI Appendix, Fig. S2). High-sensitivity, wide-field detection with large area of view and pixel count enables spectrally imaging of large samples, as we will demonstrate in the following sections.

We evaluated the accuracy of IR-OH for pathology by imaging breast tissue as a test case. For over 125 y, stains have been used for histopathologic recognition in clinical and research activities, allowing differentiation of cellular and subcellular components in tissue. For example, the hematoxylin and eosin (H&E) stain allows a trained observer to contrast epithelial cells from the surrounding stroma (Fig. 3A). Morphological features of epithelial cells are the basis for cancer diagnoses and studying progression. In contrast, vibrational imaging of tissue can provide label-free diagnoses by machine learning techniques applied to chemical constitution of tissue without stains. We demonstrate this method enabled by IR-OH imaging of an unstained breast tissue microarray at 22 discrete IR frequencies and subsequently applied random forest (RF) classification on a subset of only seven frequency bands (Materials and Methods). Fig. 3 B–D show IR absorption (1,550 cm⁻¹) and classification maps of selected tissue types, in accordance to the gold standard (H&E images, Fig. 3A). Classifier performance was assessed using receiver operating characteristic (ROC) curves (Fig. 3E), demonstrating an overall area under the curve (AUC) of 0.93, which is comparable to state-of-the-art IR imaging (71). Classification of the epithelium into disease (malignant and noncancerous subtypes) showed an AUC of ∼0.90. Representative IR-OH spectra (Fig. 3G) revealed differences that allow for classification and are consistent with FT-IR spectra (Fig. 3H). We conducted feature selection on the original 22 frequency point dataset (Fig. 3F and Materials and Methods). We observed saturation of classifier performance for the first seven most relevant frequency bands (Fig. 3D, five-class model) with only little improvement provided by using all 22 bands for classification. Interestingly, the first five most relevant bands already provided tissue type identification with >0.90 AUC as shown in Fig. 3C (four-class model), which could make IR-OH a rapid screening tool for guiding robotic surgery where localization and identification of anatomical structures is needed. Importantly, mitigation of scattering artifacts in IR-OH does not require the acquisition of additional IR frequencies for baselining spectral data. We only required ratios of the signals at various frequencies for our IR-OH protocol, which provides a speed advantage compared to QCL-based far-field IR imaging. With the subset of seven bands, our first-generation IR-OH instrument already provides full slide classification after ∼2.5 d of imaging time, and there is certainly room for further improvement by improving the sensitivity of the interferometer. In comparison, FT-IR microscopy would require about 26,000 individual frames with a 128 × 128 array detector to cover an entire TMA slide owing to the small format of the detector, as well as a large number of coadditions owing to the weak irradiance of a thermal source. Assuming a typical value of 32 coadditions, full slide imaging time is estimated to be on the order of 40 d for FT-IR microscopy. Recently, high-brightness QCL were shown to enable rapid spectral IR imaging based on discrete-frequency approaches, reducing full slide spectral imaging to less than a day. However, wide-field QCL microscopy is still fundamentally limited by diffraction; further, both image and spectral distortions (see SI Appendix, Note 5, for discussion) mitigate the widefield detection advantage and present challenges for high-quality imaging needed in pathology. We note that the use of QCL sources requires preselection of the IR frequencies prior to imaging, which might depend on the specific tissue type to be segmented. This could be done on the same instrument by spectral imaging at many data points or by FT-IR imaging of a representative sample and by performing subsequent feature selection. We further note that with IR-OH, tighter integration between machine learning and optical components will extend the abilities of this hybrid approach further (72).

Fig. 3.

IR-OH imaging of breast tissue and spectral cell type recognition. (A) H&E image of a breast tissue microarray section. (Scale bar: 500 μm.) (B) IR-OH absorption of an adjacent, unstained tissue section (at 1,550 cm⁻¹). (C) A four-class model (blood, epithelium, stroma, and other) allows rapid tissue component visualization based on five IR bands. (D) Epithelial classification (five-class model) permits both histologic cellular identification and recognition of cancer based on seven IR bands. (E) ROC curves quantify the accuracy of recognition in D by IR-OH. (F) AUC of the ROC curve increases with number of IR frequencies used for classification, showing a small number provides good recognition (G and H) Representative class spectra obtained with IR-OH and FT-IR imaging. Full classification maps are shown in SI Appendix, Figs. S3 and S4. Class mean spectra and variance are shown in SI Appendix, Figs. S5 and S6, and further AUC and ROC curves are shown in SI Appendix, Fig. S7.

IR-OH enables a different dimension in histopathology by combining visible microscopic morphology with chemical composition. The data can be used to produce computational H&E contrast, providing a stain-free means of tissue imaging that maps to extant clinical practice (Fig. 4). While absorbance images from FT-IR measurements (Fig. 4A) can now be obtained in higher detail by IR-OH imaging (Fig. 4B), the visible microscopy channel (Fig. 4C) provides a further and complementary contrast derived from a combination of factors, including density of the tissue, scattering, path length, and visible absorption that exceed simple brightfield images (16, 18). We combine absorption and morphologic data using machine learning to generate stainless H&E images at the resolution of optical microscopy (Fig. 4D). Sample morphology, e.g., the individual stromal fibers and structure of the breast acini, were reproduced well in both color and spatial detail compared to H&E images (Fig. 4E). Thus, our approach provides stain-free pathology images that rival current stain-based practice. IR-OH is thus the enabling technology needed for truly all-digital pathology without stains or human interpretation. In the short term it opens the door to research in stainless pathology by spectroscopists, microscopists, pathologists, and computer scientists for improved human interpretation and the translation of advanced machine learning algorithms developed on archival stained images (73). Extraction of texture features from the visible microscopy channel is a possibility (74, 75), such as, for example, based on Gabor filter banks, and could be combined with chemical information from the IR channel for novel synergistic tissue segmentation approaches. The required, perfect coregistration between visible and IR data is intrinsically provided by IR-OH. While this initial implementation can be greatly improved in terms of data quality by optimizing the geometry further, the combination of widefield microscopy in IR-OH is also compatible with other means of providing the sensitive visible microscopy measurements, such as methods based on quantitative phase imaging in transmission, which will allow greater morphologic detail for thin samples and unambiguously detect subcellular morphologies more precisely.

Fig. 4.

Morphology-based inspection of tissue with IR-OH to generate stainless staining images that mimic current clinical practice. (A) Uncorrected absorbance from FT-IR imaging (at 1,550 cm⁻¹), showing limited contrast due to limited optical resolving power and scattering. (B) IR-OH absorption (1,550 cm⁻¹). (C) Low-coherence interferometry images from the optical channel that are perfectly coregistered to those in B. (D) Computed H&E image using the stainless staining approach and (E) H&E image of adjacent tissue section. Full core images are shown in SI Appendix, Fig. S8. (Scale bar: 100 μm.)

Our IR-OH system can be applied to large pathology samples and is useful for whole-slide imaging. Full-slide absorbance images (400 mm²) were computationally segmented (Fig. 5A) using the machine-learned model previously developed in Fig. 3 and computationally stained (Fig. 5B). The image quality agreed well with a reference H&E image (cf. Fig. 5 B and C), providing conventional and additional disease state information for tumor detection and determination of tumor margins (further examples in SI Appendix, Fig. S9). Minimal sample preparation as provided by entirely backscattering geometry and rejection of IR scattering makes this technique appealing not only for minimally preparative histopathology (76) but also for in situ analysis in dynamic polymer systems or for forensics. In the current implementation of IR-OH, classification accuracy was limited by 1) inhomogeneity and fluctuations in the IR illumination of the sample, 2) low illumination power at the crossover points of the QCL preventing reliable measurements in some of the relevant spectral regions (e.g., 1,200 cm⁻¹ and 950 cm⁻¹ regions), and 3) the limited vertical sensitivity of our interference microscope (SI Appendix, Note 4). Optimized interferometric microscopes have been shown to reach picometer vertical sensitivity, promising more accurate classification and faster imaging speeds (77).

Fig. 5.

All-digital histopathology of an unstained breast surgical resection, combining automated recognition and traditional pathology. (A) Classification of surgical resection from IR-OH data and (B) its derived computational H&E. (C) H&E image of adjacent section. (Top) Whole-slide images. (Bottom) Digital zooms into region marked by arrow in A.

In terms of technological progress, IR-OH merges wide-field optical microscopy with vibrational spectroscopy into a hybrid microscopy platform that overcomes the diffraction limit of IR microscopy, and provides for a class of practical and cost-effective instrumentation for IR imaging. It enables a contrast mechanism for visible microscopy, providing molecular information without specialized stains or dyes. We thus access more than a century of knowledge and progress in quantitative molecular analyses through vibrational spectroscopy for visible microscopy. IR-OH also considerably enhances IR spectroscopic imaging technology by doing away with the need for IR-specialized imaging components and by leveraging recent developments in optical technology such as the megapixel visible camera employed here for increased coverage. High-frame rate, multimegapixel visible camera technology and QCL technology with consistent output power across the spectrum could lead to high-throughput chemical imaging at mm² field of view, submicron pixel size, and subsecond integration time. Building a unit of multiplexed, fixed-frequency QCL chips could further replace tunable QCL at the fraction of the cost. IR-OH could thus provide a new gamut of cost-effective IR microscopy that proves practical for routine biomedical imaging applications. We anticipate many variations on the basic IR-OH configuration presented here with innovations in QCL illumination, measurement optics of the sample expansion, and phase and amplitude contrast in optical detection. This activity will also spur for IR-OH directions that are of contemporary interest in quantitative optical microscopy, including computational methods for improved image acquisition, emerging methods that make great use of morphology such as deep learning, extraction of enhanced detail from the data, and reconstruction of the modified optical signals. The spectrally integrative approach of IR-OH is especially amenable to advanced machine learning algorithms that make use of chemical composition; morphology data; and, in future implementations, tomographic information afforded by the underlying low coherence interferometry principle.

In terms of use, the synergy of IR absorption contrast with optical microscopy in a backscattering geometry clearly holds further promise of IR-OH for minimally preparative imaging across the spectrum of applications where either visible microcopy or IR spectroscopy are useful. IR-OH especially brings an important functionality to optical microscopy and removes a major barrier of using IR absorption as a contrast for biomedical analyses. This compatibility with optical microscopy can obviate the need for stains or specialized knowledge for routine chemical analysis in biomedical research, making optical microscopy cheaper, more informative, and easier to apply. For example, the results of Fig. 5B demonstrate a potential to eliminate staining. This could cut down on precious research time and need for safely maintaining reagents and supplies, as well as human labor for a routine task. Since the tissue is not changed in any manner by IR-OH microscopy, it is still available for conventional or advanced analyses. With modern machine learning undergoing a similar democratization in availability and use, the combined application of vibrational molecular contrast and machine learning can greatly expand the analytical tools accessible to biomedical scientists. Together, the compatibility of IR-OH microscopy with optical microscopy and its synergy with emerging computational ubiquity to handle the wide palette of IR molecular contrast can change how we routinely handle, image, and understand microscopic tissue structure, enabling all-digital pathology with applications across the spectrum of biomedical science.

Materials and Methods

Sample Preparation.

The use of tissue for this study was approved by the University of Illinois Institutional Review Board via project 06684.

A paraffin-embedded serial breast tissue microarray (BR1003a; US Biomax Inc.) consisting of a total of 101 cores of nominal 1 mm diameter from 47 cases was obtained. One section was stained with H&E and imaged with a light microscope. A 5-μm-thick adjacent unstained section of the TMA was placed on a BaF₂ salt plate. We note that we chose IR transparent BaF₂ salt plates as substrate to image the same sample with both IR-OH and transmission FT-IR microscopy. As IR-OH can be based entirely on a backscattering geometry, imaging of samples placed on IR opaque substrates like glass or in aqueous media is in principle possible, which would constitute a major advantage for pathology proposes as it would remove the need for special IR transparent substrates and could allow live cell imaging and imaging of nonfixated tissue. The unstained section was deparaffinized using a 16-h hexane bath. Next, a surgical biopsy was obtained on a BaF₂ salt plate with a corresponding H&E-stained adjacent section on a glass slide. These are large tissue sections of about 20 × 20 mm² obtained from a single patient to obtain a comprehensive diagnostic profile. These are commonly used in clinical decision making, and hence, the models developed on the TMA were extended as stainless staining on the biopsy data.

A USAF 1951 optical resolution test target was fabricated in house. The target was patterned with SU-8 2005 polymer (MicroChem) on a polished 1-in.-diameter barium fluoride (BaF₂) substrate (Spectral Systems). The substrate was first cleaned with acetone and isopropyl alcohol (IPA) and then rinsed with distilled water. A 5-μm-thick layer of SU-8 polymer was spun coat on BaF₂ at 500 rpm for 5 s then ramped to 3,000 rpm for 60 s at a ramp rate of 1,500 rpm/s. The sample was then soft baked at 65 °C for 1 min and then heated to 95 °C at a ramp rate of 300 °C/min and held for 2 min. A lithography mask was contact aligned, and SU-8 was exposed to i-line UV radiation (365 nm) at 9 mW/cm² for 5 min with an optical density filter (PN NE206B; Thorlabs) to filter out deep UV. The exposed sample was heated with the same parameters as the soft bake and then developed in SU-8 developer for 4 min and rinsed with IPA.

IR-OH Setup.

The setup of our IR-OH microscope is shown in Fig. 1C. The sample is heated with a monochromatic IR laser beam (MIRcat; Daylight Solutions) that can be tuned to molecular vibration modes in the sample. The IR beam is intensity-modulated with an optical chopper at frequency $\mathrm{\Omega }=600\hspace{0.17em}\text{Hz}$ (MC2000B; Thorlabs) and directed toward the sample for illumination from the side using a combination of a concave and parabolic mirror to shape the IR beam. To probe the sample deformation induced by IR absorption, we apply stroboscopic wide-field interference microscopy. In detail, the sample is illuminated with light from a narrow-band LED emitting at a nominal ${\lambda }_0=660\hspace{0.17em}\text{nm}$ and 20 nm spectral bandwidth (M660L4; Thorlabs). The LED is strobed in synchronization with the beginning of each camera frame for a time of ∼500 μs. The reflected light $E_S$ from the sample is collected with a Mirau interference objective (50× CF IC Epi Plan DI; Nikon) where $E_S$ is interfered with a reference field ${\text{E}}_{\text{R}}$ for phase detection. To scan the delay $\text{τ}$ between sample and reference field, the sample can be translated vertically (z) by a linear piezo stage (P-611.3; Physik Instrumente). The range of vertical translation was chosen to be 4.5 and 7 μm for the USAF test target (Fig. 1) and the tissue samples (Figs. 2–4), respectively. The slight increase in travel with tissue was needed to keep the uneven surface of the tissue sections in focus and within the coherence length of the LED across the entire sample. While the sample is translated at constant velocity $z\left(t\right)=vt$, a sequence of images is registered with a monochromatic camera at high frame rate $F=500\hspace{0.17em}\text{Hz}$, yielding interferograms $I_{xy}\left(z\right)$ at each pixel (x, y). We optimize signal-to-noise ratio (SNR) by employing a million-level electron full well complementary metal-oxide semiconductor (CMOS) camera for image registration (Q-2A750; Adimec), a recently developed camera technology that significantly reduces shot-noise contributions in bright imaging conditions. The image sequence is continuously streamed to computer memory at 2 GByte/s. To reduce data volume and extract the IR absorption signal, signal demodulation is implemented in C++ for real-time processing. Demodulation is performed at frequencies S and Ω − S by 1) applying a Gaussian window to each interferogram and 2) subsequent discrete Fourier transform. The resulting datasets are saved to disk. To enable extraction of 3D information based on interferogram analysis, a decimated version of the full data was saved to disk as well, keeping every 20th frame. Note that since in our case, Ω > F, the vibration signal is detected at the low-frequency alias $\mathrm{\Omega }\prime =100\hspace{0.17em}\text{Hz}$ of the pulse rate Ω (remainder of division $\mathrm{\Omega }/F$). For clarity we do not distinguish between $\mathrm{\Omega }$ and $\mathrm{\Omega }\prime$ in the main text. The IR absorption image is obtained by dividing Ω − S by S datasets, as described by SI Appendix, Eq. S19. The optical image is obtained by demodulation at frequency S.

IR-OH Imaging.

IR-OH imaging of the USAF test sample in Fig. 2 was performed by acquiring a single field of view of groups 6 and 7 and part of group 5. A set of frames was acquired at a total of 396 frequency points and 2 cm⁻¹ spacing between the spectral range of 910 to 1,700 cm⁻¹. To remove contributions of water lines in our unoptimized system, a 2-pt Gauss filter was applied in the spectral domain. IR-OH imaging of the BR1003a breast tissue microarray (TMA) was performed by acquiring 2,917 tiles measuring ∼460 × 460 μm² each and with 12% overlap. At each sample location, 22 IR frequencies were consecutively acquired before moving to the next sample location. The 22 IR frequency points were selected according to a previously developed FT-IR classification model for breast tissue, where IR frequencies were manually determined (78, 79). The chosen IR frequencies were 1,038, 1,086, 1,142, 1,170, 1,214, 1,254, 1,282, 1,302, 1,338, 1,362, 1,386, 1,406, 1,426, 1,450, 1,486, 1,534, 1,550, 1,590, 1,630, 1,662, 1,690, and 1,718 cm⁻¹. Inhomogeneous sample illumination of the QCL beam was corrected in postprocess by 1) calculating an average image over all tiles, 2) applying a 2D polynomial fit, and 3) multiplying the inverse of the result to each tile. Stitching with in-house developed software yielded a set of 22 IR absorption images covering an area of 19 × 22 mm², which covered the entire TMA. For classification purposes, data were binned by 4 × 4 pixels to improve SNR and to reduce the dataset to a more manageable size, yielding 295 Megapixel × 22 frequencies with a pixel size of 1.2 μm. Data were then normalized to the Amide II band. No further spectral postprocessing was applied, e.g., baselining or noise reduction was not applied. Similarly, the breast biopsy was imaged by acquiring 2,808 tiles and at 25 IR frequencies, yielding 285 Megapixel image × 25 frequencies of an area of ∼20 × 20 mm². In both cases, the integration time was 8 s per tile and frequency, and the total image time was 10 s including overhead for data processing and storage. Image time of the entire sample was ∼7 d for all 22 frequencies (equivalent to 1.7 h per core). Potential for significant speed-up exists by improving the vertical sensitivity of the interferometer (SI Appendix, Note 4). For spectral comparison with FT-IR, we acquired a representative selection of 17 tiles at 221 IR frequencies with IR-OH covering the range from 910 cm⁻¹ to 1,790 cm⁻¹ in steps of 4 cm⁻¹. To remove contributions of water lines and to improve SNR, a 1-pt Gauss filter was applied in the spectral domain. Class spectra were averaged from regions labeled in classification.

FT-IR Imaging.

HD FT-IR imaging was performed on a 680-IR spectrometer coupled to a 620-IR imaging microscope (Agilent Technologies) with a 0.62 NA objective and a liquid nitrogen-cooled MCT 128 × 128 focal plane array detector. Data were acquired over the 900 to 3,800 cm⁻¹ spectral range and averaged over 16 scans per pixel. Data were subsequently corrected against a background acquired in an empty space of the BaF₂ slide with 128 scans and Fourier transformed. The spectral resolution was 4 cm⁻¹ with a pixel size of 1.1 μm. Each core was imaged by acquiring several tiles measuring ∼140 × 140 μm each and subsequent stitching using in-house software. Data were further processed using minimum noise fraction for noise reduction in a commercial software package, Environment for Visualizing Images (ENVI, Harris Geospatial Solutions, Inc., Broomfield, CO).

Supervised Classification.

The IR absorption images obtained with IR-OH were manually labeled using correlation with the consecutive marked H&E-stained glass slide images under the supervision of a pathologist as ground truth for our analysis. The following four histological classes were used: epithelium, stroma, red blood cells, and other, where “other” summarized the remaining types (necrosis, mucin, and secretions). Epithelium was further discriminated into noncancerous and malignant subtypes. As a first step for supervised classification, a tissue mask based on the IR-OH signal of the Amide I band was applied to remove empty spaces, debris, and pixels exhibiting low IR-OH signal from further analysis. For each class, 10,000 training and 10,000 validation pixels were randomly selected from the marked regions, and subsequently, IR-OH absorption spectra were extracted and normalized to the Amide II band. These spectral signatures were then used to build a random forest classifier. Notably, metrics were not defined—as typically done in the classification of FT-IR data—but rather the normalized spectral data were used as direct input for the classifier. A total of five predictors were used (1,086, 1,302, 1,338, 1,550 [reference band for normalization], and 1,630 cm⁻¹) to build a 60-tree ensemble for the four-class tissue segmentation. A total of seven predictors were used (1,086, 1,302, 1,338, 1,550 [reference band for normalization], 1,630, 1,662, and 1,718 cm⁻¹) to build a 60-tree ensemble for five-class tissue segmentations. These frequency bands were selected in a feature selection procedure (see below). The four-class classifier was based on the main cell types comprising epithelium, stroma, red blood cells, and other. A five-class classifier further discriminated between noncancerous and malignant epithelial subtypes. To assess the performance of the classifiers, we generated binary ROC curves and calculated the AUC metric for each class. AUC values were estimated for each class for three separate classification runs. In each of these runs a random subset of the pixels were sampled for training. This was done to ensure that the all subsets of the sampled pixels are representative of the data population. Mean AUCs are reported for each class. The breast tissue microarray shown in Figs. 3 and 4 was classified at a pixel size of 1.2 μm (4 × 4 pixel binning). The breast biopsy shown in Fig. 5 was classified at a pixel size of 2.4 μm (8 × 8 pixel binning) to further improve SNR and to obtain a more accurate classification.

Feature Selection.

We applied feature selection based on an iterative search algorithm to 1) reduce the number of IR frequencies needed for accurate classification and thus speed up future IR-OH imaging and 2) learn how well IR-OH data classify with only a few IR frequencies. Briefly, we developed an algorithm that tested classifier performance (AUC) for all possible band combinations, starting with n = 2 frequency points and using 1,550 cm⁻¹ as the single reference band to which all other bands were normalized. The four best frequency combinations where kept, using a cutoff of the 80 combinations with highest AUC scores, and the search algorithm was repeated for n + 1 bands. This search algorithm worked fully automatically. The found features were (most relevant first) 1,550 (reference band for normalization), 1,302, 1,630, 1,086, 1,338, 1,662, 1,718, 1,690, 1,038, 1,406, 1,486, 1,450, 1,534, 1,590, 1,282, 1,386, 1,254, 1,362, 1,214, 1,142, 1,426, and 1,170 cm⁻¹. Classifier accuracy (AUC) was evaluated using the top n features in Fig. 3F with the breast TMA. Prior to segmentation of the biopsy, we repeated the feature selection by training and validation on the TMA dataset shown in Fig. 3 and testing classifier performance with a second dataset of the same TMA to provide feedback (SI Appendix, Fig. S10). This was done to assure robustness in performance and improvements in the developing instrument, particularly because the biopsy and TMA datasets were collected at different time points during instrument development. Classifier performance was found to saturate for the top nine features: 1,550 (manually selected reference band), 1,630, 1,302, 1,338, 1,662, 1,486, 1,038, 1,690, and 1,406 cm⁻¹. These features are contained in the top 11 features of the first feature search, which confirms feature consistency. Note that we manually substituted feature 1086 by feature 1038, which captures similar tissue chemistry, because the former yielded poor segmentation of the biopsy. We attribute this to the low reproducibility when imaging at the 1086 laser line for reasons yet to be determined but which could involve power and pointing instabilities of the laser.

Computational H&E Images.

We first assigned a specific color to each cell type that represented the corresponding color obtained from the H&E stained images and subsequently generated a color image from the four-class map. To add morphology information, we combined it pixel by pixel with the optical image using a pan-sharpening approach. Briefly, we transformed the color image to a single intensity band that was matched to the histogram of the optical image. Then, we applied a linear combination of the color image with the optical image and so obtained the computational H&E image. Details about this pan-sharpening approach will be published elsewhere. All data and software developed can be requested from the corresponding author by email.