Development of a multimodal foveated endomicroscope for the detection of oral cancer

Abstract

A multimodal endomicroscope was developed for cancer detection that combines hyperspectral and confocal imaging through a single foveated objective and a vibrating optical fiber bundle. Standard clinical examination has a limited ability to identify early stage oral cancer. Optical detection methods are typically restricted by either achievable resolution or a small field-of-view. By combining high resolution and widefield spectral imaging into a single probe, a device was developed that provides spectral and spatial information over a 5 mm field to locate suspicious lesions that can then be inspected in high resolution mode. The device was evaluated on ex vivo biopsies of human oral tumors.

1. Introduction

Early detection of oral squamous cell carcinoma (SCC), the most common type of head and neck cancer, is critical to improve patient outcomes. It is estimated that there will be over 48,000 new cases of oral cancer in the United States, with nearly 10,000 deaths predicted in 2016 alone [1]. Although the oral cavity is simple to access and inspect for frequent screening, cancers are often discovered at a late stage when treatment options are more invasive and less effective. While the five year survival rate of localized oral cancers is around 80%, survival rates drop below 40% for oral cancers diagnosed at a more advanced stage [1]. Although oral examination should be trivial due to the ease of access of the orifice, around 65% of oral cancers are detected at a later stage, when treatment options become more limited [2]. Early detection of oral cancer is facilitated by clinical oral examination (COE), where a trained physician performs visual inspection and palpation of the head and neck regions to evaluate the oral mucosa for abnormal changes such as leukoplakia and erythroplakia; biopsy is required to confirm the presence of disease [2,3]. Pathologic assessment of biopsies allow for precise identification of the presence and grade of neoplastic disease including mild, moderate, or severe dysplasia or the presence of invasive carcinoma [4]. The recognition of oral carcinoma is highly dependent on the skill and experience of the clinician [2]. In an effort to improve upon the sensitivity and specificity of COE, several imaging techniques have been developed to improve the contrast between early neoplastic lesions and surrounding normal or benign tissue, with the goal of improving early detection while reducing the time and cost to obtain unnecessary biopsies.

One particularly simple and well-implemented optical technique is widefield autofluorescence imaging. Normal oral mucosa emits blue-green fluorescence (primarily due to stromal collagen) when excited by UV or blue light [5]. Neoplasia is associated with a loss of blue-green autofluorescence and increased red fluorescence [6] attributed to degradation of fluorescent crosslinks in stromal collagen, an increase in vascularity, and an increase in epithelial scattering [7–9]. Ideally, the loss of fluorescence can be used to quickly identify neoplastic regions and tumor margins. Unfortunately, stromal inflammation and other benign lesions can also exhibit similar loss of fluorescence [5,9,10]. While several commercial devices have been successfully developed to enhance observation of these autofluorescence changes with high sensitivity, the inability to consistently differentiate between benign and neoplastic lesions has led to poor adoption [11–15].

In an effort to overcome the limitations of COE and autofluorescence imaging, several modalities have been explored including nonlinear microscopy [16,17], Raman spectroscopy [18,19], fluorescence lifetime imaging (FLIM) [20,21], optical coherence tomography [22,23], spectroscopy with point spectrometers [24,25], multispectral imaging [26,27] and confocal microscopy [20,28]. While each of these methods delivers valuable information to aid in the diagnosis of oral cancer, studies have suggested that a combination of these techniques could improve performance [5,7,21,27]. For example, a pilot study with multispectral imaging demonstrated the advantages of combining spectral and spatial data acquisition over the entire oral cavity [27]. While several point spectroscopy techniques have shown high sensitivity and specificity, the sparse spatial sampling of fiber optic probes makes it challenging to interrogate the entire surface of the oral cavity [24,25]. Multispectral imaging improves upon point spectroscopy by providing both spatial and spectral data. However, these systems are still limited because they lack the spatial resolution necessary to identify the cellular changes associated with the presence of neoplasia [26,27]. Confocal microscopy, a technique employing a spatial filter to improve spatial resolution by providing optical sectioning allows for visualization of the cellular changes associated with dysplasia [28]. Similar to point spectroscopy, however, confocal devices are usually limited by a small field of view. An ideal system would combine widefield multispectral acquisition capabilities with a high resolution imaging technique, allowing for rapid localization of abnormal lesions followed by high resolution imaging to determine the presence and degree of neoplasia.

In this work we present a probe that combines a high resolution confocal microscope with widefield multispectral imaging through a single foveated endomicroscope. We previously validated the miniature foveated objective by imaging ex vivo mouse colon tissue with a confocal microscope [29]. We have now combined the foveated objective with a vibrating fiber bundle to relay the image from the objective to both a confocal microscope and an image mapping spectrometer (IMS). The IMS is a snapshot spectrometer that is capable of acquiring spatial and spectral information at high speed. We previously validated the IMS on several tissue models including an in vivo pilot study of normal and cancerous oral tissue [27,30,31]. With the combined system, we aim to improve specificity and sensitivity over either of the single modalities. In this study, we imaged human ex vivo oral samples, providing spectral data to demonstrate the differences between normal and tumor tissue along with high resolution imaging in order to image the cellular architecture of the samples. While the oral cavity is accessible enough to allow for larger probes than this system to be introduced, the advantage of this concept is the co-registration of the two modalities, which could eventually prove valuable in smaller, more difficult to reach cavities. The ultimate device we envision will utilize the proof of concept presented in this manuscript, consisting of a fully enclosed system that will combine the imaging probe with a tool channel for dispensing a fluorescent dye and an illumination source. To the best of our knowledge, this is the first reported endomicroscope that combines both multispectral fluorescence and high resolution confocal fluorescence modalities through a single miniature objective.

2. Foveated objective

The complete design, fabrication and performance of the foveated miniature objective were reported previously [29]. Briefly, the intentional introduction of distortion into the optical design of the objective provides variability in the magnification and numerical aperture across the field. With the highest magnification found in the center of the field and demagnification towards the edge, it is possible to provide a larger field of view than allowable by the Lagrange invariant while maintaining the ability to resolve fine features near the optical axis. The lens performs nominally similar to the human eye, in which peripheral vision allows for rapid localization of features of interest to then be closer inspected by the fovea of the eye – providing higher resolution and more detail. The specific parameters of the miniature objective can be found in Table 1, highlighted by the 2.7 mm in diameter objective providing a 5 mm field of view with an NA of 0.1 on axis.

Table 1. Basic optical parameters of the Foveated Objective.

Field of View	5 mm
Diameter of Objective	2.7 mm
Length of Objective	10 mm
Image size	1.44 mm
NA / Magnification on Axis	0.1 / 0.39x
NA / Magnification at edge of field	0.061 / 0.06x
Design wavelength	510 nm

The design consisted of three optical lenses – two PMMA and one Zinc Sulfide lens - to provide the majority of the optical power. The prototype lenses were built using single point diamond turning on a miniature diamond lathe, utilizing a hypodermic tube as the chassis of the objective. The performance of the objective was observed by imaging a 1951 United States Air Force resolution target across the entire field by relaying the image from the foveated objective to a commercial Zeiss Primostar microscope. The final validation of the objective was performed by combining it with a confocal microscope for imaging ex vivo mouse colon with colorectal lesions dyed with proflavine [29]. From these images it was possible to observe the presence of dysplastic tissue and demonstrate that morphology compared well to that observed in histology images. With the foveated objective fully validated, the objective required an image relay to the imaging devices that would allow the endomicroscope to be used in the body.

3. Vibrating fiber bundle integration

In order for the objective to be clinically relevant, an image guide is required to relay the image from the objective near the tissue to the imaging devices located away from the imaging target. For the foveated objective, which provides an image size of nearly 1.5 mm, relaying the complete image without losses is not trivial. Piezoelectric scanners for example do not provide a large enough field of view with high enough refresh rates required for in vivo imaging. A larger fiber bundle provides an adequate field of view, but the size, pixilation due to individual fibers and crosstalk between fibers limits achievable resolution due to undersampling. In an effort to improve sampling we present a technique of rapidly vibrating a fiber bundle to acquire data that would normally be inaccessible due to the limitations of fiber bundle pattern.

The vibrating fiber bundle setup consists of a miniature electric vibrating motor attached to a 60k element, 1.2 mm diameter fiber bundle with a core to core spacing of 4.5 µm (Myriad Fiber, Dudley, MA). The vibrating motor (273-0107, Radio Shack, Fort Worth, TV) was connected to a voltage driver supplying up to a 3 V signal to provide 16,000 RPM. The motor, which is attached to the middle of the fiber, rapidly vibrates the fiber with a maximum displacement of approximately ± 50 μm on each end. Initial setup of the vibrating system required the adjustment of the clamping pressure of the motor to the fiber bundle and the fine tuning of the applied voltage. Once these parameters were set, the system maintained alignment. The optical train of the testing system is as follows: Light from a halogen lamp passes through a 710 nm filter onto a 1951 USAF negative resolution target. To protect the surfaces of the fiber bundle and the resolution target, the image of the target is relayed by two 4x, 0.1NA Olympus objectives placed back to back (providing 1x magnification) to the fiber bundle proximal surface. A CCD camera with a 10x, 0.25NA finite correct objective and a tube lens collects the image from the distal end of vibrating fiber bundle. Images were collected with voltage applied to the vibrating motor in order to assess the improvement in resolution.

The image below (Fig. 1) displays the resolution target imaged through the fiber bundle when voltage is not applied to the motor (A), compared to when voltage is applied and the fiber vibrates (B), and without the fiber bundle (C) over the same exposure time. As the spatial sampling increases, there is a clear improvement in the resolvability of features when comparing the smallest resolvable features for the static fiber bundle (group 6, element 4 equating to 11.04 µm per line pair) to the resolvable features for the vibrating fiber (group 7, element 4 or 5.52 µm per line pair). The limit of resolution for this setup is defined by the 0.1 NA objectives and the chosen wavelength (710nm), with an ability to resolve features from group 7, element 5 (4.33 µm per line pair) according to the Rayleigh criterion for the diffraction limit. Thus the system provides nearly diffraction-limited performance when the fiber bundle is vibrating, well above the limit of a static fiber bundle, and is appropriate for imaging with the foveated objective.

Fig. 1.

Image of 1951 USAF neagative resolution target taken through a static fiber bundle (A). Image of the same target taken with a vibrating fiber bundle in order to increase the spatial sampling, providing an improvement in acievable resolution (B). Image taken with fiber bundle removed, showing the limit of resolution of the system without the fiber bundle (C).

In addition to reducing the visual impact of the fiber bundle pattern, the vibrating method minimizes the effects of debris and damage found on the fiber bundle surfaces. This is illustrated in the full field images of the resolution target taken through the foveated objective (Fig. 2). Figure 2 compares images of the resolution target when the fiber bundle is stationary (A) and when vibrating (B), demonstrating the improvement in achievable resolution and reduction of defects to the fiber bundle.

Fig. 2.

Full field image of 1951 USAF neagative resolution target taken through a static fiber bundle and the foveated objective, with red box displaying the higher resolution features and green circles indicating defects of the fiber bundle (A). Image of the same target taken with the vibrating fiber bundle and foveated obejctive in order to increase the spatial sampling, providing an improvement in acievable resolution and overall smoothing of image defects from the fiber bundle. The red box displays the higher resolution features while the green circles illustrate the reduction in the effects of the defects of the fiber bundle (B).

4. Confocal and IMS integration with foveated objective

With sufficient improvement in spatial resolution, the vibrating fiber can be used to integrate the foveated objective to the confocal microscope and the Image Mapping Spectrometer, both of which have been previously evaluated in several papers. Figure 3 shows a schematic of the entire foveated endomicroscope system with the miniature objective, the vibrating fiber bundle, the two imaging devices and a computer to display the images in real time.

Fig. 3.

Schematic of the completely integrated device with foveated objective on the left combined with the vibrating fiber bundle. A beam splitter then splits the signal to the confocal microscope and IMS. The images are then displayed on a laptop monitor.

The next step of integration required a mount to set the distance between the objective and the fiber bundle. The proximal end of the objective integratied with the distal end of the fiber bundle was accomplished with the 3D printing of a miniature mount to attach the foveated objective to the fiber bundle. The mount, designed in Solidworks, increases the outer diameter of the probe by 500 µm and relies on friction to hold both the objective and the fiber bundle. Three schematics of the mount and an image of the foveated objective in the mount are shown in Fig. 4.

Fig. 4.

Schematic view of the silver foveated objective, the gray 3D printed mount, and the black fiber bundle (A). Cross section view, displaying the lenses within the foveated objective and the spacing of the objective to the fiber bundle (B). Photograph of the foveated objective held in the mount attached to the fiber bundle (C). Isometric view of the 3D printed holding mount (D).

The custom confocal microscope was reported previously [29]. Briefly, the microscope incorporates a 455nm handheld laser pointer, commercial optics, miniature galvanometer mirrors, a spatial filter, and a photomultiplier tube to scan and collect a diffraction limited point across the region of interest. The spatial filter improves axial resolution over standard bright-field imaging by blocking out-of-focus light in the conjugate image plane. A custom LABVIEW program drives the mirrors and acquires the images with the ability to control scanning speed and amplitude. The beam is focused and scanned on the distal end of the optical fiber to collect images. Figure 5 displays an image of lens paper marked with fluorescent yellow highlighter and imaged through the fiber bundle alone (A) and through the foveated objective with the fiber bundle (B) coupled to the confocal microscope. Due to the thickness of the resolution target, confocal images of the resolution target could not be acquired. The addition of the foveated objective increases the field of view from 900 µm to 1.5 mm diameter, as indicated in Fig. 5.

Fig. 5.

Comparison of images of fluorescent lens paper imaged by the confocal microscope through a fiber bundle. Images acquired through the bare fiber bundle (A) and through the foveated objective coupled to the confocal microscope (B) show an observable increase in field of view

In addition to the high resolution images provided by the confocal microscope, broad surveillance of the tissue is accomplished by multispectral imaging with the IMS. The IMS has been described previously [32,33] but briefly is a snapshot hyperspectral imaging device, which acquires spectral and spatial data simultaneously. The system consists of customized optical parts (mirror, lenses and prisms), a bandpass spectral filter (470–670 nm), and a CCD camera incorporated into a custom built metal casing. Each image collects greater than 100,000 spectra across the field - this information can then be used to aid in the localization of suspicious lesions. In this setup, the IMS uses external 500 mW white (425–700 nm) and 670 mW blue (385–425 nm) LEDs to illuminate the tissue directly. As with the confocal microscope, the distal end of the vibrating optical fiber is imaged by the IMS to collect the widefield images in addition to the spectral information. An image of the USAF resolution target through the foveated objective is shown in Fig. 6, along with pseudocolored images at 8 different wavelengths.

Fig. 6.

Full field image of resolution target imaged through the foveated objective by the IMS with all wavelengths represented, where the scale bar is equivalent to 500 μm (A). Images of the resolution target at eight different wavelengths, pseudocolored to match the represented wavelength, where the scale bars are equivalent to 500 μm (B).

5. Imaging of ex vivo oral tissue

To assess the potential of the device to image normal and neoplastic oral tissue, fresh biopsies of clinically normal and abnormal oral tissue were imaged with the system immediately following removal. Patients gave informed consent and the study was reviewed and approved by the IRBs at Rice University and the MD Anderson Cancer Center. Eligible patients were those scheduled for surgery to resect abnormal oral cavity tissue. Punch biopsies from the lesions and clinically normal sites (when available) were obtained and imaged using the foveated endomicroscope. In this study, biospsies were obtained from six patients with invasive carcinoma found in the oral cavity. All of the biopsies from the tumors were classified between G2 and G4 according to the pathology reports, where G refers to the grade of the tumor. A G2 grade of a tumor in the oral cavity indicates moderately differentiated cells, while G3 and G4 indicate a very abnormal appearance where the cells are poorly differentiated. Utilizing this metric, clinicians can quantify the severity of the cancer.

First, widefield multispectral images were acquired as the samples were illuminated by white and then blue LEDs. The biospsies were then stained by proflavine, a fluorescent dye that stains cell nuclei was used as a contrast agent for the confocal imaging. A solution containing 0.05% w/v solution of proflavine (Sigma Aldrich, St. Louis, Missouri) in 1X PBS was applied topically, the tissue was rinsed with additional PBS, and the biopsies were imaged by the high-resolution confocal mode. For testing, the device was secured to an optical table, which allowed for external staining of the biopsied sample without moving the probe. Figure 7 displays a series of images and the corresponding spectra acquired by the foveated endomicroscope of a clinically abnormal biopsy obtained from the maxilla in a patient scheduled for total maxillectomy. The pathology report indicated an invasive, moderately to poorly differentiated squamous carcinoma.

Fig. 7.

Images acquired by the endomicroscope of a biopsy showing invasive oral SCC in widefield spectral mode when illuminated by the white and blue LEDs, where the scale bars are equivalent to 500 μm. Beneath the images are plots of the associated spectra of these two images, which were found by taking the average of the measured spectra of the an area 1 mm in diameter at the center of the fields, (indicated by an orange circle). The vertical axis of the chart is intensity (arbitrary units) and the horizontal axis is wavelengths (from 460 nm to 660 nm) (A). Reflectance images of the biopsies at eight different wavelengths, (pseudocolored to match the represented wavelengths) showing enhanced viszualitzion of vessels at 548 m, and 575 nm (areas identified by the red circles), where hemoglobin absorption is maximal - the scale bars are equivalent to 500 μm. (B).

The widefield spectral images shown in Figs. 7(A) and 7(B) show the macroscopic structure of the biopsy, while the blue LED allows for observation of the loss of fluorescence associated with the lesion before proflavine was applied. Additionally, as shown in Fig. 7(B), it is possible to observe vascularization at different depths (red circles) corresponding to the wavelength penetration depths (27), which could potentially prove valuable for in vivo samples to distinguish changes in vascularization due to abnormalities. The average spectra across the center of the field (Fig. 7(A)) can be used for further analysis.

An additional metric for spectral analysis consists of comparing the intensity of different color channels or at specific wavelengths when the reflectance spectrum is acquired, in order to determine relative changes in the spectral response. It has been reported that by analyzing the ratio of diffuse reflectance intensity in the oral mucosa at 545 nm and 575 nm, it is possible to distinguish between healthy and cancerous oral lesions due to changes in the spectra at these wavelengths, which are associated with oxygenated hemoglobin in the tissue [34–37]. While a standard RGB camera is unable to differentiate between these two wavelengths, the IMS allows for a clear distinction. Analysis of the average 545/575 ratio as measured by the device was performed for each biopsy site (Fig. 8). There is a clear trend between histologically normal and abnormal biopsies from similar anatomic sites; the ratio is lower for normal samples compared to the tumors. The ratios are not uniform between sites, most likely due to the different sources of the biopsies in the oral cavity (tongue, maxilla, roof of the mouth, buccal, and mandible). For one patient, two biopsied samples from the tumor were examined, while for another patient only the tumor was examined and no normal samples were collected, as reflected in Fig. 8.

Fig. 8.

Scatter plot of the reflectance signal intensity ratio of 545/575 nm from different biopsy sites (both normal and from cancerous oral lesions). The error bars represent the standard deviation of the average ratios, which were measured in approximately twenty points on each biopsy.

Using the widefield mode with spectral data to locate the suspicious regions of the tissue, it is then possible to acquire confocal images at the center of the field of view in order to observe morphological changes associated with the development of neoplasia. Biopsies of buccal tissue acquired from both a clinically normal and abnormal region of tissue from the same patient were imaged by the device in widefield mode (Fig. 9(A) and 9(C)), stained with proflavine and then imaged in confocal mode at the center of the field (Fig. 9(B) and 9(D)). The confocal images acquired of these samples (Fig. 9(B) and 9(D)), which were pseudocolored green to match the proflavine dye, display the differences between the negative for tumor and tumor (squamous carcinoma) biopsies as defined in the pathology report. Specially, the negative for tumor biopsy was defined as a fragment of tan-yellow fibroadipose tissue from the left buccal, while the other biopsy from the right buccal consisted of squamous carcinoma that was moderately differentiated. The confocal image of the biopsy of negative-for-tumor tissue shown in Fig. 9(B) shows round nuclei of uniform size and shape throughout the field of view. In contrast, the confocal image of the abnormal biopsy shown in Fig. 9(D) shows extracellular matrix and cellular debris; this is consistent with the pathologic diagnosis of moderately differentiated squamous cell carcinoma.

Fig. 9.

Widefield and confocal mode images of two ex vivo biopsy samples acquired by the system (A) A widefield image of a negative for tumor sample from the buccal site. (B) confocal image of the same sample treated with proflavine with a red arrow indicating individual cell nuclei. (C) Widefield image of biopsy taken from a buccal tumor. (D) Confocal image of the biopsy stained with proflavine, illustrating irregular formations, suggesting neoplasm.

5. Conclusions

A multimodal endomicroscope was developed for the rapid localization and identification of premalignant and cancerous lesions. This is made possible by a miniature foveated objective providing a combination of large field of view with high resolving power on axis. An image relay technique to improve spatial resolution was developed for the system by vibrating a fiber bundle to improve the spatial sampling, reducing the detrimental visual effects of the fiber bundle pattern as well as under-sampling. The objective and fiber bundle were integrated with a custom confocal microscope and a custom snapshot imaging spectrometer with a 3D printed holder. The device was validated by imaging a 1951 USAF resolution target and ex vivo samples of normal and cancerous oral tissue. The successful combination of widefield imaging with spectral analysis and high resolution imaging for this pilot study suggests this device should be further investigated in clinical studies. Future work will involve the testing of larger samples containing normal and abnormal structures in order to assess the ability to rapidly differentiate tissues and aid in the demarcation of tumor margins. The probe will require further rugidization/packaging for clinical implementation of the device, including the introduction of an ergonomic mount that will not affect the vibrating fiber bundle. Additionally, the optical probe will be integrated with a tool channel that will allow for dispensing of a fluorescent dye and an external illumination source. Eventual in vivo studies with additional spectral analysis will further validate the device, as active vascularization and effects of palpation during the oral examination can be observed and assessed for cancer detecting biomarkers.