Real-time automated thickness measurement of the in vivo human tympanic membrane using optical coherence tomography

Abstract

Background

Otitis media (OM), an infection in the middle ear, is extremely common in the pediatric population. Current gold-standard methods for diagnosis include otoscopy for visualizing the surface features of the tympanic membrane (TM) and making qualitative assessments to determine middle ear content. OM typically presents as an acute infection, but can progress to chronic OM, and after numerous infections and antibiotic treatments over the course of many months, this disease is often treated by surgically inserting small tubes in the TM to relieve pressure, enable drainage, and provide aeration to the middle ear. Diagnosis and monitoring of OM is critical for successful management, but remains largely qualitative.

Methods

We have developed an optical coherence tomography (OCT) system for high-resolution, depth-resolved, cross-sectional imaging of the TM and middle ear content, and for the quantitative assessment of in vivo TM thickness including the presence or absence of a middle ear biofilm. A novel algorithm was developed and demonstrated for automatic, real-time, and accurate measurement of TM thickness to aid in the diagnosis and monitoring of OM and other middle ear conditions. The segmentation algorithm applies a Hough transform to the OCT image data to determine the boundaries of the TM to calculate thickness.

Results

The use of OCT and this segmentation algorithm is demonstrated first on layered phantoms and then during real-time acquisition of in vivo OCT from humans. For the layered phantoms, measured thicknesses varied by approximately 5 µm over time in the presence of large axial and rotational motion. In vivo data also demonstrated differences in thicknesses both spatially on a single TM, and across normal, acute, and chronic OM cases.

Conclusions

Real-time segmentation and thickness measurements of image data from both healthy subjects and those with acute and chronic OM demonstrate the use of OCT and this algorithm as a robust, quantitative, and accurate method for use during real-time in vivo human imaging.

Introduction

Otitis media (OM) is a middle ear infection that is extremely common and most prevalent in children under the age of three (1). This disease initially presents as an acute infection, but can progress to chronic OM. Effusions, or fluid within the middle ear cavity, may also be present. The current gold standard for diagnosis of OM is visual inspection via an otoscope to examine the surface of the tympanic membrane (TM) (2,3). Previously, our lab developed an optical coherence tomography (OCT) system with a handheld probe to noninvasively image the in vivo human TM in cross-section, as well as any structures behind the TM and within the middle ear (4,5). Depth-resolved OCT imaging has provided a unique opportunity to quantify the in vivo thickness of the TM, and correlate thickness with the different types of OM.

OCT is a noninvasive, high-resolution, real-time imaging modality that has been extensively used in ophthalmology (6-8) as well as in a large number of other medical and surgical specialties (9). In the specialties of otolaryngology (otology), pediatrics, and primary care, this technology can also be applied to improve the visualization of the TM and the structures behind it in the middle ear (10,11). While otoscopy is the gold standard for diagnosis of OM by visualizing the surface of the TM and assessing it qualitatively (2,3), this technique and others, such as acoustic reflectometry and tympanometry, do not enable the physician to visualize if a middle-ear biofilm is present behind the TM (2,12). A biofilm is a protective microenvironment containing one or more bacterial species that are prone to develop resistance toward antibiotics and re-seed infections in chronic OM.

To facilitate the use of OCT for diagnosing and quantifying OM, and for the implementation of OCT in the clinic, an automatic and real-time algorithm to quantify the thickness of the TM is desired. While several studies have imaged the ear with OCT, both ex vivo and in vivo (13-15), there are no published algorithms for segmentation and measurement of an in vivo TM using OCT. Notably, Nguyen et al. developed a classification algorithm to characterize TMs based on OCT depth scans (A-scans), however, the algorithm was not real-time and did not account for the angular orientation of the TM (16). Furthermore, a quantitative measurement of the TM thickness can have clinical importance, as there is an association with having a biofilm behind the TM and chronic OM (17-19). Additionally, it has been shown that the in vivo TM thickness in humans differs significantly between normal healthy subjects and those with acute and chronic OM (20).

For this study, we have optimized our existing portable OCT system and handheld probe for imaging the TM and the structures immediately behind it. We have developed a real-time segmentation algorithm which detects the TM and calculates its thickness while correcting for any tilt of the TM. Our algorithm utilizes a Hough transform to detect the boundaries of the TM. The Hough transform has been extensively utilized in image processing because it allows for detection of known shapes from images (21-25). Together, the OCT system with the handheld probe, and the real-time segmentation algorithm, will allow for immediate quantification of TM thickness, provide image and segmentation data for patient records, and enable a new way to screen for and visualize OM noninvasively within the human ear.

Materials and methods

The system used for this study was a portable spectral-domain (SD) OCT system shown in Figure 1. This system and the design of the handheld probe represents an advancement over our previous design (4). Figure 1A shows a photo of the portable imaging system. The system is contained within a custom fabricated cart (35 cm in depth, 62 cm in width, 150 cm in height) that enables reliable transport between and within clinical sites.

Figure 1.

Photos and schematic of an OCT imaging system with a handheld probe. (A) Photo of the portable cart with touch screen display; (B) photo of the handheld imaging probe compatible with otoscope ear specula; (C) schematic of OCT system with handheld imaging probe. OBJ, objective; DM, dichroic mirror; MEMS, micro-electro-mechanical-system mirror; C, collimator; CCD, charge-coupled device camera; PC, polarization controller; 50/50, two-by-two fiber coupler; NDF, neutral density filter; DC, dispersion compensation; L, lens; M, mirror; DG: diffraction grating; LSC, line-scan camera; OCT, optical coherence tomography.

Light is delivered and collected through a handheld imaging probe modeled after a traditional otoscope (Figure 1B). The familiar form factor facilitates ease of use for physicians and other health professionals. The probe is ergonomically inspired (3.8 cm in depth, 10 cm in width, 20 cm in height) and lightweight (approximately 250 grams). The near-infrared beam for OCT imaging is scanned across the surface of the tissue using a micro-electro-mechanical-system (MEMS)-based mirror element (Advanced MEMS, San Francisco, CA, USA). A high-resolution color charge-coupled device (CCD)-based camera is integrated alongside the OCT optical system to provide surface images of tissue simultaneously with the OCT cross-sectional images. Illumination for the surface images is provided by an integrated fiber bundle and high-power LED, which are both located within the nose cone of the probe.

The OCT imaging engine is composed of a Michelson-based interferometer with accompanying broadband light source and high-resolution spectrometer. A schematic of this system is shown in Figure 1C. A broadband light source (T-860HP, Superlum), centered at 860 nm, provides 12 mW of output power with a full-width-half-maximum (FWHM) bandwidth of 135 nm. The spectrometer unit (Wasatch Photonics, UT) has a resolution of 0.04 nm paired with a 12-bit high speed line-scan camera (spl4096-140km, Basler), capable of recording 70,000 lines per second. Axial and transverse resolution was measured to be approximately 2.4 and 15 µm in air, respectively. Video-rate processing and display is enabled by using a graphics processing unit (GPU)-based OCT processing algorithm, which enables up to 100 frames per second, depending on number of A-lines used. The system has a measured SNR of approximately 105 dB, when using a perfect reflector in the sample arm.

The data is displayed through a custom-designed graphical user interface (LabView). Users can control important imaging parameters and easily save both OCT and standard otoscopic surface image data. Data can be saved by depressing a trigger button on the probe handle or by touching a button on the touch screen monitor. Data is “pre-triggered” from a data queue, enabling the saving of the previous N number of frames prior to pressing the save button. This ensures that desirable data seen by the user on the screen in real-time will not be lost.

Once the signal was acquired, all processing was performed using LabView 2013 and CUDA v5.5. on a GPU (GTX 680, NVIDIA). An outline of the processing steps is shown in Figure 2. First, standard OCT processing was performed involving a λ-to-k linearization, followed by a Fourier transform, and gamma intensity correction. Once the cross-sectional OCT data was processed, a noise threshold was applied to aid in feature extraction. The algorithm calculated the average intensity of the image and only segmented the images that were above a predetermined average intensity. This reduced the processing time because only frames that had enough data for analysis were processed.

Figure 2.

Thickness measurement steps. Starting from the far left, standard OCT processing is performed on the raw data and an image quality metric is calculated. If sufficient signal is measured, the algorithm continues. A Hough transform is then performed to find the angle and thickness of the TM. To correct for the tilt of the TM, a right triangle was formed and the hypotenuse (T) measured the thickness of the TM. OCT, optical coherence tomography; TM, tympanic membrane; ROI, regions-of-interest.

Before proceeding to the Hough transform, the frames were subdivided into three predefined regions-of-interest (ROI), as shown in Figure 2. This enabled the algorithm to compensate for slight bends in the TM by fitting it with different segmentations in each ROI. The total region analyzed consisted of 2,048 pixels in depth, but only 1,000 pixels were used to reduce processing times and to ensure that the sample was near the focus of the probe.

Next, as shown in the center column of Figure 2, a Hough transform was applied to each ROI using (Eq. [1]):

S_H (m, b) = ∑ S(x, mx + b) [1]

Each pixel in the Hough space corresponds to the sum of intensities along a line with slope, m, and intercept, b. Therefore, the pixel in the Hough space with the highest intensity corresponds to a line in the frame with the highest cumulative intensity. This line was used to determine the slope, m_max, of the TM in each individual ROI. The boundaries of the TM were found by determining which lines of the same slope had one tenth the cumulative intensity of the maximum line in the Hough space, and were characterized by b₁ and b₂. For each ROI, the lines y = m_max x + b₁ and y = m_max x + b₂ were then superimposed in red and green onto the display.

The thickness within each ROI was calculated as the orthogonal distance between the two lines in each ROI. To convert m_max, b₁, and b₂ into a quantitative measurement, the equations shown in the right column of Figure 2 were used. These equations corrected for the refractive index of the TM, assumed to be 1.44 from Van der Jeught, et al. (26), voxel size, and the angle of the TM. The final thickness in each ROI was determined by forming a right triangle where the hypotenuse was the final thickness, T. In the equations, the system-dependent constants c_x and c_z converted from units of pixels to microns along the x and z dimensions, respectively. The median thickness of the ROIs was calculated and displayed to give the user a final readout. The median was used to reduce the effect of outliers in the measurements.

Results

The algorithm performed at a rate of 30 frames per second with a frame size of 1,000×1,000 pixels². To test the algorithm, a phantom consisting of layers of tape was imaged as it was rotated through different angles and translated vertically to test the stability of the tilt correction capabilities. Figure 3 shows the real-time display as the phantom was rotated (from –7 to 5 degrees) and Figure 4 shows the display as the phantom was translated vertically (over 650 µm). Figure 5 shows still frame images and the median thickness of the ROIs as a function of time for each dataset. The peak-to-peak variance was approximately 5 μm within both data sets. The small area of missing points in the median thicknesses during phantom rotation was a result of the phantom surface approaching a tilt of zero degrees. Near zero degrees, spurious normal incidence reflections from within the sample arm caused the algorithm to fail. These reflections were artifacts of a non-anti-reflection coated piece of glass and should be corrected in future handheld probe designs.

Figure 3.

Real-time OCT imaging and automated segmentation and thickness measurement of a tape phantom under user rotation and at various incident angles of imaging (27). The multi-layer tape phantom is used to as a model of the human TM. The numerical values across the bottom indicate the automated thickness measurement at different points across the image. The median thickness (in microns) is given in the upper right corner. OCT, optical coherence tomography; TM, tympanic membrane. Available online: http://www.asvide.com/articles/386

Figure 4.

Real-time OCT imaging and automated segmentation and thickness measurement of a tape phantom under user translation along the direction of imaging (28). The numerical values across the bottom indicate the automated thickness measurement at different points across the image. The median thickness (in microns) is given in the upper right corner. OCT, optical coherence tomography. Available online: http://www.asvide.com/articles/387

Figure 5.

Testing on a phantom sample. Shown on top are still shots from layers of tape at different angles along with the corresponding thickness measurements. Below, the median of all three values is plotted over time for two different datasets (Figures 3,4). The peak-to-peak variation over time for both datasets was approximately 5 µm.

Next, under an Institutional Review Board protocol approved by both the University of Illinois at Urbana-Champaign and Carle Foundation Hospital, Urbana, IL, the algorithm was tested on image data from both healthy human volunteers and volunteers with ear infections to evaluate how it performed during in vivo imaging sessions. Figure 6 and the top of Figure 7 show how the probe was used in vivo and in real-time for a normal patient to obtain the data shown in Figure 7. Figure 8 shows the onscreen display in real-time. The data from Figure 8 was acquired during the filming of Figure 6 via the finger trigger button on the handheld probe. The still shots in the middle of Figure 7 show the data before and after processing. On the right side, the three ROIs can be seen with the median thickness displayed at the base of each section. The bottom of Figure 7 shows the thickness of each ROI as a function of time. Two distinct regions in time can be seen: the time when the user was still positioning the probe to bring the TM into view, and after the TM was located within the field-of-view. These two periods can also be observed in the video (Figure 6). The median thicknesses of each ROI are shown after the positioning has been achieved. Within this field-of-view, the spatially-varying thickness of the TM can be observed. On the left, the TM thickness is measured to be approximately 60 μm, while on the right it is observed to be approximately 80 μm.

Figure 6.

In vivo OCT imaging of the human tympanic membrane using a portable OCT system with a handheld probe (29). The probe and system enables real-time acquisition and display of a CCD-based image of the surface of the TM, a cross-sectional OCT image, and the automated segmentation and thickness measurement of the TM. OCT, optical coherence tomography; TM, tympanic membrane; CCD, charge-coupled device camera. Available online: http://www.asvide.com/articles/388

Figure 7.

In vivo normal TM imaging. Shown on top is a still shot from Figure 6, imaging a healthy human volunteer. In the middle is an OCT cross section with thicknesses measured in real time for each ROI (Figure 8). The traditional surface image of the TM is shown as an inset. On the bottom is a plot over time of the measured thickness in each ROI. The varying thickness across the scan is highlighted with the calculated median using the second half of the data. TM, tympanic membrane; OCT, optical coherence tomography; ROI, regions-of-interest.

Figure 8.

Real-time OCT imaging and automated segmentation and thickness measurement of the in vivo human tympanic membrane (normal) (30). This data was collected during the imaging session shown in Figure 6. The numerical values across the bottom indicate the automated thickness measurement at different points across the image. The median thickness (in microns) is given in the upper right corner. OCT, optical coherence tomography. Available online: http://www.asvide.com/articles/389

In Figure 9, the algorithm was applied to still images from patients with either acute or chronic OM. These images from subjects with OM were obtained with a prior version of the handheld probe, which exhibited a lower signal-to-noise ratio (4). Therefore, for these images, one ROI was selected manually. The algorithm finds the boundary of the TM on one side and includes the biofilm in the calculation of the TM thickness. As expected, the TM thicknesses in the OM cases are thicker than the cases of a normal TM. Additionally, the thicknesses measured with this algorithm corresponded the ranges of TM thicknesses found in acute and chronic OM, as reported by Monroy et al. (20).

Figure 9.

In vivo acute and chronic OM imaging. Shown are in vivo OCT cross sections with the corresponding thickness measurements for one acute and two chronic OM cases. Inflammation in the acute case and the possible presence of biofilm in the chronic cases resulted in varying thickness measurements. OM, otitis media; OCT, optical coherence tomography; TM, tympanic membrane

Discussion

The primary goal of this project was to improve the technologies used by physicians and healthcare providers for caring for their patients with OM. It is expected that the thickness of the TM in normal healthy subjects and those with acute and chronic OM will vary (20). However, prior to the use of OCT, no technology has been able to measure the thickness of the in vivo human TM with micron resolution, and no algorithm has been previously developed to make this quantitative measurement in real-time, such as in a busy healthcare provider’s office. The rapid assessment of TM thickness has the potential to be a new diagnostic metric in OM, and for guiding treatment strategy. Because chronic OM has been highly associated with biofilms in the middle ear, and because bacteria within biofilms are known to be highly resistant to standard antibiotic regimens, the ability to noninvasively identify the presence of biofilms has the potential to be an important clinical biomarker for directing more effective antibiotic use and for determining the optimal time to recommend surgical intervention.

To address this goal in this study, we first improved the existing handheld probe and OCT system, as discussed previously. One example of the performance improvement can be seen by comparing the images in Figures 7,9. In Figure 9, the algorithm was applied to patient images obtained with an older version of the probe. With the current probe and system, and an improved signal-to-noise ratio, the images in Figure 7 could be automatically analyzed by the algorithm and did not require manual selection of the ROIs; hence, the algorithm was fully automated. Additionally, the newer probe and system design provided more convenient data acquisition because it saved pre-trigger data that could be accessed via a finger trigger built into the probe, or by a foot trigger on the ground. This allowed the user to capture higher-quality and segmented images and videos of the in vivo TM.

The main advantages of the algorithm presented here are: (I) real-time processing and display; (II) automated segmentation and quantitative thickness measurements; and (III) functionality that accounts for various angles of the TM. This was achieved by rejecting images without significant signal in them and by limiting the amount of steps in the algorithm. After noise thresholding, the images could be directly analyzed by the Hough transform. The slope obtained by the Hough transform was then used to correct for the tilt of the sample before calculating the thickness. Implementing three separate ROIs allowed the algorithm to overcome one of the inherent limitations of the Hough transform; the shape of interest needs to be known prior to analysis of the image. In our algorithm, we assumed the TM was fairly straight, and met this criterion by dividing the image into several ROIs where the algorithm could then adapt to a slight curvature of the TM. Determining the number of ROIs to implement is a tradeoff between processing speed and the potential for a more accurate segmentation. Three ROIs were used in this study as the best compromise between speed and accuracy.

Imaging the TM with a handheld probe is an inherently unstable process because of natural motion between both the subject and the operator. The operator needs to locate the TM within the imaging field-of-view and hold the probe steady during imaging and segmentation. For this reason, we analyzed the stability of the algorithm when a tape phantom (representing an artificial TM) was tilted to several angles and was vertically translated. For both tests, the algorithm had an acceptable peak-to-peak variability of 5 µm. However, when the sample was close to horizontal, the algorithm often failed due to artifacts in the data from normal incident reflections off of the surface of the phantom or TM. When tested on human subjects, the measurements generated by the algorithm varied due to the movements of the probe, user, and/or subject, but once the desired region of the TM was within the field-of-view, the measurements stabilized in less than 1 second. The normal TM is known to vary in thickness across its diameter, and as standard use protocols are developed, it may be necessary to scan multiple regions of the TM to more accurately assess these variations in thickness and to look for any sporadic distribution of biofilm growth that can be found across the TM.

It is important to note that the algorithm does not specifically discriminate between the multiple boundaries that may be associated with the TM, such as the boundary between the ear canal space and TM, the boundary between the TM and the middle ear space/effusion, and in the case of a biofilm, the boundary between the TM and biofilm or between the biofilm and middle ear space/effusion. Instead, it finds the boundaries with the highest intensity within the image. Therefore, when analyzing TMs with biofilms, the algorithm measured the thickness of the TM and the biofilm together. This, however, is advantageous because measurements that exceed the standard thickness of the normal TM are highly indicative of pathology, such as in acute OM or chronic OM with a biofilm, as previously recognized (20). Future improvements to this algorithm and approach will investigate segmentation strategies to identify and quantify all boundaries and layers that may be present, particularly in OM with effusions and/or biofilms.

Some limitations of our algorithm exist. In addition to failing when there is a normal incidence reflection from the TM, the algorithm is also problematic when imaging regions of the TM that are highly curved or close to the ear canal wall. To address this, more ROIs could be used; however, with more ROIs, the processing and display time will be slower, and there will be less data within each ROI for comparison, making the algorithm more susceptible to noise.

Conclusions

In conclusion, we have developed a novel, automatic, and quantitative algorithm that has enabled the real-time segmentation and thickness measurement of the in vivo human TM. This algorithm and methodology, in conjunction with OCT imaging of the TM and middle ear, has been shown to be both stable and reliable for measuring the thickness of the TM, even when it may be tilted or vertically displaced within the field-of-view. Applying this algorithm to OCT images from both healthy subjects and subjects with OM, robust detection of the TM boundaries and thickness measurements were obtained. The use of OCT and this algorithm offers physicians and healthcare providers a potentially new diagnostic biomarker to rapidly distinguish normal TMs and middle ear conditions from those involving acute and chronic OM, where the thickness measurements change in response to inflammation or the presence of a middle ear biofilm. By providing a new quantitative and diagnostic biomarker, these technologies offer the future potential to improve the detection, assessment, and treatment strategies for this very common disease.