Simultaneous full-field 3-D vibrometry of the human eardrum using spatial-bandwidth multiplexed holography

Abstract.

Holographic interferometric methods typically require the use of three sensitivity vectors in order to obtain three-dimensional (3-D) information. Methods based on multiple directions of illumination have limited applications when studying biological tissues that have temporally varying responses such as the tympanic membrane (TM). Therefore, to measure 3-D displacements in such applications, the measurements along all the sensitivity vectors have to be done simultaneously. We propose a multiple-illumination directions approach to measure 3-D displacements from a single-shot hologram that contains displacement information from three sensitivity vectors. The hologram of an object of interest is simultaneously recorded with three incoherently superimposed pairs of reference and object beams. The incident off-axis angles of the reference beams are adjusted such that the frequency components of the multiplexed hologram are completely separate. Because of the differences in the directions and wavelengths of the reference beams, the positions of each reconstructed image corresponding to each sensitivity vector are different. We implemented a registration algorithm to accurately translate individual components of the hologram into a single global coordinate system to calculate 3-D displacements. The results include magnitudes and phases of 3-D sound-induced motions of a human cadaveric TM at several excitation frequencies showing modal and traveling wave motions on its surface.

1. Introduction

The eardrum, or tympanic membrane (TM), transforms acoustic energy in the ear canal (at its lateral side) into mechanical motions of the ossicles (at its medial side: the middle ear). The acousto-mechanical-transformer behavior of the TM is determined by its shape, thickness, boundary conditions, and mechanical properties. For a better understanding of hearing mechanisms, full-field-of-view techniques are required to quantify shape and nanometer-scale sound-induced displacement of the TM in three-dimensions (3-D).^1–4 Common 3-D displacement measurement techniques require repeating the measurement at least along three sensitivity directions, which rely on the assumption of reproducibility (i.e., the observed specimen behaves similarly when stimulated and measured several times). Reproducibility is not applicable in cases where the measured phenomena are of temporally varying nature, which is the case of biological tissues such as the TM. Therefore, ideally, 3-D motions of such membranes should be quantified simultaneously.

For in vivo measurements,⁵ reproducibility is affected by a number of factors including changes in static pressure preloading or slight changes in shape due to respiration, motion of the subject, and the subject’s pulse. For measurements in cadaveric specimens, although the reproducibility is typically better than in live subjects, the response of the TM over time can also be affected, e.g., by the drying of the tissue. The motivations to measure the 3-D motion of the TM in response to sound in vivo potentially include the use of the results for a more accurate diagnostic of TM and middle ear pathologies or to evaluate middle ear surgeries.

In this paper, the development of a single-frame multiplexed lensless holographic system is described in order to minimize the measurement times to address the time-varying nature of the TM. In this method, the hologram is simultaneously illuminated with three incoherently superimposed pairs of reference and object beams, such that the images corresponding to each illumination direction are reconstructed at a different position on the image plane because of the slightly different spatial carrier frequency introduced by the incident off-axis angles of the reference beams.^6,7 Due to the differences in the position of the reference beams and the wavelength of each pair of beams, the reconstruction distance and magnification of each numerically reconstructed hologram corresponding to each sensitivity vector are different. We, therefore, developed and implemented a registration algorithm to accurately translate individual views into a single global coordinate system. The registration method uses phase-only correlation (POC)⁸ and a swept cut-off filtering to improve robustness. Three images obtained from three sensitivity vectors in a reconstructed multiplexed hologram are registered and, consequently, 3-D displacement components are extracted. To validate the method, the 3-D motion of a synthetic (time-invariant) membrane is measured with the presented method and with a method involving sequential measurements from multiple illumination angles. Once the accuracy of the measuring system is verified, the sound-induced 3-D motions of human cadaveric TM samples are measured at different tonal frequencies and the results are presented.

2. Methods

2.1. Off-Axis Digital Holography

In holography, the recorded intensity at the CCD (hologram) plane, $I(k,l)$, is given by

$$I(k,l)={|R(k,l)+O(k,l)|}^2={|R(k,l)|}^2+{|O(k,l)|}^2+R^{*}(k,l)O(k,l)+R(k,l)O^{*}(k,l),$$ (1)

where $R(k,l)$ and $O(k,l)$ are the complex amplitudes of the reference and object waves, respectively, $k$ and $l$ are the coordinates of the pixels in the CCD (hologram) plane, and * stands for the complex conjugate. In an off-axis digital holographic configuration^9–12 similar to the one shown in Fig. 1(a), the CCD sensor is illuminated with a plane reference wave, $R(k,l)=R\exp [i(2\pi f_{0}x)]$, at an angle $\theta$, and an object wave, $O(k,l)$, so that the resulting interference has a fringe pattern with equidistant spacing of^13,14

$$P_F=\frac{\lambda }{2\sin \left(\frac{\theta }{2}\right)},$$ (2)

where $P_F$ is the period of the fringe pattern, $\lambda$ is the laser wavelength, and $\theta$ is the off-axis angle. Therefore, the spatial frequency of the interferogram, i.e., reciprocal of the $P_F$, can be written as

$$f_0=\frac{2}{\lambda }\sin \left(\frac{\theta }{2}\right).$$ (3)

Fig. 1.

Principles of the off-axis holographic configuration: (a) spatial carrier frequency is induced by the off-axis angle of the reference wave with respect to the system’s optical axis; and (b) four terms in the spectrum of the hologram that consists of two autocorrelation terms T1 and T2 and two cross-correlation terms T3 and T4. The bandwidths of the cross-correlation terms are half the bandwidth of autocorrelation terms.

The four terms of Eq. (1) are superimposed in the hologram (spatial) domain. However, if the hologram is recorded in an off-axis geometry, these terms can be separated by taking the Fourier transform, $\mathcal{F}$, of the intensity to obtain

$$\mathcal{F}[I(k,l)]=\mathcal{F}({|R(k,l)|}^2)+\mathcal{F}({|O(k,l)|}^2)+\mathcal{F}[R^{*}(k,l)O(k,l)]+\mathcal{F}[R(k,l)O^{*}(k,l)].$$ (4)

The spectrum of such an off-axis hologram is schematically shown in Fig. 1(b). The central parts of the hologram spectrum, $\mathcal{F}({|R(k,l)|}^2)+\mathcal{F}({|O(k,l)|}^2)$, lack the phase information and contain only intensity values and are called the zero-order diffraction (ZOD) terms. The other two terms, symmetrically located from the origin, encode both intensity and phase and are called the twin images.

Based on the Wiener–Khinchin theorem, the Fourier transforms (FTs) of the squared fields ${|R(k,l)|}^2$ and ${|O(k,l)|}^2$, termed T1 and T2 in Fig. 1(b), are equal to the autocorrelation of the FT of wave fields $R(k,l)$ and $O(k,l)$ in the Fourier domain, respectively, and the FT of the term $R^{*}(k,l)O(k,l)$ is the cross-correlation of the FT of $R^{*}$ and $O$ [and similarly for the $R(k,l)O^{*}(k,l)$ term]. It should be noted that the correlation integral, similar to the convolution integral, represents the scanning of one function by another. The range of variables over which the integral is nonzero is given by the sum of the widths of the scanning and scanned signals. In the case of autocorelation, the maximum extent is twice the width of the function being autocorrelated.¹⁵ Therefore, considering an ideal plane reference wave $R(k,l)$ providing a single spatial frequency, the bandwidths of cross-correlation terms $\mathcal{F}(R^{*}O)$ and $\mathcal{F}(R{O}^{*})$ are half of the bandwidth of the autocorrelation term $\mathcal{F}({|O|}^2)$. As shown in Fig. 1(b), assuming that the object wave field has a bandwidth of B, the bandwidth of each of the cross-correlation terms, T3 and T4, corresponds to the bandwidth of the object’s wavefront and is equal to B, and the bandwidth of the autocorrelation term T2, $\mathcal{F}({|O|}^2)$, is 2B.

To ensure that the information recorded by the off-axis holography is resolvable and useful, two requirements should be satisfied. First, the cross-correlation terms (conjugated data) should be completely separate from the autocorrelation (ZOD) terms; this requirement can be satisfied using a minimum off-axis angle of the reference wave. Second, the CCD sensor should be able to resolve the interference pattern, which can be obtained by limiting the off-axis angle to be less than the Nyquist sampling limit, which requires at least two pixels per fringe period.

2.1.1. Minimum off-axis angle

In order to separate useful data that contain phase information (conjugated data) from the autocorrelation terms, which only include intensity values (ZOD terms), a minimum off-axis angle should be considered for the reference wave. To determine this minimum angle, it is sufficient to determine the minimum carrier frequency $f_{\min }$ for which the autocorrelation and cross-correlation terms are completely separated from each other. As shown in Fig. 1(b), the minimum carrier frequency that produces separated terms occurs when $f_{\min }\ge 3\mathrm{B}$, resulting in a highest spatial frequency of at least 4B and a total spatial bandwidth of 8B. Therefore, the optimum bandwidth of the hologram in the case of the Fresnel approximation is adjusted to be 8B.

It was demonstrated that for the case of Fresnel approximation in the scalar diffraction theory, the total bandwidth of the hologram is equal to the sum of the bandwidths of the convolution kernel (quadratic phase-factor) and the bandwidth of the FT of the object field.^16,17 The spatial bandwidth of the convolution kernel can be determined by taking a partial derivative of the convolution kernel with respect to the two directions and can be calculated with $N\mathrm{\Delta }x/\lambda d$ and $N\mathrm{\Delta }y/\lambda d$, where $N^2$ is the number of pixels in the hologram plane, $\mathrm{\Delta }x$ and $\mathrm{\Delta }y$ are the pixel sizes in the $x$ and $y$ directions, and $d$ is the reconstruction distance. On the other hand, it can be geometrically shown that the maximum bandwidth of the object can be calculated from $L_x/\lambda d$ and $L_y/\lambda d$, where $L_x$ and $L_y$ are the dimensions of the object in the horizontal and vertical directions.^18,19 Assuming $L_x=L_y=L$ and $\mathrm{\Delta }x=\mathrm{\Delta }y=\mathrm{\Delta }p$, the summation of these two bandwidths defines the total bandwidth of the hologram field and can be written as

$$8\mathrm{B}=\frac{L+N\mathrm{\Delta }p}{\lambda d}.$$ (5)

Therefore, the minimum spatial frequency of 3B can be calculated with

$$f_{\min }\ge 3\mathrm{B}=\frac{3(L+N\mathrm{\Delta }p)}{8\lambda d},$$ (6)

hence, the minimum off-axis angle of the reference wave can be obtained by combining Eqs. (3) and (6) and can be written as

$${\theta }_{\min }=2{\sin }^{-1}\left[\frac{3(L+N\mathrm{\Delta }p)}{16d}\right].$$ (7)

Therefore, a minimum angle of ${\theta }_{\min }$ should be considered to avoid any overlapping of the twin images and ZOD terms. However, in order for the CCD sensor to be able to resolve the interference patterns, one needs to limit the maximum off-axis angle.

2.1.2. Maximum off-axis angle

The achievable resolution for holographic reconstruction is dictated mainly by the spatial carrier frequency of the reference wave (which influences the fringe spacing) and the sampling resolution of the CCD. Based on the Nyquist theorem, the maximum spatial frequency of the interferogram that the CCD can resolve is

$$f_{\max }=\frac{1}{2\mathrm{\Delta }p},$$ (8)

where $\mathrm{\Delta }p$ is the pixel size of the camera. Combining Eqs. (3) and (8) and considering the small angles for $\theta$, the maximum angle can be obtained using

$${\theta }_{\max }=2{\sin }^{-1}\left(\frac{\lambda }{4\mathrm{\Delta }p}\right).$$ (9)

Therefore, considering the points mentioned in Sec. 2.1 and, in particular, Eqs. (7) and (9), for a CCD sensor with $2048\times 2048\text{\hspace{0.17em}\hspace{0.17em}pixels}$ of size of $\mathrm{\Delta }p=3.45\mu \mathrm{m}$, an object size of 1 cm, and an object-to-CCD distance of 10 cm, the allowable off-axis angle should be within the range of $3.6\le \theta \le 6.5\mathrm{deg}$. These results are considered in the design of our experimental setup.

2.2. Off-Axis Multiplexed Lensless Digital Holography

For 3-D displacement measurements using off-axis multiplexed holography, three incoherently superimposed pairs of object and reference beams are simultaneously recorded with the CCD sensor. As shown schematically in Fig. 2(a), the object of interest is illuminated concurrently with three object beams from different directions to provide three sensitivity vectors that are required for 3-D displacement measurements.¹⁴ The reference beams provide slightly different spatial carrier frequencies on the CCD to enable single-frame phase extraction.²⁰ The total intensity recorded on the CCD detector, $I(k,l)$, due to incoherent superposition of three pairs of reference-object beams is^6,7

$$I(k,l)=\sum _{n=1}^3{I}_n(k,l)=\sum _{n=1}^3{|R_n(k,l)+O_n(k,l)|}^2,$$ (10)

where $k$ and $l$ are the coordinates of the pixels in the CCD (hologram) plane, $R_n$ and $O_n$ are the $n$’th reference and object waves, respectively, with $n=1$, 2, 3. Similar to single sensitivity vector off-axis holography, described in Eqs. (1) and (4), Eq. (10) can be expanded to

$$I(k,l)=\sum _{n=1}^3{a}_n(k,l)+c_n(k,l)\exp [2\pi i(f_{n\xi }x+f_{n\eta }y)]+c_n^{*}(k,l)\exp [-2\pi i(f_{n\xi }x+f_{n\eta }y)],$$ (11)

where $a_n(k,l)=R_n^2(k,l)+O_n^2(k,l)$ and $c_n(k,l)=O_n(k,l)R_n(k,l)\exp [i{\varnothing }_n(k,l)]$. The two-dimensional (2-D) FT of Eq. (11) is

$$\mathrm{FT}\{I(k,l)\}=\sum _{n=1}^3{A}_n(\xi ,\eta )+C_n(\xi -f_{n\xi 0},\eta -f_{n\eta 0})+C_n^{*}(\xi +f_{n\xi 0},\eta +f_{n\eta 0}),$$ (12)

where the uppercase notation denotes the FT components. The terms $C_n$ and $C_n^{*}$ describe the spatial frequency distribution of the twin images components that are mathematically expressed as complex conjugates of one another, each containing the required phase information. As shown in Fig. 2(b), these terms are shifted in the Fourier domain due to the off-axis angles of the reference waves and can be separated from one another by an appropriate calculation and the setting of carrier frequencies $f_{n\xi 0}$ and $f_{n\eta 0}$, as described in Sec. 2.1.

Fig. 2.

Multiplexed off-axis holography: (a) schematic of the recording procedure by simultaneous illumination with three pairs of reference and object beams; (b) spectrum of the recorded multiplexed off-axis hologram consisting of three incoherently superimposed interference patterns; and (c) representative numerically reconstructed double-exposure multiplexed hologram that is used for 3-D displacement measurements. The object of interest is a 1-cm diameter latex membrane, acoustically excited by a tone of 1.5 kHz. Sound-induced concentric fringe patterns and four painted marks used to test registration algorithms are visible on each of the reconstructed images.

Our techniques are based on single-frame lensless digital holography^21–23 in which reconstructions of the multiplexed holograms are obtained by the Fresnel–Kirchhoff integral that is efficiently computed with the 2-D FFT as

$$\mathrm{\Gamma }(m,n)=\mathrm{FFT}2[R(k,l)h(k,l)\psi (k,l)],$$ (13)

where $\mathrm{\Gamma }(m,n)$ is the complex reconstructed hologram at coordinates $m$ and $n$ in the reconstruction plane, $R(k,l)$ is the complex amplitude of the plane reference wave that is considered to be one in the numerical reconstruction, and $\psi (k,l)$ is the 2-D chirp function defined with

$$\psi (k,l)=\exp [\frac{-i\pi }{\lambda d}(k^2\mathrm{\Delta }{x}^2+l^2\mathrm{\Delta }{y}^2)],$$ (14)

where $\mathrm{\Delta }x$ and $\mathrm{\Delta }y$ are the pixels sizes of the CCD sensor, $\lambda$ is the laser wavelength, and $d$ is the reconstruction distance. Additionally, the DC component can be mathematically removed from the multiplexed hologram by recognizing that the DC component in the Fourier domain is due to the low-frequency spatial variation in the hologram, and it can thus be isolated by applying a high-pass filter.^10,11 Therefore, the optical phase of the reconstructed hologram is obtained with

$$\phi (x,y)=\mathrm{atan}2\left\{\frac{\mathfrak{I}[\mathrm{\Gamma }(m,n)]}{\mathfrak{R}[\mathrm{\Gamma }(m,n)]}\right\},$$ (15)

where $\mathfrak{I}$ and $\mathfrak{R}$ denote the imaginary and real parts of the complex number $\mathrm{\Gamma }(m,n)$. Thus, in one single frame of the camera, optical phase information corresponding to all three sensitivity vectors exists. With the use of double-exposure techniques, i.e., subtracting optical phases of two reconstructed holograms corresponding to deformed and reference states of the object, the 3-D motion components of the objects can be obtained with the method of multiple sensitivity vectors in holographic interferometry.²⁴ Considering the fact that the fringe-locus function $\mathrm{\Omega }$,²⁵ i.e., the unwrapped optical phase difference between the two states, is the scalar product of the sensitivity vector, $\mathit{K}({\mathit{K}}_x,{\mathit{K}}_y,{\mathit{K}}_z)$, with the object’s displacement vector, $\mathit{d}({\mathit{d}}_x,{\mathit{d}}_y,{\mathit{d}}_z)$, the 3-D displacements components are obtained by

$$\left[\begin{array}{c}{\mathit{d}}_x\\ {\mathit{d}}_y\\ {\mathit{d}}_z\end{array}\right]=\frac{\lambda }{2\pi }{\left[\begin{array}{ccc}{\mathit{K}}_x^1& {\mathit{K}}_y^1& {\mathit{K}}_z^1\\ {\mathit{K}}_x^2& {\mathit{K}}_y^2& {\mathit{K}}_z^2\\ {\mathit{K}}_x^3& {\mathit{K}}_y^3& {\mathit{K}}_z^3\end{array}\right]}^{-1}\times \left[\begin{array}{c}{\mathrm{\Omega }}_1\\ {\mathrm{\Omega }}_2\\ {\mathrm{\Omega }}_3\end{array}\right].$$ (16)

However, in the case of off-axis multiplexed holography, the differences in the position of each reference beam and wavelength of each pair of beams cause the position, reconstruction distance, and magnification of each image corresponding to each sensitivity vector to differ. The translation between the images could be computed analytically if the exact relative orientation and position of the reference beams were known. However, such an approach would require additional hardware, and small errors in the measurement of the orientation could lead to large errors in the estimate of the translation. It is thus more efficient to register the resulting images in a postprocessing step. Therefore, for quantification of the 3-D displacement, an image registration algorithm is required to position each image into a common global coordinate system.

2.3. Registration of Multiplexed Holograms

In order to place the images in a common coordinate system, prior to applying Eq. (16), a registration algorithm based on POC has been developed and implemented.^8,26,27 This algorithm accounts for pure translation of the reconstructed images and is an intensity-invariant approach to compensate for slight differences in the overall brightness and intensity of the images.

Considering two translated images $f(n_1,n_2)$ and $g(n_1,n_2)=f(n_1-{\delta }_1,n_2-{\delta }_2)$, with relative separations of ${\delta }_1$ and ${\delta }_2$ in the horizontal and vertical axes, respectively, their corresponding FT images, $F(k_1,k_2)$ and $G(k_1,k_2)$, are related to each other through the shift theorem with

$$G(k_1,k_2)=F(k_1,k_2)·e^{-2\pi j[\frac{k_1{\delta }_1}{M}+\frac{k_2{\delta }_2}{N}]},$$ (17)

where $M$ and $N$ are the number of pixels in the image, and $k_1$ and $k_2$ are the coordinates in the frequency domain. Therefore, the normalized cross-power spectrum of $F(k_1,k_2)$ and $G(k_1,k_2)$ is

$$R(k_1,k_2)=\frac{F(k_1,k_2)G^{*}(k_1,k_2)}{|F(k_1,k_2)G^{*}(k_1,k_2)|}=e^{2\pi j[\frac{k_1{\delta }_1}{M}+\frac{k_2{\delta }_2}{N}]},$$ (18)

where * defines the corresponding complex conjugated functions. By applying the inverse 2-D FFT to Eq. (18), the POC function, $r(n_1,n_2)$ is calculated with

$$r(n_1,n_2)={\mathcal{F}}^{-1}\{R(k_1,k_2)\}=\delta (x+{\delta }_1,y+{\delta }_2),$$ (19)

with $\delta$ being the Kronecker delta function having a single peak, whose location defines the translation between the two images⁸ and can be computed with

$$[{\delta }_1,{\delta }_2]=\mathrm{argmax}\{r(n_1,n_2)\}.$$ (20)

The phase correlation, $r$ [Eqs. (19) and (20)] was calculated from a series of subsets of $R$ with regularly decreasing high-frequency limit (equivalent to spatial filtering with regularly decreasing low-pass cut-off frequencies). The translation between two images is obtained by considering the most frequent peak of the POC functions resulting from the calculation series.

2.4. Multiplexed Holographic System

In our experimental system, incoherent superposition of the beams is implemented by using three different near-infrared external cavity tunable laser sources with wavelengths centered at 779.8, 780.2, and 780.6 nm. As shown in Fig. 3, the acousto-optic modulators (AOMs) contained in each laser delivery subsystem (LD1 to LD3) are used to switch the laser on/off to enable stroboscopic measurement capabilities.^2,21,28 A dual-channel function generator is used with one of the channels sets to acoustically stimulate the TM sample with a calibrated speaker, while the second channel is set to pulse mode to simultaneously drive all three AOMs. The duty cycle of the pulse signal sent to the AOMs is typically set to 2% to 5% of the period of the tonal stimulus; however, the multiplexed holographic system (MHS) temporal resolution, i.e., the system’s maximum detection bandwidth, is determined by the 200-kHz limit of the AOMs used to strobe the phasic illumination. As shown in Fig. 3, each laser is coupled into single-mode polarization maintaining fibers and then split into reference and object beams. The reference beams illuminate the CCD in an off-axis configuration by a beam splitter and the object beams concomitantly illuminate the sample from three different directions to define the sensitivity vectors for 3-D displacement measurements. In Fig. 3, the optoelectronic components are shown for only one of the laser delivery subsystems; however, all three laser delivery subsystems contain similar components.

Fig. 3.

Experimental system for simultaneous 3-D displacement measurements with multiplexed holography. The solid lines denote the beam paths and the dashed lines the connections between components. AOM is the acousto-optic modulator; RB and OB are the reference and object beams, respectively; BS is the beam splitter; and TM is the tympanic membrane sample. Each of the three laser delivery subsystems contains similar components.

The computing platform performs multiple tasks that include synchronizing the stroboscopic illumination of the lasers with different stimulus phases, acquiring multiplexed holograms with a 5 MPix CCD camera having a pixel size of $3.45\times 3.45\mu {\mathrm{m}}^2$, and reconstructing the holograms in real time. A live 2-D FFT is used to ensure that the components of the frequency spectrum of the hologram do not overlap with each other.²⁹

3. Validation of Measuring Capabilities

In order to validate the measuring capabilities of the MHS, the results of an artificial membrane obtained with our MHS are compared with the ones obtained with a documented repetitive holographic interferometric method.²³ The artificial sample is a thin semispherical membrane mounted on a mechanical shaker that can operate over a wide range of frequencies up to 150 kHz. The results of the vibration of this sample at 25-kHz frequency are shown in Fig. 4. Modulation and wrapped optical phase images of both twin components are shown to illustrate how the three pairs of complex conjugated reconstructed holograms are distributed.

Fig. 4.

Stroboscopic measurements of a vibrating thin semispherical membrane clamped around its perimeter: (a) and (b) are the modulation and wrapped optical phase of the real and conjugated reconstructed multiplexed holograms. The excitation frequency is 25 kHz and the membrane is concomitantly illuminated from three different directions to define the sensitivity vectors for 3-D measurements, which in this case correspond to the difference in the motion of the membrane between two stimuli phases of 0 deg and 90 deg. The images for each of the three sensitivity vectors and their corresponding conjugates (fuzzy images) are symmetrically arranged around the center of each figure panel.

To determine the 3-D deformations of the membrane, the POC image registration algorithm described in Sec. 2.3 is used to place the wrapped optical phases in a common coordinate system prior to unwrapping and applying Eq. (16). Figures 5(a)–5(c) show the registered wrapped and corresponding unwrapped optical phases of the artificial sample along three sensitivity vectors ${\mathit{K}}_1$ to ${\mathit{K}}_3$ that are used to quantify 3-D displacements of the membrane. Unwrapping of the optical phases might add a constant phase value (DC) to the unwrapped data, therefore, as shown in Fig. 5(c), a seed point at an area with no deformation is selected and the value of this point is considered as the DC, so that all the pixels’ values in this image are subtracted from this DC. This step is repeated for all three unwrapped phases so that the data are all normalized before quantification of 3-D motion.

Fig. 5.

Registered optical phases corresponding to sound-induced displacement of an artificial membrane: (a)–(c) wrapped and unwrapped optical phases corresponding to sensitivity vectors ${\mathit{K}}_1$ to ${\mathit{K}}_3$, respectively. “+” in the unwrapped optical phase along ${\mathit{K}}_3$ denotes the location of the seed point to identify the DC of the unwrapped data. The color and grayscale coded wrapped and unwrapped phases are in radians.

The registered, unwrapped, normalized phases are used to quantify 3-D deformation. To test and verify the validity of the measurements obtained with MHS, the results are compared with results obtained with a documented repetitive holographic interferometric method.²³ The results obtained with both methods are shown in Fig. 6. On visual inspection, the results of both methods are almost indistinguishable from each other. A point-by-point correlation measure indicates a Pearson correlation coefficient of 97%, 96%, and 99% along the $x$-, $y$-, and $z$-axes, respectively. It should be noted that since the resolution of the results obtained from the two methods is different (in the MHS the image resolution is $500\times 500$ while in the regular repetitive method the resolution is $1500\times 1500$), the images with higher resolution are first downsampled and then the correlations are evaluated.

Fig. 6.

Validation of the results of three-dimensional (3-D) displacement measurements versus a documented method:²³ (a) displacement components along $x$, $y$, and $z$ obtained using MHS; and (b) displacement components along $x$, $y$, and $z$ obtained with the method of repetitive holographic interferometry. $ℝ$ is the Pearson correlation coefficient between the results obtained with the two methods.

4. Representative Measurements of Three-Dimensional Motions of Human Tympanic Membrane

The cadaveric human TM of a female, 46-year-old donor was prepared by removing the bony portion of the ear canal in a lightly fixed temporal bone. The middle-ear space of the sample was widely opened, which enabled assessment of the normality of the TM and ossicles. The temporal bone was immersed in Thiel embalming solution for several weeks before the experiments to stop decay and eliminate potential pathogens.³⁰ Due to semitransparency of the mammalian TMs, the sample was coated with a thin layer of zinc oxide to increase the laser light reflection, as shown in Fig. 7(a). The effect of coating on shape and deformation patterns has been studied by several researchers and found to be negligible.^31,32 The temporal bone was held with an adjustable clamp and mounted on a post in front of the holographic system. Figure 7(b) schematically shows the simultaneous recording of a multiplexed hologram of the cadaveric human TM. As shown in this figure, the $x$-axis is along the superior-inferior direction and the $y$-axis is along posterior-anterior direction, while the $z$-axis is perpendicular to the tympanic ring plane and along the lateral side of the TM. Figure 7(c) shows a representative example of wrapped optical phase of sound-induced motion of the TM acquired with a double-exposure multiplexed hologram of the TM surface. The three fringe patterns show the similarities and differences in their spatial arrangement and their optical phase magnitudes.

Fig. 7.

Simultaneous recording of a multiplexed hologram of a cadaveric human TM in order to acquire 3-D sound-induced displacements: (a) the human temporal bone including the TM under investigations with the location of speaker and microphone; (b) schematic of the recording of the multiplexed hologram illustrating simultaneous illumination of the three light sources on the TM samples from different sensitivity vectors; and (c) acoustically induced wrapped optical phase corresponding to the stroboscopic measurement of a human TM motion at a stimulus frequency of 6.884 kHz. The optical phases correspond to the difference in the motion of the TM at two acoustic phases of 0 deg and 90 deg. Bright blurry regions facing each of them on the opposite of the image’s center are the corresponding complex conjugates.

Prior to stroboscopic measurements, the time-averaged response of the TM at different tonal stimuli was monitored³² and the excitation frequencies were chosen at the maximum motion of the TM. At each stroboscopic phase, sound-induced motions of the TM along three orthogonal axes $x$, $y$, and $z$ are calculated from the unwrapped optical phases, as described in Sec. 2.2. Then, FFT algorithms are used to reconstruct magnitudes and phases of motions along all three axes² and the results are shown in Fig. 8. The displacement patterns are simpler at lower frequencies, and as the excitation frequency increases, the complexity of the displacement maps also increases. At 0.8 kHz, one or two regions of large displacement are visible with a relatively homogenous phase along all three axes. As excitation frequency increases nodal lines appear, characterized by lines of minimum magnitude corresponding to separations between regions where the phase is different by 0.5 cycles, suggesting that the number of areas on the surface of the TM that are moving out-of-phase also increases. Such a phenomenon can be clearly seen in TM’s motion patterns obtained with excitation frequencies of 4.68 and 13.2 kHz, as shown in Fig. 8. An interesting observation is that in low excitation frequencies, the magnitude of motion along the $x$-axis is slightly greater than along the $y$-axis. Since the manubrium is located along the $x$-axis of the measuring system [Fig. 7(b)], a possible interpretation is that the TM, in response to sound, deforms more in the direction parallel to the manubrium than in the direction perpendicular to it.

Fig. 8.

Magnitudes and phases of 3-D sound-induced displacements of the human TM excited at three different frequencies obtained with FFT-reconstruction algorithm along three orthogonal axes $x$, $y$, and $z$. The displacements are normalized based on the sound pressure level obtained from the microphone. The outline of the manubrium is shown in white in all the figures.

The presence of delay in the sound transfer of the middle ear triggered the idea of the existence of traveling waves on the surface of the TM and it was suggested by several researchers.^33,34 Although our data show that the majority of the TM motions are modal, there are indications of a combination of modal and traveling wave like motions on the surface of the TM. Figure 9 shows the sound-induced motions of the TM excited with a tone of 6.884 kHz at several instances of the full-cycle vibration obtained from stroboscopic illumination at different phases of the excitation signal. Two arrows shown in Fig. 9 illustrate the main traveling waves’ paths. As shown in this figure, the motion of the first traveling wave is initiated from the posterior-inferior quadrant and travels in a semicircular way toward the posterior-superior quadrants. The second one circulates in the posterior-inferior quadrant. Considering the excitation frequency and the travel distance, a wave speed of $21\mathrm{m}/\mathrm{s}$ can be calculated.

Fig. 9.

Responses of the human TM excited with a tone of 6.884 kHz at different acoustic phases suggesting the presence of a combination of modal and traveling wave patterns. Black arrows shown in the pattern of phase 45 deg indicate the main paths of the traveling waves.

5. Discussion

In this paper, we have shown the results of simultaneous 3-D vibrometry of the human eardrum using MHS. The results show great similarities with state-of-the-art multiple sensitivity vectors holographic methods, while the recording time is drastically reduced (decrease in the acquisition time by a factor of 3 in the case of a single frame and a factor of 12 in the case of four phase-stepping techniques). The developed method has promising applications in the experimental measurement of the motion of biological membranes such as the TM. Simultaneous 3-D motion measurement is a crucial step toward holographic measurements of live biological specimens.

5.1. Choice of the Registration Technique

The success of the registration depends on the similarities in spatial patterns. In theory, if the motion of the measured object is very different along the multiple sensitivity vectors, the registration might be less accurate. However, since the sensitivity vectors are not orthogonal, the motions along all sensitivity vectors are correlated. Moreover, the pattern made by nodes (regions with no membrane motion) is likely to be similar along several vectors since the displacement is equally zero along the multiple directions. The choice of an FFT-based method over a moving correlation method is motivated by its lower computational load. Also, the FFT needs to be calculated only once for spatial filtering and registration. An alternative to use a posteriori registration would be the analytic computation of the translation between the reconstructed images. In this case, a very accurate measure of the position and orientation of all elements of the optical system would be required. Such an approach would not only be cumbersome, but small errors could lead to large errors in the translation calculation. For these reasons, the developed method is preferred.

5.2. Dynamics of Human Tympanic Membranes

Our previous studies have shown that the sound-induced motions of the mammalian TMs follow several patterns (simple, complex, ordered) at different stimulus frequency ranges.^21,32 It was shown that at low excitation frequencies (up to 1 kHz), most of the points ($>90\%$) on the surface of the TM are moving in-phase and the surface displacements are well described by low-order modal motions without any nodal points.^2,35 As the excitation frequency increases, the displacement patterns can be described with a combination of higher-order modal displacements (with multiple nodes) and traveling waves. The 3-D results shown in this paper show similar types of dynamics along all three axes, which are in the plane of the tympanic ring (along the $x$- and $y$-axes) and normal to such a plane (along the $z$-axis). The motion pattern is simple at 0.8 kHz (top 2 rows of Fig. 8), complex at 4.68 kHz (middle 2 rows of Fig. 8), and ordered at 13.2 kHz (bottom 2 rows of Fig. 8) stimuli.

6. Conclusions

Due to the time-varying nature of biological tissues like the TM, a unique method for 3-D displacement measurements based on multiplexed holography is developed that allows for simultaneous holographic measurements along multiple sensitivity vectors. The developed methodology is a critical step toward in vivo measurements of 3-D TM motions. In our approach, the hologram of an object of interest is recorded with three simultaneous incoherently superimposed pairs of reference and object beams, such that the modulation image corresponding to each illumination direction is reconstructed at a different position of the image. An image registration algorithm based on the shift theorem of the FT is implemented to register the images. The displacement measurements are in good agreement (greater than 96%) with other documented methods, while simultaneous acquisition of all three measurements reduces the effects of temporal variations of the specimens. The time needed for a given 3-D displacement measurement is decreased at least threefold. We demonstrate that the present method is a valid alternative to repetitive holographic methods and offers promising perspectives toward faster accurate displacement measurements of biological specimens.

Biographies

Morteza Khaleghi is a PhD candidate in mechanical engineering at Worcester Polytechnic Institute, Worcester, Massachusetts. His research interests include biomedical imaging and developments of optical and nondestructive metrology systems.

Jérémie Guignard received his MSc degree from the Swiss Federal Institute of Technology (EPFL), in 2009 and completed his PhD at the University of Bern, Switzerland, in 2013. He subsequently received a fellowship from the Swiss National Science Foundation (SNSF) to work on eardrum holography at the Massachusetts Eye and Ear Infirmary. His research interests include middle-ear physiology and prosthetics and biomedical image processing.

Cosme Furlong is an associate professor of mechanical engineering, working in mechanics, optical metrology, nanoengineering, science, and technology.

John J. Rosowski is a professor of otology and laryngology, and also of health sciences and technology, working in acoustics and mechanics of the external, middle, and inner ear, with an interest in comparative middle and external ear structure and function.