The effect of high-speed videoendoscopy configuration on reduced-order model parameter estimates by Bayesian inference

Abstract

Bayesian inference has been previously demonstrated as a viable inverse analysis tool for estimating subject-specific reduced-order model parameters and uncertainties. However, previous studies have relied upon simulated glottal area waveforms with superimposed random noise as the measurement. In practice, high-speed videoendoscopy is used to measure glottal area, which introduces practical imaging effects not captured in simulated data, such as viewing angle, frame rate, and camera resolution. Herein, high-speed videos of the vocal folds were approximated by recording the trajectories of physical vocal fold models controlled by a symmetric body-cover model. Twenty videos were recorded, varying subglottal pressure, cricothyroid activation, and viewing angle, with frame rate and video resolution varied by digital video manipulation. Bayesian inference was used to estimate subglottal pressure and cricothyroid activation from glottal area waveforms extracted from the videos. The resulting estimates show off-axis viewing of 10° can lead to a 10% bias in the estimated subglottal pressure. A viewing model is introduced such that viewing angle can be included as an estimated parameter, which alleviates estimate bias. Frame rate and pixel resolution were found to primarily affect uncertainty of parameter estimates up to a limit where spatial and temporal resolutions were too poor to resolve the glottal area. Since many high-speed cameras have the ability to sacrifice spatial for temporal resolution, the findings herein suggest that Bayesian inference studies employing high-speed video should increase temporal resolutions at the expense of spatial resolution for reduced estimate uncertainties.

I. INTRODUCTION

Reduced-order models have long been employed in the speech science community to explore the physics of phonation, such as onset pressure,^1,2 potential compensation mechanisms for posterior glottal openings,³ tissue and flow asymmetry effects,^4–6 the roles of intrinsic muscle activation on dynamical parameters,⁷ vocal tremor,⁸ and voice breaks,⁹ to name a few. While valuable for elucidating the underlying physical principles governing a wide array of observed speech phenomena, such studies are largely relegated to population-level assessments; that is, by virtue of employing normative material and flow properties in the modeling domain, general trends are uncovered. At the individual speaker level, subject-to-subject variability in geometry, material properties, oronasal geometry, supraglottal coordination, and any variety of other factors may result in specific performance metrics outside of the predictive normative ranges.^2,10,11

In an effort to exploit the simplicity of reduced-order models for subject-specific assessment, recent efforts have introduced inverse analysis techniques to estimate model parameters.^12–16 Briefly, the aim of inverse analysis is to determine the parameterization of a given model such that the output of the model best matches the observation data.¹⁷ In the case of voicing, the observation would be some clinical measure, such as vocal fold (VF) kinematics from high-speed videoendoscopy (HSV) or stroboscopy, oral airflow, sound pressure level, and/or other information-rich signals.¹⁵ Two general inverse analysis approaches have been applied in voicing: optimization methods and Bayesian inference.

Optimization techniques typically determine the parameter set that minimizes the error between the measurement (observation) and the model output. Such techniques have successfully determined reduced-order model parameters based on glottal dimensions extracted from HSV.^{13,14,18–21} Döllinger et al.¹⁴ conducted the first optimization studies based on dimensions extracted from HSV and were able to identify left-right VF asymmetries in a two-mass model. Since then, further studies have employed inverse analysis to identify healthy and pathological voices¹⁹ and gender-specific voice differences.¹³ While most studies have focused on recordings of sustained phonation, approaches have also been developed to estimate model parameters during non-stationary phonation.²⁰ Recently, deep learning has been employed to extract model parameters from video observations.²²

Bayesian inference, in contrast, is a statistical inference approach wherein inferred parameters are treated as random variables, leading to probability distributions of parameters instead of point estimates.¹⁷ The Bayesian framework is a natural setting as it accounts for the inherent uncertainties in clinical measurements; since there is uncertainty in the measurements, there is a range of possible parameters that can match those measurements. Parameter distributions can further propagate to model outputs,²³ such as vocal fold trajectories and collision pressure. Bayesian inference was first introduced in phonation modelling by Cataldo et al.,^12,24 wherein they employed a particle filter methodology to reconstruct the posterior distributions of reduced-order model parameters from synthetic observation data (observation data generated from a reduced-order model). Hadwin et al.¹⁵ extended the particle filter method to estimate non-stationary model parameters, demonstrating that time-varying muscle activation parameters⁷ can be estimated. In a follow-up study, the same group introduced an extended Kalman filter approach for estimating non-stationary parameters, which significantly reduced the computational expense without sacrificing accuracy.¹⁶

High-speed video has been the principal clinical measure employed in previous studies on subject-specific phonation modeling due to the high information content. Generally, HSV involves an endoscope coupled to a high-speed camera to record the VF motion.¹⁸ Glottal dimensions are then extracted from the recorded video through methods such as edge detection^18,25 or image thresholding.²⁶ Bayesian inference provides a way to account for uncertainty in the extracted dimensions and the resulting effects on the estimated parameters. To date, however, Bayesian inference has only been demonstrated using simulated HSV data, with measurement uncertainty imposed in the form of additive noise on the glottal area waveform.^15,16 In the clinic, glottal dimensions are measured from real HSV and are thus subject to practical aspects of HSV acquisition,^25,27 such as, resolution and frame rate of the camera, camera motion, viewing angle (orientation of the camera image plane with respect to the folds), lighting conditions, camera quantum efficiency, and lens aberrations, to name a few.

To obtain meaningful parameter estimates, it is necessary to understand how these imaging variables will affect the inference procedure, and how to mitigate or correct errors that may be introduced. Prior work utilizing the Bayesian framework to estimate time varying parameters from a body-cover model (BCM)^15,16 employed purely simulated measurements which lack key measurement uncertainties present in real high-speed video. This work investigates the effect of three HSV imaging variables: frame rate, spatial resolution, and viewing angle. The former may be modified to a degree via camera settings, while the latter depends on the orientation of the subject with respect to the camera, which may be difficult to control in practice. To robustly characterize their effects, a driven VF facility with trajectories computed by an underlying BCM²⁸ is employed. The glottal width is extracted from a high-speed, high resolution recording of the VF motion at varying values of each of the three imaging variables for different subglottal pressures and muscle activations. Bayesian inference is then applied to estimate subglottal pressure and muscle activation, which are compared with the ground truth values. Trends in estimated parameter biases and uncertainties are calculated as a function of the imaging variables to assess their effects on inferred parameters.

II. METHODOLOGY

A. Experimental setup

High-speed videoendoscopy is simulated experimentally using a 15-times scaled-up driven VF model facility²⁹ coupled with a NIKON D3200 DSLR camera. See Fig. 1. The medial surface, including the radii of curvature at the inferior and superior VF margins, are machined aluminum such that the geometry matches the M5 geometry of Scherer et al.³⁰ when in the neutral position (see Fig. 2). Each VF is 150 mm long in the anterior-posterior direction and is connected to a two degree-of-freedom motion system via a rigid shaft; a stepper motor and gearbox system enable rotation, θ, of the VF, while a translation stage comprising a lead screw and additional stepper motor independently provides linear motion, s, in the medial-lateral direction. These two degrees-of-freedom enable replication of the two primary VF vibration modes during modal speech, producing a mucosal wave-like motion.²⁸ The motion is akin to the motion of the “plate” in the bar-plate reduced order VF model or, equivalently, the “cover” motion in the BCM.⁷

FIG. 1.

A schematic of the simulated HSV experimental setup. A pair of rigid two-dimensional VF medial surfaces (a coronal cross-section) are driven by a motion system that provides one translational and one rotational degree of freedom, as in the bar-plate VF model (Ref. 7). A consumer DSLR is used to capture the VF motion. A calibration plate with two dots, shown as circles, allow alignment of the camera view along a specified angle α.

FIG. 2.

The mapping procedure connects points on the BCM to points on the physical VF geometry. Each physical VF is controlled by two degrees of freedom (s and θ). These are mapped to the cover mass displacements (x_u and x_l) of the BCM. The rigid VF dimensions are given by: $L_{\mathrm{u}}=17.07\hspace{0.17em}\text{mm},\hspace{0.17em}{L}_{\mathrm{l}}=15.11\hspace{0.17em}\text{mm},\hspace{0.17em}{\alpha }_{\mathrm{u}}=27.71°,\hspace{0.17em}{R}_{\mathrm{l}}=22.23\hspace{0.17em}\text{mm}$, and $R_{\mathrm{u}}=14.29\hspace{0.17em}\text{mm}$.

The motion of the rigid VF models is controlled by a BCM^7,28 without acoustic coupling. The glottal flow is modeled as one-dimensional Bernoulli flow with constant subglottal and supraglottal pressures. Muscle activation rules are used to relate lumped BCM properties to the activation of the cricothyroid, a_ct, thyroarytenoid, a_ta, and lateral cricothyroid muscles, a_lc.⁷ Each muscle activation parameter is a dimensionless number representing degree of activation, with $a_{\text{ct}},\hspace{0.17em}{a}_{\text{ta}}$, and a_lc ranging between, [0, 1], [0, 1], and [−1, 1], respectively.⁷ As such, the BCM is governed by a total of 11 parameters: six for initial positions and velocities of the masses ${\mathbf{x}}_0=(x_{\mathrm{u}0},v_{\mathrm{u}0},x_{\mathrm{l}0},v_{\mathrm{l}0},x_{\mathrm{b}0},v_{\mathrm{b}0})$, three for the muscle activations ($a_{\text{ct}},\hspace{0.17em}{a}_{\text{ta}},\hspace{0.17em}{a}_{\text{lc}}$), and two for the subglottal and supraglottal pressures $(P_{\text{sub}},P_{\text{sup}})$.

The positions of the cover masses of the BCM $(x_{\mathrm{u}},x_{\mathrm{l}})$ are related to s and θ through the mapping

$$15\left[\begin{array}{c}{x}_{\mathrm{u}}\\ x_{\mathrm{l}}\end{array}\right]=\left[\begin{array}{c}s-R_{\mathrm{l}}-L_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\theta )\\ s-R_{\mathrm{u}}-L_{\mathrm{u}}\hspace{0.17em}\text{sin}\hspace{0.17em}({\alpha }_{\mathrm{u}}-\theta )\end{array}\right],$$ (1)

where the variables are defined in Fig. 2, and the factor of 15 is used to scale the physiological displacements of the BCM into displacements of the physical models. The same BCM is used as the fitting model in the Bayesian analysis, thus providing a ground truth for assessing the accuracy of the estimations, see Sec. II B.

The positions and angles of the VFs at each time step are determined by solving the BCM equations and Eq. (1) in a loop using a 4th order Runge-Kutta solver on a National Instruments cRIO 9074 real-time controller with a time step of 1/350 ms. In each loop, the BCM is integrated in time through 25 time steps, the displacements are mapped to translations and rotations of the VFs, which are then actuated through the motion system to the calculated positions. To avoid collision of the physical folds, their motion is arrested when they are within 3 mm of each other. The dynamics of the BCM continue to progress through the collision model via overlapping masses; the physical models start moving again once the mass positions are outside of the 3 mm buffer.

Four cases were tested and imaged, wherein a_ct and P_sub were varied, while the remaining parameters remained fixed. The constant parameters were given by $(a_{\text{ta}},a_{\text{lc}})=(0,0.5),(x_{\mathrm{l}0},x_{\mathrm{u}0},x_{\mathrm{b}0})=(0,0,3)\hspace{0.17em}\text{mm},\hspace{0.17em}(v_{\mathrm{l}0},v_{\mathrm{u}0},v_{\mathrm{b}0})=(0,0,0)\hspace{0.17em}\text{mm}/\mathrm{s}$, and $P_{\text{sup}}=0\hspace{0.17em}\text{Pa}$. The variable parameters are listed in Table I.

TABLE I.

Four sets of BCM parameters investigated.

Case	1	2	3	4
a ct	0.15	0.15	0.20	0.20
Psub [Pa]	1800	2000	1800	2000

1. Simulated high-speed videoendscopy

High-speed videoendoscopy was simulated experimentally by taking a sequence of frames using a consumer DSLR camera with frame rate of 30 fps and resolution of 1920 × 1080 pixels placed approximately 100 cm from the VFs, see Fig. 1. The VFs are driven by a numerical model (BCM), so the timing of the physical VF motion can be artificially slowed by updating their positions in real-time at a rate slower than the simulation times of the numerical model. In this study, for each physiological simulation time step of 1/14 ms, 200 ms of real-time passes as the cRIO moves the VFs to the updated positions. As a result, the motion of the VFs is slowed by a factor of 2800. At 30 fps, this corresponds to an equivalent HSV frame rate of 84 000 fps for the physiological folds. Typical HSV frame rates in the clinic range from 2000 to 4000 fps, with research-caliber systems reporting frame rates up to 20 000 fps.³¹

The viewing angle, α, is investigated by shifting the camera about an arc centered at a point between the VFs in 2.5° increments up to a total offset of 10°. A study on laryngoscopy for endotracheal intubation found angular variations in practitioner positioning on the order of 10° during the procedure,³² suggesting this upper bound is reasonable in the present work.

To investigate the effect of frame rate, the simulated 84 000 fps video is downsampled by including only every d_temporalth frame to construct lower frame rate videos. Similarly, spatial resolution is explored by spatially downsampling each video frame. Spatial resolution is measured using the magnification factor, the ratio of physical distances in the image to the equivalent pixel distances. Note that high fidelity spatial resolutions correspond to low magnification factors. We estimate that HSV used in clinical settings have magnification factors of ≈ 0.04 mm px⁻¹, which assumes a 512 × 512 px camera³³ and a 20 mm glottal length³⁴ that spans the entire image height. The spatial resolution of the simulated HSV at full resolution at experimental scales is approximately 0.1 mm px⁻¹, which corresponds to 0.0066 mm px⁻¹ at physiological scales, roughly six times better than that obtained in clinical settings. Spatial downsampling is performed by averaging the intensity of $d_{\text{spatial}}\times d_{\text{spatial}}$ pixels in the high resolution image to create a single pixel at the lower resolution.

A total of five angles of view, seven frame rates, and five spatial resolutions are considered that span the range of realistic values in clinical settings; see Table II. Since frame rate and spatial resolution are explored through numerical downsampling, only one video is required for each set of BCM parameters and viewing angles, leading to a total of 20 recordings. Each recording captures approximately 17 oscillations. A sample video showing the BCM and physical vocal fold trajectories is available in the Supplementary Material.³⁵

TABLE II.

HSV parameters investigated.

$\alpha \hspace{0.17em}[°]$	0, 2.5, 5.0, 7.5, 10.0
dtemporal [a.u.]	1, 8, 16, 32, 64, 512, 1024
Frame rate [fps]	84 000, 10 500, 5250, 2625, 1312, 164, 82
$d_{\text{spatial}}$ [a.u.]	1, 4, 8, 16, 32
Magnification
factor [mm px−1]	≈ 0.0066, 0.026, 0.053, 0.11, 0.21

Mapping of the in silico model to the actual glottal width of the physical folds is imperfect owing to unaccounted for experimental uncertainties, including imperfect camera calibration, view angle setting, and stepper motor backlash and stiction. Direct comparison of the glottal area waveform amplitude from the BCM and the physical VFs at the highest spatial and temporal resolutions, and the aligned angle of view shows that the maximum discrepancy is less than 1% for all cases.

2. Glottal width extraction

Figure 3(a) shows a sample image extracted from a video recording. The dark columns are the VFs, while the light portion in the middle is the glottal area. The two-dimensional nature of the experimental facility is evident in this figure, which exhibits negligible row-to-row variability. Therefore, glottal width is used as the observation (measurement) data for the estimation procedure instead of the glottal area.

FIG. 3.

(Color online) (a) Sample image extracted from a sample video recording. (b) The Laplacian computed using a 3 × 3 px kernel over row 100 of the image in (a). The open and closed circles are peaks of the Laplacian.

The glottal width is extracted from each row of every video frame using an edge detection algorithm based on zero crossings of the Laplacian, computed through a 3 × 3 px kernel, see Fig. 3(b). A linear fit between adjacent peaks of the Laplacian is used to determine the zero crossing points of the Laplacian with sub-pixel precision. The average width for a given frame is stored as the measurement for that frame. Data from the full frame sequence yields the glottal width waveform. Note that all glottal widths presented are scaled from the measured values to physiological values, that is divided by a factor of 15.

B. Bayesian inference

Following Hadwin et al.,¹⁵ Bayesian inference is used to estimate P_sub and a_ct of the BCM from the glottal width waveform observation data extracted from the simulated HSV. Bayesian inference computes the posterior density, given by¹⁷

$${\pi }_{\text{post}}(\mathbf{x}|\mathbf{y})=\frac{{\pi }_{\text{like}}(\mathbf{y}|\mathbf{x}){\pi }_{\text{pri}}(\mathbf{x})}{{\pi }_{\text{evi}}(\mathbf{y})},$$ (2)

where π_post, π_like, and π_pri are the posterior, prior, and likelihood densities, respectively. The x term is a vector of parameters (P_sub, a_ct, and possibly α in the present case), y is a vector of measurements (i.e., glottal widths), and π_evi is a normalizing constant that ensures the law of total probability.

The present work is concerned with how measurement errors and biases present in HSV affect the resultant estimates. In the Bayesian framework, such effects are propagated through the likelihood density, as it embodies how our model connects to the measurements. Thus, this work employs uninformative priors and focuses on the likelihood density. Specifically, an additive Gaussian error model is used, resulting in a likelihood of the form¹⁵

$${\pi }_{\text{like}}\left(\mathbf{y}|\mathbf{x}\right)={\left(2\pi {\sigma }^2\right)}^{-k/2}\hspace{0.17em}\text{exp}\hspace{0.17em}\left(\frac{-1}{2{\sigma }^2}{\Vert \mathbf{y}-F(\mathbf{x})\Vert }^2\right),$$ (3)

where k is the number of elements in y, σ is the standard deviation of the measurement noise, and F(x) represents the forward model, which in this case is the glottal width computed from the BCM described in Sec. II A. The first three oscillations are excluded to remove transients in the glottal width waveform and the first data point in the waveform is set at a peak. The glottal widths for each frame are then calculated as

$$y_{i,\text{model}}=\text{max}(2\text{min}(x_{\mathrm{u},i},x_{\mathrm{l},i}),0),$$ (4)

where i is the time index for the forward model, corresponding to a given frame of the video. Note the glottal width waveform extracted from the video was similarly clipped and the signals aligned in time through cross-correlation.

The measurement noise standard deviation was conservatively chosen as a constant value of σ = 0.12 mm, corresponding to approximately 10% of the maximum glottal width at the highest resolution; a scaling factor is applied at the lower resolutions. To measure the effect of resolution on the measurement uncertainty (the scaling factor), the variance of the glottal width over rows of the image was computed for each frame and averaged over all frames for all of the acquired videos. Figure 4 shows the relative measurement uncertainties follow an approximately linear increase in the uncertainty with respect to d_spatial. The final downsampling factor was not used for fitting since it results in reduced glottal width variances, a result of non-linear effects at very poor spatial resolutions. The computed rate of linear increase, found from a linear fit to the data in Fig. 4, is then used as the scaling factor, leading to the measurement noise standard deviation

$$\sigma =0.12\sqrt{1+0.11(d_{\text{spatial}}-1)}.$$ (5)

FIG. 4.

(Color online) The time averaged variance of the glottal width over rows of frames normalized by the variance at the highest resolution for various cases of known parameters and angles of view. Each point corresponds to the time averaged variance computed for one of the known $(P_{\text{sub}},a_{\text{ct}})$ cases at each of the angles of view $\alpha =(0°,2.5°,5.0°)$.

To compute estimates, a simple direct approach is pursued wherein Eq. (3) is evaluated over a grid of parameters. Since a maximum of three parameters are considered here (P_sub, a_ct, and α), the approach is computationally tractable. The estimated posteriors are characterized using a point and uncertainty estimate. The maximum likelihood estimate (MLE) is used to compute parameter estimates, and the posterior covariance matrix is used to quantify uncertainty in the estimated values.

C. Measuring Changes in uncertainty

Uncertainty in the estimated parameter values is quantified in terms of variance. Specifically, the covariance matrix is computed as

$$\Sigma ={\int }_{\Omega }\left(\mathbf{x}-\overline{\mathbf{x}}\right){\left(\mathbf{x}-\overline{\mathbf{x}}\right)}^{\mathrm{T}}{\pi }_{\text{post}}(\mathbf{x}|\mathbf{y})\hspace{0.17em}\mathrm{d}\mathbf{x},$$ (6)

where $\Omega =[0,\hspace{0.17em}\infty )\times [0,\hspace{0.17em}1]$ is the domain of x, and $\overline{\mathbf{x}}$ denotes the posterior mean. This covariance matrix contains information about the uncertainty in all estimated parameters, as well as how the estimates of the parameters vary together subject to small changes in the measurements. A measure of the total uncertainty is computed as

$$u=\sqrt{\text{det}\left(\Sigma \right)}=\sqrt{{\lambda }_1{\lambda }_2},$$ (7)

where Σ is computed using Eq. (6), and λ₁, λ₂ are the eigenvalues of Σ. This measure is a geometric average of the total spread of the estimated posterior density.

Since each posterior depends on the level of measurement noise σ inherent to a particular imaging system/configuration, a fixed level of measurement noise σ = 0.012 cm is assumed for all videos (except under spatial resolution changes) to facilitate direct comparisons between cases at the same spatial resolution without the additional confound of varying noise levels. Relative changes in uncertainty with respect to the highest resolution source video (no downsampling of frame rate or resolution) at α = 0° are reported to elucidate trends with respect to the investigated imaging parameters, computed as

$$u_{\text{rel}}(\alpha ,d_{\text{spatial}},d_{\text{temporal}})=\frac{u(\alpha ,d_{\text{spatial}},d_{\text{temporal}})}{u(0°,1,1)}.$$ (8)

D. Laplace approximation

To analyze the effect of frame rate, resolution, and angle on the uncertainty, we use the Laplace approximation for the posterior and decompose it into contributions from the three respective factors. The Laplace approximation computes the posterior through approximation by a local Gaussian about the MLE estimate³⁶ using the Jacobian of the forward model, F. This results in an approximation for the covariance matrix given by

$$\Gamma \approx {\sigma }^2{\left(J^{T}J\right)}^{-1},$$ (9)

where Γ is the approximate covariance matrix of the posterior, σ is the standard deviation of the measurement error, and J is the Jacobian of F calculated at the MLE. The Jacobian measures the sensitivity of the glottal width to the parameters for each measurement. This approximation is used later to illustrate the relative effects of uncertainty between the three imaging variables.

E. Accounting for viewing angle

Intuitively, off-axis viewing angles are expected to reduce the observed glottal width, which will in turn bias parameter estimates. To mitigate this bias, viewing angle can be included as a parameter to be estimated, which necessitates a viewing model that relates viewing angle to the glottal width observed by the camera. To do this, we note that the visible points on the VFs in the BCM are the medial surface corners of the upper and lower masses, that have (x, z) coordinates (see Fig. 2) given by

$$\left\{(x_{\mathrm{l}},0),(x_{\mathrm{l}},T_{\mathrm{l}}),\hspace{0.17em}(x_{\mathrm{u}},T_{\mathrm{l}}),(x_{\mathrm{u}},T_{\mathrm{u}}+T_{\mathrm{l}})\right\}.$$ (10)

Since the distance between the inferior and superior edges is small, we can neglect the effect of perspective. Accounting for perspective could be done using a pinhole camera model for example; however, this requires the focal length of the camera and the camera location relative to the VFs to be known, which adds additional parameters to the estimation procedure, increasing computational complexity and parameter uncertainty. For simplicity, we employ a parallel projection model.

For an image plane with angle α, the projected distance of a point on the image plane from the origin is the dot product with the unit vector of the image plane $[\text{cos}\hspace{0.17em}(\alpha ),-\text{sin}\hspace{0.17em}(\alpha )]$. The projected BCM medial surface point locations are then

$$\left\{\begin{array}{c}{x}_{\mathrm{l}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha ),\hfill \\ x_{\mathrm{l}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-T_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha ),\hfill \\ x_{\mathrm{u}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-T_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha ),\hfill \\ x_{\mathrm{u}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-(T_{\mathrm{u}}+T_{\mathrm{l}})\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha )\hfill \end{array}\right\}.$$ (11)

Projected points on the opposing VF can be calculated by negating the x coordinates, since symmetric oscillations are assumed. The edges that are viewed in the image are then the edge points with the minimum/maximum values; all other points on the VFs will not be visible. As a result, the apparent glottal width can be expressed as

$$\begin{array}{c}\text{min}\left\{\begin{array}{c}{x}_{\mathrm{l}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha ),\hfill \\ x_{\mathrm{l}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-T_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha ),\hfill \\ x_{\mathrm{u}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-T_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha ),\hfill \\ x_{\mathrm{u}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-(T_{\mathrm{u}}+T_{\mathrm{l}})\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha )\hfill \end{array}\right\}\\ -\text{max}\left\{\begin{array}{c}-x_{\mathrm{l}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha ),\hfill \\ -x_{\mathrm{l}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-T_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha ),\hfill \\ -x_{\mathrm{u}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-T_{\mathrm{l}}\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha ),\hfill \\ -x_{\mathrm{u}}\hspace{0.17em}\text{cos}\hspace{0.17em}(\alpha )-(T_{\mathrm{u}}+T_{\mathrm{l}})\hspace{0.17em}\text{sin}\hspace{0.17em}(\alpha )\hfill \end{array}\right\},\end{array}$$ (12)

where min and max account for the viewed edges between the left and right VFs, respectively, for the coordinate system shown in Fig. 2. By incorporating this model of projection into the BCM, α can be treated as an unknown parameter to be estimated.

III. RESULTS AND DISCUSSION

Figure 5 shows the glottal width waveform extracted from the highest resolution video for $(P_{\text{sub}},a_{\text{ct}})=(1800\hspace{0.17em}\text{Pa},0.15)$. As discussed in Sec. II B, the initial few oscillations have been removed and time 0 is set when the waveform is at a maximum. The glottal width never reaches zero because of the aforementioned buffer, imposed to prevent collision of the physical VFs. All four cases listed in Table I exhibit similar glottal width waveforms, albeit with differing oscillation frequencies and waveform amplitudes. A high degree of periodicity is observed in all cases.

FIG. 5.

(Color online) Sample glottal width waveform extracted from HSV for $(P_{\text{sub}},a_{\text{ct}})=(1800\hspace{0.17em}\text{Pa},0.15)$ in physiological dimensions.

A. Viewing angle

As the viewing angle increases from α = 0°, the apparent glottal width observed by the camera decreases, as shown in Fig. 6. This effect is marginal for viewing angles below 5°, but becomes pronounced as alpha increases to 10°. This is due to two main effects. The first is a geometric projection; the area projected onto the image plane is multiplied by a factor of $\text{cos}\hspace{0.17em}(\theta )$. Under the small angle offsets that occur in clinical HSV³² and employed herein ($\alpha \le 10°$), this effect is small. The second, more significant, effect is due to imaging different VF edges, either superior or inferior. For α = 0°, the observed glottal width is always between either both inferior or both superior margins of the VFs (except for very specific times when these two edges coincide in the image plane). However, with off-axis viewing, the observed glottal width may be between inferior-superior (or vice versa) edges; that is, under certain conditions, the superior edge can block the view of the inferior edge that would otherwise be observed when α = 0°.

FIG. 6.

(Color online) Observed glottal width as a function of camera viewing angle α for (P_sub, a_ct) = (1800 Pa, 0.15). Angles of view correspond to: –––– $0°$, - - - $5°$, -·-· $10°$.

1. Estimation excluding viewing angle

In this section, we estimate subglottal pressure and muscle activation without the inclusion of viewing angle as a parameter. Figure 7 presents the estimated parameters for all four cases in Table I for α ranging from 0° to 10° with the highest temporal and spatial resolution. Figures 7(a) and 7(b) shows that, as α increases, the estimated value of P_sub decreases and a_ct slightly increases. For α > 0°, the apparent glottal width decreases without impacting the fundamental frequency (though the waveform shape can be affected), which results in the observed decrease in estimated P_sub with increasing α. Due to the dynamics of the BCM, decreasing P_sub leads to a decrease in frequency. Since the frequency of vibration remains constant as the viewing angle is changed, however, the decrease in frequency must be compensated for by a small increase in a_ct. Note that the deviations in these estimates from the values at α = 0° constitute a bias in the estimates.

FIG. 7.

(Color online) The estimated (a) P_sub and (b) a_ct with increasing viewing angle. (c) The relative uncertainty with increasing viewing angle. Known reference parameters are: (P_sub, a_ct) = –––– (1800 Pa, 0.15), - - - (2000 Pa, 0.15), -·-· (1800 Pa, 0.20), …. (2000 Pa, 0.20).

The relative uncertainty of the estimated parameters increases as α increases, as seen in Fig. 7(c). This is a result of the posterior being calculated about the biased estimates; the observed trend in relative uncertainty is a product of the BCM behavior at the biased parameter estimates due to α, and not the change in α itself. It can be seen that while cases (P_sub, a_ct) = (2000 Pa, 0.15), (1800 Pa, 0.20), and (2000 Pa, 0.20) have similar trends in relative uncertainty, (1800 Pa, 0.15) does not due to the different biased parameter estimates of this case.

2. Estimating viewing angle

We now include α as an estimated parameter in an attempt to correct the bias observed in (Sec. III A 1). Considering the α = 0° and 7.5° cases with (P_sub, a_ct) = (1800 Pa, 0.15) as exemplars, we employ the parallel projection model (see Sec. II E) to estimate the viewing angle. Table III summarizes mean estimates and marginal standard deviations with and without α as an estimated parameter. Note that in marginal likelihoods, the effect of the other variables are removed by integrating the likelihood over those variables.¹⁷ Thus, each marginal likelihood represents the likelihood for the associated parameter, independent of the other parameters. At α = 0°, Table III shows that estimates both with and without the angle of view agree reasonably well with the known parameter values. When the angle of view is included as an estimated parameter, a slight improvement in the estimates P_sub is observed. However, a slight increase in uncertainty is observed on the estimated parameters due to the larger number of parameters; for the same model, more parameters generally results in more ways for the model to fit a measurement and therefore higher uncertainty. When α = 7.5°, Table III shows significant improvements in estimates. When the angle of view is not estimated, P_sub is estimated as 1644 Pa due to the apparent decrease in glottal width from the offset angle of view. This bias of over 8% is effectively eliminated when α is included as an estimated parameter. While uncertainty was seen to increase for both P_sub and a_ct for the α = 0° case, uncertainties do not show consistent increases at α = 7.5°. This is due to the correction in the bias, which results in the covariance being computed about different parameter sets.

TABLE III.

Parameter estimates with (shaded) and without (not shaded) estimating angle of view at $\alpha =0°,7.5°$ for the case $(P_{\text{sub}},a_{\text{ct}})=(1800\hspace{0.17em}\text{Pa},0.15)$. Marginal standard deviations are shown in parentheses.

$\alpha \hspace{0.17em}[°]$	Psub (σ) [Pa]	act (σ)	α(σ)
0	1793 (1.8)	0.151 (8.5 × 10−5)	-
0	1805 (2.4)	0.151 (9.1 × 10−5)	0.36 (0.25)
7.5	1644 (1.9)	0.157 (1.0 × 10−4)	-
7.5	1832 (2.0)	0.150 (7.4 × 10−5)	6.3 (0.04)

We note that the known estimates are not always within the uncertainties of the posterior due to the fact that some modelling errors were neglected in this study. These include errors in the stepper motor motion system, mapping from the BCM to the rigid VFs, and experimental calibration. For example, at α = 7.5° the estimated α is about $6.3\pm 0.04°$, clearly not containing the true value. This error could be due, at least in part, to uncertainty in the angular placement of the camera in the experiment. Although the angle was nominally placed at 7.5°, errors in the calibration procedure results in some uncertainty in this value. Herein, the focus is purely on the effects of the imaging system and thus no attempt has been made to identify or correct for these unmodelled errors.

A further note, the ability to estimate viewing angle in this case is due to the changes in shape of the glottal width waveform that arise due to changes in viewing angle. This requires that the model behavior be a good representation of the underlying physics; that is, changes in the observed glottal waveform are consistent between the model and the observation. While this is achieved in the current work due to the use of a reduced-order model for the control and fitting models, application of the method for clinical HSV requires deeper evaluation, which is left for future work.

B. Frame rate

As the frame rate is decreased the number of time points used to sample the glottal width waveform decreases. Figure 8 presents the estimated parameters for the four different cases in Table I at different frame rates with α = 0° and full spatial resolution. Figures 8(a) and 8(b) show that the estimate is unaffected by the decreasing frame rate, down to a frame rate of about 300 fps, which corresponds approximately with the Nyquist frequency for the observation data. For frame rates around or below the Nyquist frequency, we expect the estimates to deteriorate due to aliasing of the glottal width waveform. We note that the majority of high-speed imaging systems employed to record VF motion have frame rates in excess of 1000 fps, which is well above the typical VF oscillation frequencies in modal voice and thus aliasing artifacts evident in the estimations for very low frame rates are not expected to be of practical significance.

FIG. 8.

(Color online) The estimated (a) P_sub and (b) a_ct with decreasing frame rate. (c) The relative uncertainty with decreasing frame rate. Known reference parameters are: $(P_{\text{sub}},a_{\text{ct}})=$ –––– (1800 Pa, 0.15), - - - (2000 Pa, 0.15), -·-· (1800 Pa, 0.20), …. (2000 Pa, 0.20).

The relative uncertainty, presented in Fig. 8(c), is inversely proportional to the frame rate, wherein halving the frame rate doubles the uncertainty down to the Nyquist frequency. The reason for this linear trend can be analyzed using the Laplace approximation given in Eq. (9). Frame rate influences uncertainty by removing measurements, which reduces the number of rows in the Jacobian. Suppose we are downsampling by a factor of 2; that is, every second frame is retained (all the even frames). To see the impact of this on the Jacobian, we reorganize the rows of J. Specifically, all the even rows (the rows of those frames being kept) are placed at the top and all the odd rows (the frames being dropped) at the bottom. Thus, we can represent the original Jacobian product as

$$\begin{array}{cc}{J}^{T}J& ={\left[\begin{array}{c}{J}_{\mathrm{e}}\\ J_{\mathrm{o}}\end{array}\right]}^T\left[\begin{array}{c}{J}_{\mathrm{e}}\\ J_{\mathrm{o}}\end{array}\right]=J_{\mathrm{e}}^T{J}_{\mathrm{e}}+J_{\mathrm{o}}^T{J}_{\mathrm{o}}\end{array},$$ (13)

where J_e is the Jacobian for the downsampled video. Since the underlying forward model behavior is smooth in time and the time step is small, it follows that J_e and J_o are approximately equal. Therefore, $J_{\mathrm{e}}^T{J}_{\mathrm{e}}\approx (1/2)J^{T}J$. Thus, the covariance matrix for the downsampled case (${\Sigma }_{\text{ds}}$) can be expressed as

$${\Sigma }_{\text{ds}}={\sigma }^2{\left(J_{\mathrm{e}}^T{J}_{\mathrm{e}}\right)}^{-1}\approx {\sigma }^2{\left(\frac{1}{2}{J}^{T}J\right)}^{-1}\approx 2\Sigma .$$ (14)

This shows that halving the number of frames halves the covariance; in other words, the covariance has an inversely proportional relationship with the number of frames. Using this in the measure of total uncertainty [Eq. (7)] yields

$$\left|{\Sigma }_{\text{ds}}\right|=2^m\left|\Sigma \right|,$$ (15)

where m is the number of parameters being estimated. Thus, the relative uncertainty [Eq. (8)] becomes

$$u_{\text{rel}}=\sqrt{\left|{\Sigma }_{\text{ds}}\right|}/\sqrt{\left|\Sigma \right|}=\sqrt{2^m}.$$ (16)

In the present study, m = 2 as two parameters were estimated, which means that for a reduction in frame rate by a factor of 2, the relative uncertainty increases by 2, as was observed in Fig. 8(c). Similar to the MLE, this linear relationship no longer holds below the Nyquist frequency since the glottal width is not accurately captured, which invalidates the linear assumption of the Laplace approximation.

C. Spatial resolution

As the spatial resolution of the imaging system decreases, the apparent glottal width is slightly decreased in some regions and increased in others. Figure 9 shows the glottal width waveform for various degrees of downsampling. The waveforms obtained from the high resolution videos generally agree, while the apparent glottal width is biased to be less than its true value at the peaks, and overestimated in the troughs for low resolution videos. At the highest magnification factor (lowest resolution), significant errors are observed. This is expected since low resolution images will result in more substantial blurring of the VF edge, reducing edge detection accuracy.

FIG. 9.

(Color online) The measured glottal width waveform for various degrees of spatial downsampling. Each resolution corresponds to a magnification factor of approximately: –––– 0.013 mm px⁻¹, - - - 0.053 mm px⁻¹, -·-· 0.21 mm px⁻¹ for (P_sub, a_ct) = (1800 Pa, 0.15).

The effect of spatial resolution on the MLE and relative uncertainty is shown in Fig. 10 for all cases with fixed frame rate ($d_{\text{temporal}}=1\hspace{0.17em}[a.u.]$) and α = 0°. Figures 10(a) and 10(b) show that at high spatial resolutions, the estimates are in agreement with the “ground truth” since the resolution was sufficient to capture the glottal width without biasing. At lower spatial resolutions however (magnification factors greater than $0.053\hspace{0.17em}\text{mm}\hspace{0.17em}\hspace{0.17em}{\text{px}}^{-1}$), the measured glottal width is biased, leading to an underestimation of P_sub, whereas a_ct increases slightly to compensate for the reduced frequency induced by the lowered P_sub. Similar to the cases with downsampling of the frame rate, downsampling the spatial resolution of the video results in an increase in the relative uncertainty. This is the result of the scaling factor applied to σ (described in Sec. II B). Linearity is seen for small magnification factors (from 0.007 to 0.027 mm px⁻¹), but increases for larger magnifications factors due to the biased parameters. The relative uncertainties for all four cases are nearly identical.

FIG. 10.

(Color online) The estimated (a) P_sub and (b) a_ct, and (c) relative uncertainty with decreasing spatial resolution. Known reference parameters are: (P_sub, a_ct) = ––––– (1800 Pa, 0.15), - - - (2000 Pa, 0.15), -·-· (1800 Pa, 0.20), …. (2000 Pa, 0.20). Note that the x axis corresponds to different downsampling factors of the spatial resolution.

D. Combined effects of frame rate, resolution, and viewing angle

Figure 11 shows the MLE parameter estimates for the case (a_ct, P_sub) = (0.15, 1800 Pa), under combined changes in resolution, frame rate, and viewing angle. Figure 11 illustrates how the MLE estimate is affected by simultaneous changes in all three imaging parameters. Within the region where resolution is less than 0.053 mm px⁻¹ and frame rate is above 1000 fps, MLE estimates are approximately constant with respect to changes in frame rate and resolution since the glottal width waveform is well resolved. As the viewing angle is offset, the MLE estimate becomes biased but remains constant at different frame rates and resolutions. This indicates that the MLE estimate is only dependent on viewing angle for sufficiently high fidelity videos.

FIG. 11.

(Color online) The estimated (a) P_sub and (b) a_ct. The known reference parameters are: $(P_{\text{sub}},a_{\text{ct}})=(1800\hspace{0.17em}\text{Pa},0.15)$ and viewing angles are: ––––– $0°$, -·-· $7.5°$, …. $10.0°$.

When the resolution is greater than 0.053 mm px⁻¹, the glottal width waveform is aliased and, as a result, errors are introduced in the MLE estimate. For α = 0°, decreasing resolution decreases P_sub, with the exception of a few outliers at the lowest frame rates. This effect is due to underestimated vibration amplitudes (Fig. 9), which leads to underestimating P_sub. For α = 7.5° and 10°, decreasing resolution increases P_sub. The reason can be seen from Fig. 12 showing the glottal width at low and high resolutions at α = 10°. In contrast to the aligned angle of view, the glottal width amplitude is not underestimated at the offset angle of view, but rather overestimated near glottal closure.

FIG. 12.

(Color online) The measured glottal widths at the highest —- and lowest -·-· resolutions at $\alpha =10°$ for (P_sub, a_ct) = (1800 Pa, 0.15).

When frame rate is less than 1000 fps, there is temporal aliasing of the glottal width. For example, at a frame rate of 82 fps, a mere ten points are used to resolve 16 oscillations. This leads to large errors in the MLE estimates since multiple waveforms can fit these discrete points satisfactorily. Figure 11 shows that at α = 0° these low frame rates decrease P_sub, while at α = 10° low frame rates increase P_sub. At α = 7.5°, there is a non-monotonic impact of P_sub at low frame rates. In some cases, the additional error induced by the poor frame rate counteracts the error induced by the viewing angle. These are just unique cases, however, specific to the particular measurements and parametrizations, as can be seen by comparing the α = 7.5° and 10° cases. For different known parameters and recordings, this trend will not be repeated consistently.

Figure 13 shows how relative uncertainty [Eq. (8)] is affected by simultaneous changes in all three parameters. First, it can be seen that relative uncertainty scales exponentially with frame rate (linearly on the log-log plot in Fig. 13). As the resolution is decreased, the scaling with frame rate shifts up by a constant. In a log-log plot, this illustrates a uniform multiplicative relationship induced by changes in resolution. At the lowest frame rates, small deviations from the linear behavior are seen, which are due to biases in the MLE estimate seen in Fig. 11. Comparing Fig. 11(a) with 11(b) shows that the linear relationship with respect to frame rate still holds at offset viewing angles but with an increase in relative uncertainty compared to the aligned viewing angle. The fact that relative uncertainty increases from the viewing angle is due to the biased MLE estimate. Changes in resolution at the offset viewing angle shift the relative uncertainty upwards on the log-log plot, thus showing the multiplicative relationship between resolution and frame rate. The reason for the multiplicative combination can be seen from the Laplace approximation [Eq. (9)]. The total uncertainty is the product of a Jacobian term and the σ term that are influenced solely by resolution and frame rate, respectively. While the Jacobian term depends on the MLE and therefore could be affected nonlinearly by the viewing angle, Fig. 11 shows that the MLE is nearly constant for sufficiently high frame rates and resolution. Thus there is no additional affect by the viewing angle.

FIG. 13.

(Color online) The relative uncertainty at (a) $\alpha =0°$ and (b) $\alpha =10°$ with changing spatial and temporal resolutions for the case $(P_{\text{sub}},a_{\text{ct}})=(1800\hspace{0.17em}\text{Pa},0.15)$. Different spatial resolutions correspond to: –––– ≈ 0.0066 [mm px⁻¹], -·-· ≈ 0.053 [mm px⁻¹] and …. ≈ 0.21 [mm px⁻¹].

The multiplicative relationship suggests that estimate uncertainty can be reduced while using the same camera by increasing frame rate and, to a lesser degree, improving spatial resolution. Many high-speed cameras have the ability to increase frame rate at the expense of spatial resolution to remain within a given memory bandwidth limit. That is, a camera capable of capturing one 1024 × 1024 image every second can, roughly, capture two 512 × 1024 images every second. Since the frame rate has considerably higher impact on uncertainty, this suggests that frame rates should be increased at the expense of spatial resolution for improved certainty on estimates, within reason. In this work, it was found that magnification factor (resolution) should be less than 0.053 mm px⁻¹, which corresponds to having more than ≈ 300 px over a nominal 15 mm glottal length. This recommendation is based on how resolution affects the ability to detect glottal width but does not take into account effects near glottal closure, where poor resolution can more substantially impact the glottal area waveform. Thus, for small glottal amplitudes where such effects will be present, one should apply a more conservative resolution. The combined effects for all cases and conditions are available in Supplementary Materials.³⁵

There are a number of limitations associated with this study. First off, we consider only three imaging effects (frame rate, resolution, and viewing angle), but a number of other factors also affect HSV quality and associated measured glottal area waveforms, such as lighting conditions, camera motion, and view obstruction by the arytenoid hooding, which will be explored in future work. Another limitation is that only two parameters from the BCM were estimated. A larger numbers of parameters pose a number of challenges to the estimation, including broad credibility intervals and a larger computational expense. Future work will explore the use of prior information or additional measurements, such as glottal flow rate, to expand the estimated parameter set while maintaining meaningful credibility intervals.

IV. CONCLUSIONS

This study investigated the effect of three HSV imaging variables on Bayesian inference: the viewing angle, and frame rate and spatial resolution of the imaging device, and methods are proposed to mitigate issues arising from these variables. For the viewing angle, offset angles of view lead to biased parameters due to an underestimated glottal width. A parallel projection model for incorporating viewing angle as a parameter to be estimated was presented and shown to alleviate the bias.

The frame rate and spatial resolution of the camera were found to have relatively small biasing effects on the estimated parameters for clinical-grade high-speed cameras. However, they did influence the level of uncertainty in the estimate. In particular, increases in frame rate decreased uncertainty in estimates more than equivalent increases in spatial resolution. Under combined changes in frame rate and spatial resolution, it was seen that uncertainty in estimates grew multiplicatively with respect to the two changes. That is, if a decrease in spatial resolution increased uncertainty by a factor of 2 and a reduction in frame rate increased uncertainty by a factor of 1.5, the total change in uncertainty under both spatial and temporal resolution changes would be a factor of 3 (2 × 1.5). This is expected to hold up to a limit where the decreased spatial resolution results in highly inaccurate edge detection. Spatial resolution had a considerably lower impact on uncertainty, and, therefore can be sacrificed to increase frame rate for improved estimates. The exact limit to which this can be done and the specific trade-off in uncertainty, however, will likely vary for individual imaging scenarios.