Reliability and Diagnostic Accuracy of Semi-Automated and Automated Acoustic Quantification of Vocal Tremor Characteristics

Abstract

Purpose:

This study compared three methods of acoustic algorithm-supported extraction and analysis of vocal tremor properties (i.e., rate, extent, and regularity of intensity level and fundamental frequency modulation): (a) visual perception and manual data extraction, (b) semi-automated data extraction, and (c) fully automated data extraction.

Method:

Forty-five midvowel sustained [a:] and [i:] audio recordings were collected as part of a scientific project to learn about the physiologic substrates of vocal tremor. This convenience data set contained vowels with a representative variety in vocal tremor severity. First, the vocal tremor properties in intensity level and fundamental frequency tracks were visually inspected and manually measured using Praat software. Second, the vocal tremor properties were determined using two Praat scripts: automated with the script of Maryn et al. (2019) and semi-automated with an adjusted version of this script to enable the user to intervene with the signal processing. The reliability of manual vocal tremor property measurement was assessed using the intraclass correlation coefficient. The properties as measured with the two scripts (automated vs. semi-automated) were compared with the manually determined properties using correlation and diagnostic accuracy statistical methods.

Results:

With intraclass correlation coefficients between .770 and .914, the reliability of the manual method was acceptable. The semi-automated method correlated with manual property measures better and was more accurate in diagnosing vocal tremor than the automated method.

Discussion:

Manual acoustic measurement of vocal tremor properties can be laborious and time-consuming. Automated or semi-automated acoustic methods may improve efficiency in vocal tremor property measurement in clinical as well as research settings. Although both Praat script-supported methods in this study yielded acceptable validity with the manual data measurements as a referent, the semi-automated method showed the best outcomes.

Supplemental Material:

https://doi.org/10.23641/asha.28873088

Tremor is a neurological movement disorder typified by involuntary and approximately rhythmic movements of one or more body parts (Crawford & Zimmerman, 2011; Elble, 2016; Louis, 2016). When tremor affects muscles of the speech system, it can result in a vocal tremor. Vocal tremor is a neurological voice disorder that results from tremor observed within speech structures and is typically associated with other neurological disorders, such as essential tremor, Parkinson's disease, Huntington's disease, amyotrophic lateral sclerosis, multiple sclerosis, dystonia, multiple system atrophy, cerebellar degeneration, progressive supranuclear palsy, and stroke (Hlavnička et al., 2020; Sulica & Louis, 2010; Wolraich et al., 2010). Torrecillas et al. (2021) characterized the clinical phenotypes for the various types of vocal tremor using combined tremor classification criteria from the International Parkinson and Movement Disorder Society (IPMDS; Bhatia et al., 2018) and laryngeal features from the American Academy of Otolaryngology–Head and Neck Surgery (AAO-HNS; Merati et al., 2005). Several studies have documented the association of tremulous voice modulation with oscillatory movements of the tongue (Bove et al., 2006; Gamboa et al., 1998; Jiang et al., 2000; Lester et al., 2013; Sulica & Louis, 2010), larynx (Ackermann & Ziegler, 1991; Bove et al., 2006; Finnegan et al., 2003; Gamboa et al., 1998; Koda & Ludlow, 1992; Sulica & Louis, 2010; Tomoda et al., 1987), pharyngeal constrictors (Bove et al., 2006; Lester et al., 2013; Sulica & Louis, 2010), soft palate (Bove et al., 2006; Gillivan-Murphy et al., 2016; Panda et al., 2013; Sulica & Louis, 2010), and respiratory musculature (Hachinski et al., 1975; Tomoda et al., 1987). Vocal tremor has been shown to significantly affect communication, participation, quality of life, and well-being (Dromey et al., 2002; Gillivan-Murphy et al., 2019; Mendoza & Carballo, 1999; Merati et al., 2005; Sulica & Louis, 2010).

To document vocal tremor based on the acoustic speech signal, the signal first needs to be decomposed into vocal tremor–related features, which typically are fundamental frequency (f₀) and sound intensity level (IL). Once isolated, these time-varying features can be inspected for the presence of a more or less cyclic modulation with a rate between 3 and 12 Hz (Duffy, 1995). Subsequently, the modulation properties rate (i.e., frequency), extent (i.e., depth or amplitude), and regularity (i.e., stability or periodicity) can be described as measurements of vocal tremor. Maryn et al. (2019) explored the concurrent and diagnostic validity of software to automatically decompose the time-varying features of speech signals and determine tremor modulation properties. Such measurement is regarded as pertinent and clinically useful for multiple reasons. That is, measuring vocal tremor modulation properties has potential for use in comparisons and differential diagnosis of patient groups, such as healthy individuals, and those with Parkinson's disease, essential vocal tremor, laryngeal dystonia, Huntington's disease, and myotonic dystrophy (Barkmeier-Kraemer et al., 2018; Dominguez et al., 2018; Gillivan-Murphy et al., 2019; Ramig et al., 1988; Tanaka et al., 2011). Measuring vocal tremor properties also enables clinicians to track the severity of tremor affecting the voice over time during clinical treatment. Current treatment approaches for vocal tremor include (a) injection of botulinum toxin into laryngeal or strap musculature (Adler et al., 2004; Bove et al., 2006; Gurey et al., 2013; Kendall & Leonard, 2011; Nelson et al., 2019; Sulica & Louis, 2010; Warrick et al., 2000); (b) intake of systemic medication (Julius & Longfellow, 2016; Khoury & Randall, 2022; Ondo, 2016), such as primidone (Hartman & Vishwanat, 1984; Nida et al., 2016), propanolol (Koller, 1985; Koller et al., 1985), methazolamide (Busenbark et al., 1996; Muenter et al., 1991), sodium oxybate (Simonyan & Frucht, 2013), octanoic acid (Lowell et al., 2019, 2021), and cannabinoids (Millman et al., 2023); (c) deep-brain stimulation, with described beneficial effect when applied on ventro-intermediate thalamic nucleus (Erickson-DiRenzo, Kuijper, et al., 2020; Erickson-DiRenzo, Sung, et al., 2020; Ho et al., 2019; Lu et al., 2023; Ruckart et al., 2023), caudal zona incerta (Hägglund et al., 2016), and ventro-latero-posterior thalamic nucleus (Matsumoto et al., 2016); and (d) behavioral speech/voice therapy with strategies for reducing perceived vocal tremor (Barkmeier-Kraemer, 2013, 2016, 2019; Barkmeier-Kraemer et al., 2011; Ghosh et al., 2023). In addition, preliminary findings have shown that the voice symptoms in those with essential vocal tremor may be altered during sensorimotor modulation, such as transcutaneous laryngeal vibrotactile stimulation (Dwenger, 2022, 2024). Regardless of the treatment approach, however, it is important to quantify the changes in the properties and severity of vocal tremor and to follow up on its treatment outcomes.

Various clinical and scientific methods have been applied to the differential diagnosis and documentation of vocal tremor (Maryn et al., 2019), including auditory–perceptual, visual–perceptual/spectrographic, acoustic, aerodynamic, and laryngoscopic methods. However, acoustic methods are of particular interest, considering its noninvasiveness, relatively low cost, ease of application, quantitative output, mostly objective protocol, and potential for automatic or semi-automatic extraction and processing of different speech signal properties. Furthermore, clinical application of acoustic methods is relevant because it addresses the speech signal associated with the symptoms or perceived abnormal speaking patterns. Anand et al. (2012), for example, conducted an acoustic-perceptual study of vocal tremor and explored how perceived vocal tremor severity (PVTS) varies with f₀, rate and extent of modulation in f₀, and signal-to-noise ratio as an indicator of dysphonia severity. Their findings confirmed that the perceived severity of vocal tremor increases with extent of frequency modulation. They also found that, above a modulation rate of 8 Hz, the interaction between extent and rate of frequency modulation became relevant. However, only four tremorous speakers were studied, and IL modulation was not evaluated. Erickson-DiRenzo, Sung, et al. (2020) is another example of a recently published study in which an acoustic interface was used for the manual extraction of f₀ and IL modulation properties and measures, in this case for tracking seven patients' vocal tremor before, during, and after placement of a ventral intermediate nucleus stimulator. Their data also confirmed that perceived severity of vocal tremor increases predominantly with extent of f₀ modulation, as well as with extent of IL modulation. However, only seven tremorous speakers were studied. Finally, Maryn et al. (2019) studied the correlation between PVTS and vocal tremor properties (i.e., rate, extent, and regularity of f₀ and IL modulation) in 56 subjects with various degrees of vocal tremor. They, too, found extent of f₀ and IL modulation to provide the highest correlation with PVTS. Furthermore, via multivariate linear regression analysis including all properties, Maryn et al. (2019) constructed the Acoustic Phonatory Tremor Index (APTI), which showed a highly acceptable r_Spearman = .848 with PVTS.

The acoustic method as described in detail in Maryn et al. (2019) consisted of a completely automated set of algorithms to (a) isolate the time-varying f₀ and IL trajectories, (b) determine the properties of these trajectories, and (c) present graphically the trajectories, measured properties, and APTI within the Praat program. This tool is considered as objective as possible given it is entirely automated in which the user cannot interfere with the measurements once the algorithm is started. However, vocal tremor properties measured from manually delineated modulation cycles as, for example, in Erickson-DiRenzo, Sung, et al. (2020), can still be considered most trustworthy and, therefore, gold standard. Thus, the reliability of the output of any automated or semi-automated acoustic phonatory tremor analysis tool must be confirmed by comparing it with the output of manual methods. In other words, before clinically implementing an acoustic method for vocal tremor measurement, it is crucial to establish its accuracy as a more objective supplement or complement to a manual method. As such, a preliminary comparison between the fully automated tool from Maryn et al. (2019) and manual determination of vocal tremor properties in a few vowel recordings was completed; however, some discrepancies emerged. Figure 1 illustrates the output of a fully automated and accurate acoustic analysis. Rate of modulation was correctly estimated in the f₀ and IL traces with all modulation cycles accurately indicated. Thus, tremor extent and regularity were also accurately measured. However, Figure 2 shows the output of a fully automated yet inaccurate acoustic analysis due to an erroneous tremor rate in the irregular IL trace. Instead of 5.31 Hz as manually indicated (2), the program spuriously selected the spectral peak at 3.92 Hz, leading to faulty modulation cycle detection. Despite this error in the measurement of the IL trace, the tremor rate was accurate in the f₀ trace (1). Figure 3 displays the output of a fully automated but also inaccurate acoustic analysis due to erroneous modulation cycle choices. Based on manual annotation (1), the tremor rate indeed is circa 5.4 Hz (at least in the first second of the recording). However, some of the tremor cycles were erroneously located (2), resulting in an inaccurate tremor extent measurement. From these examples, it can be observed that errors occurred in small-ranged and/or irregular modulations: The IL contours in Figures 2 and 3 only had a range of 3.6 dB and 2.5 dB, respectively, and the f₀ trace in Figure 3 only had a range of 5.3 Hz. In contrast, the f₀ traces in Figures 1 and 2 had a range of 61.3 Hz and 21.0 Hz, respectively. The range of the IL trace in Figure 1 was not much larger with 4.6 dB but was relatively regular and, therefore, more accurate in detecting rate and modulation cycles. Because of these exemplary discrepancies, it was concluded that the completely automated method for acoustic vocal tremor analysis may not be sufficiently correct for clinical use and decision making. Thus, a semi-automated tool may be required and more appropriate, especially in cases where there is some irregularity within f₀ and IL modulation cycles. This conclusion initiated adjustments to the acoustic phonatory tremor analysis tool from Maryn et al. (2019), as described in the Methods section. Overall, the adjustments enable the clinician/user to intervene when required, for example, to omit erroneous cycles from the f₀ or IL modulations or to specify the tremor rate so it only includes the actual rate and not a period doubling-based rate.

Figure 1.

Illustration of correct vocal tremor analysis by the script of Maryn et al. (2019). f₀ = fundamental frequency; IL = intensity level.

Figure 2.

Illustration of erroneous tremor rate in the IL trace by the script of Maryn et al. (2019) in the first [i:] of Subject 2. (1) Accurate tremor rate in the f₀ trace. (2) Inaccurate tremor rate in the IL trace. f₀ = fundamental frequency; IL = intensity level.

Figure 3.

Illustration of erroneously located tremor cycles by the script of Maryn et al. (2019) in the first [i:] of Subject 6. (1) Accurate tremor rate. (2) Inaccurate location of four tremor cycles in the f₀ trace. f₀ = fundamental frequency; IL = intensity level.

The present study had two goals. First, the reliability of the automated and semi-automated acoustic tools in measuring the f₀ and IL modulation properties in vocal tremor was assessed based on correlation with manual acoustic measurement as the standard referent. Second, these tools' diagnostic/clinical validity in differentiating between manually determined absence or presence of vocal tremor was investigated using the receiver operating curve analysis.

Method

Experimental Ethics

Institutional review board approval for this study was obtained at the University of Utah (IRB Protocol 00118469).

Subjects

Fifteen adults enrolled in this institutional review board–approved research study at the University of Utah. Demographic and clinical characteristics are in Table 1. The subjects were identified for participation based on a diagnosis of essential tremor as previously documented in their medical records. Following informed consent to participate, subjects underwent a standard neurologic examination by a movement disorder specialist consisting of a physical examination for Parkinsonism using the Unified Parkinson's Disease Rating Scale, Part III (UPDRS Part III; Movement Disorder Society Task Force on Rating Scales for Parkinson's Disease, 2003), and an assessment of tremor using the Fahn–Tolosa–Marin Tremor Rating Scale (Fahn et al., 1988) to determine if criteria for the diagnosis of essential tremor or essential tremor plus were met. All 15 subjects in this study met criteria for a diagnosis of essential tremor as determined by the movement disorder specialist. The criteria used were based on the updated consensus statement by the IPMDS (Bhatia et al., 2018) and includes:

Table 1.

Demographic and clinical characteristics of the 15 participants in this study.

Characteristic	ET + VT	ET – VT	Total
n	10	5	15
Age in years, M (SD)	71 (5)	47 (4)	65 (12)
Gender, n (%)
Male	7 (70%)	5 (100%)	12 (80%)
Female	3 (30%)	0 (0%)	3 (20%)
Race, n (%)
White	7 (70%)	5 (100%)	12 (80%)
Asian	1 (10%)	0 (0%)	1 (7%)
African American	1 (10%)	0 (0%)	1 (7%)
American Indian/Alaska Native	0 (0%)	0 (0%)	0 (0%)
Native Hawaiian	0 (0%)	0 (0%)	0 (0%)
Did not disclose	1 (10%)	0 (0%)	1 (7%)
Ethnicity, n (%)
Hispanic or Latino	0 (0%)	0 (0%)	0 (0%)
Non-Hispanic	10 (100%)	4 (80%)	14 (93%)
Did not disclose	0 (0%)	1 (20%)	1 (7%)
Disease duration in years, M (SD)	27 (22)	33 (13)	28 (22)
Progression in disease, n (%)	5 (50%)	0 (0%)	5 (33%)
VHI 30-item total score, M (SD)	45 (33)	24 (34)	39 (37)
QUEST overall health status, M (SD)	81 (6)	63 (31)	76 (19)
QUEST quality of life, M (SD)	82 (13)	65 (29)	78 (21)
QUEST communication score, M (SD)	45 (29)	21 (22)	38 (32)
QUEST physical score, M (SD)	44 (20)	57 (33)	48 (27)

Note. ET = essential tremor; VT = vocal tremor; VHI = Voice Handicap Index; QUEST = Quality of Life in Essential Tremor Questionnaire.

1. bilateral upper limb action (kinetic and postural) tremor;

2. at least 3 years in duration;

3. with or without tremor in other locations (e.g., head, voice, or lower limbs);

4. absence of other neurologic signs, such as dystonia, ataxia, or Parkinsonism; and

5. essential tremor plus: tremor with the characteristics of essential tremor and additional neurological signs of uncertain significance (such as impaired tandem gait, questionable dystonic posturing, memory impairment, or other mild neurologic signs of unknown significance) that do not make an additional syndrome classification or diagnosis (including essential tremor with tremor at rest).

In addition, participants underwent a standard clinical voice assessment including patient questionnaires (e.g., 30-item Voice Handicap Index [VHI] and Quality of Life in Essential Tremor [QUEST]), acoustic and aerodynamic recordings, and nasoendoscopy to assess laryngeal structure and function and as part of the cranial nerve examination to determine the presence or absence of vocal tremor affecting speech structures associated with essential tremor. Ten (67%) subjects were classified with vocal tremor. Guidance for classification of tremor affecting the upper airway structures was taken from the Neurolaryngology Committee of the American Academy of Otolaryngology—Head and Neck Surgery report by Merati et al. (2005). This report included the following:

1. idiopathic action tremor affecting the voice as part of essential tremor;

2. tremor affecting muscles of the larynx, pharynx, palate, other articulators, and/or respiratory musculature; and

3. tremor is not task specific (occurs during phonation and respiration).

Additional inclusion criteria for the present study were:

1. no botulinum toxin injections within 3 months of participation with a return of at least 50% of symptoms at the time of participation;

2. English as their primary communication language;

3. normal hearing or no more than mild–moderate unilateral hearing loss;

4. no prior history of radiation treatment to the head and neck structures resulting in functional changes to the voice, breathing, or swallowing;

5. no other neuromotor disorder other than essential tremor that affects voice or speech;

6. no speech abnormalities due to a stroke or traumatic brain injury;

7. no psychiatric disorder that may affect the ability to participate; and

8. no history of surgical treatment with deep-brain stimulation or focused ultrasound.

Vowel Samples

Speech sounds were captured with an AKG C520 head-mounted condenser microphone positioned at ~5 cm from the mouth and at an azimuthal angle of circa 45°, in accordance to Titze and Winholtz (1993) and Švec and Granqvist (2010, 2018). This microphone was connected to a Zoom H6 Pro audio recorder (Zoom corporation) for signal recording with a sampling rate of 44.1 kHz and saved in WAV format. To obtain appropriate sound levels, sound pressure calibration for head-mounted microphones were conducted as described in Švec and Granqvist (2018, p. 456) with a Class 1 SoundTrack LxT Sound Level Meter (Larson Davis, PCB Piezotronics, Inc.) as criterion.

Prior research demonstrated that sustained phonation is the optimal speech context for identifying the presence of vocal tremor through acoustic measurement (Barkmeier et al., 2001; Brown & Simonson, 1963; Dromey et al., 2002; Lederle et al., 2012; Lester et al., 2013). In addition, sustained [a:] and [i:] were previously used to measure acoustic rate and extent of f₀ and IL modulations across differing pitch and loudness levels (Barkmeier-Kraemer et al., 2011; Erickson-DiRenzo, Sung, et al., 2020; Lederle et al., 2012; Shao et al., 2010). Therefore, the participants were instructed to produce [a:] and [i:] for 5 s at a comfortable pitch and loudness. From these audio recordings, midvowel portions of 2 s of [a:] and [i:] with optimal representation of tremor modulation were extracted using the program Praat (Paul Boersma and David Weenink, Institute for Phonetic Sciences), deidentified, and coded before processing with the following algorithm for measurements of rate and extent of acoustic modulation because of vocal tremor.

Manual Measurements of Vocal Tremor

The senior author (J.B.K.) provided the training to manual measurement raters. One co-author (K.D.) with 4 years of experience collecting these samples and measures completed the manual tremor measurements. Another co-author (S.K.) with 1 year of experience collecting these samples and measures completed measurements on 30 (33%) of the files for interrater reliability. To assess interrater reliability, intraclass correlation coefficients (ICCs; ICC model: two-way random effects; ICC type: single rater; ICC definition: absolute agreement; Koo & Li, 2016) between the manual measurements of the two raters were calculated and listed in Table 2. With single-rater ICCs between .770 and .914 across the four manually determined modulation properties, interrater reliability was considered acceptable. Upon post hoc review, there were no particular samples that explained instances of disagreement between the two raters. Disagreement predominantly pertained to deciding on presence/absence of vocal tremor modulation and was associated with small extent of modulation in f₀ or IL, measured on average as 1.7% (±0.9%). The manual tremor property measurements on all vowel samples by the first rater were taken as the standard referent in the further study.

Table 2.

Intraclass correlation coefficients, with their 95% confidence intervals (CIs) and statistical significance, as indicators of interrater reliability of the manual measurement of vocal tremor properties by two raters.

Manual modulation property	Single-rater ICC	95% CI		p	Interpretation
		Lower bound	Upper bound
f 0ModRate	.803	.582	.914	< .001	Moderate–excellent
$f_0{\mathrm{Mod}}_{\text{Mean}}^{\text{Extent}}$	.914	.807	.963	< .001	Good–excellent
ILModRate	.770	.478	.910	< .001	Poor–excellent
${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$	.887	.715	.958	< .001	Moderate–excellent

Note. Interpretation is based on Portney and Watkins (2000). ICC = intraclass correlation coefficient.

For the manual measurements of vocal tremor, the f₀ (in Praat: “pitch”) and IL (in Praat: “intensity”) contours of each coded acoustic file were visually inspected using Praat (Version 6.1.38). Before raters completed the measurements, each file's f₀ and IL contours were adjusted individually to assure adequate visualization of modulations using a consistent method across speaker recordings. The f₀ range was adjusted to be ±100 Hz of the mean f₀ of each file. Similarly, the IL settings ranged within ±10 dB of the mean IL of each file. The following measures were obtained manually for each file with calculations completed within Excel.

For f₀ and IL modulation rate, the total number of visible modulation cycles was identified in the 2-s segment analyzed, divided by 2 s. The number of cycles was determined by counting the number of peak-to-peak or valley-to-valley cyclic modulations. The following equation was set in Excel for each file: $\text{rate}\left(\mathrm{Hz}\right)=\frac{N\text{cycles}}{\text{duration}\left(\mathrm{s}\right)}$.

For f₀ and IL modulation extent, each cycle's extent was calculated by finding the maximum and minimum values for that cycle. For the extent of IL modulation, values were first converted from IL (in dB) into sound pressure level (SPL, in pascal or Pa). The extent of each cycle in the 2-s segment was averaged to determine the average magnitude of cyclic modulation: $f_0\text{extent}\left(\%\right)=\frac{\left(f_0\text{maximum}-f_0\text{minimum}\right)}{f_0\text{maximum}+f_0\text{minimum}}\times 100$ and $\mathit{IL}\text{extent}\left(\%\right)=\frac{\left(\text{SPLmaximum}-\text{SPLminimum}\right)}{\text{SPLmaximum}+\text{SPLminimum}}\times 100$.

Acoustic Analyses of Vocal Tremor

All acoustic signal processing was achieved through custom programming/scripting in the program Praat by the first author (Y.M.). An automated analysis was first run using the tool as described in Maryn et al. (2019) followed by a semi-automated analysis. The steps in this semi-automated acoustic method were similar to Maryn et al. (2019), but to overcome the above-described difficulties, an interface was built to enable the researcher/clinician/user to intervene when required (i.e., to adjust the tremor's frequency range so it contains the user-approved f₀ and/or IL modulation rate, delete signal parts that were erroneously indicated as modulation cycles, or state that there is no tremor-based modulation in f₀ and/or IL). Information about the rate, extent, and regularity of f₀ modulation and IL modulation in their respective traces was extracted via fast Fourier transformation (i.e., FFT, or spectral analysis) in Praat, as outlined in the automatic and user-intervened steps in the Appendix and the figures in Supplemental Material S1. The final graphical and numerical output of this semi-automated Praat tool resulted from this program.

This study investigated the congruence between the manual and the automated/semi-automated measurements. Given that the manual method only provided measures of IL modulation extent and rate, and f₀ modulation extent and rate, this study continued with the four analogous automated/semi-automated measures:

• ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ : marker of the mean extent of the IL modulation, as measured in the IL modulation trace after spectral processing.

• ILMod^Rate: frequency of the peak of the Fourier spectrum of the IL modulation trace, as a spectral marker of rate of IL modulation.

• $f_0{\mathrm{Mod}}_{\text{Mean}}^{\text{Extent}}$ : marker of the mean extent of the f₀ modulation, as measured in the f₀ modulation trace after spectral processing.

• f ₀Mod^Rate: frequency of the peak of the Fourier spectrum of the f₀ modulation trace; as a spectral marker of rate of f₀ modulation.

Results

Comparison Between Measurement Methods

Table 3 summarizes the descriptive data for the 12 variables (four tremor modulation properties × three measurement methods) in this study. Because analysis with the one-sample Kolmogorov–Smirnov test revealed that most of the variables (all except: automated f₀Mod^Rate with p = .052, manual ILMod^Rate with p = .181, manual ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ with p = .191, and semi-automated ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ with p = .082) were not normally distributed, nonparametric analysis of variance with Friedman repeated-measures test for three related and dependent samples was used to compare modulation properties between the three methods. These overall comparisons showed significant differences for all four modulation properties. Therefore, post hoc Wilcoxon signed-rank tests were used to juxtapose pairs of related samples after Bonferroni correction (i.e., results were considered statistically significant at p ≤ .017). For f₀Mod^Rate, mean of measurements differed slightly (only 0.15 Hz between manual and automated, and only 0.32 Hz between manual and semi-automated) but significantly between manual and the other two methods. For ILMod^Rate, mean of measurements also differed slightly (only 0.39 Hz between manual and automated, and only 0.32 Hz between manual and semi-automated) but significantly between manual and the other two methods. For ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ , mean of measurements differed significantly between all three methods. The difference was small (only 0.19%) between automated and semi-automated. However, the differences were large (10.24% between manual and automated, and 10.05% between manual and semi-automated) between manual and the other two methods. The latter can be explained mostly by the difference in scale (in pascal for the manual method, and in dB for the automated/semi-automated methods). For $f_0{\mathrm{Mod}}_{\text{Mean}}^{\text{Extent}}$ , there were no significant differences between the methods.

Table 3.

Minimum, maximum, mean, and standard deviation of the 12 variables (four tremor modulation properties × three measurement methods) of this study.

Method	f0ModRate (Hz)				${\mathit{f}}_{\mathbf{0}}{\mathbf{\text{Mod}}}_{\mathbf{\text{Mean}}}^{\mathbf{\text{Extent}}}$ (%)
	Min	Max	M	SD	Min	Max	M	SD
Manual	3.10	10.00	4.40	1.03	0.50	30.80	6.35	7.09
Automated	3.25	5.78	4.55	0.73	0.79	47.39	7.82	9.99
Semi-automated	3.25	10.47	4.72	1.03	0.66	31.52	6.47	6.93
	ILMod Rate (Hz)				${\mathbf{\text{ILMod}}}_{\mathbf{\text{Mean}}}^{\mathbf{\text{Extent}}}$ (%)
Method	Min	Max	M	SD	Min	Max	M	SD
Manual	2.90	6.60	4.26	0.83	1.00	33.80	12.08	7.52
Automated	3.11	10.26	4.65	1.32	0.25	4.91	1.84	1.21
Semi-automated	3.08	7.18	4.58	0.88	0.37	7.10	2.03	1.37

Correlations Between Manually and Acoustically Determined Tremor Properties

Table 4 lists the Pearson correlation coefficients (r_Pearson) between the manual measures and the automated measures with the tool from Maryn et al. (2019), as well as between the manual measures and the semi-automated measures. These correlations are also illustrated by the scatterplots in Figure 4. The raters considered tremulous modulation to be absent in 32 of 90 (36%) f₀ traces and in 38 of 90 (42%) IL traces; therefore, r_Pearson was only calculated on the data retrieved from the cases where tremor modulation was determined to be present (i.e., 58 f₀ and 52 IL traces). All Pearson correlations were statistically significant. The strength of r_Pearson values between manual and automated methods varied from low for f₀Mod^Rate to strong for ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ . However, the strength of r_Pearson values between manual and semi-automated methods was strong for all four measured tremor properties. Using Fisher r-to-z transformation, r_Pearson with manual markers was significantly higher for the semi-automated markers than for the automated markers, except for ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ . However, although r_Pearson for ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ from the automated method was highest, its difference with the r_Pearson for ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ from the semi-automated method was only 0.031 and not statistically significant. In general, Table 4 shows that the semi-automated method outperformed the automated method in terms of their correlations with the manual method across all rate and extent of tremor modulation measures.

Table 4.

Pearson correlation coefficients and their statistical significance between manual and automated tremor measures as well as between manual and semi-automated tremor measures, together with the statistical significance of the difference between the paired r_Pearson values.

Acoustic modulation property	Manual and automated			Manual and semi-automated			Between-rPearson differences
	N	r Pearson	p	N	r Pearson	p	z	p
f 0ModRate	58	.502	< .001	58	.933	< .001	−5.92	< .001
$f_0{\mathrm{Mod}}_{\text{Mean}}^{\text{Extent}}$	58	.811	< .001	58	.995	< .001	−8.47	< .001
ILModRate	52	.709	< .001	52	.918	< .001	−3.42	< .001
${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$	52	.957	< .001	52	.926	< .001	1.38	.168

Note. N = number of vowel samples; r_Pearson = Pearson correlation coefficient; p = significance level; z = value after Fisher's r-to-z transformation.

Figure 4.

Scatter plots to illustrate the correlations between manually and acoustically (automated or semi-automated) determined rate and extent of modulations in the f₀ and IL traces. f₀ = fundamental frequency; IL = intensity level.

Incongruency Between Manual and Automated Measurements

Visual inspection of the automated measures-based scatter plots of Figure 4 indicated six signals for which one or more outlier data (marked by an arrow) located away from the linear regression line. These were signals for which there was obvious incongruence between manually and automated acoustically determined modulation measurements. Table 5 presents the erroneous and corrected measurements on these signals and description of the phenomena that may have led to the errors. From this analysis, main signal characteristics that may have induced error were multiplophonic voice break (see Figure 5), unvoiced interval (by the software interpreted as f₀ = 0 Hz; see also Figure 5), and varying modulation rates (see Figure 6).

Table 5.

Description of characteristics of signals for which manual and automated tremor property analysis disagreed.

Subject	Recording	Modulation	Property	Measurement			Signal characteristics
				Manual	Automated	Semi-automated (corrected)
1	[i:], 1	f 0	Rate, Hz	5.70	3.34	5.44	Varying modulation rates with the script choosing the lower rate
5	[a:], 3	f 0	Rate, Hz	10.00	4.42	10.47	Varying modulation rates with the script choosing the lower rate
2	[i:], 1	A	Rate, Hz	5.30	8.45	5.51	Varying modulation rates with the script choosing the higher rate
2	[a:], 1	f 0	Extent, %	6.00	25.66	5.38	Instances with no period detection leading to erroneous f0 = 0 Hz in the script
7	[a:], 3	A	Rate, Hz	4.80	10.26	5.12	Varying modulation rates with the script choosing the higher rate
		f 0	Extent, %	11.50	36.67	12.37	Short voice breaks with diplophonia, subsequent drops in f0 and then erroneously high extent measures Instances with no period detection leading to erroneous f0 = 0 Hz in the script
12	[a:], 1	f 0	Extent, %	14.50	47.39	12.31	Instances with no period detection leading to erroneous f0 = 0 Hz in the script

Note. f ₀ = fundamental frequency.

Figure 5.

Graphs illustrating five intervals not detected as voiced and thus considered unvoiced by the script (U1, U2, U3, U4, and U5) and two multiplophonic interval (M1 and M2). All these intervals result in clear drops in the f₀ modulation trace, resulting in erroneous f₀ modulation extent measurements. f₀ = fundamental frequency.

Figure 6.

Graphs illustrating variance in period of 10 modulation cycles (MCs) and consequently also in modulation rate (between 3.482 Hz and 6.369 Hz, as averages of respectively the two lowest and seven highest points).

Vocal Tremor Diagnosis Accuracy

To determine the accuracy in diagnosing vocal tremor of the acoustic methods relative to the manual methods of tremor measurement, positive cases were considered those in which tremor modulation was determined as present (i.e., 58/90 f₀ traces and 52/90 IL traces), whereas negative cases were those where tremor modulation was absent (i.e., 32/90 f₀ traces and 38/90 IL traces). This accuracy of the automated and semi-automated acoustic methods was investigated using the receiver operating characteristics curve (ROC). Table 6 lists the areas under ROC (A_ROC), as well as the threshold scores that yielded the best combination of sensitivity and specificity as expressed in Youden's index. Compared to manual acoustic measurements, vocal tremor diagnosis accuracy was determined acceptable for all automated and semi-automated acoustic measures of modulation extent with A_ROC values ranging between 0.888 for automated ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ and 0.920 for semi-automated ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ .

Table 6.

Diagnostic accuracy statistics of automatically and semi-automatically deciding on the presence/absence of tremorous modulation, with manual decision as the referent measure.

Metric	Automated		Semi-automated
	${\mathit{f}}_{\mathbf{0}}{\mathbf{\text{Mod}}}_{\mathbf{\text{Mean}}}^{\mathbf{\text{Extent}}}$	${\mathbf{\text{ILMod}}}_{\mathbf{\text{Mean}}}^{\mathbf{\text{Extent}}}$	${\mathit{f}}_{\mathbf{0}}{\mathbf{\text{Mod}}}_{\mathbf{\text{Mean}}}^{\mathbf{\text{Extent}}}$	${\mathbf{\text{ILMod}}}_{\mathbf{\text{Mean}}}^{\mathbf{\text{Extent}}}$
AROC	0.904	0.888	0.904	0.920
Sensitivity	0.724	0.827	0.741	0.865
Specificity	0.969	0.921	0.937	0.921
Threshold score	1.594%	0.833%	1.690%	0.730%

Note. A_ROC = area under receiver operating characteristics curve.

Discussion

Vocal tremor is a neurogenic voice disorder that can significantly impact a person's communication and quality of life. It is important to have a readily available methodology to quantify the distinct vocal tremor acoustic properties, for example, for tracking treatment outcomes and changes in the severity of vocal tremor over time. This study investigated the concurrent validity of the two algorithms (automated or semi-automated) for measuring acoustic vocal tremor properties compared to manual extraction of these measures. Specifically, the correlations between manually and acoustically determined vocal tremor properties were gathered and compared for the strength of association. Pearson correlation coefficients between manual and automated acoustic measurements as well as between manual and semi-automated acoustic measurements were all significant. However, for the automated algorithm r_Pearson ranged from .502 (i.e., moderate) to .957 (i.e., strong) and was thus variable across the four markers, whereas for the semi-automated algorithm, all r_Pearson exceeded .9 (i.e., strong). With the semi-automated method, enabling the user to correct errors from the automated signal processing, this finding substantiated our hypothesis that it may be better to use a semi-automated algorithm. The semi-automated method was similar to the manual method, with f₀ and IL traces and added information shown in Praat's graphical output window and consequently visually processed. It was noted that there are significant differences in modulation properties between measurement methods. In general, these differences were small. Only in ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ there were large differences. This, however, can be explained by the difference in unit upon which modulation extent was calculated: pascal in the manual method versus dB in the automated/semi-automated. In future studies, differences can be assessed with ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ measurements based on variables with the same unit.

This study also investigated the reliability of the manual measurements of vocal tremor properties when executed by different trained clinicians. To date, the reliability of visual-acoustically determined modulation properties in f₀ and IL traces as standard in a perceptual-acoustic study has not yet been studied. In the present study, interrater reliability was considered good to excellent. A similar methodology has been used by Lederle et al. (2012) and Lester et al. (2013) for visual inspection and measurement of f₀ and IL acoustic modulation using the program Praat. However, visual–perceptual assessment of other graphical outputs in the field of voice disorders may be more common and, therefore, their reliability more studied. For example, reliability analysis of narrow-band spectrographic voice sound assessment showed moderate to excellent agreement, depending on the feature considered (Bastilha et al., 2021). Agreement between neurolaryngologists in their visual assessment/interpretation of graphical laryngeal-electromyographic traces has also been shown to be between moderate and high (Ho et al., 2019). Our visual trace manual reliability measurements were comparable to other established methods. It was interesting, however, to see in post hoc review that there may be some difficulty in visually perceiving vocal tremor acoustic modulation at lower extents of modulation, particularly around 1.7%. Nevertheless, the manual measurements were used as the referent upon which the validity of the automated and semi-automated acoustic algorithms was assessed in the present study.

Another goal of the present study was to determine how accurate the two acoustic (automated and semi-automated) algorithms are in differentiating between tremorous and nontremorous phonation. Upon visual/manual inspection of the f₀ and IL traces in the program Praat, the minimum extent of modulation of the traces from tremorous samples was found to be 0.5% for $f_0{\mathrm{Mod}}_{\text{Mean}}^{\text{Extent}}$ and 1.0% for ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ . ROC analysis indicated threshold values for the automated and semi-automated measures of extent of f₀ and IL modulation (see Table 6). For example, semi-automated ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ < 0.730% has a strong chance of corresponding with absence of vocal tremor. However, confirmation from additional study with more tremorous as well as nontremorous participants is required to substantiate general use of these thresholds. Based on ROC analysis, diagnostic precision was equal between automated and semi-automated measurements of $f_0{\mathrm{Mod}}_{\text{Mean}}^{\text{Extent}}$ . However, for ${\text{ILMod}}_{\text{Mean}}^{\text{Extent}}$ , vocal tremor diagnosis precision of the semi-automated measurements was higher than that of the automated measurements. Therefore, as expected, the semi-automated measurements appeared to perform superiorly in terms of correlation with manual measurements and diagnostic precision. This similarity between manual and semi-automated methods is most likely explained by including visual inspections before calculating the vocal tremor properties from the individual acoustic modulation cycles.

Zooming in to signal characteristics that may lead to incongruency between manual and automated measurements of vocal tremor modulation properties, as presented in Table 5, has shown that signal intervals without voiced period detection, multiplophonic intervals, and varying modulation rate may induce erroneous modulation rate and/or extent measurements. Semi-automated analysis enables the user to correct for such phenomena. Furthermore, future Praat scripts may incorporate signal processing steps to reduce their influence, such as interpolation to bridge unvoiced intervals and period detection algorithms that help reducing unwanted f₀ drops because of multiplophonia.

Limitations and Future Directions

There are a few limitations of this study that guide future investigations regarding the measurement and quantification of vocal tremor acoustic properties. First, only 15 subjects with essential tremor and various vocal tremor severity levels were included. This limits the clinical representativity of this study's outcomes. Second, only two raters visually inspected the f₀ and IL traces. Although their reliability was determined acceptable, it would be interesting to investigate the reliability of such inspection by more raters and to search more in depth for causes of disagreement within and between raters. Furthermore, using the semi-automated tool will require the user to (a) understand how vocal tremor properties are analyzed and measured, (b) to recognize when the program-generated measures differ from their manual measures, and (c) to identify signal characteristics, such as multiplophonic intervals or varying modulation rates resulting in incongruence between acoustic and manual measurements. As only one user applied the semi-automated tool, future work will also need to assess intra-user and inter-user reliability across a sample representative of vocal tremor types and severity levels. In case of insufficient reliability, training of semi-automated acoustic measurement of vocal tremor properties and its impact on reliability could be evaluated. Third, depending on the speech structures involved, vocal tremor may not only affect f₀ (predominantly related to the phonatory part of the speech system) and IL (predominantly related to the respiratory part of the speech system, although glottal width modulation during horizontal oscillation of the vocal folds can also affect IL), it may also, for example, involve the first and second formants (both related to the vocal tract due to tremor affecting the articulators comprising the speech system; Barkmeier-Kraemer & Story, 2010). It would, therefore, be important to add analyses of the properties of modulation of formant traces to the semi-automated algorithm. This could help to facilitate the evaluation of the relative contribution of these supralaryngeal tremor properties to vocal tremor, or the analysis of co-occurrence of f₀ and formant modulations in cases with oscillating laryngeal position/height. Fourth, vibrato in the singing voice has been studied as a surrogate model of vocal tremor (Lester-Smith & Story, 2016). Thus, it would be interesting to apply the acoustic methods in this study to characterize vibrato modulation properties. Finally, the tools developed and tested in this study can also be applied for identification and characterization of unique acoustic modulation features and classification of vocal tremor between various medical conditions (e.g., essential vocal tremor, vocal tremor associated with laryngeal dystonia, or vocal tremor associated with Parkinson's disease).

Data Availability Statement

The data sets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Appendix

Algorithm Steps Programmed in the Program Praat to Find Rate, Extent, and Regularity of Vocal Tremor Modulations in the Fundamental Frequency (f₀) and Sound Intensity Level (IL) Traces

Number	Type	Process
1	Au	Drawing a narrow-band spectrogram to enable a visual–perceptual interpretation of possible modulation in the speech signals: window length = 0.03 s, maximum frequency = 2000 Hz, time step = 0.002 s, frequency step = 20 Hz, window shape = sine-squared (Hanning)
2	Au	Application of the following ranges: range of tremor rate = from 3 to 12 Hz, range of tremor search rate = from 2 to 24 Hz
f 0 tremor: Repeat until user's approval of correct analysis
3	Au	Extraction and drawing of f0 trace with Praat's cross-correlation method: minimum f0 = 60 Hz, maximum f0 = 400 Hz, time step = 0.02 s, silence threshold = 0.3, voicing threshold = 0.45, octave cost = 0.01, octave jump cost = 0.8, voiced/unvoiced cost = 0.14
4	Au	Setting the mean value of the f0 trace to 0 to remove zero offset
5	Au	Windowing of the f0 trace with a Hanning function to remove spectral artefacts resulting from truncation of the time series
6	Au	Fast Fourier transformation (FFT) of the f0 trace to yield its FFT spectrum
7	Au	One-to-one translation of the f0 trace's FFT spectrum into a long-term average spectrum (LTAS) for further processing of the spectral data. Drawing of the LTAS
8	Au	Determination of frequency (and its corresponding period as $\frac{1}{\text{frequency}}$) and amplitude corresponding with the f0-based LTAS maximum/peak within the predefined range of tremor search rate
9	Au	Computation of the linear trend line through the f0-based LTAS over the range of tremor search rate
10	Au	Determination of the spectral peak's prominence above the linear trend line
11	Au	Determination of minimal f0 and maximum f0 in every period of the f0 trace
12	Au	Calculation of f0 tremor extent for every period as $\frac{\text{maximum}{f}_0-\text{minimum}{f}_0}{\text{maximum}{f}_0+\text{minimum}{f}_0}\times 100$ (Lederle et al., 2012; Lester et al., 2013)
13	Au	Calculation of mean and median extents of f0 tremor
14	Au	Determination and drawing of a rectangle derived from x (i.e., time) and y (i.e., f0) coordinates of the minimum and maximum data points in every period. These rectangles thus incorporated half duration and extent of every tremor period in a single metric and were therefore hypothesized to summarize elemental tremor cycle geometry.
15	Au	Calculation of absolute perturbation in rectangle surfaces between adjacent f0 tremor cycles as $\frac{\sum _{i=1}^{N-1}\left|{\text{surface}}_i-{\text{surface}}_{i+1}\right|}{N-1}$
16	Au	Calculation of relative perturbation in rectangle surfaces between adjacent f0 tremor cycles as $\frac{\frac{\sum _{i=1}^{N-1}\left|{\text{surface}}_i-{\text{surface}}_{i+1}\right|}{N-1}}{\frac{{\sum }_{i=1}^N{\text{surface}}_i}{N}}\times 100$
17	UI	Decision on visual presence of tremor in f0 trace
18	UI	Decision on correctness of f0 tremor rate
19	UI	Removal of unfit f0 tremor modulations
20	UI	User approval if f0 tremor is present, f0 tremor rate is correct and there are no more unfit f0 tremor modulations.
IL tremor: Repeat until user's approval of correct analysis
21	Au	Extraction and drawing of IL trace: minimum f0 = 60 Hz, time step = default
22	Au	Setting the mean value of the IL trace to 0 to remove zero offset
23	Au	Windowing of the IL trace with a Hanning function to remove spectral artefacts resulting from truncation of the time series
24	Au	Fast Fourier transformation of the IL trace to yield its FFT spectrum
25	Au	One-to-one translation of the IL trace's FFT spectrum into an LTAS for further processing of the spectral data. Drawing of the LTAS
26	Au	Determination of frequency (and its corresponding period as $\frac{1}{\text{frequency}}$) and amplitude corresponding with the IL-based LTAS maximum/peak within the predefined range of tremor search rate
27	Au	Computation of the linear trend line through the IL-based LTAS over the range of tremor search rate
28	Au	Determination of the spectral peak's prominence above the linear trend line
29	Au	Determination of minimal amplitude and maximum amplitude in every period of the IL trace
30	Au	Calculation of IL tremor extent for every period as $\frac{\text{maximum}\mathrm{IL}-\text{minimum}\mathrm{IL}}{\text{maximum}\mathrm{IL}+\text{minimum}\mathrm{IL}}\times 100$ (Lederle et al., 2012; Lester et al., 2013)
31	Au	Calculation of mean and median extents of IL tremor
32	Au	Determination and drawing of a rectangle derived from x (i.e., time) and y (i.e., IL) coordinates of the minimum and maximum data points in every period. These rectangles thus incorporated half duration and extent of every tremor period in a single metric and were therefore hypothesized to summarize elemental tremor cycle geometry.
33	Au	Calculation of absolute perturbation in rectangle surfaces between adjacent tremor cycles as $\frac{\sum _{i=1}^{N-1}\left|{\text{surface}}_i-{\text{surface}}_{i+1}\right|}{N-1}$
34	Au	Calculation of relative perturbation in rectangle surfaces between adjacent tremor cycles as $\frac{\frac{\sum _{i=1}^{N-1}\left|{\text{surface}}_i-{\text{surface}}_{i+1}\right|}{N-1}}{\frac{{\sum }_{i=1}^N{\text{surface}}_i}{N}}\times 100$
35	UI	Decision on visual presence of tremor in IL trace
36	UI	Decision on correctness of IL tremor rate
37	UI	Removal of unfit IL tremor modulations
38	UI	User approval if IL tremor is present, IL tremor rate is correct, and there are no more unfit IL tremor modulations
39	Au	Final graphical and numerical output with addition of the Acoustic Phonatory Tremor Index (APTI) based on Maryn et al. (2019)

Note.  Au = automatic; UI = user interference. Adapted from Maryn et al. (2019).

Funding Statement

This work was funded by the National Institute on Deafness and Other Communication Disorders (R01DC016838, PI: Barkmeier-Kraemer; P50DC019900, PI: Simonyan) with partial support from the University of Utah Voice, Airway, Swallowing Translational (VAST) Research Lab.

Acknowledgments

This work was funded by the National Institute on Deafness and Other Communication Disorders (R01DC016838, PI: Barkmeier-Kraemer; P50DC019900, PI: Simonyan) with partial support from the University of Utah Voice, Airway, Swallowing Translational (VAST) Research Lab.