The diagnostic utility of patient-report and speech-language pathologists’ ratings for detecting the early onset of bulbar symptoms due to ALS

Abstract

Objective

This study aimed to determine the diagnostic utility of clinician speech ratings and patient self-report for detecting early bulbar changes associated with amyotrophic lateral sclerosis (ALS), compared to instrumentation-based speech measures.

Methods

Thirty-six individuals with ALS and 17 healthy control participants were included. Patients’ awareness of early bulbar motor involvement was assessed using self-reported scores on the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R). Clinicians’ detection of early bulbar motor involvement was assessed through perceptual speech ratings by two experienced speech-language pathologists. Participants with ALS were grouped as ‘bulbar pre-symptomatic’ or ‘bulbar symptomatic’ based on self-report and clinician ratings, and compared to healthy controls on six instrumentation-based speech measures. ROC analysis was used to compare the sensitivity and specificity of perceptual and instrumentation-based measures for detecting bulbar changes in pre-symptomatic individuals.

Results

Early bulbar changes that were documented using instrumentation-based measures were undetected by both patients and clinicians. ROC analyses indicated that instrumentation-based measures outperformed clinicians’ scaled severity ratings, and that percent pause time was the best measure for differentiating healthy controls from bulbar pre-symptomatic individuals with ALS.

Conclusions

Findings suggested that instrumentation-based measures of speech may be necessary for early detection of bulbar changes due to ALS.

Introduction

Declines in speech and swallowing associated with amyotrophic lateral sclerosis (ALS) substantially diminish quality of life and shorten survival (1,2). Despite these devastating consequences, there are currently no agreed-upon standards for early detection or monitoring the progression of bulbar symptoms. ALS is a clinical diagnosis, and the prolonged diagnostic delay in ALS is well-documented (3,4). Early detection and monitoring of bulbar deterioration have been a significant clinical challenge because overt changes in speech and other oral-motor functions may not occur until muscle weakness progresses to a critical level (5). Precise markers of bulbar motor decline are critically needed in clinical care for reducing diagnostic uncertainty and delay, monitoring disease progression and planning treatments, and in intervention research for improving trial participant selection and serving as objective clinical outcomes.

Current best-practices for bulbar assessment rely on subjective clinician judgments of speech and swallowing symptoms, evaluation of oral structures and functions, and patients’ self-reports of functioning (6,7). To evaluate speech characteristics, clinicians listen for deviations in articulation, voice quality, resonance, respiration, and prosody (7). Prior work has questioned the reliability of these clinician-based judgments (8), although judgments of speech severity under carefully controlled listening conditions have successfully differentiated people with mild dysarthria from healthy controls (9,10). One of the most widely-used patient-report measures for ALS is the ALS Functional Rating Scale – Revised (ALSFRS-R) (11), a 12-item, provider-guided questionnaire in which patients rate their current motor function. It includes three items on bulbar function: speech, swallowing, and saliva management. The ALSFRS-R has been shown to be highly reliable (12), easy to administer, and a useful predictor of ALS rate of progression and survival (13,14). the responsiveness of these metrics to early bulbar changes due to ALS.

Recent studies using instrumentation-based protocols (15,16) have demonstrated that, prior to declines in intelligibility, people with early-stage ALS show declines in articulator movement speeds, speaking rate, and percent pause time (17), and that articulatory (i.e., maximum and minimum lip and jaw velocities and number of repetitions in a DDK task) and phonatory measures (i.e., max F0 during a high pitch task) were most responsive to early decline in bulbar function (16). Similar instrumentation-based measures have also been shown to be sensitive to subtle or pre-symptomatic speech changes associated with Parkinson’s disease (18,19) and Huntington’s disease (20, 21). Although these measures show promise for early detection, most instrumentation-based approaches are not currently well-suited for clinical evaluations, due to the cost of the equipment, and the time and expertise required for data collection, analysis, and interpretation. Therefore, in the near-term, clinician judgment and patient self-report will likely remain the preferred clinical assessments of bulbar motor status. Despite their widespread current use, to our knowledge, no studies have examined the efficacy of clinician and self-report measures for detecting bulbar onset and monitoring bulbar motor decline.

The current study evaluated the diagnostic utility of clinician ratings of speech and patient self-report for detecting early bulbar involvement, using instrumentation-based measures as a standard. The following research questions were addressed: 1) Can patients with ALS detect early changes in their bulbar function? 2) Can SLPs perceptually detect early changes in bulbar function of people with ALS?, and 3) How does the diagnostic efficacy of speech-language pathologists (SLPs) perceptual speech ratings compare to instrumentation-based speech measures?

Methods

Participants

Participants with ALS

Thirty-six individuals with ALS were included in this study. Participants were selected from a larger, longitudinal study of bulbar decline in ALS, and met the following recruitment criteria: 1) diagnosis of ALS made by a neurologist, in accordance to the revised El Escorial criteria (22), 2) English as their primary language, 3) no prior history of progressive neurological disorders, and 4) normal hearing and adequate vision and literacy skills to read stimuli. Participants were selected based on the following additional inclusion criteria: 1) normal speech intelligibility (>96%) and 2) normal speaking rate (>150 wpm) on the Speech Intelligibility Test (SIT) (23). Although patients were enrolled in a longitudinal study, only one visit from each included participant was chosen for the current study. Demographic characteristics of participants with ALS, including information about disease onset, is listed in Table 1.

Table 1.

Demographic characteristics of participants with ALS

Subject	Gender	Age	Intelligibility	Speaking Rate	Onset	SLP Group	Patient-reported Group	ALSFRS Bulbar Subscore	ALSFRS Total Score
ALS01	M	47.2	97.3	161	Bulbar	SLP_Symp	Self_Symp	6	36
ALS02	F	66.2	100.0	168	Spinal	SLP_Symp	Self_Symp	7	37
ALS03	M	60.3	100.0	171	Generalized	SLP_Symp	Self_Symp	7	n/a
ALS04	F	62.8	100.0	185	Spinal	SLP_Symp	Self_Symp	8	34
ALS05	M	57.1	99.1	206	Spinal	SLP_Symp	Self_Symp	9	41
ALS06	M	75.2	98.2	188	Spinal	SLP_Symp	Self_Symp	10	37
ALS07	F	63.4	100.0	180	Spinal	SLP_Pre	Self_Symp	10	35
ALS08	F	48.6	99.1	158	Spinal	SLP_Pre	Self_Symp	10	36
ALS09	M	51.7	99.1	177	Spinal	SLP_Symp	Self_Symp	11	36
ALS10	M	77.2	96.4	177	Spinal	SLP_Symp	Self_Symp	11	34
ALS11	F	63.5	98.2	150	Spinal	SLP_Symp	Self_Symp	11	41
ALS12	M	46.0	100.0	196	Spinal	SLP_Symp	Self_Symp	11	24
ALS13	M	67.4	99.1	163	Spinal	SLP_Symp	Self_Symp	11	44
ALS14	F	51.3	98.2	174	Spinal	SLP_Symp	Self_Symp	11	33
ALS15	M	46.9	100.0	215	Spinal	SLP_Pre	Self_Pre	12	35
ALS16	M	49.2	100.0	237	Spinal	SLP_Pre	Self_Pre	12	42
ALS17	M	70.5	100.0	263	Spinal	SLP_Pre	Self_Pre	12	36
ALS18	M	60.9	98.2	188	Spinal	SLP_Pre	Self_Pre	12	41
ALS19	F	54.5	100.0	205	Spinal	SLP_Pre	Self_Pre	12	32
ALS20	F	72.3	98.2	197	Spinal	SLP_Symp	Self_Pre	12	28
ALS21	F	46.1	100.0	177	Spinal	SLP_Symp	Self_Pre	12	35
ALS22	M	56.9	97.3	168	Unknown	SLP_Symp	Self_Pre	12	39
ALS23	M	73.3	100.0	163	Unknown	SLP_Symp	Self_Pre	12	29
ALS24	F	63.2	100.0	161	Spinal	SLP_Symp	Self_Pre	12	40
ALS25	M	67.3	100.0	187	Spinal	SLP_Pre	Self_Pre	12	41
ALS26	M	68.0	100.0	183	Spinal	SLP_Pre	Self_Pre	12	43
ALS27	M	40.9	100.0	209	Spinal	SLP_Pre	Self_Pre	12	40
ALS28	F	50.7	99.1	159	Spinal	SLP_Symp	Self_Pre	12	27
ALS29	F	60.4	96.4	156	Spinal	SLP_Symp	Self_Pre	12	34
ALS30	M	61.0	99.1	159	Spinal	SLP_Symp	Self_Pre	12	45
ALS31	M	59.0	100.0	188	Spinal	SLP_Symp	Self_Pre	12	45
ALS32	F	73.1	97.3	201	Spinal	SLP_Pre	Self_Pre	12	30
ALS33	M	42.8	100.0	215	Spinal	SLP_Pre	Self_Pre	12	44
ALS34	M	55.4	99.1	226	Spinal	SLP_Pre	Self_Pre	12	41
ALS35	M	52.8	98.2	216	Spinal	SLP_Symp	Self_Pre	12	45
ALS36	M	56.0	100.0	202	Spinal	SLP_Pre	Self_Pre	12	25

Control participants

Seventeen healthy adults (8 M, 9 F) also participated in this study. Control participants had no known history of neurological disease, respiratory disease, craniofacial surgery or hearing loss, and spoke English as their primary language (Table 1).

Procedures

All participants completed a standard research protocol that included the ALSFRS-R, the SIT, and instrumentation-based measurement of speech designed to capture function of individual subsystems and system-wide speech function (16,24).

Patient-reported measure of function

ALSFRS-R scores were used as a patient-reported measure of function. An ALSFRS-R bulbar subscore was calculated for each participant with ALS based on their responses to the three test items pertaining to bulbar function (speech, salivation, and swallowing). Each item was rated on a 5-point scale (0=severe impairment, 4=normal function), yielding a maximum bulbar subscore of 12. Thus, subscores less than 12 indicated impaired function in one or more of the bulbar domains.

SLP perceptual speech analysis and measures

Perceptual speech ratings were based on recordings of each participant’s productions of 10 sentences from the SIT. The SIT was developed as a test of speech intelligibility for speakers with motor speech disorders, based on studies showing the utility of measuring intelligibility and speaking rate in patients with dysarthria (25, 26). The test has strong inter-rater reliability (r = .94) (23). The produced sentences differed across subjects and were randomly generated by the SIT software. Recordings were trimmed to remove extraneous conversation and peak-amplitude was normalized to reduce bias due to differences in recording volume.

Two certified SLPs completed two perceptual rating tasks presented using a web interface. SLPs were blinded to the speakers’ diagnosis while judging speech samples. In the first task, SLPs listened to speech samples of all ALS and control participants in a randomized order, and made binary judgments regarding whether or not they thought the speaker had dysarthria. In the second task, SLPs heard the same set of speech samples, randomized in a different order, and rated their overall severity using a visual analog scale (VAS). The VAS was a vertical line, 300 pixels in length (approximately 100 mm) with no additional markings, anchored with ‘no dysarthria’ at the bottom and ‘severe dysarthria’ at the top. Instructions were based on a previously published protocol by Sussman & Tjaden (10). SLPs completed the severity rating task between two and four days after the dysarthria judgment task.

Instrumental analysis and measures

The speech measures were a subset of 61 collected instrumentation-based measures reflecting function across speech subsystems (16,24). Six measures were chosen based on previous work that has shown them to be responsive to early changes in bulbar function and predictive of speech decline (17). The six measures were derived from either acoustic and orofacial kinematic recordings, and included percent of pause time (PPT), maximum fundamental frequency (Max F0), nasalance, maximum velocity of lip opening (Max velocity UL_LL), diadochokinesis rate (DDK rate), and articulation rate. Procedures for each measure are listed in Table 2, and have been previously described in detail (14). In addition to individual subsystem measures, speaking rate and speech intelligibility were obtained from the SIT, representing global measures of speech function. These measures were primarily used as selection criteria for the current study; however, speaking rate was also analyzed as an outcome measure, because declines in speaking rate are known to precede declines in intelligibility among individuals with ALS (27).

Table 2.

Summary of included acoustic and kinematic subsystem measures

Speech subsystem	Measure	Experimental Task	Procedures
Respiratory	Percent pause time (%) (PPT)	Reading of Bamboo passage	Pauses > 300ms were identified using a custom MATLAB program, Speech Pause Analysis (SPA)*; PPT= total pause duration/total passage duration ×100
Phonatory	Max F0 (Hz)	‘high pitch’ phonation of /a/	Participants raised their pitch from a normal level to the highest pitch possible while vocalizing /a/, and held the high pitch for 5 seconds. Multi-dimensional Voice Profile (MDVP, Model 5105) software was used to measure the maximum pitch obtained during the sustained vowel.
Resonatory	Nasalance (%)	Sentence repetition ‘Buy Bobby a puppy’	KAYPentax, Model 6400 nasometer was used to measure nasalance during sentence production.
Articulatory	Max velocity (mm/s) of UL_LL	Sentence repetition ‘Buy Bobby a puppy’	3D optical motion capture system (Motion Analysis Corp.) tracked movement of reflective markers on each participant’s face; maximum velocity of relative movement between the upper and lower lips was calculated using SMASH (38)
	DDK rate (syll/sec)	Repeat/ta/as quickly and clearly as possible on one breath	DDK rate = # syllable repetitions/total duration (seconds)
	Articulation rate	Reading of Bamboo passage	Articulation rate = # words produced/total speech duration of passage (excluding pauses > 300ms)
System-level	Intelligibility	Reading 10 randomly generated sentences (5–15 words in length) from the Speech Intelligibility Test	Each participant’s speech sample was transcribed orthographically by one naive listener. Intelligibility = % words correctly identified
	Speaking rate (wpm)		Speaking rate = # words produced/total duration (min)

Statistical analyses

To determine whether participants with ALS detected their early bulbar changes, they were stratified into the following groups based on their ALSFRS-R bulbar subscores: participants with a bulbar subscore of 12 were labeled bulbar pre-symptomatic (Self_Pre; n= 22); participants with a bulbar subscore < 12 were labeled bulbar symptomatic (Self_Symp; n= 14). These groups were subsequently evaluated for differences in instrumentation-based measures and perceptual severity ratings using a series of one-way ANOVAs.

To determine if SLPs were able to detect early bulbar changes, participants with ALS were re-grouped based on SLP’s binary dysarthria judgments. Individuals who both SLPs rated as having dysarthria (n= 22) were considered to be bulbar symptomatic (SLP_Symp group), and individuals rated by one (n= 8) or both (n= 6) SLPs as not having dysarthria were considered to be bulbar pre-symptomatic (SLP_Pre). After grouping, a series of ANOVAs was used to compare the SLP-Pre and SLP_Symp groups to healthy adults on instrumentation-based measures and perceptual severity ratings. Tukey’s HSD posthoc tests were conducted to test pairwise contrasts on measures that had a significant omnibus test.

Receiver operating characteristic (ROC) analyses were used to compare how well perceptual severity ratings and instrumentation-based measures differentiated healthy controls from self-reported bulbar pre-symptomatic individuals with ALS (Self_Pre group). Area under the curve (AUC) values were used to estimate the overall strength of each measure for detecting early changes in bulbar function. Optimal cutpoints were obtained from ROC analysis, and used to calculate sensitivity, specificity, and positive and negative likelihood ratios for each measure.

Results

Patient self-report

Twenty-two of the 36 participants with ALS rated themselves as having no bulbar deficit on the ALSFRS-R. The median bulbar subscale score for the symptomatic bulbar patients was ten. Among symptomatic bulbar participants, the median score on the Speech question of the ALSFRS-R was three.

Instrumentation-based assessment of patient-derived groups

Results of one-way ANOVAs comparing self-rated bulbar presymptomatic (Self_Pre) individuals with ALS, symptomatic (Self_Symp) individuals with ALS, and healthy adults revealed significant group effects on the following measures: speaking rate, F(2,50) = 3.24, p=.048, articulation rate, F(2, 49) = 4.2, p=.021, percent pause time, F(2, 49) = 7.73, p=.001, maximum velocity of lip opening, F(2, 47) = 4.44, p=.017, and DDK rate, F(2, 28) = 5.71, p=.008. No significant group differences were found for maximum F0, nasalance, or perceptual severity rating. Descriptive statistics for significant variables are presented in Figure 1a.

Figure 1.

Group differences on instrumentation-based speech measures and clinicians’ perceptual speech severity ratings. Panel a) displays results from groups based on patient self-report (i.e., self-reported bulbar pre-symptomatic (Self_Pre), self-reported bulbar symptomatic (Self_Symp), and controls). Panel b) displays results from groups based on clinician’s dysarthria judgments (i.e., SLP-rated bulbar pre-symptomatic (SLP_Pre), SLP-rated bulbar symptomatic (SLP_Symp), and controls).

On pairwise testing, the Self_Pre group had significantly faster articulation rates and DDK rates than the Self_Symp group (p < .05), and both ALS groups had significantly faster maximum velocities of lip opening and significantly longer percent pause times compared to the healthy controls (p < .05). Interestingly, the Self_Pre group also tended to have faster rates of speaking, articulation, and DDK than the control participants. Although group differences did not meet statistical significance, Cohen’s d effect sizes were large for these contrasts. Effect sizes for pairwise contrasts between Self_Pre and control groups are shown in Figure 2.

Figure 2.

Cohen’s d effect sizes for pairwise contrasts between self-reported bulbar presymptomatic individuals with ALS and healthy controls.

Speech-language pathologists’ ratings

Reliability

Speech-language pathologists both judged 22 of the participants with ALS as having dysarthria, compared to only 14 participants who reported bulbar symptoms on the ALSFRS-R. Of the 14 participants who self-reported bulbar symptoms, 12 (86%) were rated by both SLPs as having dysarthria, and the remaining two were rated by one SLP as having dysarthria. SLPs’ ratings of healthy controls yielded a high number of false positives; seven of the 18 healthy controls (39%) were rated as having dysarthria by both SLPs, and 16 were rated as having dysarthria by at least one SLP.

Inter- and intra-rater reliability was calculated for the perceptual rating tasks. For the dysarthria judgment task, overall agreement between SLPs was 72.6%. Agreement was higher for ALS speakers (82%) than healthy controls (50%). Intra-rater reliability, based on re-judgment of 15% of samples, was 70%–90%. For the severity rating task, Pearson’s product moment correlations indicated moderate inter-rater reliability for VAS ratings, r = .52.

Instrumentation-based assessment of clinician-derived groups

Results of one-way ANOVAs comparing the SLP-rated pre-symptomatic (SLP_Pre) group, symptomatic (SLP_Symp) group, and healthy adults revealed significant group effects on the following measures: speaking rate, F(2,50) = 7.62, p=.001; articulation rate, F(2, 49) = 4.12, p=.02; percent pause time, F(2,49) = 11.43, p < .001; maximum velocity of lip opening, F(2, 47) = 4.46, p=.02; and severity rating, F(2, 50) = 12.94, p<.001. No significant group differences were found for maximum F0, nasalance, or DDK rate. Descriptive statistics for significant variables are presented in Figure 1b.

Individuals in the SLP_Pre group had significantly faster articulation rates and speaking rates than the control participants and the SLP_Symp group (p<.05 for each). Individuals in both ALS groups had significantly faster maximum velocities of lip opening than healthy controls (p<.05). Individuals in the SLP_Symp group had significantly longer percent pause times and significantly higher severity ratings than healthy controls and those in the SLP_Pre group (p < .05).

Diagnostic efficacy of instrumentation-based measures and perceptual severity ratings

ROC analyses (see Figure 3) were conducted to compare the efficacy of perceptual severity ratings and instrumentation-based measures for differentiating individuals in the Self_Pre group from healthy controls. AUC values are listed in Table 3, and sensitivity, specificity, and likelihood ratios are listed in Table 4.

Figure 3.

ROC curves displaying the performance of instrumentation-based speech measures and clinicians’ speech severity ratings for differentiating between self-reported bulbar presymptomatic individuals with ALS and healthy controls.

Table 3.

Area under the curve (AUC) values for acoustic, kinematic, and perceptual speech measures.

Measure	AUC	95% CI (DeLong)
		Lower	Upper
% Pause time	0.83	.70	.97
Max Vel UL_LL	0.72	.55	.89
Articulation rate	0.72	.55	.88
Speaking rate	0.65	.47	.83
DDK rate /ta/	0.71	.49	.92
Severity rating	0.52	.33	.71

Table 4.

Sensitivity, specificity, positive and negative likelihood ratios for acoustic, kinematic, and perceptual severity ratings, based on ROC-derived optimal cutpoints.

Measure		Value	95% CI
			Lower	Upper
% Pause time (cut point = 17.4%)	Sensitivity	68.18%	45.13%	86.14%
	Specificity	93.75%	69.77%	99.84%
	Positive Likelihood Ratio	10.91	1.6	74.34
	Negative Likelihood Ratio	0.34	.18	.63
Max Vel UL_LL (cut point = 205 mm/s)	Sensitivity	47.62%	25.71%	70.22%
	Specificity	100%	78.2%	100%
	Positive Likelihood Ratio	na*	na*	na*
	Negative Likelihood Ratio	0.52	0.35	.79
Articulation rate (cut point = 212 wpm)	Sensitivity	59.09%	36.35%	79.29%
	Specificity	62.5%	35.43%	84.80%
	Positive Likelihood Ratio	1.58	.77	3.24
	Negative Likelihood Ratio	0.65	.35	1.23
Speaking rate (cut point =182 wpm)	Sensitivity	68.18%	45.13%	86.14%
	Specificity	70.59%	44.04%	89.69%
	Positive Likelihood Ratio	2.32	1.05	5.11
	Negative Likelihood Ratio	0.45	.23	.89
DDK rate /ta/(cut point = 4.65 syl/sec)	Sensitivity	91.67%	61.52%	99.79%
	Specificity	53.85%	25.13%	80.78%
	Positive Likelihood Ratio	1.99	1.08	3.66
	Negative Likelihood Ratio	0.15	.02	1.08
Severity rating (cut point = 25.5)	Sensitivity	45.45%	24.39%	67.79%
	Specificity	29.41%	10.37%	55.96%
	Positive Likelihood Ratio	0.64	.37	1.12
	Negative Likelihood Ratio	1.85	.81	4.25

^*A positive likelihood ratio could not be calculated, since the specificity was 100%.

Perceptual severity ratings by SLPs did not differentiate groups well, with AUC values at chance levels, low sensitivity and specificity, and very weak positive and negative likelihood ratios. AUC values indicated that percent pause time (PPT) was the best measure for differentiating individuals in the Self_Pre group from healthy controls, with moderate sensitivity and high specificity, as well as a large positive likelihood ratio and moderate negative likelihood ratio. AUC values indicated maximum velocity of lip opening, articulation rate, and diadochokinesis rate were moderate group differentiators. Lip opening velocity had high specificity but low sensitivity, whereas diadochokinesis rate had high sensitivity but low specificity.

Discussion

There were two primary findings from the current study: (1) patients’ self-report and clinicians’ perceptual judgments were inadequate for detecting early bulbar involvement, and (2) instrumentation-based speech measures performed better at differentiating bulbar pre-symptomatic patients with ALS from healthy controls than clinicians’ scaled severity ratings.

Patient self-report in early disease stages

Results suggested that individuals with ALS do not notice the earliest changes in their bulbar function. When participants with ALS were stratified based on their self-reported bulbar scores, both participant groups (i.e., those who reported bulbar symptoms (Self_Symp) and those who reported normal bulbar function (Self_Pre)), differed significantly from healthy controls on instrumentation-based speech measures. Despite reporting no bulbar symptoms, individuals in the Self_Pre group paused more and had faster lip opening velocities than controls. Trends in other variables (i.e., faster articulation rates, speaking rates, and DDK), although not statistically significant, were also consistent with the increasing movement speeds. We have previously reported increased articulator movement speeds and greater consistency of speech movement patterns during early stage ALS, which is followed by a significant slowing of movement in later stages (28, 29, 30). Increases in speaking rate have also been reported in individuals with pre-manifest Huntington’s disease (21), followed by slowed speech rate in symptomatic individuals with Huntington’s disease (31). It is currently unknown if the transient increase in articulatory movement speed is due to behavioral compensation or primary disease effects.

These findings indicate that patient self-ratings on the ALSFRS-R are not sufficient to determine whether early changes in bulbar function are occurring. It is possible that subtle changes perceived by individuals were not revealed in ALSFRS-R scores because it is a global scale and its bulbar function test items are broad. However, since functional speech (i.e., speaking rate or intelligibility) was not affected and speech became faster rather than slower, it is unlikely patients would perceive these changes as concerning, even if they were noticed.

Clinician perceptual judgment of speech in early disease stages

Results also suggest that SLPs’ auditory perceptual judgments are not sufficient to detect early bulbar changes in people with ALS. When participants with ALS were stratified based on SLP’s binary dysarthria judgments, both participant groups (i.e., those judged as having dysarthria (SLP_Symp) and not having dysarthria (SLP_Pre)) significantly differed from healthy controls on instrumentation-based measures of speaking rate, articulation rate, and maximum lip velocity. In addition, the reliability of SLPs’ dysarthria judgments was relatively weak. The high false positive rate was particularly problematic, and suggests SLPs may have trouble perceptually differentiating speech changes associated with normal aging from those related to early disease processes.

Instrumentation-based detection of early bulbar changes

Results of ROC analyses demonstrated that SLPs’ severity ratings were no better than chance for differentiating self-reported, bulbar pre-symptomatic individuals with ALS from healthy controls, but that instrumentation-based speech measures performed better. Among the instrumentation-based measures, percent of pause time (PPT) was the best group differentiator, although it had stronger specificity than sensitivity. Its positive likelihood ratio of 10 suggests that having PPT over 17% on the Bamboo passage greatly increases the likelihood of having ALS, compared to a PPT below this cutpoint. This cutpoint is similar to the value found to differentiate groups in prior studies (32). Although PPT does not differentiate between speech breathing pauses and non-breathing pauses, the 300 ms minimum pause threshold used in this study has been previously shown to be an optimal threshold for obtaining stable estimates of pausing in speakers with ALS (33) and prior research has shown that breathing-related pauses are typically over 150–250 ms in duration at normal speaking rates (34, 35). Nonetheless, our pause data may include some non-breathing pauses in addition to breathing pauses. Prior research on pausing during passage reading in healthy adults has shown that breathing pauses are longer in duration and occur less frequently than non-breathing pauses, and are driven both by linguistic factors (e.g., syntactic boundaries, emphatic stress) as well as physiologic need (36). In bulbar pre-symptomatic stages of ALS, increased pause time may reflect subclinical changes in respiratory function or cognitive-linguistic skills as well as articulatory function (28). Although it was not the focus of the current investigation, future studies examining early changes in respiration (e.g., forced vital capacity), cognitive-linguistic skills, and breathing vs. non-breathing pauses are needed to determine if changes in PPT are specific indicators of bulbar decline. Velocity of lip opening, articulation rate, and DDK rate also showed potential diagnostic value for detecting early bulbar changes in bulbar pre-symptomatic patients with ALS, with AUC values in the moderate range.

Although these preliminary data were promising, the 95% confidence intervals for sensitivity, specificity, and likelihood ratios were large, suggesting the need for caution when interpreting findings. In addition, findings motivate the need for future research examining the discriminative ability of combined acoustic and kinematic measures, which could lead to development of an assessment protocol for detection of early bulbar changes associated with ALS.

Collectively, results suggest that instrumentation-based measures of speech may be necessary when the clinical objective is early diagnosis of bulbar deterioration in ALS. These findings are consistent with recent studies demonstrating that changes in instrumentation-based measures precede intelligibility declines in speakers with ALS (16,37), and have important implications for clinical assessment and future research. Present findings specifically call into question the validity of clinicians’ perceptual speech ratings for assessing people with subtle dysarthria symptoms. Clinician ratings in this study were based on purely auditory judgments of speech samples, and it is possible that with additional information available in a clinical setting (e.g., visual information, cranial nerve exam) SLPs may more accurately detect early bulbar changes. Findings also suggest that measures of pausing and rate of speech movement may be useful to include in assessment protocols of bulbar function in early disease stages. The integration of instrumentation-based measures into clinical practice will require additional research that establishes robust normative data, validates diagnostic protocols, and develops more automated collection and analysis methods. These findings also have important implications for future research because they demonstrate the efficacy of instrumentation-based measures for identifying appropriate candidates for clinical trials and for tracking disease progression.

Biographies

Kristen Allison is a postdoctoral research fellow in the Speech and Feeding Disorders Lab at MGH Institute of Health Professions. Her research interests focus on characterization and treatment of dysarthria in adults and children.

Yana Yunusova is an associate professor in the Department of Speech-Language Pathology at the University of Toronto and director of Speech Production Laboratory. Her research focuses on the motor speech disorders due to ALS, Parkinson’s disease and stroke.

Thomas Campbell is a professor of Communication Disorders and the executive director of the Callier Center for Communication Disorders at the University of Texas at Dallas. His research focuses on the identification of physiological, environmental, and genetic variables for the early identification of speech and language disorders, and development of communication technology.

Jun Wang is an assistant professor of Biomedical Engineering and Communication Disorders at the University of Texas at Dallas and director of the Speech Disorders and Technology Lab. His research focuses on silent speech recognition, motor speech disorders, and neuroscience for speech communication.

James Berry is a neurologist at Massachusetts General Hospital, where he is the ALS Clinic Unit Chief. His research focuses on behavioral and biomarkers of ALS.

Jordan Green is a professor at the MGH Institute of Health Professions and director of the Speech and Feeding Disorders Lab. His research focuses on how neurologic impairments, such as ALS, affect speech and swallowing function.