Predicting dysphagia onset in patients with ALS: The ALS Dysphagia Risk Score

Abstract

Purpose:

For patients diagnosed with ALS, dysphagia can result in aspiration, malnutrition, and mortality. The purpose of this study was to develop a clinical prediction model capable of identifying patients with ALS at imminent risk for developing swallowing complications.

Methods:

A retrospective cohort study using the Pooled Resource Open-Access ALS Clinical Trials Database (PRO-ACT) was conducted. After dividing the PRO-ACT database into development and validation cohorts with dysphagia defined from the ALS Functional Rating Scale (ALSFRS), a multivariable Cox proportional hazards regression model estimated the probability of dysphagia at 3, 6, and 12-months with subsequent evaluation of model discrimination and calibration.

Results:

With 2057 participants in the development cohort and 1891 in the validation cohort, the Cox model included 7 clinical variables: spinal-onset; bulbar, fine and gross motor ALSFRS subscale scores; respiratory impairment; functional progression rate; and time from diagnosis. The cumulative incidence of dysphagia was 18% at 3-months, 29% at 6-months, and 45% at 12-months. The mean predicted probability of dysphagia development ranged from 4.5% in the bottommost risk decile to 40% in the topmost decile at 3 months, 10%–72% at 6 months, and 25%–93% at 12 months. In the validation cohort, the model had good discrimination and calibration with an optimism corrected c-statistic of 0.70 and calibration slope of 0.96.

Conclusions:

The ALS dysphagia risk score can be used to identify patients with ALS at high risk for self-reported dysphagia development who would benefit from a comprehensive swallowing assessment and proactive dysphagia management strategies.

Introduction

Dysphagia occurs in 63–92% of patients with ALS depending on their site of symptom onset.¹ It is one of the most severe and debilitating symptoms of the disease,(CITE) and may result in poor nutrition and aspiration pneumonia, both of which have been found to increase mortality in this population.(CITE)

At present, no clinical guidelines for the assessment and management of dysphagia in this population exist.^2,3 Practice patterns surrounding dysphagia evaluation and management vary,^2,3 but are largely reactive in response to the onset of dysphagia symptoms.⁴ For patients with ALS, once dysphagia manifests, treatment options become limited to compensatory strategies, diet modifications, and feeding tube placement.⁴ This has resulted in a call for a “paradigm shift” from reactive to proactive dysphagia management approaches, including interventions to increase functional reserve, such as resistance training, that may improve or help maintain swallowing function.⁴ Barriers to proactive care include the variability in the clinical presentation and the rate of progression of swallowing impairments and the time constraints of multidisciplinary assessment of these patients.^2,5

While “gold standard” dysphagia assessments may identify objective impairments in swallowing prior to the onset of subjective dysphagia symptoms,^6,7 currently “gold-standard” dysphagia assessments are not regularly performed in busy multidisciplinary clinics². A clinical prediction model containing routinely collected clinical variables could serve as a valuable risk stratification tool to identify patients who are likely to self-report future dysphagia symptoms and may benefit from comprehensive dysphagia assessments and proactive dysphagia management strategies. As such, the objective of this work was to develop and validate a multivariable clinical prediction model to quantify the 3-, 6-, and 12-month risk of self-reported dysphagia development in patients with ALS.

Methods

Study Population

Data used in the preparation of this article were obtained from the Pooled Resource Open-Access ALS Clinical Trials ( PRO-ACT) Database and has been volunteered by PRO-ACT Consortium members. October 22nd, 2019, all rights and obligations under this Terms of Use were assigned to The ALS Association. The Pooled Resource Open-Access ALS Clinical Trials Database (PRO-ACT)⁸ is the largest database containing worldwide clinical trial data from patients diagnosed with ALS with currently over 10,700 de-identified clinical patient records pooled from 23 Phase II/III clinical trials between the years of 1990–2015. None of the interventions tested proved to be clinically effective. This dataset was selected for its size and inclusion of easily and reliably obtainable clinical variables relevant for model development. Participants were excluded from this study if they 1) had no reported ALS Functional Rating Scale or Functional Rating Scale-Revised (ALSFRS or ALSFRS-R) scores in the dataset or 2) had a score of less than 4 on the swallowing question of the ALSFRS or ALSFRS-R on their first study visit, indicating the presence of dysphagia at time of study entry. The ALSFRS-R adds two additional respiratory questions to the respiratory subscale to the ALSFRS.

Prior to model development, the more recent participant PRO-ACT data, (i.e. participants with ALSFRS-R scores) were set aside for external model validation. The study was reviewed and approved by the Institutional Review Board at Tufts Medical Center in Boston, Massachusetts. Informed consent was waived due to the de-identifiable nature of the data.

Outcome

The outcome of interest was time to dysphagia, defined as a change in score from 4 (i.e. no impairment) to <4 (i.e. any impairment) on the swallowing question of the ALSFRS or ALSFRS-R. We predicted the probability of this outcome at 3-, 6- and 12-month time horizons to align with typical regularly scheduled clinic visits. Participants who did not develop dysphagia were censored at their last recorded ALSFRS score or at 365 days, whichever came first.

Variable Selection

Based on the published literature^9,10 and expert opinion, the candidate predictor variables included age, sex, race, site of symptom onset, time from diagnosis, weight, height, forced vital capacity, slow vital capacity, riluzole use, family history of ALS, ALSFRS bulbar subscale score, ALSFRS fine motor subscale score, ALSFRS gross motor subscale score, ALSFRS respiratory subscale score, and whether or not the participant received the placebo or treatment in the clinical trial. ALSFRS bulbar, fine motor, and gross motor scale scores each range from 0–12 with 12 being no impairment and 0 being severe impairment. The ALSFRS respiratory subscale includes one question and ranges from 0–4, with 4 being no impairment and 0 being severe impairment. Total ALSFRS scores were calculated by summing the 4 ALSFRS subscale scores. Total ALSFRS scores range from 0–40 with 40 being no impairment and 0 being severe impairment. For participants with ALSFRS-R scores, we derived total ALSFRS by excluding the two added respiratory questions.¹¹ To account for disease decline, we calculated a functional progression rate by dividing the decrease in ALSFRS score (score of 40 indicates no impairment) by the time (in months) elapsed since symptom onset to their first visit.¹² Rate of decline was log-transformed to normalize its skewed relation to the outcome. The site of symptom onset was categorized as either spinal or not spinal. All predictor variables were obtained at the first visit recorded in the dataset.

Missing data for candidate variables were imputed by multiple imputation to create 10 complete data sets using predictive mean matching for imputing continuous variables; logistic regression for imputation of binary variables; and polytomous logistic regression for imputation of categorical variables. Variables missing over 50% including a family history of ALS, forced vital capacity scores, and slow vital capacity scores, were excluded from the model.

Statistical Analysis

A multivariable Cox proportional hazards regression model was fit to each of the 10 imputed data sets using all candidate variables as described above. Parameter estimates and standard errors were then pooled using Rubin’s rules.¹³ Variables in the full model that were not associated with time to dysphagia development (p>0.05) were removed in the development of a simplified model, except for the site of symptom onset because of its well-accepted association with the dysphagia outcome and ALSFRS gross motor subscale score because it is needed to calculate the rate of decline. A likelihood ratio test was performed to assess for differences between the full and simplified model.

Both the full and simplified models were assessed for discrimination, calibration, and potential for over-fitting and optimism. Concordance statistics (c-statistics) were used to assess model discrimination. To assess model calibration, calibration plots at 3, 6, and 12-months were used to compare observed and predicted outcomes. Harrell’s Eavg and E90 which uses the difference between the observed and predicted values using locally estimated scatterplot smoothing (LOESS) regression was used to estimate the observed probabilities for each individual.¹⁴ Internal model validation was used to assess for over-fitting and optimism using bootstrap resampling.¹⁵ We performed 200 bootstrap samples on each of the 10 imputed datasets to calculate a uniform shrinkage factor for both models. The uniform shrinkage factor from the full model was used to calculate optimism corrected model coefficients in the simplified model to avoid overfitting.¹⁶ The c-statistic from the bootstrapped samples was averaged to calculate an optimism corrected c-statistic for both the full and simplified models.

To assess the validity of the simplified model, we evaluated the baseline hazard at 3, 6, and 12 months and shrank beta coefficients from the simplified model in the external validation data set to estimate predicted outcomes. To determine the clinical utility of the prediction model, we compared its clinical benefit versus simply referring everyone to swallowing evaluation (treat all) or referring no one (treat no one). Using the prediction model, we examined the range of probabilities from zero to one as decision thresholds. These results are displayed as decision curve plots for the application of the simplified model in the external validation dataset.^17,18 All statistical analyses were performed using R Studio version 1.2.1335.¹⁹

Results

Of the 10,723 participants in the dataset, 4,216 were excluded as ALSFRS or ALSFRS-R data were not available. An additional 2,559 participants were excluded for having dysphagia at baseline (ALSFRS or ALSFRS-R < 4). Thus, splitting the 3,948 remaining participants eligible for study inclusion resulted in 2,057 patients for model development and 1,891 for the external validation cohort. Table 1 describes participant characteristics for those who did or did not develop dysphagia. On average, those who developed dysphagia were more likely to be female and have non-spinal onset of their symptoms, faster rates of functional progression, and lower ALSFRS subscale scores.

Table 1.

Participant characteristics for those with and without dysphagia development.

	No Dysphagia n=2265	Dysphagia n= 1683
Age, years (mean (sd))	55.1 (11.3)	55.4 (11.8)
Male sex (%)	1553 (68.6)	1091 (64.8)
Race (%)
White	2082 (94.4)	1545 (94.4)
African American	24 (1.1)	18 (1.1)
Asian	46 (2.1)	34 (2.1)
Other	54 (2.4)	39 (2.4)
Height, cm (mean (sd))	171.7 (9.4)	171.0 (9.7)
Weight, kg (mean (sd))	78.2 (18.4)	77.4 (18.2)
Months from Diagnosis (mean (sd))	8.8 (9.3)	8.2 (9.0)
Rate of Functional Disease Progression, points per month (median (IQ range))	1.7 (1.1, 2.8)	1.8 (1.1, 2.8)
Non-spinal Onset (%)	188 (8.3)	341 (20.3)
ALSFRS
Bulbar Subscale Score (0–12) (mean (sd))	11.8 (0.6)	11.11 (1.2)
Fine Motor Subscale Score (0–12) (mean (sd))	8.6 (2.7)	7.7 (3.0)
Gross Motor Subscale Score (0–12) (mean (sd))	7.8 (2.9)	7.4 (3.0)
Respiratory Subscale Score (0–4) (mean (sd))	3.8 (0.5)	3.7 (0.6)
Riluzole Use = Yes (%)	1285 (73)	894 (73)

The Cox model included the following initial baseline visit covariates: site of symptom onset (spinal versus not spinal); presence of bulbar impairment; bulbar, fine motor, and gross motor ALSFRS subscale scores; presence of respiratory impairment; functional progression rate; and time since diagnosis. Table 2 provides hazard ratios for both the full and simplified models. Both models had good discrimination (c-statistic = 0.71 and optimism corrected c-statistic = 0.70) and were well-calibrated at each time horizon. There were no differences in model fit between the full and simplified models in a likelihood ratio test (p=.39) and minimal differences in model overfitting that might limit generalizability with uniform shrinkage factors of 0.96 and 0.98 for the full and simplified models, respectively.

Table 2.

Hazard ratios for full and simplified models

	Hazard Ratio Full Model (CI)	Hazard Ratio Simplified Model (CI)
Age (per decade)	1.04 (0.98–1.10)	–
Male sex	0.94 (0.78–1.13)	–
Race (vs White)
African American	0.80 (0.45–1.41)	–
Asian	0.88 (0.54–1.43)	–
Other	0.93 (0.63–1.35)	–
Height (per 10cm)	1.04 (0.93–1.05)	–
Weight (per 10kg)	0.97 (0.93–1.00)	–
Months from Diagnosis	0.98 (0.97–.99)	0.98 (0.97–.99)
Rate of Functional Disease Progression (points per month)	1.30 (1.13–1.49)	1.29 (1.13–1.47)
Site of Onset = Not Spinal	1.10 (0.92–1.31)	1.11 (0.95–1.29)
ALSFRS
Bulbar Subscale Score (0–12)	0.57 (0.53–0.60)	0.60 (0.56–0.63)
Fine Motor Subscale Score (0–12)	0.93 (0.90–0.95)	0.93 (0.90–0.96)
Gross Motor Subscale Score (0–12)	0.96 (0.96–1.02)	0.99 (0.96–1.02)
Respiratory Subscale Score (0–4)	0.83 (0.74–0.92)	0.82 (0.74–0.91)
Riluzole Use = Yes	0.94 (0.77–1.16)	–
Randomized to Placebo arm of trial	0.97 (0.82–1.14)	–

At 3-months, the predicted probability of dysphagia development was 4% for participants in the lowest risk decile and 40% for those in the highest risk decile. At 6-months, the predicted probability of dysphagia development was 10% for participants in the lowest risk decile and 72% for those in the highest risk decile. At 12-months, the predicted probability of dysphagia development was 25% for participants in the lowest risk decile and 93% for those in the highest risk decile.

When applying the simple model to the validation cohort, discrimination improved with c-statistics of 0.74 at 3 months, 0.72 at 6 months, and 0.70 at 12 months. The cumulative incidence of dysphagia was 18% at 3 months, 29% at 6 months, and 45% at 12 months. At 3 months, the mean absolute error in predicted probabilities was 3% with 90% of the predictions having less than a 6% absolute error across deciles when compared to the observed outcome rates. At 6 months the mean absolute error in predicted probabilities was 4% with 90% of the predictions having less than 6% absolute error when compared to the observed outcomes rates. At 12 months the mean absolute error in predicted probabilities was 7% with 90% of the predictions having less than 10% absolute error when compared to the observed outcome rates. Figure 1 (A–C) displays the external validation calibration plots.

Figure 1.

External validation calibration curves for the simplified model at 3-months (A), 6-months (B), and 12-months (C)

Calibration plots for the predicted versus observed probability of developing dysphagia over 3 months (A), 6 months (B) and 12 months (C). The black lines denote actual model performance (i.e., model predicted vs. observed probabilities). For reference, the gray lines would be perfect calibration (i.e., predicted probabilities exactly match the observed probabilities). The markers indicate the mean predicted probabilities for 3 risk groups –lowest risk group, moderate risk group, and high-risk group. The bars underneath the curves indicate the probability distribution of predicted dysphagia probabilities for study patients at each time point.

Decision curves in the external validation dataset¹⁷ showed that the simplified model performed at least as well as refer all or refer no one approaches across threshold probabilities at all three time horizons but most notably provides net benefit when the decision threshold (i.e. the risk score at which a clinician decides to refer) falls between 15%–50% for the 3 month prediction (Figure 2A); between 20%–80% for the 6 month prediction (Figure 2B); and between 20%–95% for the 12 month prediction (Figure 2C).

Figure 2.

Decisions curves for the simplified model at 3-months (A), 6-months (B), and 12-months (C)

Decision curves to evaluate the clinical net benefit of the prediction model versus refer all or none for comprehensive swallowing assessment. The horizontal axis represents the threshold probability of developing dysphagia over 3 months (A), 6 months (B) and 12 months (C) where the threshold incorporates both the probability having dysphagia and the value or concern about dysphagia with lower thresholds consistent with greater concern and higher likelihood to refer for a comprehensive swallowing assessment. The vertical axis is the net benefit of referring for comprehensive swallowing assessment, which in this case refers to the true positives for dysphagia. The solid black line depicts the strategy of referring no one for comprehensive swallowing assessment. The solid grey line depicts the strategy of referring everyone for comprehensive swallowing assessment. The dashed line depicts the benefit of using the predictive model to determine referral for a swallowing assessment. As an example for interpreting 2A, at a threshold probability of 0.2, the model leads to a net benefit of approximately 0.09 indicating that applying the model in 100 patients will detect 9 additional patients with dysphagia instead of none (no referral) and 6 additional patients with dysphagia instead of referral for all patients with a threshold probability ≥0.2. When the model’s dashed line falls above the grey and black lines, the model has superior net benefit (based on incremental true positive probabilities) relative to the other strategies over the following intervals: .15-.50, .20-.80, and .20-.95 for 3, 6, and 12 months respectively. Because the risk for harm associated with referral for comprehensive swallowing assessment is low and because referral has potential benefit, the threshold probability for referral would usually be low with the model providing incremental benefit over referring all or none.

To calculate the ALS dysphagia onset risk score, we created an online calculator with the simplified model (https://www.bridgetjperry.com/alsdysphagiariskscore).

Discussion

Based on the PRO-ACT database, the ALS dysphagia onset risk score estimates an individual’s likelihood of developing dysphagia at 3-, 6-, and 12-months. The model contains seven variables routinely collected in clinical encounters and stratifies patients into risk groups that differ substantially in their risk of developing dysphagia with good discrimination and calibration in the validation dataset. Thus, because of the clinical consequences of dysphagia in patients with ALS, this tool could facilitate timely referral to comprehensive dysphagia evaluation by a speech-language pathologist and proactive implementation efforts to avoid or mitigate nutritional and infective consequences and to incorporate patient preferences for management options.

Of the 7 variables in the final model, five were statistically significant predictors of dysphagia development; the association between dysphagia development and site of symptom onset did not reach significance, presumably because those who developed dysphagia early (i.e. before study entry) were excluded. Unsurprisingly, the score on the ALSFRS bulbar subscale served as the strongest predictor of dysphagia development. Higher scores on this subscale (indicative of less bulbar impairment) were protective of dysphagia development. Because all participants had scores of 4 on the swallowing question to be included in the study, differences in bulbar impairment subscale scores were driven by impairments in salivation and/or dysarthria. In some particpants, as a result of treatment, salivation scores have been found to improve over time, making them less likely to be drivers of dysphagia risk.²⁰ Future exploratione of dyarthria alone as a predictor of dysphagia onset may be warranted.

The ALSFRS respiratory score was also strongly associated with dysphagia development. For each point decrease on question 10 of the ALSFRS or ALSFRS-R score, the hazard of dysphagia increased by 28%. This finding was consistent with previous literature linking impaired respiratory function to both dysphagia and aspiration in patients with ALS.²¹ ALSFRS fine motor subscale scores were also significantly associated with dysphagia development. Indicative of less severe disease status, higher ALSFRS fine motor subscale scores were protective of dysphagia development. To our knowledge, although associations between dysphagia and the loss of handgrip strength^22,23 exist, the association between ALSFRS fine motor scores and dysphagia development has not yet been reported. We did not find any association between ALSFRS gross motor subscale scores and dysphagia development.

Our study suggests that the incidence of dysphagia increased in people with higher initial functional progression rates. To our knowledge, no prior studies have explored the relationship between the rate of functional decline and the onset of swallowing impairments. A study by Rong et al.⁹ found no association between the rate of speech loss and rate of disease decline as calculated by the time elapsed between initial and follow-up study assessment divided by the change in ALSFR-R total scores between assessments. Research suggests that using time from symptom onset divided by loss of total ALSFRS score may be more sensitive to disease decline than using the change in time over 4-month intervals and may explain discrepancies in our findings.¹²

In the absence of clinical guideline recommendations, it is not surprising that a recent survey of 34 multidisciplinary ALS clinics across the United States found that roughly 40% do not routinely report clinical swallowing evaluations, and half refer fewer than 15% of their patients for modified barium swallowing studies (MBSS), the “gold standard. The known drawbacks to clinical swallowing evaluations or MBSS (e.g., longer appointment time, an additional appointment, and radiation exposure) would appear to be minor relative to the potential to prevent complications from dysphagia such as malnutrition, aspiration pneumonia, and mortality. As such the threshold for the decision to refer patients for more comprehensive swallowing assessments should be relatively low. Decision curve analysis suggests that the dysphagia risk score could substantially enhance referral for screening if the threshold for referring (i.e. the probability at which clinicians and patients should consider swallowing evaluation to be beneficial) is in the ~15% to ~50% range at 3 months, a range that seems to be highly clinically relevant.

Limitations

Because the PRO-ACT database consists of data from participants in large clinical trials, the dataset may not be representative of the ALS population as a whole in both measured and unmeasured characteristics. The dataset does not include a trial indicator variable, so the model was unable to account for clustering by trial. Additionally, inclusion in this study is contingent upon not developing dysphagia prior to entry into the clinical trial. As such, the dysphagia risk score may not be well-calibrated to those who develop dysphagia in the very early stages of the disease. Finally, particularly given this selection bias, any causal interpretation of variable effects should be avoided.

Most importantly, this study relies on a single question to define the development of dysphagia on a scale that is not reflective of the “gold-standard” diagnosis of dysphagia derived from a modified barium swallow study. Some research suggests that objective impairments in swallowing efficiency and swallowing safety may precede subjective dysphagia symptoms.^6,7 Currently, “gold-standard” swallowing assessments in large datasets do not exist, and “gold-standard” dysphagia assessments are not regularly performed², so their clinical utility for predicting dysphagia onset remains uncertain. Despite this, we feel that a model containing routinely collected clinical variables, such as in our simple model, could serve as a valuable risk stratification tool to identify patients most likely to benefit from a comprehensive dysphagia assessment by a speech language pathologist. Additionally, this model could be useful for stratifying patients enrolled in clinical trials, especially when dysphagia outcomes might be relevant.. Finally, patient-reported outcomes may be more clinically meaningful (even if less sensitive) than image-based tests, which may be viewed as surrogate outcomes.²⁴

Conclusions

The ALS dysphagia onset risk score predicts the 3-, 6- and 12-month risk of dysphagia onset for patients’ diagnosed with ALS using seven easily obtainable clinical variables. Findings from this study should help identify patients with ALS who are at high risk for developing dysphagia. Early identification, referral, and evaluation of patients at risk for developing dysphagia may help patients preserve their health as swallowing disorders directly impact their survival and quality of life.