Discriminant ability of the 3-ounce water swallow test to detect aspiration in amyotrophic lateral sclerosis

Abstract

Background:

Given the need for quick and accurate dysphagia screening tools to optimize referral workflows and resource utilization in fast-paced multidisciplinary amyotrophic lateral sclerosis (ALS) clinics, we evaluated the discriminant ability of the 3 oz. water swallow test (WST) to detect aspiration in individuals with ALS.

Methods:

A total of 212 paired 3 oz. WST (index test) and standardized videofluoroscopic swallow studies (reference test) were completed in individuals with a confirmed diagnosis of ALS. Blinded raters analyzed swallowing safety using the validated penetration-aspiration scale (PAS) (non-aspirator: PAS <6; aspirator: PAS ≥ 6). Receiver operating characteristic curve analysis, area under the curve (AUC), sensitivity, specificity, and positive and negative predictive values (PPV, NPV) were calculated.

Key Results:

Index test: 78 (36.8%) WSTs were scored as a fail and 134 (63.2%) as a pass. Reference test: Aspiration was confirmed in 67 (31.6%) reference tests with 145 (68.4%) reference tests verified as having no aspiration. Sensitivity and specificity of the 3 oz. WST to detect radiographically confirmed aspiration was 55.2% and 71.7% respectively (AUC: 0.635, PPV: 47.4%, NPV: 77.6%).

Conclusions & Inferences:

In this dataset, the 3 oz. WST did not demonstrate adequate sensitivity or specificity to detect aspiration in people with ALS as a stand-alone dysphagia screening tool.

Introduction:

The majority of dysphagia screening tools have been validated in stroke survivors,¹ limiting generalizability of validation data and their use in other high-risk dysphagia populations where screening accuracy may differ.^2,3 This is relevant as the underlying pathophysiology of diseases often differ between patient populations,^4–8 thereby leading to variable clinical manifestations of dysphagia. Individuals with amyotrophic lateral sclerosis (ALS) are at high risk of dysphagia due to the rapid and progressive upper and lower motor neuron degeneration that impacts bulbar musculature and can lead to impairments in swallowing safety and efficiency. Recent data radiographically confirmed a high prevalence of swallowing safety (48%) and efficiency (73%) impairments in this patient population.⁴ Importantly, presence of dysphagia has been associated with social isolation and reduced mental well-being,^9,10 aspiration pneumonia,¹¹ and death.¹²

Given the rapid nature of ALS disease progression, accurate and frequent monitoring of swallowing function is a critical component of patient care models.^9,13,14 The fast-paced nature of typical multidisciplinary ALS clinics necessitate quick and efficient dysphagia screening methods that would ideally identify high risk patients for subsequent instrumental evaluation. Currently, no validated dysphagia screening tools exist for use in ALS, resulting in use of non-specific and inaccurate monitoring methods.^15,16 A published survey from the Northeastern ALS (NEALS) consortium reported that the most commonly used instrument to monitor bulbar function across clinical sites was the ALS functional rating scale revised (ALSFRS-R),^17,18 which was subsequently found to yield a sensitivity of 78% and specificity of 62% to detect radiographically confirmed dysphagia.¹⁶ Given these gaps in knowledge and clinical care, there exists a critical need to validate a pragmatic, non-invasive, and accurate screening tool for standard implementation across ALS multidisciplinary clinics.^15,19,20 In response, the NEALS Bulbar Committee published preliminary guidelines for the clinical evaluation of bulbar function in ALS²⁰ that included the 3 ounce water swallow test (3 oz. WST) and the Yale Swallow Protocol.²¹ The 3 oz. WST involves giving a patient a cup containing 3 oz. of water and asking them to drink continuously, without stopping. Criteria for failing the 3 oz. WST includes stopping, coughing, or throat clearing during or within one minute of task completion. Importantly, the 3 oz. WST was validated in a cohort of patients post-stroke where it demonstrated a sensitivity of 80% and specificity of 54%.²² More recently, the 3 oz. WST was validated in a heterogenous acute inpatient population (N=3,000) where it demonstrated a sensitivity of 96.5% and specificity of 48.7% for detecting aspiration.²³ Of note was the finding that results varied based on the underlying medical diagnosis causing dysphagia with sensitivity ranging from 90.9%−100% and specificity ranging from 25.4%−67.3%.²³ This latter finding along with recent 3 oz. WST data in cardiac surgical patients²⁴ highlight the need to develop and validate population-specific screening tools.

Therefore, we aimed to determine the discriminant ability of the 3 oz. WST to detect radiographically confirmed aspiration status in people with ALS. Based on previous data examining the utility of the 3 oz. WST in patients post-stroke²² and in patients with suspected dysphagia in the inpatient hospital setting,²³ we hypothesized that the 3 oz. WST would have high sensitivity and low specificity for detecting aspiration in people with ALS.

Methods and Materials:

Study participants:

This prospective research study was approved by our University’s Institutional Review Board and all study participants provided written informed consent. Inclusion criteria were: 1) confirmed diagnosis of probable or definite ALS based on the El-Escorial criteria revised,²⁵ 2) consuming some form of oral intake by mouth, 3) no allergies to barium, and 4) not pregnant. The present study was part of a longitudinal natural history study wherein participants attended serial research evaluations every three months. For this report, collected data from research study visits where participants underwent both the 3 oz. WST and a videofluoroscopic swallowing study (VF) were included.

Procedures:

3 oz. WST:

Participants were seated in a Trans-Motion Medical TMM3 Videofluoroscopy Swallow Study Treatment Chair (Ocala, FL) and the 3 oz. WST was administered following previously published guidelines.^22,23 The research speech-language pathologist (SLP) presented participants with 3 oz. (90cc) of room temperature purified water in a 10 oz. clear graduated plastic cup to ensure accurate volume measurement (Medline Graduated Disposable Plastic Drinking Cups, The Betty Mills Company, Inc., San Mateo, CA) . Participants were given the option to perform the test with or without a straw, based on how they typically drink liquids at home and participant preference. The SLP gave the instruction “Here is a cup of water. I would like you to continuously drink this, without stopping.” The testing was video recorded with a standalone camcorder (Canon Vixia HFR30) to allow for subsequent viewing. After participants finished the 3 oz. WST, they were observed for up to one minute for overt signs of aspiration including coughing or throat clearing or the inability to drink continuously without stopping. The research SLP recorded whether the participant passed or failed the 3 oz. WST per established criteria and noted the reason for failure (Pass: uninterrupted task completion with no cough or throat clear; Fail: interrupted drinking or presence of a cough or throat clear during or immediately following the drinking task).

Videofluoroscopic Swallowing Study:

Participants underwent a standardized VF procedure in a comfortable upright seated position in the same chair used for the 3 oz. WST. VF images were obtained in a lateral plane using continuous fluoroscopy with a Phillips BV Endura System fluoroscopic C-arm unit (GE OEC 8800 Digital Mobile C-Arm System). A TIMS Dicom system (Version 3.2, TIMS Medical, TM, Chelmsford, MA) recorded VF images at a rate of 30 frames per second and in segmented bolus trial clips. The following standardized bolus protocol was administered using Varibar barium sulfate products (Bracco Imaging, Monroe Township, NJ), which have been mapped to the IDDSI framework accordingly: thin liquid trials (IDDSI level 0), thin honey trials ( IDDSI level 3), pudding trials (IDDSI level 4), and graham crackers (IDDSI level 7).²⁶ All swallows were cued unless noted: one saliva swallow, three trials of 5mL thin liquid via a 30mL medicine cup, one comfortable cup sip of thin liquid from a cup filled to 90mL (uncued), one sequential drinking challenge of the remaining thin liquid from the 90mL cup (uncued), three trials of 5mL thin honey via spoon, two trials of 5mL pudding via spoon, ¼ graham cracker coated with pudding, and a 13mm EZ-Disk barium tablet. For command swallows, participants were instructed to “Hold the bolus in your mouth and wait until I tell you to swallow it” and “Swallow now.” For the comfortable cup sip and sequential cup sip trials, participants were instructed to “Take a comfortable sip and swallow whenever you’re ready” and to “Drink the remaining liquid in this cup continuously until it’s gone.” If participants had physical mobility limitations due to ALS limb involvement, the research SLP assisted with administering bolus trials. VF examinations were terminated if participants had three episodes of gross aspiration and/or if they had >75% pharyngeal residue despite efforts to clear residue from the pharynx.

Data Analysis:

Two trained raters performed duplicate and blinded analysis of VF images using the validated penetration-aspiration scale (PAS).^27,28 PAS scores were derived for each swallow of every administered bolus trial. One hundred percent agreement between independent raters was required with meetings held for instances of discrepant ratings. Following previously utilized protocols, the worst PAS score across all trials was utilized for subsequent statistical analyses.²⁹ PAS scores were classified as follows: 1–2 “safe”, 3–5 “penetration”, and 6–8 “aspiration.” Data were input into our secure online database (Research Electronic Data Capture [REDCap])^30,31 by trained research assistants.

Statistical Analysis:

Data were extracted from our secure online REDCap^30,31 database and exported into R Studio³² and Statistical Package for the Social Sciences (SPSS, Version 27)³³ for statistical analysis. Descriptive statistics were used to characterize participant demographics, PAS scores, and 3 oz. WST outcomes. Receiver operating characteristic (ROC) curve analysis, area under the curve (AUC), sensitivity, specificity, and positive and negative predictive value (PPV, NPV) were computed.

Results:

A total of 212 corresponding 3 oz. WSTs and VF exams were performed in 76 participants (median number of test visits per participant: 2). Demographic information at the baseline research visit is provided in Table 1. Percentage agreement across raters (inter-rater agreement) for raw PAS scores before discrepancy meetings was 89%. Data are summarized in Table 2

Table 1:

Summary of participant demographics.

Patient characteristics (N=76).	Mean (SD)
Age (years)	63.51 ± 10.99
Duration of Symptoms (months)	23.8 ± 18.8
ALSFRS-R Total Score	36.2 ± 7.26
ALSFRS-R Bulbar Score	9.38 ± 2.40
	Frequency (%)
Sex
Male	37 (48.7%)
Female	39 (51.3%)
Race
White	70 (92.1%)
Black	3 (4.0%)
Native Hawaiian or other Pacific Islander	1 (1.3%)
Unknown	2 (2.6%)
Ethnicity
Not Hispanic	71 (93.4%)
Hispanic	4 (5.3%)
Unknown	1 (1.3%)
Disease Onset type
Spinal	39 (51.3%)
Bulbar	32 (42.1%)
Mixed	5 (6.6%)

Table 2:

Two-by-two contingency table demonstrating aspiration status based on videofluoroscopic images and pass/fail screening results based on the 3oz water swallow test in people with amyotrophic lateral sclerosis.

		Instrumental Exam
		Aspirator	Non-aspirator
Screen	Fail	37 (true +)	41 (false +)	n=78
	Pass	30 (false −)	104 (true −)	n=134
		n=67	n=145

3 oz. WST (Index Test):

Of the 212 3-oz WSTs performed, 134 tests were rated as a pass (63.2%) and 78 rated as a fail (36.8%). Specific fail criteria included coughing (n=59, 75.6%) and the inability to complete the task without interruption (n= 31, 40.8%).

Videofluoroscopy (Reference Test):

PAS scores for the 212 VF exams performed are presented in Figure 1, revealing that 41.5% of exams were classified as safe (n=88), 26.9% with confirmed penetration (n=57), and 31.6% revealed aspiration (n=67). Therefore, 68.4% were classified as non-aspirators and 31.6% as aspirators.

Figure 1:

Distribution of penetration aspiration scale scores.

ROC Analysis:

The frequency of aspiration status by 3-oz. WST screening result (pass/fail) across the 212 research exams is presented in Table 2 and the resultant ROC curve is presented in Figure 2. Obtained AUC was 0.63 (95% CI: 0.55, 0.72); sensitivity: 55.2% (95% CI: 43, 67); specificity: 71.7% (95% CI: 64, 79), PPV: 47.4% (95% CI: 36, 59), NPV: 77.6% (95% CI: 70, 84); false positive rate: 28.3%, and false negative rate: 44.8%. The overall accuracy for the 3 oz WST test to detect aspiration was 66.5% (95% CI: 60, 73).

Figure 2:

Receiver operating characteristic curve for determining aspiration status in people with amyotrophic lateral sclerosis.

Discussion:

Results revealed that the 3-oz WST screening tool was inadequate in the ALS population, demonstrating low sensitivity to detect radiographically confirmed aspiration (55.2%) and a moderate specificity to identify radiographically confirmed non-aspiration events (71.7%). Airway safety status was misclassified in 71 tests, with 30 false negatives (aspirators who passed the 3 oz. WST) and 41 false positives (non-aspirators who failed the 3 oz. WST). The NPV for this data set was 77.6%, implying that ~78% of the 3 oz. WST ‘pass’ results, were non-aspirators. In contrast, the PPV was only 47.4%, indicating that only 47% of the 3 oz. WST ‘fail’ scores were aspirators. Thus, more than 50% of people with ALS who underwent the 3 oz. WST and aspirated would not have been identified. These findings illustrate that the 3 oz. WST does not have sufficient sensitivity and specificity for detecting aspiration in people with ALS.

The discriminant ability observed in the current investigation in people with ALS differs from the original validation study of the 3 oz. WST, which examined the accuracy of this screening method in a large cohort of hospitalized inpatients where it demonstrated a superior sensitivity of 96.5% but an inferior specificity of 48.7%.²³ Comparison of obtained NPV and PPV in our ALS cohort to the original validating study revealed that the 3 oz. WST demonstrated a lower NPV (77.6% vs. 97.9%) but higher PPV (47.4% vs. 35.9%). The current findings are, however, comparable to a recent investigation in 196 postoperative cardiac surgical patients who underwent the 3 oz WST and fiberoptic endoscopic evaluations of swallowing with blinded PAS ratings. In this study, Dallal York and colleagues reported both a sensitivity and specificity of 63% for the 3 oz. WST to detect aspiration, a NPV of 82%, and a PPV of 39%.²⁴

Several recent investigations examined the sensitivity and specificity of non-invasive clinical tools in people with ALS. A patient reported outcome measure, the Eating Assessment Tool-10 (EAT-10) demonstrated superior clinical utility to the 3 oz WST with an EAT-10 score > 8 yielding a sensitivity of 86% and specificity of 72% to identify radiographically confirmed aspiration.³⁴ Further, voluntary cough metrics including cough volume acceleration and peak expiratory flow rate demonstrated a higher sensitivity of 91 and 83%, and specificity of 82 and 74%, respectively, to detect radiographically confirmed aspiration.³⁵ Finally, similar to the current study, the ALSFRS-R bulbar subscale demonstrated inadequate discriminant ability to detect aspiration in people with ALS with a sensitivity of 78% and specificity of 62%.¹⁶

A potential explanation for the noted low sensitivity of the 3-oz. WST to detect aspiration in the current study could relate to the known high rates of ‘silent’ aspiration (no cough or throat clear) in this patient population.^4,12,36,37 Further, blunted upper airway sensory responses (i.e., reduced urge-to-cough) were reported in ALS silent aspirators, indicating degradation of airway sensory receptors responsible for eliciting the cough motor response used to score the 3-oz. WST.³⁸ Indeed, the most common aspiration response profile observed during VF was no cough or throat clear, with silent aspiration noted in 52% of observed aspiration episodes.

The suboptimal specificity (correctly classifying non-aspirators) in the current study may be attributable to disrupted respiratory-swallow coordination due to the insidious respiratory decline inherent in ALS. People with ALS may compensate for decreased tidal volume by increasing their respiratory rate to maintain adequate minute volume. While this is a necessary compensation to maintain homeostasis, as respiratory rate increases to be ≥30, the duration of a breath and a swallow become equal, which increases the likelihood that maladaptive respiratory-swallow patterns (e.g., inhaling after the swallow) emerge,³⁹ increasing a patient’s risk of aspirating.⁴⁰ Given that the 3 oz WST is a task that requires prolonged swallow duration, it is possible that people with ALS may demonstrate interrupted cup drinking due to increased respiratory rate.

Future research studies may consider investigating respiratory-swallow patterns of people with ALS during sequential cup drinking to determine whether this may be an underlying reason for the noted 41 false positives (non-aspirators who failed the 3 oz. WST). In addition to this, future studies in people with ALS are warranted to examine the diagnostic utility of other recommended clinical dysphagia screening methods such as the Yale Swallow Protocol,⁴¹ measurements of isometric tongue strength, pulmonary function, and reflexive/voluntary cough metrics.¹⁹ While it is likely that a combinatorial approach incorporating several screening items may prove more accurate than any singular dysphagia screening item, these data do not support the inclusion of the 3-oz WST for detecting aspiration status in individuals with ALS. For example, the use of a patient reported outcome measure combined with measurements of isometric tongue strength and the 3 oz. WST may lead to improved sensitivity and specificity for detecting aspiration in people with ALS compared to each individual test. These examples provide future directions for work in this area. Future research studies should also determine whether the diagnostic utility of dysphagia screening tools change over time in people with ALS due to disease progression, and subsequent progression of swallowing impairments. Investigating the natural longitudinal progression of swallowing impairments in people with ALS is an important first step to determining this.

Limitations:

Although we included a relatively large data set of 212 tests, it is possible that the 3 oz. WST may perform better in a larger group of people with ALS given that one of the original validation studies included a cohort of 3000 patients.²³ In addition, patients were recruited from a single geographic region, limiting the geographic diversity of our sample. Another important limitation to note is that we could only include data from research visits in which people with ALS completed both the 3 oz. WST and the standardized VF procedure. For example, some people with ALS in our longitudinal research study elect to only receive one VF procedure per year rather than undergoing a VF at each visit. As a result, our data set may have biased the study findings. Future studies may consider replicating this study by conducting a large, multi-site study to ensure a large and heterogenous sample that can be further stratified. Additionally, ALS is a progressive neurodegenerative disease, and as such, swallow function in people with ALS is not stable over time. Although not examined in the current research study, future research studies may consider examination of the discriminant ability of the 3 oz. WST longitudinally in people with ALS to determine whether the sensitivity and specificity of this dysphagia screening method changes with the temporal evolution of swallowing in people with ALS.

Data availability statement:

Data available on request due to privacy/ethical restrictions.