A simple bedside stroke dysphagia screen, validated against video-fluoroscopy, detects dysphagia and aspiration with high sensitivity

Abstract

Background

Early identification of dysphagia is associated with lower rates of pneumonia after acute stroke. The Barnes-Jewish Hospital-Stroke Dysphagia Screen (BJH-SDS) was previously developed as a simple bedside screen performed by nurses for sensitive detection of dysphagia and was previously validated against the speech pathologist’s clinical assessment for dysphagia. In this study, acute stroke patients were prospectively enrolled to assess the accuracy of the BJH-SDS when tested against the gold-standard test for dysphagia, the video-fluoroscopic swallow study (VFSS).

Methods

Acute stroke patients were prospectively enrolled at a large tertiary care inpatient stroke unit. The nurse performed the BJH-SDS at the bedside. After providing consent, patients then underwent VFSS for determination of dysphagia and aspiration. The VFSS was performed by a speech pathologist who was blinded to the results of the BJH-SDS. Sensitivity and specificity were calculated. Pneumonia rates were assessed across the five year period over which the BJH-SDS was introduced into the Stroke Unit.

Results

A total of 225 acute stroke patients were enrolled. Sensitivity and specificity of the screen to detect dysphagia were 94% and 66%, respectively. Sensitivity and specificity of the screen to detect aspiration were 95% and 50%, respectively. No increase in pneumonia was identified during implementation of the screen (p=0.33).

Conclusion

The BJH-SDS, validated against video-fluoroscopy, is a simple bedside screen for sensitive identification of dysphagia and aspiration in the stroke population.

Introduction

Dysphagia is a well-recognized complication of both acute ischemic and hemorrhagic stroke. Its prevalence varies depending on the method and timing of the evaluation, affecting between 37 and 78% of acute stroke patients.[1, 2] Post-stroke dysphagia is independently associated with pneumonia, the latter which is known to significantly increase the burden of stroke, by causing greater morbidity, mortality, and healthcare costs.[2–4] Formal dysphagia screening protocols have been associated with significant reductions in pneumonia risk following stroke.[5, 6]

The Barnes-Jewish Hospital Stroke Dysphagia Screen (BJH-SDS) was developed in 2006 as a simple bedside tool for identifying dysphagia in the Stroke Unit at Barnes-Jewish Hospital.[7] Prior to 2006, the speech language pathologist (SLP) was required to evaluate every stroke patient in the hospital for possible dysphagia. Such requirements were not only time and labor intensive for the hospital’s speech therapy service, but also resulted in numerous patients being kept without food for unnecessarily long periods of time. In our previous study, the BJH-SDS demonstrated simplicity (timed to take less than two minutes on average) and high intra- and inter-rater reliability (94 and 92%, respectively) amongst hospital nurses.[7] It was also found to have high sensitivity and moderate specificity when validated against the clinical bedside swallow test, the Mann Assessment Swallowing Ability (MASA).[8]

Given the promising early findings of the BJH-SDS with regard to its simplicity, reliability, and accuracy when tested against the MASA, the next step and aim of the current study was to validate the BJH-SDS against the gold-standard for dysphagia and aspiration detection, the video-fluoroscopic swallow study (VFSS). A secondary aim was to evaluate rates of pneumonia over the period of time the BJH-SDS was introduced into the Stroke Unit to assess for any compromise in patient safety during screen implementation.

Methods

Data collection and Administration of the BJH-SDS

The study was approved by the institutional review board to ensure ethical conduct of research studies with human participants. Written informed consent was obtained in all participants. Acute stroke patients were prospectively enrolled from the Barnes-Jewish Hospital inpatient stroke service (an urban, tertiary care referral center admitting 1300 stroke patients annually). Criteria for inclusion were clinical diagnosis of stroke (either ischemic or hemorrhagic) and age ≥ 18 years. Patients with decreased level of alertness preventing participation in the VFSS (defined as a score of 2 on the “Alertness” component of the MASA) were excluded. Patients with physical limitations preventing the ability to sit upright (ex. intubation or if the treating physician had ordered the patient to have the head of the bed flat) were excluded. Patients with confirmed or suspected pregnancy were excluded. The components of the BJH-SDS (Figure) were chosen based on several guiding principles including ease of administration, ease of interpretation (objective rather than subjective findings), and pre-existing research supporting each item’s relationship to dysphagia. The rationale supporting the design of the BJH-SDS was provided in the previous study. ⁷

Figure.

The Barnes-Jewish Hospital Stroke Dysphagia Screen (BJH-SDS)

After the patient was admitted to the Stroke Unit, the patient’s nurse administered the BJH-SDS and recorded the screen results in the patient’s chart. This study was not meant to test nursing ability to perform the screen as this was demonstrated in the previous study, in which 50 nurses demonstrated inter-rater reliability and test-retest reliability (measured by Cohen κ) to be 94% and 92%, respectively. The screen result was recorded as “fail” if any one of the 5 items tested were abnormal (Glasgow Coma Scale < 13, facial/tongue/palatal asymmetry or weakness, or signs of aspiration on the 3 ounce water test) or “pass” if all 5 items tested were normal. After the BJH-SDS was completed, the VFSS was performed within 8 hours to avoid significant change in the neurological examination (mean 2 hours; range 0–8 hours).

Detection of Dysphagia and Aspiration on the VFSS

For the VFSS, subjects were assigned a licensed SLP who was blinded to the results of the BJH-SDS. The Dysphagia Outcomes Severity Scale (DOSS) was utilized as the functional scale for identifying dysphagia on the VFSS.[9] This 7-point scale, chosen for its high reliability, was scored based on diet recommendations, level of assistance, and modifications required for safe oral intake. A score of ≤5 on the DOSS was the pre-specified definition for dysphagia. The New Zealand Index of Multidisciplinary Evaluation of Swallowing (NZIMES) was utilized as the functional scale for identification of aspiration on the VFSS. [10] A score of ≥ 2 was the pre-specified definition for aspiration. The NZIMES was chosen as it has a cut-off for clearly defining aspiration below the level of the true vocal cords. The SLP recorded each VFSS on a DVD which was reviewed with the radiologist (who was also blinded to the BJH-SDS results) until consensus was reached.

Statistical analysis

The primary aim was to measure the sensitivity, specificity, and positive and negative predictive values of the BJH-SDS for detection of dysphagia and aspiration as identified by the gold-standard, VFSS. A power analysis, based on a 35% prevalence of dysphagia, determined that 225 subjects would be needed to provide precise estimates of sensitivity and specificity, within 10% of the true values.

The secondary aim was to assess any major deleterious impact on pneumonia rates with the utilization of the BJH-SDS in the Stroke Unit. Prior to the development of the BJH-SDS, it was the responsibility of the SLP to screen patients for dysphagia following stroke. Between 2006 and 2008 there was a gradual transition of the screening responsibility to nursing staff with the SLP intervening for those patients that failed the BJH-SDS. By 2008, nursing had completely assumed responsibility for screening swallow function by using the BJH-SDS. A retrospective analysis was performed on all patients with a primary stroke diagnosis ICD-9 code (431, 432.9, 433, 434 with a fifth digit of 1, and 435) who were admitted between 1/1/2006 to 12/31/2010. The secondary ICD-9 diagnosis codes for pneumonia (480, 481, 482, 486, and 507) were collected for each stroke patient to determine annual pneumonia rates (2006–2010). To evaluate homogeneity across the 5 years, age, gender, and length of stay (LOS) were compared. Gender and pneumonia rates were compared with chi-square tests followed by Cochran-Armitage tests (CA) for trend to identify any linear association that may exist between these proportions and admission year. Age and LOS was compared across admission years by analysis of variance (ANOVA) followed by Spearman correlations (ρ) to assess the association between these variables and admission year.

Results

Between January 2009 and June 2010, 225 acute ischemic stroke patients provided written consent and were prospectively enrolled. To ensure that our patient sample was representative of the general stroke population admitted to our hospital, the mean age, gender, race, and median stroke severity as measured by admission National Institutes of Health Stroke Scale (NIHSS) of the study population was compared to all stroke patients admitted to Barnes-Jewish Hospital between January 2009 and June 2010 (N=1,821). No significant differences were identified (Table 1).

Table 1.

Baseline characteristics of study sample in comparison to stroke population

	Study Sample Jan 2009–June 2010	Stroke Population Jan 2009–June 2010	P-value
Age, mean ± standard deviation*	63 ± 15	64 ± 15	0.79
Female, %†	49.1%	53%	0.27
Race†
Caucasian, %	56.6%	55.6%
African American, %	42.4%	42.4%	0.55
Other, %	0.9%	2.0%
NIHSS, median [interquartile range]‡	5 [2, 13]	5 [1, 11]	0.08

^*Students t-test

^† Chi-square test

^‡ Wilcoxon’s test

NIHSS=National Institutes of Health Stroke Scale

Accuracy of the BJH-SDS for detection of dysphagia and aspiration

Sensitivity, specificity, and positive and negative predictive values of the screen for detection of dysphagia as measured by VFSS were: 94% (95% CI 88%–98%), 66% (57%–75%), 71% (63%–79%), and 93% (85%–97%), respectively. Sensitivity, specificity, positive and negative predictive values for detection of aspiration were: 95% (86%–99%), 50% (42%–58%), 41% (33%–50%), 96% (90%–99%), respectively (Table 2). Two patients were excluded from the aspiration analysis due to oral dysfunction so severe that they were unable initiate a pharyngeal swallow response.

Table 2.

Accuracy of BJH-SDS for detection of dysphagia and aspiration validated against video-fluoroscopic swallow study (VFSS)

BJH-SDS vs. VFSS for Dysphagia*	Screen Pass	Screen Fail	Total
Dysphagia on VFSS	6	100	106
No Dysphagia on VFSS	79	40	119
Total	85	140	225
Sensitivity	94% (95% CI†=88%–98%)
Specificity	66% (95% CI=57%–75%)
Positive Predictive Value	71% (95% CI=63%–79%)
Negative Predictive Value	93% (95% CI=85%–97%)

BJH-SDS vs. VFSS for Aspiration+	Screen Pass	Screen Fail	Total
Aspiration on VFSS	3	57	60
No Aspiration on VFSS	82	81	163
Total	85	138	223§
Sensitivity	95% (95% CI 86%–99%)
Specificity	50% (95% CI 42%–58%)
Positive Predictive Value	41% (95% CI 33%–50%)
Negative Predictive Value	96% (95% CI 90%–99%)

^*Prevalence = 47%

^†CI=Confidence Interval

⁺Prevalence = 27%

^§ Two subjects who failed the screen were excluded due to severe oral dysphagia preventing swallow response during VFSS.

Rates of Pneumonia during Implementation of the BJH-SDS

We wanted to ensure that the simplicity of the new screen was not at the expense of an increase in dysphagia-related complications, specifically pneumonia. Over the five year period during which the BJH-SDS was introduced, 4961 patients were admitted with primary diagnosis of stroke. There were no significant differences across admission years for gender (p=0.33 by chi-square; p=0.16 by CA), pneumonia rates (p=0.33 by chi-square; p=0.18 by CA), or age (p=0.62 by ANOVA). There was no significant linear association between age and admission year (ρ=−0.01, p=0.31). A significant reduction in LOS across the admission years was noted (p=0.003 by ANOVA using rank-transformed data), which suggested that the population over the five years could have changed. However, given the median LOS across groups remained 3.0 days and no correlation between LOS and admission year (ρ=−0.04) was found, population homogeneity over the five years was confirmed.

Discussion

In 2003, the Joint Commission, in collaboration with the American Stroke Association, developed performance measures for Primary Stroke Centers.¹⁷ This resulted in 10 harmonized measures that were subsequently implemented by the Stroke Performance Measure Consensus Group in 2006.¹⁷ Recognizing the high prevalence of dysphagia following stroke, a measure requiring dysphagia screening prior to oral intake was included. Importantly, this measure required that the screening tool be an “evidence-based bedside testing protocol”. Hinchey et al. (2005) demonstrated that dysphagia screening was associated with better patient outcomes than no screen.[5] Moreover, protocols including a checklist and water swallow test resulted in the best patient outcomes. [5, 11] In 2007, the National Quality Forum issued a recommendation to eliminate the dysphagia screen as a core stroke measure,¹⁴ a recommendation adopted by Joint Commission in 2010.¹⁹ The rationale for removing this measure was the lack of a valid, reliable, standardized screening tool rather than the lack of importance in identifying dysphagia.

The BJH-SDS validation studies (including both the previous and current studies) include several important strengths with regard to study design and testing for an acute stroke dysphagia screen. While several other dysphagia tools have been studied in acute stroke patients, [12, 13] few have included as comprehensive rigorous methods as performed here. These characteristics include: previously tested inter and intra-rater reliability, performing the VFSS within 8 hours of initial screening (mean 2 hours), blinding of the screen results to the SLP performing the VFSS, adequately powering the study to precisely estimate sensitivity, and validation against both the clinical bedside swallow test (MASA) and the gold-standard radiological swallowing test (VFSS). Moreover, the screen is generalizable to the real-world busy hospital acute stroke setting in which nurses who are not specialists in dysphagia will be administering the dysphagia screen on admission at the bedside. The training for the BJH-SDS takes 10 minutes with an average test completion time of two minutes per patient. Given the screen’s high sensitivity, we would not anticipate an increase in pneumonia rate, and when examined longitudinally, there was no increase as the screen was implemented by nursing staff. While the BJH-SDS demonstrated high sensitivity, the specificity was only moderate: 66% for dysphagia and 50% for aspiration.

While specificity is often sacrificed in favor of sensitivity for screening tests, the downside is that patients who have normal swallowing function are delayed in resuming a normal diet until they are seen by a SLP. The BJH-SDS specificity is at the higher end of the range seen across other dysphagia screens, which varies from 49–67%. [7, 11] While ensuring high sensitivity for dysphagia screening is important to avoid placing patients at increased aspiration risk, if future screens could demonstrate high sensitivity, simplicity, reliability, and improve specificity, then patient satisfaction and nutritional status would likely improve.

This study has limitations. While the DOSS for identifying dysphagia was previously validated and shown to be reliable, we did not re-demonstrate inter-rater reliability within our study team, although the SLPs performing the VFSS interpretation are highly trained in performing and scoring the DOSS as it is a scale used clinically. While used clinically, inter-rater reliability for the NZIMES aspiration component is not well documented in the literature and was not established for the SLPs performing VFSS interpretation. Based on previous studies that indicated high reliability by SLP’s when identifying aspiration on the VFSS, using the aspiration component of the NZIMES as a binary indicator of aspiration was deemed sufficient for the purpose of this study. Our assessment of pneumonia rates in relation to screen implementation is limited by our gradual introduction of the screen into the Stroke Unit which prevents a direct comparison of pneumonia rates before and after the screen was introduced. Moreover, there are several other clinical variables that could impact pneumonia rates besides dysphagia screening.

Conclusions

The BJH-SDS is a simple bedside screening tool that can be used by nursing staff to efficiently and sensitively identify swallowing impairments in the acute stroke population.