Validation of a Dysphagia Screening Tool in Acute Stroke Patients

Abstract

Background

Although many dysphagia screening tools exist, none has high sensitivity and reliability or can be administered quickly with minimal training.

Objective

To design and validate a swallowing screening tool to be used by health care professionals who are not speech language pathologists to identify dysphagia and aspiration risk in acute stroke patients.

Methods

In a prospective study of 300 patients admitted to the stroke service at an urban tertiary care hospital, interrater and test-retest reliabilities of a new tool (the Acute Stroke Dysphagia Screen) were established. The tool was administered by nursing staff when patients were admitted to the stroke unit. A speech language pathologist blinded to the results with the new tool administered the Mann Assessment of Swallowing Ability, a clinical bedside evaluation, with dysphagia operationally defined by a score less than 178.

Results

The mean time from admission to screening with the new tool was 8 hours. The mean time between administration of the new tool and the clinical bedside evaluation was 32 hours. For the new tool, interrater reliability was 93.6% and test-retest reliability was 92.5%. The new tool had a sensitivity of 91% and a specificity of 74% for detecting dysphagia and a sensitivity of 95% and a specificity of 68% for detecting aspiration risk.

Conclusions

The Acute Stroke Dysphagia Screen is an easily administered and reliable tool that has sufficient sensitivity to detect both dysphagia and aspiration risk in acute stroke patients.

One of the most common and earliest problems to emerge after a stroke is difficulty swallowing (dysphagia). Depending on the method of assessment and how dysphagia is defined, the prevalence of dysphagia in stroke patients ranges from 30%¹ to 67%.² Dysphagia may result in aspiration, which occurs in approximately 20% to 25% of stroke patients.^3-5 Swallowing abnormality is associated with higher mortality⁴ and is strongly correlated with pneumonia.^3,6 Pneumonia is approximately 7 times more likely to develop in patients who aspirate after stroke than in patients who do not.⁷ In addition to mortality and chest infection, a significant relationship exists between dysphagia and malnutrition.⁸

The Joint Commission has recognized the impact that dysphagia has on stroke patients. As part of its standardized stroke measure set for primary stroke centers, the Joint Commission requires that all patients admitted with a stroke diagnosis be screened before they can resume an oral diet.⁹ Perry¹⁰ identified 6 screening tools that are available: the “Any Two,” the Burke Dysphagia Screening Test, the Timed Test, the 3-oz water swallow test, the Bedside Swallowing Assessment, and the Standardized Swallowing Assessment. The Gugging Swallowing Screen, published in 2007, is the most recent addition to the swallowing screening tools that are available.¹¹ Although many tools are available, none of them meets essential screening criteria: high sensitivity, high reliability, quick administration, and minimal training for reliable administration.

In the “Any Two” screening study,¹² consecutive patients were recruited within 5 days of acute hospital admission for stroke. Patients who were agitated, had clouding of consciousness, or had a history of oropharyngeal dysphagia, oropharyngeal structural damage, or other neurological disease that could produce dysphagia were excluded. All patients were examined with computed tomography or magnetic resonance imaging to confirm the stroke. Concurrent validity of the Any Two screening was established with a videofluoroscopic evaluation of swallowing. Six clinical features were identified in that study: dysphonia; dysarthria; abnormal, volitional cough; abnormal gag reflex; cough after swallow; and voice change after swallowing. The presence of any 2 of those features was considered to indicate dysphagia. Table 1 shows the sensitivity and specificity of the Any Two screening; however, its reliability was not reported.

The prevalence of dysphagia in stroke patients ranges from 30% to 67%.

Table 1.

Sensitivity, specificity, and reliability for dysphagia screening methods

	"Any two"	BDST	Timed test	3-oz water swallow test	BSA	SSA	GUSS	ASDS
No. of subjects	59	139	115	44	129	200	50	300
Concurrent validity	Yes with VFSS	No	No	Yes with VFSS	Yes with VFSS	No	Yes with FEES	Yes with MASA
Sensitivity: dysphagia, %	92% for at least moderate dysphagia	NR	NR	NR	NR	97	NR	91
Sensitivity: aspiration, %	NR	NR	100	76	68	NR	100	95
Interrater reliability, %	NR	NR	NR	NR	NR	88	83	92
Test-retest reliability, %	NR	NR	NR	NR	NR	NR	NR	93
Specificity, %	67	NR	52	59	67	90	50	Aspiration,74 Dysphagia, 68

Abbreviations: ASDS, Acute Stroke Dysphagia Screening; BDST, Burke Dysphagia Screening Test; BSA, Bedside Swallowing Assessment; FEES, fiberoptic endoscopic evaluation of swallowing; GUSS, Gugging Swallowing Screen; MASA, Mann Assessment of Swallowing Ability; NR, not reported; SSA, Standardized Swallowing Assessment; VFSS, videofluoroscopic swallowing screening.

In the study of the Burke Dysphagia Screening Test,¹³ patients admitted to an inpatient rehabilitation unit were recruited. All were evaluated at a mean of approximately 5 weeks (SD, 3 weeks) from the time of stroke for the following reasons: bilateral hemispheric stroke, brainstem stroke, pneumonia during the acute stroke phase, coughing associated with feeding or during a 3-oz water swallow test, persistent failure to consume half of meals, prolonged time required for feeding, and non oral feeding program in progress. Data on the first 3 items were obtained via chart review and the rest were obtained by observing a patient during a meal. No tool for used for concurrent validity, and participants “failed” the screening if pneumonia or recurrent upper airway obstruction developed. Neither sensitivity nor specificity was reported for either aspiration or dysphagia (Table 1). The authors claim that this is not a dysphagia screening but rather a screening for dysphagia-related medical complications.

Only 2 of the 7 screening tools had reliability reported and none had test-retest reliability reported.

In the Timed Test study,¹⁴ patients admitted with suspected stroke were recruited; evaluation occurred within 72 hours of acute hospital admission. The swallowing screening consisted of an initial presentation of a single teaspoon of water that was followed by the timed intake of 100 to 150 mL of water. Abnormality was defined as either swallowing capacity outside the 95% prediction interval for age and sex or coughing during the test or wet/hoarse voice after the test. The validity of this screening was evaluated by comparing the results with symptoms, by a referral to a speech language pathologist, and by the need for intervention by a speech language pathologist. Reliability was not reported, but sensitivity and specificity are noted in Table 1.

In the 3-oz water swallow test study,¹⁵ forty-four sequential patients from a stroke rehabilitation unit were recruited. Inclusion criteria were 1 or more of the following features: bilateral hemispheric stroke, brainstem stroke, pneumonia during the acute stroke phase, coughing associated with feeding, failure to consume half of meals, prolonged time required for feeding, and nonoral feeding program in progress. Any patient who had 1 or more of these features was referred for a 3-oz water swallow test. The patient swallowed 3 oz of water uninterrupted, and abnormality was defined by coughing during swallowing or 1 minute after completion. The presence of a wet/hoarse voice quality after swallowing also was scored as abnormal. Concurrent validity was established by means of videofluoroscopic evaluation of swallowing. The authors did not report reliability, but sensitivity and specificity were reported (Table 1).

Our goal was to create a simple dysphagia screen that health care professionals could use to detect swallowing difficulty in stroke patients quickly and accurately.

In the Bedside Swallowing Assessment study,¹⁶ consecutive patients were recruited 24 hours after admission with a confirmed diagnosis of stroke. Patients were excluded if they were outside the 24-hour window, had severe illness, or if consent was refused or could not be obtained. The swallowing ability of the participants was assessed by a physician daily for the first 4 days and then at day 7, at day 28, and at 6 months. A speech language pathologist also assessed the participants up to 3 times during the first week and then again at day 28 and at 6 months. The clinical bedside assessment by the physician included assessment of consciousness, head and trunk control, breathing pattern, lip closure, palatal movement, laryngeal function, voluntary cough, and gag. The physician then presented the patient with 5 mL of water 3 times and then 60 mL of water.

The assessment by the speech language pathologist included similar criteria to those in the physician's assessment but did so in a much more detailed manner (eg, considered lip closure at rest, during eating/drinking, and in speech rather than simply a global statement of lip closure normal/abnormal). Patients were seen as having a compromised swallow (ie, they failed the screening) if they were considered to be at risk of aspirating after the clinical bedside evaluation. The authors did not present the criteria for what defined aspiration risk. Concurrent validity was established with the modified barium swallow. Reliability was not reported, but sensitivity and specificity for aspiration are noted in Table 1.

In the Standardized Swallowing Assessment study,¹⁷ patients admitted with the clinical diagnosis of stroke were recruited. The screening consisted of 3 sections. The first section ensured that the patient was physically able to participate in the screening and included level of alertness/responsiveness and the ability of the patient to be positioned upright with some degree of head control. The second section evaluated voluntary cough, salivary management, the ability to lick the top and bottom lip, respiratory function, and vocal quality. If the patient showed abnormalities in any of these criteria, the screening was considered “failed” and a referral was made to a speech language pathologist. If all items on sections 1 and 2 were passed, section 3 involved 3 trials of water from a teaspoon. If no abnormality was noted, then a half glass of water was presented. If no dysfunction was noted, then an appropriate meal was ordered and the patient was supervised during the meal.

Abnormality was defined as coughing, choking, or breathlessness while swallowing, or a wet/gurgly voice after swallowing. The authors of that study used a summary judgment based on clinical indicators found in the patient's medical record as to the presence or absence of dysphagia. This judgment was used as the “gold standard” for identification of dysphagia and was what established concurrent validity for this screening. Reliability, sensitivity, and specificity for this screening are all reported in Table 1.

In the Gugging Swallowing Screen,¹¹ patients admitted to a stroke unit with a first-ever stroke and suspected dysphagia between Monday and Thursday were recruited. Exclusion criteria included multiple infarcts seen on computed tomography scans or magnetic resonance images, known dysphagia of another cause, or somnolence/coma. The screening consists of 2 subtests: 1 an indirect assessment and 1 a direct asessment of swallowing function. The first subtest was the indirect measure, which included assessment of alertness, voluntary cough, and salivary management. The second subtest was the direct swallowing assessment, which included trials with semisolids, liquids, and solids. Observation of the presence of, absence of, or delay in swallowing response was made. In addition, the presence of coughing before, during, or up to 3 minutes after swallowing was noted, as well as voice changes immediately after swallowing. The presence or absence of drooling was noted as well. Reliability, sensitivity, and specificity were all established by the authors (Table 1). Concurrent validity was established with fiberoptic endoscopic evaluation of swallowing.

Only 2 of the 7 screening tools studied had reliability reported, and none had test-retest reliability reported. Three of 7 screening tools did not use a reference-standard evaluation method to establish concurrent validity. All of the screening tools except the Burke Dysphagia Screening Tool reported sensitivity to either dysphagia or aspiration, but none reported sensitivity to both. The 3-oz water test and the Bedside Swallowing Assessment have reported sensitivities less than 80% for aspiration. The Any Two screening has a reported sensitivity of 92% for at least moderate dysphagia. For measures that reported it, specificity ranged from 50% to 90%, an acceptable range because these are screenings and the emphasis should be on high sensitivity, not high specificity. Two of the screening tools excluded comatose patients, which was a significant limitation, because decreased level of consciousness frequently occurs in acute stroke patients, and any screening tool for dysphagia must be able to be used in this subset of patients. Several of the screening tools have high sensitivity to dysphagia or aspiration, concurrent validity, and interrater reliability, although none of the screenings included all of these items. These limitations in the current screening tools led our facility to develop the Acute-Stroke Dysphagia Screen (ASDS; see Figure).

The Acute Stroke Dysphagia Screen requires minimal training and can be administered in less than 2 minutes.

Figure.

Screening tool developed to detect difficulty swallowing in stroke patients.

Our goal was to create a simple dysphagia screening that could be used by nurses and other health care professionals to detect swallowing difficulty quickly and accurately in patients who have had a stroke. To achieve that goal, we established the following criteria to guide the development of the ASDS: sensitivity to dysphagia and aspiration, high interrater and test-retest reliability, concurrent validity established by using an objective swallowing assessment, quick administration, and minimal training for a nurse or other health care professional to become proficient in its administration. We then conducted a study to validate this measure against a well-established measure for clinical bedside evaluation and to establish its interrater reliability for nurses who were to administer and interpret the results of the instrument.

The Acute Stroke Dysphagia Screen was administered concurrently with the Mann Assessment of Swallowing Ability to 300 patients with stroke.

Methods

This study was performed at Barnes Jewish Hospital within the Washington University Medical Center and was approved by the Human Research Protection Office at the Washington University School of Medicine. Informed consent was obtained from all participants before their data were included in this study.

Measure Development

In developing the ASDS, 4 guiding principles shaped the selection of the items comprising the measure: (1) clinical indicators could not be cumbersome to administer; (2) the measures had to be present, when possible, in the daily practice of the health care professionals administering the screening; (3) components of the screening had to be able to be evaluated objectively; and (4) each item had to have research evidence supporting its relationship to dysphagia. Based on these guiding principles, components of the ASDS included a measure of level of consciousness, several items that together indicated whether a patient had dysarthria, and a 3-oz water swallow test.

Smithard et al⁴ showed that level of consciousness of a patient was correlated with aspiration; therefore, a measure of level of consciousness was included in the ASDS. The Glasgow Coma Scale was chosen to quantify level of consciousness because it has long been accepted as a means to assess level of consciousness and it has high interrater reliability.¹⁸ To evaluate dysarthria, we turned to a 1999 study by Logemann et al,¹⁹ who did a retrospective analysis of 28 items and found that the best single predictor of oral dysphagia was the presence of dysarthria. Further, Daniels et al¹² reported a correlation between dysphagia and dysarthria.

For the purposes of the ASDS, dysarthria was deemed to lack sufficient objectivity to be assessed by professionals who were not speech language pathologists. Therefore, we included in our screening components contributing to dysarthria that can be evaluated objectively. Facial, lingual, and palatal symmetry were all included in the instrument because they are easy to evaluate visually and can independently, or in combination, contribute to dysarthria. In addition, if these 3 components are assessed, 4 out of the 5 cranial nerves that play a role in swallowing can be assessed.²⁰ The 3-oz water swallow was first proposed by DePippo et al.¹⁵ They showed that sensitivity of the 3-oz water swallow alone for dysphagia was only 76%, which is insufficient for a single-item screening mechanism. The Burke Dysphagia Screening Test included the 3-oz water swallow with other clinical indicators; its sensitivity for dysphagia-related medical complications was 92%.¹³

Establishing the Reliability of the Measure

Once the design of the screening tool was completed, all nurses on the stroke service were trained by a licensed speech language pathologist to administer it. The in-service training session for the nurses took approximately 10 minutes to complete. Fifty nurses then viewed a video and independently scored the ASDS (see Figure) to establish interrater reliability. Two weeks later, 15 of the original 50 nurses were randomly selected to view the video again and independently score the ASDS again to establish test-retest reliability. Once reliability was established, the ASDS was implemented in daily nursing practice. Each nurse on the stroke service was required to use the ASDS to screen his or her patients on admission to the unit. Each item on the ASDS was scored as either present or absent. If any of the items were marked as present, the patient failed the screening. If all of the items were marked as absent, the patient passed the screening.

During the implementation period, the department of speech language pathology continued its practice of evaluating swallowing for all patients who were admitted with stroke. The swallowing evaluation was done with the Mann Assessment of Swallowing Ability (MASA), a clinical bedside evaluation tool for stroke patients.²¹ The MASA has been validated against the modified barium swallow and provides a clear operational definition for both dysphagia and aspiration risk, as well as a severity scale. A MASA score of 178 is considered to be the cutoff for the absence of dysphagia, and a MASA score of 170 is considered to be the cutoff for the absence of aspiration risk. Interrater reliability for the MASA had been established before this study.

Results

Between June 14 and December 22, 2006, a total of 480 patients were admitted to the stroke service, 400 had at least 1 MASA, and 300 participants provided consent to participate in the study and were assessed with both the MASA and the ASDS. The mean time from admission to administration of the ASDS was 8 hours; the mean time from admission to initial administration of the MASA was 32 hours. For the 300 participants who were also administered the ASDS, results were analyzed to determine the sensitivity and specificity for dysphagia and the presence of aspiration risk. The prevalence of dysphagia in the study group was 29% (95% confidence interval, 21%-34%), quite close to the estimate of 34% based on the initial MASA score for 891 stroke patients seen by the stroke management and rehabilitation team service in 2006. Sensitivity for aspiration was 95% (95% confidence interval, 85%-98%) and specificity was 68% (95% confidence interval, 62%-74%; Table 1). Sensitivity for dysphagia was 91% (95% confidence interval, 82%-95%) and specificity was 74% (95% confidence interval, 64%-80%; Table 3). The positive predictive value for dysphagia was 54% and for aspiration was 44%; the negative predictive value for dysphagia was 95% and for aspiration 98%. Interrater reliability for the ASDS as measured by Cohen kappa was 93.6%, and the test-retest reliability was 92.5%.

The Acute Stroke Dysphagia Screen has high interrater reliability.

Table 3.

Frequency of failed and passed Acute Stroke Dysphagia Screening by cutoff scores for dysphagia on the Mann Assessment of Swallowing Ability

Result on Acute Stroke Dysphagia Screening	Score on Mann Assessment of Swallowing Ability
	≥178	<178
Failed	54	80
Passed	158	8

Discussion

This study was designed to create a brief screening tool that allows patients to resume the least restrictive diet as early as possible while minimizing aspiration risk. The ASDS requires minimal training and can be administered in less than 2 minutes. The reliability of this instrument is also very high. Further, the ASDS is good for detecting dysphagia and aspiration risk in patients, with a sensitivity greater than 90% for both (Table 1).

As an illustration, Barnes Jewish Hospital admitted 891 patients with a stroke diagnosis in 2006. With a dysphagia prevalence of 34%, 303 of these patients had dysphagia and 588 had no swallowing impairment. We estimate that the ASDS would have correctly detected dysphagia in 276 of the 303 patients with dysphagia. Of the 27 patients who would not have been identified, only 15 would have scored in the aspiration risk range on the MASA; less than 2% of the total. Without the screening instrument in place, the 588 patients with no swallowing impairment would have had to wait 32 hours to resume an oral diet. Use of the ASDS would have allowed 435 of those patients to resume an oral diet within 8 hours of admission.

The Acute Stroke Dysphagia Screen has high sensitivity and reliability.

This study had 3 limitations that will be addressed in future research by our team. The first issue is the 24-hour time difference between when the ASDS was administered and when the MASA was administered. The impact that spontaneous recovery had on our data must be investigated. The second concern revolves around the use of a clinical bedside assessment of swallowing to establish the presence of dysphagia and aspiration risk. Multiple studies have documented the lack of sensitivity to both dysphagia and aspiration that is present in a clinical bedside evaluation.^4,22 These studies were crucial in the selection of the MASA as the clinical bedside measure for the current study. The MASA was developed and validated with the modified barium swallow, correctly predicting aspiration in 26 of 28 aspirators, resulting in a sensitivity of 93%.²¹ Therefore, we regard the MASA as the reference standard for clinical bedside assessment.

Finally, the MASA was less sensitive for identifying dysphagia than for detecting aspiration risk, most likely because of a poor operational definition of dysphagia on the modified barium swallow when the MASA was developed. Future research should validate the ASDS with the modified barium swallow. It is essential that a well-defined operational definition of dysphagia be established for the modified barium swallow before the next study is started. The ASDS is currently the best tool available to screen patients for dysphagia and aspiration risk.

Table 2.

Frequency of failed and passed Acute Stroke Dysphagia Screening by cutoff scores for aspiration on the Mann Assessment of Swallowing Ability

Result on Acute Stroke Dysphagia Screening	Score on Mann Assessment of Swallowing Ability
	≥170	<170
Failed	74	60
Passed	163	3