The Stroke Assessment of Fall Risk (SAFR): predictive validity in inpatient stroke rehabilitation

Abstract

Objective

To evaluate relative accuracy of a newly developed Stroke Assessment of Fall Risk (SAFR) for classifying fallers and non-fallers, compared with a health system fall risk screening tool, the Fall Harm Risk Screen.

Design and setting

Prospective quality improvement study conducted at an inpatient stroke rehabilitation unit at a large urban university hospital.

Participants

Patients admitted for inpatient stroke rehabilitation (N = 419) with imaging or clinical evidence of ischemic or hemorrhagic stroke, between 1 August 2009 and 31 July 2010.

Interventions

Not applicable.

Main outcome measure(s)

Sensitivity, specificity, and area under the curve for Receiver Operating Characteristic Curves of both scales’ classifications, based on fall risk score completed upon admission to inpatient stroke rehabilitation.

Results

A total of 68 (16%) participants fell at least once. The SAFR was significantly more accurate than the Fall Harm Risk Screen (p < 0.001), with area under the curve of 0.73, positive predictive value of 0.29, and negative predictive value of 0.94. For the Fall Harm Risk Screen, area under the curve was 0.56, positive predictive value was 0.19, and negative predictive value was 0.86. Sensitivity and specificity of the SAFR (0.78 and 0.63, respectively) was higher than the Fall Harm Risk Screen (0.57 and 0.48, respectively).

Conclusions

An evidence-derived, population-specific fall risk assessment may more accurately predict fallers than a general fall risk screen for stroke rehabilitation patients. While the SAFR improves upon the accuracy of a general assessment tool, additional refinement may be warranted.

Introduction

A disproportionate number of stroke patients, as many as 48%,¹ fall during inpatient rehabilitation; of those falls, nearly one-third lead to potentially serious injuries.² Hospital-related falls are associated with long lengths of stay and poor outcomes,³ as well as reduced physical activity owing to fear of additional falls¹ and diminished dignity.⁴ To prevent these negative outcomes, preventive strategies are needed for patients at high risk for falls.

To most effectively target preventive strategies, it is necessary to reliably identify patients at greatest risk of falls. Hospitals seek to identify high-risk patients using either published fall risk assessments developed for general hospital populations, or internally developed tools that have not been adequately validated in all the populations for which they are used. These measures, including the recently published and not well tested PREDICT-FIRST, often rely primarily upon demographic risk factors (e.g. age; gender) or general clinical characteristics (e.g. use of antihypertensive, antianxiety, or antidepressant medications; urinary incontinence; history of previous falls) to indicate fall risk. However, these characteristics may be less relevant to fall risk after stroke than are stroke-specific disabilities and impairments. As a result, existing fall risk tools may lack sensitivity and specificity for likely fallers in stroke rehabilitation.⁵ We developed and piloted a stroke-specific tool, the Stroke Assessment of Fall Risk (SAFR), as a quality improvement project,⁶ to improve fall prediction during the inpatient stroke rehabilitation stay. This study evaluated the accuracy of the SAFR in classifying fallers and non-fallers compared with an unpublished fall risk screening tool developed by our health system, the Fall Harm Risk Screen.

Methods

Participants were all patients admitted consecutively to two inpatient stroke rehabilitation units in the same academic health center between 1 August 2009 and 31 July 2010 who had imaging and/or clinical evidence of acute stroke (ischemic or hemorrhagic). Those with comorbid traumatic brain injury or degenerative neurological disorders (e.g. Parkinson’s disease and multiple sclerosis) were excluded.

This prospective quality improvement study was approved by our academic health center’s institutional review board. We retrospectively collected demographic (gender, age, race/ethnicity, and hospital unit) and clinical data (stroke etiology, hemisphere, location, and fall prevention interventions utilized during the stay) from each participant’s medical record. We also recorded hospital Fall Harm Risk Screen admission scores, and completed the SAFR based on the rehabilitation admission interdisciplinary clinical evaluation. Finally, we prospectively tracked and recorded fall occurrence during the rehabilitation stay.

The Fall Harm Risk Screen is a facility-developed, three-item scale that assesses three levels of fall risk (low, medium, and high) based on patient functional ability, history of falls, and the nurse’s clinical judgement of fall risk. Nursing staff completes the scale at rehabilitation admission, and periodically throughout the stay. The Fall Harm Risk Scale is used for all patients throughout the health system, regardless of diagnosis (the Fall Harm Risk Scale is provided in Appendix A, available online).

The SAFR is scored using clinical documentation from the first 72 hours of the inpatient rehabilitation admission. It assesses seven stroke-specific risk factors identified from the published literature^1–3,7 and clinical audits. These comprise four impairments (impulsivity, hemi-neglect, static, and dynamic sitting balance) and three functional limitations (lowest score on three Functional Independence Measure items: transfers, problem solving, and memory). Impulsivity and hemi-neglect are scored dichotomously (0, absent; 7, present). The remaining items are scored using a 7-point scale similar to the Functional Independence Measure, but with zero indicating no impairment or deficit, and seven indicating the most severe impairment or deficit. The total score is a sum of item scores (0, low risk of falls; 49, highest risk of falls) (the SAFR is provided in Appendix B, available online.)

A fall was defined as unplanned contact with the floor. Fall occurrence was gleaned from hospital incident reports, and participants were coded accordingly (fall/no fall).

Analyses were performed using SAS Version 9.2 (SAS Institute, Cary, North Carolina) with a two-tailed significance level of α = 0.05 for all tests. We characterized the sample using descriptive statistics; we then compared fallers with non-fallers on key demographic and clinical attributes using chi-square tests and Mann–Whitney U-test or independent sample student’s t-tests, as appropriate. We constructed Receiver Operating Characteristic curves using logistic regression to determine the accuracy of fall identification of the SAFR and the Fall Harm Risk Screen. We then chose a clinically meaningful cut point for “at risk to fall” and calculated positive predictive value for each tool. Additionally, we conducted posthoc analyses of the SAFR items using Receiver Operating Characteristic curves to assess individual item performance.

Results

Of the 446 patients admitted for stroke rehabilitation during the study period, 27 (6%) were excluded owing to comorbid neurological conditions, for a final sample of 419 participants. A description of the sample is provided in Table 1. A total of 68 participants (16%) experienced at least one fall during their inpatient rehabilitation stay; of those participants who fell, 10 people (2% of the sample, 15% of the fallers) fell more than once. Fallers and non-fallers differed with respect to age, use of chair alarms, and use of restraints during inpatient rehabilitation. The median age of fallers was significantly younger than that of non-fallers (fallers M = 63.7 ± 13.5 years, non-fallers M = 68.2 ± 15.7 years, p = 0.026). Fallers were also significantly more likely to have a chair alarm (χ²₁ = 21.23, p < 0.001, odds ratio (OR) = 4.3, 95% confidence interval (CI) (2.2, 8.3)) or a restraint (χ²₁ = 23.98, p < 0.001, OR = 3.7, 95% CI (2.1, 8.3)) during their inpatient rehabilitation stay. The area under the curve was 0.56 (95% CI (0.50, 0.62)) for Fall Harm Risk Screen, and 0.73 (95% CI (0.67, 0.79)) for SAFR (Figure 1); it was significantly more accurate than the Fall Harm Risk Screen (χ²₁ = 17.28_, p < 0.001). At a clinically meaningful cut point of 27, the positive predictive value for the SAFR was 0.29 and the negative predictive value was 0.94, yielding sensitivity and specificity of 0.78 and 0.63, respectively (Table 2). A Fall Harm Risk Screen score of two produced a positive predictive value of 0.19 and a negative predictive value of 0.86, yielding sensitivity and specificity of 0.57 and 0.48, respectively (Table 2). Posthoc analyses of the seven SAFR items revealed that the two dichotomous-scored items (impulsivity and hemi-neglect) were less predictive than the five ordinal-scored items (Table 3). Area under the curve values ranged from 0.55–0.69 for individual items, indicating the overall score (area under the curve = 0.73) provided a more accurate classification of fall risk than any one risk factor (Table 3).

Table 1.

Demographic and clinical characteristics of the sample.

Variable	Entire sample (N = 419)	Falls occurrence
		Non-fallers (n = 351)	Fallers (n = 68)	Test statistic (p value)
Age in years, mean ±SD	67.5 ± 15.5	M= 68.2 ± 15.7	63.7 ± 13.5	t =2.23 p= 0.026
Gender, n (%) female	202 (48.2)	176 (50.14)	26 (38.24)	χ2(1) = 3.23 p= 0.07
Race, n (%) white	310 (73.99)	260 (74.07)	50 (73.53)	χ2(3) = 4.11 p= 0.25
Stroke hemisphere, n (%) left	188 (45.30)	158 (45.53)	30 (44.12)	χ2(3) = 0.47 p= 0.93
Stroke etiology, n (%) ischemic	333 (79.86)	281 (80.52)	52 (76.47)	χ2(1) = 0.58 p= 0.45
Stroke type, n (%)
Cortical	73 (17.59)	60 (17.29)	13 (19.12)	χ2(3) = 0.61 p= 0.89
Subcortical	80 (19.28)	67 (19.31)	13 (19.12)
Cortical/subcortical	188 (45.30)	156 (44.96)	32 (47.06)
Brainstem/cerebellar	74 (17.83)	64 (18.44)	10 (14.71)
Restraint use*, n (%) present	155 (36.99)	112 (31.91)	43 (63.24)	χ2(1) = 23.98 p < 0.001
Chair alarm use*, n (%) present	239 (57.04)	183 (52.14)	56 (82.35)	χ2(1) = 21.23 p <0.001

Figure 1.

Predictive ability of Stroke Assessment of Fall Risk (SAFR) and Fall Harm Risk Screen (FHRS).

FHRS: Fall Harm Risk Screen; ROC: Receiver Operating Characteristic Curve; SAFR: Stroke Assessment of Fall Risk.

Table 2.

Predictive ability of Stroke Assessment of Fall Risk at cut point score of 27 vs. Fall Harm Risk Screen at cut point score of two.

		Predicted outcomes
		Fall	No fall
Stroke Assessment of Fall Risk (cut point = 27)	Fall	TP, n = 52	FN, n = 15	Sensitivity = TP/(TP + FN) = 0.78
	No fall	FP, n = 128	TN, n = 222	Specificity = TN/(TN + FP) = 0.63
		PPV = TP/(TP + FP) = 0.29	NPV = TN/(TN + FN) = 0.94

		Predicted outcomes
		Fall	No fall
Fall Harm Risk Screen (cut point = 2)	Fall	TP, n = 39	FN, n = 29	Sensitivity = TP/(TP + FN) = 0.57
	No fall	FP, n = 170	TN, n = 181	Specificity = TN/(TN + FP) = 0.48
		PPV = TP/(TP + FP) = 0.19	NPV = TN/(TN + FN) = 0.86

FN: false negative; FP: false positive; NPV: negative predictive value; PPV: positive predictive value; TN: true negative; TP: true positive.

Table 3.

Stroke Assessment of Fall Risk item performance.

Item	AUC	Standard error	95% Wald CI
Impulsivity	0.60	0.0327	0.53–0.66
Hemi-neglect	0.55	0.0306	0.49–0.61
Static	0.69	0.0302	0.63–0.75
Dynamic sitting balance	0.69	0.0330	0.62–0.75
Transfer	0.69	0.0314	0.62–0.75
Problem-solving	0.67	0.0325	0.60–0.73
Memory	0.66	0.0333	0.59–0.72

AUC: area under the curve; CI: confidence interval.

Discussion

In our sample of 419 stroke patients, the Fall Harm Risk Screen identified inpatient post-stroke fallers no better than chance, while the SAFR accurately identified fallers nearly 75% of the time, representing a clinically important improvement in fall identification accuracy. Like many inpatient fall risk screens, such as the Morse scale,⁸ Hendrich II,⁹ and PREDICT_FIRST,¹⁰ the Fall Harm Risk Screen is based on general risk factors, such as medications, comorbidities, and gait disturbances, as well as on non-modifiable risk factors, such as age and gender. In stroke rehabilitation, every patient scores at high fall risk on these tools, yet not every patient will fall. Preventive strategies may be initiated for every patient, reducing the vigilance provided to those truly at risk. While the recently published PREDICT_FIRST’s predictive accuracy was similar to that of the SAFR (area under the curve = 0.73) in a sample of rehabilitation patients comprising a variety of diagnoses,¹⁰ it underestimated the rate of falls in a sample of stroke rehabilitation patients.⁵ In contrast, the SAFR was derived from stroke-specific indicators, and which may lead to more accurate prediction. Moreover, with its focus on modifiable risk factors, the SAFR may suggest patient-specific rehabilitative strategies to therapeutically modify each patient’s specific risk indicators, providing greater clinical value than that provided by a simple risk prediction tool.

The SAFR’s sensitivity (0.78) suggests that it will accurately identify 78% of fallers at the chosen cut point of 27. However, results also suggest that the SAFR will rate 37% of patients who do not fall as being “at risk” (based on calculating 1-specificity), resulting in application of unnecessary preventive interventions for some patients in stroke rehabilitation. The Fall Harm Risk Screen’s sensitivity will correctly identify 57% of fallers; yet, given its specificity, 48% of non-fallers will be incorrectly designated at risk to fall and will receive unnecessary fall prevention interventions. Implementing unneeded interventions wastes staff time (e.g. through increased surveillance of “at risk” patients) and uses costly resources (bed and chair alarms, enclosure beds). We believe that the SAFR, with its improved sensitivity and specificity, represents a clinically desirable improvement in fall risk designation over the currently used Fall Harm Risk Screen, although it may benefit from further development to improve precision.

There were unexpected differences in our sample between subjects who fell and those who did not fall. First, fallers were significantly younger than non-fallers. Traditionally, increasing age has been associated with greater fall occurrence among older adults,¹¹ although this relationship may not extend to the inpatient stroke rehabilitation population.⁷ Inpatient stroke rehabilitation falls are frequently associated with impulsive behavior, poor judgment, or calculated risk-taking by patients.¹² It is possible that the younger persons in our sample were more active, or were more prone to attempt ‘risky’ behaviors, such as standing or walking unassisted. This conclusion is supported by the fact that almost half (47.5%) of persons who fell scored as “impulsive” on the SAFR, while only 27.5% of persons who did not fall were scored as “impulsive”. Of note, persons scored as “impulsive” were slightly younger (M = 63.8) compared with persons scored as not impulsive (M = 69.1).

Second, subjects who fell were more likely to have chair alarms or restraints, which are intended to prevent falls. However, we do not know whether these strategies were initiated prior to the fall, or afterward, to prevent further falls. Future analyses should attempt to identify the temporal sequence of intervention initiation and fall occurrence.

Study limitations

A key limitation is that some SAFR items rely on narrative medical charting of clinician impression, instead of standardized assessment scales. Incorporating objective measures of impairments, such as impulsivity and neglect, into clinical care may improve overall scoring accuracy. Further, we used hospital incident reports to identify falls. Relying on incident reports may result in under-identification of falls.¹³ However, our health system actively encourages recording of all falls to identify trends and improve patient care processes, so the likelihood of under reporting is low. Further, we only evaluated the ability of the SAFR to predict the occurrence of any fall during rehabilitation, rather than examining its ability to identify “repeat fallers”. However, the proportion of repeat fallers was such a low percentage of the sample (2%) that meaningful analysis of repeat falls was precluded. Although we provided some preliminary evidence of item performance, further analyses should be performed to assess item-level characteristics. Future work using psychometric testing (i.e. Rasch analysis) could determine whether some items should be weighted based on importance, to increase their contribution to the SAFR score.

Finally, we did not exclude unanticipated physiologic falls from our study. Unanticipated physiologic falls are falls resulting from a medical event (e.g. syncopal episode, seizure), which are not directly related to post-stroke impairments or activity limitations. In her classic text, Morse¹⁴ notes that unanticipated physiologic falls occur unpredictably among all hospitalized patients, have little relationship to the factors that place patients at increased fall risk, and should not be a focus of fall risk assessment initiatives. The predictive accuracy of both the SAFR and the Fall Harm Risk Screen may have been slightly decreased because we did not exclude unanticipated physiologic falls.¹⁴ Current hospital policy requires reporting of all falls, including unanticipated physiologic falls, so some of these may have been included in our data. However, the rate of unanticipated physiologic falls on our unit is very low; thus, the likelihood that our results were significantly affected by unanticipated physiologic falls is minimal. A strength of this study is that we relied upon routinely collected clinical data to complete the SAFR, eliminating burdensome “double documentation” for nurses and therapists.

Preliminary results suggest that in stroke inpatient rehabilitation, an evidence-derived, population-specific fall risk assessment may more accurately predict fallers than a general fall risk screen. The SAFR shows promise as such an assessment. Increasing fall prediction accuracy may help to decrease incident fall rates in inpatient stroke rehabilitation, thereby minimizing the harmful consequences of falls.

Clinical messages.

• A stroke-specific fall risk assessment, based on common post-stroke impairments and activity limitations, can improve fall prediction among inpatient stroke rehabilitation patients.

• Persons of advanced age exhibiting impulsivity, neglect, impaired balance, memory impairment, impaired problem solving skills, and deficits in transfers, may be at particularly increased risk of falls.