Translating Evidence-Based Falls Prevention into Clinical Practice in Nursing Facilities: Results and Lessons from a Quality Improvement Collaborative

Abstract

OBJECTIVES

To describe the changes in process of care before and after an evidence-based fall reduction quality improvement collaborative in nursing facilities.

DESIGN

Natural experiment with nonparticipating facilities serving as controls.

SETTING

Community nursing homes.

PARTICIPANTS

Thirty-six participating and 353 non-participating nursing facilities in North Carolina.

INTERVENTION

Two in-person learning sessions, monthly teleconferences, and an e-mail discussion list over 9 months. The change package emphasized screening, labeling, and risk-factor reduction.

MEASUREMENTS

Compliance was measured using facility self-report and chart abstraction (n = 832) before and after the intervention. Fall rates as measured using the Minimum Data Set (MDS) were compared with those of nonparticipating facilities as an exploratory outcome.

RESULTS

Self-reported compliance with screening, labeling, and risk-factor reduction approached 100%. Chart abstraction revealed only modest improvements in screening (51% to 68%, P<.05), risk-factor reduction (4% to 7%, P = .30), and medication assessment (2% to 6%, P = .34). There was a significant increase in vitamin D prescriptions (40% to 48%, P = .03) and decrease in sedative-hypnotics (19% to 12%, P = .04) but no change in benzodiazepine, neuroleptic, or calcium use. No significant changes in proportions of fallers or fall rates were observed according to chart abstraction (28.6% to 37.5%, P = .17), MDS (18.2% to 15.4%, P = .56), or self-report (6.1–5.6 falls/1,000 bed days, P = .31).

CONCLUSON

Multiple-risk-factor reduction tasks are infrequently implemented, whereas screening tasks appear more easily modifiable in a real-world setting. Substantial differences between self-reported practice and medical record documentation require that additional data sources be used to assess the change-in-care processes resulting from quality improvement programs. Interventions to improve interdisciplinary collaboration need to be developed.

Falls remain a major source of morbidity for nursing facility residents.^1,2 Therefore, a substantial body of research has focused on fall reduction in this setting.^3–10 Several randomized, controlled trials have demonstrated that multiple-risk-factor reduction can substantially reduce fall rates and prevent fractures. Risk factors addressed in these trials included environmental hazards, adaptive equipment maintenance, psychotropic drug use, physical therapy or exercise, staff education, postfall problem solving, and hip protectors. To implement the interventions, studies have provided on-site personnel and interdisciplinary consult teams.^4,6,8–10

Translating these studies into effective clinical practice using existing nursing facility staff remains a challenge. The Resident Assessment Protocol for falls of the nursing home Resident Assessment Instrument was an early attempt to operationalize multiple-risk-factor reduction on a national scale, but an evaluation of falls rates before and after the introduction of the Resident Assessment Protocol in 1991 showed no change.¹¹ Other attempts to promote multiple-risk-factor reduction in nursing facilities using pharmacy consultants and quality assurance training have had mixed results.^12–14 In general, low levels of facility participation and nonsignificant changes in provider behavior have been reported.

A widely used method for translating evidence from trials into clinical practice is the Quality Improvement Collaborative (QIC). Quality improvement teams from each facility participate in all-day learning sessions and teleconferences during which they meet with clinical experts, exchange ideas, and generate ‘‘enthusiasm and a shared sense of commitment.’’^15–17 Between sessions, teams apply what they have learned by systematically testing small changes over time using ‘‘Plan-Do-Study-Act’’ cycles. The Institute for Healthcare Improvement has popularized the QIC method, and the U.S. Health Resources and Services Administration, the Veterans Health Administration, and the United Kingdom’s National Health Service have used it. However, recent studies have questioned the effectiveness of the QIC method and called for an improved evidence base for quality improvement initiatives.^15,18

The current study reports the results of a statewide QIC focused on fall prevention in nursing facilities. The primary objective was to describe the changes in clinical practice resulting from QIC participation. As an exploratory analysis, it was desired to test whether facility fall rates were different in participating facilities than in nonparticipating facilities.

METHODS

The Carolinas Center for Medical Excellence, a federally designated Quality Improvement Organization (QIO), initiated the QIC on behalf of the North Carolina Division of Facility Services, who funded the program using civil monetary penalties collected from nursing facilities. Clinical experts from Duke University Medical Center served as faculty in the QIO. The Duke University Medical Center institutional review board reviewed and approved the study.

Study Facilities

Forty-two nursing facilities from across the state, representing 10% of facilities in North Carolina, agreed to participate in the collaborative. Six facilities dropped out of the project immediately after the first learning session, leaving 36 participating facilities. Analyses of the effectiveness of the collaborative were limited to those 36 participating facilities. Characteristics of the study facilities are compared with those of nonparticipating facilities in Table 1. Each facility sent a team of two to three staff members to participate in the QIC. Disciplines represented by the team members included nursing, administration, physical therapy, and pharmacy.

Table 1.

Characteristics of Participating and Nonparticipating Facilities

Characteristic	Participating (n = 36)	Nonparticipating (n = 353)	P-value
Bed size, mean ± SD	100.9 ± 43.5	104.5 ± 48.0	<.001
For profit, %	58.3	77.9	<.01
Percentage of residents with falls in previous 30 days, mean ± SD	14.1 ± 0.06	10.1 ± 0.06	<.001

SD = standard deviation.

Intervention

A ‘‘change package’’ was developed based upon the interventions reported in clinical trials. The change package included:

• – monthly environmental assessments, including adaptive equipment repair

• – screening residents for level of fall risk using the Morse Risk Assessment Scale or another validated assessment

• – ‘‘labeling’’ high-risk residents with chart, room, and armband stickers to increase staff awareness during day-to-day care

• – multiple-risk-factor reduction and injury prevention in high-risk residents, including correction of orthostatic hypotension; psychoactive medication reduction; physical therapy referral; and use of calcium, vitamin D, and hip protectors.

• – postfall assessment with repeat multiple-risk-factor reduction assessment to ameliorate new risk factors.

The project was conducted using a modified QIC format.^16,17 Two in-person learning sessions, five educational teleconferences, and an e-mail discussion group facilitated this process. A toolkit of customizable forms, stickers, posters, and educational material was also provided to support each of the five changes (available from authors upon request).

Measures

Three sources of data were used: facility-reported data submitted as part of the project (available only for participating facilities), data obtained through an on-site record abstraction before and after the QIC (available only for participating facilities), and data on falls from the Minimum Data Set (MDS) before and during the QIC (available for all facilities in the state).

Participating facilities were asked to submit monthly data on four process indicators reflecting the change concepts: proportion of new admissions and re-admissions screened for level of fall risk, proportion of high-risk residents ‘‘labeled’’ with armbands and stickers, proportion of high-risk residents with multiple-risk-factor reduction attempted, and proportion of postfall risk factor assessments completed. In addition, facilities were asked to submit the number of falls occurring each month and their resident-days for the month. Participating facilities used a variety of definitions of fall; they were instructed to use their own definition of a fall and keep it consistent throughout the study, although the Centers for Medicare and Medicaid Services definition of a fall became more stringent during the study period, requiring facilities to record ‘‘near falls’’ on the MDS. Most facilities did not include ‘‘near falls’’ in their self-reported data.

To supplement the facility-submitted data, trained, registered nurse–level employees of the QIO completed on-site medical record abstractions. Charts were abstracted for two time periods: May to August 2002 (the preintervention period) and November 2002 to February 2003 (the during-intervention period). Eligible records were identified by the presence of an admission, re-admission, or significant change MDS assessment since the change package encouraged facilities to evaluate residents during these events (n= 832). Chart abstraction focused on documentation of the process measures described above, with the exception of ‘‘labeling’’ high-risk residents and monthly environmental assessments. In addition, abstractors recorded the use of hip protectors, the use of medication assessment forms, psychoactive medication prescriptions (including benzodiazepines, neuroleptics, anticholinergics, and sedative hypnotics), and falls.

Finally, data from quarterly MDS assessments archived in the state MDS repository were used to measure the proportion of residents with falls in the facility (item J4a, fall in past 30 days). The same data collection time periods as described for the chart abstraction measure were used.

Analysis

Facility self-reported data were graphed to assess change in the aggregate use of the change concepts and fall rates over time. Comparisons between chart-documented compliance with the change concepts before and during the intervention were made using chi-square tests. As an exploratory analysis, self-reported, chart-documented, and MDS fall rates were compared before and after the intervention using t tests. The proportion of facilities with improvement was compared using chi-square tests. When possible, comparisons were made with nonparticipating facilities. Additional analyses stratified by level of participation were conducted to examine the effect of level of participation on falls rates (test for trend). The analyses of MDS fall rates and self-reported process measures were performed using the facility as the unit of analysis. Analyses of the change in chart abstraction process variables in the two time periods were performed using the GLIMMIX procedure (SAS Institute, Inc., Cary, NC) to account for potential of clustering by facility. As a sensitivity analysis, MDS analyses were repeated at the aggregated and facility levels and using only admission, readmission, and change of status MDS (more closely matching the intervention emphasis but potentially including falls occurring before admission). The results did not change substantively and are not reported here.

RESULTS

Participation

Facility participation varied by type of collaborative activity and declined over time. Participation was greatest for the first learning session, followed by teleconferences. Facilities varied in how consistently they submitted their self-reported adherence to the change concepts. When analyses were restricted to facilities that consistently submitted the self-reported data, the overall pattern was unchanged. Thus, all submitted data are reported below.

Facilities were classified into three groups based on their level of participation across all activities: high participants were facilities who participated in 66% or more of the activities (n = 5), medium participants were those who participated in 33% to 66% of activities (n = 16), and low participants were facilities who participated in less than 33% of activities (n = 15).

Self-Reported Compliance

The self-reported rate for assessment of level of fall risk increased from 86% to 100% over the course of the collaborative. Labeling high-risk residents increased from 75% to 99%. The proportion of high-risk residents reported to have multiple-risk-factor reduction completed increased from 62% to 99%. Postfall assessments improved from 81% to 98% (Figure 1).

Figure 1.

Self-reported facility compliance (n = 36) with the falls-prevention change concepts (assessment of fall risk on admission, labeling high-risk residents, multiple-risk-factor reduction in high-risk residents, postfall risk factor assessment) over the course of the quality improvement collaborative.

Compliance Measured According to Chart Abstraction

A total of 1,398 records were available from the facilities. Of those, 832 contained an admission, readmission, or change of status MDS assessment during study period and were eligible for a complete abstraction. There were no significant differences in the demographic characteristics (age, sex, race, prior falls) of the residents whose charts were abstracted in the preintervention and during-intervention periods. Table 2 presents the adherence for four of the targeted change concepts during the pre- and during-intervention time periods as measured using chart abstraction. Screening for falls increased from 51% to 68% (P<.01), but there were no significant changes in multiple-risk-factor reduction (defined as completion of any of the recommended interventions) or postfall assessments. Although prescription of most of the targeted medications was unchanged, a significant reduction in use of sedative-hypnotics (19% to 12%, P<.01) and a significant increase in vitamin D prescriptions (40% to 48%, P<.05) were observed.

Table 2.

Use of Quality Improvement Collaborative Change Concepts Measured by Chart Abstraction (N = 36)

	Preintervention Period	Intervention Period
Change Concept	Mean
Screening for level of fall risk	51.2	67.7†
Multiple risk factor reduction (any)	3.8	6.8
Postfall assessment*	7.2	8.3
Hip protectors ordered	1.5	3.5
Calcium prescription	36.9	39.7
Vitamin D prescription	40.3	47.9†
Benzodiazepine prescription	33.3	30.5
Benzodiazepine reduction	23.4	24.0
Neuroleptic prescription	27.9	29.5
Neuroleptic dose reduction	29.2	21.9
Sedative-hypnotic prescription	19.0	12.4†
Sedative-hypnotic dose reduction	28.6	24.0
Falls recorded	28.6	37.5

^*Abstracted only for cases in which there was a fall.

^†Statistically significant difference between two time periods, P<.05.

Falls

There were no significant changes in fall rates according to any measure. The self-reported fall rate declined modestly from 6.1 falls per 1,000 resident-days to 5.6 falls per 1,000 resident-days, whereas falls measured according to chart abstraction increased slightly (Table 2). Using MDS quarterly assessments, there was no significant association between level of facility participation and the proportion of residents with falls (Table 3).

Table 3.

Proportion of Residents with at Least One Fall Before and During the Quality Improvement Collaborative and Proportion with Reduction in Falls Rates According to Participation Level Based on Minimum Data Set (MDS) Quarterly Assessments

	Facility Participation Level
	None						Medium
	(n = 353)		Any (n = 36)		Low (n = 15)		(n = 16)		High (n = 5)
Falls Measure	Pre	During	Pre	During	Pre	During	Pre	During	Pre	During
Residents with fall on MDS (%)	12.3	11.0	18.2	15.4	13.9	13.1	24.2	18.9	12.0	11.3
Facilities with reduction in falls rates (%)	49.9		55.6		53.3		56.3		60.0*

^*Test for trend nonsignificant, P>.05.

DISCUSSION

This study sheds some light on the process of adopting evidence-based practices into routine clinical use in nursing facilities. Insights about effectiveness, adoption of change, and data collection may be helpful for others engaged in this process.

The goal of facilities participating in the QIC was to reduce their residents’ fall rates. Despite evidence that facilities adopted some of the care-process changes, especially screening, no substantial improvements in fall rates were observed. At least two factors related the study’s design may partly explain the failure to observe significant changes. The small number of facilities participating in the collaborative limited ability to detect differences in falls rates. Moreover, the Centers for Medicare and Medicaid Services change to the more-stringent fall definition at the beginning of the intervention period also had the effect of making it more difficult to show a decrease in falls after the intervention. An alternative explanation is that the low uptake of several key care process changes, especially multifactorial risk-factor reduction, reduced the effectiveness of the intervention.

The nonrandomized study design limits the ability to attribute any process improvements to the QIC; changes could represent ‘‘regression to the mean’’ or unmeasured confounding. Nevertheless, randomized trials of quality improvement initiatives present substantial challenges,^15,19 and important lessons can be learned from this study.

Insights into the process of translation from study to practice are evident from examining which change concepts the facilities were able to adopt and which were largely neglected. According to self-report and chart abstraction, facilities were able to implement routine screening for fall risk without difficulty. In contrast, implementation of multiple-risk-factor reduction processes was disappointingly infrequent before and after the QIC. Although poor documentation may explain this in part, the Diffusion of Innovation model, summarized below, presents a useful means of interpreting this finding. First, the extent to which opinion leaders have been persuaded that a new practice will be effective influences the decision to adopt new practices.^20,21 Evidence supporting the effectiveness of falls risk screening has been in the literature for 10 years longer than the literature on multiple-risk-factor reduction. More facilities may have made the decision to adopt risk screening before the initiation of the QIC; the effect of the interventions would therefore have been to lend greater support to a practice that had already been introduced into the facility culture. A second factor concerns the complexity of the intervention. Although great care was taken in developing the change package to make the risk reduction practical and easy to apply in the nursing facility setting, risk-reduction protocols are inherently complex, requiring changes in interaction patterns between charge nurses and physicians and between charge nurses and direct care workers and additional time to engage in clinical reasoning about costs and benefits. Systems interventions that promote the adoption of complex practices need to be developed and studied.²³

A further lesson in the challenges of altering prescribing habits in nursing facilities was learned from this study. The significant reduction in sedative-hypnotic prescriptions, although encouraging, is unlikely by itself to result in substantial improvements in fall rates. Reductions in the use of neuroleptics and benzodiazepines were not observed, perhaps reflecting a perceived lack of effective alternatives to managing behavioral disturbance in this population or a ‘‘floor effect’’ due to high regulatory oversight of these agents. The QIC teams were primarily composed of nursing and rehabilitation staff, with few pharmacy and no medical team representatives. Although teleconferences and written materials targeted specifically to prescribers were delivered, a more-intensive intervention may be required to change prescribing patterns. Further QICs should identify ways to better engage a wider range of nursing facility staff in the process of multiple-risk-factor reduction, because multiple levels of personnel, not just the senior leadership, will ultimately affect the decision to adopt new care practices.

Measurement issues raised by this study are also important. Falls were measured using chart abstraction, facility self-report, and MDS, and differences were found in the magnitude and direction of change. Although previous studies have documented more underreporting of falls in the MDS than in the medical record,²² MDS represents the only source of data available for researchers wishing to compare nonparticipating facilities. There were also large differences between compliance with change package elements measured using self-report and chart abstraction. One explanation is that, despite providing customizable documentation sheets, not all fall-prevention activities were documented in the chart. Although the realities of nursing home staffing make chart underreporting likely, it is likely that the near-perfect self-reported information is also biased. It is likely that these measures represent different but related components of the quality of care. Alternate modes of quality assessment, such as direct observation or resident interview, may be necessary to more fully capture actual care delivery in real-world settings.

CONCLUSION

This study illustrates the many challenges encountered in design, implementation, and measurement of quality improvement initiatives. Interventions to improve interdisciplinary collaboration need to be developed. Explicit use of models such as Rogers Diffusion of Innovation²⁰ to identify intervention components that may require additional time or facility support may facilitate improved adoption of more-complex intervention packages. It is difficult to implement advances made in clinical trials into clinical practice.