The impact of an electronic health record on nurse sensitive patient outcomes: an interrupted time series analysis

Abstract

Objectives

To evaluate the impact of electronic health record (EHR) implementation on nursing care processes and outcomes.

Design

Interrupted time series analysis, 2003–2009.

Setting

A large US not-for-profit integrated health care organization.

Participants

29 hospitals in Northern and Southern California.

Intervention

An integrated EHR including computerized physician order entry, nursing documentation, risk assessment tools, and documentation tools.

Main outcome measures

Percentage of patients with completed risk assessments for hospital acquired pressure ulcers (HAPUs) and falls (process measures) and rates of HAPU and falls (outcome measures).

Results

EHR implementation was significantly associated with an increase in documentation rates for HAPU risk (coefficient 2.21, 95% CI 0.67 to 3.75); the increase for fall risk was not statistically significant (0.36; −3.58 to 4.30). EHR implementation was associated with a 13% decrease in HAPU rates (coefficient −0.76, 95% CI −1.37 to −0.16) but no decrease in fall rates (−0.091; −0.29 to 0.11). Irrespective of EHR implementation, HAPU rates decreased significantly over time (−0.16; −0.20 to −0.13), while fall rates did not (0.0052; −0.01 to 0.02). Hospital region was a significant predictor of variation for both HAPU (0.72; 0.30 to 1.14) and fall rates (0.57; 0.41 to 0.72).

Conclusions

The introduction of an integrated EHR was associated with a reduction in the number of HAPUs but not in patient fall rates. Other factors, such as changes over time and hospital region, were also associated with variation in outcomes. The findings suggest that EHR impact on nursing care processes and outcomes is dependent on a number of factors that should be further explored.

Introduction

Kaiser Permanente is the largest not-for-profit integrated healthcare delivery organization in the USA, with 8.7 million members across eight regions and the District of Columbia. In March 2010, it completed the implementation of an organization-wide electronic health record (EHR), KP HealthConnect. The implementation of KP HealthConnect is the largest private deployment of an EHR system to date.¹ Internationally, a number of governments, including those of the UK and the USA, are investing in the implementation of EHRs under the widely held belief that it will improve efficiency and enable improvements in healthcare quality and patient outcomes.^2–4 However, empirical evidence of the benefit of health information technologies is mixed.^5–8

The system-wide implementation of an integrated EHR has potential benefits, such as increased organizational efficiency, improved communication, improved practitioner performance (eg, improved prescribing), better quality of care and improved outcomes.⁵ Evaluations of health information technologies have largely focused on physician behavior and patient outcomes associated with medical management.⁵ Nurses comprise the largest professional group in the healthcare workforce, yet the impact of health information technology on the quality of nursing care is largely unexplored.⁹

Adverse events such as hospital acquired pressure ulcers (HAPUs) and falls are directly related to the quality of nursing care,¹⁰ which could be enhanced through the use of health information technologies.⁸ For example, risk assessment tools in KP HealthConnect identify patients at risk of developing a pressure ulcer or falling; nurses are alerted and prompted to implement preventive measures (figure 1).

Figure 1.

(A) Example of alert for pressure ulcers. (B) Flow sheet documenting preventive actions for pressure ulcers.

We report here on a study examining the impact of KP HealthConnect implementation on nursing care process and outcome measures across Kaiser Permanente hospitals in Northern and Southern California. We hypothesized that KP HealthConnect implementation would be associated with improved documentation of HAPU and fall risk, and lower HAPU and fall rates.

Methods

Setting and study design

Kaiser Permanente is an integrated healthcare delivery system comprising Kaiser Foundation Health Plan, Inc. (KFHP), Kaiser Foundation Hospitals (KFH), and separate Permanente Medical Groups in each geographic region. KFHP contracts with KFH and the Permanente Medical Groups to provide hospital and medical services to KFHP members and the community. Each region in Kaiser Permanente has their own organizational structure for KFH and the Permanente Medical Groups. In the Northern and Southern California regions, KFH currently operates 34 hospitals; we examined 29 hospitals that were continuously operating during the period we studied. We used an interrupted time series design, collecting monthly or quarterly data on HAPU and fall rates and risk from the first quarter of 2003 through the fourth quarter of 2009. Data were available for all hospitals included in the analysis for the entire period of the study.

KP HealthConnect

The intervention of interest was KP HealthConnect implementation. KP HealthConnect includes computerized physician order entry, computerized decision support, as well as communication and documentation of all inpatient and outpatient laboratory, pharmacy, and clinical care activities. A secure patient portal, kp.org, allows members to view portions of their record and to securely email providers, order prescription refills, and arrange appointments. KP HealthConnect is vendor supplied, but content supporting clinical functions is developed by teams of Kaiser Permanente clinicians and information specialists. The inpatient nursing component is consistent across all hospitals and was developed consensually by clinical nurses and information specialists.

KP HealthConnect was implemented in the first hospital in California in 2006, followed by a rolling implementation program across other sites; the last hospital went live in March 2010. We considered the intervention as occurring in individual hospitals in the month the system went live.

Outcome measures and data sources

Process measures included the proportion of patients with a completed fall risk assessment and the proportion with a HAPU risk assessment completed within 24 h of admission. Outcome measures included fall rates, defined as the number of falls per 1000 patient days, where falls were defined as an unplanned descent to the floor, and HAPU rates, defined as the percentage of all patients on the day of data collection with stage I, II, III, IV, or unstageable ulcers.

Data on process and outcome measures were obtained from the Collaborative Alliance for Nursing Outcomes.¹¹ Since 1996, the Collaborative Alliance for Nursing Outcomes has been collecting unit level outcomes data on nurse staffing, care process, and nursing sensitive outcome measures for California hospitals, including all Northern and Southern California Kaiser Permanente hospitals. Data on process and outcome measures were collected consistently across the period we studied. Data on fall rates and documentation of fall risk were obtained monthly; HAPU data were collected quarterly. All data were collected at the unit level (medical/surgical/ICU unit) for each hospital.

Statistical analysis

Data were synchronized by using the implementation month at each hospital as time zero; they were then aggregated across facilities. Multivariate regression analysis was used to examine the effect of time, Kaiser region (Northern or Southern California), and KP HealthConnect implementation on outcomes. Although staffing levels have been identified as a predictor of quality of nursing care,¹² we did not include staffing as an independent variable in the analysis. In 2004, the state of California instituted minimum nurse staffing requirements in acute care hospitals;¹² as a result, registered nurse staffing varied little across sites during the period we studied. SAS V.9.1.3 was used to analyze data.

The Kaiser Permanente Northern California Institutional Review Board considered the study exempt from IRB approval.

Results

Nursing care process measures

Over the time period of the study, mean rates for risk assessment documentation for pressure ulcers and falls increased. The introduction of KP HealthConnect was associated with increased risk assessment documentation for pressure ulcers (coefficient 2.21, 95% CI 0.67 to 3.75) but not for falls (0.36; −3.58 to 4.30) (figure 2). Documentation of both pressure ulcer risk (0.25; 0.16 to 0.34) and fall risk (0.53; 0.32 to 0.74) increased over time. Risk of pressure ulcers was more likely to be documented in hospitals in Northern California (1.27; 0.23 to 2.31) and risk of falling was more likely to be documented in hospitals in Southern California (−2.68; −5.19 to −0.17). The multivariate model for pressure ulcer risk documentation explained 17% of the variance; the model for fall risk documentation explained only 7% of the variance.

Figure 2.

Impact of an electronic health record system on documentation of risk for falling or hospital acquired pressure ulcer (HAPU).

Nursing sensitive outcome measures

Between 2003 and 2009, mean HAPU rates decreased, while mean fall rates remained fairly stable. KP HealthConnect implementation was associated with a decrease in HAPU rates (coefficient −0.76, 95% CI −1.37 to −0.16) (figure 3) but not with changes in fall rates (−0.091; −0.29 to 0.11) (figure 4). HAPU rates also decreased significantly over time (−0.16; −0.20 to −0.13); time was not associated with changes in fall rates (0.0052; −0.01 to 0.02). Hospital region was a significant predictor of variation for both HAPU (0.72; 0.30 to 1.14) and fall rates (0.57; 0.41 to 0.72). The multivariate model for HAPU rates explained 28% of the variance in outcomes; however, the model for fall rates explained very little variance (5%). These results are presented in table 1.

Figure 3.

Impact of implementation of an electronic health record system on hospital acquired pressure ulcer (HAPU) rate.

Figure 4.

Impact of implementation of an electronic health record system on fall rate.

Table 1.

Regression results for process and outcome measures

Dependent variable	Independent variable	Coefficient (95% CI)	SE	t Statistic	p Value	Adj R2
Fall risk documentation	EHR implementation	0.36 (−3.58 to 4.30)	2.01	0.18	0.86	0.07
	Region	−2.68 (−5.19 to −0.17)	1.28	−2.1	0.04
	Time	0.53 (0.32 to 0.74)	0.11	4.98	<0.0001
HAPU risk documentation	EHR implementation	2.21 (0.67 to 3.75)	0.79	2.81	0.005	0.17
	Region	1.27 (0.23 to 2.31)	0.53	2.41	0.02
	Time	0.25 (0.16 to 0.34)	0.05	5.21	<0.0001
Fall rate	EHR implementation	−0.09 (−0.29 to 0.11)	0.1	−0.90	0.37	0.05
	Region	0.57 (0.41 to 0.72)	0.08	7.09	<0.001
	Time	0.01 (−0.01 to 0.02)	0.01	0.60	0.55
HAPU	EHR implementation	−0.76 (−1.37 to −0.16)	0.31	−2.46	0.01	0.28
	Region	0.72 (0.30 to 1.14)	0.21	3.37	<0.0001
	Time	−0.16 (−0.20 to −0.13)	0.02	−8.61	<0.0001

EHR, electronic health record; HAPU, hospital acquired pressure ulcer.

Discussion

The introduction of KP HealthConnect was associated with an increase in documentation of pressure ulcer risk assessments and a reduction in HAPU prevalence of 13% after controlling for time trends and organizational region. However, a similar result did not occur regarding the rate of fall risk documentation, and there were no apparent reductions in fall rate over the period we studied. There were changes in both process measures and outcome measures over time, independently of the introduction of the EHR. Documentation of risk assessment for both pressure ulcers and falls increased over the time period, and HAPU rates decreased significantly. In addition, there were differences in process and outcome measures between the two Kaiser regions (Northern and Southern California), indicating that organizational factors may also have an influence on outcomes. Although the introduction of the EHR was associated with a significant increase in risk assessment documentation for pressure ulcers (but not fall risk assessment) and a reduction in pressure ulcer rates, the amount of variance explained by the regression models was small, indicating that a number of factors together with the EHR have influenced the differences observed.

Previous studies have also found that using health information technology with higher levels of functionality (such as KP HealthConnect) is associated with variable improvements in outcomes.¹³ Organizational factors are important in determining the successful implementation and use of health information technology.¹⁴ Organizations characterized by collaboration, team work, and supportive leadership are more likely to report successful technology implementation and use.¹⁵ The ‘fit’ between the organization and the technology and including users in system development also contribute to successful technology use.^16–19 Similarly, a number of organizational features can lead to improved patient outcomes such as reduced HAPU rates.²⁰ These include a prioritization of safety within the organization, integrating safety practices and awareness into daily organizational functions, engagement of senior management with safety issues, allocation of organizational resources to promoting safety, and leadership.^20–22 In our study the region where hospitals were located (Northern or Southern California) was associated with differences in both process and outcome measures. There were organizational differences in this time period between the regions in terms of their emphasis on quality improvement programs for HAPU. As noted in table 1, regional differences in both HAPU and fall rates are very significant. In 2007, an initiative to reduce HAPUs was launched in all Kaiser Permanente Northern California hospitals (known as the HEROES program); it included incorporating Ascension Health's SKIN care bundle into the KP HealthConnect build and the education of frontline nursing staff.²³ The senior executive team in Northern California spearheaded the initiative and deployed resources to support it. Organization resources were not focused to the same degree on fall reduction.

As part of the HEROES initiative, the way in which risk assessment information for pressure ulcers was related to interventions to try and reduce pressure ulcers was also altered in the KP HealthConnect build. For HAPU and fall risk, nurses carry out individualized assessments for each patient. KP HealthConnect then calculates an overall risk score and alerts the nurse when a patient is at risk of falling or developing a pressure ulcer. For HAPUs, the system was adjusted to provide flowsheets prompting nurses to carry out appropriate actions related to the SKIN bundle (eg, turning the patient frequently); similar prompts for fall prevention were not built into the EHR application. Particular features (or ‘clinical content’) in the EHR also affect their use by individuals, impacting quality of care processes and outcomes.²⁴ Clinical decision support that has been positively linked to improvements in outcomes include automatically providing a recommendation for action to clinicians, as part of their workflow and at the time and location of decision making.²⁵ Using individualized risk assessments as the basis for tailored care planning to support nurse decision making is also associated with improved patient outcomes.²⁶ Although there is no direct evidence from our study to link these organizational factors and changes in KP HealthConnect functionality to differences in outcomes, what this does provide is a possible explanation for why we observed associations between Kaiser regions, time, and the implementation of KP HealthConnect on process measures and outcomes related to pressure ulcers.

There were less obvious relationships between the implementation of the EHR and improvements in risk assessment for falls, and fall rates. Fall risk documentation increased over the time period, but fall rates did not appear to vary. One possible reason for EHR implementation not impacting on fall rate documentation was the already high levels of documentation in place before implementation. However, this is the case for both pressure ulcer and risk documentation; indeed the level of pressure ulcer documentation was higher both before and after EHR implementation. It is therefore more likely that there was no additional benefit in terms of fall risk documentation with implementation of KP HealthConnect over and above the changes that were already happening over this time period. There were considerable external pressures on all healthcare organizations in the USA over the time period of the study to reduce adverse events such as pressure ulcers and falls,²⁷ which would suggest one would expect to see a reduction in the incidence of such outcomes over time, independently of the introduction of the EHR. This is the pattern we observed for pressure ulcers, but not for falls. There are potentially a number of problems with the accuracy of recording of fall rates,²⁸ which have been recognized by a number of organizations including Kaiser Permanente. The lack of variation in fall rate numbers observed in our study may therefore be due to a simultaneous decrease in fall rates with an improvement in documentation of falls (as better systems have been introduced).

Strengths and weaknesses of the study

Significant strengths of our study include the use of a consistent technology across multiple settings. A common limitation of other studies examining the effect of EHR implementation is the wide variation in technology capabilities across settings.¹³ In conjunction with cross-sectional data, which fail to take into account changes in outcomes over time, this variation makes it difficult to assess the effect of EHRs on quality. Our use of an interrupted time series design controlled for changes in outcomes over time unrelated to KP HealthConnect. There are also limitations to our study that deserve mention. We did not adjust for the mix of patients at each hospital; there may have been systematic differences affecting the outcomes of interest that are not accounted for in the analysis. Nor did we did adjust for variations in organizational characteristics; for instance, ‘early adopter’ hospitals that opted to implement KP HealthConnect first are more likely to have been enthusiastic about EHR use, which may have affected its impact. Due to staggered implementation, the number of observations available for analysis after implementation varied for individual hospitals. As a result, we may have not examined the full impact of the EHR on process and outcome measures. As already highlighted, the variance in process and outcome measures explained by our analysis was small. Although we found some association between the implementation of the EHR and process and outcome measures, the effect was relatively slight. This indicates that a number of other factors that we have not accounted for in our study design may have had a larger impact on the differences in process and outcome measures than those included in this study.

Future research

This study addresses the impact of an EHR on nursing care processes and outcomes. However, it does not address how nurses used the system in practice or sociotechnical factors that may influence its use. Further research examining variables contributing to the effective use of EHRs—linking organizational factors, technology design, and user behavior—are required to understand the causal mechanisms by which they can help improve healthcare quality and safety. As noted in the introduction section of this article, our understanding of the impact of health information technology on nurses and nursing practice is underdeveloped, as is our understanding of how it can best support nursing practice. More studies examining the impact of health information technology on nursing care processes and outcomes are required.

Conclusion

KP HealthConnect is a sophisticated, vendor supplied EHR; any benefits associated with its implementation can be potentially realized by other healthcare organizations considering investing in an EHR system. Our results suggest that implementing an EHR has the potential to contribute to improvements in both the processes and outcomes of nursing care. However, a variety of other factors, such as organizational and cultural features and external influences, also need to be taken into account when evaluating the effect of technology such as EHR on the quality of care provided by nurses. Further research is required to improve our understanding of how technology such as EHR can be used effectively to help improve the quality of care received by patients.