Abstract
Background:
Electronic health record (EHR) usability issues represent an emerging threat to the wellbeing of nurses and patients; however, few large studies have examined these relationships.
Objective:
To examine associations between EHR usability and nurse job (burnout, job dissatisfaction, and intention to leave) and surgical patient (inpatient mortality and 30-day readmission) outcomes.
Methods:
A cross-sectional analysis of linked American Hospital Association, state patient discharge, and nurse survey data was conducted. The sample included 343 hospitals, 1,281,848 surgical patients, and 12,004 nurses. Logistic regression models were used to assess relationships between EHR usability and outcomes, before and after accounting for EHR adoption level (comprehensive vs. basic or less) and other confounders.
Results:
In fully adjusted models, nurses who worked in hospitals with poorer EHR usability had significantly higher odds of burnout [odds ratio (OR)=1.41; 95% confidence interval (CI), 1.21-1.64], job dissatisfaction (OR=1.61, 95% CI, 1.37-1.90) and intention to leave (OR=1.31, 95% CI, 1.09-1.58) compared to nurses working in hospitals with better usability. Surgical patients treated in hospitals with poorer EHR usability had significantly higher odds of inpatient mortality (OR=1.21, 95% CI, 1.09-1.35) and 30-day readmission (OR=1.06, 95% CI, 1.01-1.12) compared to patients in hospitals with better usability. Comprehensive EHR adoption was associated with higher odds of nurse burnout (OR=1.14, 95% CI, 1.01-1.28).
Conclusion:
Employing EHR systems with suboptimal usability was associated with higher odds of adverse nurse job outcomes and surgical patient mortality and readmission. EHR usability may be more important to nurse job and patient outcomes than comprehensive EHR adoption.
Keywords: Electronic health records, Burnout, Usability, Patient outcomes, Nursing
INTRODUCTION
The National Academy of Medicine recently named inadequate technology usability as a significant contributor to the alarming levels of clinician burnout. [1] Specifically, electronic health record (EHR) usability has been linked to unfavorable job outcomes, including burnout and job dissatisfaction, among physicians [2-4] and advanced practice nurses.[5] EHR usability is defined as the extent to which the system can be used effectively, efficiently, and satisfactorily to complete tasks.[6] Usability issues, such as workflow disruptions and burdensome documentation, have also been reported widely among nurses [7-10] who constitute another large group of EHR users. Yet, the relationships between EHR usability and the job outcomes of hospital staff nurses, including burnout, job dissatisfaction, and intentions to leave their current position, have not been explored.
Given clinicians’ reliance on the EHR for patient care delivery, poor EHR usability has also emerged as a patient safety threat. Usability problems have been linked to actual and near-miss adverse events, such as medication errors. [11-13] Most of these analyses, however, were informed by qualitative incident reports.[13] There is a crucial need for larger observational studies that provide more generalizable data about the relationships between EHR usability and a broader set of patient outcomes. Adult surgical patients comprise a significant proportion of hospitalized patients, but no studies to our knowledge have considered how EHR usability may be associated with their care outcomes, including mortality and readmissions.
While evaluating the effects of EHR usability on the outcomes of nurses and patients, it is also important to account for the hospital’s EHR adoption level. It has been long-recognized that adoption of comprehensive EHR systems with more complex functionalities, such as clinical decision support tools, may be necessary to achieve optimal quality and safety outcomes.[14] Recent evidence regarding the effects of comprehensive EHR adoption on clinician end-users and patients, however, remains largely mixed.[15-17] A simultaneous examination of the effects of EHR usability and comprehensive EHR adoption on nurse job and patient outcomes could help to elucidate explanations for these equivocal findings.
To address these gaps in the literature, the objective of this study was to examine the associations between EHR usability and nurse job (burnout, job dissatisfaction, and intention to leave) and surgical patient (inpatient mortality and 30-day readmission) outcomes.
METHODS
Study Design
We sought to study hospitals representing the full range of EHR adoption by focusing on 4 large states (California, Florida, New Jersey, and Pennsylvania) from which patient outcomes could be obtained for all non-federal hospitals. A retrospective, cross-sectional analysis was performed using secondary data sources collected between 2015-2016, including: 1) the American Hospital Association (AHA)Annual Survey of Hospitals, 2) the AHA Healthcare Information Technology (IT) database, 3) patient discharge abstracts obtained from state agencies, and 4) the RN4CAST-US nurse survey. The hospitals included in the study were all those represented in all four data sources. Databases were linked for the analysis using a common hospital identifier. This study was reviewed the University of Pennsylvania Institutional Review Board.
Data Sources and Sample
The AHA Annual Survey of Hospitals provided data about the structural characteristics of study hospitals, including size, teaching status, and high-technology procedure capabilities. The AHA IT Database included variables indicating each hospital’s EHR functionalities and the degree to which they were adopted in clinical units.
Patient discharge abstracts were obtained from agencies representing the four states in the study, including California’s Office of Statewide Health Planning and Development, Florida’s Agency for Health Care Administration, the New Jersey Department of Health and Senior Services, and the Pennsylvania Health Care Cost Containment Council. Patient discharge data included information about demographics, primary and secondary diagnosis and procedure codes, inpatient mortality, and readmissions. All adult patients between the ages of 18-99 who were discharged between January 1, 2015 and December 31, 2016 for a set of general, orthopedic, or vascular surgical procedures were included in the sample. These procedures were selected for study because they are commonly performed at most hospitals and risk adjustment models have been validated. [18-20]
The RN4CAST-US nurse survey was conducted among a random sample of 30% of currently licensed registered nurses (RNs) in California, New Jersey, Pennsylvania, and Florida using state board licensure lists. Surveys were mailed to the homes of approximately 231,000 RNs followed by reminder postcards. The survey included questions related to demographics, job outcomes, patient workload, and EHR usability. Nurses were asked to provide the name of their primary employer which allowed for linkage to AHA and patient discharge data, as well as the creation of hospital-level measures. The overall survey achieved a 26% response rate, followed by an intensive follow-up survey of non-responders that yielded an 87% response rate.[21] Nurses are informants about hospitals in this study and thus, more important than the response rate is the number of nurses responding for each hospital. The average number of direct care nurse respondents per hospital was 35 and ranged from 10-146 across hospitals. Further details about the survey methodology are available elsewhere. [21,22]
Measures
EHR usability.
EHR usability was measured using a seven-item scale on the RN4CAST-US survey that was adapted from previous studies.[23, 24] Using a four-point Likert scale ranging from “strongly disagree” to “strongly agree,” nurses reported their level of agreement with the following statements about their primary EHR system’s features: ability to access patient information quickly in the system, system interference with the provision of patient care, ease of use, trust in the system's patient assessment and medication data, the system's ability to assist in completing work efficiently, and the ability to easily share information with other healthcare team members. The scale has a Cronbach’s alpha coefficient of 0.87 which indicates good internal consistency and exceeds the recommended value of 0.80 for applied research.[25] A hospital-level EHR usability score was derived by calculating the mean of all seven usability items for each individual nurse and then averaging across all nurses within a hospital. For all primary analysis, tertiles were used to categorize the EHR usability score for each hospital as: “poorer” (lowest tertile), “moderate” (middle tertile), and “better” (highest tertile) usability.
EHR adoption level:
Based on AHA IT Database information, hospital EHR adoption level was classified as 1) basic or less, or 2) comprehensive. Adoption levels were defined using established Office of the National Coordinator for Health Information Technology guidance.[26] Hospitals with a basic EHR had each of the following components either fully implemented on at least one clinical unit or across all units: 1) electronic clinical documentation of demographics, problem lists, medication lists, clinician notes, and discharge summaries, 2) electronic laboratory, radiologic and diagnostic test reports, and 3) computerized provider order entry (CPOE) for medications. Hospitals with comprehensive EHR systems had the three core components of a basic system as well as 14 additional functionalities, such as clinical decision support, CPOE for labs/radiology, and consultant report viewing, that were implemented fully across all units.
Nurse Job Outcomes.
The three nurse job outcomes were obtained from the RN4CAST-US survey and included: 1) burnout, measured by a score of 27 or greater on the well-validated emotional exhaustion subscale of the Maslach Burnout Inventory,[27] 2) job dissatisfaction, measured using a single item that asked respondents to rate their satisfaction with their current job on a 4-point Likert scale ranging from “very dissatisfied” to “very satisfied” (“very dissatisfied” and “a little dissatisfied” were classified as “dissatisfied” for the analysis), and 3) intention to leave, measured using a dichotomous, single item on the survey that asked respondents to indicate whether they planned to leave their current job in the next year.
Patient Outcomes.
Patient outcomes were obtained from the state discharge databases and included: 1) inpatient mortality within 30 days of admission, and 2) readmission within 30 days for any cause. Patients were excluded from the 30-day readmission measure who died during the index admission (no opportunity for readmission) or who were identified as an internal transfer (readmitted on the same day as index admission discharge).
Covariates.
Several hospital, nurse, and patient characteristics were included in the analysis to account for potential confounding. Hospital characteristics obtained from the AHA Annual Survey included size (≤ 100 beds, 101-250 beds, or >250 beds), teaching status (non-teaching, minor teaching, or major teaching), and high-technology procedure capability (i.e., ability to perform open-heart surgeries and/or major organ transplants), and state. Hospital characteristics obtained from RN4CAST-US included nurse staffing (mean number of patients/nurse) and percentage of baccalaureate-prepared nurses.[28] Nurse job outcome models included the hospital controls outlined above with the addition of several individual nurse characteristics obtained from RN4CAST-US: age, sex, years of RN experience, and unit type (medical/surgical, intensive care, or other). Patient outcome models included the hospital controls outlined above with the addition of individual patient characteristics obtained from the state discharge databases: age, sex, race/ethnicity, transfer from an outside hospital, surgical diagnosis-related group (DRG), and 27 comorbidities originally defined by Elixhauser [29] and adapted by Volpp and colleagues. [30]
Data Analysis
Characteristics and outcomes of nurses and patients in the sample were examined using descriptive statistics. Distributions of the study hospital characteristics were examined for the overall sample and by EHR usability level. Logistic regression models accounting for clustering of nurses and patients within hospitals were used to examine associations between hospital-level EHR usability tertiles and each nurse/patient outcome. Unadjusted models of EHR usability and individual outcomes were examined first, followed by models adjusting for EHR adoption level (comprehensive vs. basic or less). Finally, a fully adjusted model that included EHR usability, EHR adoption level, and all measured covariates was examined. We also conducted a sensitivity analysis that examined hospital-level EHR usability as a continuous variable in the predictive models. SAS 9.4 was used to conduct the analysis.
RESULTS
The final sample included 12,004 RNs and 1,281,848 patients embedded in 343 hospitals across the four states. Characteristics and outcomes of the nurses and patients in the sample are displayed in Table 1. On average, nurses were 46.6 years of age and had over 18 years of RN experience. Approximately 90% of nurses in our sample were female. Nearly three-quarters of respondents were employed full-time (73.7%). One-third (33.1%) of nurses in the sample reported high emotional exhaustion, or burnout. Over 20% of nurses were dissatisfied with their primary job and 12% reported an intention to leave their jobs in the next year.
Table 1.
Characteristics and Outcomes of Nurses (n=12,004) and Surgical Patients (n=1,281,848)
| Nurse Characteristics | |
|---|---|
| Age, mean (SD) | 46.6 (12.3) |
| Female, n (%) | 10,819 (90.3%) |
| Years of RN experience, mean (SD) | 18.3 (12.9) |
| Full-time employed, n(%) | 8,731 (73.7%) |
| Unit Type, n(%) | |
| Medical-Surgical | 3,444 (29.5%) |
| Intensive Care | 2,444 (20.5%) |
| Other | 5,671 (48.6%) |
| Nurse Job Outcomes | |
| Burnout (high emotional exhaustion), n (%) | 3,160 (33.1%) |
| Dissatisfied with Primary Job, n (%) | 2,049 (20.9%) |
| Intention to Leave, n (%) | 1,440 (12.0%) |
| Patient Characteristics | |
| Age, mean (SD) | 61.9 (16.6) |
| Male, n (%) | 588,641 (45.9%) |
| Race/Ethnicity, n(%) | |
| Non-Hispanic white | 913,964 (72.1%) |
| Non-Hispanic black | 114,180 (9.0%) |
| Hispanic | 169,334 (13.4%) |
| Other race | 69,774 (5.5%) |
| Elixhauser comorbidities (Top 5), n(%) | |
| Hypertension | 738,579 (57.6%) |
| Obesity | 220,794 (17.2%) |
| Diabetes mellitus | 205,296 (16.0%) |
| Chronic lung disease | 201,965 (15.8%) |
| Deficiency Anemia | 173,234 (13.5%) |
| Surgical procedure, n(%) | |
| General | 458,593 (35.8%) |
| Orthopedic | 650,817 (50.8%) |
| Vascular | 172,438 (13.5%) |
| Transfer from outside hospital, n(%) | 34,476 (2.7%) |
| Patient Outcomes | |
| Inpatient mortality, n(%) | 10,070 (0.8%) |
| 30-day readmission, n(%) | 106,796 (8.4%) |
Note: Percentages may not add to 100 due to rounding and small amounts of missing data (<5%). Sample size for nurse outcomes range from 9,546 (emotional exhaustion) to 12,004 (intention to leave). Denominator for 30-day readmissions=1,271,778
Our census of surgical patients had a mean age of 61.9 years and approximately 46% were male. The sample was predominantly white (72.1%), with black and Hispanic patients comprising 9% and 13% of the sample, respectively. The most common comorbidities in our sample included hypertension (57.6%), followed by obesity (17.2%) and uncomplicated diabetes mellitus (16.0%). Over half (50.8%) of patients in the sample were admitted for an orthopedic procedure, while approximately 36% were admitted for general surgery, and 14% for a vascular procedure. Fewer than 1% (0.8%) of the surgical patients in our sample died in the hospital within 30 days of admission, while 8.4% were readmitted to the hospital within 30 days.
Table 2 displays the characteristics of study hospitals overall and by EHR usability level. Statistically significant differences were noted in usability by EHR adoption level and teaching status. Nearly half (46%) of hospitals that adopted comprehensive EHR systems were classified as having better EHR usability compared to 23% of hospitals with basic EHR systems or less. Better usability was also associated with teaching status, with nearly half of major teaching hospitals (52.6%) being classified in the better usability group, compared to minor (24.8%) and non-teaching (37.5%) facilities.
Table 2.
Characteristics of Study Hospitals Overall and by Degree of EHR Usability (n=343)
| Overall (n=343) |
Poorer EHR Usability (n=116) |
Moderate EHR Usability (n=113) |
Better EHR Usability (n=114) |
P | |
|---|---|---|---|---|---|
| EHR Adoption Level, n (%) | |||||
| Basic EHR or less | 189 (55.1%) | 77 (40.7%) | 69 (36.5%) | 43 (22.8%) | <0.001 |
| Comprehensive EHR | 154 (44.9%) | 39 (25.3%) | 44 (28.6%) | 71 (46.1%) | |
| Size, n (%) | |||||
| ≤ 100 beds | 15 (4.4%) | 9 (60.0%) | 2 (13.3%) | 4 (26.7%) | 0.08 |
| 101-250 beds | 124 (36.2%) | 48 (38.7%) | 38 (30.7%) | 38 (30.7%) | |
| >250 beds | 204 (59.5%) | 59 (28.9%) | 73 (35.8%) | 72 (35.3%) | |
| Teaching Status, n (%) | |||||
| Non-teaching | 144 (42.0%) | 41 (28.5%) | 49 (34.0%) | 54 (37.5%) | <0.001 |
| Minor teaching | 161 (46.9%) | 70 (43.5%) | 51 (31.7%) | 40 (24.8%) | |
| Major teaching | 38 (11.1%) | 5 (13.2%) | 13 (34.2%) | 20 (52.6%) | |
| Technology status, n (%) | |||||
| Low | 134 (39.1%) | 51 (38.1%) | 38 (28.4%) | 45 (33.6%) | 0.28 |
| High (performs open heart surgery and/or organ transplants) | 209 (60.9%) | 65 (31.1%) | 75 (36.9%) | 69 (33.0%) | |
| State, n (%) | |||||
| CA | 133 (38.8%) | 43 (32.3%) | 35 (26.3%) | 55 (41.4%) | 0.10 |
| FL | 41 (12.0%) | 13 (31.7%) | 18 (43.9%) | 10 (24.4%) | |
| NJ | 74 (21.6%) | 28 (37.8%) | 29 (39.2%) | 17 (23.0%) | |
| PA | 95 (27.7%) | 32 (33.7%) | 31 (32.6%) | 32 (33.7%) |
Note: P-values generated from chi-square for all variables, except for teaching status where Fisher’s exact test was used. 53 hospitals had adopted less than a Basic EHR. Percentages may not add to 100 due to rounding. EHR usability categories were defined using tertiles of the hospital-level EHR usability score: poorer (lowest tertile), moderate (middle tertile), and better (highest tertile).
Unadjusted and adjusted models of EHR usability and nurse job outcomes are presented in Table 3. In unadjusted models, poorer EHR usability was associated with higher odds of burnout (odds ratio [OR] 1.42, 95% confidence interval [CI] 1.23-1.63, p<0.001), job dissatisfaction (OR 1.71, 95% CI 1.45-2.02, p<0.001) and intention to leave (OR 1.30, 95% CI 1.10-1.55, p=0.003) as compared to nurses working in hospitals with better EHR usability. After fully adjusting for EHR adoption level and other potential confounders, the observed associations between poor EHR usability and poor nurse job outcomes remained fairly stable: burnout (OR 1.41, 95% CI 1.21-1.64, p<0.001), job dissatisfaction (OR 1.61, 95% CI 1.37-1.90, p<0.001) and intention to leave (OR 1.31, 95% CI 1.09-1.58, p=0.004). Of note, adoption of a comprehensive EHR was associated only with higher odds of burnout in the fully adjusted models (OR 1.14, 95% CI 1.01-1.28, p=0.03).
Table 3.
Unadjusted and Adjusted Odds Ratios Indicating the Effects of Electronic Health Record (EHR) Usability on Nurse Job Outcomes
| Unadjusted | Adjusted for EHR Adoption Level | Fully Adjusted | ||||
|---|---|---|---|---|---|---|
| Outcome | OR (95% CI) | P | OR (95% CI) | P | OR (95% CI) | P |
| Burnout (High Emotional Exhaustion) | ||||||
| EHR Usability (reference: Better) | ||||||
| Poorer | 1.42(1.23-1.63) | <0.001 | 1.46 (1.25-1.70) | <0.001 | 1.41 (1.21-1.64) | <0.001 |
| Moderate | 1.14 (1.00-1.30) | 0.05 | 1.16 (1.02-1.32) | 0.02 | 1.14 (1.00-1.30) | 0.06 |
| Comprehensive EHR (reference: basic EHR or less) | -- | 1.09 (0.97-1.23) | 0.17 | 1.14 (1.01-1.28) | 0.03 | |
| Job Dissatisfaction | ||||||
| EHR Usability (reference: Better) | ||||||
| Poorer | 1.71 (1.45-2.02) | <0.001 | 1.77 (1.48-2.12) | <0.001 | 1.61 (1.37-1.90) | <0.001 |
| Moderate | 1.32(1.12-1.55) | 0.001 | 1.35 (1.15-1.59) | <0.001 | 1.24 (1.07-1.44) | 0.004 |
| Comprehensive EHR (reference: basic EHR or less) | -- | 1.10 (0.96-1.26) | 0.16 | 1.09 (0.96-1.23) | 0.18 | |
| Intent to Leave | ||||||
| EHR Usability (reference: Better) | ||||||
| Poorer | 1.30 (1.10-1.55) | 0.003 | 1.24 (1.04-1.48) | 0.02 | 1.31 (1.09-1.58) | 0.004 |
| Moderate | 1.02 (0.86-1.21) | 0.80 | 0.98 (0.83-1.17) | 0.85 | 1.04 (0.87-1.24) | 0.66 |
| Comprehensive EHR (reference: basic EHR or less) | -- | 0.86 (0.74-0.99) | 0.04 | 0.88 (0.76-1.01) | 0.08 | |
Notes: OR=odds ratio. CI=confidence interval. EHR usability categories were defined using tertiles of the hospital-level EHR usability score: poorer (lowest tertile), moderate (middle tertile), and better (highest tertile). Odds ratios in job outcome models are from robust logistic regression models adjusted for hospital characteristics (size, teaching status, high-technology procedure capability, state, nurse staffing level, % of nurses with bachelor’s degree or higher), nurse characteristics (age, sex, years of RN experience, unit type(medical/surgical, intensive care, other) and the clustering of nurses within hospitals. Sample size for nurse outcomes range from 9,546 (emotional exhaustion) to 12,004 (intention to leave).
Poorer EHR usability was not associated with surgical patient outcomes in unadjusted models: inpatient mortality (OR 1.11, 95% CI 0.99-1.25, p=0.08) and 30-day readmission (OR 0.98, 95% CI 0.92-1.05, p=0.59). EHR usability emerged as a statistically significant predictor of inpatient mortality and 30-day readmissions in the fully adjusted models that adjusted for EHR adoption level and additional patient and hospital covariates (Table 4). In the fully adjusted models, patients in hospitals with poorer EHR usability had significantly higher odds of inpatient death (OR 1.21, 95% CI 1.09-1.35, p<0.001) and 30-day readmission (OR 1.06, 95% CI 1.01-1.12, p=0.01) compared to patients in hospitals with better EHR usability. Adoption of a comprehensive EHR system was not associated with patient outcomes in the fully adjusted models. Our sensitivity analyses that measured EHR usability as a linear variable demonstrated similar effects to the categorical measure in the nurse and patient outcome models (See Supplementary File: Appendix Tables A and B).
Table 4.
Unadjusted and Adjusted Odds Ratios Indicating the Effects of Electronic Health Record (EHR) Usability on Surgical Patient Outcomes
| Unadjusted | Adjusted for EHR Adoption Level |
Fully Adjusted | ||||
|---|---|---|---|---|---|---|
| Outcome | OR (95% CI) | P | OR (95% CI) | P | OR (95% CI) | P |
| Inpatient Mortality | ||||||
| EHR Usability (reference: Better) | ||||||
| Poorer | 1.11 (0.99-1.25) | 0.08 | 1.12 (1.00-1.26) | 0.05 | 1.21 (1.09-1.35) | <0.001 |
| Moderate | 1.04 (0.94-1.15) | 0.44 | 1.05 (0.95-1.16) | 0.36 | 1.07 (0.96-1.19) | 0.22 |
| Comprehensive EHR (reference: basic EHR or less) | -- | 1.03 (0.94-1.13) | 0.49 | 0.98 (0.91-1.07) | 0.70 | |
| 30-Day Readmission | ||||||
| EHR Usability (reference: Better) | ||||||
| Poorer | 0.98 (0.92-1.05) | 0.59 | 0.98 (0.92-1.05) | 0.63 | 1.06 (1.01-1.12) | 0.01 |
| Moderate | 0.99 (0.92-1.05) | 0.68 | 0.99 (0.92-1.06) | 0.71 | 1.03 (0.98-1.08) | 0.30 |
| Comprehensive EHR (reference: basic EHR or less) | -- | 1.00 (0.95-1.06) | 0.89 | 0.98 (0.94-1.02) | 0.25 | |
Notes: OR=odds ratio, CI=confidence interval. EHR usability categories were defined using tertiles of the hospital-level EHR usability score: poorer (lowest tertile), moderate (middle tertile), and better (highest tertile). Odds ratios are from robust logistic regression models adjusted for hospital characteristics (size, teaching status, high-technology procedure capability, state, nurse staffing level, % of nurses with bachelor’s degree or higher), patient characteristics (age, male sex, race/ethnicity, transfer status, surgical diagnosis-related group, Elixhauser comorbidities) and the clustering of patients within hospitals. N=1,281,848 for inpatient mortality. N= 1,271,778 for 30-day readmissions.
DISCUSSION
We found that poor EHR usability was associated with adverse nurse job and surgical patient outcomes in a sample of over 300 hospitals located in 4 large U.S. states. Nurses practicing in hospitals with poor EHR usability had significantly higher odds of burnout, job dissatisfaction, and intention to leave their current positions compared to nurses using EHRs with better usability. Further, our results demonstrate previously undocumented associations between EHR usability and patient outcomes. Surgical patients receiving care in hospitals with poorer EHR usability were 21% more likely to die in the hospital following their procedure compared to patients in hospitals with better usability. A smaller, yet statistically significant, effect was also observed between poorer EHR usability and higher odds of 30-day readmission.
Our findings linking poor EHR usability to burnout and job dissatisfaction in a large sample of hospital staff nurses are strongly aligned with prior studies of physicians and advanced practice nurses.[2-5] To our knowledge, this study is among the first to link EHR usability to intention to leave, which has significant implications for nurse turnover and its associated costs.[31] Our study adds to the growing body of literature documenting the growing frustration of nurses with underperforming EHR systems.[7, 8] The literature is replete with reports of nurses and other clinicians frequently engaging in workarounds to perform essential tasks due to EHR system failures.[10, 32] Our study builds upon these findings and suggests that employing EHR systems with suboptimal usability may be partly responsible for the growing prevalence of burnout among nurses.
Our results also demonstrate that EHR usability issues may have dangerous consequences for patients that are greater than previously thought. Specifically, we found increased odds of inpatient mortality and readmissions following common surgery in hospitals with poorer EHR usability. Nurses routinely use the EHR in the performance of one of their primary roles: patient surveillance. [33] As the healthcare providers who interface with patients most frequently, nurses are often the first to detect and act upon clinical deterioration.[34] EHR systems with poor usability can significantly hinder a nurse’s ability to quickly access trusted information for decision-making and communicate with other members of the healthcare team. Lapses in these critical care processes may contribute to significant delays or interruptions to the provision of both inpatient and post-discharge care, and in turn, medical errors and other poor outcomes.[13, 32] Our findings are especially concerning for patients given the pervasiveness of EHR usability problems.[9, 12, 35]
Apart from burnout, we did not observe a significant relationship between comprehensive EHR adoption and nurse job and patient outcomes while accounting for usability. This finding suggests that unmeasured differences in EHR system usability could account for the equivocal findings to date of the effects of comprehensive EHR adoption on patient outcomes. While some research demonstrates that the adoption of more complex systems may be needed to experience the promised effects of EHRs, [17, 36] our results suggest that system usability may ultimately be more important to nurse job and patient outcomes. In our sample, comprehensive EHR system adoption was associated with higher odds of burnout. This finding is aligned with a prior study by Singh and colleagues [37] who found that use of a comprehensive EHR was associated with perceived information overload. While Singh and colleagues concluded that information overload may lead to missing important patient information, our work implies that the emotional wellbeing of nurses may also be compromised. Future research on the relationship between EHR adoption and clinician and patient outcomes should incorporate standardized measures of system usability in order to obtain the most comprehensive assessment of how EHRs affect the delivery of care.
Calls for the improvement of EHR usability have been growing.[38,39] Despite federal requirements for user-centric designs, evidence suggests that significant variation exists in how vendors meet this criteria to obtain EHR certification.[40] Further, recent evidence suggests that usability ratings for EHRs installed by several of the nation’s largest vendors have not significantly improved over time.[41] In response to increasing usability issues and rising rates of clinician burnout, the Department of Health and Human Services (HHS), including the Office of the National Coordinator for Health Information Technology (ONC) and the Centers for Medicare and Medicaid Services (CMS) released a set of strategies aimed at reducing EHR-associated clinician burden. [42] Several potential strategies were proposed, including aligning the EHR to clinical workflow, streamlining and standardizing the presentation of clinical information, and securing end-user perspectives in the implementation process. The report also outlined concrete plans to reduce documentation burden associated with EHRs, which also plays a significant role in how clinicians assess usability.[3] The findings of the current study both confirm and amplify the significance of these national policy efforts.
Our findings have implications for EHR vendors and hospital leadership teams by underscoring the critical importance of nurse involvement in the development, selection, implementation, and modification of EHR systems. Failing to solicit the input of nurses at each of these timepoints may have adverse effects on nurse wellbeing, job satisfaction, and retention, as well as patient outcomes. The engagement level of nurses in EHR-related decisions may be largely determined by existing organizational structures and processes.[43] In prior work, we have found that better nurse job and patient outcomes occur in hospitals that are supportive of nurses and promote nurse involvement in organizational decision-making. [44,45] Nurses working in hospitals with these characteristics are also more likely to report greater satisfaction with the EHR and involvement with EHR selection or modification. [9]
We acknowledge some limitations to our study. Due to its correlational design, we cannot draw conclusions about the causality of the relationships we observed. Omitted variables may have biased our results. The AHA IT database does not provide the number of years that each hospital’s EHR had been in use. This is important because the relationship between EHR adoption and patient mortality may improve over time.[46] Likewise, EHR usability and its relationships with nurse job and patient outcomes may also take time to mature. Poor EHR usability may be an indicator of other unmeasured hospital characteristics, such as inadequate resources and low clinician engagement, that are associated with adverse nurse job and patient outcomes. These concerns are tempered by our large sample of hospitals that represents a wide range of EHR adoption levels and structural characteristics. We were also unable to examine the usability of specific EHR components. While our study focused on staff nurses, other clinicians, such as physicians and therapists, may have different needs when interacting with EHR systems. Inpatient healthcare is highly team-based and EHR vendors should consider this when designing systems.[43] Our sample was limited to assessments of hospital-based EHRs. It is likely that our patient outcomes, especially readmissions, may also be influenced by the EHR systems used by ambulatory or post-acute care providers. Finally, the generalizability of our study may be limited due to the examination of hospitals, nurses, and patients in four U.S. states. Our study hospitals, however, represent over 20% of all acute care hospitals and patient discharges nationally and include geographically diverse areas of the country. We have also found that RN4CAST-US respondents had similar characteristics (e.g., age, education) to a national sample of nurses included in the 2017 Current Population Survey.[47]
Nurses and other clinicians rely heavily upon the EHR in the provision of patient care for clinical decision making, care planning, patient surveillance, medication ordering and administration, and communication with other members of the healthcare team. When the EHR system does not allow this work to be performed efficiently and effectively, nurse burden increases, and patient outcomes are threatened. Improving EHR usability may be critical to reducing nurse burnout and to realizing the full potential of EHRs to improve the quality and safety of healthcare.
Supplementary Material
Acknowledgements:
A portion of the study’s findings were presented at the AcademyHealth Annual Research Meeting in Washington, D.C., on June 4, 2019.
Funding: This project was supported by grant number R21HS023805 (Kutney-Lee) from the Agency for Healthcare Research and Quality, and grant number R01NR014855 (Aiken) from the National Institute of Nursing Research, National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality, the National Institutes of Health, or Department of Veterans Affairs.
Footnotes
Conflict of Interest: The authors have no conflicts of interest to declare.
Contributor Information
Ann Kutney-Lee, Center for Health Outcomes and Policy Research, University of Pennsylvania School of Nursing, Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA.
J. Margo Brooks Carthon, Center for Health Outcomes and Policy Research, Leonard Davis Institute for Health Economics, University of Pennsylvania School of Nursing, Philadelphia, PA.
Douglas M. Sloane, Center for Health Outcomes and Policy Research, University of Pennsylvania School of Nursing, Philadelphia, PA.
Kathryn H. Bowles, NewCourtland Center for Transitions and Health, Leonard Davis Institute for Health Economics, University of Pennsylvania School of Nursing, Philadelphia, PA.
Matthew D. McHugh, Center for Health Outcomes and Policy Research, Leonard Davis Institute for Health Economics, University of Pennsylvania School of Nursing, Philadelphia, PA.
Linda H. Aiken, Center for Health Outcomes and Policy Research, Leonard Davis Institute for Health Economics, University of Pennsylvania School of Nursing, Philadelphia, PA.
REFERENCES
- 1.National Academies of Science, Engineering, and Medicine. Taking Action Against Clinician Burnout: A Systems Approach to Professional Well-Being. Washington, D.C.: The National Academies Press; 2019. [PubMed] [Google Scholar]
- 2.Melnick ER, Dyrbye LN, Sinsky CA, et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020;95(3):476–487. [DOI] [PubMed] [Google Scholar]
- 3.Kroth PJ, Morioka-Douglas N, Veres S, et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019;2(8):e199609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016;91(7):836–48. [DOI] [PubMed] [Google Scholar]
- 5.Harris DA, Haskell J, Cooper E, et al. Estimating the association between burnout and electronic health record-related stress among advanced practice registered nurses. Appl Nurs Res 2018;43:36–41. [DOI] [PubMed] [Google Scholar]
- 6.Bevan N International standards for HCI and usability. Int J Hum-Comput Stud 2001;55:533–552. [Google Scholar]
- 7.Topaz M, Ronquillo C, Peltonen L, et al. Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: Results from an international survey. AMIA Annu Symp Proc 2016;2016:2016–2025. [PMC free article] [PubMed] [Google Scholar]
- 8.Staggers N, Elias BL, Makar E, et al. The imperative of solving nurses' usability problems with health information technology. J Nurs Adm 2018;48(4):191–196. [DOI] [PubMed] [Google Scholar]
- 9.Kutney-Lee A, Sloane DM, Bowles KH, et al. Electronic health record adoption and nurse reports of usability and quality of care: The role of work environment. Appl Clin Inform 2019;10(1):129–139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gephart S, Carrington JM, Finley B. A systematic review of nurses' experiences with unintended consequences when using the electronic health record. Nurs Admin Q 2015;39(4):345–356. [DOI] [PubMed] [Google Scholar]
- 11.Howe JL, Adams KT, Hettinger AZ, et al. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018;319(12):1276–1278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ratwani RM, Savage E, Will A, et al. Identifying electronic health record usability and safety challenges in pediatric settings. Health Aff 2018;37(11):1752–1759. [DOI] [PubMed] [Google Scholar]
- 13.Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: A systematic review. J Am Med Inform Assoc 2017;24(2):246–250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Blumenthal D, Tavenner M. The "meaningful use" regulation for electronic health records. N Engl J Med 2010;363(6):501–504. [DOI] [PubMed] [Google Scholar]
- 15.Yuan N, Adams Dudley R, Boscardin WJ, et al. Electronic health records systems and hospital clinical performance: a study of nationwide hospital data. J Am Med Inform Assoc 2019;26(10):999–1009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Krousel-Wood M, McCoy AB, Ahia C, et al. Implementing electronic health records (EHRs): health care provider perceptions before and after transition from a local basic EHR to a commercial comprehensive EHR. J Am Med Inform Assoc 2018;25(6):618–626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Adler-Milstein J, Everson J, Lee SY. EHR adoption and hospital performance: Time-related effects. Health Serv Res 2015;50(6):1751–1771. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.McHugh MD, Kelly LA, Smith HL, et al. Lower mortality in Magnet hospitals. Med Care 2013;51(5):382–388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Aiken LH, Cimiotti JP, Sloane DM, et al. Effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Med Care 2011;49(12):1047–1053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ma C, McHugh MD, Aiken LH. Organization of hospital nursing and 30-day readmissions in Medicare patients undergoing surgery. Med Care 2015;53(1):65–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Lasater KB, Jarrin OF, Aiken LH, et al. A methodology for studying organizational performance: A multistate survey of front-line providers. Med Care 2019;57(9):742–749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sloane DM, Smith HL, McHugh MD, et al. Effect of changes in hospital nursing resources on improvements in patient safety and quality of care: A panel study. Med Care 2018;56(12):1001–1008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Sockolow PS, Weiner JP, Bowles KH, et al. A new instrument for measuring clinician satisfaction with electronic health records. Comput Inform Nurs 2011;29(10):574–585. [DOI] [PubMed] [Google Scholar]
- 24.Otieno OG, Toyama H, Asonuma M, et al. Nurses' views on the use, quality and user satisfaction with electronic medical records: questionnaire development. J Adv Nurs 2007;60(2):209–219. [DOI] [PubMed] [Google Scholar]
- 25.Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill; 1994. [Google Scholar]
- 26.Henry J, Pylypchuk Y, Searcy T, et al. Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008-2015. ONC Data Brief 2016;35. Office of the National Coordinator for Health Information Technology: Washington, D.C. [Google Scholar]
- 27.Maslach C, Jackson SE, Leiter MP. The Maslach Burnout Inventory: Manual. Palo Alto, CA: Consulting Psychologists; 1986. [Google Scholar]
- 28.Aiken LH, Clarke SP, Cheung RB, et al. Educational levels of hospital nurses and surgical patient mortality. JAMA 2003;290(12):1617–1623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Elixhauser A, Steiner C, Harris DR, et al. Comorbidity measures for use with administrative data. Med Care 1998;36(1):8–27. [DOI] [PubMed] [Google Scholar]
- 30.Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. JAMA 2007;298(9):984–992. [DOI] [PubMed] [Google Scholar]
- 31.Li Y, Jones CB. A literature review of nursing turnover costs. J Nurs Manag 2013;21(3):405–418. [DOI] [PubMed] [Google Scholar]
- 32.Assis-Hassid S, Grosz BJ, Zimlichman E, et al. Assessing EHR use during hospital morning rounds: A multi-faceted study. PLoS One 2019;14(2):e0212816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lopez KD, Fahey L. Advocating for greater usability in clinical technologies: The role of the practicing nurse. Crit Care Nurs Clin North Am 2018;30(2):247–257. [DOI] [PubMed] [Google Scholar]
- 34.Kutney-Lee A, Lake ET, Aiken LH. Development of the hospital nurse surveillance capacity profile. Res Nurs Health 2009;32(2):217–228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Ratwani RM, Savage E, Will A, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018;25(9):1197–1201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Hessels A, Flynn L, Cimiotti JP, et al. Impact of heath information technology on the quality of patient care. Online J Nurs Inform 2015;19. [PMC free article] [PubMed] [Google Scholar]
- 37.Singh H, Spitzmueller C, Petersen NJ, et al. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med 2013;173(8):702–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ratwani RM, Reider J, Singh H. A decade of health information technology usability challenges and the path forward. JAMA 2019;321(8):743–744. [DOI] [PubMed] [Google Scholar]
- 39.Koppel R, Kuziemsky C. Usability across health information technology systems: Searching for commonalities and consistency. Stud Health Technol Inform 2019;264:649–653. [DOI] [PubMed] [Google Scholar]
- 40.Ratwani RM, Benda NC, Hettinger AZ, et al. Electronic health record vendor adherence to usability certification requirements and testing standards. JAMA 2015;314(10):1070–1. [DOI] [PubMed] [Google Scholar]
- 41.Gomes KM, Ratwani RM. Evaluating improvements and shortcomings in clinician satisfaction with electronic health record usability. JAMA Netw Open 2019;2(12):e1916651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Office of the National Coordinator for Health Information Technology. Strategy on reducing regulatory and administrative burden relating to the use of health IT and EHRs. 2020; Available from: https://www.healthit.gov/sites/default/files/page/2020-02/BurdenReport_0.pdf. Accessed December 15, 2020.
- 43.Carayon P, Hoonakker P. Human factors and usability for health information technology: Old and new challenges. Yearb Med Inform 2019;28(1):71–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Brooks Carthon JM, Hatfield L, Plover C, et al. Association of nurse engagement and nurse staffing on patient safety. J Nurs Care Qual 2019;34(1):40–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Kutney-Lee A, Germack H, Hatfield L, et al. Nurse engagement in shared governance and patient and nurse outcomes. J Nurs Adm 2016;46(11):605–612. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Lin SC, Jha AK, Adler-Milstein J. Electronic health records associated with lower hospital mortality after systems have time to mature. Health Aff 2018;37(7):1128–1135. [DOI] [PubMed] [Google Scholar]
- 47.Aiken LH, Sloane DM, Barnes H, et al. Nurses' and patients' appraisals show patient safety in hospitals remains a concern. Health Aff 2018;37(11):1744–1751. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.