Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 27.
Published in final edited form as: J Am Assoc Nurse Pract. 2021 Jan 27:10.1097/JXX.0000000000000533. doi: 10.1097/JXX.0000000000000533

Use of multifunctional electronic health records and burnout among primary care nurse practitioners

Cilgy M Abraham 1, Katherine Zheng 1, Allison A Norful 1, Affan Ghaffari 1, Jianfang Liu 1, Maxim Topaz 1, Lusine Poghosyan 1
PMCID: PMC8351449  NIHMSID: NIHMS1726347  PMID: 33534286

Abstract

Background:

Prevalence of electronic health records (EHRs) has significantly increased, and EHRs are a known contributor to clinician burnout. However, it is unknown whether the use of multifunctional EHRs is associated with nurse practitioner (NP) burnout in primary care practices. This is a major gap in the literature because 69% of practicing NPs deliver primary care services to patients.

Purpose:

This study aimed to investigate whether the use of multifunctional EHRs is associated with primary care NP burnout.

Method:

This study is a secondary analysis of cross-sectional survey data collected from NPs in two states (Pennsylvania and New Jersey). Nurse practitioners completed surveys measuring burnout, use of multifunctional EHRs, demographics, and characteristics of their practice. Use of multifunctional EHRs was operationalized using two items—computerized capabilities and electronic reminder systems. Burnout was measured using a validated, single item asking NPs to self-report their feelings of burnout. A multilevel cox regression model was built to test for associations between the use of multifunctional EHRs and NP burnout.

Results:

Of 396 NPs included, 25.3% reported burnout. The use of multifunctional EHRs did not increase primary care NP burnout (risk ratio = 0.30, 95% confidence interval = 0.13–0.71, p = .01).

Implications for practice:

With 25.3% of NPs burned out, it is imperative to reduce NP burnout. However, computerized capabilities and electronic reminder systems did not contribute to feelings of NP burnout. Future research examining other EHR components is needed to understand which features of the EHR contribute to NP burnout.

Introduction

In the United States, nearly 48% of primary care providers (PCPs) including physicians, nurse practitioners (NPs), and physician assistants are experiencing burnout (Edwards et al., 2018). Burnout is defined as an internalized feeling of emotional exhaustion, depersonalization, and feelings of low personal accomplishment (Maslach & Jackson, 1981). In 2018, the American Public Health Association identified burnout as a significant public health problem, and accumulating evidence shows that burnout negatively impacts clinicians, patients, and the entire health care system (Krisberg, 2018; West et al., 2018). Clinician burnout can lead to medical errors (West et al., 2018), increased risk of motor vehicle accidents and suicidality (Patel et al., 2018), and increased intention to leave their job (Abraham et al., 2020; West et al., 2018).

There are several known factors that contribute to clinician burnout including high workloads, poor staffing, long work hours, poor practice environments, job stress, loss of workplace autonomy, and limited support and resources to provide patient care (Abraham et al., 2020; West et al., 2018). A large body of research shows that widespread use of electronic health records (EHRs) is emerging as another major source of burnout for PCPs (Abraham et al., 2020; Babbott et al., 2014; Willard-Grace et al., 2019). Electronic health records in primary care practices include reminder systems and multiple computerized features allowing for patient medication updates, use of drug-related alerts, prescription ordering, and documentation of clinical notes during a patient visit (Huang et al., 2018). In a national study involving physicians, those who used EHRs and computerized physician order entries were less satisfied with the amount of time they spent on clerical tasks and were at greater risk for burnout (Shanafelt et al., 2016). Furthermore, 62.5% of physicians believed that the EHR did not improve their efficiency in providing patient care (Shanafelt et al., 2016), which is concerning because inefficiencies within practices can lead to physician burnout (Panagioti et al., 2018).

Since the passage of the Health Information Technology for Economic and Clinical Health Act in 2009, the use of EHRs in primary care practices increased from 25% to nearly 90% (Gordon et al., 2015). Providers increasingly use EHRs with multiple features, poor usability, and interoperability (Howe et al., 2018) and report that using these multifunctional EHRs contributes to their feelings of burnout (Babbott et al., 2014; Robertson et al., 2017). In particular, primary care physicians report greater burnout in practices with more EHR features such as electronic alerts and computerized capabilities (e.g., electronic medication prescribing, laboratory requests and results, radiology reports and images, referrals to specialists, discharge summaries, and electronic messaging to and from patients) (Babbott et al., 2014). As a result, 75% of primary care physicians attribute their feelings of burnout to EHR use (Robertson et al., 2017). One reason why the EHR may contribute to physician burnout is because of the increased time physicians spend on the EHR. For example, in a multisite study with physicians in ambulatory facilities across Illinois, New Hampshire, Virginia, and Washington, physicians spent an additional 2 hours on the EHR for every hour that they provided direct clinical care to patients (Sinsky et al., 2016). The increased time physicians spend on the EHR may stem from the complex features within the EHR, such as computerized physician order entries, patient portals, medical documentation requirements, and management of one’s inbox (Arndt et al., 2017) and can increase the risk for burnout.

Overall, the use of multifunctional EHRs containing various computerized capabilities and poor usability or operability contribute to primary care physician burnout (Babbott et al., 2014; Edwards et al., 2018; Helfrich et al., 2014; Linzer et al., 2009; Rabatin et al., 2016). However, it is unknown whether the use of multifunctional EHRs is associated with burnout in primary care NPs who, in 2016, represent about 25.2% of the primary care workforce in rural practices and 23.0% in nonrural practices (Barnes et al., 2018). Primary care NPs have obtained additional education at the Masters or at the Doctoral level and are qualified to assess, diagnose, and treat patients; manage chronic conditions; order tests and prescribe medications; and provide health and wellness to those across the lifespan (American Association of Nurse Practitioners [AANP], 2020b). In the United States, 69% of practicing NPs deliver primary care services to patients (AANP, 2020a). Compared with all PCPs, primary care NPs have the fastest growing PCP workforce with an expected growth of 93% from 2013 to 2025 in the United States (U.S. Department of Health and Human Services, 2016).

In addition to a growing NP workforce, NPs are using EHRs, which is a known predictor of clinician burnout (West et al., 2018), but it is unknown whether using multifunctional EHRs is a contributor of primary care NP burnout. With NPs delivering a significant portion of primary care to patients, if the EHR is associated with NP burnout, interventions can be designed to improve the usability and operability of EHRs in primary care practices, thus promoting NP well-being and improving patient care. The purpose of this study was to determine whether the use of multifunctional EHRs is associated with NP burnout in primary care practices.

Methods

Conceptual framework

The study is guided by the Clinician Well-Being and Resilience model (Brigham et al., 2018). Researchers at the National Academy of Medicine developed the model to show the factors associated with clinician burnout and well-being across all health care professions (Brigham et al., 2018). For this study, we adapted the model to focus on the relationship between use of multifunctional EHRs and primary care NP burnout while also considering NP and practice characteristics (Figure 1). The use of multifunctional EHRs is conceptualized as computerized capabilities and electronic reminder systems for decision support of clinical guidelines, which exist within the primary care practice (Friedberg et al., 2009). The adapted conceptual model also includes characteristics describing NPs such as age, sex, education, marital status, hours worked per week, workload, experience, and years employed in the current practice. Characteristics of the primary care practice include size, type of practice, and hours of operation.

Figure 1.

Figure 1.

Open in a new tab

Displays the adapted clinician well-being and resilience model. The adapted model shows how nurse practitioner (NP) and practice characteristics can influence NP burnout. EHR = electronic health record.

Study design and data source

This is a secondary analysis of cross-sectional survey data collected from the parent study (Poghosyan et al., 2020) that is described below.

Description of the parent study

Participant and setting.

Researchers for the parent study identified primary care NPs from the SK&A OneKey primary care practice database that contains information on NPs in office-based practices and practices owned by or operated in hospitals (DesRoches et al., 2015). To recruit NPs in primary care practices, researchers obtained data on three types of primary care practices: 1) independent solo practitioner practices in which a physician had a specialization of primary care (i.e., family medicine, general practice, internal medicine, internal medicine/pediatrics, internal medicine/preventive medicine, general preventive medicine, or geriatrics) and at-least one NP worked in the practice; 2) medical group practice sites in which the entire facility specialized in primary care and at least one NP worked in that practice; and 3) medical group practice sites with one or more physician and that encompassed at least 50% of physicians having the specialty of “primary care” and at least one NP worked in that practice. Because the number of NPs in larger states (i.e., California, Pennsylvania, and Texas) is greater than the number of NPs in smaller states (i.e., New Jersey, Arizona, and Washington), a random sampling of NPs was conducted to allow for similar numbers of NPs across states.

Data collection.

From 2018 to 2019, primary care NPs completed surveys containing measures of their use of the EHR, burnout, demographics, and characteristics of their main practice site. To collect data, researchers used a modified Dillman approach for mixed-mode surveys (Dillman et al., 2014), in which the mailings contained an online survey link to allow NPs to complete the survey electronically or on paper. In total, three mailed questionnaires were sent to NPs followed by two rounds of reminder postcards to nonresponding NPs two weeks after the first and second mailings. To obtain a higher overall NP response rate, follow-up phone calls were made to the nonresponding NPs. In these calls, some NPs were deemed ineligible (e.g., never worked or no longer worked in the practice) and others were labeled as “unknown” because researchers could not contact the NP despite three attempts to verify eligibility. In total, 1,244 NPs from six states completed the survey. However, the response rate was determined using three distinctive scenarios. If we assume that all of the “unknown,” nonresponding NPs were eligible for our study, the most conservative response rate would be 22.2%. If we assume that approximately half of the NPs who were labeled as “unknown” in the follow-up calls were ineligible, the response rate would be 25.7%. Finally, if we assume that all of the nonresponding, “unknown” NPs were ineligible, the response rate is 31.8%. We anticipate that the true response rate is between 22.2% and 31.8%.

Current study

We extracted data on all 396 primary care NPs practicing in either New Jersey or Pennsylvania because these states have similar health care markets and rankings for access to care and quality of care (U.S. News & World Reports, 2019a, 2019b). As a result, NPs in these states would be practicing in similar markets. Approval to conduct this study was obtained from the institutional review board of a large university located in the Northeast.

Measures

Use of multifunctional electronic health records.

The EHR subscale comprised two validated survey items (i.e., computerized capabilities and electronic care reminder systems) that were used to measure use of multifunctional EHRs (Friedberg et al., 2009). In previous studies investigating use of EHRs in primary care practices, provider responses from these two items (i.e., computerized capabilities and electronic care reminder systems) were combined to create an overall provider-level EHR subscale score (Friedberg et al., 2009; Martsolf et al., 2018). Below is a description of how this EHR subscale score was created.

The first item measures five computerized capabilities (i.e., recording patient history and demographics, clinical notes, patient’s medications and allergies, ordering prescriptions, and viewing laboratory or imaging results) within the EHR, and NPs responded if each capability was present in their practice. Nurse practitioners responded either “yes” or “no” to each capability. The second item measures the presence of five electronic care reminder systems for decision support of clinical guidelines (i.e., asthma/chronic obstructive pulmonary disease, cardiovascular disease, hypertension, congestive heart failure, and diabetes) by asking NPs whether each reminder system is present in their practice. Participants responded “yes” or “no” to each type of reminder system. For both of these items, a “yes” score was coded as one and a “no” score was coded as zero. Consistent with previous studies using these EHR items (Friedberg et al., 2009; Martsolf et al., 2018), responses from both EHR items were combined to create a single EHR subscale. A practice-level EHR score was calculated by averaging the individual EHR subscale responses from NPs in the same practice. Thus, using a sample of primary care NPs from New Jersey and Pennsylvania, the EHR subscale had a Cronbach alpha of 0.86.

Burnout.

Burnout was measured using a non-proprietary, single item that has been widely used across several studies (Dolan et al., 2015; Edwards et al., 2018; Helfrich et al., 2014; Helfrich et al., 2017). This measure has been validated in a national sample of NPs, primary care physicians, physician assistants, registered nurses, medical technicians, administrative clerks, and licensed practical nurses where this nonproprietary single item burnout measure was compared against a commonly used, stand-alone item from the emotional exhaustion subscale of the Maslach Burnout Inventory (MBI:EE) (Dolan et al., 2015). The nonproprietary single item measure is correlated with the stand-alone single item from the MBI:EE and has sensitivity and specificity greater than 80% and an area under the receiver operator curve of 0.93, indicating that this is an appropriate, less burdensome, cost-free, and reliable substitute to the standalone item in the MBI:EE (Dolan et al., 2015). In addition, researchers found a closely related prevalence of burnout when using the single item extracted from the MBI:EE (36.7%) compared with the nonproprietary singleitem measure (38.5%) (Dolan et al., 2015). In this study, NPs rated their overall level of burnout from 1 (I enjoy my work. I have no symptoms of burnout) to 5 (I feel completely burned out and often wonder if I can go on), with higher scores indicating greater burnout. In accordance with previous published studies, burnout responses were dichotomized so scores 3 to 5 were combined to indicate “burnout,” and scores 1 to 2 combined to indicate “no burnout” (Edwards et al., 2018; Helfrich et al., 2014).

Individual nurse practitioner and practice level covariates.

We obtained variables measuring characteristics of NPs including their race, age, sex, education, marital status, years employed in current practice, number of hours worked per week, workload, and years of NP experience, and variables measuring characteristics of the primary care practice including size, weekend and night hours of operation, and type of primary care practice.

Statistical analysis

Using SPSS statistical software (IBM Corp, 2017), we cleaned, coded, and removed outlier variables from the data set and used boxplots to check the distribution of scores for continuous variables. Variance inflation factor (VIF) statistics were used to detect the presence of multicollinearity. The absence of multicollinearity was determined when VIF was less than five (Akinwande et al., 2015).

Multiple imputation

Because up to 12% of the data were missing and the presence of missing data can limit the statistical power and lead to biased estimates (Kang, 2013), we conducted multiple imputation analyses. Multiple imputation is methodologically rigorous compared with other approaches, such as mean substitution, because each missing observation is replaced with a set of simulated potential values (McCleary, 2002; Murray, 2018). We conducted multiple imputation analyses using 10 simulated versions of the data set that was determined using the percentage of missing data (Bodner, 2008). From these 10 simulated versions, we merged and adjusted the obtained coefficients and standard errors for missing data (McCleary, 2002; Murray, 2018). This pooled, estimated data set was extracted and imported into STATA statistical software (StataCorp, 2015) for further analyses.

Multilevel analyses

To test whether the use of multifunctional EHRs predicted NP burnout, we ran a multilevel cox regression model with time as a constant variable to estimate the risk ratio. This model fits best for our analyses because the prevalence of NP burnout was greater than 10%, and when that occurs the odds ratios in logistic regression models can overestimate the results (Diaz-Quijano, 2012). Cox regression models identify the hazard ratio (Stare & Maucort-Boulch, 2016). In our study, the use of a constant time variable allows the hazard ratio to be comparable with the risk ratio (Diaz-Quijano, 2012). Therefore, we use the term “risk ratio” instead of “hazard ratio” throughout this study. In addition to reporting the risk ratio, we reported the 95% confidence interval (CI) and the p-values to show the strength of relationship between the use of multifunctional EHRs and burnout. The multilevel cox regression model contained variables measuring characteristics of NPs within level one of the models, including age, sex, race, education, marital status, years of NP experience, workload, hours worked per week, and survey modality (online or mail). The outcome variable, NP burnout, was also a level one variable because burnout is a unique feeling experienced by the individual NP. Variables measuring use of multifunctional EHRs, main practice site, weekend hours of operation, evening hours of operation, and practice size were practice-level variables within level two of the model. State was included as a fixed-effects variable.

Results

Overall, 396 NPs were included in the study (Table 1, Supplemental Digital Content 1, http://links.lww.com/JAANP/A110). Most NPs were from Pennsylvania (72.5%) and the rest were from New Jersey. The average age of NPs was 49.5 years (SD = 12.0 years), 90.4% were women, 89.4% White, 87.6% had a Master’s degree as their highest educational degree, and 79.5% were married. Nurse practitioners were employed in their current practice for an average of 6.4 years (SD = 6.3 years). Over 60% of NPs reported working in a physician owned practice. In addition, NPs reported that most of their workload consisted of providing direct patient care (average = 31.1 hours [SD = 10.1 hours]), followed by coordinating patient care (average = 5.6 hours [SD = 6.0 hours]). Nearly 25% of primary care NPs reported burnout.

Almost all primary care practices had computerized capabilities (i.e., recording patient history and demographics, clinical notes, patient’s medications and allergies, ordering prescriptions, and viewing laboratory or imaging results). However, the presence of electronic care reminder systems for decision support of clinical guidelines varied among practices with only 51.3% of NPs reported that there were electronic care reminder systems for patients with congestive heart failure, 54.8% for asthma/chronic obstructive pulmonary disease, 58.1% for cardiovascular disease, 59.1% for hypertension, and 65.2% for diabetes. The average practice-level score for use of multifunctional EHRs was 0.76 (SD = 0.24).

Multicollinearity was not detected in the final model (VIF = 1.70). In the multilevel cox regression model, a significant negative association was found between use of multifunctional EHRs and the risk of NP burnout (risk ratio = 0.30, 95% CI = 0.13–0.71, p = .01). The use of multifunctional EHRs was associated with 70% lower risk of NP burnout, after controlling for the provider- and practice-level characteristics (Table 1).

Table 1.

Multilevel cox regression model, with time as a constant variable, assessing the effect of use of multifunctional EHRs on NP burnout while controlling for provider and practice level covariates (N = 396)

Risk Ratioa p > ∣z∣ 95% CI
Use of multifunctional EHRs 0.3 0.01 0.13 0.71
Average hours worked/week 1 0.82 0.98 1.03
Age 0.99 0.41 0.97 1.01
Female 1.21 0.61 0.58 2.51
Marital status—married 0.8 0.34 0.50 1.27
Education
 Master’s degree 3.15 0.27 0.42 23.92
 Doctorate of nursing practice 2.83 0.34 0.34 23.98
 Other degree 3.04 0.38 0.26 35.8
Race—White 1.31 0.49 0.61 2.83
Years of experience as an NP 1.01 0.65 0.98 1.04
Workloadb
 Providing patient care 1.01 0.3 0.99 1.04
 Coordinating patient care 1.01 0.66 0.97 1.05
 Providing care management services 0.99 0.81 0.92 1.07
 Performing quality improvement 1.02 0.61 0.94 1.1
 Practice leadership and administrative tasks 1 0.97 0.94 1.07
Type of practice—physician practice 0.91 0.65 0.60 1.38
Open during the weekends 0.89 0.66 0.54 1.47
Open at night 0.94 0.41 0.82 1.08
Practice size 1 0.45 0.99 1.03
Survey type 1.11 0.68 0.69 1.77
State 0.8 0.33 0.50 1.26
Open in a new tab

Note: CI = confidence interval; EHR = electronic health record; NP = nurse practitioner.

a

Because we did not have NP responses recorded over various time points, we created a constant time variable. With time as a constant variable, the hazard ratio is the same as the risk ratio. Thus, we reported the risk ratio in this table.

b

Workload = measured as hours per week performing the following tasks.

Discussion

In this study, we investigated the association between use of multifunctional EHRs and primary care NP burnout. Although we found that nearly a quarter of primary care NPs from New Jersey and Pennsylvania are experiencing burnout, the use of multifunctional EHRs did not increase NP burnout. Instead, it seems that use of multifunctional EHRs (computerized capabilities and electronic care reminder systems) is associated with 70% lower risk of NP burnout.

Our findings are contrary to findings reported by other researchers who suggest that greater use of EHRs is associated with physician burnout (Babbott et al., 2014; Robertson et al., 2017). The differences in our findings may partly be because we collected survey data from 2018 to 2019, which is recent, and so primary care NPs may be in practices that are better equipped with using the EHR and practices may also have providers who are familiar with the EHR because EHRs were mandated nearly 11 years ago (Gordon et al., 2015). Furthermore, some advantages of EHRs, such as reliable prescribing, complete documentation that could be shared with other clinicians, and improved coordination of health care services, may assist NPs in providing patient care (Office of the National Coordinator for Health Information Technology, 2019), thereby reducing their workload and potentially reducing burnout. Because it is unknown whether these specific features of the EHR serve as a protective mechanism against NP burnout, future research studies examining the relationships between other components of the EHR and NP burnout are recommended.

Although in this study, the use of multifunctional EHRs is operationalized by the presence of computerized capabilities and electronic reminder systems both of which are not associated with increased NP burnout, there may still be other components of the EHR, which are not investigated in this study, contributing to NP burnout. For example, we did not investigate how much time NPs spend on the EHR and whether more time spent on the EHR is associated with NP burnout, but researchers have found that increased time spent on the EHR contributes to physician burnout (Arndt et al., 2017). Thus, we cannot conclude that the EHR as a whole reduces NP burnout, but we can only conclude that two features of the EHR (i.e., computerized capabilities and electronic reminder systems) are not associated with NP burnout. In addition, with nearly 90% of NPs reporting that they had EHRs with computerized capabilities, therefore, our results are more reflective of the impact of electronic reminder systems. Our results might suggest that NPs using more reminder systems, which assist in decision support of clinical guidelines, experience lower levels of burnout, and this finding should be further explored.

Moreover, we found that personal characteristics of NPs such as their age, race, sex, education, and marital status are not associated with burnout. Our findings are consistent with researchers who report that the fundamental causes of burnout are not personal characteristics of the individual but instead lie within the organization (Dillon et al., 2020; Harrison et al., 2017; Moss, 2019; Sillero & Zabalegui, 2018). For example, these researchers have suggested that unmanageable and increasing workloads associated with the EHR, unfair treatment of colleagues, lack of role clarity, unsupportive supervisors, and a workplace culture focused on productivity are just some features within the organization that can lead to burnout (Dillon et al., 2020; Moss, 2019; Sillero & Zabalegui, 2018). Therefore, future interventions seeking to improve organizational working conditions (e.g., reducing high workloads stemming from the EHR, improving team communication, and working with supportive colleagues) may be more fruitful in reducing burnout compared with individual, personal interventions (e.g., yoga or meditation); however, more research is needed to verify which interventions are best.

Limitations

The use of cross-sectional survey data limits our ability to determine a causal relationship between the use of multifunctional EHRs and NP burnout. In addition, one EHR variable (computerized capabilities) in this study had limited variability. With most NPs reporting that their practice had computerized capabilities, it is possible that the computerized capability items were features required to be in the EHR. Another limitation within this study is the absence of important EHR covariates such as NP training and experience with the EHR, availability of support with the EHR, and usability and functionality of the EHR that were not present in the parent study. Researchers have found that these variables can influence provider satisfaction with the EHR (Yen & Bakken, 2012), thus omitting them may bias our results. The generalizability of our findings should be used with caution because our sample originated only in New Jersey and Pennsylvania; therefore, our results are not applicable to NPs in other states.

The survey response rate is low. Although one may speculate whether the NPs in our survey are different from the overall NP population, we found that NPs in our study are relatively similar in race (89.4% White vs. 87% White nationally) and sex (90.4% female vs. 91.7% female nationally) compared with NPs from the National Nurse Practitioner Sample Survey (AANP, 2019). Despite the limitations, our results may provide new information on primary care NP use of multifunctional EHRs and their burnout. Knowing how primary care NPs are using the EHR can help in the development of future studies examining what features of the EHR help NPs to deliver coordinated primary care to patients.

Conclusion

To our knowledge, this is one of the first studies investigating primary care NPs use of multifunctional EHRs and their experiences of burnout. Although 25% of NPs reported experiencing burnout, the use of multifunctional EHRs was not associated with NP burnout. Although our findings are contrary to previous studies on EHR use and physician burnout, it may be that the NPs in our study became more adjusted and familiar with operating EHRs since the passage of the Health Information Technology Act in 2009. Nonetheless, additional research studies on NP burnout not limited to two states and with robust EHR specific covariates are needed to further examine root sources of burnout and the usability and operability of the EHR in primary care settings.

Supplementary Material

Supplementary Table

Acknowledgments:

The authors thank all the nurse practitioners who completed the survey.

Funding:

This study was supported by grant number R36HS027290 from the Agency for Healthcare Research and Quality, as well as from the Robert Wood Johnson Foundation, the Jonas Center for Nursing and Veterans Healthcare, and grant number R01MD011514 from the National Institute on Minority Health and Health Disparities. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality, the Robert Wood Johnson Foundation, the Jonas Center for Nursing and Veterans Healthcare, nor the National Institute on Minority Health and Health Disparities.

Footnotes

Disclaimers/Previous Presentation: This study was accepted as a poster presentation at Academy Health’s 2020 Annual Research Meeting.

Competing interests: The authors report no conflicts of interest.

References

  1. AANP. (2019). The state of the nurse practitioner profession: Results from the national nurse practitioner sample survey. https://www.aanp.org/news-feed/nurse-practitioner-role-continues-to-grow-to-meet-primary-care-provider-shortages-and-patient-demands.
  2. AANP. (2020a). NP fact sheet. https://www.aanp.org/about/all-about-nps/np-fact-sheet.
  3. AANP. (2020b). What’s a nurse practitioner (NP)? https://www.aanp.org/about/all-about-nps/whats-a-nurse-practitioner.
  4. Abraham CM, Zheng K, & Poghosyan L (2020). Predictors and outcomes of burnout among primary care providers in the United States: A systematic review. Medical Care Research and Review, 77, 387–401. [DOI] [PubMed] [Google Scholar]
  5. Akinwande MO, Dikko HG, & Samson A (2015). Variance inflation factor: As a condition for the inclusion of suppressor variable (s) in regression analysis. Open Journal of Statistics, 5, 754. [Google Scholar]
  6. Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, Sinsky CA, & Gilchrist VJ (2017). Tethered to the EHR: Primary care physician workload assessment using EHR event log data and time-motion observations. The Annals of Family Medicine, 15, 419–426. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Babbott S, Manwell LB, Brown R, Montague E, Williams E, Schwartz M, Hess E, & Linzer M (2014). Electronic medical records and physician stress in primary care: Results from the MEMO study. Journal of the American Medical Informatics Association: JAMIA, 21, e100–e106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Barnes H, Richards MR, McHugh MD, & Martsolf G (2018). Rural and nonrural primary care physician practices increasingly rely on nurse practitioners. Health Affairs, 37, 908–914. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bodner TE (2008). What improves with increased missing data imputations? Structural Equation Modeling: A Multidisciplinary Journal, 15, 651–675. [Google Scholar]
  10. Brigham T, Barden C, Dopp AL, Hengerer A, Kaplan J, Malone C, Martin M, McHugh M, & Nora LM (2018). A journey to construct an all-encompassing conceptual model of factors affecting clinician well-being and resilience (pp. 1–8). National Academy of Medicine Perspectives. [Google Scholar]
  11. DesRoches CM, Barrett KA, Harvey BE, Kogan R, Reschovsky JD, Landon BE, Casalino LP, Shortell SM, & Rich EC (2015). The results are only as good as the sample: Assessing three national physician sampling frames. Journal of General Internal Medicine, 30(Suppl 3), S595–S601. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Diaz-Quijano FA (2012). A simple method for estimating relative risk using logistic regression. BMC Medical Research Methodology, 12, 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dillman DA, Smyth JD, & Christian LM (2014). Internet, mail, and mixed-mode surveys: The tailored design method. John Wiley&Sons. [Google Scholar]
  14. Dillon EC, Tai-Seale M, Meehan A, Martin V, Nordgren R, Lee T, Nauenberg T, & Frosch DL (2020). Frontline perspectives on physician burnout and strategies to improve well-being: Interviews with physicians and health system leaders. Journal of General Internal Medicine, 35, 261–267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dolan ED, Mohr D, Lempa M, Joos S, Fihn SD, Nelson KM, & Helfrich CD (2015). Using a single item to measure burnout in primary care staff: A psychometric evaluation. Journal of General Internal Medicine, 30, 582–587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Edwards ST, Marino M, Balasubramanian BA, Solberg LI, Valenzuela S, Springer R, Stange KC, Miller WL, Kottke TE, Perry CK, Ono S, & Cohen DJ (2018). Burnout among physicians, advanced practice clinicians and staff in smaller primary care practices. Journal of General Internal Medicine, 33, 2138–2146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Friedberg MW, Coltin KL, Safran DG, Dresser M, Zaslavsky AM, & Schneider EC (2009). Associations between structural capabilities of primary care practices and performance on selected quality measures. Annals of Internal Medicine, 151, 456–463. [DOI] [PubMed] [Google Scholar]
  18. Gordon SH, Baier RR, & Gardner RL (2015). Primary care physicians’ use of electronic health records in Rhode Island: 2009–2014. Rhode Island Medical Journal (2013), 98, 29. [PMC free article] [PubMed] [Google Scholar]
  19. Harrison KL, Dzeng E, Ritchie CS, Shanafelt TD, Kamal AH, Bull JH, Tilburt JC, & Swetz KM (2017). Addressing palliative care clinician burnout in organizations: A workforce necessity, an ethical imperative. Journal of Pain and Symptom Management, 53, 1091–1096. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Helfrich CD, Dolan ED, Simonetti J, Reid RJ, Joos S, Wakefield BJ, Schectman G, Stark R, Fihn SD, Harvey HB, & Nelson K (2014). Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. Journal of General Internal Medicine, 29(Suppl 2), S659–S666. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Helfrich CD, Simonetti JA, Clinton WL, Wood GB, Taylor L, Schectman G, Stark R, Rubenstein LV, Fihn SD, & Nelson KM (2017). The association of team-specific workload and staffing with odds of burnout among VA primary care team members. Journal of General Internal Medicine, 32, 760–766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Howe JL, Adams KT, Hettinger AZ, & Ratwani RM (2018). Electronic health record usability issues and potential contribution to patient harm. JAMA, 319, 1276–1278. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Huang MZ, Gibson CJ, & Terry AL (2018). Measuring electronic health record use in primary care: A scoping review. Applied Clinical Informatics, 9, 15–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. IBM Corp. (2017). IBM SPSS Statistics for Windows, Version 25.0. IBM Corp. [Google Scholar]
  25. Kang H (2013). The prevention and handling of the missing data. Korean Journal of Anesthesiology, 64, 402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Krisberg K (2018). Concerns grow about burnout, stress in health care workers: New demands adding to burden. The Nation’s Health, 48, 1–15. [Google Scholar]
  27. Linzer M, Manwell LB, Williams ES, Bobula JA, Brown RL, Varkey AB, Man B, McMurray JE, Maguire A, Horner-Ibler B, Schwartz MD, & MEMO (Minimizing Error, Maximizing Outcome) Investigators. (2009). working conditions in primary care: Physician reactions and care quality. Annals of Internal Medicine, 151, 28–36, W6–W9. [DOI] [PubMed] [Google Scholar]
  28. Martsolf GR, Ashwood S, Friedberg MW, & Rodriguez HP (2018). Linking structural capabilities and workplace climate in community health centers. Inquiry: The Journal of Health Care Organization, Provision, and Financing, 55, 46958018794542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Maslach C, & Jackson SE (1981). The measurement of experienced burnout. Journal of Occupational Behaviour, 2, 99–113. [Google Scholar]
  30. McCleary L (2002). Using multiple imputation for analysis of incomplete data in clinical research. Nursing Research, 51, 339–343. [DOI] [PubMed] [Google Scholar]
  31. Moss J (2019). Burnout is about your workplace, not your people. Harvard Business Review. https://hbr.org/2019/12/burnout-is-about-your-workplace-not-your-people. [Google Scholar]
  32. Murray JS (2018). Multiple imputation: A review of practical and theoretical findings. Statistical Science, 33, 142–159. [Google Scholar]
  33. Office of the National Coordinator for Health Information Technology. (2019). What are the advantages of electronic health records? https://www.healthit.gov/faq/what-are-advantages-electronic-health-records.
  34. Panagioti M, Geraghty K, Johnson J, Zhou A, Panagopoulou E, Chew-Graham C, Peters D, Hodkinson A, Riley R, & Esmail A (2018). Association between physician burnout and patient safety, professionalism, and patient satisfaction: A systematic review and meta-analysis. JAMA Internal Medicine, 178, 1317–1330. [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
  35. Patel RS, Bachu R, Adikey A, Malik M, & Shah M (2018). Factors related to physician burnout and its consequences: A review. Behavioral Sciences, 8, 98. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Poghosyan L, Ghaffari A, Liu J, & Martsolf GR (2020). Primary care nurse practitioners’ roles and work environment in six states with variable scope of practice regulations. Unpublished manuscript. [Google Scholar]
  37. Rabatin J, Williams E, Baier Manwell L, Schwartz MD, Brown RL, & Linzer M (2016). Predictors and outcomes of burnout in primary care physicians. Journal of Primary Care & Community Health, 7, 41–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Robertson SL, Robinson MD, & Reid A (2017). Electronic health record effects on work-life balance and burnout within the I3 population collaborative. Journal of Graduate Medical Education, 9, 479–484. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Shanafelt TD, Dyrbye LN, Sinsky C, Hasan O, Satele D, Sloan J, & West CP (2016). Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clinic Proceedings, 91, 836–848. [DOI] [PubMed] [Google Scholar]
  40. Sillero A, & Zabalegui A (2018). Organizational factors and burnout of perioperative nurses. Clinical Practice and Epidemiology in Mental Health, 14, 132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, Westbrook J, Tutty M, & Blike G (2016). Allocation of physician time in ambulatory practice: A time and motion study in 4 specialties. Annals of Internal Medicine, 165, 753–760. [DOI] [PubMed] [Google Scholar]
  42. Stare J, & Maucort-Boulch D (2016). Odds ratio, hazard ratio and relative risk. Metodoloskizvezki, 13, 59. [Google Scholar]
  43. StataCorp LP. (2015). STATA statistical software: Release 14. StataCorp LP. [Google Scholar]
  44. U.S. Department of Health and Human Services, Health Resources and Services Administration. (2016). National and regional projections of supply and demand for primary care practitioners: 2013–2025. https://bhw.hrsa.gov/sites/default/files/bhw/health-workforce-analysis/research/projections/primary-care-national-projections2013-2025.pdf.
  45. U.S. News and World Reports. (2019a). Health care access rankings. https://www.usnews.com/news/best-states/rankings/healthcare/healthcare-access.
  46. U.S. News and World Reports. (2019b). Health care quality rankings. https://www.usnews.com/news/best-states/rankings/healthcare/healthcare-quality.
  47. West CP, Dyrbye LN, & Shanafelt TD (2018). Physician burnout: Contributors, consequences and solutions. Journal of Internal Medicine, 283, 516–529. [DOI] [PubMed] [Google Scholar]
  48. Willard-Grace R, Knox M, Huang B, Hammer H, Kivlahan C, & Grumbach K (2019). Burnout and health care workforce turnover. The Annals of Family Medicine, 17, 36–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Yen PY, & Bakken S (2012). Review of health information technology usability study methodologies. Journal of the American Medical Informatics Association, 19, 413–422. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Table
NIHMS1726347-supplement-Supplementary_Table.docx (23.1KB, docx)

RESOURCES