Explaining physicians' use of EMR systems and performance in the shakedown phase

Abstract

Objective

This work seeks to complement and extend prior work by using a multidisciplinary approach to explain electronic medical records (EMR) system use and consequent performance (here, patient satisfaction) among physicians during early stages of the implementation of an EMR.

Design

This was a quantitative study, with data obtained from three distinct sources: individual-level and social-network data from employees; use data from EMR system logs; and patient satisfaction data from patients and/or authorized decision-makers. Responses were obtained from 151 physicians and 8440 patient satisfaction surveys over the course of a 1-year period at the shakedown phase of an EMR system implementation.

Results

Physicians who were better connected, both directly and indirectly, to their peers—that is, other physicians—for advice on their work, used the system less than those who were less connected. In addition to such social network ties, demographic characteristics (gender and age), three personality characteristics (openness to experience, agreeableness and extroversion) and a key technology perception (perceived usefulness) predicted EMR system use.

Conclusions

For hospital administrators and other stakeholders, understanding the contributors to, and the relative importance of, various factors in explaining EMR system use, and its impact on patient satisfaction is of great importance. The factors identified in this work that influence a physician's use of EMR systems can be used to develop interventions and applications that can increase physician buy-in and use of EMR systems.

Introduction

Electronic medical records (EMR) systems organize and store medical records electronically. The various benefits of using EMR systems that have been identified include: (a) increased speed in patient encounters, documenting treatments and obtaining patient histories; (b) greater security and better access to medical records; and (c) helping pharmacies, specialists and other groups access information quickly and reliably and reducing adverse drug events in in-patient settings and ambulatory settings.^1–3 Just as much as there could be benefits of using EMR systems, there are problems, such as slowing down patient encounters at least in the early stages of implementation.⁴ Despite being available since the 1990s and their potential benefits, EMR systems are not widely used in hospitals and medical practices in the USA³ and are used to varying extents in other countries, including some European countries.⁵ For example, a recent report revealed that 67% of medical offices in the USA with four or more physicians do not use EMR systems.⁶ It is thus important to predict not only physicians' EMR system use but also how such use relates to physicians' performance.

Studies on the success of information systems note that actual system use is the key mechanism to realize IT benefits.⁷ Hence, owing to limited use of EMR systems, their benefits are not reaped. Prior research, especially in the field of medical informatics, has identified some of the barriers to EMR system adoption among physicians.^{3 8} A survey of physicians found that the reasons cited most often for physicians not to adopt an EMR system were: start-up costs, continuing costs and loss of productivity.⁸ While the costs of the technologies are likely to decrease over time, the phase of the implementation when productivity loss occurs is critical to study. This work seeks to use a multidisciplinary approach to explain EMR system use and consequent performance impacts among physicians during the early stages of EMR implementation. The early stage of the implementation, often referred to as a shakedown phase, was chosen because it is when loss in productivity and disruption in processes occur and when technological systems may be abandoned.⁹ The shakedown phase refers to the period from ‘going live’ until ‘routine use’ has been achieved and can typically last anywhere from 6 months to a year. In our context of EMR system adoption and use by physicians, the shakedown phase is likely even more critical, as it sets the tone for their future interactions with the system.

Research model

We present a holistic model (figure 1) to explain physicians' use of EMR systems and consequent performance impacts during the shakedown phase of the system implementation. We consider how physician's EMR system use is affected by the following three sets of factors: individual (physician characteristics), system (physicians' perceptions of the new EMR system), and interactions among physicians (the social influence of other physicians on EMR system use). We also expect EMR system use to affect physician performance.

Figure 1.

Research model. EMR, electronic medical records.

We leverage psychology research to examine the influence of physicians' traits, both demographics and personality, on physicians' EMR use. We also draw on research in health informatics and management information systems (MIS), and include physicians' perceptions about the new EMR system. Finally, we draw on sociology research, particularly social-networks research, as interpersonal interactions are crucial in the case of physicians' work in a hospital setting, and can be expected to influence physicians' EMR use.

Psychology research has widely used individual's demographic and personality characteristics as predictors of human behavior.^{10 11} Demographic variables, namely gender and age, have been shown to influence the use of new technologies.¹² The Big-5 personality traits inventory has been shown to influence a variety of behaviors. The traits are neuroticism (sometimes referred to as emotional instability, this expresses the tendency of an individual to experience negative emotions); conscientiousness (the trait exemplified by individuals who are painstaking and careful, and have a need for achievement); agreeableness (relating to getting along with others); openness to new experiences (categorized by imagination, curiosity, and a drive to seek out new experiences); and extraversion (a trait regarding how an individual relates to others).¹³

We draw on the health informatics and MIS research to identify two technology perceptions that should explain physicians' EMR system use: perceived usefulness (defined as the extent to which an individual believes using an EMR system can enhance their job performance) and perceived ease of use (defined as the extent to which an individual perceives using an EMR system will be free of effort).^{12 14}

A physician's workplace setting involves significant interactions with fellow physicians for a variety of matters, ranging from job-related questions to social support to general socializing. The importance of such interactions is underscored by the high stress nature of a physician's job. Therefore, a social network lens, which focuses on individual interactions within a workplace or social context, can serve as an important basis to explain behavior.¹⁵ Of the variety of social networks, we focus on advice networks that describe the nature and flow of work-related information among employees.¹⁶ Centrality is one of the most important constructs in the context of advice networks, as it represents both power and influence wielded and experienced as well as access to resources.¹⁷ Direct ties are between the focal individual and people they interact with directly, and second-order (or indirect) ties are those of individuals that interact with one's direct ties. An example of second-order ties is thus advisors of an advisor.¹⁷

Behavior so determined typically has performance impacts. Use of systems, for example, EMR systems, is typically expected to influence performance positively.^{18 19} In this case, EMR system use can be expected to contribute positively to physicians' performance.³ While there are several metrics of physicians' performance, we focus on patient satisfaction, defined as the extent to which a patient or a patient's authorized decision-maker is pleased with the overall medical care received,²⁰ which has been shown to relate to the technical quality, communication, time spent, and related aspects of the care.²¹

The online appendix (available at www.jamia.org) details the rationale underlying why each of the different factors is expected to influence EMR system use.

Method

Context, system, and participants

The study was conducted in a private hospital that was implementing a web-based enterprise-wide healthcare solution that included a component for electronic medical records. The institutional review board approval for the use of human subjects included the study of technology use in a variety of settings. The implemented EMR system had the following modules: CPOE, encounter information recording, decision support, support for all eight of the departments in the hospital (via specialty modules), compliance with physicians' quality reporting initiative (PQRI), and access to records for patients. Further, the system included advanced security features to protect data and authenticate access. This EMR system was part of the solution available to the physicians and is the focus of our study. The hospital, which had about 800 beds, provided a variety of healthcare services, including emergency care. We were provided with a list of all employees in the hospital and their contact information. The list of physicians, that is, the sampling frame, showed 244 physicians working at the hospital, with about 20% of them being contracted (1–2 days per week) as needed. For performance assessments, independent patient surveys were conducted by the hospital and mapped to each physician.

We excluded the contracted-as-needed physicians from our analysis as they had minimal interaction with other physicians and knew very few of the full-time physicians. Of the 200 full-time physicians surveyed, 151 returned complete and usable responses for a response rate just over 75%. In order to assess non-response bias, we compared the respondent demographic profile with the non-respondent profile and found them to be statistically equivalent.

All patients received a survey to assess their satisfaction with the care they received at the hospital. Although the exact number of surveys mailed was not shared with us, a total of 8440 patient and/or authorized decision-maker responses, which rated physicians, nurses, and administrative personnel (our survey only includes physicians' data), were received. Each physician had, on average, about 40 patients providing an evaluation in the 9-month period following the introduction of the EMR system. This allowed us to obtain an overall average (mean) rating for each physician.

We conducted a pilot study in a small medical practice implementing an EMR system. This was done because we did not want to taint the actual sample prior to data collection. Also, given that it was important to collect data from all members in the network, we could not use a subsample of the doctors in the hospital. The pilot study results revealed our multi-item scales to be reliable and valid.

Procedure

The timeline for the major activities and data collected is shown in table 1. Our questionnaire was combined with the questionnaire being administered by the hospital. The hospital hired an independent research firm to collect the data. Such an approach helped employees to be comfortable with sharing information honestly and helped ensure the privacy and confidentiality of responses. The hospital administration emphasized to the physicians that the survey was a critical forum for input related to the EMR system in particular and the entire IT solution in general.

Table 1.

Data-collection timeline

T0: pretraining	T1: month 1	T2: months 2–3	T3: months 4–12
Archival measures of preimplementation patient satisfaction	Training takes place over a 1-month period	Electronic medical records system is installed and available on all computers	Use over this period is obtained from system logs
	Survey of physicians to collect individual characteristics, technology perceptions, and social-network data		Archival measures of postimplementation patient satisfaction

Measures

The online appendix (available at www.jamia.org) provides the questions used in the survey that are specific to the paper. It also supplies greater detail about the measures used.

Survey of physicians

The physicians were given the option to complete the survey either online or on paper. As our survey questions were embedded within a larger survey employed by the hospital, there were many other questions that could be considered as filler questions, thus reducing demand characteristics. Further, since the dependent variables were coming from different sources, biases are further reduced. The survey of physicians measured various individual characteristics—that is, demographics and personality—technology perceptions and social network ties. The social network data, focused on advice networks, were gathered using a roster-based approach.^{22 23} While there are many centrality measures, we used a form of degree centrality quantifying each physician's first-degree (direct connections) and second-degree (indirect connections or the connections of their direct connections).

Archival logs of EMR system use

EMR system use was measured using archival system logs. As the system logged out idle users after a specified 10 min, biases due to inflation of use were minimized.

Survey of patients

Overall patient satisfaction was measured by adapting the Patient Satisfaction Questionnaire III (PSQ III). One of the key differences in the way this particular hospital used the PSQ III was to gather data about each physician who dealt with a patient. The patient and/or authorized decision-maker responded to various questions based on their experiences with each physician. The mean score of overall patient satisfaction was calculated. Thus if a physician received ratings from 50 patients or authorized decision-makers, the average (mean) on overall patient satisfaction provided the performance metric for the particular physician.

Results

We analyzed these data using partial least squares (PLS). PLS allows for the use of formative indicators²⁴ without imposing constraints on model specification and for the violation of distributional assumptions made in OLS regressions.²⁵ SmartPLS was the tool used to estimate the measurement and structural models.²⁶ The descriptive statistics, correlations, reliabilities and average variance extracted (AVE) are reported in table 2. The results provide evidence of the reliability and validity of the measures: all multi-item scales had good reliabilities (ICR >0.70) and had high loadings (>0.70) and low cross-loadings (<0.35), and the square roots of the AVEs were greater than the correlations. It should be noted that neuroticism was fairly low, and conscientiousness was very high, and both exhibited very low standard deviations—both of these findings are not particularly surprising and likely due to the context, as physicians are expected to exhibit high conscientious and low neuroticism. Most correlations exhibited the expected pattern, with most individual characteristics relating significantly to EMR system use, network centrality (direct and indirect) relating negatively to EMR system use, and both centralities (direct and indirect) and EMR system use relating positively to patient satisfaction.

Table 2.

Descriptive statistics, correlations, internal consistency reliabilities, and average variances extracted

	Internal consistency reliability	Mean	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1. Gender (1: men)	NA	NA	NA	NA
2. Age	NA	41.20	14.40	0.20***	NA
3. Neuroticism	0.71	2.20	0.40	0.04	0.04	NA
4. Conscientiousness	0.75	6.20	0.38	0.05	0.13*	0.04	0.75
5. Agreeableness	0.73	3.28	1.13	−0.12*	−0.12*	0.05	0.08	0.73
6. Openness to experience	0.70	3.51	1.20	0.14*	−0.15*	0.00	−0.07	−0.12*	0.75
7. Extraversion	0.71	4.13	1.14	−0.13*	−0.10	0.03	−0.13*	−0.07	0.12*	0.78
8. Perceived usefulness	0.87	4.10	1.31	0.16**	−0.16**	0.01	0.10	0.04	0.19**	0.08	0.83
9. Perceived ease of use	0.85	3.87	1.51	0.15*	−0.16**	0.04	0.07	0.03	0.15*	0.10	0.19**	0.82
10. Preimplementation performance	0.75	4.49	1.29	0.07	0.23***	0.02	0.06	0.08	0.17**	0.14*	0.07	0.01	0.71
11. Centrality (direct)	NA	32.30	21.60	−0.17**	0.19**	−0.02	0.09	0.15*	0.21***	0.24***	−0.22***	−0.15*	0.28***	NA
12. Centrality (indirect)	NA	21.22	14.20	−0.19**	0.14*	−0.01	0.04	0.12*	0.17**	0.20***	−0.19***	−0.12*	0.26***	0.33***	NA
13. Electronic medical records system use	NA	7.35	4.20	0.13*	−0.24***	−0.04	0.04	0.18**	0.20***	−0.22***	0.23***	0.16*	0.13*	−0.39***	−0.34***	NA
14. Postimplementation performance	0.80	5.08	1.01	0.08	0.21***	0.01	0.09	0.12*	0.17**	0.17**	0.13*	0.10	0.32***	0.30***	0.25***	0.29***	0.78

*p<0.05; **p<0.01; ***p<0.001.

Diagonal values are the square roots of average variances extracted.

The results of our structural model are shown in figure 2. Individual characteristics, technology perceptions, and social network centralities together accounted for 40% of the variance in physicians' EMR system use. Gender, age and three (of the five) personality characteristics—that is, agreeableness, openness to experience, and extraversion—predicted EMR system use. The non-significance of two predictors, that is, neuroticism and conscientiousness, was likely due to the low SD of these two constructs, as noted earlier. Perceived usefulness was a significant predictor of EMR system use. Although perceived ease of use was positively correlated with EMR system use, it was not a significant predictor. This is likely because perceived ease of use is correlated with some constructs, especially use, that are significant predictors of EMR system use. Both first-degree (direct) and second-degree (indirect) centralities negatively influence EMR system use. EMR system use had a positive impact on physician performance in terms of patient satisfaction and, together with preimplementation patient satisfaction, explained 15% of the variance in terms of this measure of physician performance. It is worth noting that it is possible that our sample size could have resulted in some null hypotheses not being rejected. A power analysis suggested that we would have detected medium effects.²⁷

Figure 2.

Results. EMR, electronic medical records.

Discussion

This research offers important contributions to research and practice.

Theoretical contributions

The key contribution of this work is the development of a comprehensive model of EMR system use. By examining factors from psychology, health informatics, MIS, and sociology, we understand their relative importance for EMR system use. It is clear that despite some positive individual characteristics and technology perceptions, the overwhelmingly strongest drivers are physicians' advice ties. Interestingly, the better-connected physicians tend to use the system less. While advice ties typically foster behavioral performance,²⁸ here, better connected physicians avoid using the system, likely due to the pressures and support from their ties. Physicians' social networks provide them with a substitute mechanism to access some of the information that is available via an EMR system and, consequently, results in lower use. Physicians with high centrality (direct or indirect) have greater access to this alternate channel that decreases their need to use an EMR system. Physicians with high centrality are also likely to be subject to greater pressure from peers to conform to prevailing norms and practices. As such, their being subject to such social influence to maintain the status quo is likely to be a deterrent to their use of an EMR system. Thus, a major implication is that EMR systems may not be used to their fullest potential by physicians with high centrality and, worse yet, may even be actively resisted by them. This requires the need for training programs in the shakedown phase of an EMR system implementation to demonstrate the advantages of EMR systems relative to accessing information from physicians' social networks and existing fragmented systems that are in place. It also requires the change management process to recognize that resistance to EMR systems will be greater among physicians with high centrality. As a result, resources and tactics to reduce such resistance should be particularly targeted at these physicians.

Another important contribution of this work is to health informatics. While research on health informatics has argued and presented evidence on the benefits of EMR systems to hospitals and patients, they have also identified physicians' adoption and use of such systems to be slow. By presenting a holistic model of physicians' EMR system use, our work complements prior work. Likewise, this holistic model complements prior work explaining EMR system adoption and use by physicians that has used a single theoretical perspective.^{8 18} Our results provide compelling support for the importance of all three sets of factors—that is, individual, technology-related and social networks—as determinants of EMR system use. Each of these sets of factors represents distinct sources of influence of EMR system use and has different implications for the management of the shakedown phase of EMR system implementation. First, there are distinct individual forces that influence the propensity of physicians' use of EMR systems. These individual factors must be considered both prior to rolling out EMR systems to physicians and also while evaluating the success of a rollout across a group of physicians who are asked to use the system. Second, with respect to technology factors, our study reveals that perceived usefulness is a key factor that influences the physicians' use of EMR systems. Interestingly, perceived ease of use is not a significant predictor of EMR use, suggesting that direct experience and training can be used to overcome the mechanistic hurdles associated with using a system for instrumental purposes. It is also possible that the perceived usefulness of systems deteriorates if they are difficult to use, a viewpoint that is consistent with the literature on technology acceptance²⁹ and with the positive correlations between perceived ease of use and perceived usefulness that we observed in these data. Third, with respect to social networks, our study reveals that the use of EMR systems by physicians is embedded in a social context that is well described by the centrality of their direct and indirect ties to other physicians. The structure of a physician's ties, then, becomes an alternate conduit for some information that is relayed through the EMR system and also for pressure to conform to established norms and practices. Collectively, our findings reveal that a focus on one set of factors and exclusion of the other two will result in an oversimplified view of the relevant forces during the critical shakedown phase.

A third contribution lies in elaborating our understanding about the impact of EMR system use on physician performance, here patient satisfaction. By systematically demonstrating the empirical relationship between EMR system use and patient satisfaction, we provide important evidence regarding the value of such systems. Of course, of concern here from a theoretical standpoint is the countervailing effect of advice network ties. On the one hand, it has a positive, direct effect on patient satisfaction (as evidenced in our post-hoc analysis). It is very likely that physicians with high centrality in advice networks also have access to knowledge from peers that can be instrumental in providing superior care to patients. They are also likely to be positioned effectively to call upon other physicians with questions or to refer patients to physicians with complementary expertise and experiences, both within and outside their own specialty. On the other hand, physicians' advice network ties have a negative effect on performance, here patient satisfaction, when operating through EMR system use. This suggests that while better-connected physicians do tend to perform better (ie, higher patient satisfaction), they do not embrace EMR systems readily. This finding poses an important scientific quandary for future research. In particular, it is important to study interventions to combat these countervailing effects—researchers should focus on interventions that will mitigate the negative forces of advice network ties on EMR system use while supporting the positive forces relating advice ties to physician performance, at least in terms of patient satisfaction. Future work should also study other aspects of the PSQ III instrument that we did not consider such as technical quality and communication. Such a study will provide a richer, more complete understanding of how EMR system use ultimately relates to patient satisfaction. It is possible that some of the other outcomes, for example, technical quality, could mediate the impact of EMR system use on patient satisfaction. More importantly, given the limitations of patient satisfaction as a quality metric, future work should focus on understanding other outcomes, such as medical errors, that are of great interest and importance to healthcare industry.

Finally, we focused on the shakedown phase because it is often the phase in which systems are abandoned, and getting through this phase is obviously important. Likewise, we focused on a specific system with certain features and functionalities. Both of these are contributions and also point to limitations. They are strengths in that the shakedown phase is important to study, and the system has a comprehensive set of features. They are limitations in that other phases of the implementation and other systems with different feature sets need to be studied to understand the boundary conditions of what we have found.

Several additional future research directions are outlined in the online appendix (available at www.jamia.org).

Practical contributions

The most important practical contribution of this work is to alert those implementing EMR systems that the better-connected physicians, who tend to be the better performing physicians in terms of patient satisfaction, will use the EMR systems the least. There are many possible intervention strategies that could be pursued to overcome these potential countervailing effects. First, it may be prudent to seek allies among physicians who are well connected so as to sway popular opinion. These physicians could be targeted to be lead users so that their viewpoints and requirements are represented, to the extent possible, in the functionalities and design of the system. Such a proactive approach can preempt resistance to the system and can even potentially lead to the grafting of opinion leaders and champions³⁰ who advocate the merits of EMR systems. Second, it may be necessary to design the system in ways that are more compatible with the ways that physicians are already trained and already work, rather than asking them to modify how they work. Third, it is possible that more extensive training and support that is not just a one-shot solution, as is the typical case in most hospitals, may be important to help physicians make the transition. Finally, much as junior physicians learn the trade from more senior physicians, in this case, it may be that a ‘buddy system’ of a junior–senior physician team could work well together in helping more senior physicians, who are likely older and likely better connected in terms of the advice network, embrace EMR systems. As such, support networks may need to be established to promote physicians' EMR system use and to overcome the negative influence of advice networks on their EMR system use.

Other types of interventions that could be pursued include far more specific things that could be done as part of the training programs that are provided. One of the potentially most malleable drivers of EMR system use in our model is the positive impact of the perception of usefulness of the EMR system. Likewise, clearly, there is evidence that EMR system use can positively impact patient satisfaction. By designing the training such that the performance benefits are clearly and unequivocally demonstrated, it may be possible to sway physicians to use EMR systems a little more. Therefore, rather than simply turning the training session into a point-and-click training session that teaches what to do and how to do it, there could be a broader emphasis on performance benefits of various features of EMR systems and a discussion of particular case studies and stories of physicians experiencing benefits. Such a case study and story-based training may be particularly suitable and resonate well with physicians, given that medical training itself hinges heavily on such an approach.

Conclusions

This paper contributes to our understanding on physicians' use of EMR systems. Despite the potential benefits of EMR system use in increasing patient satisfaction with the physicians, physicians who were better connected, both directly and indirectly, to their peers—that is, other physicians—used the system less than those who were less connected. In addition to such social network ties, demographic characteristics (gender and age), three personality characteristics (openness to experience, agreeableness, and extroversion), and a key technology perception (perceived usefulness) predicted EMR system use. Thus, the major contribution of this work that complements prior work is the development and test of a holistic model that helps us understand the relative importance of various factors in explaining EMR system use and its impact on patient satisfaction.