Abstract
User satisfaction with an electronic medical record (EMR) plays a decisive role in its implementation and subsequent use. We developed a survey tool to identify features of an EMR that contribute to user satisfaction and administered it in an adult primary care clinic.
Most physician respondents were highly satisfied with the EMR and used all of its components. The EMR decreased the time to develop a synopsis of the patient and improved communication efficiency. Most respondents valued remote access to the EMR. Electronic messaging was an important component of improved care delivery according to 80% of the respondents. Access to online references within the EMR was not valued over web-based access for most respondents.
Our results demonstrate acceptance of an EMR in adult primary care. Features such as remote access and electronic messaging were surprisingly useful and successful for primary care practice.
INTRODUCTION
Implementation of computerized order entry systems and EMR systems has not been universally successful1, and user resistance has been implicated as a primary factor in the failure of some systems2. Advantages of such systems may include improved quality of healthcare1, decreased time spent on paperwork3, increased patient satisfaction4, and financial savings5. Despite these potential advantages, implementation of EMR systems may be resisted if users are not satisfied with the system2. Physicians and nurses have preconceived concerns about security and confidentiality3,6, time incurred by EMR use5, or negative impacts on the quality of patient care3,6.
Prior work has uncovered many factors that may influence EMR user satisfaction. Factors include response time, logical and efficient flow of tasks, ability to complete desired tasks, ease of correcting mistakes, effects on an individual’s time, and proper training on the system 1,2,3,5,7,8,9. The involvement and support of users prior to and throughout the implementation of a new EMR system is also important1,2,10. Systems implemented voluntarily may be received by users better than those forced upon the users2. Prior computer experience and computing skills have also been studied as predictors of EMR acceptance, with contradictory results3,6,7,9,10. Perceived impact on the quality of patient care may be an important factor in user satisfaction with an EMR, especially when the system is perceived as negative2.
Further research exploring the specific aspects of EMR systems that contribute to user satisfaction is necessary to aid successful future development and implementation of effective computerized record systems. With this study, we attempted to define these aspects of user satisfaction in a clinic where an EMR recently had been implemented and subsequently replaced a paper medical record system. We then developed a survey to quantify these endpoints of satisfaction with the physicians and other staff of the clinic.
METHODS
Setting
We conducted this study at an adult ambulatory primary care and urgent care clinic in an academic hospital. The clinic has 25 practicing physicians, with approximately 70 residents rotating through the clinic. The clinic also employs approximately 17 nurses and 30 secretaries. Implementation of the EMR began in 2002 and was completed in just under a year.
The system includes access to all internally generated notes, reports, lab values, and scanned documents. All outside documents and letters are scanned and added to the EMR. An electronic messaging system also is incorporated into the EMR. Messages are delivered to a clinician’s mailbox and may be prioritized by the sender, and also saved to the patient’s permanent chart. Another feature allows users to create reminders about patient results or future needs. Workstations were installed in all exam rooms, nursing stations, and offices. Physicians were given the option to obtain rolling-number security key fobs for remote access to patient records across any internet connection. Current training for those new to the system takes place in the form of scheduled classes, not one-on-one training, though the development team often is available to help with problems. Further refinements and additional features continue to be added to the system, including a note-writer that replaced dictation shortly before our completed survey was administered.
Survey Development
Investigators first conducted semi-structured private interviews with a convenience sample of physicians, nurses, and secretaries in the clinic. Each subject was asked “What do you think about [the EMR]?” Investigators encouraged discussion to identify aspects of the EMR that evoked a positive or negative response from the interviewee. Comments were analyzed and used to construct a framework describing aspects in which the EMR would have an impact on satisfaction. The themes were: accuracy, speed, efficiency, communication, practice of medicine, online tools, availability of the system outside the clinic, and perceived patient satisfaction. Ninety-nine comments were collected, and 85 were assigned a theme. Ten comments did not contribute significantly to satisfaction.
Survey questions were then written to address each of these themes. Survey questions also addressed demographic information, experience in healthcare, experience with computers, experience using the EMR, and participation in the development and implementation of the EMR. The completed survey was tested for face validity with several individuals thought to be representative of the survey population.
Surveys were distributed within the clinic, along with several e-mail solicitations for their return. Surveys were distributed to seventy physicians, with 46 (66%) returned for analysis.
Statistical Analysis
The subscales of total scores were assessed for internal consistency using standardized Cronbach’s alpha coefficients, and removed from total scores if the correlation with total was less than 0.2 and the coefficient with deleted variable increased more than 5%. The total scores are calculated as the weighted sum of complete subscales with the weight equal to the inverse of completeness percentage. Means and standard deviations of total scores, and frequencies and percentages of subscale scores are listed. Spearman’s correlation coefficient was used to study uncontrolled association. The relationships between the multiple responses and satisfaction were studied using analysis of covariance (ANCOVA), adjusted for covariates. The results of fitted models for all pre-specified dependent variables are reported. All statistical comparisons were performed using two-sided tests at the 5% significant level. P values less than 0.05 were considered statistically significant. Data analysis was performed with Statistical Analysis System (SAS), version 8.2.
RESULTS
Respondents represented all ages and genders, with 37% being less than 30 years of age and 54% being males. This group had a median of 5 years in practice and a median of 2.5 years at Vanderbilt at the time of the study. Two respondents had been in practice for more than 35 years, and had been working at the study site for almost 30 years. Eight respondents (17%) saw some pediatric patients and 18 (39%) of respondents saw geriatric patients.
Implementation and EMR Performance
Providers were satisfied with the number (69%) and location (62%) of computers, the reliability of the EMR (62%) and customer support (76%). However, 53% of respondents though the EMR was too slow. Most respondents (76%) believed they and their staff were adequately trained before the system was implemented. They also felt that there was always help available if needed (78%). Satisfaction with implementation was moderately correlated with self-disclosed involvement in development (Spearman’s Rho=0.27, p=0.07), and not correlated with perceived patient satisfaction (Spearman’s Rho=0.08, p=0.62).
EMR Use
The majority of respondents used all of the EMR features listed, including updating medication (81%) and problem (81%) lists after visits, updating problem lists between visits (74%), using the computer in the exam room (87%), and using the “new results” feature (93%). Only 41% of respondents used reminders to follow up on patient issues, and 46% customized the EMR to support the workflow. Computer-based documentation was used by 88% of respondents for more than 2/3 of their clinic notes, and by 93% of respondents to some extent. Gender, computer skill, message basket use and years in practice were not associated with higher levels of acceptance; however, more satisfied respondents reported higher use of the EMR after controlling for the covariates (p=0.03).
Functional Impact (Table 1)
Table 1.
EMR Functional Satisfaction
| Questions (n = 46) | Agree | Neutral/Disagree |
|---|---|---|
| Speed: Cronbach’s Alpha=0.79 | ||
| I can develop a synopsis of a patient faster. | 66% (27) | 34% (14) |
| New results for patients are available to me sooner. | 86% (36) | 14% (6) |
| When a patient calls on the telephone I can answer his/her questions faster. | 93% (37) | 7% (3) |
| Accuracy: Cronbach’s Alpha=0.84 | ||
| Documents are contained in the correct patient’s chart more often. | 73% (30) | 27% (11) |
| Documents are more legible | 88% (35) | 12% (5) |
| Individual patient records are more complete | 68% (28) | 32% (13) |
| Efficiency: Cronbach’s Alpha=0.75 | ||
| It takes less effort to research web-based literature. | 59% (23) | 41% (16) |
| It takes less effort to review a patient’s medical history. | 86% (36) | 14% (6) |
| It takes less effort to communicate with my staff. | 88% (37) | 12% (5) |
| It takes less effort to review records when interpreting lab results | 83% (35) | 17% (7) |
| Outside Access to System: Cronbach’s Alpha=0.84 | ||
| I like the ability to access my message basket while I am away from clinic. | 84% (38) | 16% (7) |
| I like the ability to access new results while I am away from clinic. | 82% (37) | 18% (8) |
| The new system intrudes into my life while I am away from clinic in an unwelcome way. * | 20% (9) | 80% (36) |
| I value the ability to access the system from home. | 80% (36) | 20% (9) |
| The new system makes it easy for me to look up a patient’s past medical history when I am at home. | 83% (38) | 17% (8) |
| Communication: Cronbach’s Alpha=0.76 | ||
| The messaging in Starpanel allows me to respond more quickly to communication with my staff concerning patients | 93% (39) | 7% (3) |
| Sending and receiving messages in my clinic is now more convenient | 90% (37) | 10% (4) |
| When I send a message now, it is available to the intended recipient faster | 88% (37) | 12% (5) |
| There is more effective communication between attendings and residents. ** | 50% (22) | 50% (22) |
Overall Cronbach’s alpha = 0.89.
excluded from outside access to system.
excluded from communication.
Respondents were asked to comment about how the EMR affected their daily activities. The majority of clinicians either agreed or strongly agreed that the EMR resulted in efficiency gains relative to the previous environment with computer-retrievable lab results, but chart-stored notes and other documents. Over 80% of respondents liked having outside access to the system. The use of message baskets improved communication among staff for 93% of the respondents. More than 85% of the respondents believed that messaging within the clinic was a more convenient and faster form of communication. The majority (84%) of respondents liked access to their message basket while away from clinic; however, 20% of all respondents believed that that the remote access capabilities intruded on their life while they were away from clinic.
Respondent’s satisfaction with implementation was positively correlated with their perception about EMR speed (Spearman’s rho = 0.3; p = 0.04), outside access (Spearman’s rho = 0.5; p = 0.002), and EMR efficiency (Spearman’s rho = 0.4; p = 0.008), and not correlated with EMR accuracy or communication. Overall, respondent’s satisfaction with implementation was positively correlated with their perception about EMR functional impact (p=0.02). Furthermore, females and respondents with more computer skills tended to be more satisfied with EMR functional impact (p=0.02 and p=0.01 respectively).
Access to online Tools
While most respondents (68%) agreed or strongly agreed that access to online information through the EMR gave value to the EMR, the respondents varied widely in their perception about how the EMR impacted awareness of knowledge sources or patient education.
Satisfied respondents reported more benefit from EMR access to online tools (p=0.006). More computer literate respondents reported less benefit from EMR online tool access (p=0.03).
Impact on Clinic Processes and Patient Satisfaction (Table 2)
Table 2.
Clinic Process Improvement
| (Cronbach’s alpha = 0.83.)
Questions (n = 46) |
Agree | Neutral | Disagree |
|---|---|---|---|
| Takes less effort to communicate with the patient | 41% (17) | 49% (20) | 10% (4) |
| Takes less effort to focus attention on the patient | 22% (9) | 22% (9) | 56% (23) |
| Sending and receiving electronic messages in my clinic allows me to deliver more effective patient care | 80% (33) | 15% (6) | 5% (2) |
| The new system has allowed me to better organize my daily work | 69% (31) | 27% (12) | 4% (2) |
| I now have patient charts available to me more often than I did using the previous record system. | 82% (38) | 9% (4) | 9% (4) |
| I can now cross-cover other physicians’ patients more effectively | 83% (35) | 14% (6) | 3% (1) |
| The new system has made it easier for me to practice evidence-based medicine. | 56% (24) | 35% (15) | 9% (4) |
| The new system has improved patient safety. | 45% (20) | 50% (22) | 5% (2) |
| The new system has improved attending supervision of residents’ patient care. | 49% (21) | 44% (19) | 7% (3) |
| My work environment is now less stressful | 30% (13) | 42% (18) | 28% (12) |
| The new system helps me give better overall care | 62% (26) | 33% (14) | 5% (2) |
| The new system has increased patient satisfaction | 31% (13) | 62% (26) | 7% (3) |
When asked their agreement with the notion that the EMR “increased patient satisfaction in our clinic,” 62% of the responses fell in the neutral category. Of note, 50% of respondents were neutral about the EMR’s effect on patient safety, 43% were neutral about its effect on work environment stress, and 44% were neutral about its impact on resident supervision. Eighty percent of the respondents believed that messaging improved their delivery of effective care. Over 80% believed that they were able to cross-cover (manage patients cared for by another physician) more effectively, and had more chart information available with the EMR. Satisfaction with the EMR was associated with more agreement that the EMR improved the practice of medicine (Mean=40.60, SD=6.99, 11 for least improvement, 55 for most improvement) adjusted for gender, computer skill, message basket use and years in practice (p=0.006).
DISCUSSION
As electronic medical record adoption increases, a larger percentage of the market is available to help guide the future development of this technology. This survey of a fairly large group practice suggests that many features not typically present in EMRs may be worth exploring. In particular, respondents valued secure messaging, remote access to the EMR, access to new lab results, computer-based documentation, and the problem list. They did not uniformly value customization features and provider-created reminders. Before implementation, there were concerns that secure messaging was unnecessary in the presence of email, and that remote access was going to force providers to work even when at home. After implementation, respondents believed that electronic messaging allowed them to deliver more effective patient care. They liked access to this system while cross covering and while away from the clinic—two benefits that have been theorized in the literature and that are validated by this study.
Respondents did not know whether or not the use of the EMR impacted patient satisfaction. There have been numerous studies of this issue11,12,13, most of which have suggested that the use of computers in the examination room is not associated with worsening satisfaction. It is possible that primary care providers are shielded from patient perception—additional studies involving patients, nursing staff and clerical associates may yield better information.
Though respondents clearly agreed that this EMR improved the speed with which they could accomplish many tasks (Table 1), a majority also thought the system speed was too slow. Specific aspects of the system that were felt to be to slow were not addressed within this survey, but may be important in how a new system is perceived by users.
This study was performed after implementation, and asked respondents to compare the EMR to their memory of the previous system. Pre-implementation data would have been helpful to further explain some of the results of this study, especially in regard to patient satisfaction, patient safety, system speed, and messaging.
The study was conducted in only one clinic, and represents only one view of an EMR in that environment. Additional studies of multispeciality clinics and clinics of varying sizes need to be done to better understand the functionality that is most associated with high EMR acceptance.
This survey tool was constructed to address the functional needs of a clinic, rather than specific technical implementations of those functions. These respondents felt they had adequate training and support. User satisfaction with an EMR system may obviously vary with their perceptions of implementation and support. Therefore, this tool may be of use to other researchers interested in assessing the functionality of and overall satisfaction with their EMR implementations. Such data are likely to be especially useful in settings where EMR adoption has heretofore been less prevalent.
CONCLUSIONS
An implementation of an EMR in a primary care practice was associated with perceived improvements in speed and communication efficiency and information synthesis capabilities. Users valued remote access and secure messaging technologies in particular. These results confirm previous reports demonstrating favorable acceptance of an EMR with electronic messaging in adult primary care, and support continued adoption in this environment.
REFERENCES
- 1.Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994 Mar–Apr;1(2):108–23. doi: 10.1136/jamia.1994.95236142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Van Der Meijden MJ, Tange HJ, Troost J, Hasman A. Determinants of success of inpatient clinical information systems: a literature review. J Am Med Inform Assoc. 2003 May–Jun;10(3):235–43. doi: 10.1197/jamia.M1094. Epub 2003 Jan 28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Likourezos A, Chalfin DB, Murphy DG, Sommer B, Darcy K, Davidson SJ. Physician and nurse satisfaction with an Electronic Medical Record system. J Emerg Med. 2004 Nov;27(4):419–24. doi: 10.1016/j.jemermed.2004.03.019. [DOI] [PubMed] [Google Scholar]
- 4.Delpierre C, Cuzin L, Fillaux J, Alvarez M, Massip P, Lang T. A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? Int J Qual Health Care. 2004 Oct;16(5):407–16. doi: 10.1093/intqhc/mzh064. [DOI] [PubMed] [Google Scholar]
- 5.Sittig DF, Kuperman GJ, Fiskio J. Evaluating physician satisfaction regarding user interactions with an electronic medical record system. Proc AMIA Symp. 1999:400–4. [PMC free article] [PubMed] [Google Scholar]
- 6.Loomis GA, Ries S, Saywell RM, Thakker NR. If electronic medical records are so great, why aren’t family physicians using them? J Fam Pract. 2002;51:636–41. [PubMed] [Google Scholar]
- 7.Aaronson JW, Murphy-Cullen CL, Chop WM, Frey RD. Electronic medical records: the family practice resident perspective. Fam Med. 2001;33:128–32. [PubMed] [Google Scholar]
- 8.Kirshner M, Salomon H, Chin H. An evaluation of one-on-one advanced proficiency training in clinicians' use of computer information systems. Int J Med Inform. 2004 May;73(4):341–8. doi: 10.1016/j.ijmedinf.2003.11.001. [DOI] [PubMed] [Google Scholar]
- 9.Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc. 2001 Sep–Oct;8(5):499–509. doi: 10.1136/jamia.2001.0080499. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Dansky KH, Gamm LD, Vasey JJ, Barsukiewicz CK. Electronic medical records: are physicians ready? J Healthc Manag. 1999;44:440–5. [PubMed] [Google Scholar]
- 11.Johnson KB, Serwint JR, Fagan LA, Thompson RE, Wilson MH. Computer-based Documentation: Effect on Parent and Provider Satisfaction during a Pediatric Health-Maintenance Encounter. Arch Ped Adol Med. 2005 Mar;159(3):250–4. doi: 10.1001/archpedi.159.3.250. [DOI] [PubMed] [Google Scholar]
- 12.Solomon GL, Dechter M. Are patients pleased with computer use in the examination room? J Fam Pract. 1995 Sep;41(3):241–4. [PubMed] [Google Scholar]
- 13.Greatbatch D, Heath C, Campion P, Luff P. How do desk-top computers affect the doctor-patient interaction? Fam Pract. 1995 Mar;12(1):32–6. doi: 10.1093/fampra/12.1.32. [DOI] [PubMed] [Google Scholar]