Satisfaction and Perceived Quality of an Electronic Medical Record System in a Tertiary Hospital in Oman

Abstract

Objective

To evaluate the knowledge, attitude and practice (KAP) of physicians towards the Electronic Medical Record (EMR) system.

Methods

A cross-sectional survey including physicians from various clinical specialties was conducted. An existing questionnaire was adapted to assess the KAP of physicians towards the EMR system. Information was analyzed using Statistical Package for Social Sciences (SPSS) software.

Results

Out of 200 distributed questionnaires, 141 (70.5%) responses were received. Overall, only 22 physicians (15.6%) rated the current EMR system as an effective tool. A substantial proportion (29.4%) of respondents considered EMR not worth the time and effort required to use it. The majority (67.4%) reported increasing difficulty with the performance of work after applying the EMR system. The overall quality of work was perceived not to have changed (41.2% of the respondents) or declined (27.4% of the respondents). The low satisfaction and underperformance was found to be associated with younger age (p=0.032), junior designation (p=0.041), and low familiarity with computers (p=0.047).

Conclusion

We report low satisfaction and perceived quality of work among physicians in our institution with the current EMR system. Inappropriate and inadequate usage of the system was found to be the main cause of the underlying poor satisfaction.

Introduction

First used for management and administrative purposes, electronic medical record (EMR) systems are now being increasingly employed to collect and synthesize medical information. The EMR system offers support in medical decision making, promotes use of guidelines, increases coordination between different healthcare providers and is believed to improve the overall quality of care. There is a wide variation in satisfaction after the incorporation of EMR systems.

Sultan Qaboos University hospital (SQUH) is a large tertiary public-sector teaching hospital and has had a functioning Healthcare Information System (HIS) since its inception in 1991, functioning in the Departments of Radiology, Laboratory Medicine and Pharmacy. The hospital successfully adopted a fully integrated EMR (Hospital information and Hospital Management) system for patient care and administrative purposes in June 2006. As the hospital acclimatizes to new technology, the need for assessment of quality and improvement of patient care and health delivery has been perceived. Decision makers may be swayed by the general presumption that technology is of benefit to healthcare and should be wholeheartedly embraced. EMR systems should be evaluated for multiple tasks, and tests should employ both qualitative and quantitative methods.

To date, a formal evaluation of the system adopted by SQUH has not been performed. Research shows that satisfaction with information technology is more correlated with the users' perceptions of the system's effect on productivity, than its actual effect on quality of care. We initiated a study to evaluate the knowledge, attitude and usage of the EMR system in SQUH and to evaluate the impact of the EMR system on healthcare delivery from the clinician's perspective. The results of the study are presented in this report.

Methods

The study was approved by the Research Ethics Committee at SQUH. The study was conducted over a period of six months. An existing questionnaire developed and validated for evaluating EMR systems was selected and adapted for the current study. The key feature of the questionnaire is a list of 24 general clinical tasks; applicable to physicians of most specialties, and covers essential parts of their information-oriented work.¹ This standard questionnaire was modified by the research team in order to fit the local setting. The modified version was reviewed by a physician experienced in linguistics blinded to the original version. The questionnaire was then tested in a pilot study involving 30 randomly-selected physicians and validated internally.

The study participants for the pilot study comprised of members of the "Super-User" group, comprised of physician representative of various ‘clinical’ specialties in the hospital. The "clinical" specialties included; Anesthesia, Accident & Emergency, Behavioral Medicine, Dermatology, ENT, Family Practice, Hematology, Oral Health, Medicine, Obstetrics & Gynecology, Ophthalmology, Pediatrics, and Surgery. Content validity of the questionnaire was addressed through an interview where the study participants were asked to provide their assessment on whether the items in the questionnaire were supported locally, whether any additional items were required, and through studying the completed questionnaires. Most of the super-users found the clinical tasks in the questionnaire relevant and comprehensible. Based on results of the pilot study and after considering the minor differences and discrepancies that were detected in the pilot study; the questionnaire was refined for the main study.

The main study (Phase II) comprised of physicians of all clinical specialties who have been working at SQUH with the current EMR system for at least six months. The study questionnaire was distributed to the above physicians of various specialties. The completed questionnaires were collected and data entry was performed by student volunteers. The students received three training sessions (one theory and two practical) by a biostatistician. Data entry was conducted under direct supervision of a member of the research team. Dataset was cross-checked; errors and inconsistencies were corrected.

Statistical analysis was performed through descriptive statistics including; mean, median and proportions of the user characteristics and the potential outcome factors under the study. Analytic statistics were used to explore possible associations between different characteristics of the users and outcomes measured by the questionnaire, including inferential statistics (e.g. t-test and chi-square as appropriate). The scores for knowledge, attitude and practice were correlated to different user characteristics using both bivariate and multivariate robust linear regression analysis. All the statistics were carried out using SPSS.

Results

A total of 200 questionnaires were distributed. The response rate to the questionnaire in this study was 141/200 (70.5%). The demographic characteristics of subjects involved in the study are presented in Table 1. Of the 141 respondents; 55 (39%) were consultants, 28 (19.9%) were registrars and 21 (14.9%) were senior house officers (SHO). Eighty one (57.4%) participants were males. The majority of the participants were in the third to fourth decade of age.

Table 1. Characteristics of subjects included in the study.

Characteristics	Count	%
Total	141
Designation
Consultant	55	39.0
Senior registrar	10	7.1
Registrar	28	19.9
SHO	21	14.9
Resident	16	11.3
Intern	10	7.1
Regular work with patients	135	95.7
Gender
Male	81	57.4
Female	60	42.6
Age range
25-30	31	22.0
31-40	58	41.1
>40	52	36.9
Use of Track Care (years)
< 1	43	30.5
1-2	37	26.2
More than 2	62	44.0
Had Training in EMR	96	68.1
Need extra training	62	44.0
Familiarity with computers
Beginner	4	2.8
Intermediate	103	73.0
Advanced	34	24.1

The vast majority (95.7%) of respondents reported regular work with patients; at least 70% had been using the EMR system for at least one year. Approximately two thirds (68.1%) had prior training on the use of the EMR system. Majority of the subjects reported at least intermediate familiarity with computers.

Table 2 and Fig. 1 show indicators of the global assessment of the EMR. Overall, only 22 (15.4%) respondents rated the EMR to be a success in improving their work in terms of quality, performance, and timeliness. About one third of the participants (n=41; 29.4%) rated the EMR as poor. More than half of the participants (n=78; 55.1%) labeled the EMR only to be fairly successful. Evaluation of the overall quality of work also showed a similar trend; with 39 (27.9%) respondents rating it as poor. Users rated the EMR as poor with group tasks (40.6%) rather than individual tasks (36.5%), (p=0.04). About one third (31.2%) of the respondents reported poor satisfaction with the EMR system. Furthermore, more than 25% of the respondents reported poor EMR function in terms of time efficiency.

Table 2. Global assessment of EMR.*.

Indicator	Poor	Fair	Good
Time efficiency	26.3	36.5	37.2
Satisfaction with EMR	31.2	57.2	11.6
Team performance	40.6	26.8	32.6
Individual performance	36.5	33.6	29.2
Overall quality	27.9	41.2	28.7
Overall success	29.4	55.1	15.4

* All numbers are percentages

Figure 1.

Global assessment of EMR.

Intradepartmental indicators of satisfaction with the EMR are depicted in Fig. 2. Over one third of the respondents reported consistent difficulty with the use of EMR, and stated that the EMR was not user-friendly. One third of the respondents were of the opinion that the EMR did not provide information in a timely manner. However, the majority reported that the EMR provided accurate and sufficient information.

Figure 2.

Indicators of satisfaction with EMR in the departments of the respondents.

The results of multi-variate logistic regression analysis of factors affecting rating of the global success are presented in Table 3. After adjusting for other confounding factors; compared to female respondents, the male respondents exhibited a 2.1 times chance of giving a poor satisfaction report (OR= 2.1; 95% CI 1.0, 4.4). Younger age (<30 years) was three times more likely to be associated with poor rating compared to older age. In addition, junior designation was also significantly associated with poor rating (p=0.04). Compared to respondents who had prior training, respondents who had no prior training exhibited two times higher probability of reporting poor rating (OR= 2.0; 95% CI 1.2, 4.3). A weak evidence of association was observed between poor rating and the absence of regular work with patients, in addition to low familiarity with computers. However, the associations were not statistically significant.

Table 3. Factors affecting poor rating for global success.

Variable	OR	95% CI	p value
Male gender	2.1	1.0, 4.4	0.05
Junior designation (Below registrar)	1.7	1.1, 5.6	0.04
Young age (<30)	3.3	1.4, 2.9	0.02
Not working with patients	1.3	0.2, 7.3	0.78
No prior training	2.0	1.2, 4.3	0.03
No familiarity with computers	1.6	0.2, 17.5	0.68

Discussion

There has been growing recognition of the role of EMR systems in the provision of healthcare in recent years, and the use of an EMR system in a department has been proposed as a criterion of quality.² Given the competing demands of stakeholders (patients, providers, regulatory agencies, accreditation organizations, vendors, payers, and users), the structure and function of these applications are quite diverse.³ Studies have shown that the use of EMR systems has resulted in improved health outcomes.^4,5 The use of information technology systems has been linked to decreased medical errors.^2,6 Patient access to health information and personal health records through EMR systems is becoming increasingly possible,⁷ with the "patient-centered" approach of providing care having the potential of incorporating patient preferences in clinical decision making.

In 2000, WHO categorized Oman as a country with the most efficient health system in the world in terms of outcome.⁸ Although a form of computer system existed in most hospitals in Oman as early as 1990, they functioned only in some departments, such as radiology and laboratory medicine. However, in 2006, the Sultanate of Oman's digital society initiative - eOman was declared, and the country began moving towards a paperless society. In keeping with this initiative to promote the computerization of healthcare information, the Ministry of Health undertook the installation of EMR systems in various hospitals in the country. A fully integrated EMR system was first installed in Sur. A study evaluating physician satisfaction with this EMR system identified a positive impact in areas of communication, data entry and retrieval, overall patient care, and reduction of medical errors. However, the study also identified a few negative aspects, namely; loss of confidentiality of information and software related problems, particularly related to diagnosis codes.⁹

Evaluation is a challenging but necessary part of the development cycle of EMR systems.¹ The current study was initiated in order to evaluate the knowledge, attitude and usage of the newly implemented EMR system at our institution. The focus of this study was on practitioners’ performance and system efficiencies, as well as their perception of how the EMR system has impacted patient care.

For the purpose of this study; a questionnaire was developed and validated for evaluating EMR systems, which has well-established reliability.¹ Questionnaires are frequently used as a quantitative evaluation method in medical informatics, and measures of validity indicate whether an item measures up to its intended purpose. It is a significant tool to obtain an insight into what people consider and feel. There are very few validated questionnaires addressing the clinical use of EMR systems. A questionnaire is considered good if it includes both closed and open-ended questions. We used a questionnaire which was developed and validated for EMR system evaluation.¹ This standard questionnaire was modified and tested in a pilot study for content validity (relevance of tasks, accuracy of task interpretation, and estimation of EMR use) and criterion validity (correlating task-oriented EMR use to general EMR use, task performance to overall work performance, and task performance to user satisfaction).

The current study showed low satisfaction with the EMR system among physicians of clinical specialties in our institution. Nearly one-third of the participants reported that the current system was inefficient. Over 60% of users reported that the system did not improve their quality of work; while some even reporting increased difficulty in performing tasks after implementation of the EMR system. Some participants stated that the quality of their work had declined with the use of EMR. On analyzing the factors correlated with satisfaction; it was found that the male gender, age below 30 years (p=0.032), junior designations (p=0.041) and low familiarity with computers (p=0.047), were strongly associated with poor satisfaction. Respondents who did not attend the training session at the start of the EMR system, and those who did not regularly work with patients were also more likely to report poor satisfaction; however these associations were not statistically significant.

By providing a coordinated delivery of clinical services, the EMR system has been accepted as an enabling technology that allows physicians to pursue more advanced practices than would be possible with paper-based records. Previous studies have shown that the use of EMR systems has resulted in improved health outcomes.^3,4 However, these studies have also reported that healthcare providers seeking to use EMR systems in their practice are likely to face many challenges. This was a similar finding to the current study. Factors influencing the successful implementation and acceptance of an EMR system include; amount of dedicated time for training to bring all users to an appropriate level of familiarity with the specific EMR system software, presence of a "champion" or EMR problem solver and an efficient "Help Desk", and baseline levels of computer knowledge among the users. New users may not understand the scope of change required in implementing an EMR, and a great time commitment may be required prior to successful implementation.⁵ In our study, the low satisfaction associated with junior designations and respondents with low familiarity with computers may be related to this fact.

Obstacles towards the implementation of EMR systems also include; difficulty in transferring older paper-based records to the EMR system, issues based on long-term preservation and storage of data, as well as how to ensure the physical and virtual security of the archives, software problems of codification (standards that help to ensure that clinical information input and retrieval are not arbitrary), and customization (system adapted for the users and tailored to workflows specific to a user site). In the current study; an overwhelming 60% of users still resorted to paper records for some tasks not supported by EMR, such tasks included; statistics, codification, and patient transfers between institutions. (Fig. 3)

Figure 3.

Usage of EMR. (X axis shows various tasks, and Y axis the percentage usage. * indicates tasks not supported by the current version of the EMR system).

Development of "inter-hospital networks" through the EMR system although feasible, has not occurred due to software compatibility issues. No electronic networks have been built between medical institutions. Thus, communication between EMR systems in the country needs to be standardized to facilitate interaction of operations.

Additional challenges such as hardware limitations including interfacing with older technology, security and confidentiality issues, as well as lack of integrated delivery systems, in addition to reluctant providers, and prohibitive (start-up and maintenance) costs have also been important considerations negatively impacting the acceptance of EMR systems.

The main limiting factor in the current study is the fact that only the physician user-group was evaluated, and therefore lacking the contribution of other types of health personnel. Furthermore, the list of tasks used was not comprehensive and there are some areas missing, but there is scope for augmenting the list in future versions of the questionnaire.

Conclusion

The study highlights challenges and barriers, but also provides information concerning the general efficacy of the EMR system. It also serves as a benchmark while considering future system updates. The method adopted in this study can be used subsequently to compare the EMR systems being used in our institution with other EMR systems in use in the country. Further studies are required to explore the practical obstacles and methods of circumventing them.