Abstract
BACKGROUND:
Hospital Information System (HIS) is a type of health information system which is widely used in clinical settings. Determining the success rate of HISs is an ongoing area of research since its implications are of interest for researchers, physicians and managers.
AIM:
In the present study, we develop a novel instrument to measure HIS success rate based on users’ viewpoints in a teaching hospital.
METHODS:
The study was conducted in Ebnesina and Dr Hejazi Psychiatric Hospital and education centre in Mashhad, Iran. The instrument for data collection was a self-administered structured questionnaire based on ISSM, covering seven dimensions, which includes system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. The verification of content validity was carried out by an expert panel. The internal consistency of dimensions was measured by Cronbach’s alpha. Pearson’s correlation coefficient was calculated to evaluate the significance of associations between dimensions. The HIS success rate on users’ viewpoints was determined.
RESULTS:
A total of 125 users participated in the study. The instrument was validated by an expert panel with the Content Validity Index (CVI): 0.85 and Content Validity Ratio (CVR): 0.86. The overall Cronbach’s alpha value of the instrument was 0.93. The Pearson correlation coefficient showed significant positive relationships among the investigated dimensions. On average, HIS success rate in the hospital under study was 65% (CI: 64%, 67%). The dimensions of “usefulness”, “system quality”, and “net benefits” showed the highest rates of success, respectively.
CONCLUSION:
The instrument used in this study can be adopted for HIS evaluation in future studies. In the current study, a method was developed to determine the HIS success rate based on users’ viewpoints. This method allows for the comparison of HIS success rates in various hospitals. As well, our findings underscore the viewpoints of HIS users in a developing country.’
Keywords: Electronic health record, Evaluation, Hospital information systems, Information system success model, ISSM
Introduction
Hospitals have implemented health information systems to provide timely and accurate information, thus fulfilling managerial needs and improving healthcare effectiveness and efficiency at a reasonable cost [1].
Hospital Information System (HIS) is one of the health information systems which has been extensively utilised. There are major concerns regarding the quality of these systems. Therefore, it is vital to continuously evaluate HISs. As well, HIS implementation requires a great deal of investment. For example, the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 mandates about US$10 million for each hospital in the US [2]. Therefore, post-implementation evaluation of HIS success is of critical importance. Determining HIS success rate is an ongoing area of research since its implications are of interest not only for researchers but also to physicians and managers. The results of the evaluation can reveal the value of HIS and provide essential information for subsequent decisions [3].
There are many existing frameworks to evaluate health information systems. ISSM is one of the most well-known and valid frameworks to assess these systems [4]. ISSM was developed by DeLone and Mclean in 1990 [5] and was subsequently updated in 2003 [6]. ISSM is comprised of seven dimensions including system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits [6]. ISSM focuses on the assessment of technological aspects of information systems based on users’ viewpoints and explains the influence of technology on system use as well as users’ satisfaction. The success of information systems depends on the interaction of different ISSM dimensions [7]. Some of the studies which have deployed this framework to assess HIS include: Tilahun and Fritz [4], Sicotte et al., [8], Otieno et al., [9], Ojo [3], Messeri et al., [10], and Aggelidis et al., [11].
In 2014, the Iranian Ministry of Health and Medical Education mandated that public hospitals transfer patients’ medical records to SEPAS (Iran’s Health Electronic Record project). This led to a rapid growth in the implementation of HISs in hospitals across the country. Following the implementation of health electronic record (EHRs) and HISs in Saudi Arabia [12], Korea [2], and Taiwan [13], [14] the success of these systems was evaluated. For the case of Iran as a developing country, to the best of our knowledge, there have been very few studies which have evaluated the success of HISs. The studies by Tavakoli et al., [15], Abdekhoda et al., [16], and Saghaeianejad et al., [17] are some examples to this end. Nevertheless, these studies did not collectively provide conclusive evidence on the success of HISs. As such, there is a need for additional studies to gather further evidence on the effectiveness and efficiency of HISs in Iran.
In the present study, we report on the development of an instrument to evaluate the rate of HIS success based on ISSM, which can be used to compare HIS success rates in different hospitals and clinical settings. Furthermore, in a case study, we investigate the success rate of HIS based-on users’ viewpoints in a teaching hospital in Iran.
Methodology
This was a descriptive cross-sectional study.
Instrument
A self-administrated structured questionnaire was developed based on ISSM to collect data. The questionnaire comprised of seven dimensions including system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. Each dimension encompasses several evaluation measures. The questions were designed based on the ISSM evaluation measures and dimensions.
Verification of the content validity was carried out by an expert panel. The panel consisted of nine medical informatics experts. Expert validity was measured using a Content Validity Index (CVI) and Content Validity Ratio (CVR). The internal consistency of each dimension was measured by Cronbach’s alpha. Pearson’s correlation was calculated to evaluate the significance of associations between dimensions.
The final draft of the instrument consisted of two sections. The first section includes users’ characteristics such as age, gender, education level, working hours per month, ICDL certification, work experience, computer experience and HIS experience (n = 8 questions). The second section includes 60 questions on a 5-point Likert scale range from 1 (completely disagree) to 5 (completely agree), which covers 16 evaluation measures in 7 dimensions (Table 1).
Table 1.
Dimensions and evaluation measures
| Dimensions | Evaluation Measure (number of questions) | Definition |
|---|---|---|
| System Quality | Adaptability (n = 1) | “Data from different sources can be consolidated or compared without inconsistencies” [18]. |
| Reliability (n = 1) | “The dependability and consistency of access and uptime of systems” [18, 19]. | |
| Availability (n = 6) | “Access to the computer resource and infrastructure where and when it is needed”. | |
| Usability (n = 6) | “The capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfil the specified range of tasks, within the specified range of environmental scenarios. ” [20]. | |
| Information Quality | Security (n = 3) | “The safeguarding of data from misappropriation or unauthorised alteration or loss” [21]. |
| Ease of understanding (n = 3) | “Ease of determining what a data element on a report or file means, or what is excluded or included in calculating it” [18]. | |
| Completeness (n = 2) | “The comprehensive of the output information content” [21] . | |
| Personalization (n = 1) | Ability to personalize HIS for users. | |
| Relevance (n = 1) | “The degree of congruence between what the user wants or requires and what is provided by the information products and services” [21]. | |
| Service Quality | Responsiveness (n = 4) | “Willingness to help customers and provide prompt service” [22]. |
| Assurance (n = 4) | “Knowledge and courtesy of employees and their ability to inspire trust and confidence” [22]. | |
| Empathy (n = 4) | “Empathy Caring individualised attention the service provider gives its” [22]. | |
| Satisfaction | - Evaluation Measure was dimension (n = 4) | “The level of overall user’s satisfaction with HIS” [19]. |
| System Use | - Evaluation Measure was equal to dimension (n = 1) | “The number of hours that users use the HIS”. |
| System Usefulness | Evaluation Measure was equal to dimension (n = 4) | “The degree to which user believes that using the new system would improve his/her job performance” [23]. |
| Net Benefits | Evaluation Measure was equal to dimension (n = 18) | “As the “impacts” of HIS have evolved beyond the immediate user, researchers have suggested additional IS impact measures, such as workgroup impacts, inter-organizational and industry impacts, consumer impacts, and societal impacts” [6]. |
Study settings
The study was conducted in Ebnesina and Dr Hejazi psychiatric hospital and education centre. The case hospital was a 900-bed teaching hospital, which has been in operation for more than 40 years. The hospital is the largest psychiatric hospital in Northeastern Iran. Patients from five neighbouring provinces (with an approximate population of eleven million) are referred to this case hospital for professional psychological services. The hospital consists of 16 specialised and sub-specialised departments such as psychiatry emergencies, paediatrics psychiatry, geriatric psychiatry, drug abuse treatment, and adult psychiatry. Average bed occupancy and bed turnover ratio is 1.01 and 75%, respectively. The length of stay in the hospital is 9 days on average.
In 2002, Mashhad University of Medical Science implemented a customised HIS named IHIS (Iranian Hospital Information System). Currently, IHIS is implemented in all hospitals affiliated with the university. The IHIS of the case hospital covers information systems (IS) for different services including admission, discharge and transfer, inpatient, outpatient, pharmacy, laboratory, radiology, accounting, and insurance.
Participants
Target respondents were users in four departments including psychiatric departments, administrative and financial departments, nursing management departments, and para-clinics departments. Given the high number of nurses in the psychiatric departments (n = 190), a sample size of 64 participants for nurses was calculated. As well, 7 secretaries of the psychiatric departments also participated in the study, making a total of 71 participants from the psychiatric department. All users in other departments (administrative and financial departments, nursing management department, and para-clinics departments) were asked to participate in the current study, of whom a total of 61 participants agreed to participate. Thus, the total number of participants for the study was 125 individuals. Empirical data were collected targeting users with over six months of HIS working experience.
The study was approved by the ethical committee of Mashhad University of Medical Sciences before the instrument was officially distributed to all users to protect the rights and privacy of the participants.
The researches met all users in person and invited them to participate in the study. Questionnaires were provided to users who agreed to participate in the study.
Statistics
Summary statistics for characteristics of participants were calculated as frequencies and proportions. The negative responses were reversed. HIS success rate in the case hospital was calculated in two steps. First, the rate of HIS success by each user in evaluation measures, dimensions, and total dimensions of the instrument was determined. This was performed using the following formulae:
Maximum HIS success rate of evaluation measures, dimensions, and total dimensions by each user = number of questions* 5 (maximum score for each question on a 1-5 point Likert scale).
The acquired rate of HIS success of evaluation measures, dimensions, and total dimensions by each user = sum of the acquired score for each question on a 1-5 point Likert scale by each user.
HIS Success Rate = (The acquired rate of HIS success of evaluation measures, dimensions, and total dimensions by each user/Maximum HIS success rate of evaluation measures, dimensions, and total dimensions by each user) * 100.
In the following paragraphs, a sample of conducted calculations will be explained. The “Availability” evaluation measure belonged to the system quality dimension. This evaluation measure has six related questions, and users responded to each question on a 1-5 point Likert scale. The maximum possible score for “Availability” by each user was equal to 1 (one user) * 6 (number of questions) * 5 (maximum score for each question on a 1-5 point Likert scale) = 30. The HIS success rate for “Availability” by “user one” was 27. Therefore, the HIS success rate for “Availability” by “user one” = (27/30) * 100 = 90.00%.
The dimension of “system quality” consisted of 14 questions. The maximum possible score for system quality by each user was equal to 1 (one user) * 14 (number of questions) * 5 (maximum score for each question) = 70. The HIS success rate of the system quality by “user one” was 51. Therefore, the HIS success rate for system quality by “user one” = (51/70) * 100 = 72.9%.
Total dimensions of the instrument consisted of 61 questions. Accordingly, the maximum possible score of all dimensions by each user was equal to 1 (one user) * 60 (number of questions) * 5 (maximum score for each question) = 300. The HIS success rate for total dimensions of the instrument by “user one” was 205. Therefore, the HIS success rate for total dimensions of the instrument by “user one” = (205/300) * 100 = 68.3%.
In the second step, the mean and confidence interval of the HIS success rate were determined for evaluation measures, dimensions, and total dimensions of the instrument by four groups of users. These groups include para-clinics departments, administrative and financial departments, nursing management department, and psychiatric departments. An assessment of the normality of data is a necessity for many statistical tests. Hence, to compare HIS success rate among four user groups the normality of data in each ISSM dimensions was assessed. Some researchers suggested the Shapiro-Wilk test as the best option for testing the normality of data [24]. In our study, the Shapiro-Wilk test was used to assess the normality of data. In the current study, comparisons of the HIS success rate were made among four user groups using ANOVA for normal variables and Kruskal-Wallis test for non-normal variables. A p-value of less than 0.05 was considered statistically significant.
In this study, two post-hoc tests were chosen to identify significant inter-group differences at p ≥ 0.05. The Tukey honestly significant difference (HSD) test for normal variables and the Mann–Whitney test for non-normal variables was conducted. Data analysis was performed using SPSS, version 11.5 statistical software.
Results
Participants
In this study, the researchers met with 191 target users, among which 125 individuals agreed to participate in the study. Table 2 demonstrates the characteristics of the participants. Over two-thirds of the participants were women. The age range of users was 22 to 55 years, and most of the participants were aged 20-30. Only 2% of participants were without a university education. More than 77% of the participants had a two or four-year university degree. 61% of participants had 5-15 years of work experience. 68% of the users had an ICDL certificate. 80% of the users had over 3 years of HIS experience, and 68% of them had more than 6 years of computer experience.
Table 2.
Participants’ characteristics
| User’s characteristics | Sub groups | Frequency (%) |
|---|---|---|
| Sex | Male | 48 (38.4) |
| Female | 77 (61.6) | |
| Age (years) | 20-30 | 20 (16.0) |
| 30-40 | 73 (58.4) | |
| 40-50 | 30 (24.0) | |
| > 50 | 2 (1.6) | |
| Education level | High School Diploma | 2 (1.6) |
| Associate’s | 2 (1.6) | |
| Bachelor’s | 94 (75.2) | |
| Master’s | 27 (21.6) | |
| Working hours per month | < Regular + 20 overtime hours | 41 (20.8) |
| Regular + 20-60 overtime hours | 44 (35.2) | |
| Regular + 60-120 overtime hours | 30 (24.0) | |
| > Regular +120 overtime hours | 10 (8.0) | |
| Work experience (years) | < 5 | 23 (18.4) |
| 5-10 | 35 (28) | |
| 10-15 | 41 (32.8) | |
| 15-20 | 13 (10.4) | |
| 20-25 | 9 (7.2) | |
| > 25 | 4 (3.2) | |
| ICDL certification | No | 28 (22.4) |
| Yes | 97 (77.6) | |
| Computer experience (years) | > 1 | 15 (12) |
| 1-3 years | 9 (7.2) | |
| > 3 years | 101 (80.8) | |
| HIS experience (years) | < 1 year | 3 (2.4) |
| 1-3 years | 10 (8) | |
| 3-6 years | 14 (11.2) | |
| > 6 | 98 (78.4) |
Reliability and Validity
The instrument was validated by an expert panel with CVI: 0.85 and CVR: 0.86. The overall Cronbach’s alpha value of the instrument was determined as 0.93, representing high reliability. This value was between 0.476 - 0.943 for different instrument dimensions, as shown in Table 4. The value of Cronbach’s alpha was satisfactory among the five dimensions of system quality, information quality, service quality, usefulness, and net benefits. The value of Cronbach’s alpha for the satisfaction dimension was low, and its value was 47.0. Because the dimension of “system use” had just one question, its Cronbach’s alpha was not calculated. The Pearson correlation coefficient showed significant positive relationships among the investigated dimensions (p-value: 0.01). Pearson correlation coefficient was between 0.197 - 0.707 for different dimensions (Table 4).
Table 4.
Mean success rate of evaluation dimensions
| Users/Dimensions | Mean (95% Confidence Interval) | ||||
|---|---|---|---|---|---|
| Financial and administrative departments | Psychiatric departments | Nursing management department | Para-clinical department | All users | |
| System quality | 0.74 (0.71, 0.77)1 | 0.65 (0.63, 0.67) | 0.71 (0.65, 0.78)b | 0.64 (0.59, 0.70)b | 0.67 (0.65, 0.69) |
| Information quality | 0.65 (0.61, 0.69) | 0.62 (0.60, 0.64) | 0.62 (0.56, 0.68) | 0.62 (0.58, 0.66) | 0.63 (0.61, 0.64) |
| Service quality | 0.62 (0.56, 0.67) | 0.63 (0.61, 0.66) | 0.62 (0.56, 0.69) | 0.65 (0.62, 0.69) | 0-.63 (0.61, 0.65) |
| System use | 0.63 (0.47, 0.79)a | 0.30 (0.25, 0.35)b | 0.27 (0.16, 0.37)b | 0.41 (0.29, 0.54)b | 0.37 (0.32, 0.41) |
| Satisfaction | 0.68 (0.63, 0.73) | 0.63 (0.60, 0.66) | 0.68 (0.61, 0.75) | 0.66 (0.62, 0.71) | 0.65 (0.63, 0.67) |
| Usefulness | 0.78 (0.71, 0.85) | 0.71 (0.67, 0.75) | 0.77 (0.67, 0.86) | 0.76 (0.72, 0.80) | 0.74 (0.71, 0.76) |
| Net Benefits | 0.68 (0.62, 0.74) | 0.64 (0.61, 0.68) | 0.70 (0.60, 0.79) | 0.66 (0.61, 0.71) | 0.66 (0.63, 0.68) |
| Total HIS Success | 0.69 (0.65, 0.72) | 0.64 (0.62, 0.66) | 0.67 (0.61, 0.74) | 0.65 (0.62, 0.69) | 0.65 (0.64, 0.67) |
The significant results within each group of users are indicated by letters a and b; values not sharing a common letter differ significantly (P < 0.05).
Table 3.
Correlations among all dimensions (n = 125)
| Dimensions (Cronbach’s Alpha) | System Quality | Information Quality | Service Quality | System use | Satisfaction | Usefulness | Net Benefits |
|---|---|---|---|---|---|---|---|
| System quality (0.719) | 1 | 0.401** | 0.255** | 0.450** | 0.319** | 0.270** | 0.348** |
| Information quality (0.611) | 1 | 0.454** | 0.315** | 0.471** | 0.495** | 0.454** | |
| Service quality (0.785) | 1 | 0.197* | 0.329** | 0.285** | 0.352** | ||
| Satisfaction (0.476) | 1 | 0.423** | 0.444** | 0.411** | |||
| System use | 1 | 0.660** | 0.632** | ||||
| Usefulness (0.926) | 1 | 0.707** | |||||
| Net Benefits (0.943) | 1 |
Correlation is significant at the 0.01 significance level (2-tailed);
Correlation is significant at the 0.05 significance level (2-tailed).
HIS success rate
The dimensions of “usefulness”, “system quality”, and “net benefits” obtained the highest mean rates of success, respectively. The “system use”, “service quality”, and “information quality” dimensions were ranked 4th-6th. Table 4 demonstrates the mean success rates for all HIS evaluation dimensions. These results will be discussed further in the following paragraphs.
HIS success rate in system quality
The dimension of “system quality” included four evaluation measures, with a general mean success rate of 0.67 (95% CI: 65%, 69%) from the users’ perspective. Figure 1 shows the mean success rate of evaluation measures in the “system quality” dimension. Evaluation measures of “adaptability” and “reliability” acquired a mean success rate of smaller than 60%. Moreover, evaluation measures of “availability” and “usability” acquired a mean success rate of greater than 60%.
Figure 1.
The mean success rate on the dimension of system quality
The findings of this study showed that there is a significant difference in the mean success rate in the “availability” evaluation measure between para-clinical and financial and administrative departments (p < 0.001). A significant difference was observed in the psychiatric departments (p < 0.001). The mean “availability” in the financial and administrative departments, nursing management department, para-clinical, and psychiatric departments were 88%, 77%, 72%, and 68%, respectively.
The results of the study illustrated that ICDL certificate and “availability” evaluation measure had a significant difference (p-value: 0.037). The “usability” evaluation measure was significantly associated with education level. There was a significant difference in the mean success rate for the “usability” evaluation measure between the users with a diploma and bachelor’s degrees (p-value: 0.033). This difference was observed between the users with the bachelor’s and master’s degrees (p-value: 0.033). The mean success rate in this evaluation measure for the users with a diploma, bachelor’s and master’s degrees were 78%, 66% and, 64%, respectively.
The significant results within each group of users are indicated by letters a and b; values are not sharing a common letter differ significantly (P < 0.05).
HIS success rate in information system
The “information quality” dimension includes five measures, with a total of 0.63 (95%, CI: 61%, 64%) mean success rate on users’ viewpoints. The range of mean success rate on the evaluation measures in this dimension was 0.42-0.73 (Figure 2). The minimum and maximum success rates belong to “HIS privatisation” and “HIS’s information relevancy”, respectively. There was no significant difference in the mean success rate between the evaluation measures of this dimension and users’ characteristics.
Figure 2.
The mean success rate on the dimension of information quality
HIS success rate in the service system
The “service quality” dimension includes three evaluation measures. In general, this dimension acquired 0.37 (95% CI: 61%, 65%) of the mean success rate from users (Figure 3). A significant difference in the mean success rate between evaluation measures of this dimension and either users’ characteristics or groups of users’ was not observed.
Figure 3.
The mean success rate on the dimension of service quality
HIS success rate in system use
Compared to other dimensions, “system use” was assigned the least mean success rate (0.37 (95% CI: 32%, 41%)) on users’ viewpoints. A significant difference between “system use” and users’ categories was observed (p < 0.001). The “system use” in the financial and administrative departments was higher than in other groups. The mean rate of system use among users of financial and administrative departments, para-clinical departments, psychiatric departments, and nursing management department were 0.68, 0.41, 0.30, and 0.27, respectively. There was a significant difference between the ICDL certificate and system use (p < 0.001).
HIS success rate in satisfaction, usefulness, and net benefits
The mean success rate of the “satisfaction” dimension was 0.65 (95% CI: 63%, 67%). The results of the current study revealed that there is a significant difference between the mean success rate of this dimension and the number of working hours per month (p-value: 0.041). The mean success rate of “satisfaction” for users with higher working hours per month was larger in comparison with other users.
The “usefulness” dimension attained the highest success rate (0.74 (%95 CI: 71%, 76%)) on users’ viewpoints. The “net-benefits” dimension acquired 0.66 (%95 CI: 63%, 68%) of the mean success rate on users’ viewpoints. A significant difference between the mean success rate of these dimensions and users’ characteristics were not observed.
Total HIS success rate
On average, “total HIS success rate” in the case hospital was %65 (CI: 64%, 67%) (Figure 4). Mean total HIS success was not significantly associated with users’ groups. There was no significant difference between the mean total HIS success and users’ characteristics including age, gender, education, work experience, working hours per month, ICDL certificate, computer use experience, and HIS use experience.
Figure 4.
The mean total HIS success rate
Discussion
In the current study, an ISSM-based instrument was constructed to evaluate HISs. The instrument was found to have a high rate of validity and reliability. This can be applied to evaluate other HISs in future studies. As well, a method was developed to determine the HIS success rate based on users’ viewpoints. This method enables the users to compare HIS success rate in various hospitals. The association between the mean success rate between evaluation dimensions and users’ characteristics such as age, gender and users’ categories was investigated. The results of the present study highlight the characteristics and opinions of HIS users in a developing country. The most important findings of the study will be discussed in following paragraphs.
ISSM has been modified and validated in healthcare settings in a variety of empirical studies [3], [4], [9], [12], [26], [27], [28]. The present study validates a questionnaire based on the ISSM framework in the context of hospital information systems in a teaching hospital of a developing country. The results of the current study demonstrated that there is a significant correlation between the seven dimensions of ISSM, which includes system quality, information quality and service quality, system use, usefulness, satisfaction, and net benefit.
The results of the study revealed that users hold positive views on HIS’s success in the case hospital. Overall, the HIS success rate was acceptable in the evaluation measures such as usability, information security, completeness and job relevancy. Most users viewed dimensions of system quality, system use and satisfaction. Top et al., [25] illustrated that EHRs are useful and capable of enhancing the quality of information and the quality of workflow. They also found out that nurses can easily enter, access, and read data using EHRs and can conduct their work faster. According to a study by Lambooij et al., [28] users believed that EMR is easy to use and aligned with their work. They perceived that the quality of patient data is better when EMRs are easier to use and better aligned with their daily routine. The findings of the study by Bossen et al., [27] on EHR evaluation also reported similar results. In total, users had positive experiences with the EHR and its operational reliability, response time, log in and support. Moreover, EHR performance was acceptable. We observed a satisfactory average success rate in the usability evaluation measure. However, a significant difference between usability and education level was observed in our study. Users with a diploma, bachelor’s and master’s degrees had different perceptions of usability. The results of the studies by Calisir et al., [29] and Brown [30] confirm our results. The results of their study revealed that education level has a significant effect on perceived ease of use and usability. In 2018, a study was conducted on IHIS’s usability and identified usability problems. It seems that these usability problems in IHIS influenced users’ perceptions of usability with different levels of education [31].
In this study, we noticed that there is a significant difference among users’ groups and the availability of computer resources. Computer resources are more easily available in the financial and administrative departments compared with the other three groups. Meanwhile, a significant difference between system use and users’ groups was observed. Users of financial and administrative departments use HIS more often compared with other users. Rate of HIS use in users with more abundant computer resources was higher than other users. The result of the study by Lu et al. demonstrated that the availability of computer resources is a key factor in HIS use [14].
The findings of the study by Asadi et al., [32] revealed that information systems in Iran are mainly focused on financial objectives. Our findings revealed that mean system use among users of the nursing management department and psychiatric departments is generally low. The results of the study by Top et al., [25] are congruent with these findings. There might be a significant difference between the rate of system use, availability of computer resources and users groups because managers and policymakers are focused on HIS’s financial objectives. The findings of the current study also demonstrated that nursing managers had access to more computer resources compared with users of psychiatric departments. However, no significant difference was observed on HIS use rate between these two groups. Besides, findings of a study by Tubashat’s [33] showed that the professional rank of nurses does not influence the rate of HIS use.
We faced a couple of limitations in this study. In the present study, system use, which is a key measure in HIS evaluation, was evaluated by users’ self-report. This could lead to subjective evaluation of this measure. A second limitation was that the number of computers was insufficient in the psychiatric departments, mostly running at low speed. As a result, it was often the case that one user logged into the HIS and other users used that HIS under the username of the first user. Therefore, we were unable to accurately report the true amount of system use based on HIS log records.
In conclusion, in the current study, an instrument based on the ISSM framework and method was developed to determine the HIS success rate based on users’ viewpoints. This instrument covers seven dimensions including system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. The proposed method enables the researchers to determine the HIS success rate in accordance with these dimensions. The value of HIS success rate in each dimension is a quantitative measure. Hence, for future studies researchers are equipped with an objective measure to compare HIS success rates across a wide range of hospital settings.
As well, our findings underscore the characteristics and opinions of HIS users in a developing country. Our results showed that HIS in the case hospital acquired an acceptable success rate (69%) based on users’ viewpoints. Of note, some dimensions such as usability require modifications and improvements.
Footnotes
Funding: This research did not receive any financial support
Competing Interests: The authors have declared that no competing interests exist
References
- 1.Chen RF, Hsiao JL. An investigation of physicians'acceptance of hospital information systems:a case study. International journal of medical informatics. 2012;81(12):810–20. doi: 10.1016/j.ijmedinf.2012.05.003. https://doi.org/10.1016/j.ijmedinf.2012.05.003 PMid:22652011. [DOI] [PubMed] [Google Scholar]
- 2.Iqbal U, Ho CH, Li YC, Nguyen PA, Jian WS, Wen HC. The relationship between usage intention and adoption of electronic health records at primary care clinics. Computer methods and programs in biomedicine. 2013;112(3):731–7. doi: 10.1016/j.cmpb.2013.09.001. https://doi.org/10.1016/j.cmpb.2013.09.001 PMid:24091088. [DOI] [PubMed] [Google Scholar]
- 3.Ojo AI. Validation of the DeLone and McLean information systems success model. Healthcare informatics research. 2017;23(1):60–6. doi: 10.4258/hir.2017.23.1.60. https://doi.org/10.4258/hir.2017.23.1.60 PMid:28261532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Tilahun B, Fritz F. Modeling antecedents of electronic medical record system implementation success in low-resource setting hospitals. BMC medical informatics and decision making. 2015;15(1):61. doi: 10.1186/s12911-015-0192-0. https://doi.org/10.1186/s12911-015-0192-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.DeLone WH, McLean ER. Information systems success:The quest for the dependent variable. Information systems research. 1992;3(1):60–95. https://doi.org/10.1287/isre.3.1.60. [Google Scholar]
- 6.Delone WH, McLean ER. The DeLone and McLean model of information systems success:a ten-year update. Journal of management information systems. 2003;19(4):9–30. https://doi.org/10.1080/07421222.2003.11045748. [Google Scholar]
- 7.Otieno GO, Hinako T, Motohiro A, Daisuke K, Keiko N. Measuring effectiveness of electronic medical records systems:towards building a composite index for benchmarking hospitals. International journal of medical informatics. 2008;77(10):657–69. doi: 10.1016/j.ijmedinf.2008.01.002. https://doi.org/10.1016/j.ijmedinf.2008.01.002. [DOI] [PubMed] [Google Scholar]
- 8.Sicotte C, Paré G, Bini KK, Moreault MP, Laverdure G. Virtual organization of hospital medical imaging:a user satisfaction survey. Journal of digital imaging. 2010;23(6):689–700. doi: 10.1007/s10278-009-9220-x. https://doi.org/10.1007/s10278-009-9220-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Otieno OG, Toyama H, Asonuma M, Kanai-Pak M, Naitoh K. Nurses'views on the use, quality and user satisfaction with electronic medical records:questionnaire development. Journal of Advanced Nursing. 2007;60(2):209–19. doi: 10.1111/j.1365-2648.2007.04384.x. https://doi.org/10.1111/j.1365-2648.2007.04384.x. [DOI] [PubMed] [Google Scholar]
- 10.Messeri P, Khan S, Millery M, Campbell A, Merrill J, Shih S, Kukafka R. An information systems model of the determinants of electronic health record use. Applied clinical informatics. 2013;4(02):185–200. doi: 10.4338/ACI-2013-01-RA-0005. https://doi.org/10.4338/ACI-2013-01-RA-0005 PMCid:PMC3716425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Aggelidis VP, Chatzoglou PD. Hospital information systems:Measuring end user computing satisfaction (EUCS) Journal of biomedical informatics. 2012;45(3):566–79. doi: 10.1016/j.jbi.2012.02.009. https://doi.org/10.1016/j.jbi.2012.02.009. [DOI] [PubMed] [Google Scholar]
- 12.Alharthi H, Youssef A, Radwan S, Al-Muallim S, Zainab AT. Physician satisfaction with electronic medical records in a major Saudi Government hospital. Journal of Taibah University Medical Sciences. 2014;9(3):213–8. https://doi.org/10.1016/j.jtumed.2014.01.004. [Google Scholar]
- 13.Wang JY, Ho HY, Chen JD, Chai S, Tai CJ, Chen YF. Attitudes toward inter-hospital electronic patient record exchange:discrepancies among physicians, medical record staff, and patients. BMC health services research. 2015;15(1):264. doi: 10.1186/s12913-015-0896-y. https://doi.org/10.1186/s12913-015-0896-y PMCid:PMC4499194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lu CH, Hsiao JL, Chen RF. Factors determining nurse acceptance of hospital information systems. CIN:Computers, Informatics, Nursing. 2012;30(5):257–64. doi: 10.1097/NCN.0b013e318224b4cf. https://doi.org/10.1097/NCN.0b013e318224b4cf. [DOI] [PubMed] [Google Scholar]
- 15.Tavakoli N, Jahanbakhsh M, Shahin A, Mokhtari H, Rafiei M. Electronic medical record in central polyclinic of isfahan oil industry:a case study based on technology acceptance model. Acta Informatica Medica. 2013;21(1):23–25. doi: 10.5455/AIM.2012.21.23-25. https://doi.org/10.5455/aim.2012.21.23-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Abdekhoda M, Ahmadi M, Gohari M, Noruzi A. The effects of organizational contextual factors on physicians'attitude toward adoption of Electronic Medical Records. Journal of biomedical informatics. 2015;53:174–9. doi: 10.1016/j.jbi.2014.10.008. https://doi.org/10.1016/j.jbi.2014.10.008. [DOI] [PubMed] [Google Scholar]
- 17.Saghaeiannejad-Isfahani S, Jahanbakhsh M, Habibi M, Mirzaeian R, Nasirian M, Rad JS. A Survey on the Users'Satisfaction with the Hospital Information Systems (HISs) based on DeLone and McLean's Model in the Medical-Teaching Hospitals in Isfahan City. Acta Informatica Medica. 2014;22(3):179–82. doi: 10.5455/aim.2014.22.179-182. https://doi.org/10.5455/aim.2014.22.179-182. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Goodhue DL, Thompson RL. Task-technology fit and individual performance. MIS quarterly. 1995:213–36. https://doi.org/10.2307/249689. [Google Scholar]
- 19.Garcia-Smith D, Effken JA. Development and initial evaluation of the clinical information systems success model (CISSM) International Journal of Medical Informatics. 2013;82(6):539–52. doi: 10.1016/j.ijmedinf.2013.01.011. https://doi.org/10.1016/j.ijmedinf.2013.01.011. [DOI] [PubMed] [Google Scholar]
- 20.Shackel B. Usability-context, framework, definition, design and evaluation. Human factors for informatics usability. 1991:21–37. [Google Scholar]
- 21.Bailey JE, Pearson SW. Development of a tool for measuring and analyzing computer user satisfaction. Management science. 1983;29(5):530–45. https://doi.org/10.1287/mnsc.29.5.530. [Google Scholar]
- 22.Pitt LF, Watson RT, Kavan CB. Service quality:a measure of information systems effectiveness. MIS quarterly. 1995:173–87. https://doi.org/10.2307/249687. [Google Scholar]
- 23.Venkatesh V, Bala H. Technology acceptance model 3 and a research agenda on interventions. Decision sciences. 2008;39(2):273–315. https://doi.org/10.1111/j.1540-5915.2008.00192.x. [Google Scholar]
- 24.Goodhue DL. Understanding user evaluations of information systems. Management science. 1995;41(12):1827–44. https://doi.org/10.1287/mnsc.41.12.1827. [Google Scholar]
- 25.Top M, Yilmaz A, Karabulut E, Otieno OG, Saylam M, Bakır S, Top S. Validation of a nurses'views on electronic medical record systems (EMR) questionnaire in Turkish health system. Journal of medical systems. 2015;39(6):67. doi: 10.1007/s10916-015-0250-2. https://doi.org/10.1007/s10916-015-0250-2. [DOI] [PubMed] [Google Scholar]
- 26.Top M, Gider Ö. Nurses'views on electronic medical records (EMR) in Turkey:An analysis according to use, quality and user satisfaction. Journal of medical systems. 2012;36(3):1979–88. doi: 10.1007/s10916-011-9657-6. https://doi.org/10.1007/s10916-011-9657-6. [DOI] [PubMed] [Google Scholar]
- 27.Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success:approach, results, and success factors. International journal of medical informatics. 2013;82(10):940–53. doi: 10.1016/j.ijmedinf.2013.05.010. https://doi.org/10.1016/j.ijmedinf.2013.05.010. [DOI] [PubMed] [Google Scholar]
- 28.Lambooij MS, Drewes HW, Koster F. Use of electronic medical records and quality of patient data:different reaction patterns of doctors and nurses to the hospital organization. BMC medical informatics and decision making. 2017;17(1):1–11. doi: 10.1186/s12911-017-0412-x. https://doi.org/10.1186/s12911-017-0412-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Calisir F, Altin Gumussoy C, Bayram A. Predicting the behavioral intention to use enterprise resource planning systems:An exploratory extension of the technology acceptance model. Management research news. 2009;32(7):597–613. https://doi.org/10.1108/01409170910965215. [Google Scholar]
- 30.Brown IT. Individual and technological factors affecting perceived ease of use of web?based learning technologies in a developing country. The Electronic Journal of Information Systems in Developing Countries. 2002;9(1):1–5. https://doi.org/10.1002/j.1681-4835.2002.tb00055.x. [Google Scholar]
- 31.Ebnehoseini Z, Tara M, Meraji M, Deldar K, Khoshronezhad F, Khoshronezhad S. Usability Evaluation of an Admission, Discharge, and Transfer Information System:A Heuristic Evaluation. Open access Macedonian journal of medical sciences. 2018;6(11):1941–45. doi: 10.3889/oamjms.2018.392. https://doi.org/10.3889/oamjms.2018.392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Asadi F, Moghaddasi H, Rabiei R, Rahimi F, Mirshekarlou SJ. The evaluation of SEPAS national project based on electronic health record system (EHRS) coordinates in Iran. Acta Informatica Medica. 2015;23(6):369–373. doi: 10.5455/aim.2015.23.369-373. https://doi.org/10.5455/aim.2015.23.369-373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Tubaishat A. Evaluation of electronic health record implementation in hospitals. CIN:Computers, Informatics, Nursing. 2017;35(7):364–72. doi: 10.1097/CIN.0000000000000328. https://doi.org/10.1097/CIN.0000000000000328. [DOI] [PubMed] [Google Scholar]