Abstract
Background: The Integrated Health Record System, locally known as the "SIB," is the most used information system for recording public health services provided to the Iranian population. The objective of this study was to evaluate the success rate of the SIB using the Clinical Information System Success Model (CISSM).
Methods: This is a psychometric and evaluation study. The CISSM has a 26-item instrument that assesses 7 constructs in 3 following stages: (1) the socio-technical stage (facilitating conditions, social influence, information quality, and system performance(; (2) the integrated stage (system use dependency and user satisfaction); and (3) success outcome stage (net benefit). A Persian version of the CISSM instrument was validated and applied in this study. Based on this instrument, the reliability and the validity of the CISSM were assessed. The SIB success rate was evaluated using a validated CISSM. The study participants were 758 SIB users from different disciplines and different levels.
Results: Assessment of content validity, construct validity, internal consistency, and test-retest reliability showed acceptable psychometric properties of the CISSM instrument. The results demonstrated that the SIB success rate was in the moderate range (59.6%). Facilitating conditions and information quality were strong predictors of use dependency and user satisfaction, while both of these 2 constructs significantly influenced net benefit.
Conclusion: The SIB success was in the moderate range, and it needs to be enhanced. Therefore, Iranian healthcare policymakers should consider working on the most important factors influencing SIB success (facilitating conditions, information quality, use dependency, and user satisfaction) to improve SIB success.
Keywords: Information System, Electronic Health Records, Evaluation Research, Psychometrics
Introduction
↑What is “already known” in this topic:
• The integrated health record system (SIB) is the most widely used information system for recording public health services, which covers more than 80% of the Iranian population.
• Despite the widespread use, the information system success of the SIB has not been systematically evaluated.
• The clinical information system success model (CISSM) is a model for the assessment of the success of clinical information in a mandatory environment (e.g. SIB). The CISSM assesses 7 constructs in 3 stages: (1) socio-technical stage; (2) integrated stage; and (3) success outcome stage.
→What this article adds:
• The Persian version of the CISSM instrument is a valid and reliable tool to be used for the evaluation of the SIB.
• The most critical factors in SIB success are “facilitating condition”, “information quality”, “CIS use dependency” and “user satisfaction”.
•SIB success rate is at a moderate level and needed to be improved by considering the above-mentioned factors.
Nowadays, using the information system in the health sector provides considerable opportunities to improve patient care quality, reduce process errors, better clinical documentation, and increase healthcare efficiency (1,2). In order to access the improvement of the health system in Iran, an Integrated Health Record System (SIB) was provided in 2015 to facilitate health service distribution, come up with the requirements for the accomplishment of the referral system, access the achievement of the data given on public health, and conclusively enhance the quality of health services. The SIB was put forward and performed in Health Centers in Iran. All health-related data of the population collected at the time of delivering primary health services are recorded in the SIB (3). At present, exceeding 73 million people in Iran have registered in the SIB. This information system has been set up in more than 36,000 urban and rural areas, employing more than 130,000 healthcare personnel (4).
Nowadays, despite many well-designed clinical systems, few are successful at the implementation stage (5), and there are some risks related to information systems in health sectors. These risks may include the high cost of these systems and the undesirable effect of inefficient information systems on personnel and patients. Accordingly, a thorough assessment of health information systems is a significant need (1). A systematic literature review has introduced various methods and frameworks for evaluating health information systems. The result of this study shows that none of these methods and frameworks alone were sufficient for a comprehensive evaluation (6). In the current study, the clinical information systems success model (CISSM) was used because assessing the success of the SIB was considered. In 2013, Garcia-Smith and Effken designed the Clinical Information Systems Success Model (CISSM), inspired by the Delone and Mclean information system success model, to evaluate clinical system success (5).
Based on the CISSM, a study was conducted to evaluate the success of an electronic health record from the viewpoint of personnel of nursery homes (7). In another study, the effectiveness of nursing information system implementation was assessed using CISSM (8).
Several study projects were carried out to evaluate the Iranian Integrated Health Record System (3,9,10). Usability evaluation (3), the study of goals’ achievement (9), acceptance of SIB among users are some of these evaluation studies (10). To the best of our knowledge, no study has yet been published for the success evaluation of SIB. Therefore, the purpose of this study was to evaluate the success rate of the Iranian Integrated Health Record System (SIB) based on the Persian version of the clinical information success model and instrument.
Methods
This study is conducted between December 2018 and August 2019 after obtaining approval from the Ethics Committee of Iran University of Medical Sciences (IR.IUMS.REC.1397.319). The Integrated Health Record System (SIB) users voluntarily participated in this research and signed written informed consent forms before entering the study.
The clinical information system success model evaluates the clinical information system's success in mandatory environments. This model, inspired by the D&M model, was designed by Garcia-Smith and Effken in 2013. There are 3 stages in the CISSM, including the sociotechnical, the integration, and the success outcome stage. The success outcome stage consists of only 1 construct called Net Benefit. Factors of the integration stage are CIS use dependency and user Satisfaction. System Performance (Reliability, Perceived Ease of Use, and Accessibility), Information Quality (Content Completeness, Perceived Usefulness, Format, and Accuracy), Social Influence (Social Support, Service Support), and Facilitating Conditions (Work Processes, Perceived Behavioral Control) are constructs of the Sociotechnical stage (5).
This study was conducted in 3 stages. In the first stage, the psychometric properties of a 26-item instrument (CISSM instrument) were evaluated (5) based on the standard protocol of writing suggested by Streiner and Kottner (11). In the second part of the study, the validity and the reliability of the CISSM were assessed according to the didactic presentation of structural equation modeling by Cristhian M. Ringle et al. (12). Finally, the success rate of the SIB was evaluated.
Psychometric Evaluation of CISSM
Psychometric evaluation of the CISSM instrument started by the forward and backward translation process based on the World Health Organization protocol (13) followed by assessing instrument's content validity using Waltz and Bausell method for content validity index (CVI) and Lawshe content validity ratio (CVR) method based on 15 medical informatics experts' data. The CVR was evaluated using a 3-point Likert scale to grade the importance of every item in the instrument. The CVI was assessed using a 4-point scale rating to grade the relevancy of each item in the instrument. Cut points of 0.49 and 0.79 were considered acceptable for CVR and CVI, respectively (14,15).
Construct validity was evaluated using an exploratory factor analysis (EFA). The EFA was assessed based on 406 SIB users' data, including physicians, nurses, midwives, and health professionals from 3 cities affiliated to Ahvaz Jundishapur University of Medical Sciences.
To assess test-retest reliability, 90 SIB users completed the instrument twice in 2 weeks, and the intraclass correlation coefficient was calculated. Internal consistency was assessed using the Cronbach alpha.
Assessing Validity and Reliab ility of CISSM
In the second part of the study, using the structural equation modeling in SmartPLS, the quality of the measurement model and hypotheses of the clinical information system success model were tested based on another sample of 379 SIB users.
CISSM hypotheses are listed below (5):
H1. System performance can measure CIS use dependency
H2. System performance can measure user satisfaction
H3. Information quality can measure CIS use dependency
H4. Information quality can measure user satisfaction
H5. Social influence can measure CIS use dependency
H6. Social influence can measure user satisfaction
H7. Facilitating condition can measure CIS use dependency
H8. Facilitating condition can measure user satisfaction
H9. User satisfaction and CIS use dependency can predict each other
H10. User satisfaction can predict net benefit
H11. CIS use dependency can predict net benefit
To assess measurement model quality, the first step was examining convergent validity by observing the average variance extracted (AVEs) (12). When AVEs are greater than 0.5, the results of model convergence will be satisfactory (16). Observing the Cronbach alpha and composite reliability as values for internal consistency was the next stage. Both values should be greater than 0.7 (12). The discriminant validity of the structural equation modeling can be assessed by Fornell and Larcker criteria. In this method, AVE's square root in every construct should be greater than between-construct correlations (17). R2 indicates adjusted model quality (12); according to Cohen’s suggestion, models with R2 more than 26% have a large effect (18). The Stone-Geisser indicator (Q2) and Cohen’s indicator (f2) are model accuracy indicators that should be more than 0 and 0.35, respectively (12). Finally, the goodness-of-fit indicator for the observed model was calculated by measuring the square root of R2 and AVE's product for each latent variable. The adequate amount of the GOF is 0.36 (19). For hypotheses testing, path coefficient and T statistics were considered (12). CISSM hypotheses are illustrated in Figure 1.
Measurement of SIB Success Rate
Finally, the data used in the EFA and model validation were combined. This data, consisting of 785 SIB users, was used to evaluate the SIB success rate from the users' perspective. The total scores obtained from the whole questionnaire to the sum of maximum points that can be obtained were considered as success rate. The result of the success rate based on the user’s perspective was categorized as follows: first (under 25%), second, third, and fourth quartile considered poor, low, moderate, and appropriate success rate, respectively (20). In this study, comparison of CISSM constructs in gender, educational and occupational groups was done.
Statistical Analysis
The EFA was analyzed through SPSS 20. Structural equation modeling was done using SmartPLS M2.
Results
Psychometric Evaluation of CISSM
Based on CVR and CVI results, all the items were necessary and relevant, and no items were removed from the instrument. The 406 participants' age ranged from 20 to 59 years, with a mean of 36.4 (SD: 7.4). A total of 260 participants (64%) were female; 58% of the participants were health workers, 19.5% midwives, 12% physicians, and 10.5% were of other professions. Using the EFA, 7 factors with 21 items were identified. These factors were called "facilitating condition" (7 items), "social influence" (5 items), "information quality" (3 items), "system performance" (3 items), “user satisfaction” (1 item), “system use dependency” (1 item), and “net benefit” (1 item). According to the study results, because the Cronbach alpha and ICC results were in acceptable range, internal consistency and reproducibility were approved (Table 1).
Table 1. Items’ Characteristics .
Factors | Items | SD1 | CR2 | CA3 | AVE4 | ICC5 for test & retest |
Facilitating condition | q23: Completion of admission assessment | 0.860 | 0.918 | 0.896 | 0.619 | 0.906 |
q19: I can document close to the time of patient encounters | 0.840 | |||||
q20: I can document close to the location of patient encounters | 0.836 | |||||
q21: I can make referrals to other services are efficiently | 0.796 | |||||
q22: I am usually able to complete the assessment documentation within 24 h | 0.786 | |||||
q18: The CIS is compatible with other systems I use | 0.735 | |||||
q17: I have the knowledge necessary to use the CIS | 0.629 | |||||
Social influence | q14: Service Support staffs understand the specific needs of clinicians | 0.908 | 0.921 | 0.892 | 0.702 | 0.808 |
q13: Service Support staffs are always willing to help clinicians | 0.900 | |||||
q15: Service Support staffs provide enough training for clinicians | 0.881 | |||||
q9: My supervisor has been helpful in the use of the CIS | 0.751 | |||||
q11: The "super-user" on my unit has been helpful in the acceptance of the CIS | 0.732 | |||||
Information quality | q7: The CIS information is presented in a useful format | 0.890 | 0.906 | 0.845 | 0.763 | 0.855 |
q6: The CIS provides reports that seem to be just about exactly what I need | 0.883 | |||||
q8: The CIS is accurate | 0.847 | |||||
System performance | q2: The CIS can be counted on to be “up” and available when I need it | 0.891 | 0.866 | 0.768 | 0.684 | 0.687 |
q1: The CIS is easy to use | 0.850 | |||||
q4: The CIS is not subject to frequent problems and crashes | 0.731 | |||||
CIS use dependency | q24: I am dependent on the CIS to document assessments, plan and monitor patient care | 1.000 | 1.000 | 1.000 | 1.000 | 0.762 |
User satisfaction | q25: Overall, I am satisfied with the CIS | 1.000 | 1.000 | 1.000 | 1.000 | 0.793 |
Net benefit | q26: I believe that the CIS helps me to comply with hospital nursing documentation quality standards |
1.000 | 1.000 | 1.000 | 1.000 | 0.821 |
1 Standard loadings, 2 Composite reliability, 3 Cronbach alpha, 4 Average variance extracted, 5Intraclass correlation coefficient
Assessing Validity and Reliability of CISSM
In the model validation process, according to the results of Table 1, all of the AVEs were greater than 0.5, so convergent validity was satisfactory in all latent variables. Because the Cronbach alpha and composite reliability for all factors were larger than 0.7, the model's internal consistency was acceptable (Table 1). Table 2 is a Fornell and Larcker table. In this table, the AVE's square root in every construct was higher than between-construct correlations, and thus discriminant validity was provided.
Table 2. Fornell and Larcker Table .
Latent Variable | CIS use Dependency |
Net Benefit | User Satisfaction |
Facilitating Conditions | Information Quality | Social Influence | System Performance |
CIS use Dependency | 1 | ||||||
Net Benefit | 0.739 | 1 | |||||
User Satisfaction | 0.716 | 0.818 | 1 | ||||
Facilitating Conditions | 0.733 | 0.766 | 0.785 | 0.816 | |||
Information Quality | 0.581 | 0.628 | 0.694 | 0.647 | 0.874 | ||
Social Influence | 0.43 | 0.542 | 0.563 | 0.595 | 0.618 | 0.838 | |
System Performance | 0.588 | 0.635 | 0.705 | 0.74 | 0.653 | 0.591 | 0.827 |
The adjusted model had good quality because of R squares more than 26% (Table 3). Based on the result of the Stone-Geisser indicator (Q2) and the Cohen’s indicator (f2), model accuracy was perfect (Table 3). Good goodness-of-fit model index is seen in Table 3.
Table 3. Model Fitness Index .
Index | Net Benefit |
CIS Use Dependency |
User Satisfaction |
Facilitating Conditions |
Information Quality | Social Influence |
System Performance |
(f2) | 1 | 1 | 1 | 0.512 | 0.479 | 0.531 | 0.363 |
(Q2)2 | 0.697 | 0.569 | 0.658 | ||||
GOF | 0.829 | 0.773 | 0.847 | ||||
(R2) | 0.717 | 0.597 | 0.688 |
1 Cohen’s indicator, 2 Stone-Geisser indicator
According to the T statistics of path coefficients, all hypotheses were approved except relationship among social influence and user satisfaction and relationship among system (Goodness of fit indicator) performance and CIS use dependency (Fig. 1).
a. Measurement of SIB Success Rate
According to the result of this study, the SIB success rate from the users' perspective was moderate (59.6%).
According to Appendix Table 1, there is a significant mean difference between SIB users with a doctoral degree and other educational groups in all factors except social influence. Appendix Table 2 shows the success scores in the occupational groups. According to this table, no specific pattern can be identified for the results of the mean differences. Appendix Table 3 shows that the mean scores of females are higher than those in males in all factors, but these differences are not significant.
Discussion
Main Findings
This study aimed to evaluate the success rate of the Iranian Integrated Health Record System (SIB). To the best of our knowledge, the present study is the first to assess SIB success. The instrument used in this study was the CISSM instrument. The current study showed that the CISSM instrument has good validity and reliability in the SIB users' population. According to this study's results, the validity and the reliability of the clinical information system success model in the SIB users’ population were acceptable, and this model is useful to evaluate the success of the SIB. The most important predictors of the CISSM for SIB success were facilitating condition, information quality, system use dependency, and user satisfaction. According to users’ opinion, the SIB success rate was in the moderate range (59.6%). Since system support (the concept of Social Influence) is quite beneficial for the SIB, social influence scores are higher than other factors in other educational groups. Consequently, there is no significant mean difference between users with a doctoral degree and other users in this factor.
Specifically, facilitating condition had a significant and positive effect on user satisfaction and CIS use dependency (Fig. 1). These study findings revealed that the facilitating condition had the most impact on the SIB success in the socio-technical stage. It means that for the Iranian health policymakers to enhance the SIB success, the facilitating condition (record the data in a time and place close to the visiting location, referrals to other services, help the user to organize their work, and providing the necessary knowledge to users) must be given most extreme consideration. The significant effect of the facilitating condition on the integration stage is consistent with the study of Sebetci and Cetin, but the facilitating condition could only predict user satisfaction in the original CISSM study (5,21).
In the current study, social influence significantly influences CIS use dependency but not user satisfaction (Fig. 1). It shows that having good service support, helpful supervisors, and supportive organs can lead to CIS use dependency. This finding is in line with the original CISSM study (5). In a research by Sebetci and Cetin, social influence had a significant effect on user satisfaction, but its impact on CIS use dependency was not significant (21).
Information quality had a significantly positive effect on both constructs of the integrated stage, but its influence is weaker than the facilitating condition (Fig. 1). In this way, exact reports, useful format, and accuracy are predictive for user satisfaction and CIS use dependency. It is consistent with the original CISSM study (5). Sebetci and Cetin research demonstrate that information quality has only a significant influence on CIS use dependency (21). The Cronbach alpha for information quality was 0.72 and 0.61 in Tabesh and Tara study. The correlation between this construct and user satisfaction was significant in Tara study (correlation coefficient was 0.319) (20,22).
Similar to the original CISSM study, our finding revealed that system performance could influence only user satisfaction (Fig. 1). It shows being available, easy to use, and not having frequent problems is a forecasting factor for user satisfaction. The influence of system quality was significant on user satisfaction and was not significant on CIS use dependency in Sebetci and Cetin study (21). In Tabesh study, the Cronbach alpha was 0.74 for this construct. The relationship between system performance and other constructs was not evaluated (20). The Cronbach alpha was 0.72 in Tara study for system performance, and its correlation to user satisfaction was significant (correlation coefficient was 0.32) (22).
Net benefit in the success outcome stage is the users’ perception of how much the SIB helps them do standard documentation (5). Both latent variables in the integration stage (user satisfaction and CIS use dependency) significantly influence net benefit, but user satisfaction is a stronger predictor. This is consistent with the results of the original CISSM study and Sebetci and Cetin (5,21). It demonstrates that the degree of user satisfaction and user integration into the SIB in the health system can predict SIB success. The correlation between user satisfaction and net benefit was significant (correlation coefficient was 0.41) in Tara study (22).
Strengths and Limitations
The present study is the first study to evaluate the Iranian integrated health record system (SIB) success rate from the users’ perspective. For this purpose, the validity and the reliability of the clinical information system success model and its instrument were evaluated in the SIB users’ population. Furthermore, the users who participated in this study were SIB users from different professions. There are some limitations in this research. This study was performed on users of just 3 cities affiliated to Jundishapur University of Medical Sciences. It is necessary to use the opinions of more users from different universities to measure this clinical information system's success. Besides, in this study, convenience sampling was used, and future studies require random sampling.
Conclusion
According to the results of this study, SIB success rate from the viewpoint of SIB users was in the moderate range and needs to be enhanced. Therefore, Iranian healthcare policymakers should consider working on the most important factors influencing SIB success (facilitating conditions, information quality, use dependency, and user satisfaction) to improve SIB success.
Acknowledgement
We would like to express our appreciation to health officers of Ahvaz Jundishapur University of Medical Sciences for their participation and support in this study. We are also grateful to the CISSM creators for granting permission to adapt the Persian version of CISSM instrument for this study.
Etical Approval
This study approved by the Ethics Committee of Iran University of Medical Sciences (IR.IUMS.REC.1397.319).
Conflict of Interests
The authors declare that they have no competing interests.
Appendix
Appendix Table 1. Standardized mean differences of success score between education groups .
Factor | Education | PhD | Master | Bachelor | Associate degree | Diploma |
Facilitating Conditions | PhD | 0 | ||||
Master | 0.68 (0.22, 1.12) | 0 | ||||
Bachelor | 0.66 (0.31, 0.99) | -0.05 (-0.43, 0.33) | 0 | |||
associate degree | 0.56 (0.17, 0.94) | -0.11(-0.52, 0.31) | -0.06 (-0.37, 0.25) | 0 | ||
Diploma | 0.60 (0.27, 0.95) | -0.02 (-0.40, 0.36) | 0.02 (-0.23, 0.28) | 0.08 (-0.23, 0.39) | 0 | |
Social Influence | PhD | 0 | ||||
Master | 0.13 (-0.31, 0.57) | 0 | ||||
Bachelor | 0.14 (-0.19, 0.47) | 0.02 (-0.36, 0.40) | 0 | |||
associate degree | 0.14 (-0.24, 0.52) | 0.02 (-0.40, 0.44) | -.002 (-0.31, 0.31) | 0 | ||
Diploma | 0.00 (-0.33, 0.33) | -.11 (-0.50, 0.27) | -0.13 (-0.39, 0.12) | -0.13 (-0.44, 0.18) | 0 | |
Information Quality | PhD | 0 | ||||
Master | 0.13 (-0.31, 0.57) | 0 | ||||
Bachelor | 0.36 (0.02, 0.69) | 0.21(-.17,.59) | 0 | |||
associate degree | 0.60 (0.21, 0.98) | 0.42(-.01,.84) | 0.21(-.09,.52) | 0 | ||
Diploma | 0.79 (.44, 1.13) | 0.62(.23, 1.10) | 0.43(.17,.69) | 0.23(-.08,.54) | 0 | |
System Performance |
PhD | 0 | ||||
Master | 0.54 (0.09, 0.98) | 0 | ||||
Bachelor | 0.49 (0.15, 0.83) | -0.06 (-0.44, 0.32) | 0 | |||
associate degree | 0.55 (0.17, 0.93) | 0.03 (-0.39, 0.45) | 0.09 (-0.22, 0.40) | 0 | ||
Diploma | 0.65 (0.31, 0.99) | 0.15 (-0.23, 0.53) | 0.20 (-0.05, 0.46) | 0.12 (-0.19, 0.43) | 0 | |
Use Dependency | PhD | 0 | ||||
Master | 0.50 (0.05, 0.94) | 0 | ||||
Bachelor | 0.72 (0.38, 1.06) | 0.15 (-0.23, 0.53) | 0 | |||
associate degree | 0.69 (0.30, 1.07) | 0.16 (-0.26, 0.58) | 0.03 (-0.28, 0.33) | 0 | ||
Diploma | 0.74 (0.39, 1.08) | 0.23 (-0.16, 0.61) | 0.10 (-0.15, 0.36) | 0.07 (-0.24, 0.38) | 0 | |
User Satisfaction | PhD | 0 | ||||
Master | 0.44 (0.00,0.88) | 0 | ||||
Bachelor | 0.49 (0.15, 0.83) | 0.02 (-0.36, 0.40) | 0 | |||
associate degree | 0.74 (0.35, 1.13) | 0.29 (-0.14, 0.71) | 0.26 (-0.05, 0.57) | 0 | ||
Diploma | 0.87 (0.52, 1.21) | 0.42 (0.03, 0.80) | 0.4 (0.14, 0.66) | 0.14 (-0.16, 0.46) | 0 | |
Net Benefit | PhD | 0 | ||||
Master | 0.60 (0.15, 1.05) | 0 | ||||
Bachelor | 0.48 (0.14, 0.81) | 0.15 (-0.53, 0.23) | 0 | |||
associate degree | 0.60 (0.21, 0.98) | -0.04 (-0.46, 0.38) | 0.12 (-0.19, 0.43) | 0 | ||
Diploma | 0.74 (0.39, 1.10) | 0.11 (-0.27, 0.49) | 0.27 (0.01, 0.52) | 0.15 (-0.16, 0.46) | 0 |
Appendix
Appendix Table 2. Standardized mean differences of success score between occupation groups .
Factor | Occupation | Physician | Midwife | Health Worker | Health Home Worker | Other |
Facilitating Conditions | Physician | 0 | ||||
Midwife | 0.44 (0.06,0.81) | 0 | ||||
Health Worker | 0.78 (0.40,1.16) | 0.33 (0.00,0.65) | 0 | |||
Health Home Worker | 0.65 (0.31, 0.99) | 0.25 (-0.03, 0.53) | -0.05 (-0.33, 0.23) | 0 | ||
Other | 0.59 (0.14, 1.03) | 0.14 (-0.25, 0.53) | -0.21 (-0.60, 0.18) | -0.12 (-0.48,0.23) | 0 | |
Social Influence |
Physician | 0 | ||||
Midwife | 0.01 (-0.36,0.38) | 0 | ||||
Health Worker | 0.13 (-0.24,0.50) | 0.11 (-0.21, 0.43) | 0 | |||
Health Home Worker | 0.00 (-0.33,0.33) | -0.00 (-0.28, 0.27) | -0.11 (-0.39, 0.17) | 0 | ||
Other | 0.16 (-0.27,0.59) | 0.13 (-0.26, 0.52) | 0.02 (-0.37, 0.41) | 0.13 (-0.23, 0.49) | 0 | |
Information Quality | Physician | 0 | ||||
Midwife | 0.27 (-0.09,0.64) | 0 | ||||
Health Worker | 0.31 (-0.06,0.68) | 0.03 (-0.29, 0.35) | 0 | |||
Health Home Worker | 0.75 (0.40,10.08) | 0.48 (0.20, 0.76) | 0.46 (0.17, 0.74) | 0 | ||
Other | 0.11 (-0.32,0.54) | -0.16 (-0.55, 0.23) | -0.19 (-0.58, 0.20) | -0.63 (-0.99,0.26) | 0 | |
System Performance |
Physician | 0 | ||||
Midwife | 0.27 (-0.10,0.64) | 0 | ||||
Health Worker | 0.63 (0.25,1.00) | 0.35 (0.03, 0.67) | 0 | |||
Health Home Worker | 0.68 (0.34,1.02) | 0.43 (0.15, 0.71) | 0.12 (-0.16, 0.40) | 0 | ||
Other | 0.34 (-0.10,0.77) | 0.07 (-0.32, 0.46) | -0.29 (-0.68, 0.10) | -0.37 (-0.73,-0.0) | 0 | |
Use Dependency |
Physician | 0 | ||||
Midwife | 0.52 (0.15, 0.90) | 0 | ||||
Health Worker | 0.65 (0.28, 1.03) | 0.15 (-0.17, 0.47) | 0 | |||
Health Home Worker | 0.74 (0.40, 1.08) | 0.26 (-0.02, 0.54) | 0.12 (-0.16, 0.40) | 0 | ||
Other | 0.35 (-0.08,0.79) | -0.17 (-0.56, 0.22) | -0.31 (-0.71, 0.08) | -0.41 (-0.77,-0.05) | 0 | |
User Satisfaction |
Physician | 0 | ||||
Midwife | 0.37 (0.00, 0.74) | 0 | ||||
Health Worker | 0.54 (0.17, 0.92) | 0.19 (-0.13, 0.51) | 0 | |||
Health Home Worker | 0.78 (0.44, 1.12) | 0.44 (0.16, 0.72) | 0.25 (-0.03, 0.53) | 0 | ||
Other | 0.37 (-0.06,0.80) | 0.00 (-0.39,0.39) | -0.19 (-0.58, 0.20) | -0.44 (-0.80,-0.08) | 0 | |
Net Benefit | Physician | 0 | ||||
Midwife | 0.34 (-0.03,0.71) | 0 | ||||
Health Worker | 0.46 (0.09, 0.83) | 0.14 (-0.18, 0.46) | 0 | |||
Health Home Worker | 0.64 (0.30, 0.98) | 0.32 (0.04, 0.60) | 0.17 (-0.11, 0.45) | 0 | ||
Other | 0.30 (-0.13, 0.73) | -0.04 (-0.43, 0.35) | -0.18 (-0.57, 0.21) | -0.35 (-0.71,0.00) | 0 |
Appendix
Appendix Table 3. Standardized mean differences of success score between genders .
Factor | Gender |
Facilitating Conditions | -0.24 (-1.65, 1.16) |
Social Influence | -0.28 (-1.38, 0.83) |
Information Quality | -0.06 (-0.71, 0.59) |
System Performance | -0.43 (-1.06, 0.20) |
Cis Use Dependency | -0.02 (-0.26, 0.21) |
User Satisfaction | -0.23 (-0.49, 0.03) |
Net Benefit | -0.10 (-0.33, 0.14) |
Cite this article as: Bitaraf S, Janani L, Hajebi A, Motevalian SA. Information System Success of the Iranian Integrated Health Record System Based on the Clinical Information System Success Model. Med J Islam Repub Iran. 2022 (23 Mar);36:25. https://doi.org/10.47176/mjiri.36.25
References
- 1.Ammenwerth E, Gräber S, Herrmann G, Bürkle T, König J. Evaluation of health information systems - Problems and challenges. Int J Med Inform. 2003;71(2–3):125–35. doi: 10.1016/s1386-5056(03)00131-x. [DOI] [PubMed] [Google Scholar]
- 2. Lemai, Bellucci E, Nguyen LT. Electronic health records implementation: An evaluation of information system impact and contingency factors. Int J Med Inform [Internet] 2014;83(11):779–96. Available from: 10.1016/j.ijmedinf.2014.06.011. [DOI] [PubMed]
- 3. Rangraz Jeddi F, Nabovati E, Bigham R, Khajouei R. Usability evaluation of a comprehensive national health information system: relationship of quality components to users’ characteristics. Int J Med Inform [Internet] 2020;133. Available from: https://www.sciencedirect.com/science/article/abs/pii/S1386505618310244. [DOI] [PubMed]
- 4. SIB- Integrated Health Record System - Danesh Parsian Company [Internet]. [cited 2020 Oct 4]. Available from: http://dapa.ir/.
- 5. Garcia-Smith D, Effken JA. Development and initial evaluation of the Clinical Information Systems Success Model (CISSM). Int J Med Inform [Internet] 2013;82(6):539–52. Available from: https://www.sciencedirect.com/science/article/abs/pii/S1386505613000221. [DOI] [PubMed]
- 6. Eslami Andargoli A, Scheepers H, Rajendran D, Sohal A. Health information systems evaluation frameworks: A systematic review. Int J Med Inform [Internet] 2017;97:195–209. Available from: 10.1016/j.ijmedinf.2016.10.008. [DOI] [PubMed]
- 7.Yu P, Qian S. Developing a theoretical model and questionnaire survey instrument to measure the success of electronic health records in residential aged care. PLoS One. 2018;13(1):1–18. doi: 10.1371/journal.pone.0190749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tsai HH, Chiou SF, Wang TC, Wu MP, Feng RC. An empirical study on the successful implementation of nursing information system. Stud Health Technol Inform. 2016;225:846–7. [PubMed] [Google Scholar]
- 9. F. Hazhir, M. Jahanbakhsh, H.R. Peikari SS-I. Investigating the Realization of the Consequences of Using the Integrated Health System (SIB) from Users’ Point of View in Primary Healthcare Centers in Isfahan Iran - Journal of Health System Research. J Heal Syst Res [Internet]. 2020 [cited 2020 Dec 30];16(2):136–42. Available from: http://hsr.mui.ac.ir/browse.php?a_id=1126&sid=1&slc_lang=en.
- 10. Khammarnia M, Setoodehzadeh F, Peyvand M, Setayesh AH, Rezaei K, KordTamini A, et al. Evaluation of Integrated Health System Technology Acceptance among the users of Health Centers of Zahedan University of Medical Sciences: Iran. Evid Based Heal Policy, Manag Econ [Internet] 2019;3(3):154–61. Available from: http://jebhpme.ssu.ac.ir/browse.php?a_code=A-10-129-2&slc_lang=en&sid=1.
- 11. Streiner DL, Kottner J. Recommendations for reporting the results of studies of instrument and scale development and testing. J Adv Nurs [Internet] 2014;70(9):1970–9. Available from: https://onlinelibrary.wiley.com/doi/full/10.1111/jan.12402. [DOI] [PubMed]
- 12. Ringle CM, Da Silva D, Bido DDS. STRUCTURAL EQUATION MODELING WITH THE SMARTPLS ABSTRACT. Brazilian J Mark [Internet]. 2014 [cited 2021 Jan 1];13(2):56–73. Available from: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2676422.
- 13. World Health Organization. Process of translation and adaptation of instruments. Http://WwwWhoInt/Substance_Abuse/Research_Tools/Translation/En/ 2014;4–7.
- 14. Waltz CF, Bausell RB. Nursing research: Design, statistics, and computer analysis. FA Davis Co; 1981.
- 15. LAWSHE CH. a Quantitative Approach To Content Validity. Pers Psychol [Internet] 1975;28(4):563–75. Available from: http://caepnet.org/~/media/Files/caep/knowledge-center/lawshe-content-validity.pdf.
- 16. Henseler J, Ringle CM, Sinkovics RR. The use of partial least squares path modeling in international marketing. Adv Int Mark [Internet]. 2009 [cited 2021 Jan 1];20:277–319. Available from: https://www.emerald.com/insight/content/doi/10.1108/S1474-7979(2009)0000020014/full/html.
- 17. Fornell C, Larcker DF. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. J Mark Res [Internet]. 1981 Feb 28 [cited 2021 Jan 1];18(1):39–50. Available from: https://journals.sagepub.com/doi/abs/10.1177/002224378101800104.
- 18. Cohen J. Statistical power analysis for the behavioral sciences [Internet]. 2nd ed. New York: Psychology Press; 1988. Available from: http://lab.rockefeller.edu/cohenje/PDFs/159CohenBigFishLittleFishTheSciences1989.pdf.
- 19. Tenenhaus M, Vinzi VE, Chatelin YM, Lauro C. PLS path modeling. Comput Stat Data Anal [Internet]. 2005 Jan 1 [cited 2021 Jan 2];48(1):159–205. Available from: https://www.sciencedirect.com/science/article/abs/pii/S0167947304000519.
- 20. Ebnehoseini Z, Jangi M, Tara M, Tabesh H. Investigation the success rate of hospital information system (HIS): Development of a questionnaire and case study. J Healthc Qual Res [Internet]. 2021 Jan 23 [cited 2021 Feb 13];10. Available from: https://www.sciencedirect.com/science/article/abs/pii/S2603647920301202. [DOI] [PubMed]
- 21. Sebetci Ö, Çetin M. Developing, applying and measuring an e-Prescription Information Systems Success Model from the persperctives of physicians and pharmacists. Heal Policy Technol [Internet] 2016;5(1):84–93. Available from: https://www.sciencedirect.com/science/article/abs/pii/S2211883715000830.
- 22.Ebnehoseini Z, Tabesh H, Deldar K, Mostafavi SM, Tara M. Determining the hospital information system (His) success rate: Development of a new instrument and case study. Open Access Maced J Med Sci. 2019;7(9):1407–14. doi: 10.3889/oamjms.2019.294. [DOI] [PMC free article] [PubMed] [Google Scholar]