Understanding the acceptance factors of an Hospital Information System: evidence from a French University Hospital

Abstract

The goal of this study was to examine the perceived usefulness, the perceived ease of use and the perceived behavioral control of a Hospital Information System (HIS) for the care staff. We administrated a questionnaire composed of open-end and closed questions, based on the main concepts of Technology Acceptance Model. As results, the perceived usefulness, ease of use and behavioral control (self-efficacy and organizational support) are correlated with medical occupations. As an example, we found that a half of the medical secretaries consider the HIS is ease of use, at the opposite to the anesthesiologists, surgeons and physicians. Medical secretaries reported also the highest rate of PBC and a high rate of PU. Pharmacists reported the highest rate of PU but a low rate of PBC, which is similar to the rate of the surgeons and physicians. Content analysis of open questions highlights factors influencing these constructs: ergonomics, errors in the documenting process, insufficient compatibility with the medical department or the occupational group. Consequently, we suggest that the gap between the perceptions of the different occupational groups may be explained by the use of different modules and by interdependency of the care stare staff.

Introduction

The implementation of Healthcare Information Systems, such as Computerized Physicians Order Entry (CPOE), Clinical Information Systems (CIS) or Electronic Medical Records (EMR) and, more recently, Hospital Information System (HIS), is supposed to have various benefits for the medical practices, as providing easy access to documentation of patients records and accurate them^{1, 2}, billing management³, reducing potential medical errors⁴, and improving the quality of patient care³. However, previous studies have shown the use of HIS has led to unintended consequences in the actual work practices, such as increased documentation time^5,6, incompatibility with clinical workflow⁵, increasing more interruptions in medical work⁶ and system-introduced errors in patients care^7,8. How can we explain the gap between the expected benefits and the conclusions of these different studies? We suggest an explanation related to the fact that the acceptance factors are not taken enough into account by the hospital management during the post implementation stage. We argue this idea based on a survey driven in a French University hospital using a HIS. This survey is based on the main concepts of the Technology Acceptance Model (TAM)⁹.

Theoretical framework

TAM is considered a “gold standard” in the information systems research¹⁰. This model aims to identify determinants (perceived ease of use and usefulness) of attitude and usage intention. While it provides useful insights into why people adopt a technology, this model and its evolutions (known as TAM2 and TAM3¹¹), has several limitations. First, it works especially in voluntary contexts and not mandatory ones^{12, 13}. Thus, intention to use is not a relevant concept in mandatory contexts.

Second, it assumes that users face no impediments in the course of information system uses. Besides, users often do not have a total control over situations of use, especially as employees. This argument led TAM authors to improvement in the initial model¹¹.

Third, the parsimony of the TAM concepts, whereas it was considered for a long time as a strengthens of the model, has been evaluated recently as weakness because its misfit to different organizational contexts¹⁴.

Fourth, these concepts are linked to binary variables (use or not use), which may be described for simple technologies uses^{9, 15} but not for complex ones. However, complex information systems, including Enterprise Resource Planning (ERP) and, consequently, HIS, give rise to more or less advanced uses. Some features may be used, while others not. In addition, the professional context of usage is not considered enough¹⁶.

Fifth, the main potential benefit of TAM, which is its explanatory and predictive power, seems to be partial¹⁴. Sixth, previous research in the healthcare, based on the TAM use and focused on physicians, have found a positive relationship between perceived utility and acceptance but a non-significant relationship between ease of use and acceptance or ease of use and intention to use^{17, 18}.

We suggest that these limitations are related to the deterministic use of the TAM, that is, measures of causality relations between the determinants, the attitude of acceptance and the intention of use. Beyond this deterministic use, the main construct of TAM may provide better understanding about HIS acceptance. In order to take into account the criticism we have summarized above, we add the construct of perceived behavioral control, which is driven by the theory of the planned behavior and has been used in the healthcare¹⁷. This construct includes perceived internal control, which is named self-efficacy by the TAM3 authors, and perceived external control, which is linked to another concept of TAM3, that is, facilitating conditions¹¹.

The table below summarizes the constructs of the survey and their definitions.

Methodology and context

We designed a questionnaire based on these three constructs. We used the items tested by Chau & Hu¹⁹. All items were measured in a 7-point Likert’s scale, with 1 as strongly disagree and 7 as strongly agree. We added two open questions in order to investigate users’ perceptions about the advantages and the disadvantages of this information system.

This questionnaire was sent for pre-test to a panel of 20 volunteers. This pre-test and adaptation of the questionnaire prevented the definition of indicators according to the representations of researchers and thus allowed the contextualization of the issues¹⁹. The main adaptation consists on the contextualization of the HIS name: as suggested by the pre-test answers, we decided to use the software’s name instead of the generic term “HIS”.

The questionnaire was developed and administered online to the care staff, during the month of December 2013.

The survey was conducted in a large French University hospital. The target was composed by the care staff (9 000 employees with care occupations). The aim was to measure the users’ perceptions related to the daily use of the HIS, 18 months after the HIS implementation. This information system includes computerized physicians order entry, medical and nursing observation, laboratory tests results, medical prescription, operating room process management, drugs logistics management, consultation program and appointment, billing management.

Data analysis and results

1,942 questionnaires were collected. The distribution of the occupations is detailed in the table below:

We use the across-method triangulation²⁰, consisting in the combination of three methods:

• Statistical correlations between the responses to the closed questions, which were assessed by Chi-Square test, using the Sphinx software.

• Content analysis using a French textual data analysis software named Alceste, which identifies classes of speech.

• Content analysis of the open answers, which were coded independently by two researchers (the reliability was measured by the kappa coefficient).

We also used internal validation²¹, which consists in the “truth value” of the findings and their credibility for the care staff. We presented the findings to the hospital management, to the medical managers of the hospital units or services and to the care staff.

We found a very significant correlation of all the constructs with occupations. We resumes bellow the main rates (1–3 for disagree and 5–7 for agree. We do not mention the middle rate (4 on the Lickert’s scaledivided):

1. Item Perceived Ease of Use (PEOU): chi2 = 160,48, ddl = 30, P< 0.01, Cramer’s V : 20,42%. Generally, 48% of the staff care agree (vs. 35% disagree) with the statements related to the construct “the HIS is ease of use”. The majority of anesthesiologists (61%), of physicians (56%), of midwives (55%) and of surgeons (52%), and of the residents (51%), the half of nurses, 45% of care managers, 39% of auxiliary nurses, 35% of physiotherapists disagree. Only 25% of a anesthesiologists, 27% of physicians, 27% of midwives, 30% of surgeons, 33% of nurses, 35% of residents, 37% of care managers, 42% of auxiliary nurses, 45% of physiotherapists agree with this statement. Furthermore, 50% of the medical secretaries (vs 32%) consider the HIS is ease of use. Pharmacists are more divided: 41% disagree and 40% agree.

2. Item Perceived Utility (PU): chi2 = 168,61, ddl = 30, P< 0.01, Cramer’s V : 19,65%. The staff care consider the software is useful or very useful (42% agree vs. 34% disagree). More precisely, 48% of auxiliary nurses (vs. 31%), 38% of anesthesiologists (vs. 38%), 57% of care managers (vs.13%), 47% of surgeons (vs. 27%), 44% of residents (vs. 27%), 41% of physicians (vs. 38%), 43% of physiotherapists (vs. 27%), 74% of pharmacists (vs. 15%), 31% of midwives (vs. 23%), 63% of medical secretaries (vs. 16%) agree with the statement “the HIS is useful”. Besides, nurses (30%), midwives (31%) and anesthesiologists (38%) have the lowest rate of PU. Only the nurses disagree (46% vs 31%) with this statement. In contrast, pharmacists (71%), medical secretaries (63%), residents (61%) and nurses managers (57%) state the software is useful or very useful.

3. Item Perceived Behavioral Control (PBC): chi2 = 154,32, ddl = 30, 1−p = >99,99%. The staff care is divided (46% agree while 36% disagree), as surgeons (45% agree vs. 41% disagree). More precisely, 52% of anesthesiologists (vs. 32%) and 43% physicians (vs. 37%), 44% of pharmacists (vs. 39%), consider they have only little behavioral control or not at all. At the opposite, 65% of medical secretaries (vs. 20%), 64% of residents (vs. 34%), 63% of physiotherapists (vs. 22%), 56% of auxiliary nurses (vs. 25%), 45% of nurses (vs. 31%) and 42% of nurses managers (vs. 35%) states that they have behavioral control.

Furthermore, we identified 4 classes of speech by the Alceste software:

• Class 1 (majority) “Treatment plan”. Key words: prescription – care – treatment sheet – validation – error – transmission – treatment – sign – readability. At the center of this class are prescription and treatment plan, related to the difficulty to understand the computerized prescription and treatment sheet, which may lead to errors of treatment.

• Class 2 “Lack of ergonomics”. Key words: unfriendly – tool – person – think – Computer – true – Software – design. These speeches are related to the ergonomics software, considered as flawed. Some users describe as “a disaster” loss of staff time to the detriment of his care tasks. Software bugs are also highlighted.

• Class 3 “Laboratory results and examinations”. Key words: report – post – results – classify – archive – imaging. The speech emphasized results and schedule modules, which are described as unusable, as well as the non-readability of biological results.

• Class 4: “Find and view information”. Key words: Input – Support – File – information – structure – Psychiatry – enter – check. This class focuses on the difficulties of entering and viewing documents external to the University Hospital and the difficulty to find relevant information. Many people feel that the information is misfiled.

The difficulties related to the HIS may be classified into three categories, as follows:

• Software ergonomics. The HIS is viewed as a usability-faulty system. Overload of information on the screen drowns relevant information and increase the risk of error. Users mention also bugs and software failures.

• Insufficient use of documentation rules in the clinical workflow and misfiled information driving to errors in patient care.

• Insufficient compatibility with the clinical workflow in the departments (e.g. Hematology, Emergency, Pediatrics and especially Psychiatry) and for different medical occupations (anesthesiologists, physicians, surgeons, midwives), because the possibility of settings and customization was not effectively made.

In addition, the words occurences analysis shows that the mains words are: time – especially “waste of time” (727 occurrences); difficulties or difficult (402 occurences); waste (383 occurrences); prescriptions (257 occurrences); incompatibility – of the information system to the clinical work (199 occurrences); prescription errors (182 occurrences); training (insufficient) (164 occurrences); errors – especially, bugs (154 occurrences).

Discussion

The low score of PEOU and PBC or the anesthesiologists, the surgeons and the physicians may be explained by the answers to the open questions. Based on the classes of speech, we have to consider the prescription module and the treatment plan module as the critical issues of the HIS use. The lack of ergonomics of the prescription and laboratory results modules, related to the information overload, may lead to prescription or treatment errors (especially for the drugs dosage and their frequency). In this context, anesthesiologists assess the lowest score of PEOU because they are the clinical occupation for which the medical error (especially prescription error) may have the most rapid dramatic consequences. Basically, anesthesiologists, physicians, surgeons and midwives are the most concerned by this issue, contrary to the medical secretaries.

Otherwise, the findings highlight errors in the documenting process associated with the misuse of indexation rules for patients’ files and the insufficient compatibility of the HIS to the clinical process for each medical department.

We have to mention that ergonomics²², errors in the documenting process^{2, 6} and the insufficient compatibility with specific medical process⁸ have already been identified in the literature as a medical error risk. Our mixed method shows these factors are very connected. Furthermore, ergonomics and usability, correlated to a lower rate of PEOU, are a main issue, in contrast with studies suggesting physicians’ ability to learn to use the technology by themselves²³ and their relative disinterest in usability. Moreover, the PBC rates suggest that the medical staff (especially anesthesiologists) considers that organizational support (as training) is not sufficient during the post-implementation stage. This result is consistent with the literature on the ERP success which is conditioned by the managerial and technical support²⁴.

In addition to these issues supported by the literature, our study adds to the literature the gap between the HIS perceptions according to the occupational groups. As an example, we found that a half of the medical secretaries consider the HIS is ease of use, at the opposite to the anesthesiologists, surgeons and physicians. Moreover, they reported the highest rate of PBC and a high rate of PU. It would be easy to conclude that this result is the expression a conflict between the administration and the professional logics²⁵, knowing that accountability and planning are the main aims associated to an ERP and, consequently, to a HIS. Nevertheless, an in-depth look at the results emphasizes a gap inside the clinic and care occupational groups perceptions. Thus, pharmacists reported the highest rate of PU but they are more skeptical about the PEOU and PBC. Answers to open questions suggest that this result is related to the drugs traceability, which is on expected benefit of the HIS, under the condition of the improvement of both the ergonomics and the indexation rules for patients’ files. Otherwise, the different clinical and care occupational groups do not need to use the same features. Anesthesiologists, surgeons and physicians use the prescription module, while nurses and auxiliary nurses use the care sheet. Nurses are very impacted by the misuses of the prescription module by the clinical staff, which may explain their low rate of PU and PEOU. In line with this argument, we suggest that the most nurses state that they have PBC because the right use of prescription module is not their task. Thus, we may consider our results in relation to the use of the different modules and features according to the occupational groups (e.g. use of the office module for medical secretaries, use of the prescription and the results laboratory modules for the physicians and surgeons) and the exposure to the medical liability and medical malpractice.

Although each occupational group uses different modules or feature of the HIS, they are interdependent, meaning that an error in the information workflow may have consequences on the clinical and care professionals involved in the workflow (related to a patient). Thus, medical secretaries have to use the indexation rules for the patients’ files for physicians, surgeons and anesthesiologists to find the relevant information related to the identity and the admission file. Clinical occupational groups have to use the right features for medical observation and prescription for pharmacists to deliver drugs and for nurses to administrate dugs and care delivery. Consequently, the PU, PEOU and PBC may influence thsese constructs for the other occupational groups.

Conclusion and future directions

Our study shows that perceived usefulness, ease of use and behavioral control are correlated to medical occupations. Furthermore, it outlines three main points that may explain this correlation: ergonomics, errors in the documenting process and insufficient compatibility with specific clinical care process. Generally, information systems are implemented according to a standard vision of clinical workflow. Our research results suggest that the HIS have to be settled and customized taking into account particular clinical care process. In addition, these results can help planners, managers and healthcare software editors to understand key issues affecting HIS perceptions and use.

This research provide better understanding inside the HIS care staff perceptions. Besides, the generic items of the questionnaire, driven from TAM, may be completed by more specific items considering each feature and module of the HIS (medical observations, prescriptions, care treatments, laboratory results), as suggested by the answers to the open questions. Indeed, medical secretaries, nurses managers, nurses or physicians do not use the same features; thus, theirs perceptions and their expectancy about the HIS are different.

Further research have to provide also more insight into the various features and modules used by the medical and the care employees, in different departments, and on their interdependency in both the information and care chain.

Figure 1.

Occupational characteristics

Table 1.

Definitions of the constructs

Perceived Utility (PEU)	“the degree to which a person believes that using a particular system would enhance his or her job performance”9
Perceived Ease of Use (PEOU)	“the degree to which a person believes that using a particular system would be free of effort”9.
Perceived Behavioral Control (PBC)	“The degree to which an individual beliefs that he or she has the ability to perform specific task/job using computer” (self-efficacy) “The degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system.” (facilitating conditions)11.