Does hospital information technology infrastructure promote the implementation of clinical practice guidelines? A multicentre observational study of Japanese hospitals

Abstract

Objectives

It remains unclear whether insufficient information technology (IT) infrastructure in hospitals hinders implementation of clinical practice guidelines (CPGs) and affects healthcare quality. The objectives of this study were to describe the present state of IT infrastructure provided in acute care hospitals across Japan and to investigate its association with healthcare quality.

Methods

A questionnaire survey of hospital administrators was conducted in 2015 to gather information on hospital-level policies and elements of IT infrastructure. The number of positive responses by each respondent to the survey items was tallied. Next, a composite quality indicator (QI) score of hospital adherence to CPGs for perioperative antibiotic prophylaxis was calculated using administrative claims data. Based on this QI score, we performed a chi-squared automatic interaction detection (CHAID) analysis to identify correlates of hospital healthcare quality. The independent variables included hospital size and teaching status in addition to hospital policies and elements of IT infrastructure.

Results

Wide variations were observed in the availability of various IT infrastructure elements across hospitals, especially in local area network availability and access to paid evidence databases. The CHAID analysis showed that hospitals with a high level of access to paid databases (p<0.05) and internet (p<0.05) were strongly associated with increased care quality in larger or teaching hospitals.

Conclusions

Hospitals with superior IT infrastructure may provide higher-quality care. This allows clinicians to easily access the latest information on evidence-based medicine and facilitate the dissemination of CPGs. The systematic improvement of hospital IT infrastructure may promote CPG use and narrow the evidence-practice gaps.

Strengths and limitations of this study.

• We integrated a hospital questionnaire survey and administrative claims data at hospital level in the analysis.

• We described the present state of information technology (IT) infrastructure of Japanese acute care hospitals and investigated its association with a composite quality indicator (QI) of adherence to the guidelines for perioperative antibiotic prophylaxis, using a chi-squared automatic interaction detection analysis.

• As an infrastructure to promote evidence-based practice, we focused on IT infrastructure such as accessibility to the internet and other information sources, access to paid medical evidence databases, and medical library and intranet usability within hospitals.

• The QI that we used was limited to adherence to the guidelines for perioperative antibiotic use and therefore describes only one aspect of healthcare quality.

• As we used administrative claims data, we could not know the clinical information in detail including appropriate exceptions in clinical practice.

Introduction

Due to the growth of renewed medical information and frequent updates to clinical practice guidelines (CPGs) in the internet era, clinicians can find it difficult to keep abreast of the latest evidence. The availability and usability of hospital information technology (IT) infrastructure such as wireless local area networks (LAN) and medical evidence databases may affect the ability of clinicians to update their knowledge and practice, which can influence the quality of provided care. These infrastructure elements may facilitate accessibility to various updated CPGs, which would be essential for CPG implementation in daily practice.¹

CPGs for various diseases have been developed worldwide not only to help clinicians but also to promote shared decision making with patients.^2–4 However, CPGs continue to be underused even in countries even where CPGs are well developed over the last several decades. These gaps between medical evidence and clinical practice (ie, ‘evidence-practice gaps’) can lead to the provision of substandard or potentially harmful care to patients.^5–11 System-level barriers as well as individual-level barriers to evidence-based practice have been revealed in previous studies.^12–17 For example, institutional equipment, technological capital and accessibility to guideline-related resources have been found to be important in addition to individual awareness, familiarity and agreement with the contents.^15–17 Clinical quality indicators (QIs) can monitor clinicians’ adherence to the guidelines, but they are not necessarily utilised to assess guideline implementation.¹⁸ Furthermore, there is no clear evidence regarding whether hospital IT infrastructure may affect the quality of provided care using these QIs.

On the other hand, research related to the adoption of health IT at the organisational level is growing, and a number of studies have reported positive effects on quality, safety and efficiency.^19–21 However, most of these studies have focused on clinical decision support systems, order entry, telecommunication systems, e-prescriptions^{19 20} and strategic management systems.²¹ It thus remains unclear as to whether the lack of an adequate IT infrastructure for medical information retrieval is a crucial system-level barrier for CPG implementation. In Japan, over 180 evidence-based CPGs have been assessed and disseminated by the government-funded Medical Information Network Distribution Service (Minds) Guideline Centre²² over the last decade, but the actual use of these CPGs in daily clinical practice remains unknown.

This multicentre study aimed to describe the present state of IT infrastructure provided in Japanese acute care hospitals and to investigate its association with healthcare quality, taking into account hospital size, hospital policies promoting evidence-based practice.

Methods

Data sources

We integrated a hospital questionnaire survey and administrative claims data at hospital level in the analysis. Data were obtained from hospitals enrolled in the quality indicator/improvement project (QIP), which is an ongoing project launched in 1995 to monitor and improve clinical performance in acute care hospitals across Japan through the analysis of administrative claims data.^{23 24} Currently, over 500 QIP member hospitals voluntarily submit data for analysis, and the project generates periodic reports of clinical and economic performance. The participating hospitals vary widely in type (eg, teaching status and hospital ownership), region of location, patient and physician volume, bed numbers and composition of specialties.

The Minds-QIP project, as a part of the activities of the Minds Guideline Centre, was initiated in 2014, with the objective of effectively implementing and disseminating CPGs across Japan. A survey was conducted as part of this project by mailing questionnaires to the hospital administrators (including general managers) of QIP member hospitals between January and March 2015. The questionnaire included items on hospital policies regarding evidence-based practice and hospital IT infrastructure, actual provision of IT infrastructure (including LAN deployment and usability of medical evidence databases), QI monitoring and the use of clinical pathways. This study focused on hospital policies and IT infrastructure. The questionnaire was developed based on literature reviews, discussions with experts and semistructured face-to-face interviews with several hospital administrators and IT managers from five major teaching hospitals. Survey respondents were asked to answer questions from a concise list about their institution and policies as representatives of their hospital (see online supplementary appendix table).

Hospital policy and IT infrastructure

In order to identify each hospital’s policies promoting evidence-based practice, the questionnaire included items on whether the hospital has an explicit policy to enhance IT infrastructure intending to improve information accessibility, whether it explicitly recommends the practice of evidence-based medicine, and whether it explicitly encourages the use of CPGs.

The questionnaire was also designed to focus on the following three elements of hospital IT infrastructure: (1) accessibility to the internet and other information sources, including wired/wireless LAN availability; (2) access to paid medical evidence databases in English and Japanese; and (3) medical library and intranet usability, such as the availability of a well-organised intranet interface, number of full-time medical librarians and activities for improving the medical library.

Hospital quality of care

As a measure of hospital quality of care, a QI of adherence to CPGs for perioperative antibiotic prophylaxis was calculated using diagnosis procedure combination (DPC) administrative claims data from the QIP. The DPC is a Japanese case-mix classification system for hospital reimbursements, and >1600 hospitals nationwide had adopted this system by 2016. The DPC database includes information on hospital codes, patient demographics, admission and discharge dates, admission routes, outcomes, primary and secondary diagnoses based on International Classification of Diseases (10th revision) codes, comorbidities, complications, surgeries performed and high cost procedures.^{23 24} DPC data from April 2013 to March 2014 were used as these were the most recent data available for analysis.

The QI of interest for this study was a composite score (range: 0–100) that indicated a hospital’s proportion of adherence to CPGs for perioperative antibiotic prophylaxis,²⁵ and was aggregated from the results of the following 11 surgical types (ie, target QI in figure 1): evacuation of intracranial haematoma, gastrectomy, cholecystectomy, total hip replacement, mastectomy for breast cancer, thyroid surgery, prostate cancer surgery, uterine myoma surgery, uterine cancer surgery, benign ovarian tumour surgery and ovarian cancer surgery. The QI score was calculated based on the administrative data when and what kind of the antibiotics were used, and this medication information is very precise in the Japanese administrative data. Therefore, the QI score is accurate in any hospital.

Figure 1.

Flow diagram of the subject hospital selection process. *Target QI indicates QI of perioperative antibiotic prophylaxis of the 11 surgical procedures mentioned in the manuscript. QI, quality indicator; QIP, quality indicator/improvement project.

Statistical analysis

We first calculated descriptive statistics for the hospitals’ and respondents’ baseline characteristics, which included hospital bed numbers, teaching status, number of full-time physicians, number of resident physicians (representing younger physicians who may be more likely to incorporate IT into their practice), as well as respondent sex, age and appointment. The responses to the questionnaire items related to hospital policies and IT infrastructure were also summarised. The main items of interest consisted of yes/no questions; we calculated each hospital’s number of positive responses within each item. The hospitals were categorised into subgroups based on these response numbers, and the mean QI score was calculated for each subgroup.

Finally, we performed a chi-squared automatic interaction detection (CHAID) tree analysis to identify factors that determine hospital quality of care. The independent variables included hospital size and teaching status, hospital policies regarding the promotion of evidence-based practice, and IT infrastructure (accessibility to the internet and other information sources, access to paid medical evidence databases, and medical library and intranet usability within hospitals). The dependent variable was the mean QI score for perioperative antibiotic prophylaxis. CHAID tree analysis repeatedly uses χ² statistics to split independent variables into child nodes^26–28 to identify the relative interactions between the independent variables and the outcome variables. This method is a classification tree algorithm, that is, often utilised as a data mining method in fields with complex datasets, such as marketing, healthcare²⁷ and nursing.²⁸ We used the exhaustive CHAID algorithm, a modified version of the basic algorithm that performs a more thorough merging and testing of independent variables.²⁹ Statistical calculations were performed using SPSS V.20.0J software and Decision Tree.

Patient and public involvement

There was no patients or public involvement in the design and analysis of this study.

Results

Baseline characteristics, hospital policy and IT infrastructure

From the 239 hospitals that responded to the questionnaire (response rate: 57.2%), we were able to calculate and integrate the target QI data for 153 hospitals. Hospitals with data on at least one target QI were included. The hospital selection flow diagram is presented in figure 1. The baseline characteristics of the participant hospitals and respondents are shown in table 1. The median number of hospital beds was 303 (range: 63–1161). Approximately 75% of all the hospitals were teaching hospitals; the mean number of junior and senior residents in each hospital was ~21.

Table 1.

Characteristics of the hospitals and respondents* (153 hospitals)

Hospital characteristics
Beds, mean±SD (range); median	339±182 (63–1161); 303
Teaching hospitals, n (%)	115 (75.2)
Hospital size, n (%)
>500 beds	25 (16.3)
≤500 beds	128 (83.7)
Full-time physicians, mean±SD (range)	61.9±43.3 (8–268)
Resident physicians, mean±SD (range)	21.2±29.2 (0–197)

Respondent characteristics	n (%)
Sex
Male	125 (81.7)
Female	19 (12.4)
No response	9 (5.9)
Age (years)
20–29	5 (3.3)
30–39	21 (13.7)
40–49	28 (18.3)
50–59	50 (32.7)
60–69	39 (25.5)
No response	10 (6.5)
Appointment
Hospital administrator (physician)	69 (45.1)
Chief general manager (non-physician)	38 (24.8)
Others	34 (22.2)
No response	12 (7.8)

*Respondents answered the questions as representatives of their hospital.

Table 2 shows the results of the survey on hospital policies and IT infrastructure. Almost all respondents reported that their hospitals had an explicit policy to enhance IT infrastructure (94.1%). However, the provision of wireless LAN (71.9%) and access to paid medical evidence databases in English (54.9%) was limited. Further, an intranet homepage was provided only in a minority of hospitals (27.5%).

Table 2.

Hospital policies and IT infrastructure (153 hospitals)

Questionnaire items	n (%)
Hospital policies
Explicit policy to enhance IT infrastructure to improve accessibility to medical information (YES)	144 (94.1)
Explicit recommendation for the utilisation of evidence-based medicine (YES)	88 (57.5)
Explicit recommendation for adherence to clinical practice guidelines (YES)	84 (54.9)
Accessibility to the internet and other information sources
Electronic health records and internet access
Access to both electronic health records and the internet	110 (71.9)
Other	43 (28.1)
Wireless LAN
Available with no limitations/with limited access points	110 (71.9)
Not available	43 (28.1)
Major locations with wired LAN access (multiple answers allowed)
Outpatient clinics/wards	98 (64.1)
Other locations (including medical offices and library)	144 (94.1)
Access to paid medical evidence databases (multiple answers allowed)
Igaku Chuo Zasshi (ICHUSHI) database <in Japanese>	118 (77.1)
Medical databases such as UpToDate, Clinical Key, Ovid and DynaMed	84 (54.9)
Medical library and intranet usability within the hospital
Provision of an intranet homepage with user-friendly interface	42 (27.5)
Number of full-time medical librarians
≥1	66 (43.1)
0	87 (56.9)
Medical library activities (multiple answers allowed)
Periodic meetings held to improve the information retrieval environment	84 (54.9)
Continuously working to improve library services and usability	60 (39.2)
Participation in hospital librarian associations and communication with other hospital librarians	25 (16.3)
Other	23 (15.0)

IT, information technology; LAN, local area network.

Figure 2 shows the information sources freely available or specifically provided by the participating hospitals. There were large variations in the provision of paid medical evidence databases, and hospitals tended to subscribe to the Japanese-language database (77.1%) rather than the English-language databases (9.8%–46.4%). In general, the print editions of various CPGs and medical information were provided more frequently than the electronic editions, and there were relatively few hospitals that provided CPGs in either edition (41.2% in the print edition and 15.0% in the electronic edition).

Figure 2.

Information sources freely available or specifically provided by the participating hospitals (153 hospitals).

Correlates of hospital quality of care

Table 3 shows the mean QI scores for the use of perioperative antibiotics according to the various independent variables. Hospitals with a lower number of positive responses to items related to hospital policies and IT infrastructure tended to have a lower QI score. Using CHAID analysis, we identified three major correlates of QI score (figure 3); hospital size and teaching status were the strongest correlates. The subgroup of ‘≤500 bed non-teaching hospitals’ had the lowest QI score (73.1 points, node 2). The other subgroup (comprising larger or teaching hospitals) was divided into two groups based on the provision of access to paid medical evidence databases. The derived subgroup of ‘Japanese and/or English databases’ was further divided into two groups according to (1) accessibility to the internet, (2) wireless LAN availability and (3) wired LAN availability at outpatient clinics/wards; hospitals in the subgroup that had the highest number of positive responses to these three items had the highest QI score (87.2 points, node 6) among all nodes. In contrast, the subgroup that had positive responses to two or fewer of these items had a lower QI score (83.1 points, node 5). The subgroup of hospitals with no IT infrastructure elements had the lowest QI score (75.1 points, Node 4) among the larger or teaching hospitals. These results indicated that the provision of access to paid medical evidence databases and accessibility to the internet (including LAN availability) were strongly associated with hospital quality of care.

Table 3.

QI scores of hospital groups according to the number of positive responses to questionnaire items on hospital policies, IT infrastructure and hospital size (153 hospitals)

No of questionnaire items with positive responses	Mean QI score	n
Hospital policies *
0	78.73	4
1	81.92	51
2	84.95	29
3	78.92	69
IT infrastructure
Accessibility to the internet and other information sources†
0	78.59	5
1	82.20	32
2	78.70	62
3	83.32	54
Access to paid medical evidence databases‡
0	72.55	25
1	79.76	54
2	84.88	74
Medical library and intranet usability within the hospital§
0	81.70	21
1	78.45	45
2	80.25	39
3	84.40	26
4–6	83.27	22
Hospital size and teaching status
>500 bed non-teaching	87.64	4
>500 bed teaching	83.28	21
≤500 bed teaching	83.16	94
≤500 bed non-teaching	73.10	34

Questionnaire items are as follows.

*Three items: having explicit policy to enhance IT infrastructure, explicit recommendation for the utilisation of evidence-based medicine, and explicit recommendation to use clinical practice guidelines.

†Three items: electronic health records and internet availability, wireless LAN availability, wired LAN availability at outpatient clinics/wards.

‡Two items: provision of access to the Japanese medical database and access to English-language medical databases.

§Six items: provision of access to an intranet homepage, one or more full-time medical librarians, periodic meetings for library improvement, continuously working to improve library services and usability, communication with other hospital librarians and others.

IT, information technology; LAN, local area network; QI, quality indicator.

Figure 3.

χ² automatic interaction detection tree diagram showing the correlates of the QI score. *These values indicate the numbers of positive responses to questionnaire items related to (i) electronic health records and internet availability, (ii) wireless LAN availability and (iii) wired LAN availability at outpatient clinics/wards. **P=0.0499, in detail. DB, database; LAN, local area network; QI, quality indicator.

Discussion

In this multicentre study, we observed wide variations in the provision of IT infrastructure across hospitals in Japan. Our results indicated that hospitals with superior IT infrastructure tended to have higher adherence to CPGs for perioperative antibiotic prophylaxis. Using a CHAID tree analysis, we found that the provision of access to paid medical evidence databases and accessibility to the internet (including LAN availability) were strong indicators of quality of care in larger or teaching hospitals

Despite the wide availability of new medical evidence, clinicians are not always able to acquire and apply the most recent and relevant information at the right moment in their daily practice. The lack of adequate IT infrastructure may affect the ability of clinicians to access this information, thereby contributing to evidence-practice gaps. There are >8000 hospitals in Japan, of which 80% (and almost all clinics) are privately owned.³⁰ Different leadership approaches among these hospitals may have resulted in considerable variations in IT infrastructure. Our analysis found that there was an overall inadequate provision of LAN, and accessibility to the internet and electronic health records was limited among the hospitals. In addition, the print editions of various CPGs and medical information were provided more frequently than electronic editions. Previous studies on the activities to improve accessibility to medical information within specific hospital networks^{31 32} have indicated the importance of hospital leadership in the development of IT infrastructure.

Our CHAID analysis found that the three most important correlates of hospital quality of care were hospital size and teaching status, access to paid medical evidence databases and high accessibility to the internet. Notably, hospital policies and library/intranet usability were not identified as major correlates. There are likely several reasons for the identification of hospital size and teaching status, access to paid medical evidence databases and high accessibility to the internet as the most important factors. First, from an economic perspective, teaching hospitals tend to be large and more likely to have the economic capability to provide resources such as IT infrastructure and full-time librarians engaged to work on intranet development. The results in table 2 indicate that more than half of the participating hospitals do not hire full-time medical librarians. Interviews with the administrators of several leading teaching hospitals prior to the survey revealed that some administrators were actively working to enhance their hospital’s intranet environment. This included the hiring of full-time librarians to create user-friendly intranet homepages designed to guide clinicians to the most recent and relevant clinical information. Our results are consistent with those of previous reports that show improvements in medical library functionality can improve patient health outcomes while reducing the time needed for clinicians to search for required information.^{33 34} From a cultural perspective, ‘teaching’ nature of the teaching hospitals makes them more committed to evidence-based thinking and promote evidence-based practice. In the Japanese context, highly motivated teaching staffs tend to gather in large-scale teaching hospitals, and these cultural elements may also explain our results in part.

Second, we found that hospital administrators tended to provide access to free medical databases first, followed by the paid Japanese database and finally the paid databases in English (figure 2). Besides the high cost of subscribing to English-language databases, the administrators may have prioritised the Japanese database due to the possible language barrier for Japanese clinicians; this phenomenon has also been observed in Taiwan.³⁵ However, the failure to provide medical databases in English raises concerns that the clinicians may be unable to retrieve the newest relevant information in a timely manner, which can directly impact daily clinical practice. In addition, the availability of information in English may be crucial to the efficient dissemination and effective implementation of CPGs.

Third, the importance of internet accessibility (including LAN availability) to healthcare quality has been similarly observed in previous studies from the USA and UK.^{1 21 31} Our study sample included a fairly high proportion of hospitals without wireless LAN (28.1%) or with limited wired LAN availability at outpatient clinics and wards (64.1%). In order to encourage the implementation of IT infrastructure that facilitates easy retrieval of evidence in all types of hospitals, it may be necessary to develop a standardised assessment tool for hospital IT infrastructure and to include such assessments as a component of hospital accreditation.

In daily clinical practice, clinicians have limited time to search for and retrieve medical information. Thus, an ideal search platform would allow the use of clinical questions with several keywords and provide the requested information promptly and accurately. In addition to improving the accessibility and usability of online information, it may be useful to actively provide paid medical evidence databases (especially English-language databases) at the hospital level to supply multiple layers of information ranging from abstracts to full-text articles, as well as recommendations for CPGs and their evidence sources. In order to maximise the use of this system, individual physicians should work to improve not only their English language skills but also their information searching skills and ability to implement new knowledge into practice.

Our findings suggest that larger or teaching hospitals would have the most potential for improvements in IT infrastructure that can lead to better quality of care (see nodes 4, 5 and 6 in figure 3). As Doebbeling et al noted, IT implementation is dependent on the support of hospital management and ‘should be tailored to the needs of the organisation, and not as a ‘one size fits all’ solution’.³⁶ It is also necessary to conduct balanced assessments of the costs and effectiveness of these IT infrastructures in order to efficiently support the implementation of evidence into practice under limited budgets.

In a broader context, barriers to implementing CPG recommendations in daily practice vary greatly at the individual level of specialists and physicians (eg, perception, education, incentives, professional autonomy), the institutional level (eg, physician leadership, hospital policies, finance, institutional culture, teamwork, IT infrastructure), national level (eg, policies to promote CPG use, hospital accreditation) and the social level (eg, a culture of shared-decision making with patients, information derived from mass media).^37–39 Yet, given the growing importance of IT use in an innovative society, the impact of IT adoption on healthcare quality warrants far more consideration of the types of relationships that were revealed in this study. According to previous research, IT itself needs to be understood as having two distinct components—information technology and communication technology—and these differently affect the autonomy of workers.⁴⁰ Further examination is needed to clarify these issues in order to implement IT in practice settings.

There are several limitations to this study. First, the respondents were hospital administrators, which means the results may not indicate the usability of various IT infrastructure elements from a physician’s perspective. In addition, as this was a self-reported survey, the possible presence of social desirability bias may have caused these respondents to underestimate the barriers being investigated. Because individual physicians are the primary target users and are likely to be the link between IT infrastructure and quality of care, studies focusing physicians are required in the future to clarify the quality improvement mechanism in detail. Second, the survey was conducted in Japan, which may limit the generalisability of our results to other countries. Third, the QI that we used was limited to adherence to the CPGs for perioperative antibiotic use and therefore describes only one aspect of healthcare quality. Fourth, as we used administrative claims data, we could not know the clinical information in detail including appropriate exceptions in clinical practice. However, in this study, we focused on perioperative antibiotic prophylaxis of the 11 surgeries, which we could identify accurately based on the information of surgical procedure and drug use from the database. Finally, we were not able to identify the amount of investment or the affordability of each hospital’s IT infrastructure, and further studies are needed to examine the total effect of these issues on the quality of hospital care.

Conclusions

Hospitals with superior IT infrastructure may provide higher-quality care. The provision of access to paid medical evidence databases and accessibility to the internet were strongly associated with hospital quality of care, and may be key factors for improving healthcare quality in larger or teaching hospitals. These infrastructure elements may allow healthcare professionals to retrieve the latest information on evidence-based medicine with greater ease and facilitate the dissemination of CPGs in the internet era. Hospitals should focus on establishing adequate IT infrastructure to support the effective implementation of CPGs. The systematic improvement of IT infrastructure in hospitals may support greater adherence to CPGs and narrow the evidence-practice gaps.