Impact of nonintrusive clinical decision support systems on laboratory test utilization in a large academic centre

Abstract

Background:

The near-universal prevalence of electronic health records (EHRs) has made the utilization of clinical decision support systems (CDSS) an integral strategy for improving the value of laboratory ordering. Few studies have examined the effectiveness of nonintrusive CDSS on inpatient laboratory utilization in large academic centres.

Methods:

Red blood cell folate, hepatitis C virus viral loads and genotypes, and type and screens were selected for study. We incorporated the appropriate indications for these labs into text that accompanied the laboratory orders in our hospital’s EHR. Providers could proceed with the order without additional clicks. An interrupted time-series analysis was performed, and the primary outcome was the rate of tests ordered on all inpatient medicine floors.

Results:

The rate of folate tests ordered per monthly admissions showed no significant level change at the time of the intervention with only a slight decrease in rate of 0.0109 (P = .07). There was a 43% decrease in the rate of hepatitis C virus tests per monthly admissions immediately after the intervention with a decrease of 0.0135 tests per monthly admissions (P = .02). The rate of type and screens orders per patient days each month had a significant downward trend by 0.114 before the intervention (P = .04) but no significant level change at the time of the intervention or significant change in rate after the intervention.

Discussion:

Our study suggests that nonintrusive CDSS should be evaluated for individual laboratory tests to ensure only effective alerts continue to be used so as to avoid increasing EHR fatigue.

1 |. INTRODUCTION

Rising health care costs in the United States (US) have created a national interest in high-value care and methods to reduce wasteful practices.¹ Finding solutions for improving appropriate utilization of medical resources often focuses on modifying provider behavior.² With the near-universal prevalence of electronic health records (EHRs) in this country, the utilization of clinical decision support systems (CDSS), combined with specific clinical guidelines for appropriate laboratory testing,³ can be used as an integral strategy for improving the value of laboratory ordering.⁴

The CDSS can be considered intrusive or nonintrusive in the context of the EHR platform for provider ordering. Examples of intrusive CDSS include pop-up windows requiring active acknowledgement of browsing messages and “hard stops” to ordering practices. An example of a nonintrusive CDSS is text information accompanying orders without an active requirement for provider acknowledgement. Intrusive CDSS are more expensive to create and often lead to provider alert fatigue and dissatisfaction.^5-8 Provider surveys show that intrusive CDSS are preferred for alerts such as drug interactions while nonintrusive CDSS are preferred for less urgent information.⁹ Nonintrusive CDSS, therefore, could be a practical method to reduce laboratory utilization.

Prior studies have addressed the use of CDSS on laboratory ordering practices in the outpatient setting,^10-12 but few have addressed the utility in the inpatient setting with only a recent emphasis on targeting redundant laboratory ordering.^13,14 We conducted a prospective pilot study to evaluate the effect of nonintrusive CDSS on the inpatient utilization of serum and red blood cell (RBC) folate levels, hepatitis C virus (HCV) viral loads (VL) and genotypes, and type and screens (T&S) in a large academic centre.

2 |. METHODS

2.1 |. Laboratory test selection

We created utilization messages for the following laboratory tests: serum and RBC folate, HCV VL and genotype, and T&S. We chose these tests because we could provide clear appropriate use messaging and we had evidence of overutilization at an institutional level and from the research literature.

In 1998, the US government implemented a mandatory fortification of all cereal grains with folic acid. With this intervention, folate deficiency in the US population decreased to less than 1%, making routine folate testing unnecessary.^15,16 Additionally, the lack of gold standard for diagnosing folate deficiency makes the sensitivity and specificity of current biological assays unknown.¹⁷ Therefore, we recommended against routine testing of serum and RBC folate in our utilization message.

Appropriate indications for HCV VL and genotype were determined after consultation with infectious disease experts at our institution as well as a literature review. Testing indications for HCV VL included either suspicion of acute HCV or the need to distinguish between chronic and cleared HCV in patients with positive antibodies but no prior VL.^18,19 We advised that HCV genotype be reserved for patients imminently starting HCV therapy to guide the treatment regimen.¹⁹ Notably, these appropriate indications are rare in the hospitalized patient.

The American Association of Blood Banks requires a T&S sample drawn within 3 days of transfusion.²⁰ These standards are based on the timing of red cell alloantibody formation and their detection after a stimulating event such as transfusion or pregnancy. Most of the haemolytic transfusion reactions due to anamnestic antibody production occur 3 to 14 days after transfusion.²¹ Therefore, for patients who are at low risk of bleeding, are not pregnant, and have not had prior positive antibody screens, we advised that a single T&S is sufficient for their entire hospital admission. High-risk patients, however, including those with active bleeding, haemoglobinopathies, symptomatic anaemia, pregnancy, recent transfusions, or prior positive antibody screen, should continue to have a T&S every 3 calendar days. We based this recommendation on our analysis of inpatient T&S orders in our institution. T&S orders were placed every 3 days on all patients on the inpatient General Medicine services regardless of their risk of needing a transfusion. Less than 5% of these patients eventually received a blood transfusion.

2.2 |. Intervention

The hospital system consisted of over 1000 acute-care inpatient beds with 9 different inpatient General Medicine services each with an average of about 20 patients. There was a total of over 250 patient beds for the Department of Medicine with 20 intermediate care beds available for patients on all Medicine services.

We incorporated the above appropriate indications for the selected laboratory tests into text that accompanied the laboratory orders in our hospital’s EHR (Figure 1). Minimal programming time was involved in the text creation. If desired, providers could ignore the messages and proceed with the order without additional clicks.

FIGURE 1.

Educational text accompanying orders for serum and RBC folate, HCV VL and genotype, and T&S (ABO)

In addition to these nonintrusive messages, brief topic reviews were provided through 1-page 1-minute guides (OMG) to the Department of Medicine on the appropriate indications for each test as well as a noon conference to the medicine housestaff regarding the indications. The folate interventions were implemented in December 2014, HCV interventions in May 2015, and T&S in February 2016. The primary outcome was the total number of tests ordered either per monthly admissions or monthly patient days on all inpatient medicine floors, excluding the intensive and intermediate care units. All data collection ended on July 1, 2016, when our hospital system switched EHR platforms. The hospital’s institutional review board approved the study.

2.3 |. Statistical analysis

Patient demographic information including admission diagnosis was collected for the same number of months before and after each intervention. Chi-square analyses were performed to identify differences in age, gender, race, and admission diagnosis between the preintervention and postintervention group for each time intervention. The admission diagnoses of gastrointestinal bleeding, haemorrhage, anaemia, and liver disease were chosen as ICD-9/10 categories likely to affect the frequency of folate, HCV, and T&S orders.

An interrupted time-series (ITS) analysis was performed using SAS software v9.4 (SAS Institute Inc, Cary, North Carolina) with the autoregressive procedure. As noted in the equation below, the ITS model measured both a level change at the time of the intervention as well as a change in slope after the intervention while controlling for any pre-existing baseline trend.

$$\text{Test Rate}=\beta 0+\beta 1\times \text{Time}+\beta 2\times \text{Intervention}+\beta 3\times \text{Time after intervention}+e$$

β₀ represents the intercept at the beginning of the time series. β₁ estimates the overall trend of the test rate prior to the intervention where time is continuous and measured in months at the start of the study period. β₂ captures the immediate impact of the intervention on the test rate with intervention representing a binary variable with 0 for preintervention and 1 for postintervention. β₃ captures the trend after the intervention where time after intervention is continuous and measured in months after the start of the intervention. e is the standard error.

The 3 interventions (brief educational guide, noon conference, and nonintrusive ordering message) were counted as one given their proximity in occurrence. During the month of the intervention, the data for that month were analysed as either before or after the intervention depending on whether or not most of the days that month had the intervention. Serum and RBC folate were analysed together as one test as were HCV VL and genotype. For folate and HCV testing, the number of tests ordered was normalized to the number of admissions per month and reported as a rate of orders per monthly admissions. The T&S orders were presented as a rate of orders per patient day per month given the nature of the test often requiring repeating orders on the same patient throughout the same hospitalization. For all three laboratory tests, the estimates of the coefficients β₁, β₃, and β₂ were reported in the results in that order.

3 |. RESULTS

The demographic information for all admissions to medicine floors in the hospital was stratified by intervention period for each separate laboratory test (Table 1). There were no significant differences in age, gender, race, or admission diagnoses for any of the tests before or after the intervention.

TABLE 1.

Patient demographics on all medicine admissions stratified by intervention period

	Folate			HCV			T&S
	Pre	Post	P value	Pre	Post	P value	Pre	Post	P value
Age			.24			.20			.22
13-39	2893 (20.4)	2880 (20.9)		1962 (21.3)	1898 (20.6)		1245 (21.9)	1243 (21.8)
40-59	5406 (38.1)	5097 (37.1)		3399 (36.8)	3454 (37.6)		2124 (37.4)	2073 (36.3)
60-79	4570 (32.2)	4519 (32.9)		2996 (32.5)	3039 (33.1)		1844 (32.5)	1866 (32.7)
80-110	1324 (9.3)	1255 (9.1)		872 (9.4)	802 (8.7)		469 (8.3)	530 (9.3)
Gender			.23			.15			.89
Male	7134 (50.3)	6814 (49.6)		4518 (49.0)	4597 (50.0)		2854 (50.2)	2862 (50.1)
Female	7059 (49.7)	6937 (50.4)		4711 (51.0)	4596 (50.0)		2828 (49.8)	2850 (49.9)
Race			.43			.50			.91
White	5702 (40.2)	5421 (39.4)		3610 (39.1)	3642 (39.6)		2222 (39.1)	2277 (39.9)
African American	7432 (52.4)	7240 (52.7)		4900 (53.1)	4799 (52.2)		2978 (52.4)	2970 (52.0)
Asian	194 (1.4)	209 (1.5)		133 (1.4)	144 (1.6)		94 (1.7)	90 (1.6)
Other	865 (6.1)	881 (6.4)		586 (6.3)	608 (6.6)		388 (6.8)	375 (6.6)
Admission diagnosis			.06			.06			.10
GI bleeding	207 (1.5)	211 (1.5)		160 (1.7)	143 (1.6)		97 (1.7)	123 (2.2)
Haemorrhage	191 (1.3)	191 (1.4)		50 (0.5)	71 (0.8)		87 (1.5)	98 (1.7)
Anaemia	442 (3.1)	342 (2.5)		268 (2.9)	234 (2.5)		118 (2.1)	117 (2.0)
Liver disease	85 (0.6)	83 (0.6)		72 (0.8)	53 (0.6)		33 (0.6)	20 (0.4)
Total admissions	14193	13751		9229	9193		5682	5712

Abbreviations: HCV, Hepatitis C virus; T&S, type and screens.

Figure 2 shows the rates of folate, HCV VL and genotype, and T&S over their respective time periods. The number of folate tests ordered per total number of monthly admissions was calculated for 30 consecutive months (January 2014 through June 2016). There was a significant downward trend in the rate of folate tests ordered per monthly admissions by 0.0055 before the intervention (P < .0001) and a significant increase from baseline by 0.0056 tests per monthly admissions after the intervention (P < .0001). The results show that there was no significant level change in test rate at the time of the intervention with only a slight decrease in rate of 0.0109 (P = .07). The rate of folate tests ordered per monthly admissions did decrease across the time period.

FIGURE 2.

Interrupted time-series analysis showing the rate of folate, HCV VL and genotype, and T&S orders over time on the Internal Medicine hospital floors. Vertical line represents the intervention. Predicted values from the autoregressive model along with the corresponding 95% confidence intervals are shown in dashed lines

The number of HCV VL and genotypes ordered per total number of monthly admissions was determined over 30 consecutive months (January 2014 through June 2016). There was no significant increasing or decreasing trend found in the number of HCV tests being ordered per month before or after the intervention. Before the intervention, the number of HCV tests per monthly admissions was increasing at a rate of 0.000279, which was not significant, and similarly, postintervention, they were increasing at a rate of 0.000971, which was not significant. However, the average change in number of HCV tests per monthly admissions immediately after the start of the intervention while controlling for the baseline trend was a decrease of 0.0135 tests per monthly admission or a 43% decrease (P = .02).

The rate of T&S orders per monthly patient days was calculated for 16 consecutive months (February 2015 through June 2016). There was a significant downward trend in the rate of T&S orders per patient days each month by 0.114 before the intervention (P = .04) and no significant change in rate after the intervention with a decrease of 0.038 (P = .85). The intervention itself was not associated with a significant level change in rate with a decrease of 0.248 tests per patient days each month at the time of the intervention (P = .73).

4 |. DISCUSSION

This study suggests that the impact of nonintrusive CDSS should be evaluated for individual laboratory tests to ensure only effective alerts continue to be used so as to avoid increasing EHR fatigue. The CDSS can still be a low-cost intervention for continuing educational efforts on appropriate ordering practices for selected laboratory tests. The folate results showed a significant pre-existing trend of a decrease in the number of folate tests ordered per monthly admissions. This baseline decrease before the intervention was most likely associated with an increase in awareness from informal educational lectures within the department. The data did demonstrate a decrease in the number of HCV VL and genotype tests ordered per monthly admissions immediately after the intervention. However, given the few number of these tests being ordered, the clinical significance of this decrease is likely minimal. The pre-existing trend of a decrease in T&S orders per patient days each month over the time period was maintained even after the intervention. The effect of the intervention on this laboratory test was diminished by the truncated postintervention time period.

Prior work has shown that noninterruptive notifications do not increase the appropriate utilization of baseline laboratory testing.¹² Because we did not evaluate the appropriateness of individual patients’ orders, whether the pre-existing trends in reduction of folate and T&S orders specifically represented a reduction in unnecessary testing was outside of the scope of our analysis. The specific laboratory tests in this study were targeted at our institution, but nonintrusive messaging can continue educational efforts regarding the appropriate ordering of other tests as long as a clear message is displayed. In fact, targeting laboratory tests that have high volume and show no pre-existing decreasing trends would be important.

Our study has several limitations that need to be noted. First, this was not a randomized control trial, and practicing patterns of resident physicians may change as a result of their experience in the academic year. An ITS analysis as well as analysis of demographic data before and after the intervention helped to account for the pre-existing trends in the ordering rate of the studied laboratory tests. Second, the main intervention of the study was accompanied by an educational lecture at the Department of Medicine noon conference and an educational guide for appropriate laboratory test ordering available to all house staff. These initiatives were promoted by the Department of Medicine High-Value Care Committee with the goal of promoting a culture of high-value care and education. Perhaps these initiatives can partly explain the pre-existing decrease in the rate of folate and T&S ordering that was seen in the analysis. Third, our academic centre converted EHR platforms on July 1, 2016, which unfortunately truncated our data collection but only for the T&S results.

Future efforts to deliver high-value care will need to find innovative methods for impacting the use of laboratory tests in large hospital systems. Our findings suggest that nonintrusive CDSS consisting of text accompanying EHR laboratory orders do not have significant impact on laboratory test utilization. More work at different hospital centres will need to be done in order to find more effective means of influencing ordering practices.