Changing patterns of routine laboratory testing over time at children’s hospitals

Abstract

Background:

Research into low-value routine testing at children’s hospitals has not consistently evaluated changing patterns of testing over time.

Objectives:

To identify changes in routine laboratory testing rates at children’s hospitals over ten years and the association with patient outcomes.

Design, Settings, and Participants:

We performed a multi-center, retrospective cohort study of children aged 0–18 hospitalized with common, lower-severity diagnoses at 28 children’s hospitals in the Pediatric Health Information Systems database.

Main Outcomes and Measures:

We calculated average annual testing rates for complete blood counts, electrolytes, and inflammatory markers between 2010 and 2019 for each hospital. A > 2% average testing rate change per year was defined as clinically meaningful and used to separate hospitals into groups: increasing, decreasing, and unchanged testing rates. Groups were compared for differences in length of stay, cost, and 30-day readmission or ED revisit, adjusted for demographics and case mix index.

Results:

Our study included 576,572 encounters for common, low-severity diagnoses. Individual hospital testing rates in each year of the study varied from 0.3 to 1.4 tests per patient day. The average yearly change in hospital-specific testing rates ranged from −6% to +7%. Four hospitals remained in the lowest quartile of testing and two in the highest quartile throughout all ten years of the study. We grouped hospitals with increasing (8), decreasing (n = 5), and unchanged (n = 15) testing rates. No difference was found across subgroups in costs, length of stay, 30-day ED revisit, or readmission rates. Comparing resource utilization trends over time provides important insights into achievable rates of testing reduction.

INTRODUCTION

Overuse of diagnostic testing is a highly prevalent type of low-value care¹ in pediatric settings and provides no net benefit to patients or healthcare systems. This type of overuse leads to excess healthcare costs,^1,2 wasted staff time to perform testing, and reduced quality of care.^3,4 It also exposes patients to potential harms such as radiation, phlebotomy pain, interrupted sleep, and iatrogenic anemia and worsens patient and family experience overall.^5,6 Overuse of diagnostic testing also can lead to an unnecessary cascade effect of further testing or treatments and overdiagnosis with no meaningful impact on patient outcomes.^3,7 Reducing the overuse of diagnostic testing is vital to improving the value of pediatric healthcare.

Previous studies have identified wide variations in rates of routine inpatient testing with no clear difference in patient outcomes.^8–12 This variation between hospitals in testing rates does not appear to be randomly distributed. Hospitals with high rates of testing for one condition tend to have high rates of testing for other conditions and high rates of testing throughout early and late time points of inpatient admissions.^8,11 Cultural or environmental factors at an institutional level may contribute to these diagnostic testing patterns.

However, all of these studies aggregate longitudinal data into one overall measure to identify high and low performers, leading to a loss of perspective on how testing has changed over time. For example, sites with steady improvement over time might appear as average performers when collapsing data across time points into one measure. Making significant changes to system-level practice culture and reliably sustaining that change throughout time are difficult improvement science endeavors.^13,14 Analyzing hospital-level performance time trends over a specified period can help push beyond identifying hospitals with high average performance to identifying hospitals that have successfully changed practice and those that have a record of sustained change. These model sites may provide a rich learning environment for future intervention development.

The objectives of this study were to (1) determine hospital-level changes to routine testing rates over a 10-year period in children’s hospitals and (2) evaluate whether hospitals with increasing, decreasing, or sustained rates of testing had meaningful differences in clinical outcomes.

METHODS

Study design

We performed a multicenter, retrospective cohort study of hospitalized children at children’s hospitals contributing data to the Pediatric Health Information Systems (PHIS) database (Children’s Hospital Association, Lenexa, KS). Participating hospitals submit clinical, demographic, and billing data for all encounters, including International Classification of Diseases, 10th Revision, Clinical Modification codes. Data are deidentified at the time of submission and reviewed regularly for data quality. Of the 49 PHIS hospitals, we excluded hospitals with incomplete data (n = 11) during the study timeframe and hospitals that did not submit length of stay (LOS) in hours (n = 10), limiting our ability to measure this outcome precisely, leaving 28 hospitals in the study. This study was deemed exempt from review according to policies at the lead author’s Institutional Review Board.

Study population

The study cohort included all inpatient or observation encounters from 2010 to 2019 for patients 0–18 years of age with diagnoses that fell into the 10 most common all patient refined diagnosis-related groups (APR-DRG; 3M Healthcare) categories defined from our previous study on routine laboratory testing (e.g., asthma, bronchiolitis, urinary tract infections; details available in Supporting Information S1: Data 1).¹² In PHIS, the severity of illness score is assigned per 3M Healthcare’s APR-DRG grouper algorithm (1 = Minor, 2 = Moderate, 3 = Major, 4 = Extreme). The focus of our study was routine laboratory testing in patients with low illness severity, so we only included patients within the “minor” severity of illness category. Similarly, we excluded patients with complex chronic conditions¹⁵ and encounters with intensive care unit utilization. Finally, we excluded transfers in from other hospitals due to the inability to determine testing that may have occurred at the transferring institution.

Testing rates

We measured rates of testing in three common, high-volume laboratory test categories defined in our previous publication: inflammatory markers (e.g., c-reactive protein, erythrocyte sedimentation rate), electrolyte testing (e.g., basic metabolic panels), and complete blood counts (Supporting Information S1: Data 2).¹² A routine laboratory testing rate was determined for each hospital and each year during the study time period, defined as the total number of these tests performed divided by the total number of patient days for the included patient encounters.

Identifying hospitals with increasing or decreasing rates of testing

To evaluate changing patterns of testing for each hospital over time, we determined each hospital’s annual testing rate quartile. We then calculated each hospital’s rate ratio (RR) as their average annual change in testing rate using Poisson regression and determined the trend (RR > 1 was increasing, RR < 1 decreasing) and significance of the trend using p < 0.010 for this calculation. Because the large data set predisposes to high rates of statistical significance, the study team established a separate consensus measure of what constituted a clinically meaningful change, which was an overall testing change of >2% per year over the 10-year time period. This threshold was chosen by consensus based on prior improvement studies demonstrating this to be an achievable change in routine testing.^16–19 We then used this measure of clinically meaningful change to group hospitals into one of three mutually exclusive groups: increasing testing rates, decreasing testing rates, and the remaining hospitals with no clinically significant change in testing rates.

Outcome measures

We compared hospitals with increasing, decreasing, and no change in testing rates for differences in clinical outcomes over time. Outcome measures included LOS in hours, cost of hospitalization in US dollars (estimated from charges using hospital and year-specific cost-to-charge ratios and inflated to 2019 dollars using the medical component of the Consumer Price Index), 30-day all-cause readmission rate, and 30-day all-cause emergency department (ED) revisit rate. These outcomes are common measures in retrospective and quality improvement studies to measure the effect of changes in utilization directly and as balancing measures against potential adverse outcomes from changes in testing rates.^{8,11,12,16,18–21} Outcomes were adjusted for patient severity using the Hospitalization Resource Intensity Scores for Kids (H-RISK) score, a pediatric-specific relative cost weight used to measure resource intensity of patient admissions based on the APR-DRG (3M Health-care).²² To evaluate if changes in case mix over time impacted results, we also determined hospital-specific case mix indices calculated as the mean H-RISK score of hospitalizations for each hospital and study year.

Analysis

We summarized categorical variables with frequencies and percentages and continuous variables with geometric means and geometric standard deviations because of the skewed distribution of the data. Hospital volume was described via a median and range. We compared demographic and clinical characteristics across hospital groups using χ² and Wilcoxon rank tests. Because of ongoing research and discussions around the ways inequity in healthcare affects the overall value of care provided,^23,24 we included racial and ethnic categories available in the PHIS database to examine whether these social constructs were associated with a hospital’s ability to change testing rates. The method of collecting racial and ethnic category data (e.g., patient self-report) was not available from PHIS. We compared outcomes between hospital groups using a general linear mixed effects model with binomial distribution for readmissions and ED revisits and normal distribution for cost and LOS after log-transformation. Models were adjusted for H-RISK and clustered by hospital with random intercepts. Yearly changes to case mix index for each hospital were compared over time to evaluate for clinically significant changes. Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc.) and p values < .05 were considered statistically significant for these analyses.

RESULTS

Study cohort

We included 576,572 patient encounters across 28 hospitals after exclusions were applied (Supporting Information S1: Data 3). Most patients were in the 0- to 4-year-old age group, male, non-Hispanic White, and had government payor insurance (Table 1). In the hospitals with decreasing testing, there was a ~10% higher proportion of private payor. Over 99% of patient encounters were discharged to home, and the case-mix index determined via H-RISK was generally low.

TABLE 1.

Demographics for study overall and for hospital groups based on changing testing rates over time.

	Overall	Decreasing	No change	Increasing	p
N hospitals	28	5	15	8
N encounters	576,572	58,206	343,928	174,438
Age, years
0–4	372,624 (64.6)	39,003 (67)	220,284 (64)	113,337 (65)	<.001
5–9	109,388 (19)	10,765 (18.5)	65,802 (19.1)	32,821 (18.8)
10–14	60,910 (10.6)	5576 (9.6)	36,822 (10.7)	18,512 (10.6)
15–18	33,650 (5.8)	2862 (4.9)	21,020 (6.1)	9768 (5.6)
Gender
Male	319,819 (55.5)	32,382 (55.6)	190,919 (55.5)	96,518 (55.3)	0.321
Female	256,709 (44.5)	25,822 (44.4)	152,974 (44.5)	77,913 (44.7)
Race
Non-Hispanic White	236,429 (41)	26,041 (44.7)	139,081 (40.4)	71,307 (40.9)	<.001
Non-Hispanic Black	141,084 (24.5)	10,323 (17.7)	98,331 (28.6)	32,430 (18.6)
Hispanic	138,631 (24)	14,083 (24.2)	71,414 (20.8)	53,134 (30.5)
Asian	15,164 (2.6)	2848 (4.9)	7550 (2.2)	4766 (2.7)
Other	45,264 (7.9)	4911 (8.4)	27,552 (8)	12,801 (7.3)
Payor
Government	339,884 (58.9)	29,868 (51.3)	206,273 (60)	103,743 (59.5)	<.001
Private	206,121 (35.7)	25,626 (44)	119,737 (34.8)	60,758 (34.8)
Other	30,567 (5.3)	2712 (4.7)	17,918 (5.2)	9937 (5.7)
Disposition
HHS	1864 (0.3)	684 (1.2)	1001 (0.3)	179 (0.1)	<.001
Home	570,566 (99)	57,383 (98.6)	339,713 (98.8)	173,470 (99.4)
Skilled	299 (0.1)	18 (0)	193 (0.1)	88 (0.1)
Other	3843 (0.7)	121 (0.2)	3021 (0.9)	701 (0.4)
CMI (H-RISK)	0.53 (0.10)	0.54 (0.09)	0.53 (0.10)	0.53 (0.09)	<.001
Hospital volume, encounters median [IQR]	18,752 [11,513, 27,088]	12,115 [10,080, 12,714]	19,431 [16,803, 27,701]	20,825 [12,641, 30,220]	.077

Note: p Value determined via χ² or Wilcoxan rank tests comparison among study groups.

Abbreviations: CMI, case mix index; HHS, home health services, H-RISK, Hospitalization Resource Intensity Scores for Kids; IQR, interquartile range.

Testing rates

Hospital-level routine laboratory testing rates in 2010 varied from 0.32 to 1.41 tests per patient day (Figure 1) with the lowest quartile testing rates varying from 0.32 to 0.46 and the highest quartile varying from 0.88 to 1.41 tests per patient day. In 2019, the last year of the study timeframe, hospital-level routine laboratory testing rates varied from 0.32 to 1.36 tests per patient day with the lowest quartile testing rates varying from 0.32 to 0.53 and the highest quartile varying from 0.89 to 1.36 tests per patient day.

FIGURE 1.

Heat map of hospital-level routine testing rates over time. Hospital-level variation in routine testing rates displayed as a heat map. Individual hospitals are displayed as rows and year is displayed in columns. Color values correspond to quartiles of testing rate for each specific year. Median testing rate and 25th percentile (Q1) and 75th percentile (Q3) testing rates for a specific year displayed at the bottom of the chart. Rate ratio is the average annual change in testing rate. Clinically meaningful changes to rates of testing determined by a rate ratio >1.02 or <0.98 (i.e., annual change of ≥2%). Hospitals were sorted by 2010 testing rate.

Hospital changes in rates of testing over time

RRs for changes to hospital testing over time varied from 0.94 to 1.07 (median 1.00, interquartile range: 0.98–1.03), indicating an average yearly change in testing rates ranging from −6% to +7% for hospitals (Figure 1). When considering clinically meaningful changes as per our study definition, eight hospitals had a clinically meaningful increase in testing rates, five hospitals had a clinically meaningful decrease, and 15 hospitals had no change (Figure 2). The average RR for the decreasing group was 0.96, for the increasing group was 1.04, and for the no change group was 1.00. In the increasing testing group, four of eight hospitals were in the lowest quartile of testing rate in the first year of the study (2010), and one was in the top quartile (Figure 1). In the decreasing group, two of five hospitals were in the highest quartile and none were in the lowest quartile for testing rate in the first year of our study.

FIGURE 2.

Trends in routine testing rates over time by hospital groups. Annual hospital-specific rates of routine laboratory testing per patient day. Each figure represents a subgroup of hospitals based on increasing, decreasing, or no change in testing rates over time. Gray lines indicate individual hospital testing rates. Black lines indicate unweighted average of hospital testing rates by year.

Outcomes

Over the 10-year study timeframe, there were no significant differences in adjusted hospital encounter costs, LOS, 30-day ED revisit, or 30-day readmission rates between the three hospital subgroups (Table 2). For all three subgroups, hospitalization costs decreased between $53 and $124 per hospitalization per year, and LOS decreased between 0.31 and 1.04 h per year. Case mix index remained stable over time for the hospitals in the study (Supporting Information S1: Data 4).

TABLE 2.

Adjusted clinical outcomes for hospitals with decreased, no change, or increased rates of testing.

	Decreased testing	No change testing	Increased testing	p
Hospital costs: Mean annual change in cost in US dollarsa (95% CI)	−123.96 (−226.08, −21.83)	−90.83 (−147.13, −34.53)	−52.96 (−105.83, −0.08)	.400
LOS (h): Mean annual change in hours (95% CI)	−1.04 (−1.59, −0.49)	−0.31 (−0.64, 0.02)	−0.54 (−1.28, 0.19)	.082
30-day ED revisit: Odds ratio (95% CI)	1.01 (0.99, 1.03)	1.01 (1.01, 1.02)	1.02 (1.01, 1.03)	.547
30-day readmission: Odds ratio (95% CI)	1.01 (0.99, 1.03)	1.01 (1.00, 1.02)	1.01 (1.00, 1.02)	.800

Abbreviations: CI, confidence interval; ED, emergency department; LOS, length of stay.

^aInflated to 2019 US dollars.

DISCUSSION

In this multicenter study of routine laboratory testing in hospitalized children with common, lower-severity inpatient diagnoses, we identified three groups of hospitals with different patterns of hospital-level testing rate changes over 10 years. Most hospitals had no clinically meaningful change to their rates of testing during the 10 years of the study, though we identified groups of hospitals with both increasing and decreasing testing rates over time. A common argument for higher utilization of testing is preventing more severe clinical outcomes. However, in this low-severity population, there were no differences in any of these clinical outcomes across hospital groups, supporting the idea that higher value care may be safely achieved through lower rates of testing. Surprisingly, there was also no clear statistical difference in the rate of change of the cost of hospitalization between groups. This may be explained by the relatively low cost of routine testing compared to hospitalization overall or a lack of power to detect small changes in cost with the number of hospitals in this study. However, it was clear that reduced testing did not lead to increased costs for hospitals.

This study adds to the growing body of evidence around the low value of repeated routine laboratory testing in many situations for low-severity hospitalized children. Across the low-severity patient population in this study, rates of routine laboratory testing varied by three- to fourfold during most years of the study. It is unlikely that this wide range of testing rates is appropriate or contributes to high-value clinical care. The implications of excess testing are broad. With over 500,000 patient encounters, if hospitals in this study changed their routine testing to a rate closer to the lower quartile median (~20% lower), ~100,000 tests could potentially be avoided over 10 years. Estimating the financial impact of this reduced testing via incorporating the direct and indirect costs of routine laboratory testing is difficult,²⁵ but prior studies using hospital-level financial sources have found estimates of $10–50 per test which would correlate with a potential cost savings of $1–5 million over 10 years.^16,19 As seen in other studies, low-value tests that are low-cost but high-frequency can quickly add significantly to total costs.²⁶ Further identifying which types of routine testing are the highest priority or best opportunity for deimplementation may be an important next step in this line of research to guide value improvement efforts.

And though there were no differences across groups in the specific clinical outcomes measured directly in this study, it is likely that hospitals with higher testing rates had some important differences in several unmeasured outcomes. Specifically, overuse of laboratory testing is likely to impact two areas that are particularly important to patient experience: sleep and pain. Poor sleep and pain from excess phlebotomy have both been identified as significant causes of negative experiences in the hospital for patients and families.^5,6,27,28 Reducing common laboratory test utilization has been associated with reduced phlebotomy attempts and reduced early morning awakenings for patients and families in addition to reduced staff workload related to blood draws and results interpretation.^{16,17,19,29–33} Together, our study, along with these prior studies, highlight the continued challenges of low-value routine laboratory testing and a need to focus on system-wide efforts toward diagnostic stewardship and reducing overuse.^34,35

In this study, the changes in testing rates over time also identified interesting hospital practice patterns. Five hospitals consistently reduced testing rates over 10 years with no clear worsening of clinical outcomes. We know from previous studies that changing clinical practice patterns and hospital-wide culture change can be difficult to achieve,^36,37 so this degree of consistency would be unusual without specific and consistent interventions. In addition, a second pattern emerged around sustainability. Four hospitals remained in the lowest quartile of testing throughout the 10-year time period with little year-over-year variation and no change in measured outcomes. We know that sustaining high-value practice over time can also be difficult.^14,38 Understanding the strategies that have been effective at these four sites over a prolonged time period could be useful to identify sustainment strategies in value improvement efforts. Finally, though hospitals that began in the highest testing quartile and lowest testing quartile were both able to reduce testing, none of those in the lowest quartile were able to reduce testing to a clinically meaningful degree, indicating a potential floor for testing rates across systems. There may be more opportunity to reduce testing for those with higher initial testing rates. Finding an achievable “floor” for testing rates may be useful in situations where the ideal rate of utilization is nuanced or unknown and where undertesting remains a risk. Similar to an achievable benchmark of care (ABC),²⁰ this could be a pragmatic measure based on real-world practice to quantify what measure of improvement is feasible in healthcare around routine testing. Though less concrete than an ABC, this approach may reduce the risk of miscategorizing undertesting as ideal testing like a straightforward ABC calculation might.

It is unclear based on our findings if there is an important relationship between payor mix (i.e., higher proportion of private payor) and a hospital’s ability to consistently reduce testing. It could be important to know if this relationship is related to something like a payor financial incentive or alternately a difference in access to care and follow-up, both of which may facilitate decreased rates of testing. With a continued push to increase high-value care in pediatric settings, this finding supports the idea that we should continue to gather data where we can on the equity and access implications of value improvement work.³⁹

A positive deviance approach has been posited as an effective method to improve the value of healthcare via studying the specific systems that consistently demonstrate the best performance.^40,41 In the case of routine laboratory testing, the results of this study highlight the feasibility and benefits of measuring data over time to identify these positive deviants. Some of the patterns described in this article may not have been identified in other research methods that combine data over time into one overall performance measure. Other studies have looked at changes to practice patterns over time across a combined cohort of hospitals,^42–44 which is useful for understanding overall healthcare utilization trends. These studies do not break down their results to the level of hospital-specific changes and how they may be different from overall trends. A few recent articles have investigated hospital-specific changing practice patterns.^10,21 Nabower et al. evaluated hospital-specific practice patterns in acute gastroenteritis and similarly found that many hospitals had minimal changes in routine diagnostic testing. In that study, the hospitals with the highest reduction in costs were associated with decreased LOS and diagnostic testing.¹⁰ Overall, the ability to identify hospitals with specific changes over time to their testing patterns may facilitate learning and future research to improve and sustain high-value diagnostic testing in healthcare systems.

LIMITATIONS

Our study has several important limitations. The study uses an administrative database which may lack important clinical variables for interpreting the trends identified. Specifically, in this administrative data set, it is difficult to determine the location of testing (e.g., the ED vs. acute care ward) which may be an important consideration for evaluating the value of testing. We looked at testing per patient day and do not know the clinical indication for testing or how testing was used for decision making. This study investigated a subset of lower severity, uncomplicated patients that have previously been shown to encompass a large proportion of admissions; however, it may not be applicable to higher severity or more complex inpatients.^45,46 In particular, further study in the population of patients with complex chronic conditions is necessary to see if similar results occur in more complex populations. Extrapolating results from this study to nonchildren’s hospitals may not be appropriate. The retrospective study design makes it difficult to know with certainty if any of the associations identified in this study are causative or if other secular trends or changes to hospital populations influence the outcomes, though our limitation of the study population to a low severity, uncomplicated patient population minimizes this possibility. Finally, our study established a consensus-based definition of a clinically meaningful change in testing based on achievable changes in prior studies, but further research may be needed to determine how best to define this threshold more specifically.

CONCLUSION

This multicenter study identified a cohort of hospitals that reduced rates of routine testing over time with no change in commonly measured patient outcomes and hospitals that sustained low testing rates over time. It can be difficult to change hospital-wide practice patterns and culture, and the results of this study highlight one method to identify sites with success in changing hospital-level utilization trends over time.