Hospital-Level Variation in Genetic Testing in Children’s Hospitals Neonatal Intensive Care Units from 2016 to 2021

Abstract

Purpose:

To examine variation in genetic testing between neonatal intensive care units (NICUs) across hospitals over time.

Methods:

We performed a multicenter large-scale retrospective cohort study using NICU discharge data from the Pediatric Hospital Information System Database between 2016 and 2021. We analyzed variation in the percentage of NICU patients who had any genetic testing between hospitals and over time. We used a multivariable multilevel logistic regression model to investigate the potential association of patient characteristics and genetic testing.

Results:

The final analysis included 207,228 neonates from 38 hospitals. Overall, 13% of patients had at least one genetic test sent, though this varied from 4% to 50% across hospitals. Over the study period, the proportion of patients tested increased, with the increase disproportionately borne by hospitals already testing high proportions of patients. On average, patients who received genetic testing had higher illness severity. Controlling for severity, however, only minimally reduced the degree of hospital-level variation in genetic testing.

Conclusion:

The percentage of NICU patients who undergo genetic testing varies among hospitals, and increasingly so over time. Variation is largely unexplained by differences in severity between hospitals. The degree of variation suggests that clearer guidelines for NICU genetic testing are warranted.

BACKGROUND

The neonatal intensive care unit (NICU) has been a launch point for genetic testing.^1,2 Many conditions warranting NICU admission have an underlying genetic etiology, and 9% of all level IV NICU patients received a genetic diagnosis in a recent study.³ Compared to patients with other types of disease, NICU patients with genetic diseases have longer, costlier admissions that more frequently end in death.^3–6 Early identification of genetic disease promises to improve clinical care and inform decision making for this high acuity population.⁷ Yet, the complex and frequently uncertain information produced by modern genetic tests also has the potential to be misunderstood or misused by both clinicians and parents, particularly if not accompanied by sufficient support from clinical geneticists and genetic counselors.^8–10

No guidelines exist to standardize genetic testing practices between NICUs. As such, hospitals are left to develop their own testing policies and practices. Genetic testing has evolved rapidly over the past two decades, but the absence of standardization means that progress may be unevenly distributed between hospitals.¹¹ Insurance coverage of inpatient genetic testing varies between states, which likely increases variation in practice.¹² Generally, extreme variation is considered a marker of poor quality care.^15,16 The degree to which genetic testing practices vary between NICUs has not been studied.

Believing that describing genetic testing practices would be a first step toward assessing the need for standardization, we used data from NICU admissions at children’s hospitals in the Pediatric Hospital Information System (PHIS) Database to analyze variation in genetic testing among hospitals over time. PHIS compiles patient-level care data from 49 freestanding children’s hospitals including diagnostic codes, demographic and payer information, and billing data.

METHODS

We performed a retrospective multi-center large-scale cohort study using PHIS data from NICU discharges between January 2016 and December 2021. We excluded incomplete patient entries and hospitals with missing years of data for the study period.

Measurements

We extracted data on length of stay, admission year, gestational age, in-hospital mortality, biologic sex, race, ICD diagnosis codes, lab tests billed, binary flags signifying presence or absence of any complex chronic conditions (CCC)¹⁷ or congenital anomalies, All Patient Refined Diagnosis Related Groups (APR-DRG) severity levels, and whether a patient was inborn or transferred to the hospital. A medical geneticist (K.T.W) curated a list of 29 Current Procedural Terminology lab codes that denote genetic testing (Supplemental Table 1). We similarly refined a list of genetic diagnoses. We also extracted data from each hospital’s website on the number of clinical geneticists (MD or equivalent) working at the institution as of June 2022.

Statistical Analysis

The primary outcome was a binary indicator of any genetic testing during the hospital encounter. We used t-tests, chi-squared tests, and ANOVA tests as appropriate to compare patients that did and did not receive genetic testing. We calculated the percentage of patients who received genetic testing at each hospital for each year and across the study period. We performed linear regression analysis and graphed this percentage over time by hospital and looked separately at the trends for the five hospitals with the highest overall testing over the study period, heretofore “high testing hospitals,” and lowest overall testing percentages, or “low testing hospitals.” We also analyzed hospital-level variation in the mean number of genetic tests sent per patient.

We used a multivariable multilevel logistic regression model to investigate the potential association of patient or hospital characteristics with genetic testing. We included hospital as a random effect and then sought to assess variation attributable to the hospital after controlling for various patient characteristics. We ultimately fit a pair of models with differing fixed-effects – the first an intercept-only model, the second with patient-level fixed effects. We built the second model using forward stepwise procedures by adding patient- and hospital- level characteristic, including admission year, in-hospital mortality, CCC, congenital anomalies, severity level, gestational age, biologic sex, race, length of stay, and admission volume (annual hospital mean). We included characteristics that were statistically significant predictors (individually) at level α=.20 in a multivariable regression model. In the final multivariable model, we retained only those characteristics with P-values less than 0.05. We calculated the intra-class correlation coefficient (ICC) statistic for both models to measure the proportion of variation attributable to the random effects, in this case hospital. We obtained the Best Linear Unbiased Predictors (BLUPs) to estimate and graphically display the predicted effect each hospital had on the probability of testing. Using the final model, we made in-sample predictions of the probability of genetic testing and plotted the mean and standard deviation by hospital. We performed all data analysis using Stata version 16.1 (StataCorp, College Station, TX).

RESULTS

Our final analysis included 188,025 admissions from 38 hospitals (Supplemental Table 2). We excluded 95,878 entries representing 34% of all admissions due to missing data.

Of included admissions, 24,521 (13%) had genetic testing. Over the study period, the percentage of patients with testing increased (Figure 1; β=0.021, CI: 0.011–0.031, P<.001). The top five highest testing hospitals increased at a faster rate (β=0.063, CI: 0.024–0.10 P=.003) and the bottom five hospitals hospital increased at a slower rate (β=0.006, CI: 0.002–0.010, P=.008). Between hospitals, the percentage of patients with genetic testing ranged from 4% to 41% (Figure 2). Among patients who underwent genetic testing, the mean number of tests was 2.28 (SD 1.58), with an interhospital range of 1.14 (SD 0.49) to 3.50 (SD 1.46) (Supplemental Figure 1). Hospitals which tested a greater percentage of patients also sent more tests per patient (R²=0.75, P<.001). The most common tests sent were chromosomal microarray (16,668, 17%), tissue culture for chromosomal analysis (15, 056, 15%), chromosome analysis (13,342, 14%), sanger sequencing (10,588, 11%), and in situ hybridization (7,015, 7%). Over the study period microarrays and sanger sequencing accounted for an increasing proportion of testing. The most common genetic diagnoses among these patients are listed in the appendix (Supplemental Table 3).

Figure 1. Percent of patients who underwent genetic testing over time.

Fitted lines represent fractional polynomial lines for all hospitals, the top five highest testing hospitals, and bottom five lowest testing hospitals.

Figure 2. Unadjusted and adjusted genetic testing percentage by hospital.

Each point represents a hospital and error bars indicate 95% confidence intervals. Panel 2A shows the actual testing variation. Panel 2B shows the variation in predicted testing rates calculated using best linear unbiased prediction based on the final multivariate regression model.

Compared to patients without genetic testing, patients with testing had higher mortality rate (Supplemental Table 4, 9.6% vs. 2.9%, P<.001), longer length of stay (7.0 weeks vs. 3.2 weeks, P<.001), were more likely to have a CCC (90.6% vs 48.0%, P<.001) and a congenital anomaly (29.4% vs 5.1%, P<.001), and were less likely to be of Black race (18.9% vs 21.4%, P<0.001). Genetic testing was increasingly likely as a patient’s APR-DRG severity level increased (P<.001). Patients admitted later in the study period were more likely to have a genetic test sent (R²=0.070, P<.001).

Consistent with these findings, increases in the following patient-level characteristics were associated with testing: year of admission, gestational age, in-hospital mortality, CCC and congenital anomaly flags, APR-DRG severity level, and length of stay (P<.001 for all). Patients who were Black were less likely and patients of race Other were slightly more likely to have a genetic test sent compared to White patients (P<0.001). Biologic sex was not associated with genetic testing (P=.058). We also individually considered the hospital-level characteristic of admission volume, which was significant (Supplemental Figure 2, P<.001). All significant patient-level factors retained significance while admission volume was no longer significant in the multivariate model. For continuous variables, we explored the relationship between deviance residuals and the patient-level fixed effect, which led us to log-transform length of stay resulting in a better fitting model. The final model included mortality, CCC, congenital anomalies, severity level, gestational age, length of stay (log-transformed), race, and year of admission (Supplemental Table 4). The random effects variance was significantly different from 0 in the intercept-only model (σ²=0.47; 95% CI 0.30 – 0.74) and patient fixed-effects model (σ²=0.35; 95% CI 0.22 – 0.56), justifying the choice of hospital as the random intercept. Compared to the intercept-only model, the multivariate model was associated with a 23% drop in the Akaike’s information criterion (136574 to 105587) and the Likelihood-ratio test comparing models was significant (P<0.001). The proportion of variation in genetic testing that was attributable to hospital as measured by the ICC was 12.3% (95% CI 8.2%–18.1%) in the unadjusted model and 9.3% (95% CI 6.1%–14.0%) in the adjusted model (Figure 2).

In several measures, high testing hospitals and low testing hospitals deviated in opposite directions from the remaining hospitals. High testing hospitals had a higher percentage of transferred patients (69% vs. 16%, P<.001), lower admission volume (Supplemental Figure 2, 644.41 vs. 1652.94, P<.001), and greater average number of clinical geneticists (7.14 vs. 3.93, P<.001).

DISCUSSION

This study of genetic testing in NICUs reveals large and increasing hospital-level variation that is not explained by patient-level characteristics. This suggests that if the same complex patient was born at different hospitals, their likelihood of receiving genetic testing could vary substantially.

We interpret these findings cautiously for three main reasons. First, coding practices may be inconsistent between hospitals.¹⁸ If hospitals vary in how they separate versus bundle billing codes, the total number reported per patient would be less precise. Given the standardization imposed by the PHIS reporting system, this is, however, unlikely to explain the degree of variation we observe. Second, the PHIS records likely exclude genetic testing that is done on a research basis. Research testing, though, is even more likely to differ across hospitals than clinical testing, so we expect this leads to an underestimation of variation. Finally, we are limited by the information in the database assessing whether genetic testing was appropriate. Even some represented variables may not fully capture difference in patient characteristics. For instance, the congenital anomaly flag may be enriched for particular diagnoses for which genetic testing is more or less appropriate at certain hospitals. However, since our model controlled for most factors that have been previously associated with likely genetic disease^3,5,6, we believe the majority of the observed variation cannot be explained by differences in the patient characteristics between hospitals.

What might explain the observed variation in testing practices? The correlation between the percentage of testing sent and the mean number of tests sent per patient suggests that some hospitals have a generally more pro-testing culture than others. One reason for variation may be differences in reimbursement policies for inpatient genetic testing, varying ability to conduct genetic tests in house, or hospital administrations’ response to these policies. Also, varied prenatal testing practices and inter-state differences in the breadth of newborn screening may reasonably account for some variation. Hospitals may also be more likely to test because they have greater availability of genetic resources, which is supported by the higher number of geneticists at high-testing hospitals and the general observation that greater capacity in healthcare leads to greater utilization.¹⁹ High-testing hospitals seem to be referral centers, with a higher percentage of transferred patients and lower admission volume. Such centers may be culturally more likely to adopt new technology early. Patients may also be appropriately transferred to obtain genetic testing. The lower testing among Black patients raises the concerning possibility that bias may influence decisions about testing, though we cannot definitively exclude other explanations for this finding. Finally, the idiosyncrasy of medical cultural evolution in the absence of clear testing guidelines likely contributes to this variation. Further work is needed to explore the extent to which each of these factors contributes to variation.

This degree of variation for any reason would be troubling and the widening gap adds urgency to the problem. These findings imply that some patients are more likely than others to receive genetic testing based on where they are hospitalized. If testing is benefitting patients, denying patients access to testing based on location represents inequality.⁷ If testing poses risks, either to patients through misuse or discrimination or to society by creating an unjustified financial burden, the burden is unequally borne.^8–10,14 As the types of genetic tests used become more complex, risks increase. A more complete understanding of the benefits and harms associated with genetic testing will facilitate consensus and generation of guidelines for genetic testing, which in turn can begin to reverse the trend of increasing variation. Future work should explore how inequalities in genetic testing may extend to or differ in other hospital and outpatient contexts.

Data Availability:

All PHIS data are publicly available.