Abstract
Objective
To examine changes in referral rates of very low birthweight (VLBW, birthweight <1500 g) infants to high-risk infant follow-up (HRIF) in California and identify factors associated with referral before and after implementation of a statewide initiative in 2013 to address disparities in referral.
Study design
We included VLBW infants born 2010–2016 in the population-based California Perinatal Quality Care Collaborative who survived to discharge home. We used multivariable logistic regression to examine factors associated with referral and derive risk-adjusted referral rates by NICU and region.
Results
Referral rate improved from 83.0% (pre-initiative period) to 94.9% (post-initiative period); yielding an odds ratio of 1.48 (95% CI: 1.26–1.72) for referral in the post-initiative period after adjustment for year. Referral rates improved the most (>=15%) for infants born >=33 weeks gestation, with birthweight 1251–1500g, and born in intermediate and lower-volume NICUs. Post-initiative, Hispanic ethnicity, small for gestational age status, congenital anomalies, and major morbidities were no longer associated with decreased odds of referral. Lower birthweight, outborn status and higher NICU volume were no longer associated with increased odds of referral. African American race was associated with lower odds of referral, and higher NICU level with higher odds of referral during both time periods. Referral improved in many previously poor-performing NICUs and regions.
Conclusion
HRIF referral of VLBW infants improved substantially across all sociodemographic, perinatal and clinical variables following the statewide initiative, although disparities remain. Our results demonstrate the benefit of a targeted initiative in California, which may be applicable to other quality collaboratives.
Despite increased survival rates, very low birthweight (VLBW) infants remain at high risk for neurodevelopmental impairment, with 30–50% of infants having disabilities at 2 to 3 years of age.1–6 High-risk infant follow-up (HRIF) programs follow VLBW and other high-risk infants after they are discharged from the neonatal intensive care unit (NICU) with the goal of surveilling growth and neurodevelopment and identifying evolving difficulties that may be improved with early intervention.3, 5, 7 The American Academy of Pediatrics (AAP) emphasizes the importance of HRIF as a vital resource for optimizing the care of VLBW infants after discharge.8, 9
Despite the importance of HRIF programs, several studies have identified significant gaps in compliance with follow-up appointments. Attendance rates range from 50–70%, and there are significant social and demographic disparities associated with poor follow-up, including non-white parents, single parent families and lower maternal education.10–13 Hintz et al evaluated the referral patterns of VLBW infants born from 2010–2011 in California and found that only 80% of eligible infants were referred to HRIF at NICU discharge. Significant variability existed in referral among regions (8%–98%) and across NICUs (<5%–100%). Sociodemographic disparities also existed, with maternal African American and Hispanic race-ethnicity associated with 35–50% lower odds of referral.14 These findings prompted an initiative to increase referral rates across California which involved the launch of site-specific, online reports that matched NICU discharges with confirmed HRIF referral. This initiative was implemented in July 2013 with the goal of improving referral rates for all eligible infants.
Our objective was to determine the impact of this targeted statewide initiative by examining the change in referral rates before and after the implementation of this initiative and identifying risk factors associated with non-referral of VLBW infants before and after implementation.
Methods
The California Perinatal Quality Care Collaborative (CPQCC) collects data on infants admitted to 140 NICUs in California and includes >95% of VLBW infants born in the state. The CPQCC network collects data from member NICUs using an expanded version of the Vermont Oxford Network dataset.15 CPQCC data are abstracted by trained NICU personnel. The California Children’s Services (CCS) is a federally and state-funded program that provides support for services related to medically-eligible conditions and provides funding for most NICUs and HRIF programs. CPQCC and CCS partnered in 2009 to restructure the existing statewide HRIF to develop the CPQCC-CCS HRIF program, which provides a series of visits for eligible children through age 3 in one of approximately 70 HRIF clinics across the state. All VLBW infants, regardless of gestational age, are eligible for HRIF, and CCS expects these children to be referred and followed. The assigned personnel at the discharging CPQCC NICU complete a Referral/Registration Form via the web-based CPQCC-CCS HRIF Reporting System, referring the infant to a HRIF program, that then accepts the case, contacts the family, and arranges follow-up appointments. CPQCC and CPQCC-CCS HRIF databases are linked via a probabilistic linkage algorithm with >99% success.16, 17
We included infants born from 2010 to 2016 from the linked CPQCC and CPQCC-HRIF databases with birthweight <1500 grams who survived to discharge home. The initiative was formally implemented July 1, 2013; therefore, the pre-initiative period included infants born January 2010 to June 2013, and the post-initiative period included infants born July 2013 to December 2016. This initiative included creation of NICU site-specific, confidential online reports that matched NICU discharges to home with confirmed referral to HRIF through the web-based system. The report was limited to specific highest-risk clinical categories eligible for HRIF, including VLBW, and highlighted cases of referral failure for individual NICUs to review online. A 6-month pre-implementation period included education about the report, feedback plans to the sites, and expectation that all high-risk infants, including VLBW, were referred to HRIF at NICU discharge. Although VLBW infants were an initial priority for this initiative, there were other high-risk groups who were included in the initiative, including infants with moderate or severe hypoxic ischemic encephalopathy, infants who received active cooling and infants requiring extracorporeal membrane oxygenation (ECMO).
Education regarding the report was provided to all CPQCC member hospitals during annual CPQCC data training sessions in 2013 and 2014. In anticipation of future deliverables for referral rates, awards were given to NICUs with referral rates of 100% for VLBW infants for the preceding three continuous years. Beginning in 2014, the expectation was set that 100% of all VLBW infants should be referred to HRIF. This was communicated broadly and consistently to CPQCC NICUs in advance of implementation. Centers that achieved 100% referral rates received annual awards, which were publically announced to the entire CPQCC membership, highlighting the specific centers. Centers that did not meet this expectation were charged a penalty fee of a small percentage of the year’s CPQCC membership dues.
We evaluated several patient-level, NICU and regional factors potentially associated with referral. Patient-level factors were derived from CPQCC and included prenatal care, maternal age (<19 years, 20–34 years, 35+ years), maternal race/ethnicity (African American, Hispanic, non-Hispanic white, Asian/Pacific Islander, Other), infant sex, multiple gestation, gestational age, birthweight, small for gestational age status, need for surgery, presence of congenital anomaly, and presence of major morbidity. Major morbidity is a composite outcome used in other studies and includes the presence of infection (early or late sepsis), severe retinopathy of prematurity, bronchopulmonary dysplasia (use of oxygen at 36 weeks postmenstrual age), severe intraventricular hemorrhage, periventricular leukomalacia or necrotizing enterocolitis.7 NICU factors included birth location (inborn or outborn), discharging NICU level of care, and discharging NICU volume. NICU level was divided into three levels (regional, community and intermediate) based on CCS guidelines, AAP designation and the services provided at each NICU.18 NICUs that choose not to participate in the CCS program were defined as “non-CCS.” NICU volume was based on the VLBW discharge volume in either the pre-initiative or post-initiative periods. NICUs were grouped according to quartile of VLBW discharge volume in each period. California regions were defined according to the California Department of Public Health Regional Perinatal Programs of California and were deidentified in this analysis.
Statistical analyses:
Referral rate was calculated as the total number of referrals received by HRIF divided by the total number of eligible infants. We examined the crude odds ratio of referral in the post-initiative compared with the pre-initiative period, as well as the odds ratio derived from a model that included birth year, because referral rates were changing over time, especially during the pre-intervention period. We examined the change in referral rates between the pre-initiative and post-initiative periods for each category of each covariate and calculated 95% confidence intervals for the risk difference to reflect the precision of the estimates. We used chi-squared tests to assess differences in referral across categories of each covariate, before and after implementation. We used multivariable logistic regression models to evaluate independent associations of factors with referral to HRIF during each time period. Models included variables that were identified a priori to be relevant and significant in either the pre-initiative or post-initiative period in unadjusted analyses (P < .05). These models excluded infants who had missing data on any covariates. Because birthweight and gestational age are correlated, multivariable models included birthweight only. Confidence intervals for odds ratios that did not include 1.0 were considered significant. Because referral is only mandated for eligible infants cared for in a CCS NICU, we excluded infants in the final model who were never cared for in a CCS NICU during any point in their hospitalization (15 infants in the pre-initiative period and 2 infants in the post-initiative period).
The final multivariable logistic regression model was used to estimate the probability of the HRIF referral rate for each infant. The expected referral rate for each hospital was calculated by summing the probabilities of referral for all infants that were discharged from that hospital. The ratio for observed rate to expected rate (O/E ratio) was calculated for each hospital. If the O/E ratio was >1.0, that indiciates that the hospital had more referrals than would be expected based on its patient mix. The hospital O/E ratio was then multiplied by the overall CPQCC network referral rate to obtain an adjusted referral rate. Risk-adjusted referral rates were similarly calculated for each region. NICUs were deidentified, numbered and listed in ascending order of their pre-initiative risk-adjusted referral rates. Regions were similarly deidentified and designated by letters and listed in ascending order of their pre-initiative risk-adjusted referral rates. This study was approved by the California Committee for the Protection of Human Subjects and the Stanford University Institutional Review Board.
Results
There were 17,942 VLBW infants in the pre-initiative period and 17,736 infants in the post-initiative period cared for in 139 NICUs in California. Referral increased from 83.0% of eligible VLBW infants in the pre-initiative period (11,712/14,106) to 94.9% in the post-initiative period (13,399/14,116), yielding an unadjusted odds ratio of 3.81 (95% CI: 3.49–4.16). After adjustment for birth year, the odds of referral in the post-initiative period was 1.29 (95% CI: 1.24–1.35). Table I displays the referral rates by year in the pre- and post-initiative periods.
Table 1.
HRIF Referral Rates of VLBW Infants in California by Year
| Birth Year | Referred n (% of Eligible) | |
|---|---|---|
| Pre-Initiative | 2010 | 3164 (77.7) |
| 2011 | 3382 (83.5) | |
| 2012 | 3446 (85.2) | |
| Jan-Jun 2013 | 1720 (88.8) | |
| Post-Initiative | Jul-Dec 2013 | 1798 (87.9) |
| 2014 | 3970 (96.5) | |
| 2015 | 3862 (96.5) | |
| 2016 | 3769 (95.6) | |
Referral rate varied according to a variety of factors in the pre- and post-initiative periods (Table 2). In the pre-initiative period, African American and Hispanic race/ethnicity and small for gestational age status were associated with lower referral. Earlier gestational age, lower birthweight, need for surgery, presence of congenital anomaly, presence of major morbidity, outborn birth status, discharge from regional and community NICUs, and discharge from NICUs with higher VLBW volume were associated with greater referral. For every factor, referral was greater in the post-initiative compared with the pre-initiative period. Rates were still significantly different across a variety of factors. HRIF referral was most improved among SGA infants born at greater than 33 weeks [22.7% (95% CI: 19.6–25.7)], those with birthweight 1250–1499 g [15.3% (95% CI: 14.0–16.6)], and those discharged from intermediate level [25.4% (95% CI: 20.5–30.3)] and lower-volume NICUs [22.0% (95%CI: 16.4–27.6)]. African American and Hispanic infants had greater magnitude in improvement of referral [12.8% (95% CI: 10.8–14.9)] and [13.7% (95% CI: 12.5–14.7)] compared with white infants [9.6% (95% CI: 8.2–11.0)], although the confidence intervals for African American and white infants do overlap slightly.
Table 2.
HRIF Referral Rates of VLBW Infants in California in the Pre- and Post-initiative Periods
| Pre-initiative (N=14106) | Post-initiative (N=14116) | ||||
|---|---|---|---|---|---|
| Referred n (% of eligible) | Missing | Referred n (% of eligible) | Missing | Change in Referral Rate % (95% CI) | |
| Overall | 11712 (83.0) | 13399 (94.9) | 11.9 (11.1–12.6) | ||
| Prenatal Care2 | 35 | 18 | |||
| No | 331 (84.4) | 535 (97.8) | 13.3 (9.5–17.1) | ||
| Yes | 11354 (83.0) | 12856 (94.8) | 11.8 (11.1–12.5) | ||
| Maternal age2 | 7 | 6 | |||
| <19 years | 978 (84.4) | 729 (97.6) | 13.2 (10.8–15.6) | ||
| 20–34 years | 7831 (83.0) | 8988 (94.9) | 11.9 (11.0–12.7) | ||
| 35+ years | 2899 (82.6) | 3676 (94.5) | 11.8 (10.4–13.3) | ||
| Maternal Race/Ethnicity1,2 | 39 | 61 | |||
| African American | 1575 (81.7) | 1621 (94.6) | 12.8 (10.8–14.9) | ||
| Hispanic | 5088 (81.9) | 6123 (95.6) | 13.7 (12.5–14.7) | ||
| White | 3249 (84.6) | 3441 (94.2) | 9.6 (8.2–11.0) | ||
| Asian/PI | 1469 (84.3) | 1780 (94.5) | 10.2 (8.2–12.2) | ||
| Native American/Other | 298 (84.9) | 373 (93.3) | 8.3 (3.9–12.8) | ||
| Infant Sex | 1 | 3 | |||
| Female | 5781 (82.8) | 6676 (94.9) | 12.1 (11.1–13.1) | ||
| Male | 5930 (83.3) | 6720 (95.0) | 11.7 (10.7–12.7) | ||
| Multiple Gestation1,2 | 1 | 2 | |||
| No | 8594 (83.6) | 9802 (95.2) | 11.6 (10.8–12.4) | ||
| Yes | 3117 (81.5) | 3595 (94.2) | 12.7 (11.3–14.2) | ||
| Gestational Age1,2 | 3 | 0 | |||
| 25 weeks and less | 1888 (89.5) | 2002 (97.0) | 7.5 (6.0–9.0) | ||
| 26–29 weeks | 5703 (84.9) | 6378 (95.6) | 10.7 (9.7–11.6) | ||
| 30–32 weeks | 3331 (80.2) | 3936 (93.5) | 13.3 (11.9–14.7) | ||
| 33+ weeks | 788 (70.0) | 1083 (92.7) | 22.7 (19.6–25.7) | ||
| Birthweight1,2 | 0 | 0 | |||
| 750g and less | 1698 (89.2) | 1808 (97.3) | 8.1 (6.5–9.7) | ||
| 751–1000g | 2733 (86.4) | 2989 (95.9) | 9.5 (8.1–10.9) | ||
| 1001–1250g | 3232 (84.0) | 3745 (95.2) | 11.2 (9.8–12.5) | ||
| 1251–1500g | 4049 (78.0) | 4857 (93.3) | 15.3 (14.0–16.6) | ||
| Small for Gestational Age (SGA)1,2 | 13 | 7 | |||
| <=32wk estimated GA | 2537 (81.4) | 3007 (94.3) | 12.9 (11.3–14.5) | ||
| >=33wk estimated GA | 788 (70.0) | 1083 (92.7) | 22.7 (19.6–25.7) | ||
| AGA | 8377 (85.1) | 9302 (95.4) | 10.3 (9.5–11.2) | ||
| Any surgery1,2 | 0 | 0 | |||
| No | 9456 (81.7) | 11107 (94.4) | 12.7 (11.9–13.5) | ||
| Yes | 2256 (89.2) | 2292 (97.6) | 8.4 (7.0–9.7) | ||
| Congenital anomaly2 | 5 | 2 | |||
| No | 10562 (82.8) | 12169 (94.7) | 11.9 (11.1–12.6) | ||
| Yes | 1146 (84.8) | 1228 (96.9) | 12.1 (10.0–14.2) | ||
| Major Morbidity1,2 | 628 | 628 | |||
| No | 7227 (83.2) | 8521 (94.6) | 11.4 (10.5–12.3) | ||
| Yes | 4113 (85.9) | 4318 (96.5) | 10.6 (9.4–11.7) | ||
| Birth Location1,2 | 0 | 0 | |||
| Inborn | 10569 (82.3) | 12450 (94.7) | 12.5 (11.7–13.2) | ||
| Outborn | 1143 (90.9) | 949 (97.7) | 6.8 (5.0–8.6) | ||
| Discharging NICU level1,2 | 5 | 1 | |||
| Regional | 4094 (90.9) | 4891 (99.5) | 8.6 (7.7–9.4) | ||
| Community | 7232 (85.9) | 8194 (98.3) | 12.4 (11.6–13.2) | ||
| Intermediate | 322 (71.1) | 191 (96.5) | 25.4 (20.5–30.3) | ||
| Non CCS | 59 (8.2) | 122 (18.4) | 10.2 (6.6–13.8) | ||
| Discharging NICU volume1,2 | 0 | 1 | |||
| Lowest quartile | 240 (43.6) | 396 (65.6) | 22.0 (16.4–27.6) | ||
| 2nd quartile | 1477 (74.3) | 1741 (87.0) | 12.7 (10.3–15.1) | ||
| 3rd quartile | 2699 (77.2) | 3100 (94.5) | 17.2 (15.6–18.8) | ||
| 4th quartile | 7296 (90.4) | 8162 (99.2) | 8.8 (8.1–9.5) | ||
p<0.05 in the pre-initiative period
p<0.05 in the post-initiative periods
Results of multivariable logistic regression analysis in the pre-initiative and post-initiative periods are shown in Table III. In the pre-initiative period, maternal African American race, Hispanic ethnicity, small for gestational age status, the presence of major morbidity and presence of congenital anomaly were associated with lower odds of referral. Odds of referral were higher for infants with lower birthweights. Outborn birth status, discharge from regional and community NICUs, and discharge from NICUs with higher VLBW volume were associated with greater odds of referral. In the post-initiative period, Hispanic ethnicity, small for gestational age status, presence of congenital anomaly and the presence of major morbidity were no longer associated with decreased odds of referral. Lower birthweight categories, outborn birth status and higher discharge NICU volume were no longer associated with increased odds of referral. However, African-American race remained associated with decreased odds of referral. The odds of referral also remained greater with increasing NICU level.
Table 3.
Results of multivariable logistic regression of association of factors with referral of VLBW infants to HRIF in California in the pre- and post-initiative periods.
| Pre-Initiative (N=13370) | Post-Initiative (N=13398) | |
|---|---|---|
| Adjusted OR (95% CI) | Adjusted OR (95% CI) | |
| Prenatal Care | ||
| No | ref | ref |
| Yes | 1.08 (0.77–1.40) | 0.99 (0.46–2.13) |
| Maternal Age | ||
| <19 years | ref | ref |
| 20–34 years | 0.98 (0.81–1.19) | 0.82 (0.43–1.56) |
| 35+ years | 1.00 (0.80–1.24) | 0.85 (0.44–1.66) |
| Maternal Race/Ethnicity | ||
| African American | 0.60 (0.50–0.71) | 0.64 (0.41–0.98) |
| Hispanic | 0.62 (0.54–0.72) | 0.76 (0.55–1.04) |
| White | ref | ref |
| Asian/PI | 0.86 (0.70–1.04) | 1.20 (0.78–1.84) |
| Other | 1.23 (0.81–1.87) | 0.76 (0.36–1.63) |
| Infant Sex | ||
| Male | ref | ref |
| Female | 1.05 (0.95–1.18) | 1.01 (0.78–1.30) |
| Small for Gestational Age (SGA) | ||
| ≤32wk SGA | 0.86 (0.76–0.98) | 0.97 (0.71–1.32) |
| ≥33wk SGA | 0.49 (0.40–0.59) | 0.71 (0.44–1.16) |
| Appropriate for GA | ref | ref |
| Any Surgery | ||
| No | ref | ref |
| Yes | 1.18 (0.99–1.41) | 0.99 (0.63–1.56) |
| Congenital Anomalies | ||
| No | ref | ref |
| Yes | 0.78 (0.65–0.93) | 0.80 (0.48–1.32) |
| Major Morbidity | ||
| No | ref | ref |
| Yes | 0.78 (0.68–0.89) | 0.77 (0.55–1.07) |
| Birthweight | ||
| 750g and less | 1.75 (1.42–2.17) | 1.11 (0.67–1.86) |
| 751–1000g | 1.45 (1.24–1.70) | 1.21 (0.84–1.74) |
| 1001–1250g | 1.25 (1.10–1.43) | 1.34 (0.98–1.86) |
| 1251–1500g | ref | ref |
| Birth Location | ||
| Inborn | ref | ref |
| Outborn | 1.56 (1.24–1.97) | 1.15 (0.59–2.25) |
| Discharging NICU Level | ||
| Regional | 1.49 (1.11–2.00) | 5.20 (1.92–14.11) |
| Community | 1.35 (1.04–1.75) | 2.30 (0.96–5.52) |
| Intermediate | ref | ref |
| Non CCS | 0.02 (0.02–0.03) | 0.01 (0.003–0.02) |
| Discharging NICU Volume | ||
| Lowest quartile | ref | ref |
| 2nd quartile | 3.04 (2.31–4.01) | 1.18 (0.79–1.78) |
| 3rd quartile | 2.39 (1.86–3.08) | 0.66 (0.41–1.06) |
| 4th quartile | 3.95 (3.03–5.14) | 1.31 (0.76–2.28) |
The distribution of race/ethnicity by discharging NICU level and discharging NICU volume is shown in Table 4 (available at www.jpeds.com). The percentage of white infants discharged from NICUs with the lowest VLBW volume was 3.1% in the pre-initiative period and 3.6% in the post-initiative period. This is compared with 4.4% and 6.2% of African American infants in the pre- and post-initiative periods, respectively.
Table 4.
“online”: Distribution of race/ethnicity in VLBW infants in California by discharging NICU level and discharging NICU volume
| Race/Ethnicity n (%) | |||||
|---|---|---|---|---|---|
| African-American | Hispanic | White | Asian/PI | Other | |
| Pre-Initiative Period | |||||
| Discharging NICU level | |||||
| Regional | 638 (33.1) | 1845 (29.7) | 1311 (34.2) | 573 (32.9) | 119 (33.9) |
| Community | 1152 (59.8) | 3931 (63.4) | 2114 (55.1) | 1014 (58.3) | 192 (54.7) |
| Intermediate | 59 (3.1) | 190 (3.1) | 133 (3.5) | 62 (3.6) | 9 (2.6) |
| Non-CCS | 78 (4.1) | 239 (3.9) | 281 (7.3) | 91 (5.2) | 31 (8.8) |
| Discharging NICU volume | |||||
| Lowest quartile | 85 (4.4) | 237 (3.8) | 119 (3.1) | 87 (5.0) | 14 (4.0) |
| 2nd quartile | 225 (11.7) | 817 (13.2) | 655 (17.1) | 232 (13.3) | 51 (14.6) |
| 3rd quartile | 544 (28.3) | 1684 (27.2) | 773 (20.2) | 390 (22.4) | 90 (25.8) |
| 4th quartile | 1072 (55.7) | 3465 (55.9) | 2285 (59.6) | 1033 (59.3) | 194 (55.6) |
| Post-Initiative Period | |||||
| Discharging NICU level | |||||
| Regional | 621 (36.3) | 2045 (31.9) | 1380 (37.8) | 708 (37.6) | 139 (34.8) |
| Community | 1006 (58.7) | 4040 (63.1) | 1987 (54.4) | 1038 (55.1) | 228 (57.0) |
| Intermediate | 13 (1.0) | 91 (1.4) | 63 (1.7) | 25 (1.3) | 6 (1.5) |
| Non-CCS | 73 (4.3) | 228 (3.6) | 222 (6.1) | 112 (6.0) | 27 (6.8) |
| Discharging NICU volume | |||||
| Lowest quartile | 106 (6.2) | 247 (3.9) | 133 (3.6) | 103 (5.5) | 15 (3.8) |
| 2nd quartile | 188 (11.0) | 834 (13.0) | 601 (16.5) | 294 (15.6) | 65 (16.3) |
| 3rd quartile | 430 (25.1) | 1617 (25.3) | 727 (19.9) | 411 (21.8) | 86 (21.5) |
| 4th quartile | 989 (57.7) | 3707 (57.9) | 2191 (60.0) | 1075 (57.1) | 234 (58.5) |
We calculated observed and risk-adjusted rates of HRIF referral for the 114 NICUs that had at least 20 VLBW infants discharged home and for the California regions for both the pre-initiative and post-initiative periods (Figure). The number of NICUs referring <80% of eligible infants decreased from 28% (32/114) in the pre-initiative period to 7% (8/114) in the post-initiative period. The number of NICUs referring >95% of eligible infants increased from 47/114 (41%) in the pre-initiative period to 96/114 (84%) in the post-initiative period. The five California regions with referral rates <85% in the pre-initiative period all surpassed 85% referral in the post-initiative period.
Figure 1.
Risk-Adjusted Referral Rates of VLBW infants in California by NICU (A) and by Region (B) in the Pre-initiative and Post-initiative Periods
Discussion
Our findings demonstrate an improvement in the referral of VLBW infants to HRIF clinic in California from 83.0% to 94.9%, after the implementation of a targeted statewide initiative. After adjustment for birth year, which may serve as a proxy for background trends in changes in referral rate, the post-initiative period remained associated with an increased odds of referral compared with the pre-initiative period. Following the implementation of this initiative, referral rates were higher across all of the studied sub-groups, NICUs, and regions.
Substantial improvement was notable in certain key groups of infants. Although this initiative targeted all VLBW infants, the improvement in referral of infants born at greater than 33 weeks gestational age and with birthweight 1251–1500g is particularly important. These infants had particularly low referral rates before the initiative compared with more premature and lower birthweight infants, but the differences were much narrower post-implementation. This may have been due to a provider perception of lower risk. However, several studies have demonstrated that moderate and late preterm infants are at increased risk of developmental delays, and school and behavioral problems.19–23 In addition, these infants have greater special education needs and greater use of early intervention programs than full-term infants.19, 24, 25
Although referral rates of African-American and Hispanic infants were lower than white infants in the pre-initiative period (81.7% and 81.9% compared with 84.6%), there was a greater magnitude in improvement for these infants compared with white infants between the 2 periods. Despite this improvement, African-American race remained associated with lower adjusted odds of referral in the post-initiative period. This remaining disparity could be related to differential access to high quality healthcare. Significant racial/ethnic variations in quality of care exist between and within NICUs.26 African-American patients are more likely to deliver VLBW infants at lower-level NICUs, and hospitals that have high percentages of African-American patients were under-resourced and delivered lower quality of care.27–29 In our population, a greater percentage of African-American infants than white infants were discharged from NICUs in the lowest quartile of discharging volume in both the pre- and post-initiative periods. Because lower-level and lower-volume NICUs have lower referral rates, further investigation is needed to evaluate the relationship between referral rates by race/ethnicity and specific NICU characteristics in California.
Notable improvement in referral was observed in intermediate-level NICUs and NICUs with lower VLBW volume, although there remains continued room for improvement. Regional NICUs still had greater odds of referral compared with intermediate-level NICUs. In addition, despite a 22% improvement in referral by the NICUs in the lowest quartile of discharging volume, referral rate was only 65.6% in the post-initiative period. These findings highlight additional opportunities for improvement. Referral to HRIF should be considered a quality indicator and the lower referral rates by lower-level and lower-volume NICUs may be reflective of difficulties in providing optimal quality of care. In general, this aligns with prior research that shows that VLBW and other high-risk infants born at lower-level and smaller NICUs have poorer outcomes.30–33 These NICUs may have fewer resources and may lack the infrastructure and personnel to easily facilitate the referral process, which could contribute to lower referral rates. With recent de-regionalization of neonatal care leading to an increased proportion of preterm infants being born and cared for in nontertiary hospitals30, 34, it is crucial for all hospitals that care for high-risk infants to maintain a standard quality of care. It is unclear how referral rates are related to other medical outcomes of VLBW infants, and future studies should investigate whether referral rates correlate with other markers of quality of care. Additional investigation of the specific NICUs and regions with lower referral rates is necessary to better delineate remaining barriers to referral, identify further opportunities for improvement and guide resource allocation and utilization.
Referral was associated with different factors in the pre-initiative period compared with the postinitiative period. In the pre-initiative period, although presence of major morbidity or congenital anomaly both had higher referral rates in unadjusted analyses, they paradoxically had lower odds of referral in regression analyses. This is similar to findings in the previous analysis of referral in California, which may be due to adjustment for multiple factors, but also could be because HRIF referral may be missed for infants who otherwise may have several other subspecialty follow-up appointments.14
This study has several strengths. First, we had the unique opportunity to use a population-based dataset that includes nearly all VLBW infants in California to provide a comprehensive analysis of HRIF referral patterns. No other state has an existing infrastructure that allows referral to HRIF to be tracked and analyzed. Ensuring that all eligible infants are referred to HRIF and assessing any barriers to referral is crucial to ensuring actual follow-up. In addition, this study demonstrates that the statewide initiative targeting all eligible VLBW infants in California resulted in a reduction in disparities in HRIF referral throughout the state; this has the potential to serve as a model and be replicated in other states and frameworks.
A limitation of this study is that it is observational, and we cannot definitively conclude that the initiative caused the increase in referral rates. There may be other unmeasured factors that contributed to the increase in referral rates. Our results show that referral rates were increasing each year prior to the implementation of the initiative, and adjustment for year may serve as a proxy for secular trends caused by other factors. Following implementation of the initiative in July 2013, referral rates did not improve in the subsequent six months, which is when ongoing educational efforts regarding the initiative were occurring. The greatest increase in referral rates between years was observed from 2013 to 2014. This aligns with the expectation set that 100% of VLBW infants born in 2014 be referred to HRIF by all CPQCC NICUs. We considered alternatives to our statistical approach to assess the effect of the initiative, including evaluating models checking interaction of the initiative with each covariate. We ultimately preferred to have distinct models for each time period, due to parsimony and to facilitate comparisons with the prior study, although we acknowledge that other approaches could also be applied (eg, regression discontinuity or further tests of interaction).
Another limitation is that it is possible that information on referral may be missing for some infants, leading to an underestimation of referral in either the pre- or post-initiative periods. This is unlikely given that CPQCC and CPQCC-CCS HRIF databases are linked by a robust linkage algorithm based on several different variables. Any missing data on referral are likely randomly distributed in the pre- and post-initiative periods, because we have no reason to suspect that referral would be either under- or over-captured in the database in either period. In addition, our analyses had very little missing data with our final models for the pre- and post-initiative periods including more than 94% of all eligible infants. Finally, we used conventional criteria for statistical significance and precision (p<0.05 and 95% confidence intervals) given the relatively narrow scope of our analysis but acknowledge that more conservative cut-offs would further minimize any false positive findings.
In summary, these results demonstrate the effects of a targeted quality improvement initiative to increase referral of VLBW infants to HRIF at NICU discharge in California, provide an analysis of several key factors associated with success, and highlight ongoing challenges. The remaining disparities associated with referral provide opportunities for additional initiatives focused on lower-level and lower-volume NICUs, and further analysis of persistent racial disparities related to referral. These evaluations will help guide resource and fiscal planning at the local and state level. Further research is needed to evaluate how these referral rates translate to actual HRIF attendance in California.
Acknowledgments
Supported by the National Institutes of Health (F32 HD096778–01). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Also supported in part by the Stanford Maternal and Child Health Research Institute, and the American Academy of Pediatrics Marshall Klaus Neonatal-Perinatal Health Services Research Award.
Abbreviations:
- VLBW
very low birthweight
- NICU
neonatal intensive care unit
- HRIF
high-risk infant follow-up
- CPQCC
California Perinatal Quality Care Collaborative
- CCS
California Children’s Services
- SGA
small for gestational age
- CI
confidence interval
- OR
odds ratio
Footnotes
Publisher's Disclaimer: This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Portions of this study were presented at the Pediatric Academic Societies Meeting, April 28, 2019, Baltimore, Maryland; Stanford Department of Pediatrics Research Retreat, April 19, 2019, Palo Alto, California; California Association of Neonatologists Annual Meeting, March 1, 2019, Coronado, California.
The authors declare no conflicts of interest.
References
- [1].Fanaroff AA, Wright LL, Stevenson DK, Shankaran S, Donovan EF, Ehrenkranz RA, et al. Very-low-birth-weight outcomes of the National Institute of Child Health and Human Development Neonatal Research Network, May 1991 through December 1992. Am J Obstet Gynecol 1995;173:1423–31. [DOI] [PubMed] [Google Scholar]
- [2].Fanaroff AA, Hack M, Walsh MC. The NICHD neonatal research network: changes in practice and outcomes during the first 15 years. Semin Perinatol. 2003;27:281–7. [DOI] [PubMed] [Google Scholar]
- [3].Fanaroff AA, Stoll BJ, Wright LL, Carlo WA, Ehrenkranz RA, Stark AR, et al. Trends in neonatal morbidity and mortality for very low birthweight infants. Am J Obstet Gynecol 2007;196:147.e1–8. [DOI] [PubMed] [Google Scholar]
- [4].Stoll BJ, Hansen NI, Bell EF, Walsh MC, Carlo WA, Shankaran S, et al. Trends in Care Practices, Morbidity, and Mortality of Extremely Preterm Neonates, 1993–2012. JAMA. 2015;314:1039–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Younge N, Goldstein RF, Bann CM, Hintz SR, Patel RM, Smith PB, et al. Survival and Neurodevelopmental Outcomes among Periviable Infants. N Engl J Med 2017;376:617–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Hintz SR, Kendrick DE, Wilson-Costello DE, Das A, Bell EF, Vohr BR, et al. Early-childhood neurodevelopmental outcomes are not improving for infants born at <25 weeks’ gestational age. Pediatrics. 2011;127:62–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Stoll BJ, Hansen NI, Bell EF, Shankaran S, Laptook AR, Walsh MC, et al. Neonatal outcomes of extremely preterm infants from the NICHD Neonatal Research Network. Pediatrics. 2010;126:443–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].Newborn AAoPCoFa. Hospital discharge of the high-risk neonate. Pediatrics. 2008;122:1119–26. [DOI] [PubMed] [Google Scholar]
- [9].Vohr B, Wright LL, Hack M, Aylward G, D H Follow-up care of high-risk infants. Pediatrics. 2004;114 (Suppl 1):1377–97. [Google Scholar]
- [10].Ballantyne M, Stevens B, Guttmann A, Willan AR, Rosenbaum P. Maternal and infant predictors of attendance at Neonatal Follow-Up programmes. Child Care Health Dev 2014;40:250–8. [DOI] [PubMed] [Google Scholar]
- [11].Harmon SL, Conaway M, Sinkin RA, Blackman JA. Factors associated with neonatal intensive care follow-up appointment compliance. Clin Pediatr (Phila) 2013;52:389–96. [DOI] [PubMed] [Google Scholar]
- [12].Perenyi A, Katz J, Flom P, Regensberg S, Sklar T. Analysis of compliance, morbidities and outcome in neurodevelopmental follow-up visits in urban African-American infants at environmental risk. J Dev Orig Health Dis 2010;1:396–402. [DOI] [PubMed] [Google Scholar]
- [13].Wolke D, Söhne B, Ohrt B, Riegel K. Follow-up of preterm children: important to document dropouts. Lancet. 1995;345:447. [DOI] [PubMed] [Google Scholar]
- [14].Hintz SR, Gould JB, Bennett MV, Gray EE, Kagawa KJ, Schulman J, et al. Referral of very low birth weight infants to high-risk follow-up at neonatal intensive care unit discharge varies widely across California. J Pediatr 2015;166:289–95. [DOI] [PubMed] [Google Scholar]
- [15].Horbar JD. The Vermont-Oxford Neonatal Network: integrating research and clinical practice to improve the quality of medical care. Semin Perinatol. 1995;19:124–31. [DOI] [PubMed] [Google Scholar]
- [16].Jaro MA. Probabilistic linkage of large public health data files. Stat Med 1995;14:491–8. [DOI] [PubMed] [Google Scholar]
- [17].Herrchen B, Gould JB, Nesbitt TS. Vital statistics linked birth/infant death and hospital discharge record linkage for epidemiological studies. Comput Biomed Res 1997;30:290–305. [DOI] [PubMed] [Google Scholar]
- [18].Newborn AAoPCoFA. Levels of neonatal care. Pediatrics. 2012;130:587–97. [DOI] [PubMed] [Google Scholar]
- [19].Chyi LJ, Lee HC, Hintz SR, Gould JB, Sutcliffe TL. School outcomes of late preterm infants: special needs and challenges for infants born at 32 to 36 weeks gestation. J Pediatr 2008;153:25–31. [DOI] [PubMed] [Google Scholar]
- [20].Woythaler MA, McCormick MC, Smith VC. Late preterm infants have worse 24-month neurodevelopmental outcomes than term infants. Pediatrics. 2011;127:e622–9. [DOI] [PubMed] [Google Scholar]
- [21].Shah P, Kaciroti N, Richards B, Oh W, Lumeng JC. Developmental Outcomes of Late Preterm Infants From Infancy to Kindergarten. Pediatrics. 2016;138. [DOI] [PMC free article] [PubMed]
- [22].Petrini JR, Dias T, McCormick MC, Massolo ML, Green NS, Escobar GJ. Increased risk of adverse neurological development for late preterm infants. J Pediatr 2009;154:169–76. [DOI] [PubMed] [Google Scholar]
- [23].Schonhaut L, Armijo I, Pérez M. Gestational age and developmental risk in moderately and late preterm and early term infants. Pediatrics. 2015;135:e835–41. [DOI] [PubMed] [Google Scholar]
- [24].Curry AE, Pfeiffer MR, Slopen ME, McVeigh KH. Rates of early intervention referral and significant developmental delay, by birthweight and gestational age. Matern Child Health J 2012;16:989–96. [DOI] [PubMed] [Google Scholar]
- [25].Morse SB, Zheng H, Tang Y, Roth J. Early school-age outcomes of late preterm infants. Pediatrics. 2009;123:e622–9. [DOI] [PubMed] [Google Scholar]
- [26].Profit J, Gould JB, Bennett M, Goldstein BA, Draper D, Phibbs CS, et al. Racial/Ethnic Disparity in NICU Quality of Care Delivery. Pediatrics. 2017;140. [DOI] [PMC free article] [PubMed]
- [27].Gould JB, Sarnoff R, Liu H, Bell DR, Chavez G. Very low birth weight births at non-NICU hospitals: the role of sociodemographic, perinatal, and geographic factors. J Perinatol 1999;19:197–205. [DOI] [PubMed] [Google Scholar]
- [28].Lake ET, Staiger D, Horbar J, Kenny MJ, Patrick T, Rogowski JA. Disparities in perinatal quality outcomes for very low birth weight infants in neonatal intensive care. Health Serv Res 2015;50:374–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [29].Lake ET, Staiger D, Edwards EM, Smith JG, Rogowski JA. Nursing Care Disparities in Neonatal Intensive Care Units. Health Serv Res 2018;53 Suppl 1:3007–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Phibbs CS, Baker LC, Caughey AB, Danielsen B, Schmitt SK, Phibbs RH. Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants. N Engl J Med 2007;356:2165–75. [DOI] [PubMed] [Google Scholar]
- [31].Cifuentes J, Bronstein J, Phibbs CS, Phibbs RH, Schmitt SK, Carlo WA. Mortality in low birth weight infants according to level of neonatal care at hospital of birth. Pediatrics. 2002;109:745–51. [DOI] [PubMed] [Google Scholar]
- [32].Apfeld JC, Kastenberg ZJ, Sylvester KG, Lee HC. The Effect of Level of Care on Gastroschisis Outcomes. J Pediatr 2017;190:79–84.e1. [DOI] [PubMed] [Google Scholar]
- [33].Kastenberg ZJ, Lee HC, Profit J, Gould JB, Sylvester KG. Effect of deregionalized care on mortality in very low-birth-weight infants with necrotizing enterocolitis. JAMA Pediatr 2015;169:26–32. [DOI] [PubMed] [Google Scholar]
- [34].Haberland CA, Phibbs CS, Baker LC. Effect of opening midlevel neonatal intensive care units on the location of low birth weight births in California. Pediatrics. 2006;118:e1667–79. [DOI] [PubMed] [Google Scholar]