Characteristics and outcomes in preterm infants ≤26 weeks’ gestation with early HRF at 18 to 26 months follow-up.
Abstract
OBJECTIVES:
To evaluate the survival and neurodevelopmental impairment (NDI) in extremely low birth weight (ELBW) infants at 18 to 26 months with early hypoxemic respiratory failure (HRF). We also assessed whether African American infants with early HRF had improved outcomes after exposure to inhaled nitric oxide (iNO).
METHODS:
ELBW infants ≤1000 g and gestational age ≤26 weeks with maximal oxygen ≥60% on either day 1 or day 3 were labeled as “early HRF” and born between 2007 and 2015 in the Neonatal Research Network were included. Using a propensity score regression model, we analyzed outcomes and effects of exposure to iNO overall and separately by race.
RESULTS:
Among 7639 ELBW infants born ≤26 weeks, 22.7% had early HRF. Early HRF was associated with a mortality of 51.3%. The incidence of moderate-severe NDI among survivors was 41.2% at 18 to 26 months. Mortality among infants treated with iNO was 59.4%. Female sex (adjusted odds ratio [aOR]: 2.4, 95% confidence interval [CI]: 1.8–3.3), birth weight ≥720 g (aOR: 2.3, 95% CI: 1.7–3.1) and complete course of antenatal steroids (aOR: 1.6, 95% CI: 1.1–2.2) were associated with intact survival. African American infants had a similar incidence of early HRF (21.7% vs 23.3%) but lower exposure to iNO (16.4% vs 21.6%). Among infants with HRF exposed to iNO, intact survival (no death or NDI) was not significantly different between African American and other races (aOR: 1.5, 95% CI: 0.6–3.6).
CONCLUSIONS:
Early HRF in infants ≤26 weeks’ gestation is associated with high mortality and NDI at 18 to 26 months. Use of iNO did not decrease mortality or NDI. Outcomes following iNO exposure were not different in African American infants.
What’s Known on This Subject:
The incidence of early hypoxemic respiratory failure and inhaled nitric oxide therapy are common in preterm infants ≤26 weeks’ gestation. There is limited information regarding developmental outcomes and survival at 18 to 26 months by race outside randomized trials.
What This Study Adds:
In preterm infants ≤26 weeks’ gestation, early hypoxemic respiratory failure is associated with high mortality and severe neurodevelopmental impairment among survivors at 18 to 26 months. Therapy with inhaled nitric oxide did not reduce mortality or severe neurodevelopmental impairment in preterm infants of all races.
Surfactant therapy, advanced ventilation techniques, and continuous positive airway pressure have significantly reduced pulmonary morbidity in extremely preterm infants.1 Despite these measures, some preterm infants continue to suffer from hypoxemic respiratory failure (HRF).2 Some of these infants may have associated persistent pulmonary hypertension of newborn (PPHN).3 Off-label use of inhaled nitric oxide (iNO) is not recommended in preterm infants.4,5 However, newer guidelines from various societies recommend the selective use of iNO among preterm infants with HRF associated with PPHN physiology.6–9 Limited data are available from randomized controlled trials (RCTs) to guide neonatal providers in counseling parents regarding the neurodevelopmental outcomes of HRF at ≤26 weeks’ gestation in the first few postnatal days.2,10 Observational studies demonstrate the widespread use of iNO for early HRF in the NICU11,12 without any evidence of benefit, even among infants with PPHN. Also, iNO exposure was associated with higher mortality among preterm neonates whose respiratory distress syndrome (RDS) was not accompanied by PPHN even after adjusting for birth size and maximum support rating.12 However, these studies do not include neurodevelopmental outcomes after discharge from the NICU.
A recent individual participant data meta-analysis suggests a significant reduction in the composite outcome of death or bronchopulmonary dysplasia (BPD) with iNO treatment among African American (AA) infants.13 AA adults were found to have lower bioavailability of NO compared with white individuals, providing biological plausibility for a differential effect by race.14,15 It is unknown if survival and long-term neurodevelopmental benefits are seen in infants with early HRF treated with iNO. Our primary objective was to report the 18- to 26-month outcomes of extremely low birth weight (ELBW) infants born at ≤26 weeks’ gestation at birth with early HRF. We hypothesized that extremely preterm infants with early HRF would have higher morbidity, mortality, and neurodevelopmental impairment (NDI) at 18 to 26 months of age compared with those without HRF. We further hypothesized that iNO would not improve survival, BPD, or neurodevelopmental outcomes among ELBW infants with early HRF. Finally, we also hypothesized that ELBW infants with HRF whose parents self-reported as African American would have lower mortality, BPD, and NDI after exposure to iNO.13,16
Methods
The Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Neonatal Research Network (NRN) Generic Database prospectively collects data on preterm infants with a birth weight (BW) of ≤1000 g and gestational age (GA) ≤26 weeks. Follow-up data on these infants at 18 to 26 months are available. These data are collected at each site after institutional review board approval. We included all infants born from January 1, 2007, to December 31, 2015.
Data on Early HRF
On day 1 and day 3, the maximum fraction of inspired oxygen (Fio2) was recorded. Infants whose maximal Fio2 was ≥0.6 on either of these days were labeled as “early HRF.”17 Because higher inspired oxygen requirement is common on the day of birth (day 0) among preterm infants, we did not include maximal Fio2 on that day in our definition of early HRF. Exposure to iNO during the neonatal period (first 28 days of postnatal age) was also collected.8,18 BW, GA, sex, race, ethnicity, duration of rupture of membranes (ROM), Apgar scores, antenatal steroids (ANS), and maternal chorioamnionitis data were collected. A complete course of ANS was defined as a mother receiving 2 doses of steroids before delivery. The prolonged preterm rupture of membranes (PPROM) was defined as ROM for ≥18 hours before delivery. Infants with congenital anomalies and those treated with iNO after 28 days of life were excluded. We further analyzed a subgroup of infants with HRF whose mothers self-identified as African American and studied the effect of iNO exposure in this population.
Data on Discharge Outcomes
Clinical characteristics such as BPD defined by the oxygen reduction test,19 patent ductus arteriosus (PDA), culture-positive sepsis (both early and late), necrotizing enterocolitis (NEC), retinopathy of prematurity (ROP), intraventricular hemorrhage (IVH), and periventricular leukomalacia (PVL) were collected (definitions in Supplemental Materials).
Data on Outcomes at 18 to 22 Months
The early HRF group was analyzed for characteristics associated with survival at discharge and at 18 to 26 months follow-up. Outcomes at 18 to 26 months between infants exposed versus not exposed to iNO were evaluated. Neurodevelopmental assessments at the corrected age 18 to 22 months (for infants born January 1, 2007, through June 30, 2012) or corrected age 22 to 26 months (for infants born July 1, 2012, through December 31, 2015) were performed by certified examiners. The Bayley Scales of Infant Development 3 (Bayley-III) includes composite scores (mean of 100 and a SD of 15) for cognitive, language, and motor skills. Scores range from 55 to 145, with a lower score indicating a greater degree of delay.20 A standardized neurosensory examination was also performed. The severity of cerebral palsy was defined on the basis of the Gross Motor Function Classification System level, consisting of a scale from I (mild impairment) to V (severe impairment).21,22
For purposes of this analysis, the composite outcome of NDI consists of at least 1 of the following outcomes: moderate-severe cerebral palsy (Gross Motor Function Classification System level ≥Il), a cognitive composite score of <85 on the Bayley-III, severe hearing loss (bilateral permanent hearing loss that interferes with the ability to understand or communicate with or without amplification), or severe visual impairment (no usable vision, usually defined as legally blind).20
Statistical Approach
Infant demographic and maternal characteristics were summarized and compared by using Student t test for continuous variables and χ2 tests for categorical variables. Specifically, unadjusted comparisons of demographic and clinical outcomes between all infants with and without HRF were performed. Additionally, among infants with HRF, exposure to iNO versus no exposure was compared.
A multivariable logistic regression model was constructed to evaluate factors associated with early HRF. A propensity score to quantify the probability of receiving iNO therapy for each subject was created from a multivariable logistic regression model. The propensity scores were ranked and classified into similar groups on the basis of their ranks. Six groups were created that included infants with and without iNO. Subjects within each group had a similar probability of iNO treatment and were balanced on key covariates.23,24 Separate multivariable logistic regression models were then constructed to identify factors associated with survival at discharge and intact survival at 18 to 26 months follow-up, adjusting for the propensity scores and other covariates.
A list of a priori risk factors was considered in all the initial models for propensity scores, risk factors for HRF, survival at discharge, and intact survival at follow-up. The list included the following variables: BW, GA, sex, ethnicity, duration of ROM, Apgar scores, ANS, maternal chorioamnionitis, delivery by cesarean delivery (small for gestational age [SGA]), birth centers, year of birth, and use of iNO. The PPROM and ANS were not correlated, and both the variables were considered in the model. Additionally, the groups based on the propensity scores were also considered as a covariate in both the models for survival at discharge and intact survival at follow-up models to even out any imbalance. The final model for each outcome was selected by using the backward elimination method, in which covariates with P > .05 were dropped from the model. In the model for intact survival, PPROM was not significant but was added in the final model as a predefined clinically important factor.
Statistical interaction between PPROM and ANS was evaluated and was significant only in the model for survival at discharge. The interaction effects between iNO and other covariates were evaluated and were not significant in any of the models and were excluded. Specifically, the interaction between iNO and AA race was not significant (P = .323). As a predefined secondary objective, subgroup analyses for only AA infants were performed, including an adjusted model for survival without physiologic BPD at discharge and intact survival at follow-up.
Model fit for all the logistic models were assessed by using Hosmer–Lemeshow test. The time of death in infants with and without HRF was compared by using the Kaplan-Meier curves. All reported P values are 2-sided and considered statistically significant at <.05. Analyses were performed with SAS/STAT software, Version 9.4 (SAS Institute, Inc, Cary, NC).
Results
During the period 2007 to 2015, there were 7639 ELBW infants born at ≤26 weeks’ gestation within the NRN centers (Fig 1). A total of 1732 (23%) of these infants were exposed to ≥0.6 Fio2 during days 1 and 3 and were labeled early HRF.
Characteristics and Outcome of Early HRF
ELBW infants with early HRF had lower BW, GA, and 5 min Apgar scores compared with infants without early HRF (Table 1). The incidence of PPROM and ANS was significantly lower in the early HRF group. Preterm infants with lower BW (<720 g), male sex, SGA, or delivery by cesarean were more likely to be associated with early HRF (Supplemental Table 6). The BW of 720 g was chosen as a cutoff after including weight as a continuous variable in the initial statistical analysis. Higher GA, maternal chorioamnionitis, PPROM, and ANS decreased the risk of HRF.
TABLE 1.
Characteristics | Early HRF, n = 1732 | Infants Without Early HRF, n = 5907 | P |
---|---|---|---|
Birth wt, g, mean ± SD | 664.5 ± 144.9 | 727.9 ± 139.4 | <.0001 |
GA, wk, mean ± SD | 24.4 ± 1.1 | 24.8 ± 1.0 | <.0001 |
Male sex, n (%) | 946 of 1731 (54.7) | 2889 of 5905 (48.9) | <.0001 |
Small for GA, n (%) | 208 of 1731 (12.0) | 371 of 5905 (6.3) | <.0001 |
PROM, n (%) | |||
>18 h | 336 of 1683 (20.0) | 1609 of 5745 (28.0) | <.0001 |
>24 h | 289 of 1666 (17.3) | 1419 of 5686 (25.0) | <.0001 |
>120 h | 176 of 1668 (10.6) | 753 of 5707 (13.2) | .0042 |
Race, n (%) | |||
AA | 685 of 1732 (39.5) | 2469 of 5907 (41.8) | .0947 |
White | 722 of 1732 (41.7) | 2226 of 5907 (37.7) | .0026 |
Hispanic | 250 of 1732 (14.4) | 864 of 5907 (14.6) | .8418 |
Other | 63 of 1732 (3.6) | 319 of 5907 (5.4) | .0031 |
Chorioamnionitis, n (%) | 244 of 1716 (14.2) | 1087 of 5872 (18.5) | <.0001 |
ANS, n (%) | 983 of 1722 (57.1) | 3926 of 5865 (66.9) | <.0001 |
Delivery by cesarean section, n (%) | 1141 of 1729 (66.0) | 3695 of 5904 (62.6) | .0097 |
Apgar scores <4 at 1 min, n (%) | 1091 of 1714 (63.7) | 2952 of 5864 (50.3) | <.0001 |
Apgar scores <4 at 5 min, n (%) | 427 of 1716 (24.9) | 886 of 5878 (15.1) | <.0001 |
BPD, traditional, n (%) | 741 of 977 (75.8) | 2867 of 4923 (58.2) | <.0001 |
BPD, physiologic definition, n (%) | 713 of 971 (73.4) | 2755 of 4883 (56.4) | <.0001 |
PDA, treatment and surgery, n (%) | 899 of 1728 (52.0) | 3143 of 5898 (53.3) | .3546 |
Postnatal steroid use for BPD, n (%) | 347 of 1636 (21.2) | 1050 of 5663 (18.5) | .0156 |
NEC stage II/III, n (%) | 172 of 1731 (9.9) | 722 of 5907 (12.2) | .0093 |
Early-onset sepsis, n (%) | 59 of 1731 (3.4) | 149 of 5902 (2.5) | .047 |
Late-onset of sepsis, n (%) | 510 of 1504 (33.9) | 1824 of 5730 (31.8) | .1251 |
ROP stage ≥III, n (%) | 298 of 989 (30.1) | 1062 of 4929 (21.5) | <.0001 |
Grade 3 or 4 IVH and PVL, n (%) | 630 of 1614 (39.0) | 942 of 5770 (16.3) | <.0001 |
Length of stay survivors, d, mean ± SD | 132.9 ± 55.8 | 117.1 ± 46.2 | <.0001 |
Mortality in the NICU, n (%) | 800 of 1727 (46.3) | 1073 of 5896 (18.2) | <.0001 |
18–26-mo outcomes among NICU survivors; neurodevelopment outcome of infants with or without HRF | n = 814 | n = 4095 | |
Intact survival, no death or severe NDI, n (%) | 460 of 812 (56.7) | 2728 of 4090 (66.7) | <.0001 |
Death after NICU discharge, n (%) | 30 of 814 (3.7) | 60 of 4095 (1.5) | <.0001 |
Moderate-severe NDI, n (%) | 322 of 782 (41.2) | 1302 of 4030 (32.3) | <.0001 |
Cerebral palsy, n (%) | 173 of 783 (22.1) | 518 of 4032 (12.8) | <.0001 |
Moderate to severe cerebral palsy, n (%) | 100 of 782 (12.8) | 254 of 4032 (6.3) | <.0001 |
Bayley-III cognitive, mean ± SD | 84.4 ± 16.7 | 88.2 ± 15.1 | <.0001 |
Hearing impairment, n (%) | 35 of 783 (4.5) | 112 of 4028 (2.8) | .012 |
Vision impairment, n (%) | 22 of 781 (2.8) | 53 of 4032 (1.3) | .0019 |
Thirty infants died after NICU discharge before follow-up (Fig 1). Of the 927 infants with early HRF discharged from the NICU, 115 (12.5%) infants were lost to follow-up or did not have complete data on neurodevelopmental outcomes available. The overall incidence of death (in-hospital plus postdischarge) and/or moderate-severe NDI in this population was 71.2% (1152 of 1617 patients). A nonresponse bias analysis to account for infants lost to follow-up was performed, and there was no significant difference in the characteristics other than the treatment of PDA, which was higher in the group with follow-up (Supplemental Table 7). Among 782 infants with complete follow-up data, 41.2% (322 of 782) had moderate-severe NDI. Among infants with early HRF, higher BW (≥720 g), higher GA, female sex, Apgar score >4 at 1 minute and at 5 minutes, prolonged rupture of membrane (PROM) >18 hours and/or ANS, and center were associated with increased survival to discharge from the NICU (Table 2). Exposure to iNO was associated with reduced survival to NICU discharge. Preterm infants ≤26 weeks with HRF had significantly lower survival probability compared with those without HRF as shown in Fig 2. ANS, higher BW (≥720 g), and female sex were associated with intact survival at 18 to 26 months (Table 3).
TABLE 2.
Factors Associated With Survival at Discharge in ELBW Infants With O2 ≥ 60% on Day 1 and Day 3 | P | aOR | 95% CI | 95% CI |
---|---|---|---|---|
Birth wt ≥720 g, yes versus no | <0.0001 | 2.028 | 1.567 | 2.623 |
GA | <0.0001 | 1.400 | 1.254 | 1.562 |
Female, yes versus no | 0.0046 | 1.367 | 1.101 | 1.696 |
AA, yes versus no | 0.0014 | 1.390 | 1.088 | 1.777 |
Apgar >4 at first minute, yes versus no | 0.0085 | 1.291 | 1.009 | 1.652 |
Apgar >4 at fifth minute, yes versus no | 0.0420 | 1.462 | 1.118 | 1.913 |
Not exposed to iNO, yes versus no | <0.0001 | 1.967 | 1.467 | 2.638 |
PROM >18 and ANS | ||||
Both PROM and ANS versus No PROM or ANS | <0.0001 | 1.535 | 1.108 | 2.127 |
Only ANS versus No PROM or ANS | <0.0001 | 1.450 | 1.124 | 1.870 |
Only PROM versus No PROM or ANS | 0.3401 | 1.261 | 0.692 | 2.298 |
Center variation | <0.0001 | Differs with each center (Supplemental Table 8) | Differs with each center (Supplemental Table 8) | Differs with each center (Supplemental Table 8) |
TABLE 3.
Factors Associated With Intact Survival at 18–26 mo in ELBW Infants With O2 ≥ 60% on Day 1 and Day 3 | P | aOR | 95% CI | 95% CI |
---|---|---|---|---|
Birth wt ≥ 720 g, yes versus no | <0.0001 | 2.277 | 1.669 | 3.107 |
Female, yes versus no | <0.0001 | 2.427 | 1.789 | 3.293 |
PROM >18 hr | 0.2134 | 0.999 | 0.999 | 1.000 |
Complete course of ANS, yes versus no | 0.0082 | 1.551 | 1.120 | 2.147 |
Exposure to iNO and Outcomes in Early HRF
As expected, HRF was also associated with increased exposure to iNO, with considerable variation in exposure between centers. Extremely preterm infants with early HRF with the following characteristics were more likely to receive iNO: PPROM, ANS, Apgar score <4 at 1 minute, and birth in the year 2011 or after. There was a wide variation between centers regarding iNO use (range: 1%–38%, Supplemental Table 8). Characteristics of infants with early HRF based on exposure to iNO are shown in Table 4.
TABLE 4.
Characteristics | Exposed to iNO, n = 338 | Not Exposed to iNO, n = 1394 | P |
---|---|---|---|
Birth wt, g, mean ± SD | 652.4 ± 150.8 | 667.5 ± 143.3 | .085 |
GA, wk, mean ± SD | 24.4 ± 1.2 | 24.4 ± 1.1 | .8154 |
Male sex, n (%) | 185 of 338 (54.7) | 761 of 1393 (54.6) | .9727 |
Small for GA, n (%) | 57 of 338 (16.9) | 151 of 1393 (10.8) | .0022 |
ROM, n (%) | |||
>18 h | 100 of 320 (31.3) | 236 of 1363 (17.3) | <.0001 |
>24 h | 88 of 314 (28.0) | 201 of 1352 (14.9) | <.0001 |
>120 h | 63 of 316 (19.9) | 113 of 1352 (8.4) | <.0001 |
Race, n (%) | |||
AA | 112 of 338 (33.1) | 573 of 1394 (41.1) | .0072 |
White | 167 of 338 (49.4) | 555 of 1394 (39.8) | .0013 |
Hispanic | 44 of 338 (13.0) | 206 of 1394 (14.8) | .4088 |
Other | 13 of 338 (3.8) | 50 of 1394 (3.6) | .8193 |
Chorioamnionitis, n (%) | 60 of 331 (18.1) | 184 of 1385 (13.3) | .0235 |
ANS, n (%) | 226 of 334 (67.7) | 757 of 1388 (54.5) | <.0001 |
Delivery by cesarean, n (%) | 231 of 337 (68.5) | 910 of 1392 (65.4) | .27 |
Apgar scores <4 at 1 min, n (%) | 231 of 334 (69.2) | 860 of 1380 (62.3) | .0197 |
Apgar scores <4 at 5 min, n. (%) | 91 of 334 (27.2) | 336 of 1382 (24.3) | .2659 |
Exposure to iNO was associated with reduced survival to NICU discharge. Infants with early HRF exposed to iNO in the neonatal period had a higher rate of combined morbidities and in-hospital mortality of 54.1% compared with 44.4% in infants not exposed to iNO. There was no significant difference in survival when early HRF infants exposed to iNO were stratified on the basis of PPROM (Supplemental Table 9). In addition, when we compare infants with no PROM or PROM <120 hours, the survival is 118 of 251 = 47%, which is identical to survival of 45% with PROM >120 hours. The incidence of BPD, PDA requiring treatment, sepsis, length of stay, and mortality were higher with exposure to iNO among extremely preterm infants with HRF (Table 5). There was no difference in the incidence of grade 3 or 4 IVH and PVL (34.6% with iNO exposure vs 40.2% among infants not exposed to iNO, P = .0653). The incidence of morbidities such as cerebral palsy and hearing and vision impairment at discharge were not different among survivors with early HRF exposed to iNO and not exposed to iNO (Table 5).
TABLE 5.
Characteristics | Exposed to iNO, n = 338 | Not Exposed to iNO, n = 1394 | P |
---|---|---|---|
BPD, traditional, n (%) | 157 of 172 (91.3) | 584 of 805 (72.5) | <.0001 |
BPD, physiologic definition, n (%) | 157 of 171 (91.8) | 556 of 800 (69.5) | <.0001 |
PDA, treatment and surgery, n (%) | 202 of 337 (59.9) | 697 of 1391 (50.1) | .0012 |
Postnatal steroid use for BPD, n (%) | 107 of 324 (33.0) | 240 of 1312 (18.3) | <.0001 |
NEC stage II/III, n (%) | 27 of 338 (8.0) | 145 of 1393 (10.4) | .182 |
Early-onset sepsis, n (%) | 18 of 338 (5.3) | 41 of 1393 (2.9) | .0304 |
Late-onset of sepsis, n (%) | 124 of 312 (39.7) | 386 of 1192 (32.4) | .0145 |
ROP stage ≥III, n (%) | 63 of 178 (35.4) | 235 of 811 (29.0) | .0911 |
Grade 3 or 4 IVH and PVL, n (%) | 112 of 324 (34.6) | 518 of 1290 (40.2) | .0653 |
Length of stay survivors, d, mean ± SD | 150.8 ± 70.2 | 129.3 ± 51.8 | <.0001 |
Mortality in the NICU, n (%) | 183 of 338 (54.1) | 617 of 1389 (44.4) | .0013 |
18–26 mo, characteristics among NICU survivors | n = 137 | n = 677 | |
Intact survival, no death or severe NDI, n (%) | 73 of 137 (53.3) | 387 of 675 (57.3) | .3833 |
Death after NICU discharge, n (%) | 7 of 137 (5.1) | 23 of 677 (3.4) | .332 |
Moderate-severe NDI, n (%) | 57 of 130 (43.8) | 265 of 652 (40.6) | .4982 |
Cerebral palsy, n (%) | 32 of 129 (24.8) | 141 of 654 (21.6) | .4166 |
Moderate to severe cerebral palsy, n (%) | 16 of 129 (12.4) | 84 of 653 (12.9) | .8862 |
Bayley-III cognitive, mean ± SD | 84.1 ± 17.4 | 84.4 ± 16.6 | .8339 |
Hearing impairment, n (%) | 7 of 130 (5.4) | 28 of 653 (4.3) | .5805 |
Loss of vision, n (%) | 3 of 129 (2.3) | 19 of 652 (2.9) | .712 |
AA ELBW Infants
AA preterm infants had a similar incidence of early HRF (21.7% vs 23.3%) compared with other races (Fig 3). AA infants with HRF had a significantly lower rate (16.35%) of iNO use compared with other races (21.6%) (P = .0072). We did observe center variations in the use of iNO in AA infants. Two centers had higher rates of iNO use in AA infants when compared with other races, whereas 2 centers used less iNO (Supplemental Table 8). Overall, some centers with more AA infants used less iNO among all infants, accounting for differences in rates of iNO use (Supplemental Table 8). The number from each center and the decision to use iNO appears to be variable based on individual provider preference, center protocols and different interpretation of the literature. Among extremely preterm infants with early HRF exposed to iNO, there was no difference in the incidence of physiologic BPD among AAs (86.8%), as compared with infants of other races (94.1%, P = .1). Among AA infants with early HRF, 70% developed BPD compared with 50% without early HRF. There was no difference in BPD rates in AA infants with early HRF (70.1%) compared with other races (75.6%, P = .056).
Among all extremely preterm infants (with and without early HRF), AA infants had similar NICU mortality (25.4% vs 23.9%) but higher postdischarge mortality by 18 to 26 months (2.1% vs 1.1%). Adjusted analyses considering only AA infants revealed similar findings as the overall models for survival at discharge and intact survival at follow-up. AA infants overall had higher chances of survival compared with non-AA infants (adjusted odds ratio [aOR]: 1.4, 95% confidence interval [CI]: 1.09–1.8). However, within the AA population, death and NDI in infants with HRF was much higher (473 of 641: 89.3%) compared with those infants without HRF (1152 of 2198: 52.4%). Among HRF infants exposed to iNO, intact survival (no death or NDI) was not significantly different between AA and other races (aOR: 1.5, confidence interval: 0.6–3.6) (Supplemental Table 10). The AA infants who did not receive iNO had higher survival without physiologic BPD at discharge (aOR: 4.6, 95% confidence interval: 2.0–11.0). The mortality before discharge in this subgroup with HRF was 45.7%, as compared with 19% without HRF. For the AA subgroup, infants with BW >720 g and female sex had higher survival at discharge. Similarly, for the AA infants, the intact survival at follow-up was not significantly different for infants with and without iNO therapy.
Discussion
In our study, we focus on death or intact survival at follow-up as the primary outcome in preterm neonates ≤26 weeks’ gestation with early HRF. Early HRF, as defined as a requirement of ≥60% oxygen on day 1 or day 3, was observed in 22.7% of ELBW infants born at ≤26 weeks’ gestation in our study. A striking finding was that early HRF was associated with higher in-hospital mortality (46.2% vs 18.2%), postdischarge mortality (3.7% vs 1.3%) and severe NDI among survivors (41.2% vs 30%) compared with those without HRF.
The association between early HRF and lower BW, GA, and male sex was expected given the higher association of RDS with these characteristics.25 Interestingly, chorioamnionitis, PROM, and exposure to ANS were associated with lower incidence of early HRF. We speculate that chorioamnionitis and PROM are associated with exposure to inflammatory cytokines and acceleration of lung maturation and lower incidence of early HRF.26 Similarly, exposure to ANS is associated with decreased incidence of RDS, acceleration of fetal lung maturation, along with decreased pulmonary vascular resistance and vasodilator response to nitric oxide and catecholamines.27–29
Despite formal recommendations against such use, almost one-fourth of extremely preterm infants with early HRF were exposed to iNO, with significant center variation.4,5 Among infants with early HRF, factors such as race, SGA status, PPROM, chorioamnionitis, and ANS were associated with a higher incidence of exposure to iNO. These factors are likely associated with increased severity of HRF and/or associated pulmonary hypertension (PH). Also, PPROM can be associated with pulmonary hypoplasia and PH, resulting in therapy with iNO.30 In our study, the survival of 45% in the cohort of infants with PPROM >120 hours may warrant further investigation of the use of iNO in this patient population. Although the duration of ROM was collected, we did not collect information about pulmonary hypoplasia or echocardiographic evidence of PH in our patients. Exposure to iNO was an independent factor associated with mortality at discharge, but no causality was identified. We speculate that the severity of illness is higher in the iNO-treated infants and the higher mortality reflect a “last-ditch effort” use in moribund preterm infants.
Infants exposed to iNO with early HRF had a higher incidence of BPD, PDA, sepsis, mortality, and postnatal steroid use, suggesting higher acuity in these patients. In contrast to the largest RCT in which researchers evaluated iNO for HRF in preterm neonates,2 we observed a tendency toward a lower incidence of severe IVH and PVL in infants exposed to iNO with BW <750 g (P = .0503). Among survivors, iNO did not alter the rate of NDI. The incidence of death or NDI was high (75%) in the current study, and these rates are similar to those reported by Hintz et al31 (87%) in an iNO group enrolled in the NRN trial with BW ≤1000 g). In contrast to the findings of Askie et al,13 who examined prophylactic use of iNO for BPD prevention using individual patient data analysis (in preterm infants <34 weeks), we found that iNO has no impact on short- or long-term outcomes in extremely premature infants (≤26 weeks) with early HRF irrespective of race.32 More recent evidence linking iNO exposure in the NICU to childhood cancer further emphasizes the need for the selective use of iNO among preterm infants with HRF33 and the possible role of free radicals in carcinogenesis.34
This current analysis supports the National Institutes of Health Consensus Development Conference statement discouraging routine rescue therapy with iNO and emphasizes the need for further studies evaluating the selective use of iNO on the basis of risk of BPD, presence of pulmonary hypoplasia, race, or presence of PH.3,7 The use of iNO among preterm infants continues to be high in the United States.35,36 Authors of two editorials recently speculated that the persistent use of iNO in preterm infants is because clinicians feel that the evidence from RCTs cannot be generalized to individual clinical situations.37,38 Similarly, researchers in three other studies have suggested that iNO could be beneficial in preterm neonates if specific criteria are followed.39–41 Providers’ previous experience with iNO, the safety profile in term infants, and finally, a perception of offering patient-centered care with involvement of parents in shared decision-making all might also contribute to high iNO use among preterms with early HRF.42 With our study, we provide additional evidence to support the findings of the NRN RCT, which enrolled critically ill ELBW infants with HRF.2,31
Authors of a recent individual participant data meta-analysis have shown a reduced incidence of BPD after iNO use among preterm infants, with the most significant effect in AA infants.13 Our analysis did not include patient data from randomized trials and has the limitations of a retrospective analysis. However, it includes prospectively collected individual patient data from an established neonatal research network of academic institutions. We therefore evaluated our registry data from a geographically diverse multicenter network in the United States to see if extremely preterm AA infants with early HRF benefited from exposure to iNO. The mortality data for 2017 demonstrate a threefold difference in mortality rate from neonatal respiratory distress of newborn among non-Hispanic white race compared with non-Hispanic Black individuals (8.5 vs 24.1 per 100 000 live births).43 The rate of mortality from chronic respiratory disease originating in the perinatal period was also significantly lower among non-Hispanic white compared with Black individuals (2.4 vs 8.5 per 100 000 live births).43 In our study, among infants with early HRF, without adjusting for factors, the incidence of mortality and physiologic BPD were not different between the AA compared with the other races. We found no benefit in survival or neurodevelopmental outcome with iNO use. Regardless of the use of iNO, extremely preterm AA infants had lower trends of BPD (70.1% vs 75.6% among other races, P = .056). However, after exposure to iNO, no difference in the incidence of BPD was observed. Although the incidence of early HRF is similar among AA infants compared with other races (21.7% vs 23.3%), the use of iNO was significantly lower among them. This difference is related to center-differences in iNO use and differences in the proportion of AA infants. We cannot rule out differences in medication use based on race that has been observed in other studies or differing baseline HRF severity.44 Further investigation to understand racial differences in resource use and higher post-NICU discharge mortality among AA preterm neonates is warranted.
Additional information on the cause of HRF, oxygenation, blood gas parameters, echocardiographic evidence of PH, oligohydramnios, pulmonary hypoplasia, and the time of initiation of iNO and response were not available and are significant limitations to this analysis. Hence, we were unable to stratify and study the effects of iNO on patients with PH, pulmonary hypoplasia on mortality, and NDI. It is possible that a subgroup of patients with oligohydramnios, pulmonary hypoplasia, and PH pathophysiology may respond well to iNO as described previously.8,30 We defined HRF on the basis of inspired oxygen requirements and did not have data on Pao2 or Spo2 to use the standard definition of HRF using oxygenation index. Finally, definition of race was based on maternal self-identification.45 Furthermore, iNO exposure duration may be brief in some of these neonates, and although propensity modeling was done in these neonates, all confounding factors may not have been adjusted. Also, because most grade III/IV IVHs occur during the first 3 days of life, especially in infants with HRF, the exposure to iNO and IVH could be incidental.
There are several strengths to the current study. To our knowledge, this is the first study in which early HRF in extremely preterm infants ≤26 weeks was evaluated. Complete neurodevelopmental and mortality outcome data were available in 93.4% of the original cohort. To date, this is the largest study evaluating early HRF and iNO use in ELBW infants that includes neurodevelopmental outcomes. In combination with other published studies, these outcome data could be used to develop RCTs with improved targeting to specific high-risk groups.
Conclusions
Early HRF in ELBW infants at ≤26 weeks is associated with high morbidity and mortality during the NICU course. Inhaled NO use in this cohort was associated with higher mortality and did not improve neurodevelopmental outcomes at 18 to 26 months. More than three-fourths of ELBW infants with HRF receiving iNO died or had severe NDI at 18 to 26 months. The outcomes following iNO exposure were not different among AA infants. This prognostic information should be discussed with parents during shared-decision making before initiating iNO in ELBW infants with early HRF.42
Acknowledgments
We thank all the parents, neonates, nurses, practitioners, respiratory therapists, paramedical staff, and physicians who provided care to these neonates.
Glossary
- AA
African American
- ANS
antenatal steroid
- aOR
adjusted odds ratio
- BPD
bronchopulmonary dysplasia
- BW
birth weight
- ELBW
extremely low birth weight
- GA
gestational age
- HRF
hypoxemic respiratory failure
- iNO
inhaled nitric oxide
- IVH
intraventricular hemorrhage
- NDI
neurodevelopmental impairment
- NEC
necrotizing enterocolitis
- NICHD
Eunice Kennedy Shriver National Institute of Child Health and Human Development
- NRN
Neonatal Research Network
- PDA
patent ductus arteriosus
- PH
pulmonary hypertension
- PPHN
persistent pulmonary hypertension of newborn
- PPROM
prolonged preterm rupture of membranes
- PROM
prolonged rupture of membranes
- PVL
periventricular leukomalacia
- RCT
randomized controlled trial
- RDS
respiratory distress syndrome
- ROM
rupture of membranes
- ROP
retinopathy of prematurity
- SGA
small for gestational age
Footnotes
Dr Chandrasekharan conceptualized, wrote the manuscript, helped with interpretation of data reviewed, revised, and submitted the final manuscript; Dr Lakshminrusimha conceptualized and wrote the manuscript and helped with interpretation of data, supervision, and critical review; Ms Chowdhury collected data, analyzed data, and wrote part of the manuscript; Drs Van Meurs, Keszler, Kirpalani, Walsh, and Higgins conceptualized and helped with interpretation of data, supervision, and critical review; Dr Das conceptualized the data analysis and helped with interpretation of data, supervision, and critical review; Dr McGowan reviewed and interpreted the follow-up data and helped with supervision, and critical review; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
This trial was registered at www.ClinicalTrials.gov (identifier NCT00063063).
Data Sharing: Data reported in this article may be requested through a data use agreement. Further details are available at https://neonatal.rti.org/index.cfm?fuseaction=DataRequest.Home.
A complete list of nonauthor contributors appears in Supplemental Materials.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: The National Institutes of Health and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (U10 HD27871, U10 HD53119, UG1 HD21364, UG1 HD21373, UG1 HD21385, UG1 HD27851, UG1 HD27853, UG1 HD27856, UG1 HD27880,UG1 HD27904, UG1 HD34216, UG1 HD36790, UG1 HD40492, UG1 HD40689, UG1 HD53089, UG1 HD53109, UG1 HD68244, UG1 HD68270, UG1 HD68278, UG1 HD68263, UG1 HD68284, UG1 HD87226, UG1 HD87229) and the National Center for Advancing Translational Sciences (NCATS) (UL1 TR6, UL1 TR41, UL1 TR42, UL1 TR77, UL1 TR93, UL1 TR442, UL1 TR454, UL1 TR1117) provided grant support through cooperative agreements for the Neonatal Research Network. Eunice Kennedy Shriver National Institute of Child Health and Human Development staff provided input into the study design, conduct, analysis, and article drafting; National Center for Research Resources and NCATS cooperative agreements provided infrastructure support to the Neonatal Research Network. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- 1.Polin RA, Carlo WA; Committee on Fetus and Newborn; American Academy of Pediatrics . Surfactant replacement therapy for preterm and term neonates with respiratory distress. Pediatrics. 2014;133(1):156–163 [DOI] [PubMed] [Google Scholar]
- 2.Van Meurs KP, Wright LL, Ehrenkranz RA, et al.; Preemie Inhaled Nitric Oxide Study . Inhaled nitric oxide for premature infants with severe respiratory failure. N Engl J Med. 2005;353(1):13–22 [DOI] [PubMed] [Google Scholar]
- 3.Giesinger RE, More K, Odame J, Jain A, Jankov RP, McNamara PJ. Controversies in the identification and management of acute pulmonary hypertension in preterm neonates. Pediatr Res. 2017;82(6):901–914 [DOI] [PubMed] [Google Scholar]
- 4.Cole FS, Alleyne C, Barks JD, et al. . NIH Consensus Development Conference statement: inhaled nitric-oxide therapy for premature infants. Pediatrics. 2011;127(2):363–369 [DOI] [PubMed] [Google Scholar]
- 5.Kumar P; Committee on Fetus and Newborn; American Academy of Pediatrics . Use of inhaled nitric oxide in preterm infants. Pediatrics. 2014;133(1):164–170 [DOI] [PubMed] [Google Scholar]
- 6.Abman SH, Hansmann G, Archer SL, et al.; American Heart Association Council on Cardiopulmonary, Critical Care, Perioperative and Resuscitation; Council on Clinical Cardiology; Council on Cardiovascular Disease in the Young; Council on Cardiovascular Radiology and Intervention; Council on Cardiovascular Surgery and Anesthesia; and the American Thoracic Society . Pediatric pulmonary hypertension: guidelines from the American Heart Association and American Thoracic Society [published correction appears in Circulation 2015;133(4):e368]. Circulation. 2015;132(21):2037–2099 [DOI] [PubMed] [Google Scholar]
- 7.Kinsella JP, Steinhorn RH, Krishnan US, et al. . Recommendations for the use of inhaled nitric oxide therapy in premature newborns with severe pulmonary hypertension. J Pediatr. 2016;170:312–314 [DOI] [PubMed] [Google Scholar]
- 8.Chandrasekharan P, Kozielski R, Kumar VH, et al. . Early use of inhaled nitric oxide in preterm infants: is there a rationale for selective approach? Am J Perinatol. 2017;34(5):428–440 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Handley SC, Steinhorn RH, Hopper AO, et al. . Inhaled nitric oxide use in preterm infants in California neonatal intensive care units. J Perinatol. 2016;36(8):635–639 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kinsella JP, Cutter GR, Walsh WF, et al. . Early inhaled nitric oxide therapy in premature newborns with respiratory failure. N Engl J Med. 2006;355(4):354–364 [DOI] [PubMed] [Google Scholar]
- 11.Ellsworth KR, Ellsworth MA, Weaver AL, Mara KC, Clark RH, Carey WA. Association of early inhaled nitric oxide with the survival of preterm neonates with pulmonary hypoplasia. JAMA Pediatr. 2018;172(7):e180761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Carey WA, Weaver AL, Mara KC, Clark RH. Inhaled nitric oxide in extremely premature neonates with respiratory distress syndrome [published correction appears in Pediatrics 2018;142(4):e20182150]. Pediatrics. 2018;141(3):e20173108. [DOI] [PubMed] [Google Scholar]
- 13.Askie LM, Davies LC, Schreiber MD, Hibbs AM, Ballard PL, Ballard RA. Race effects of inhaled nitric oxide in preterm infants: an individual participant data meta-analysis. J Pediatr. 2018;193:34–39.e2 [DOI] [PubMed] [Google Scholar]
- 14.Kalinowski L, Dobrucki IT, Malinski T. Race-specific differences in endothelial function: predisposition of African Americans to vascular diseases. Circulation. 2004;109(21):2511–2517 [DOI] [PubMed] [Google Scholar]
- 15.Saul S. FDA approves a heart drug for African-Americans. The New York Times. June 24, 2005. Available at: https://www.nytimes.com/2005/06/24/health/fda-approves-a-heart-drug-for-africanamericans.html#:∼:text=The%20Food%20and%20Drug%20Administration,pump%20by%20relaxing%20blood%20vessels. Accessed June 4, 2020 [PubMed]
- 16.Ballard RA, Truog WE, Cnaan A, et al.; NO CLD Study Group . Inhaled nitric oxide in preterm infants undergoing mechanical ventilation. N Engl J Med. 2006;355(4):343–353 [DOI] [PubMed] [Google Scholar]
- 17.Baczynski M, Ginty S, Weisz D, et al. . Outcomes of hypoxic respiratory failure at birth associated with previable rupture of membranes. J Perinatol. 2018;38(8):1087–1092 [DOI] [PubMed] [Google Scholar]
- 18.Kumar VH, Hutchison AA, Lakshminrusimha S, Morin FC III, Wynn RJ, Ryan RM. Characteristics of pulmonary hypertension in preterm neonates. J Perinatol. 2007;27(4):214–219 [DOI] [PubMed] [Google Scholar]
- 19.Jobe AH, Bancalari E. Bronchopulmonary dysplasia. Am J Respir Crit Care Med. 2001;163(7):1723–1729 [DOI] [PubMed] [Google Scholar]
- 20.Younge N, Goldstein RF, Bann CM, et al.; Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network . Survival and neurodevelopmental outcomes among periviable infants. N Engl J Med. 2017;376(7):617–628 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Palisano R, Rosenbaum P, Walter S, Russell D, Wood E, Galuppi B. Development and reliability of a system to classify gross motor function in children with cerebral palsy. Dev Med Child Neurol. 1997;39(4):214–223 [DOI] [PubMed] [Google Scholar]
- 22.Palisano RJ, Hanna SE, Rosenbaum PL, et al. . Validation of a model of gross motor function for children with cerebral palsy. Phys Ther. 2000;80(10):974–985 [PubMed] [Google Scholar]
- 23.Braitman LE, Rosenbaum PR. Rare outcomes, common treatments: analytic strategies using propensity scores. Ann Intern Med. 2002;137(8):693–695 [DOI] [PubMed] [Google Scholar]
- 24.Truog WE, Nelin LD, Das A, et al.; NICHD Neonatal Research Network . Inhaled nitric oxide usage in preterm infants in the NICHD Neonatal Research Network: inter-site variation and propensity evaluation. J Perinatol. 2014;34(11):842–846 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Anadkat JS, Kuzniewicz MW, Chaudhari BP, Cole FS, Hamvas A. Increased risk for respiratory distress among white, male, late preterm and term infants. J Perinatol. 2012;32(10):780–785 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Richardson CJ, Pomerance JJ, Cunningham MD, Gluck L. Acceleration of fetal lung maturation following prolonged rupture of the membranes. Am J Obstet Gynecol. 1974;118(8):1115–1118 [DOI] [PubMed] [Google Scholar]
- 27.Deruelle P, Houfflin-Debarge V, Magnenant E, et al. . Effects of antenatal glucocorticoids on pulmonary vascular reactivity in the ovine fetus. Am J Obstet Gynecol. 2003;189(1):208–215 [DOI] [PubMed] [Google Scholar]
- 28.Gao Y, Zhou H, Raj JU. Antenatal betamethasone therapy potentiates nitric oxide-mediated relaxation of preterm ovine coronary arteries. Am J Physiol. 1996;270(2 Pt 2):H538–H544 [DOI] [PubMed] [Google Scholar]
- 29.Roberts D, Brown J, Medley N, Dalziel SR. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database Syst Rev. 2017;3:CD004454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Chock VY, Van Meurs KP, Hintz SR, et al.; NICHD Neonatal Research Network . Inhaled nitric oxide for preterm premature rupture of membranes, oligohydramnios, and pulmonary hypoplasia. Am J Perinatol. 2009;26(4):317–322 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hintz SR, Van Meurs KP, Perritt R, Poole WK, Das A, Stevenson DK, et al. . Neurodevelopmental outcomes of premature infants with severe respiratory failure enrolled in a randomized controlled trial of inhaled nitric oxide. J Pediatr. 2007;151(1):16–22, 22–3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Jensen EA, Kirpalani H. Inhaled nitric oxide to prevent bronchopulmonary dysplasia in preterm infants–less than a silver bullet. J Pediatr. 2014;165(6):1079–1081 [DOI] [PubMed] [Google Scholar]
- 33.Dixon F, Ziegler DS, Bajuk B, et al. . Treatment with nitric oxide in the neonatal intensive care unit is associated with increased risk of childhood cancer. Acta Paediatr. 2018;107(12):2092–2098 [DOI] [PubMed] [Google Scholar]
- 34.Vali P, Vento M, Underwood M, Lakshminrusimha S. Free Radical Damage Can Cause Serious Long-Lasting Effects In: Acta Paediatr, vol. 107 2018:2099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Carey WA, Ellsworth MA, Harris MN. Inhaled nitric oxide use in the neonatal intensive care unit: rising costs and the need for a new research paradigm. JAMA Pediatr. 2016;170(7):639–640 [DOI] [PubMed] [Google Scholar]
- 36.Ellsworth MA, Harris MN, Carey WA, Spitzer AR, Clark RH. Off-label use of inhaled nitric oxide after release of NIH consensus statement. Pediatrics. 2015;135(4):643–648 [DOI] [PubMed] [Google Scholar]
- 37.Soll RF. Inhaled nitric oxide for preterm infants: what can change our practice? Pediatrics. 2018;141(3):e20174214. [DOI] [PubMed] [Google Scholar]
- 38.Finer NN, Evans N. Inhaled nitric oxide for the preterm infant: evidence versus practice. Pediatrics. 2015;135(4):754–756 [DOI] [PubMed] [Google Scholar]
- 39.Ahmed MS, Giesinger RE, Ibrahim M, et al. . Clinical and echocardiography predictors of response to inhaled nitric oxide in hypoxic preterm neonates. J Paediatr Child Health. 2019;55(7):753–761 [DOI] [PubMed] [Google Scholar]
- 40.Golfar A, Bhogal J, Kamstra B, et al. . Outcome of preterm neonates with a birth weight <1,500 g with severe hypoxemic respiratory failure rescued by inhaled nitric oxide therapy and high-frequency oscillatory ventilation. Neonatology. 2017;112(3):274–280 [DOI] [PubMed] [Google Scholar]
- 41.Suzuki S, Togari H, Potenziano JL, Schreiber MD. Efficacy of inhaled nitric oxide in neonates with hypoxic respiratory failure and pulmonary hypertension: the Japanese experience. J Perinat Med. 2018;46(6):657–663 [DOI] [PubMed] [Google Scholar]
- 42.Manja V, Guyatt G, Lakshminrusimha S, Jack S, Kirpalani H, Zupancic JAF, et al. . Factors influencing decision making in neonatology: inhaled nitric oxide in preterm infants. J Perinatology. 2019;39(1):86–94 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kochanek KDMS, Xu JQ, Arias E. Deaths: Final Data for 2017. National Vital Statistics Reports, vol. Vol 68 Hyattsville, MD: National Center for Health Statistics; 2019:9. [PubMed] [Google Scholar]
- 44.Goyal MK, Kuppermann N, Cleary SD, Teach SJ, Chamberlain JM. Racial disparities in pain management of children with appendicitis in emergency departments. JAMA Pediatr. 2015;169(11):996–1002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Hwang SS, Burris HH, Collins JW Jr., Kirpalani H, Wright CJ. Moving beyond race and ethnicity to understand the effect of inhaled nitric oxide on bronchopulmonary dysplasia prevention. J Pediatr. 2018;201:298–300 [DOI] [PubMed] [Google Scholar]