A Predictive Model for Neurodevelopmental Outcome Following the Norwood Procedure

Abstract

Neurodevelopmental outcomes following the Norwood procedure for single right ventricular lesions are worse than those in the normal population. It would be valuable to identify which patients at the time of Norwood discharge are at greatest risk of neurodevelopmental impairment later in childhood. As such, we sought to construct and validate a model to predict poor neurodevelopmental outcome using variables readily available to the clinician. Using data from 14-month neurodevelopmental outcome of the Single Ventricle Reconstruction (SVR) trial, a Classification and Regression Tree (CART) analysis model was developed to predict severe neurodevelopmental impairment, defined as a Psychomotor Development Index (PDI) of <70 on the Bayley Scales of Infant Development®-II. The model was then validated using data from subjects enrolled in the Infant Single Ventricle (ISV) trial.

PDI scores were <70 in 138 of 313 subjects (44%). Predictors of PDI<70 were: post-Norwood length of ICU stay >46 days, genetic syndrome or other anomalies, birth weight <2.7 kg, additional cardiac surgical procedures, and use of ≥5 medications at hospital discharge. Using these risk factors, the CART model correctly identified 75% of SVR subjects with PDI<70. When the CART model was applied to 70 subjects from the ISV trial, the correct classification rate was 67%.

This model of variables from the Norwood hospitalization can help identify infants at risk for neurodevelopmental impairment. However, given the overall high prevalence of neurodevelopmental impairment and the fact that nearly one-third of severely affected children would not have been identified by these risk factors, close surveillance and assessment for early intervention services is warranted for all infants following the Norwood procedure.

Neurodevelopmental impairment constitutes a major morbidity among children and adolescents with complex congenital heart disease; among these, patients with hypoplastic left heart syndrome (HLHS) and other forms of single right ventricle are at greatest risk[1, 2]. Previous studies have identified patient- and procedure-related factors that are associated with neurodevelopmental deficits in children with HLHS who have undergone the Norwood procedure[3–5]. The ability to predict which children will have the most severe neurodevelopmental impairment could help to leverage early intervention strategies in face of constraints of distance and financial resources, and also would be valuable when counseling parents. Whereas traditional multivariable linear regression techniques have been useful in identifying risk factors, the mathematical formulas resulting from these analyses are not intuitive for busy clinicians to use. Ideally, one would like to construct a user-friendly predictive instrument to accurately classify children into risk categories using variables readily available to the clinician.

Classification and Regression Tree (CART) is an analytic method that can identify high- and low-risk patient subgroups through a series of binary splits[6, 7]. As such, CART can simplify risk stratification. To develop a clinically useful risk stratification model of mid-term neurodevelopment in patients after the Norwood procedure, the present analysis applies the CART analytic technique to a 14 month neurodevelopmental assessment of subjects enrolled in the Single Ventricle Reconstruction (SVR) trial carried out by the Pediatric Heart Network. The model was then validated using data from subjects enrolled in the Infant Single Ventricle (ISV) trial also performed by Pediatric Heart Network investigators.

Methods

Subjects

The details of the SVR trial design have been published [8, 9]. Subjects were eligible for inclusion if they had a diagnosis of HLHS or related single, morphologic right ventricular anomaly and planned Norwood procedure. Exclusion criteria included any major congenital or acquired extra-cardiac abnormality that could independently influence transplant-free survival at 12 months following randomization. The study protocol was approved by the Institutional Review Board or Research Ethics Board at each participating center, and written informed consent was obtained from a parent/guardian prior to randomization.

A validation of the model (described below) was undertaken using data from subjects that had single ventricle anomalies and underwent the Norwood procedure as part of the Pediatric Heart Network Infant Single Ventricle (ISV) randomized trial of enalapril versus placebo examining a primary endpoint of somatic growth[10]. This validation sample consisted only of subjects enrolled in the ISV trial, but not enrolled in the SVR trial. The details of this trial have been published [11]. In brief, inclusion criteria consisted of age ≤45 days at enrollment, age >1 week if born at 35 weeks’ gestation, presence of single-ventricle physiology, and planned superior cavopulmonary connection. Exclusion criteria included aortic oxygen saturation <65%, creatinine >1.0 mg/dL, absolute neutrophil count <1000 cells/mL, and chromosomal or recognizable phenotypic syndrome of non-cardiac congenital abnormalities associated with growth failure.

Neurodevelopmental Assessment

The Bayley Scales of Infant Development-II Revised (BSID-II) allows a standardized assessment of cognitive and motor development for children ages 1 month through 42 months [12], yielding summary scores for two domains: the Psychomotor Development Index (PDI) and the Mental Development Index (MDI). The mean±SD for PDI and MDI scores in the normative population is 100±15. BSID-II is an older instrument than the more recently released Bayley Scales of Infant and Toddler Development, Third Edition (BSID-III). The BSID-II may be more sensitive than the BSID-III for identifying impairment in several high-risk groups including the congenital heart disease population[13, 14].

All examiners completed protocol training, and submitted a video of administration of the instrument that was then reviewed by a senior neuropsychologist (D.C.B.). The BSID-II was administered in either English or Spanish depending upon the dominant language spoken in the home. The results of this 14 month assessment have previously been presented [15]. The completion rate for the BSID-II examination among transplant-free survivors was 313 of 364 (86%). The mean age (±SD) was 14.3±1.1 months (range, 12.2–19.5 months).

Study Design and Measurements

In the SVR trial subjects were randomly assigned to either a modified Blalock-Taussig shunt (MBTS) or a right ventricle-to-pulmonary artery shunt (RVPAS). In addition to genetics evaluations performed as part of routine clinical care, an optional research genetics evaluation was offered. Subjects were classified with regard to whether they had 1) a specific genetic syndrome, or 2) other anomalies (i.e., not identified with a syndrome). The distribution of genetic syndromes in the SVR trial has been reported previously [16].

For the present CART analysis, the primary outcome is the binary indicator of PDI score of <70. While the BSID-II yields both a PDI and MDI scale, the psychomotor domain has generally been found to be more impaired in young children with congenital heart disease [17, 18]. As such, one would expect nearly all subjects with MDI scores of <70 to also have PDI scores of <70. The cutpoint of <70, two standard deviations below the mean for the standard population, was chosen since this is generally recognized to represent severe impairment and would qualify a child for early intervention services in nearly all states [19, 20].

Statistical Methods

Continuous variables are presented as mean±SD and categorical variables are presented as counts and percentages. CART is a nonparametric technique that can identify from among a large number of variables and their interactions which are the most predictive of outcome. The CART method was used to generate a decision tree by a series of binary splits (recursively partitioning) of the data to predict the dichotomous variable PDI < or ≥70. The CART model selects variables that provide the highest discriminatory power and the variables which contribute the most to the outcome are listed first at the top of the tree. Candidate predictors were demographic and clinical data available through the time of the Norwood procedure hospital discharge (Table 1). Since variability in administration might impact BSID-II scores, we also examined a second decision tree that required site to be included as a candidate predictor to adjust for potential differences in testing among centers. Receiver operating curves were constructed and the area under the curve for the CART analysis is reported, with a higher value indicating better prediction.

Table 1.

Candidate Predictor Variables for CART Model

Norwood Procedure Hospitalization
Patient Characteristic	Preoperative	Operative	Postoperative to Discharge
Gender	Prenatal diagnosis	Assigned treatment	ECMO
Ethnicity	Fetal intervention	Shunt at end of operation	CPR
HLHS	Age at diagnosis	Cross clamp time	Open sternum
Aortic atresia	Birth weight	Total support time, min	ICU days
Genetic or other anomaly	Gestational age	Perfusion type	# surgeries
	APGAR scores	DHCA time	# complications
	Highest lactate	RCP time	# medications at discharge
	Mechanical ventilation	Hematocrit	Oxygen saturation
	Catheter interventions	Lowest temperature during CPB
		Ultrafiltration
		Aprotinin
		Steroids
		α-blockade
		ECMO*

CPB-cardiopulmonary bypass, CPR- cardiopulmonary resuscitation, DHCA- deep hypothermic circulatory arrest, ECMO-extracorporeal membrane oxygenation, HLHS-hypoplastic left heart syndrome, ICU-intensive care unit, RCP-regional cerebral perfusion

^*initiated in the operating room

The prediction rule arising from the CART analysis was validated using independent data from 70 subjects enrolled in the ISV trial. As with the SVR trial, ISV trial subjects underwent neurodevelopmental assessment at 14 months with the BSID-II.

Analyses were performed using SAS version 9.2 (SAS Institute, Inc., Cary, NC) and an open-source adaptation of the CART algorithm (RPART library in R version 2.13.0).

Results

Development of the Model

The mean (±SD) age at completion of the BSID-II was 14.3±1.1 months. The cohort was 62% male, had a weight z-score of −0.7±1.1 at BSID-II completion, was 20% Hispanic, and 90% had HLHS. PDI scores were profoundly lower than the normative population: scores were <70 (>2 SD below expected mean) in 138 of 313 subjects (44%), significantly greater than the expected frequency of 2%. The variables that remained in the discriminative CART model included intensive care unit (ICU) stay of >46 days following the Norwood procedure, genetic or other anomalies, birth weight of <2.7 kg, additional cardiac surgeries following the Norwood procedure during the hospitalization for the Norwood, and use of five or more medications at the time of Norwood hospitalization discharge (Figure 1). Notably all of the subjects with an intensive care unit (ICU) stay of more than 46 days had PDI scores of <70. None of the procedure-related variables, such as shunt type, use of deep hypothermic circulatory arrest, or anatomic variables such as aortic atresia remained in the model.

Figure 1.

Classification analysis for PDI <70 at 14 months of age. Shaded boxes indicate terminal nodes. A terminal node occurs when it is not possible to find any variable that separates this subsample into any more homogeneous subgroups. CART terminates disaggregation at this level.

The CART tree correctly identified 75% of subjects (Tables 2a & 2b). Ninety-four of the 138 subjects (68% sensitivity) with PDI <70 were correctly classified and 140 of the 175 subjects (80% specificity) with PDI≥70 were correctly classified. Receiver operating characteristic curve analysis shows the area under curve (AUC) of the CART calculation is 0.78. Because of the possibility that testing site might influence BSID-II scores, we explored whether incorporation of site might alter the model. The forced inclusion of site yielded a nearly identical AUC of 0.76. Since this CART model was meant to be applied to general clinical care and site did not significantly improve the AUC, we did not include site in the final model.

Table 2a.

Observed vs. CART-predicted PDI Score in SVR Cohort

Actual Observation
Classification tree (Predicted)	PDI<70	PDI≥70	Total
PDI<70	94	35	129
PDI≥70	44	140	184
Total	138	175	313

Table 2b.

Performance Measures of CART Model

Sensitivity	68%
Specificity	80%
Positive Predictive Value	73%
Negative Predictive Value	76%
Accuracy	75%

Independent Validation Sample

The validation of the final CART model used data from 70 ISV trial subjects with single ventricle anomalies who had undergone the Norwood procedure (Table 3). Of the 70 subjects, 25 (36%) had PDI scores of <70, which is not significantly different from the SVR trial rate of PDI<70 of 44% (P=.23). However, one predictive variable, the presence of a genetic syndrome or anomaly, was identified less frequently in the ISV cohort. Only 3% of the ISV subjects were confirmed as having a genetic syndrome or non-syndromic anomaly, as compared to 25% from the SVR trial (P<.001).

Table 3.

ISV Trial Subjects by CART Node Grouping

Actual Observation
Classification tree (Predicted)	PDI<70	PDI≥70	Total
PDI<70	8	6	14
PDI ≥70	17	39	56
Total	25	45	70

The validation of the model using data from the ISV trial correctly predicted the classification in 67% of subjects. Eight of the 25 subjects (32%) with PDI <70 were correctly classified and 39 of the 45 subjects (87%) with PDI≥70 were correctly classified. Therefore, the model performed better with regard to identifying infants who did not have severe impairment.

Discussion

Infants who undergo the Norwood procedure are at high risk of neurodevelopmental impairment as measured by the BSID-II administered just after 12 months of age. Indeed, in the SVR trial, almost half of the subjects undergoing neurodevelopmental follow-up at age 14 months had PDI scores below 70, consistent with significant developmental impairment. Many risk factors for adverse outcome, such as the presence of genetic syndromes and other anomalies, low birth weight, and perioperative morbidity, are manifest during the Norwood hospitalization. We thus hypothesized that CART analysis using clinical data available at the time of hospital discharge from the Norwood procedure would allow us to classify patients into high and low risk for severe neurodevelopmental impairment. Using the comprehensive database of the SVR trial, we found that the best model for predicting PDI score < 70 correctly classified 75% of the subjects evaluated at age 14 months but performed less well in our validation cohort from the ISV trial, correctly classifying 67% of patients.

The limited strength of the model may have several causes. Critical factors that predispose an infant with single right ventricle to later developmental impairment may be hard to quantify or may not be routinely measured. For example, it is known that a proportion of these children may manifest pre-operative brain injury on MRI even in the absence of acute clinical insults [21, 22]. In addition, neurodevelopmental status at 14 months is influenced by many factors that occur beyond the initial Norwood hospitalization. Nearly all infants with HLHS undergo an additional cardiac surgery, the stage II procedure, in the first year life and may require additional unplanned interventions. As such, an infant with a relatively smooth Norwood hospitalization and no other overt risk factors in the neonatal period may have later exposures that increase the risk of neurodevelopmental impairment. One particular limitation to the model in the ISV validation sample relates the presence of genetic syndromes or other anomalies. Genetic abnormalities may affect both cardiac and brain development [23, 24]. The ISV study protocol excluded subjects with a chromosomal or recognizable phenotypic syndrome of noncardiac congenital abnormalities associated with growth failure. The more stringent exclusion criteria for some genetic syndromes in the ISV trial may in part account for the worse performance of the CART model in the validation sample. The relative importance of genetic syndromes and other anomalies for neurodevelopmental outcome does underscore the need for a comprehensive genetic assessment in this population.

In spite of the aforementioned limitations, the CART analysis does provide some meaningful insights into factors that may influence neurodevelopmental outcome. It is known that an increased length of hospital stay has an association with subsequent developmental deficits [25, 26]. All subjects with an extended ICU stay (>46 days) after the Norwood procedure had PDI scores of <70. Birth weight of <2.7 kg and the need for additional cardiac surgery during the Norwood hospitalization also were associated with later impairment. The number of medications prescribed to an infant at hospital discharge has not previously been explored as a predictor of later developmental status. However, this measure emerged as a risk factor that is likely a surrogate marker for overall health status; its identification demonstrates the value of the agnostic approach of CART analysis. Interestingly, intraoperative variables such as duration of deep hypothermic circulatory arrest or anatomic features such as aortic atresia did not contribute to risk stratification in the CART model. This finding is consistent with previous reports documenting that intraoperative variables explain only a small portion of variability in developmental outcomes [5, 18].

In summary, comprehensive measurement of risk factors at the time of Norwood discharge in infants with HLHS and other single RV anomalies correctly classifies 75% of infants at risk for severe neurodevelopmental impairment. However, the best model would misclassify a considerable number of subjects as lower risk, who at 14 months had PDI scores of <70. If one were to limit referral for early intervention based on an early prediction model alone, such as the CART tree, these high risk children might not receive appropriate services. Because of the high prevalence of neurodevelopmental impairment, close surveillance and early intervention services are warranted for all young children following the Norwood procedure.