Technical Performance Score is Associated with Outcomes after the Norwood Procedure

Abstract

Objectives

Technical Performance Score (TPS) has been reported in a single center study to predict outcomes after congenital cardiac surgery. We sought to determine the association of TPS with outcomes in patients undergoing Norwood procedure in the Single Ventricle Reconstruction Trial.

Methods

We calculated TPS (Class 1-Optimal; Class 2-Adequate; Class 3-Inadequate) based on predischarge echocardiograms analyzed in a Core Laboratory and unplanned reinterventions that occurred prior to discharge from the Norwood hospitalization. Multivariable regression examined the association of TPS with time to first extubation, Norwood length of stay (LOS), death/transplantation, unplanned post-discharge reinterventions, and neurodevelopment at 14 months of age.

Results

Among 549 patients undergoing a Norwood procedure, 356 (65%) had an echocardiogram adequate to assess atrial septal restriction/arch obstruction or an unplanned reintervention, enabling calculation of TPS. In multivariable regression adjusting for preoperative variables, a better TPS was an independent predictor of shorter time to first extubation (p=0.019), better transplant-free survival before Norwood discharge (p<0.001; odds ratio 9.1 for Inadequate vs. Optimal score), shorter hospital LOS (p<0.001), fewer unplanned reinterventions between Norwood discharge and Stage II (p=0.004), and higher Bayley II Psychomotor Development Index at 14 months (p=0.031). TPS was not associated with transplant-free survival after Norwood discharge, unplanned reinterventions after Stage II or Bayley II Mental Development Index at 14 months.

Conclusion

TPS is an independent predictor of important outcomes after Norwood and may serve as a tool for quality improvement.

Introduction

Technical Performance Score (TPS) is a quality improvement tool that was developed to determine the technical adequacy of an intended surgical procedure and has been validated for a subset of common congenital cardiac operations at a single center.^1–8 For neonates undergoing the Norwood procedure at a single institution, an optimal TPS mitigated the effects of preoperative physiological status and severity of illness during the initial hospital stay.^{2, 3, 9} However, TPS requires validation in a multicenter study.

We sought to determine the validity of TPS across multiple centers using the database of the Pediatric Heart Network’s Single Ventricle Reconstruction (SVR) trial.^10–15 Specifically, we explored whether TPS predicted early and late outcomes, including resource utilization. We hypothesized that TPS could also identify patients at higher risk for reintervention in the inter-stage phase.

Materials and Methods

We performed a secondary analysis of data in the SVR trial, for which inclusion and exclusion criteria, study design, and data collection have been described.^10–15

Technical Performance Scoring System

All subjects were assigned a TPS based on the following data obtained prior to Norwood hospital discharge: 1) post-operative, protocol-driven transthoracic echocardiographic findings interpreted by a Core Laboratory, and 2) unanticipated surgical or catheter-based reinterventions in areas of Norwood repair prior to discharge from the hospitalization associated with the Norwood procedure. The TPS module used in our analyses, as modified from the original reported by Bacha et al.², is summarized in Supplemental Table 1.

In brief, surgical procedures were divided into components that were assigned a score of Class 1 (optimal), Class 2 (adequate), or Class 3 (inadequate) based on specific echocardiographic criteria and unplanned reinterventions at surgical repair sites that occurred prior to discharge from the Norwood hospitalization (Supplemental Table 1). The overall classification of the operation as Class 1, 2 or 3 was based upon the highest Class assignment in any of the component sub-procedures. All components of the Norwood TPS module in its current version are given equal weight.

Two additional classes were created to allow inclusion of all subjects in the SVR trial. Class 4 included subjects who had no Core Lab echocardiograms and no unanticipated reinterventions in the area of Norwood repair prior to discharge or death. Class 5 included subjects with echocardiograms inadequate for TPS assignment and no unanticipated Norwood surgical reinterventions prior to discharge or death.

Outcomes

Our primary outcome was time to initial endotracheal extubation because it is a well known surrogate for resource use. We did not choose mortality as our primary endpoint because a vast majority of Class 4 (no echocardiogram) patients died and thus could not be assigned a TPS. Secondary outcomes included early mortality or transplantation, defined as occurring prior to Norwood discharge or within 30 days from the Norwood procedure, whichever was longer; Norwood hospitalization length of stay (LOS); late mortality or transplantation; unplanned reinterventions following Norwood discharge; and Psychomotor Development Index (PDI) and Mental Development Index (MDI) scores of the Bayley Scales of Infant Development® – Second Edition.

Classification of unplanned reinterventions after Norwood discharge

The following post-discharge reinterventions were considered to be attributable to the Norwood procedure technique: a) any reintervention on the aortic arch, atrial septum, or ascending aorta/proximal pulmonary artery (PA) connection, b) extensive PA rehabilitation, excluding simple PA augmentation at time of Stage II or Fontan procedure, and c) any reintervention on the modified Blalock-Taussig Shunt (MBTS) or right ventricle-to-pulmonary artery shunt (RVPAS) between Norwood discharge and Stage II procedure.

The following post-discharge reinterventions were considered not to be attributable to the Norwood procedure technique: a) coiling of aortopulmonary or venovenous collaterals, b) uncomplicated Stage II procedure or Fontan procedure, and c) any intervention on the superior vena cava after the Stage II procedure or Fontan connections after Fontan procedure.

Statistical analysis

The distributions of patient and procedural characteristics by TPS Class were compared using a chi-square or Fisher exact test for categorical variables and a Kruskal-Wallis test or a one-way analysis of variance for continuous variables.

Cox regression modeling identified factors associated with time to initial extubation. Factors associated in univariate analysis at the 0.20 level were candidate predictors in stepwise multivariable modeling. Our multivariable analyses included only patient factors and preoperative medical variables, because intraoperative and postoperative outcomes are in the causal pathway of the measures used to calculate the TPS. TPS was then added to the multivariable model to assess whether it was an independent predictor. Kaplan-Meier estimation with logrank test described the association of time to initial extubation and TPS. The same approach was applied for secondary outcome variables, using linear regression for continuous variables and logistic regression for dichotomous outcomes. We assessed the reliability of TPS as an independent predictor of each outcome by creating 1000 samples with bootstrapping to determine the percentage of samples in which TPS was significant at the 0.05 level conditional on the demographic and pre-surgical variables already in the multivariable model. A reliability of at least 50% was set as the criterion for retaining the TPS term as an independent predictor.

We assessed whether the three components of the TPS (distal arch gradient, atrial septal defect (ASD) gradient, unplanned re-interventions prior to Norwood discharge) were associated with Norwood hospitalization LOS, early mortality/transplant, and pre-Stage II procedure reintervention. For the echocardiographic components, the medium and high categories for distal arch gradient and for mean ASD gradient were combined prior to modeling due to sparse data.

Sensitivity analyses were also conducted, utilizing data of subjects with missing and incomplete echocardiograms (Class 4 and 5, respectively). TPS was assumed to be optimal for all, and then inadequate for all, to set bounds on the relationships between time to initial extubation and technical performance.

A p-value of 0.05 was considered to be statistically significant. All analyses were performed using SAS version 9.3 (Statistical Analysis System®, SAS Institute, Cary, NC).

Results

The SVR trial randomized 555 subjects, of whom five did not undergo a Norwood procedure and one withdrew in the first week. Among the remaining 549 subjects, 485 (88%) had postoperative echocardiograms adequate for calculating TPS, and 452 (82%) survived transplant-free to Norwood discharge. Protocol driven echocardiograms were performed at a median of 14 days (Interquartile range (IQR), 8 – 21 days). A TPS could be assigned to 356 subjects (65%) based on echocardiograms or unanticipated reinterventions in areas of Norwood repair; an additional 56 subjects (10%) were Class 4, and 137 (25%) were Class 5. Baseline characteristics are shown in Supplemental Table 2. Subjects in Class 4 (no echocardiogram) had the lowest birth weight and the highest prevalence of preterm birth and percentage of unknown genetic status. There was no difference in patient factors between Class 5 and subjects with an assigned TPS (Supplemental Table 3). Among subjects with a TPS, the distribution was: Class 1, 72%; Class 2, 12%; Class 3, 17%.

Time to first extubation

Among the 452 (82%) subjects discharged post-Norwood alive without transplantation, subjects with TPS Class 3 underwent extubation later than those in Classes 1 or 2 (Figure 1; logrank test, p<.001), with median (IQR) days to extubation of 9 (5–15) vs. 6 (4–8) vs. 6 (3–7), respectively. After adjusting for center, the variables independently associated with longer time to first extubation included lower gestational age, Hispanic ethnicity, and presence of a genetic syndrome/anomalies (model R²=0.21). When added to this model, TPS was an independent predictor (model R² =0.23, p=0.019, reliability 76%). Class 3 subjects had a longer time to first extubation (Table 1).

Figure 1.

Time to first extubation, early mortality and Norwood hospital length of stay by TPS classes. Class 3 subjects had a significantly longer time to first extubation, higher early mortality and longer Norwood hospital length of stay. TPS: Technical Performance Score

Table 1.

Multivariable Cox regression model for time to first extubation^* (n=324, R²=0.23).

		Hazard ratio	95% CI	p
Center				<.001
Gestational age, (months)		0.88	0.82, 0.95	<.001
Hispanic				0.03
Yes		1.40	1.04, 1.90
No		Ref
Genetic syndrome/other anomalies				0.03
Yes		1.51	1.12, 2.06
No		Ref
Unknown		1.19	0.84, 1.66
TPS				0.02
Class 1-Optimal		0.59	0.41, 0.85
Class 2-Adequate		0.62	0.38, 1.00
Class 3- Inadequate		Ref

^*Excluding data of subjects who died or underwent cardiac transplant during the hospitalization.

CI: confidence interval, TPS: technical performance score

Early mortality/transplant

Three pre-Norwood characteristics were independently associated with early mortality/transplant: obstructed pulmonary venous return (OPVR), cardiac or other surgeries prior to the Norwood operation, and genetic syndrome/anomalies. When added to this model, TPS was an independent risk factor (p<0.001, reliability 98%). Class 3 subjects had higher odds of early mortality/transplant, whereas Classes 1 and 2 had similar risks of early mortality/transplant (Supplemental Table 4; Figure 1). The R² increased from 0.14 to 0.26; (Table 2).

Table 2.

Multivariable logistic regression model for early death/transplant (n=356, max-rescaled R²=0.26)

		Odds ratio	95% CI	p
Pre-Norwood intubation		2.38	1.01, 5.61	0.05
Genetic syndrome/other anomalies				0.002
Yes		0.83	0.23, 3.01
No		Ref
Unknown		4.43	1.72, 11.4
TPS				<0.001
Class 1 Optimal		0.11	0.04, 0.26
Class 2 Adequate		0.19	0.05, 0.75
Class 3 Inadequate		Ref

CI: confidence interval, TPS: technical performance score

Norwood hospital length of stay in transplant-free survivors

The median LOS (n=329) was significantly longer in Class 3 than in Classes 1 and 2 (median 50 vs. 23 and 20 days, respectively) (Figure 1). TPS was an independent predictor of Norwood LOS (p<0.001, reliability 100%), when added to the multivariable model that included center and genetic syndrome/anomalies. The R² increased from 0.15 to 0.20, (Table 3).

Table 3.

Multivariable Cox Regression Model for Norwood Length of stay^* (n=329, R²=0.20)

		Hazard Ratio (95% CI)	p value
Center			0.01
Genetic syndrome/other anomalies			0.001
Yes		1.63 (1.21, 2.19)
No		Ref
Unknown		1.65 (1.18, 2.30)
TPS			<.001
Class 1 Optimal		0.44 (0.31, 0.63)
Class 2 Adequate		0.35 (0.22, 0.56)
Class 3 Inadequate		Ref
Pre-Norwood complication			0.03
Yes		1.33 (1.02, 1.72)
No		Ref

^*Excluding data of subjects who died or underwent cardiac transplant during the hospitalization.

CI: confidence intervals, TPS: technical performance score

Late reinterventions attributable to the Norwood

Data were analyzed in two periods: a) after Norwood discharge but before Stage II, b) post-Stage II procedure. Among the 329 subjects with a TPS Class, including 37 (11%) subjects who died between Norwood discharge and Stage II surgery, subjects in Class 1 were the least likely to have had a reintervention between Norwood discharge and Stage II (logistic regression p=0.003, reliability 78%, Table 4, Supplemental Table 5). TPS was not associated with late reinterventions from the Stage II operation to age 12 months.

Table 4.

Multivariable logistic regression model for pre-Stage II reintervention (n=318, R²=0.12)

		Odds ratio	95% CI	p
Prenatal diagnosis of CHD				0.01
Yes		2.77	1.24, 6.18
No		Ref
SES score		1.06	1.01, 1.12	0.03
TPS				0.01
Class 1 Optimal		Ref
Class 2 Adequate*		2.93	1.41, 6.11
Class 3 Inadequate		2.02	0.94, 4.32

^*Class 2 vs. 3 do not differ (p=0.681).

CHD: congenital heart disease, SES: socio economic status, TPS: technical performance score

Mortality/transplant post-Norwood discharge

Among the 448 transplant-free survivors to Norwood discharge, 96 had late mortality (n=85) or transplant (n=11) by 3 years post-randomization. Independent predictors of a higher hazard of death/transplant between Norwood discharge and 3 years post-randomization included preterm delivery, presence of OPVR, and highest pre-Norwood lactate level. When added to this model, TPS was not an independent predictor (Supplemental Tables 6, 7).

Neurodevelopmental outcomes

Distributions of PDI and MDI scores are presented in Supplemental Figure 2. Independent preoperative predictors of lower PDI score were center, lower birth weight, and genetic syndrome/anomalies. When TPS was added to the model, worse TPS Class was associated with lower PDI score (p=0.031, reliability 66%), with the model R² increasing from 0.19 to 0.22. Lower MDI score was independently associated with center, lower birth weight, lower Apgar score at minute 1, intubation for respiratory failure or metabolic acidosis and genetic syndrome/anomalies. When added to this model, TPS Class was not associated with MDI score.

TPS components

We analyzed the three components of TPS available in the SVR trial database - distal arch gradient, ASD gradient, and unplanned pre-discharge reinterventions in areas of Norwood repair - to test their associations with selected outcomes. Unplanned predischarge reinterventions in areas of Norwood repair could occur in any component of the Norwood TPS. Supplemental Table 8 depicts the reinterventions that resulted in Class 3 TPS assignment, while Supplemental Table 9 summarizes the echocardiographic findings in the 59 subjects assigned to Class 3. In multivariable analysis, pre-Norwood discharge reintervention was significantly associated with a higher early mortality/transplant rate (OR 8.36, CI 3.54–19.7, p<0.001) and longer median (IQR) Norwood LOS [51 (33–86) vs. 23 (16–36) days, p<0.01]. Distal arch gradient ≥20 mmHg was significantly associated with a higher odds of pre-Stage II reintervention (OR 3.28, CI 1.59–6.76, p<0.001). ASD gradient was not associated with these three outcomes.

Sensitivity Analyses

To assess the effect of exclusion of Class 5 (incomplete echocardiogram) subjects on time to first extubation, we included 137 Class 5 subjects in Cox regression analyses under two extreme assumptions: that all were Class 1 and all were Class 3. When all Class 5 subjects were assumed be Class 1, TPS remained a significant (p<0.001) independent predictor. In contrast, when all Class 5 subjects were presumed to be Class 3, TPS was no longer associated (p=0.99) with days to first extubation. However, since the characteristics of subjects with an incomplete echocardiogram were similar to those with an assignable TPS, this latter sensitivity assumption (100% Class 3) is discordant with the observed percentage of subjects with a score who were assigned to Class 3 (17%).

We did not perform a sensitivity analysis regarding time to first extubation on Class 4 (no echocardiogram) subjects because only 2 of the 56 Class 4 subjects survived to Norwood discharge and had time-to-extubation data. Instead, we performed a sensitivity analysis using the regression model for early mortality/transplant including Class 4 subjects assuming that all had either Class 1 or 3 TPS. When Class 4 subjects were all assumed to be Class 1, TPS remained associated (p=0.050) with early mortality/transplant. However, we expect that Class 4 subjects would be most likely to have had an Inadequate TPS (Class 3) rather than an Optimal (Class 1) TPS, because 96% of the Class 4 subjects died or underwent transplant. When Class 4 subjects were all assigned to Class 3, TPS was strongly associated with mortality (p<0.001).

Discussion

While the role of human factors in outcomes after surgery has been well described^16–18, technical adequacy of the repair may still be the single most important factor in determining outcomes. This premise has been proven in other surgical fields^19–21 in which assessment models have been based on methods such as video recordings. Indeed, models of surgical skills assessment have been incorporated into surgical training programs^22–25. However, a tool using routine clinical information for measurement of technical adequacy has not been validated. As previously described at a single center^3–9, TPS is based upon information available in routine clinical care, including postoperative echocardiography and reintervention in anatomic areas of surgical repair. Moreover, TPS has been associated with early outcomes, such as mortality, major adverse events, longer LOS and higher costs, as well as mid-term outcomes after hospital discharge, such as transplant-free survival and unplanned reinterventions.

In this multicenter study, we found that TPS was an independent predictor of both early and mid-term outcomes after the Norwood procedure. Specifically, our multivariable model, which included baseline patient factors and preoperative variables, found that a worse TPS was significantly associated with longer time to first extubation, higher early mortality or transplantaton, longer Norwood LOS, more unanticipated interventions between Norwood discharge and the Stage II procedure, and lower PDI scores at 14 months. We did not find an association of TPS with unanticipated reinterventions after the Stage II procedure, 14-month MDI score, or late mortality/transplant. The association of worse TPS with lower 14-month PDI score but not MDI score is consistent with the nearly universal observation that PDI scores are more affected than MDI scores in children with congenital heart disease.^26–29 Early death in subjects with inadequate repair may explain the lack of association of TPS with late mortality/transplant, as well as the higher rate of reintervention before Stage II among subjects in Class 2 compared with Class 3. Analyses of the components of the TPS showed reinterventions were significantly associated with early mortality and Norwood LOS, whereas higher arch gradients were associated with pre-Stage II reinterventions. This multicenter study validates the generalizability of TPS across centers that vary widely in geographic location and volume.

The effect of TPS on Norwood outcomes is relatively modest compared with that of other independent risk factors measured in the SVR trial. This is, in part, because regression models in the current study could not include intraoperative and postoperative variables that are in the causal pathway of the factors on which calculation of the TPS is based. For example, in an earlier manuscript from the SVR trial, extracorporeal membrane oxygenation (ECMO) and open sternum on the day of the Norwood procedure were among the most powerful independent predictors of 30 day and hospital mortality, as well as time to first extubation after the Norwood procedure, the primary outcome in the current study³⁰. Similarly, independent predictors for interstage mortality included use of modified Blalock-Taussig shunt rather than right ventricular-to-pulmonary artery shunt in subjects with no or mild postoperative AVVR and greater number of post-Norwood complications³¹. Our regression models did, however, consider patient and preoperative medical risk factors previously shown to be independent risk factors for adverse outcomes before Norwood discharge and in the interstage, such as low birth weight,³⁰ lower gestational age,³¹ and anatomic factors, e.g., the presence of aortic atresia/mitral atresia.³¹ Of note, TPS is among the most modifiable of the various risk factors demonstrated in the SVR trial.

TPS has been based on transthoracic echocardiogaphy performed before hospital discharge. A scoring system for intraoperative echocardiography might provide timely information and elucidate components of an “inadequate” procedure that would benefit from immediate intraoperative revision. Indeed, recent work has shown intraoperative revision of residual lesions improves in-hospital outcomes.⁵

Our analysis should be viewed in light of its limitations. We were unable to calculate TPS for one third of subjects because of missing or incomplete protocol echocardiograms together with absence of reinterventions at sites of initial surgical correction. Mortality was a secondary study outcome, because its analysis with respect to TPS was conditional on early survival; that is, the vast majority of subjects without echocardiograms (91%) died early (i.e., before the protocol echocardiogram was performed -Class 4 subjects). Although it seems likely that most of these subjects would have had Class 3 TPS, our study design does not allow us to test this hypothesis directly. In sensitivity analyses, assuming that all of these subjects had either Class 1 or Class 3 TPS, TPS remained an independent predictor of mortality before Norwood discharge. The primary endpoint of this study, time to initial extubation, may be affected by center practices and protocols. However, center was considered as a covariate in multivariable models, and other measures of overall hospital complexity, such as length of stay in the CICU, are even more affected by center variation.³² Finally, the components of the TPS were derived by consensus of experts, rather than by determination of cut-points from analyses of prospectively collected data, and TPS Classes 1 and 2 were often similar in their prediction of outcomes. A prospective multicenter study of the Pediatric Heart Network, currently in its planning phase, will refine the cut-points of the TPS using both a multicenter expert panel and analysis of prospectively collected data.

In summary, TPS is an independent predictor of both early and mid-term outcomes in patients with hypoplastic left heart syndrome and other single right ventricle anomalies undergoing the Norwood procedure. With further multicenter refinement and testing, TPS may be used as a tool for quality improvement, with potential for impact not only on patient outcomes, but also on resource utilization in this costly, high-risk population.

Abbreviations

ASD

Atrial Septal Defect

CI

Confidence Interval

IQR

Interquartile Range

LOS

Length of Stay

MBTS

Modified Blalock-Taussig Shunt

MDI

Mental Developmental Index

OPVR

Obstructed Pulmonary Venous Return

OR

Odds Ratio

PA

Pulmonary Artery

PDI

Psychomotor Developmental Index

RI

Reinterventions

RVPAS

Right Ventricle to Pulmonary Artery Shunt

SES

Socio Economic Status

SVR

Single Ventricle Reconstruction

TPS

Technical Performance Score