The Prediction and Cost of Futility in the NICU

Abstract

Aim

To quantify the cost and prediction of futile care in the Neonatal Intensive Care Unit (NICU).

Methods

We observed 1813 infants on 100 000 NICU bed days between 1999 and 2008 at the University of Chicago. We determined costs and assessed predictions of futility for each day the infant required mechanical ventilation.

Results

Only 6% of NICU expenses were spent on nonsurvivors, and in this sense, they were futile. If only money spent after predictions of death is considered, futile expenses fell to 4.5%. NICU care was preferentially directed to survivors for even the smallest infants, at the highest risk to die. Over 75% of ventilated NICU infants were correctly predicted to survive on every day of ventilation by every caretaker. However, predictions of ‘die before discharge’ were wrong more than one time in three. Attendings and neonatology fellows tended to be optimistic, while nurses and neonatal nurse practitioners tended to be pessimistic.

Conclusions

Criticisms of the expense of NICU care find little support in these data. Rather, NICU care is remarkably well targeted to patients who will survive, particularly when contrasted with care in adult ICUs. We continue to search for better prognostic tools for individual infants.

INTRODUCTION

Neonatal Intensive Care Unit (NICU) care is expensive – and sometimes babies die despite our best efforts. We address two questions here: first, how much money do we spend on babies who die, and on babies who are predicted to die, in the NICU? Second, which medical caretakers are better, or worse, at predicting NICU deaths?

At the outset, we will restrict our analyses to infants who require mechanical ventilation during their NICU stay. For these infants, an alternative to continued NICU intervention – namely, extubation and compassionate care – is available if parents and medical caretakers agree. We will also extend our analyses over time – that is, we will not restrict ourselves to predictions that are made before birth, in the delivery room, or even during the first NICU day. Rather, we will gather information over the entire time that the infant remains ventilated in the NICU, in an attempt to maximize our prognostic power.

METHODS

Prediction protocol

For 10 years (1999–2008), we identified 1813 ventilated infants in the NICU at the University of Chicago. On each day of each infant’s mechanical ventilation, we attempted to ask attending physicians, fellows, residents, registered nurses (RNs) and neonatal nurse practitioners (NNPs) one question: ‘Do you think this baby will die in the NICU or survive to be discharged?’ This question was followed by an assessment of the respondent’s confidence level – low, medium or high. Respondents were asked these questions privately, in a space separate from the patient’s bedside. Researchers also attempted to isolate respondents from each other at the time of questioning to prevent cross-respondent interference. At all times, respondents were allowed to give their prediction as ‘uncertain’ or ‘decline to participate’. Researchers attempted to obtain at least three predictions for each baby on each day of mechanical ventilation.

Characterizing predictions

For each infant, for each day of mechanical ventilation, we began by determining which level of respondent (i.e. attending, fellow, resident, nurse, NNP) had predicted ‘live’, ‘die’ or ‘uncertain’. No individual was identified at any phase of the data collection or analysis. All responses accompanied by an admission of ‘low confidence’ and all responses of ‘uncertain’ were not analysed further. A ‘discordant’ ventilator day was defined as a day on which the infant received at least one optimistic prediction [‘survive to NICU discharge’ with medium or high confidence], while also receiving at least one pessimistic prediction [‘die before NICU discharge’ with medium or high confidence]. ‘Concordant’ ventilator days were characterized by unanimous predictions with medium or high confidence – either every respondent predicted ‘live’ or every respondent predicted ‘die’.

Predictions were also characterized along a separate dimension as ‘correct’ or ‘incorrect’. ‘Correct’ predictions were either predictions of ‘live’ followed by patient discharge from the NICU (optimistic/correct), or ‘die’ followed by patient demise (pessimistic/correct). Conversely, ‘incorrect’ predictions were either ‘live’ for patients who eventually died (optimistic/incorrect), or ‘die’ for patients who survived to discharge (pessimistic/incorrect).

Determining NICU costs

Recognizing that the large majority (>80%) of patients in our NICU are reimbursed on a per diem basis (as opposed to a fixed percentage of charges generated), we approximated costs for each infant’s hospitalization by determining the number of bed days occupied from birth to NICU discharge [i.e. length of stay (LOS)]. We were able to determine LOS for 1660 of 1813 (92%) ventilated infants treated in our NICU during the 10-year study period. The other 153 infants had been back-transferred to Level II hospitals, and their LOS data were not available to us.

We assessed two distinct definitions of futile NICU care: (i) all bed days occupied by infants who died before NICU discharge; and (ii) all bed days occupied by NICU nonsurvivors after they had been predicted to die. We then further stratified futile care using increasingly stringent predictions of ‘die’ (e.g. 1 prediction of ‘die’ on a single day; >1 prediction of ‘die’; unanimous prediction of ‘die’; multiple days of predictions of ‘die’). We contrasted the bed days occupied by patients who survived, died, had been predicted to die and had never been predicted to die. In an attempt to increase the generalizability of our observations, we also determined cost data for the subset of ventilated infants whose birth weight was <1000 g (ELBW).

Statistical analyses

Parametric comparisons among patient groups were made using t tests and ANOVA. Statistical corrections were made for repeated analyses of the same data set. Non-parametric comparisons were performed using chi-square tests. Multigroup chi-square analyses were performed to determine whether any particular caregiver group (e.g. fellows, nurses, attendings, NNPs) was significantly more optimistic, pessimistic, correct or incorrect. To account for multiple comparisons using the same data set, statistical significance was accepted at an alpha level of p < 0.01.

This study was approved the Institutional Review Board of University of Chicago.

RESULTS

Costs of futile care in the NICU

Table 1 displays demographic data for 1813 ventilated patients in the University of Chicago NICU between 1999 and 2008. About 246 (14%) of 1813 ventilated infants died. Not surprisingly, nonsurvivors were smaller, younger and had shorter length of stay than survivors. There was no difference in sex or race distribution comparing survivors and nonsurvivors.

Table 1.

Comparison of surviving and nonsurviving ventilated infants

	Nonsurvivors (n = 246)				Survivors (n = 1567)
	Average	SD	Median	Sum	Average	SD	Median	Sum
BW	1437.6	1104.5	802.5		1775.4	1052.0	1382.5
GA	29.3	6.1	26.0		31.4	5.4	30.0
LOS	29.9	45.4	13	7357	61.4	53.1	50	96 217

BW, birth weight; GA, gestational age; LOS, length of stay.

Figure 1 displays our central finding regarding ‘futile’ NICU costs. Data from 1660 ventilated infants and 95 360 NICU bed days are represented. No matter what definition of futility is used [e.g. all bed days occupied by nonsurvivors; nonsurvivor bed days occupied after one (or more) predictions of nonsurvival], the vast majority (>90%) of NICU bed days were occupied by infants who survived to be discharged home.

Figure 1.

Futile expenses for nonsurviving ventilated NICU infants.

Figure 1 also displays NICU costs restricted to 583 ventilated ELBW infants. 49 889 NICU bed days are represented. The 77% of ventilated ELBW infants survived, ranging from 26% survival for birth weight (BW) < 500 g to 93% for BW 875–1000 g. Three points are clear from an examination of the ventilated ELBW population. First, only 7.8% of ELBW bed-days were occupied by ELBW nonsurvivors. Second, 6.0% of ELBW bed days were occupied by ELBW infants who had ever been predicted to die. Finally, only 4.7% of ELBW NICU bed days were occupied by non-surviving ELBW babies after they had been predicted to die – and could, in this sense, ethically be classified as futile. By analysing the ELBW population in greater detail, we observed that for infants with BW < 500 g, at most 29% of NICU bed days were futile, falling to 8% for babies with BW 501–750 g and 2% for babies with BW 751–1000 g.

Absolute NICU costs for ventilated infants can also be calculated from our data. The 1660 infants accounted for a total of 95 360 NICU bed days. Reimbursed at $2000/day, the overall cost to society of NICU care for these 1660 ventilated infants was approximately $191 M. About 1422 of these infants survived. Consequently, the NICU cost for each surviving ventilated infant was approximately $114 000. NICU costs for ventilated ELBW infants can be calculated similarly. 583 ELBW infants accounted for a total of 50 661 bed days, costing $101 M. 447 ELBW infants survived. Consequently, the overall NICU cost per surviving ELBW infant was approximately $226 000.

Accuracy of predictions of NICU outcome

Caretaker predictions were obtained on 22 899 days of mechanical ventilation (average 2.9 predictions/ventilator day) for 1813 infants. 30% of ventilated infants received at least one prediction of ‘die before d/c’. Only 40% of infants predicted to die actually did so. For increasingly stringent criteria (corroborated or unanimous prediction of death on a single day), positive predictive value (PPV) of predictions of ‘die’ improved slightly (59% and 68%; p < 0.01). We observed the same pattern when we restricted our analysis to the 583 ventilated ELBW infants in our study. The 45% of ventilated ELBW infants received at least one prediction of ‘die before d/c’. Only 51% of infants predicted to die actually did so. For increasingly stringent criteria (corroborated or unanimous prediction of death), PPV of predictions of ‘die’ improved slightly (62% and 71%; p < 0.01).

As Table 2 reveals, 1430 infants (79% of all ventilated infants) never had even a single day of disagreement on whether the infant would live or die. The 383 infants (21% of the population of ventilated infants) had at least 1 day of ‘discordant’ predictions – where one caretaker predicted ‘live’ and another predicted ‘die’. These infants were smaller (930 vs. 1406 g), younger (27 vs. 30 weeks) and more likely to die (37% vs. 7%) than the ‘concordant’ infants (all p < 0.01).

Table 2.

Comparison of concordant and discordant ventilated infants

	Concordant points	Discordant points
n	1430	383
GA	1406	930
BW	30	27
LOS	40	58
Days of ventilation	6	28
Mortality	104 (7.3%)	142 (37.1%)
Number of prediction days	12 144	10 755
Number of discordant days	0	1242

GA, gestational age; BW, birth weight; LOS, length of stay.

All comparisons p < 0.01.

The 1303 babies (92% of concordant infants; 72% of all ventilated infants) had unanimous predictions of ‘live’ on every day of mechanical ventilation. The 97% of these infants survived. In contrast, 127 (8% of concordant infants) had at least 1 day where all caretakers predicted ‘die before discharge’ – 52% of these infants died.

For the 383 discordant babies, a total of 3663 predictions were offered on 1242 discordant ventilator days. On average, discordant predictions were observed on 3.2 (12%) of 26 days of ventilation for these infants. Figure 2 displays the accuracy of predictions of ‘die before discharge’ for discordant patients, grouped by level of NICU provider (attending, fellow, resident, nurse and NNP). There was no significant difference in accuracy comparing the provider groups – the PPV ranged between 51% and 64%.

Figure 2.

Positive predictive value of predictions of ‘die before NICU discharge’.

Table 3 displays the effect of optimism and pessimism on predictive accuracy for each provider group on each day of discordant predictions. Attendings and neonatology fellows were significantly more optimistic than the other respondent groups, predicting ‘live’ 71% and 66% of the time, respectively. Residents, RNs and NNPs were significantly more pessimistic, with live predictions of 49%, 47% and 44%, respectively. The relative optimism of attendings and fellows was reflected in a higher percentage of correct predictions for patients who survived (67% of the discordant population). In contrast, RNs and NNPs, who were more likely to be pessimistic, more accurately predicted the discordant patients who eventually died.

Table 3.

Effects of optimism and pessimism on prediction accuracy

	Optimistic correct (%)	Optimistic incorrect (%)	Pessimistic correct (%)	Pessimistic incorrect (%)
Attending	46.1*	25.0*	13.1*	15.8*
Fellow	46.3*	19.7	17.6	16.4*
Resident	31.9	17.9	20.0	30.2
RN	30.2	17.0	20.7	32.1*
NNP	32.4	12.9*	26.4*	28.3

RN, registered nurse; NNP, neonatal nurse practitioners.

^*p < 0.01; chi-square.

DISCUSSION

NICU care has often been criticized along two dimensions. First, some have claimed that NICU care is simply too expensive – that we spend too much money on individual infants and that cumulatively these costs become prohibitive for society as a whole (1–3). Second, NICU care has been criticized for futile over-treatment of infants doomed to die – becoming a cruel perversion of the physician– patient relationship (4–7). We attempted to gather data that might support, or not, each of these criticisms.

We restricted our analyses to infants who required mechanical ventilation in our NICU, precisely because for these infants, an ethical alternative to continued intervention, namely extubation and compassionate care, is available. We report here observations on 1813 infants cared for more than 100 000 NICU bed days between 1999 and 2008 at the University of Chicago. This is the largest study, of which we are aware, that directly addresses questions of futility and prediction in the NICU. We draw several conclusions from our data.

1. For infants who require mechanical ventilation, only 7% of overall NICU expenses are spent on nonsurvivors, and in this sense can be thought of as futile.

2. If nonsurvivor expenses are considered futile only after a prediction of death has been offered, then futile NICU expenses fall to 4% for the entire population of ventilated infants.

3. The selective targeting of NICU resources in favour of NICU survivors was observed to be independent of mortality rates – NICU care is preferentially directed to survivors, even for the smallest ventilated ELBW infants, at the highest risk to die.

4. The vast majority (>75%) of ventilated NICU infants are correctly predicted to survive on every day, by every caretaker, at every level (attendings, fellows, residents, nurses, NNPs). For our entire population of ventilated NICU patients, fewer than 5% of prediction days were characterized as discordant.

5. For discordant patients – that is, patients for whom some caretakers predict ‘live’ while others predict ‘die’ on the same day – attendings and fellows tended to be optimistic, and nurses tended to be pessimistic. However, predictions of ‘die before discharge’ for even the most accurate caretaker group (neonatology fellows) were wrong more than one time in three.

We discuss the implications of these observations in turn.

The average cost to society for each surviving ventilated infant in our NICU was approximately $114 000. For each survivor with BW < 1000 g, the average cost to society was approximately $225 000. These cost estimates agree well with previously published data (8). After discharge, the average NICU survivor has been reported to generate lifetime medical costs in an amount approximately equal to the cost of their original NICU stay (9,10). Consequently, the cumulative lifetime medical costs of our ventilated ELBW infants are approximately $450 000. Assuming that NICU survivors live 70 years, these costs amount to approximately $6400/ELBW survivor/year. Formulas exist to adjust for future deficits in ‘quality of life’. Assuming a ‘worst case’ scenario, that the average ELBW NICU survivor has a quality of life equal to 0.7 of a term infant, the maximum calculated cost of NICU care per ELBW patient per quality-adjusted life year is $9100. This figure agrees well with previous published values (8,11,12) and is roughly one-fifth that of many interventions that are widely used by U.S. Medicare and accepted for adult patients, such as coronary artery bypass grafts or renal dialysis. (13).

Futile costs can also be compared between NICUs and adult medical intensive care units (MICUs). In our population of ventilated infants, only 8% of NICU bed days were devoted to infants who would not survive to discharge. In contrast, nearly 40% of MICU costs are devoted to patients who will die before hospital discharge (14,15). In the United States each year, approximately 30 000 infants die. In contrast, one hundred times more adults (approximately 3 million) die yearly in the United States, nearly one-third of whom will be admitted to a MICU within 6 months of their death (16–18). Consequently, from a public policy standpoint, there are 100 times more dying adults than dying infants, and adult intensive care unit (ICU) care is at least five times less efficient (that is, less accurately directed towards patients who will survive to be discharged). By these accounts, it is clear that if end-of-life cost savings for society as a whole are desired, the MICU, and not the NICU, will have to be the site of cost reduction.

If excessive costs cannot be invoked as a valid criticism of NICU care, perhaps the cruelty of over-treatment for doomed infants can. These concerns have been raised previously by both neonatologists (6,7) and NICU parents (5). In response, our data would suggest two findings – first, in aggregate, nearly 80% of ventilated infants will survive to be discharged from the NICU, and every caretaker knows it, every day. Second, it is not as easy as it might appear to know when care is ‘futile’. In this study, we analysed over 67 000 caretaker predictions offered on 22 000 days of mechanical ventilation for 1813 ventilated infants in our NICU. Our central finding is that, when asked on a daily basis to identify which of their patients was likely to ‘die before discharge’, all levels of NICU caretakers – attendings, fellows, residents, nurses, NNPs – were likely to be wrong. No group even reached 65% correct. Moreover, increasingly stringent prediction criteria – such as requiring more than one prediction of ‘die’ on a single day, unanimous predictions of ‘die’ on a single day or multiple days of predictions of ‘die’ – improved predictive power only slightly. More than one in four infants with a unanimous prediction of ‘die in the NICU’ survived to be discharged. We think it unlikely that parents will be persuaded to shift from NICU intervention to palliative care when the factual basis of a recommendation is not much better than flipping a coin.

Several limitations of our study can be acknowledged explicitly. Concerns might arise regarding the possibility of ‘self-fulfilling prophesy’ in our study – that is, once patients have been predicted to die, they will receive less intensive care and thus be more likely to die. This phenomenon has been noted in other ICU settings (19,20). However, one of our central findings is that medical caretaker predictions of ‘die before NICU discharge’ are wrong nearly half the time. If a self-fulfilling prophecy were at work, these predictions should be much more correct – or, conversely, to the extent that a self-fulfilling prophecy is playing a role in our findings, our ‘real’ predictive power is even less than 50/50. Moreover, we have previously reported that the frequency of ‘negotiated deaths’ in our NICU is extremely low (approximately 5%), a figure which we attribute to the demographics of our parent population – poor, inner city, black, religious, distrustful of medical authority and believing in miracles. These parents, in our experience, are extremely likely to choose ICU intervention over palliative care, even in situations where the outcome seems grim. This phenomenon has been previously reported in other ICU settings for a similar patient demographic (21).

We recognize that our predictive measure, surveying the clinical intuitions of medical caretakers on each day of mechanical ventilation, is unorthodox. Why study the predictive power of serial intuitions at all? We, and others, have previously demonstrated that objective measures of illness severity – whether obtained in the delivery room (Apgar scores) or on the first day of life (SNAP), have well-known inaccuracies when used for individual prognostication (22–25). Moreover, we have previously reported that serial observations of SNAP scores over time become less, not more, accurate in predicting the outcome of death in the NICU or survival with significant neurological morbidity (26).

But infants do change during their NICU stay, and parents, doctors, nurses and policy makers are all interested in making up-to-date assessments of their future outcome. Physicians and nurses who work in NICUs, and who must make decisions or recommendations for individual babies, always consider the data from objective measures when they assess a baby and make a clinical judgment about prognosis. But they consider other nonquantifiable things, too. Their unique expertise lies in their ability to consider both objective measures and subjective ones. In the end, they are left with an ‘intuition’ or a ‘professional judgment’ about the baby’s prognosis. This judgment is difficult to quantify. But it is important. We tried to quantify that judgment by asking physicians and nurses to commit to a prediction of outcome for each baby on each day that the baby remained on the ventilator. This is, admittedly, an unusual sort of prognostic variable. It is, however, the one that clinicians use in their actual practice day-in and day-out. We wanted to test the prognostic power of clinical intuitions for the outcome of death in the NICU while children were being mechanically ventilated in the NICU. We found that the predictive power of these intuitions was disappointingly low.

Finally, we have no long-term outcome data for the 1813 infants described here. For many parents, death in the NICU may not be the worst outcome – rather, survival with significant disability may be most feared (27,28). For other parents and children, living with disability seems to be remarkably well accepted (29–31). It is likely that many of the infants who were predicted to die and yet survived to NICU discharge will be disproportionately likely to have abnormal neurological development, compared to comparable BW/GA infants who were never predicted to die. We have previously demonstrated in a smaller cohort of ventilated NICU patients that predictions of ‘die before discharge’, coupled with abnormal head ultrasounds, predicted the combined outcome of either death or neurological morbidity by age 2 with 96% accuracy (32). However, as only 20% of ventilated NICU patients, and fewer than 5% of all ventilated NICU bed days, were associated with any prediction of ‘die before discharge’, accounting for impairment in NICU survivors predicted to die will impact our futility calculations only slightly. Finally, we recognize that our data represent observations in only one NICU, and would welcome replication in other NICUs.

In sum, we report here the costs and prediction of futile care for infants sick enough to require mechanical ventilation in the NICU. We found that over 90% of all NICU bed days were occupied by infants who were eventually discharged home, independent of their birth weight, gestational age or mortality risk. This figure rose above 95% when the definition of futility was adjusted to consider only bed days accumulated after a prediction of die had been made. For nearly 80% of ventilated infants, on every day that they required ventilation, there was never disagreement among caretakers about the fact that these infants would survive – and they did. For the remaining 20% of ventilated infants, all levels of caretakers were wrong at least one-third of the time in predicting their demise. We continue to search for better prognostic tools. We owe it to ourselves, our patients and their families.