A Cost-effectiveness Analysis Comparing a Clinical Decision Rule Versus Usual Care to Risk Stratify Children for Intraabdominal Injury After Blunt Torso Trauma

Abstract

Objectives

Recently a clinical decision rule (CDR) to identify children at very low risk for intraabdominal injury needing acute intervention (IAI) following blunt torso trauma was developed. Potential benefits of a CDR include more appropriate abdominal computed tomography (CT) use and decreased hospital costs. The objective of this study was to compare the cost-effectiveness of implementing the CDR compared to usual care for the evaluation of children with blunt torso trauma. The hypothesis was that compared to usual care, implementation of the CDR would result in lower CT use and hospital costs.

Methods

A cost-effectiveness decision analytic model was constructed comparing the costs and outcomes of implementation of the CDR to usual care in the evaluation of children with blunt torso trauma. Probabilities from a multicenter cohort study of children with blunt torso trauma were derived; estimated costs were based on those at the study coordinating site. Outcome measures included missed IAI, number of abdominal CT scans, total costs, and incremental cost-effectiveness ratios. Sensitivity analyses varying imputed probabilities, costs, and scenarios were conducted.

Results

Using a hypothetical cohort of 1,000 children with blunt torso trauma, the base case model projected that the implementation of the CDR would result in 0.50 additional missed IAIs, a total cost savings of $54,527, and 104 fewer abdominal CT scans compared to usual care. The usual care strategy would cost $108,110 to prevent missing one additional IAI. Findings were robust under multiple sensitivity analyses.

Conclusions

Compared to usual care, implementation of the CDR in the evaluation of children with blunt torso trauma would reduce hospital costs and abdominal CT imaging, with a slight increase in the risk of missed intraabdominal IAI.

Intraabdominal injury is a leading cause of morbidity and mortality in children. Failure to rapidly identify these injuries can lead to preventable morbidity and mortality.^1–4 Abdominal computed tomography (CT) is the primary diagnostic imaging modality in the diagnosis of pediatric intraabdominal injury.^5,6 Clinicians, however, are inaccurate in identifying children who require CT imaging for suspected intraabdominal injury, with fewer than 2% of children with abdominal CT imaging requiring acute intervention.^7–9

Overuse of abdominal CT scans results in increased health care costs and prolonged emergency department (ED) stays and adds to the risk of radiation induced malignancy.^10,11 Both the underuse and the overuse of abdominal CT scanning have potential adverse effects that could be reduced by the appropriate targeting of scanning to only those children at risk of clinically important injuries.

Recently, our group led the development of a clinical decision rule (CDR) in the Pediatric Emergency Care Applied Research Network (PECARN) to identify children with blunt torso trauma who are at low risk for clinically important intraabdominal injury (defined as death or requiring an acute intervention).¹² The rule was derived in a multicenter, prospective, observational study of 12,044 children with blunt torso trauma. Children without any risk factors in the decision rule (evidence of abdominal wall trauma, Glasgow Coma Scale score less than 14, abdominal tenderness on examination, evidence of thoracic trauma, complaints of abdominal pain, decreased breath sounds on examination, or history of emesis) are considered at very low risk for intraabdominal injury needing acute intervention (IAI) requiring needing acute intervention (0.12%; 95% confidence interval [CI] = 0.04% to 0.26%) and thus do not require routine abdominal CT scanning. It is estimated that by eliminating abdominal CT imaging in low-risk children with blunt torso trauma, imaging would decrease by 23% (95% CI = 22% to 24%) in this population.¹²

After development of a CDR, it is recommended that an economic analysis be conducted to evaluate the economic implications of the rule.¹³ This is often done in the form of a cost-effectiveness analysis, which evaluates both costs and consequences of alternative strategies.^14,15 The objective of this study was to model the cost-effectiveness of implementing the CDR compared to usual care in the evaluation of children with blunt torso trauma. We hypothesized that compared to usual care, implementation of the decision rule would result in lower estimated CT use and lower hospital costs, although it would increase the rate of missed intraabdominal injury needing intervention.

METHODS

Study Design

We used a decision analytic approach to estimate incremental costs and outcomes when implementing the CDR compared to usual care. The study was approved by the University of California, Davis, institutional review board.

Model Creation

We constructed a decision tree model using TreeAge Pro 2011 (TreeAge Software, Inc., Williamstown, MA) decision analytic software. In the model, we compared two clinical strategies: usual care, and implementation of the CDR (Figure 1). We defined the usual care strategy as the management of children with blunt torso trauma by clinicians not using the CDR. We derived the clinical course and probabilities (e.g., abdominal CT use, hospital admission, IAI) from PECARN study data in which clinicians did not have the CDR available to them to manage children with blunt torso trauma (i.e., they used usual care).¹² We defined the CDR strategy as the hypothetical implementation of the PECARN study-based CDR. In the CDR strategy, we classified children as CDR positive (CDR+) if the child had one or more of the clinical risk factors present and CDR negative (CDR− or low risk) if the child had no risk factors present. The primary objective of the rule was to serve as a decision support tool for clinicians to identify children with blunt torso trauma at low risk for IAI and thus unlikely to require abdominal CT. There is no implication that children who are positive for any of the variables in the rule (i.e., CDR+) should undergo abdominal CT. Thus, in the decision model, we modeled CDR− patients to not have abdominal CT scans and modeled CDR+ patients to follow usual care (i.e., what actually occurred in the observational cohort of children in the PECARN study). Both strategies were subject to the accuracy of abdominal CT. Key parameters used in the model, along with 95% CI, are shown in Table 1.

Figure 1.

Abbreviated decision tree. CDR = clinical decision rule; IAI = intraabdominal injury requiring acute intervention.

Table 1.

Probability Parameters^*†

Parameter Name	Base Case Value, % (95% CI)
Prevalence of IAI	1.7 (1.5–1.9)
Sensitivity of CDR strategy‡	97.0 (93.7–98.9)
Specificity of CDR strategy	42.5 (41.6–43.4)
Sensitivity of usual care strategy	99.5 (96.7–100.0)
Specificity of usual care strategy	56.2 (55.3–57.1)
Sensitivity of CDR+ arm	99.5 (96.6–100.0)
Specificity of CDR+ arm	42.2 (41.0–43.3)
Accuracy of abdominal CT in detecting IAI
Sensitivity	98.2 (94.3–99.5)
Specificity	100.0 (98.1–100.0)
% of IAI requiring therapeutic laparotomy or angiography	61.6 (54.5–74.4)

CDR = clinical decision rule; CDR+ = at least one CDR criterion present; IAI = intraabdominal injury requiring acute intervention.

^*See Data Supplement S1 for complete list of probabilities.

^†Probabilities are derived from PECARN study data.¹²

^‡Sensitivity of CDR strategy refers to accuracy of CDR to identify patients with IAI (one or more CDR criterion present) and does not imply CT imaging.

The time horizon for the model included the acute injury period based on the assumption that clinical outcomes from IAI occur in a period of days to weeks, as well as the lack of long-term outcome data for children with blunt torso trauma available in the literature or the PECARN cohort study.¹²

Study Setting and Population

We based our model cohort on the PECARN study population, which included children with blunt torso (thorax and abdomen) trauma occurring less than 24 hours prior to initial ED evaluation.¹² Complete inclusion and exclusion criteria for the PECARN cohort study have been previously described.¹²

Study Model

We projected outcome measures for a hypothetical cohort of 1,000 children with blunt torso trauma. These included IAI, missed IAI, number of abdominal CT scans, total costs, and incremental cost-effectiveness ratio (cost per missed IAI). For the usual care strategy, we defined missed IAI as an IAI not initially recognized by the clinician; that is, the patient did not receive an initial abdominal CT at the time of evaluation in the ED. For the CDR strategy, we defined missed IAI as an IAI missed by the decision rule; that is, the patient was considered low risk (CDR−) by the rule but had an IAI or if the patient was not low risk (CDR+) and usual care did not result in an initial ED abdominal CT scan and the patient had an IAI. We defined intraabdominal injury as any radiographically apparent injury to the spleen, liver, urinary tract (from the kidney to the bladder), gastrointestinal tract, pancreas, gallbladder, adrenal gland, intraabdominal vascular structure, or traumatic fascial defect (traumatic abdominal wall hernia). We defined an IAI as any of the following: death due to the IAI, an intervention at laparotomy deemed therapeutic or necessary, angiographic embolization due to bleeding from the IAI, blood transfusion for anemia secondary to abdominal hemorrhage from the IAI, or administration of IV fluids for at least two nights in those patients with pancreatic or gastrointestinal injuries. We did not evaluate quality-adjusted life-years (QALY) due to the lack of health-related quality-of-life data on children with blunt torso trauma.

Model Assumptions

The primary assumption of the model was that, in the CDR strategy, no low-risk children (CDR−) would have initial ED abdominal CT scans. CDR+ children (i.e., those with one or more risk factors) were assumed to undergo ED abdominal CT based on clinician judgment and at a similar proportion received by children in the PECARN study (i.e., usual care). Finally, in both the usual care and the CDR implementation strategies, we derived the probabilities for ED disposition (admission or discharge) from those observed in the PECARN study.

Probabilities

We primarily derived the process and outcome probabilities for the usual care and CDR cohorts from the PECARN study data (Data Supplement S1, available as supporting information in the online version of this paper).¹² Probabilities for the usual care cohort represent actual proportions observed in the PECARN study. For the CDR cohort, we derived the probabilities for each branch after hypothetically applying the CDR to the PECARN study population.

Costs

We derived the model costs for individual units of resources used for diagnostic testing, treatment, and hospital admission (boarding costs) from the financial department at the study site (Data Supplement S2, available as supporting information in the online version of this paper). These costs represented estimated 2011 hospital costs (as opposed to charges or reimbursements).

We calculated total costs for each of the seven unique end nodes by summing the estimated hospital costs for all resources used (Table 2 and Data Supplement S3, available as supporting information in the online version of this paper). We estimated all costs in United States dollars. The number of units used for each resource (e.g., days in the intensive care unit or number of units transfused) were estimated based on mean units used for children in each of the end nodes based on the PECARN data (Data Supplement S4, available as supporting information in the online version of this paper).

Table 2.

Total Costs for Study End Nodes^*† (Base Case)

End Node	Total Cost, $
Patient with IAI requiring therapeutic laparotomy or angiography	16,804
Patient with IAI requiring a blood transfusion or IV fluids for 2 or more nights	9,420
Patient with no IAI, receiving an abdominal CT scan, and is admitted to hospital	2,370
Patient with no IAI, receiving an abdominal CT scan, and is discharged home	1,234
Patient with no IAI, not receiving an abdominal CT scan, and is admitted to hospital	1,817
Patient with no IAI, not receiving an abdominal CT scan, and is discharged home	681
Patient with a missed IAI	11,483

IAI = intraabdominal injury requiring acute intervention.

^*See Data Supplements S2 and S3 for individual microcosts.

^†Estimated 2011 hospital costs.

Sensitivity Analysis

We used three different types of sensitivity analyses to evaluate the robustness of the model output, for three different objectives. First, we used probabilistic sensitivity analysis to address the issue of parameter uncertainty. We assigned beta distributions (best fit for binomial data) to all probability parameters, based on the point estimates and their 95% CIs derived from the PECARN data. In each of the 3,000 rounds of the simulation we conducted, the value of an input parameter was sampled from a specified distribution. The results of the simulations are presented as a cost-effectiveness scatter plot and an acceptability curve.¹⁶

Second, we used univariate (i.e., one way) sensitivity analysis to assess the generalizability of the model output. For example, because a substantial portion of the PECARN data were derived from pediatric hospitals staffed primarily by pediatric emergency medicine fellowship-trained physicians, the assumed distributions for sensitivity and specificity of physician diagnostic accuracy may not be reflective of practice in other settings with other practitioners. Univariate sensitivity analysis is complementary to the probabilistic sensitivity analysis by evaluating model projections when input parameters are changed to certain extreme values and to identify parameters that have substantial effect on the model. However, the potential for input interactions is unaccounted for with univariate sensitivities. We varied one model input by 20% of its base case value at a time and computed the effect on the incremental cost-effectiveness ratio. We summarized the result in a tornado diagram (bar chart where data are ordered vertically with the largest bar appearing at the top of the chart, the second largest appearing second from the top, and so forth). A tornado diagram allows readers to easily compare the relative importance of variables.

The third sensitivity analysis evaluated model structure uncertainty. In the base case model, we assumed that children with no CDR criteria present (i.e., low risk) did not receive abdominal CT scans. In real practice, a child may receive an abdominal CT scan even if his or her risk of having IAI is identified by the CDR as low. We tested this possible condition by running the model with a new branch added to the CDR arm where a proportion of low risk children received abdominal CT scans (the dotted line in Figure 1).

RESULTS

The main results are shown in Table 3. Per 1,000 children with blunt torso trauma, the CDR strategy was projected to fail to identify 0.50 more IAI, but had 104 fewer abdominal CTs compared to the usual care strategy. The total costs of the CDR strategy saved $54,527 per 1,000 children with blunt torso trauma compared to the usual care strategy. The incremental cost to detect one additional IAI was $108,110.

Table 3.

Base Case Output (per 1,000 Children With Blunt Torso Trauma)^*

Events	CDR	Standard Care	Difference
Number of abdominal CT scans obtained	343	447	104
Number of abdominal CT scans per IAI diagnosis	21	27	6
Number of missed IAI	0.59	0.09	–0.50
Total costs, $	2,293,140	2,347,667	54,527
Incremental cost per additional IAI detected, $			108,110

CDR = clinical decision rule; IAI = intraabdominal injury requiring acute intervention.

^*Estimated 2011 hospital costs.

The results of the probabilistic sensitivity analysis are shown in Figure 2. As demonstrated in Figure 2A, the usual care strategy cost more than the CDR strategy in all 3,000 simulations. In 2,238 (74.6%) simulations the usual care strategy diagnosed more IAI than the CDR strategy. Figure 2B displays the acceptability curves demonstrating that the CDR strategy has a higher likelihood to be cost-effective when the amount one is willing to pay for an extra IAI diagnosis is less than $113,600. The CDR strategy has greater than 99% chance of being the cost-effective choice when the amount one is willing to pay is less than $50,000.

Figure 2.

Probabilistic sensitivity analysis: cost-effectiveness scatter plot and acceptability curves. (A) Scatter plot of 3,000 simulations. Each dot represents the extra cost and number of IAI diagnoses for the usual care strategy compared to the CDR strategy for one simulation. (B) Acceptability curves derived from the 3,000 simulations. The higher the amount one is willing to pay to diagnose an additional IAI, the lower the probability of the CDR strategy being more cost-effective than the usual care strategy. For example, when the amount one is willing to pay for an extra IAI diagnosis is $50,000, the CDR is cost-effective (i.e., has an incremental cost-effectiveness ratio greater than $50,000) nearly 100% of the time (of the 3,000 simulations). When the amount one is willing to pay for an extra IAI diagnosis is $113,600, half of the simulations will have an incremental cost-effectiveness ratio less than $113,600 (usual care is more cost-effective) and half will be greater (CDR strategy is more cost-effective). CDR = clinical decision rule; IAI = intraabdominal injury requiring acute intervention.

We changed the input parameters by 20% of the base case values to conduct univariate sensitivity analyses (Figure 3). The tornado diagram demonstrates the change in the incremental cost-effectiveness ratio as the values of probability and cost parameters change. The most sensitive parameter is the sensitivity of the CDR: when the CDR becomes less accurate in identifying IAI patients, it is less expensive to “buy” an extra IAI diagnosis through usual care. The second most sensitive parameter is the specificity of clinician’s diagnostic accuracy for usual care (based on actual CT scans obtained by clinicians). The purpose of the CDR is to aid clinicians in determining which children with blunt torso trauma do not need abdominal CT scans. If clinicians already have high accuracy in excluding patients requiring CT imaging, the CDR would be less useful. Among cost inputs, total cost of abdominal CT scan was the most sensitive parameter.

Figure 3.

Univariate sensitivity analysis: tornado diagram and threshold analysis. *Tornado diagram displays the univariate sensitivity analyses for various parameters from most sensitive (top) to least sensitive (bottom) parameters. The sensitive variable is modeled as an uncertain value while all other variables are held at baseline values. Solid bars refer to increases in parameters and striped bars refer to decreases in parameters. Values above $200,000 and below $0 per extra IAI diagnosis are not shown. † To evaluate the sensitivity of clinician diagnostic accuracy, we used the point estimate of the usual care strategy for both the usual care strategy and the CDR+ arm. This is because changes in clinician diagnostic accuracy (e.g., different level of training) would affect both the usual care strategy and the CDR+ arm similarly (i.e., it is unlikely that sensitivity would change in one but not the other). We were unable to do this with the specificity of clinician diagnostic accuracy since the point estimates of the usual care strategy and CDR+ arm were very different (56 and 41%, respectively). This is because the CDR+ arm has a cohort of patients with a much higher risk for IAI (since the low-risk patients have been screened out). CDR = clinical decision rule; CDR+ = at least one CDR criterion present; IAI = intraabdominal injury requiring acute intervention.

We also evaluated the uncertainty of the model structure, specifically the possibility that in real-world practice, clinicians may still obtain abdominal CT scans in low-risk patients with implementation of the CDR (in the base case model, no low-risk patients receive abdominal CT scans in the CDR strategy). As Figure 4 demonstrates, even if clinicians obtain abdominal CTs in up to 17.4% of low-risk patients, the incremental cost of avoiding a missed IAI is greater than $50,000.

Figure 4.

Model structural uncertainty: one-way sensitivity analysis of percentage of low-risk patients receiving abdominal CT. The base case analysis assumes that no low-risk patients in the CDR strategy receives an abdominal CT. However, in the real-world setting, clinicians may still obtain some abdominal CT scans in these patients. Here, we demonstrate that as more low-risk patients receive an abdominal CT, the incremental cost-effectiveness ratio decreases. At an abdominal CT scan proportion of 28.2%, the CDR strategy becomes dominated (more missed IAI and more costly) by the usual care strategy. CDR = clinical decision rule; IAI = intraabdominal injury requiring acute intervention; ICER = incremental cost-effectiveness ratio.

DISCUSSION

The results of our decision model for the evaluation of children with blunt torso trauma demonstrate that compared to a usual care strategy, implementation of the CDR is associated with lower abdominal CT use and lower total costs, but an increased risk for missed IAI. Quantitatively, this tradeoff can be summarized as a cost of $108,110 per additional IAI detected with the usual care strategy.

Typically, cost-effectiveness analyses evaluate an intervention such as knee arthroplasty¹⁷ or renal transplant¹⁸ that is more effective and more expensive than other or no interventions. The usual measure of effectiveness is a QALY, which takes into account both the quantity and quality of life enhanced by health care interventions. A year of perfect health is worth a QALY of 1, a year of less than perfect health is worth less than 1, and death is worth 0.¹⁹ While the use of QALYs is less than perfect, it does provide a common metric to assess the extent of the benefits gained from an intervention in terms of health-related quality of life.^14,15

In our cost-effectiveness analysis, the intervention under evaluation (i.e., implementation of the CDR) is less costly and less effective. While the usual care strategy may diagnose additional IAI compared to the CDR strategy, many would agree that a cost of $108,110 per additional IAI detected represents poor use of resources. All health systems have limited resources, and they should be used efficiently and effectively, where potential health gains are greatest. Resources expended on the usual care strategy are subsequently not available for other interventions, which may have substantially greater effectiveness (i.e., large opportunity cost to the usual care strategy).

We were unable to use QALY as a measure of effectiveness due to lack of data. We instead used the diagnosis of missed IAI as our comparator metric, which we felt is a relevant measure of effectiveness in the acute injury period. The cost to detect one additional IAI is one perspective that can be derived from this analysis. To put this dollar figure into context, interventions with a cost per QALY of up to $45,000 to 50,000 are likely to be considered for approval for funding by the UK National Health Service.¹⁹ One could argue that the value of one QALY is much higher than the value of detecting an additional IAI, particularly since the additional IAI detected with the usual care strategy would likely not result in death or significant morbidity. Unfortunately very limited data exist on the clinical courses of children with missed IAIs. Reassuringly, no low-risk patient with missed IAI in the CDR strategy required therapeutic laparotomy or died in the PECARN study.¹² In addition, delayed diagnoses of gastrointestinal injuries in children have not been shown to increase morbidity or mortality.²⁰ The expected low morbidity with missed IAI would make the difference in cost between usual care and implementation of the decision rule unacceptably high, given that the resources used to increase the likelihood of finding an IAI could potentially be used for interventions with greater potential effects on child health. If, however, the additional IAI detected with the usual care strategy resulted in death or significant morbidity, the CDR strategy clearly would be less cost-effective (at $108,110 per additional IAI detected).

Although the CDR resulted in the failure to identify 0.50 injuries per 1,000 children, it is likely that this estimate is higher than what would be expected in actual practice. Clinicians who suspect the presence of IAI in patients who are CDR− would likely obtain screening tests (laboratory tests and abdominal ultrasound) that would help to identify those patients who have IAIs despite being CDR−. All patients in the PECARN study who were missed by the CDR had laboratory abnormalities suggestive of injuries or hemoperitoneum that could be detected with ultrasonography.¹² However, we did not include screening tests and their potential effect on the effectiveness or costs ($161 for abdominal ultrasound and $22 for laboratory testing) of either strategy.

Perhaps even more important than lower costs of care, the CDR strategy reduced radiation exposure compared with the usual care strategy. In the base case model, the usual care strategy resulted in 104 additional abdominal CT scans compared with the CDR strategy. Each year in the United States, more than 600,000 injured children are evaluated for IAI and approximately 138,000 patients (23%) receive abdominal CT scans.²¹ Implementation of the CDR strategy by eliminating abdominal CT imaging in low-risk children with blunt torso trauma would decrease abdominal CT imaging by 23%.¹² Using the estimated overall pediatric lifetime risk for fatal, radiation-induced malignancy with a single abdominal CT scan of 0.0679%, implementation of the CDR strategy could result in 21 fewer future deaths from radiation-induced malignancies per year in the United States.²² An additional 62 nonfatal radiation-induced malignancies could also be prevented.²³

LIMITATIONS

Our results should be interpreted in the context of certain limitations. The CDR strategy as applied in the decision model is hypothetical. Real-world implementation of the CDR strategy may have different probabilities than those imputed in our model. For example, clinicians likely would obtain abdominal CT scans in some low-risk patients even with implementation of the CDR in actual clinical situations. While we conducted a sensitivity analysis on this particular model assumption, we could not evaluate all potential differences in probabilities between our base case model and real-world implementation. It is possible that the rule may perform differently in other settings and by other practitioners, such as in nonacademic centers.²⁴ Implementation of the CDR may also increase CT imaging in CDR+ patients. This may occur since low-risk patients (CDR−) are now screened out, thus leaving clinicians with a cohort of children at higher risk for IAI and the potential to image more children than what was observed with usual care. In addition, implementation of the decision rule may also lead to reduced imaging but increased admissions (for serial abdominal exams), which may increase hospital costs. Modeling the decision rule using the data set from which it was derived biases toward the success of the rule. However, our sensitivity analyses demonstrated that the accuracy of the CDR and usual care strategies would have to be substantially lower to alter the results of the base case model. Because the accuracies of CDRs are often lower in external validation studies, we analyzed decision rule test characteristic ranges that were far inferior than reported in prior external validation studies.^25,26 Furthermore, the CDR was derived at children’s referral centers where clinicians are very experienced in the care of injured children. The CDR is likely to be more helpful to practitioners in non-children’s hospitals who are less experienced evaluating injured children.

We only included outcome measures in the acute injury period (e.g., CT use and missed IAI). Including longer time horizons and utility weights (e.g., use of QALYs) would provide more patient-oriented outcomes measures. A longer time horizon would also allow us to incorporate the risk of radiation-induced malignancies from abdominal CT use into our model. We were unable to incorporate these outcomes, however, due to the lack of available data and literature on these long-term measures for children with blunt torso trauma.

The use of single-center cost data might not be generalizable to those in different settings (e.g., facilities that are not Level I trauma centers) or geographic locations. We did explore the possibility of using national cost data, specifically data from the Health Costs and Utilization Project.²⁷ Several limitations, however, prevented our use of these data, including the lack of costs for nonadmitted patients and the unreliability of reported CT imaging. Finally, differences in the medicolegal climate, access to ultrasound, availability to specialty providers, and ability to follow-up discharged ED patients may also limit the generalizability of our results.

CONCLUSIONS

Compared to usual care, implementation of the clinical decision rule in the evaluation of children with blunt torso would reduce hospital costs and abdominal computed tomography imaging with a slight increase in risk of a missed intraabdominal injury needing acute intervention.