Pediatric Trauma Assessment and Management Database: Leveraging Existing Data Systems to Predict Mortality and Functional Status after Pediatric Injury

Abstract

BACKGROUND:

Efforts to improve pediatric trauma outcomes need detailed data, optimally collected at lowest cost, to assess processes of care. We developed a novel database by merging 2 national data systems for 5 pediatric trauma centers to provide benchmarking metrics for mortality and non-mortality outcomes and to assess care provided throughout the care continuum.

STUDY DESIGN:

Trauma registry and Virtual Pediatric Systems, LLC (VPS) from 5 pediatric trauma centers were merged for children younger than 18 years discharged in 2013 from a pediatric ICU after traumatic injury. For inpatient mortality, we compared risk-adjusted models for trauma registry only, VPS only, and a combination of trauma registry and VPS variables (trauma registry VPS). To estimate risk-adjusted functional status, we created a prediction model de novo through +purposeful covariate selection using dichotomized Pediatric Overall Performance Category scale.

RESULTS:

Of 688 children included, 77.3% were discharged from the ICU with good performance or mild overall disability and 17.6% with moderate or severe overall disability or coma. Inpatient mortality was 5.1%. The combined dataset provided the best-performing risk-adjusted model for predicting mortality, as measured by the C-statistic, pseudo-R², and Akaike Information Criterion, when compared with the trauma registry-only model. The final Pediatric Overall Performance Category model demonstrated adequate discrimination (C-statistic = 0.896) and calibration (Hosmer-Lemeshow goodness-of-fit p = 0.65). The probability of poor outcomes varied significantly by site (p < 0.0001).

CONCLUSIONS:

Merging 2 data systems allowed for improved risk-adjusted modeling for mortality and functional status. The merged database allowed for patient evaluation throughout the care continuum on a multiinstitutional level. Merging existing data is feasible, innovative, and has potential to impact care with minimal new resources. (J Am Coll Surg 2017;224:933e944.

Injury is the leading cause of death and disability in children.¹ During the past 3 decades, the American College of Surgeons Committee on Trauma has fostered the creation of trauma systems nationally, and has worked to improve patient care through organized data systems, including the National Trauma Data Bank (NTDB) and the Trauma Quality Improvement Program (TQIP). These allow for hospital benchmarking and they establish a platform for institutional performance improvement.^2–5 These data systems do not capture important non-mortality outcomes, such as functional status or quality of life outcomes. These outcomes are arguably more important in pediatric trauma than adult trauma due to significantly lower mortality after traumatic injury in the pediatric population (11.39 vs 75.97 per 100,000).¹ Morbidity, as measured by functional status impairment and reduced health-related quality of life, is especially important in the evaluation of pediatric trauma care because of the high burden of traumatic brain injury and risk for neurologic sequela leading to lifelong disabilities. Functional status has been studied in small populations of injured children,^6,7 however, it has not been adopted and integrated into national registries and data systems, limiting multi-and intra-institutional assessment, risk-adjusted benchmarking, and national quality-improvement efforts.

Additional data sources exist outside of the American College of Surgeons Committee on Trauma, but they either lack the ability to appropriately risk adjust for injury severity or are limited to specific hospitals or sub-populations. For example, the Pediatric Health Information System is a database with clinical and resource use data, however, it is limited to fewer than 50 children’s hospitals, uses only administrative claims data, and lacks data elements critical for risk adjustment in trauma, including the Abbreviated Injury Scale and Glasgow Coma Scale (GCS) scores.⁸ The Healthcare Cost and Utilization Project is a comprehensive source of hospital data involving hundreds of hospitals nationally. It is also based on administrative data and lacks essential risk-adjustment variables. Finally, and most importantly, the Pediatric Health Information System and the Health-care Cost and Utilization Project family of databases lack non-mortality functional outcomes data, which limits their ability to effectively evaluate the quality of pediatric trauma care.⁹

The Pediatric NSQIP captures detailed process of care data, but also lacks critical trauma-specific variables, including injury type, intent, and mechanism, pre-arrival and arrival physiologic data, and anatomical Injury Severity Scores. Also, NSQIP is procedure-based, and a significant portion of trauma care is nonoperative. Finally, NSQIP is a sample only, and with the heterogeneity of pediatric trauma, arguably fails at capturing the majority of the pediatric trauma population.

The Virtual Pediatric Systems, LLC (VPS) database is a collaborative involving more than 140 hospitals and includes comprehensive data from the pediatric ICU (PICU) phase of care.¹⁰ The VPS database includes detailed physiologic and laboratory data, functional status outcomes at PICU discharge, and 3 validated scores for mortality prediction in critically ill children (Pediatric Index of Mortality [PIM2], Pediatric Risk of Mortality [PRISM3], and the Pediatric Logistic Organ Dysfunction [PELOD]).^11–14 The non-mortality outcomes collected at PICU discharge include the Pediatric Overall Performance Category (POPC) and the Pediatric Cerebral Performance Category, modeled after the traditional Glasgow Outcomes Scale, and have been shown to correlate to more-specific neuropsychological tests and also to post-discharge long-term outcomes.¹⁵

Under the guidance of the Institute for Healthcare Improvement’s Triple Aim—calling for improving the health of populations (injured children) and the patient experience of care (targeting functional status optimization), all while reducing costs—we created the Pediatric Trauma Assessment and Management (PTAM) database. The PTAM is a novel database created through the merging of 2 independent data systems, capitalizing on both systems’ specific strengths, infrastructure, and investment, to improve assessment of care quality provided to critically injured children throughout their hospitalization and to, for the first time, create multi-institutional risk-adjusted models for functional status impairment in the pediatric trauma population.

The primary objectives of this study were to prove the feasibility of a multi-institutional merger of existing data with improved ability to assess care metrics across the care continuum, test the utility of PTAM to improve on risk-adjusted inpatient mortality modeling, and create a risk-adjusted model for functional status at PICU discharge to provide a platform for benchmarking of non-mortality outcomes for critically injured children on a multi-institutional level.

METHODS

Data sources

The PTAM database was created by merging institutional trauma registry and PICU data at 5 pediatric trauma centers for all children meeting the following eligibility criteria: discharged from the PICU during calendar year 2013; younger than 18 years of age at the time of discharge; and at least 1 documented ICD-9-CM code 800–959.9 indicating a traumatic injury; and/or indication of traumatic injury in the VPS participant profile page. All PICU data were obtained through the VPS database, which was submitted to the PTAM data-coordinating center by the national office. All VPS data were collected according to the 2013 VPS data manual.¹⁶ The trauma registry data were collected in compliance with the National Trauma Data Standard 2013 data dictionary¹⁷ and were sent directly from each participating institution to the PTAM data-coordinating center at the University of Washington’s Harborview Injury Prevention and Research Center. The VPS site coordinators and trauma registrars undergo extensive training on data abstraction and coding, and have continuing education for continued qualifications. The VPS sites undergo routine data quality checks on submitted data, as well as quarterly estimates of inter-rater reliability. For calendar year 2013, there were 110 PICUs abstracting and entering data with an aggregate inter-rater reliability of 96.7%. Additionally, for the same time period, the 5 designated study sites’ inter-rater reliability scores ranged from91.5% to 99.5%, with a mean score of 96.2%. Quality checks are built into the VPS software and the National Trauma Data Standard-vetted vendor software at each institution.

Additional data elements were captured independently by trained data abstractors at each site to test the feasibility of capture for additional data elements considered important by the authors. These included select laboratory values at presentation, hemoglobin levels before transfusion, transfusion indications, intracranial pressure monitor use, repeat CT imaging after transfer and during inpatient stay, mechanical deep venous thromboembolism prophylaxis, focused assessment with sonography for trauma use, cervical collar indication and removal timing, and time of initiation of enteral and parenteral nutrition. The additional data elements were collected and managed using Research Electronic Data Capture (REDCap) hosted at the University of Washington.¹⁸ See eTables 1, 2, and 3 for a full list of all data elements captured. A data dictionary was created to mirror the NTDB National Trauma Data Standard 2013 Data Dictionary format,¹⁹ including a data source hierarchy guide, edit checks, and appropriate null values for each data element. An interactive presentation on the additional data elements and their associated data dictionary was disseminated to all sites, including a web-based video training session (contact authors for slide deck and recording).

Data merge

All data were transferred in a deidentified manner, without personal health information. Only a site-based unique identifier was transferred to the data-coordinating center to assist with the merge process. This site-based identifier was entered into each trauma registry data file before transfer, and into all REDCap software profiles. At the data-coordinating center, the site-based identification number and a site number were concatendated to create a unique identifier for each study participant. Data were merged using a 1:1:1 merge strategy within each site, then site data were combined to create the PTAM database including all databases and all sites.

Before verification of participant inclusion, immediate merge was highly successful: 91% matched all 3 databases, with site-specific complete match rates ranging from 84% to 99%. Participants without a VPS file were excluded (n = 35) to focus on the effectiveness of the VPS complement to the trauma registry data, leaving 96% (661 of 692) fully matched. After the merge, 4 participants were noted to be 18 years or older and were excluded. Among the remaining 688, twenty-five children (3.6%) had a VPS record only, 4 children (0.6%) had VPS and REDCap records but no trauma registry record, and 2 children (0.3%) had a trauma registry and VPS record but no REDCap record. The 31 records without complete data were included because these participants still met inclusion criteria.

Pediatric Trauma Assessment and Management Database pilot sites

The 5 pediatric trauma centers involved in the study included 4 verified Level I pediatric trauma centers and 1 Level II pediatric trauma center: Children’s Hospital of Philadelphia (Level I, Philadelphia, PA), Children’s Hospital of Los Angeles (Level I, Los Angeles, CA), Kosair Children’s Hospital (Level I, Louisville, KY), Children’s Hospital of Wisconsin (Level I, Milwaukee, WI), and Akron Children’s Hospital (Level II, Akron, OH). The data-coordinating center was located at the University of Washington’s Harborview Injury Prevention and Research Center in Seattle, WA.

Outcomes data

In-hospital mortality was captured from trauma registry data and was defined as death before hospital discharge or discharge/transfer to hospice care. Functional outcomes measures at ICU discharged were obtained from VPS and included the POPC and the Pediatric Cerebral Performance Category (eTable 4). The POPC was chosen for risk-adjusted model building due to its reflection of overall functional morbidity, and was dichotomized into good/ poor based on earlier stratification of the Glasgow Outcomes Score and Glasgow Outcomes Score-extended in studies of pediatric trauma victims.²⁰ Patients qualified as having “good” functional status if they were discharged with good overall performance (POPC = 1) or mild overall disability (POPC = 2). Patients were defined as having “poor” functional status if they were discharged with moderate or severe overall disability (POPC = 3, 4) or coma or vegetative state (POPC = 5). Additional data collected from VPS data included the mortality prediction tools (PIM2 and PRISM3) and PELOD scores (baseline, delta, and comprehensive).

Missing data

Multiple imputation with chained equations (10 iterations) was used to address missing data for emergency department systolic blood pressure, pulse (hear rate), respiratory rate, and motor GCS modified for paralysis and intubation. Complete-case analysis using only patients with observed values for all included elements in a model was also completed, however, due to risk of introducing bias and reducing sample size and power, imputed analyses are presented.

Statistical analysis

Descriptive statistics were completed with frequencies and column proportions, in addition to means and SD for normally distributed data and medians and interquartile ranges for non-normally distributed data. Nonimputed missing data were included if they accounted for >5% of data for that variable. All variables were accounted for in their original form with the exception of injury type, intent, mechanism, and emergency department GCS. Injury type, intent, and mechanism were categorized based on ICD-9 primary external cause of injury codes (E-codes). Notably, motor vehicle collision included motor vehicle collision with participant as the occupant, motorcycle collisions, motor vehicle collision-pedestrian collisions, motor vehicle collision-bicycle collisions, and motor vehicle collision-not otherwise specified. The emergency department GCS was modified by the emergency department GCS qualifier variable provided by the NTDB to account for paralysis and intubation. Multivariable logistic regression was used to create risk-adjusted models for in-hospital mortality and functional outcomes at PICU discharge.

Mortality modeling

To test the utility of merging data sources to improve on previously established mortality models we compared mortality models with data elements from the trauma registry dataset only (ie modified Pediatric TQIP model), from the VPS dataset only (using PIM2, PRISM3, and PELOD estimates), and from the merged PTAM dataset. Comparisons were made between the full PTAM (trauma registry+VPS model) and the trauma registry- and VPS-only nested models using a Wald test with p value set at 0.05. The 2014 Pediatric TQIP model included more than 12 variables (2014 Benchmark Report²¹), which risked overfitting the study data and/or not converging due to the limited sample size. The modified Pediatric TQIP mortality (trauma registry-only) model was tested iteratively to find the most reduced model with the best performance. This trauma registry-only mortality model included age in years, mechanism of injury, transfer status, emergency department systolic blood pressure, emergency department motor GCS, emergency department heart rate, maximum head Abbreviated Injury Scale, and congenital comorbidity. Our goal was to improve on the trauma registry-only mortality model estimation by improving its accuracy, its efficiency (ie parsimony), or both, with the combined trauma registry+VPS model.

Functional-status modeling

Purposeful covariate selection through multivariable logistic regression was used to create a predictive model de novo for functional status at PICU discharge (dichotomized POPC).²² Dataset-specific comparisons were not completed for POPC model estimation due to the lack of POPC in the trauma registry. Purposeful covariate selection with the PTAM data was completed through an iterative process, including the following core steps:

1. Univariable analyses were completed for variables from all data sources (see eTable 5), keeping variables with a Wald test’s p value <0.25 and/or those that were clinically relevant. Variables with missingness >10% were not included unless clinically relevant.

2. A full multivariable model was then fitted with all covariates identified for inclusion in step 1. All covariates with Wald test <0.10 were removed. The smaller was compared with larger model using the partial likelihood ratio test or the Wald test (for nested model, same sample).

3. Estimated coefficients were compared between the larger model (step 1) and the smaller model (step 2). Variables originally excluded in step 1 were reintroduced until the relative change in estimated coefficients was minimal.

4. Each variable not selected in step 1 was then placed one at a time back into the model obtained at the end of steps 2 and 3. The Wald test or partial likelihood ratio test (if categorical with more than 2 levels) were used to rule in/ out significant covariates and/or precision variables, creating the preliminary main effects model.

5. Categorical variable cut points and fractional polynomials, splines, and log-linear relationship of continuous variables were then tested, creating the main effects model.

6. Clinically important interactions were tested to obtain the preliminary final model.

7. Model adequacy and fit were tested with the diagnostics described here.

All univariate and multivariable regressions were completed controlling for clustering by site.

Variables built for testing from existing data elements included binary variables for age-based hypotension and age-based tachycardia in the emergency department setting and within the first hour of ICU admission, mass-based tachycardia (based on allometric scaling constants using mean mass-based pulse = 208.0 * (VPS_weight ˆ (−0.283))),^23,24 single worst injury score,^25,26 3 worst injury scores,²⁶ and interaction terms between age and emergency department pulse, age and systolic blood pressure (emergency department and first ICU hour), age and head injury, and age and mechanism of injury.

Model performance

To test model performance, we compared each model’s discrimination, calibration/goodness-of-fit, and parsimony. The area under the curve, also known as the C-statistic, was used to determine model discrimination to predict the outcome of interest (eg death or poor functional status). To assess calibration, we used the Hosmer-Lemeshow goodness-of-fit test to compare observed outcomes and predicted outcomes across deciles of risk. Calibration refers to the agreement between observations and predictions. Because the Hosmer-Lemeshow test is often criticized for the risk-grouping approach,²² we also used the McFadden’s Pseudo-R² to supplement goodness-of-fit evaluation.²⁷ To assess parsimony, or efficiency, we used the Akaike Information Criterion (AIC), which assesses model fit and penalizes for complexity of the model. To further test the utility of the merged dataset and the adequacy of purposeful covariate selection for model building to estimate functional status at PICU discharge, we used backwards-stepwise regression at the0.05 level to determine if, via another statistical approach, important covariates came from all 3 datasets.

Ethics

Institutional Review Board approval was obtained from all institutions who contributed data and from the University of Washington as the data-coordinating center.

RESULTS

A total of 688 children were included in the study. Demographic and injury characteristics are presented in Table 1. Two-thirds of the children were male and the mean age was 7.2 (SD 5.9) years; however, 21.7% of children included were younger than 1 year of age. Eighty-four percent of injuries were unintentional, 14% were assaults, 1% were self-inflicted, and 1% were undetermined. Blunt injury (77%) was the most common injury mechanism, with falls (32%) and motor vehicle collisions (25%) predominating. Most notably, 58% of study participants had a maximum head Abbreviated Injury Scale ≥3 and 35.6% had an Injury Severity Score >15, indicating severe injury. A substantial proportion of the study population qualified as an “assault” victim based on the E-code matrix for intent associated with the injury. Half of those children were younger than 24 months old, highlighting the potential for abuse in this population. With the combined data sources, physiologic, laboratory, and process of care data were available following each child across the care continuum, from the prehospital setting via emergency medical services run sheets, through the stabilization phase of care in the emergency department, and the resuscitation phase in the PICU (Fig. 1; eTables 1, 2, and 3 for full list of data elements).

Table 1.

Demographic and Injury Characteristics of the Pediatric Trauma Assessment and Management Database Study Population (n = 688)

Characteristic	Data
Age
Younger than 1 y, n (%)	143 (21.7)
1 to 4 y, n (%)	155 (23.5)
5 to 12 y, n (%)	190 (28.8)
13 to 17 y, n (%)	171 (26.0)
Median age, y (IQR)	6 (1.5–13)
Sex, male, n (%)	446 (67.7)
Race/ethnicity, n (%)
White	353 (51.3)
Black/African American	142 (20.6)
Hispanic	47 (6.8)
Asian	15 (2.2)
Other	79 (11.5)
Missing	52 (7.6)
Payment method, n (%)
Medicaid/Medicare	313 (47.5)
Commercial insurance	229 (34.8)
Self-pay	27 (4.1)
Other	90 (13.7)
Injury type, n (%)
Blunt	508 (77.1)
Penetrating	26 (4.0)
Burn	17 (2.6)
Other/not otherwise specified	108 (16.4)
Injury mechanism, n (%)
Fall	211 (32.0)
Motor vehicle collision	163 (24.7)
Struck by/against (object)	68 (10.3)
Pedestrian, cyclist, other transport	66 (10.0)
Firearm	22 (3.3)
Cut/pierce	4 (0.6)
Burn	17 (2.6)
Other/not otherwise specified	108 (16.4)
Injury Severity Score
0 to 15, n (%)	406 (59.0)
16 to 25, n (%)	153 (22.2)
>25, n (%)	92 (13.4)
Missing, n (%)	37 (5.4)
Median Injury Severity Score (IQR)	10 (9–17)
Head Abbreviated Injury Scale score, n (%)
No head injury	166 (25.2)
1 or 2	104 (15.8)
3	224 (34.0)
4 or 5	160 (24.3)
Not otherwise specified	5 (0.8)
Emergency department GCS motor score, n (%)
1/paralyzed	62 (9.9)
1/valid	24 (3.9)
2 to 3	9 (1.5)
4 to 5	68 (10.8)
GCS 6/obeys commands	462 (73.9)
Transferred from another facility, n (%)	392 (57.1)
Pre-injury Pediatric Overall Performance Category, n (%)
Good overall performance (1)	614 (89.2)
Mild overall disability (2)	59 (8.6)
Moderate overall disability (3)	11 (1.6)
Severe overall disability (4)	4 (0.6)
Pre-Injury Pediatric Cerebral Performance Category, n (%)
Normal (1)	652 (94.8)
Mild disability (2)	28 (4.1)
Moderate disability (3)	6 (0.9)
Severe disability (4)	2 (0.3)

GCS, Glasgow Coma Scale; IQR, interquartile range.

Figure 1.

Select data elements available along multiple phases of the care continuum. ALT, alanine transaminase; AST, aspartate transaminase; ED, emergency department; GCS, Glasgow Coma Score; PICU, pediatric ICU; PT/PTT, prothrombin time/partial thromboplastin time.

Outcomes

The majority of children were healthy before their injury, with 89% having good overall performance (Table 1). The POPC and Pediatric Cerebral Performance Category scores at PICU discharge, along with the mean difference between baseline scores and discharge scores, are presented in Table 2. Fewer than half of children (42%; n = 289) had no change in their POPC score from base line to PICU discharge. Forty percent (n = 272) had a change in POPC by 1 category (eg from good overall performance to mild overall disability or from mild to moderate overall disability), and 10% (n = 70) had a change by 2, and 5% (n = 33) by 4 or 5 categories. The PIM2, PRISM3, PELOD, disposition, and length of stay summary data are also presented.

Table 2.

Pediatric Trauma Assessment and Management Database Outcomes

Outcomes measure	Data
Inpatient mortality, n (%)	36 (5.2)
POPC at PICU discharge
Good overall performance (1), n (%)	232 (34.0)
Mild overall disability (2), n (%)	296 (43.3)
Moderate overall disability (3), n (%)	94 (13.8)
Severe overall disability (4), n (%)	24 (3.5)
Coma or vegetative state (5), n (%)	2 (0.3)
Brain death (6), n (%)	35 (5.1)
Change in POPC, mean (SD)	−0.95 (1.2)
PCPC at PICU discharge
Normal (1), n (%)	487 (71.3)
Mild disability (2), n (%)	101 (14.8)
Moderate disability (3), n (%)	38 (5.6)
Severe disability (4), n (%)	20 (2.9)
Coma or vegetative state (5), n (%)	2 (0.3)
Brain death (6), n (%)	35 (5.1)
Change in PCPC, mean (SD)	−0.55 (1.2)
Pediatric Index of Mortality score, n (%)
0% to 10% risk of mortality	632 (91.9)
11% to 30% risk of mortality	21 (3.1)
31% to 50% risk of mortality	8 (1.2)
51% to 70% risk of mortality	7 (1.0)
71% to 90% risk of mortality	11 (1.6)
91% to 100% risk of mortality	9 (1.3)
Pediatric Risk of Mortality score, n (%)*
0% to 10% risk of mortality	607 (93.0)
11% to 30% risk of mortality	15 (2.3)
31% to 50% risk of mortality	4 (0.6)
51% to 70% risk of mortality	3 (0.3)
71% to 90% risk of mortality	10 (1.5)
91% to 100% risk of mortality	15 (2.3)
Pediatric Logistic Organ Dysfunction Scale, n	(%)
0% to 10% risk of mortality	612 (89.0)
11% to 30% risk of mortality	40 (5.8)
31% to 50% risk of mortality	0 (0)
51% to 70% risk of mortality	0 (0)
71% to 90% risk of mortality	17 (2.5)
91% to 100% risk of mortality	19 (2.8)
Disposition (patients discharged alive), n (%)
Home without service	544 (87.0)
Rehab facility or home with services	69 (11.0)
Transferred to another acute care facility	13 (2.1)
Length of stay, d, mean (SD); median (IQR)
Trauma registry hospital LOS	7.3 (10.9); 4 (2–8)
Virtual Pediatric Systems, LLC  PICU physical LOS	3.1 (5.0); 1.4 (0.7–2.9)

^*Thirty-five (5.1%) patients met Pediatric Risk of Mortality exclusion criteria. IQR, interquartile range; LOS, length of stay; PICU, pediatric ICU; PCPC, Pediatric Cerebral Performance Category; POPC, Pediatric Overall Performance Category.

Mortality model performance

A total of 36 patients died from their injuries (5.2%). All risk-adjusted mortality models performed well; however, the PTAM (trauma registry+VPS) model had improved discrimination compared with the trauma registry-only model (Fig. 2; p < 0.001). The McFadden’s pseudo R² value was highest in the PTAM model, and displayed less goodness-of-fit in the VPS-only model and the lowest in the trauma registry-only model (0.754 trauma registry+VPS vs 0.673 VPS-only vs 0.4577 trauma registry-only). The AIC was lowest for the VPS-only model, highlighting its single parameter estimation (96.3). The PTAM model had a better AIC compared with the trauma registry-only model (99.6 vs 169.4).

Figure 2.

Receiver operating characteristic by data source. Mortality model comparisons for discrimination (area under the curve [AUC]), goodness-of-fit (McFadden’s R²), and parsimony (Akaike Information Criterion [AIC]); n = 583. Chi-square test for equality of AUC estimates between the trauma registry (TR)+ Virtual Pediatric Systems, LLC (VPS) model and the TR-only model is statistically significant with p < 0.001. *McFadden’s R², a measure of goodness-of-fit.^{27 †}TR-only covariates: age, mechanism of injury, transfer status, emergency department systolic blood pressure, Glasgow Coma Scale score, maximum head Abbreviated Injury Scale score, and congenital comorbidity. ^‡VPS-only model: performance of the Pediatric Index of Mortality 2 (PIM2).

Functional status model performance

The final model for functional status at ICU discharge for severely injured children contained variables from all 3 data sources (Table 3). Backwards-stepwise regression also resulted in covariates from each of the 3 available data sources (Table 3). The final model of 14 variables demonstrated adequate discrimination (C-statistic = 0.896; Fig. 3) and calibration (Hosmer-Lemeshow goodness-of-fit p = 0.65; Fig. 4). Discrimination and calibration plots for the complete case analyses are also available (eFigure 1). Assessing the site as a main effect, the probability of poor outcomes varied significantly (Wald test p < 0.0001). Observed vs expected ratios for poor outcomes by site are presented in Figure 5.

Table 3.

Final Multivariable Logistic Regression Model for Risk-Adjusted Functional Status at ICU Discharge (n = 504)

Source	Purposeful covariate selection	Backwards stepwise regression (p < 0.05)
TR	Age	—
TR	Race/ethnicity	Race/ethnicity
TR	Payment	—
TR	ED motor GCS	ED motor GCS
TR	Mechanism of injury	Mechanism of injury
TR	Maximum head Abbreviated Injury Scale	—
TR	Single worst injury	Single worst injury
TR	ISS	ISS
VPS	Age-based hypotension first hour in ICU	Systolic blood pressure (first 12 hours in ICU)
VPS	Baseline POPC	Baseline POPC
VPS	Mas-based tachycardia, binary	—
VPS	Non-head trauma*	Non-head trauma
VPS	Recovery from surgery	—
		Baseline Pediatric Logistic Organ Dysfunction scale
		Cardiac massage†
PTAM	Hematuria	Hematuria

^*Might or might not also have concurrent head injury.

^†History of cardiac massage or cardiac arrest immediately before or during admission (ie as a result of injury).

ED, emergency department; GCS, Glasgow Coma Scale; ISS, Injury Severity Score; POPC, Pediatric Overall Performance Category; PTAM, Pediatric Trauma Assessment and Management Database; TR, trauma registry, VPS, Virtual Pediatric Systems, LLC.

Figure 3.

Discrimination plot for functional status model prediction (n = 504). Area under receiver operating characteristic curve = 0.8958.

Figure 4.

Calibration plot for functional status model prediction (n = 504). POPC, Pediatric Overall Performance Category.

Figure 5.

Observed vs expected probability of poor functional outcomes at ICU discharge by site (n = 504).

DISCUSSION

The Pediatric Trauma Assessment and Management database was created by merging 2 national independent data systems across multiple institutions. This feasibility study shows that this dataset provides a platform to improve risk-adjusted modeling for in-hospital mortality and to develop non-mortality outcomes prediction models for critically injured children.

The PTAM mortality model outperformed both the trauma registry-only and VPS-only models. For the well-validated predictors of mortality in the nontrauma PICU population (ie VPS PIM2 and PRISM3 scores^13,14), we improved both external and face validity in the trauma population by controlling for essential trauma registry-specific covariates, including mechanism of injury, GCS, maximum head Abbreviated Injury Scale, and Injury Severity Score. Compared with the trauma registry-only mortality model, we improved both its accuracy of discrimination (improved area under the curve) and its efficiency (improved AIC). The AIC in the PTAM model was also comparable with the AIC in the VPS model, despite the model’s relative complexity. A few noteworthy components of the model include hematuria and age-based hypotension in the first hour in the ICU. Although hematuria is often evaluated, it is rarely, if ever, available in large datasets. Its inclusion here highlights the need to look further into hematuria’s role in predicting poor functional status as an end point, particularly as a predictor of other injury. Age-based hypotension in the first hour of the ICU emphasizes the impact of persistent hypotension beyond initial stabilization in the emergency department phase of care.

The American College of Surgeons Committee on Trauma created TQIP to “improve quality of trauma care through robust risk-adjusted benchmarking of trauma centers,”²⁸ and this study provided evidence that benchmarking is possible for functional status after severe injury, arguably one of the most important primary outcomes in pediatric trauma. The model created in our study for functional status at discharge relied on data elements from all 3 data sources, highlighting the importance of information beyond what is normally provided in available datasets. Historically, the only covariates available for NTDB analyses are initial emergency medical services and emergency department data and ICD-9 codes for procedures and diagnoses. There is no ability to risk adjust for baseline functional status, laboratory parameters, or physiologic data after initial stabilization.

Studies have previously evaluated functional status as measured by the POPC at ICU discharge in the general PICU population,^29,30 and there are mixed opinions about the utility of the POPC in trauma registries. Pollack and colleagues³⁰ reported the POPC was less sensitive than Functional Status Scale to measure outcomes after critical illness. Fiser and colleagues³¹ found the POPC at ICU discharge to correlate with more detailed instruments later in care, including to the Bayley Scales of Infant Development and Stanford-Binet Intelligence Quotients at hospital discharge, and to the Vineland Adaptive Behavior Scales at 1 and 6 months post discharge. An outcomes assessment instrument must be valid, responsive, reliable, and efficient,³² and although the best instrument is debated in the literature,^32–34 it is clear that trauma registries should capture information beyond mortality. Although the POPC might not be the ideal instrument, this study highlighted both its utility and the fact that it is immediately available for hundreds of injured children throughout the nation, with no additional data abstraction or cost.

Finally, although there is evidence that quality-focused databases like the NSQIP have used risk-adjusted benchmarking methodology to improve outcomes,³⁵ there is administrative resistance due to the cost of participation, which can be prohibitively expensive for many centers. The goal of this work is to leverage existing data—and existing investments in those data sources—to improve care for critically injured children, optimize their outcomes, and provide a framework for multi-institutional benchmarking for functional status and processes of care throughout the care continuum.

Limitations

Important limitations should be noted. Although the final model for functional status at ICU discharge performed very well, this study is limited by its sample size and subsequently compromised power. There might exist variability by site in type of patient admitted to the ICU based on differences in indication and threshold for ICU admission, and in resources that affect ICU use (eg step-down unit availability). There exist critiques of the analytic strategies used (ie risk of unreliable Hosmer-Lemeshow estimates with small groups, imperfection of pseudo-R² techniques, pros and cons of logistic compared with hierarchical linear or general linearized modeling and/ or bootstrapping).^22,36,37 Although all critiques were valid, the methods were chosen to optimize both statistical accuracy and face validity. We chose to test parameters throughout the care continuum when modeling functional status to fully assess what combination of covariates created the best-fitting model and to test the hypothesis that the combined data sources would portend improved model accuracy and fit. Depending on the goal of the model, however, variables along the care continuum can introduce confounding if on the “causal pathway.” The merged database provides the ability to deliver models unique to various goals.

Although this database is restricted to critically injured children, the mortality and morbidity burden from pediatric trauma is predominantly among the most critically ill children admitted to the ICU. There are limitations to the POPC; it is not highly sensitive to small changes in functional status and it is captured at ICU discharge only. Although it correlates to specific outcomes at 1 and 6 months post discharge, capturing functional status at hospital discharge or at 30 days, 6 months, and/or 12 months after discharge might be more ideal.

Next steps

Efforts to improve pediatric trauma outcomes depend on the availability of detailed data to assess care and outcomes, optimally at the lowest cost. This feasibility study demonstrated the ability of the PTAM database to leverage already existing data sources—and investments in those data sources. The PTAM database allows intensivists to risk adjust with mechanism and severity of injury data, and allows trauma researchers to evaluate non-mortality outcomes, and detailed laboratory and process of care data. Most importantly, PTAM has the capacity to “scale up” with new minimal resource investments, increasing its potential scope. Future success will be measured by the ability to sustain and expand collaboration to additional sites. Our vision is an electronic-linked database making merged data available to collaborators for investigation, permitting within- and between-hospital comparisons, and allowing identification of quality measures to improve care.

CONCLUSIONS

To mitigate burden from pediatric injury, we must focus on quality care optimization that minimizes death and maximizes functional status and quality of life after nonfatal injury. Merging 2 independent national data sources allowed us to assess variability in care provided to critically injured children throughout hospitalization, improve on existing risk-adjusted modeling for inpatient mortality, and create risk-adjusted non-mortality outcomes prediction models for critically injured children. A collaborative approach between data entities, the American College of Surgeons Committee on Trauma and VPS, is vital to sustainability. To truly impact the lives of injured children, it is essential that we work to translate data into improvements in patient care. The PTAM database potentiates the ability to assess and measure change in patient care and outcomes beyond each dataset alone.

Abbreviations and Acronyms

AIC

Akaike Information Criterion

GCS

Glasgow Coma Scale

NTDB

National Trauma Data Bank

PELOD

Pediatric Logistic Organ Dysfunction

PICU

pediatric ICU

PIM2

Pediatric Index of Mortality

POPC

Pediatric Overall Performance Category

PRISM3

Pediatric Risk of Mortality

PTAM

Pediatric Trauma Assessment and Management Database

REDCap

Research Electronic Data Capture

TQIP

Trauma Quality Improvement Program

VPS

Virtual Pediatric Systems, LLC