Collider bias in trauma comparative effectiveness research: The stratification blues for systematic reviews

Abstract

Background

Collider bias, or stratifying data by a covariate consequence rather than cause (confounder) of treatment and outcome, plagues randomized and observational trauma research. Of the seven trials of prehospital hypertonic saline in dextran (HSD) that have been evaluated in systematic reviews, none found an overall between-group difference in survival, but four reported significant subgroup effects. We hypothesized that an avoidable type of collider bias often introduced inadvertently into trauma comparative effectiveness research could explain the incongruous findings.

Methods

The two most recent trials, a single-site pilot and a multi-site pivotal study, provided data for a secondary analysis to more closely examine the potential for collider bias. The two trials had followed the a priori statistical analysis plan to subgroup patients by a post-randomization covariate and well-established surrogate for bleeding severity, massive transfusion (MT), ≥10 units of red blood cells within 24 hours of admission. Despite favorable HSD effects in the MT subgroup, opposite effects in the non-transfused subgroup halted the pivotal trial early. In addition to analyzing the data from the two trials, we constructed causal diagrams and performed a meta-analysis of the results from all seven trials to assess the extent to which collider bias could explain null overall effects with subgroup heterogeneity.

Results

As in previous trials, HSD induced significantly greater increases in systolic blood pressure (SBP) from prehospital to admission than control crystalloid (p=0.003). Proportionately more HSD than control decedents accrued in the non-transfused subgroup, but with paradoxically longer survival. Despite different study populations and a span of over 20 years across the seven trials, the reported mortality effects were consistently null, summary RR=0.99 (p=0.864, homogeneity p=0.709).

Conclusions

HSD delayed blood transfusion by modifying standard triggers like SBP with no detectable effect on survival. The reported heterogeneous HSD effects in subgroups can be explained by collider bias that trauma researchers can avoid by improved covariate selection and data capture strategies.

Introduction

Well recognized now, survival bias, or the failure to exclude antecedent survival time that the intervention under study could not possibly affect, has cast doubt on many published reports of trauma comparative effectiveness research.^1–7 While randomization controls survival and other types of bias,⁸ insidious collider bias continues to plague trauma trials as well as observational studies. Collider bias is a type of selection bias that is often introduced inadvertently into trauma resuscitation research by restriction, stratification or adjustment on a covariate that is a consequence (collider) rather than a cause (confounder) of both the treatment and outcome of interest. A familiar example of collider bias is Berkson’s bias or fallacy.⁹ In his 1946 landmark paper,¹⁰ renowned physician and statistician, Joseph Berkson, recounted how physicians hypothesized that cholecystic disease caused or aggravated diabetes after observing a correlation between the two diseases in hospitalized patients. In his hypothetical data analysis, Berkson demonstrated that, although the two diseases were truly not associated with one another in the total population at risk, a spurious positive association arose in an analysis restricted to hospitalized patients. The corresponding analysis for the remaining non-hospitalized population produced an equal and opposite spurious negative association. These puzzling findings were the result of subdividing the total population on an invalid covariate, hospital admission, a consequence (outcome) of both cholecystic disease and diabetes rather than a risk factor or contributing cause (e.g., age or dietary history).

Collider bias is common in trauma resuscitation research due to the difficulty of ascertaining the extent and severity of injury and hemorrhage before initiating the intervention of interest, and the use of poor proxies often not established until hours later. Of the seven trials of prehospital hypertonic saline in dextran (HSD) for hypovolemic trauma patients^11–17 evaluated in systematic reviews,^18–26 none found an overall between-group difference in survival, but four reported significant subgroup effects.^11,12,14,16 Reconciling the promising subgroup results with consistently null overall findings has been challenging.^18–26 We hypothesized that a preventable type of collider bias could explain the incongruities.

Methods

Source Data

The two most recent HSD trials, a single-site pilot¹¹ and a multi-site pivotal study,¹² provided data for a secondary analysis to assess collider bias. The trials were funded by the National Heart Lung and Blood Institute of the National Institutes of Health and approved by the U.S. Food and Drug Administration.^11,12,27 The trials compared the effects of prehospital infusion of an initial 250 ml bolus of crystalloid solution containing either HSD (7.5% NaCl with 6% dextran-70) or the standard-of-care, i.e., lactated Ringer’s in the pilot;¹¹ normal saline (0.9%) in the pivotal trial.¹² The trials had similar designs with a few exceptions.^11,12,27 The pivotal trial included a third, hypertonic saline without dextran.¹² We excluded patients in that arm (n=256) due to the inability to test for homogeneity across the two trials and hypertonic saline solutions. The pivotal trial¹² (n=596) included patients with penetrating or blunt injuries, and patients with a prehospital systolic blood pressure (SBP) ≥70 mmHg were required to have a prehospital heart rate ≥108 beats per minute to enroll only hemorrhagic shock patients. The pilot study¹¹ (n=209) enrolled blunt trauma patients only, required prehospital SBP≤90 mm Hg, and defined the outcome as 28 day survival free of acute respiratory distress syndrome. De-identified data sets and data dictionaries were acquired and verified. Our Institution’s Committee for the Protection of Human Subjects approved the project.

Criteria for a Valid Covariate

The prevailing guidance for comparative effectiveness research asserts that, regardless of their intended purpose as a confounder or effect modifier, valid covariates or stratification variables must be either defined with known patient status before treatment begins or known to be unaffected by the treatment.^28,29 Variables that either change in response to treatment or develop only post-randomization are not considered valid covariates. We applied these criteria and constructed causal diagrams to evaluate whether massive transfusion (MT), >=10 units of red blood cell (RBC) transfusions with 24 hours of admission to the emergency department (ED), status fulfilled the criteria for a valid covariate

Statistical Analyses

Following the a priori statistical analysis plan, patient data in the original trials^11,12 were stratified by the post-randomization covariate and well-established surrogate for bleeding severity, MT.^1,3,6,30 Because randomization was expected to balance potential confounders across study arms, the data were stratified by MT to reveal HSD modifying effects, or whether the MT subgroup benefited more than the alternate subgroups (receiving 1–9 and 0 RBCs, respectively).^11,12 Despite favorable HSD effects in the MT subgroup, opposite effects in the 0-RBC subgroup halted the pivotal trial early. To assess whether collider bias could explain null overall, but significant MT subgroup effects, we conducted secondary analyses of standardized data elements combined from the two trials^11,12 and performed a meta-analysis summarizing results from all seven trials1.^11–17 Univariate and multivariable analyses included chi square tests, the Breslow-Day test for homogeneity of stratum-specific effect estimates, logistic regression and Cox proportional hazards modeling for HSD mortality outcomes, t tests for 24-hour RBC counts, linear regression for HSD-induced changes from prehospital to admission SBP, adjusted for prehospital SBP and Wilcoxon rank sum tests for differences in median survival times. Covariates for the mortality analyses included trial identity (pilot¹¹ or pivotal¹²) and 24-hour RBC count. Our meta-analysis of the 1,695 patients in 7 trials used the fixed effects confidence interval method.^11–17 Hypothesis tests were two-sided and p values less than 0.05 were considered statistically significant. Data were analyzed with SAS version 9.3 (SAS Institute, Cary NC) and Stata release 12 (StataCorp, College Station TX).

Results

To fulfill the criteria for a valid covariate, a hypothetical early indicator of bleeding severity would have been ascertained before the infusion of prehospital crystalloid (Figure 1a). As a valid covariate, the hypothetical early indicator of bleeding severity could have either confounding or modifying effects that appropriate stratification or modeling strategies would reveal.^28,29 However, because resuscitation with hypertonic saline is known to influence the indications for blood transfusion in trauma patients (e.g., increasing SBP),^13–26 and the receipt of one or more RBC transfusions depends on the duration of injury survival (Figure 1b), the 24 hour sum count of RBC transfusions used for stratification in the HSD trials fulfills the definition for an invalid collider covariate.^{31, 32} Causal diagramming using a directed acyclic graph or DAG (Figure 1c)³³ shows the irreversible path between prehospital HSD infusion and the 24 hour sum of RBCs precluding meaningful interpretation of mortality analyses stratified by RBC category. The true association between HSD and mortality is confounded by the spurious association introduced by the causal effects of early mortality and HSD on the 24 hour sums of RBC transfusions, or reverse causation.³³ The well-known peak mortality rates in the minutes to hours following injury occurrence^{34, 35} limit hemorrhaging patient’s opportunity to receive RBC transfusions.

Figure 1. Causal directions for a valid covariate (1a), a collider covariate (1b) and their causal pathways in a directed acyclic graph or DAG (1c) of prehospital HSD trials.

1a. Valid covariate: Causal arrows point from valid covariate (pre-intervention injury and bleeding severity status) to both intervention (prehospital HSD or control crystalloid) and outcome (24 hour injury survival). 1b. Collider bias: Causal arrows point from both intervention and outcome to collider (surrogate severity indicator, 24 hour sum of RBC transfusions). 1c. DAG: Solid lines depict causal paths. A valid covariate causes both intervention and outcome. Stratifying on a valid covariate would reveal the true causal path between the intervention and outcome (the '?' represents the unknown causal relationship, if any, that the trial is intended to determine). A collider is a consequence of both intervention and outcome. The dashed line reflects the spurious association between the intervention and outcome introduced by stratification on a collider (invalid) covariate. The direction of the causal arrow at the bottom of the dashed line reflects misinterpretation of the spurious association as a causal effect of treatment on outcome.

In the combined trial data (n=805),^11,12 overall 30-day mortality was similar in the HSD and control groups (Table 1). Across the arms of the two trials, randomization appeared to balance potential confounders, however, in the pilot trial, a larger proportion of HSD than control patients were transported by helicopter and intubated.¹¹ The overall mortality relative risk (RR for HSD vs. control) was somewhat higher (RR=1.31, 95% CI=0.82–2.09) than that of the pivotal trial (RR=1.01, 95% CI=0.77–1.35), though the two estimates did not differ significantly (p value=0.363). HSD and control crystalloid had similar effects on mortality (Table 1), across three time intervals following enrollment (homogeneity p value=0.339). Among HSD and control decedents, there were no differences in times to death overall or when subdivided into time intervals after enrollment (Table 1). Mortality hazard ratios estimated from Cox proportional hazards models (data not shown) were very similar to the RRs estimated from the logistic regression models. There were no between-group differences in the 24 hour sum count of RBCs transfused (Table 1).

Table 1.

Overall between-group differences in outcome (before stratification) using combined data set.

Outcome	HSD Group	Control Group	Relative Risk or difference	p value	95% Confidence Interval
Overall 30 day Mortality	89/330=0.27	118/475=0.25	RR = 1.09	0.497	RR = 0.86 – 1.37
Mortality within 6 hours	48/330=0.14	73/475=0.15	RR = 0.95	0.748	RR = 0.68 – 1.32
Mortality between 6 – 24 hours	12/282=0.04	17/402=0.04	RR = 1.01	0.986	RR = 0.49 – 2.07
Mortality between 1 – 30 days	29/270=0.11	28/385=0.07	RR = 1.48	0.121	RR = 0.90 – 2.42
Mean # of days to death overall (SD)	2.10 (4.57)	1.95 (4.33)	Diff = 0.15	0.814	Diff = −1.37 – 1.08
Mean # of hours to death between 0–6	1.81 (1.17)	2.25 (1.46)	Diff = −0.446	0.080	Diff = −0.05 – 0.94
Mean # of hours to death between 6–24	6.77 (5.44)	7.21 (6.34)	Diff = −0.44	0.847	Diff = −4.19 – 5.08
Mean # of days to death between 1–30	5.15 (6.35)	6.65 (6.07)	Diff = −1.50	0.365	Diff = −1.80 – 4.80
Mean 24 hour sum RBC count (SD)	5.03 (8.09)	5.21 (8.74)	Diff = −0.18	0.774	Diff = −1.02 – 1.37
% in 0 RBC subgroup	136/329=41.3%	189/473=40.0%	Diff = 1.3%	-	-
% in 1–9 RBC subgroup	133/329=40.4%	208/473=44.0%	Diff = −3.6%	-	-
% in >=10 RBC subgroup	60/329=18.2%	76/473=16.1%	Diff = 2.1%	0.548*	-
Mean SBP change Δ, prehospital to ED admission - overall	Pre=64.11 N=318 Δ =51.79 (44.06)	Pre=62.09 N=458 44.62 (46.37)	Adjusted† Diff = 8.61	0.003	Adjusted† Diff = 2.93 – 14.29
Died within 30 days	Pre=46.35 N=83 Δ =44.85 (59.88)	Pre=52.09 N=109 27.76 (33.12)	A†Diff=13.52	0.089	A Diff = −2.00 – 29.04

^*p value for overall between-group difference in proportionate distribution across 3 RBC subgroups

^†Adjusted for pre-hospital SBP. A Diff = Adjusted difference.

As observed in previous trials,^13–26 among patients surviving to ED admission, prehospital to admission SBP increased significantly more for HSD than control patients in the combined data set (Table 1). Given that low or declining SBP can be one of the indications for transfusion, we explored the hypothesis that the greater SBP increases among HSD patients (Table 1) reflected HSD-induced physiologic changes that may have delayed the initial RBC transfusion. Because the trials had not captured data on the timing of RBC transfusions, we sought indirect evidence of delayed initial transfusion in the HSD group using data available on changes in SBP and survival time. The difference between HSD and control group SBP changes (adjusted for prehospital SBP) was greater for decedents than survivors (Table 1) and the differences tended to increase with decreasing survival times (data not shown).

Stratification by the 24-hour RBC count confirmed the significantly heterogeneous overall mortality differences across RBC subgroups previously reported for both trials^11,12 (Table 2). The most extreme heterogeneity occurred in the earliest time interval (Table 2). However, patients dying during transport to the ED had no chance for RBC transfusion, and their mortality did not differ between study groups (4/330 = 0.012, HSD, 6/475 = 0.013, controls, p=0.949). The subgroup of patients classified as receiving 0 RBCs included patients who died early, before any RBCs were transfused (23 in the HSD group; 21 in the control group) and those who survived 24 hours without transfusion (113 in the HSD group; 168 in the control group, Table 2). The 7 prehospital HSD trials^11–17 included in the meta-analysis reported consistent in-hospital mortality effects (HSD vs. control RRs ranged from 0.65 to 1.35, homogeneity test p value=0.709). The overall summary RR was 0.99 (95% CI = 0.85–1.15, p=0.864, see supplementary material).

Table 2.

Group mortality and relative risk, by time of death and 24 hour sum RBC count

Outcome	HSD Group	Control Group	Relative Risk	p value	95% Confidence Interval
Overall 30 day Mortality				0.023*
0-RBC Subgroup	30/136=0.22	22/189=0.12	1.89	0.012	1.14 – 3.14
1–9 RBC Subgroup	29/133=0.22	51/208=0.24	0.89	0.564	0.60 – 1.33
>=10 RBC Subgroup	30/60=0.50	44/76=0.58	0.86	0.359	0.63 – 1.19
Mortality within 6 hours				0.065*
0-RBC Subgroup	19/136=0.14	17/189=0.09	1.55	0.159	0.84 – 2.88
1–9 RBC Subgroup	17/133=0.13	28/208=0.13	0.95	0.856	0.54 – 1.67
>=10 RBC Subgroup	12/60=0.20	27/76=0.355	0.56	0.047	0.31 – 1.01
Mortality between 6 – 24 hours				0.616*
0-RBC Subgroup	4/117=0.03	4/172=0.02	1.47	0.578	0.37 – 5.76
1–9 RBC Subgroup	2/116=0.02	6/180=0.03	0.52	0.405	0.11 – 2.52
>=10 RBC Subgroup	6/48=0.13	7/49=0.14	0.88	0.796	0.32 – 2.41
Mortality between 1 –30 days				0.085*
0-RBC Subgroup	7/113=0.06	1/168=0.006	10.41	0.006	1.30 – 83.44
1–9 RBC Subgroup	10/114=0.09	17/174=0.10	0.90	0.776	0.43 – 1.89
>=10 RBC Subgroup	12/42=0.29	10/42=0.24	1.20	0.620	0.58 – 2.47

^*p value for homogeneity test across the three RBC subgroups

Figure 2 is a theoretical timeline to illustrate how stratifying the 24-hour mortality data by patients’ 24-hour RBC count could have introduced collider bias. If deaths actually occurred at the same rate in each study group, but HSD-induced physiologic changes delayed the initial RBC transfusion,^{18, 22} then the cut-off point for HSD decedents’ maximum pre-transfusion hours to death would be shifted to the right. The right shift would classify proportionately more HSD than control deaths as pre-transfusion and fewer HSD than control deaths as MT (as observed in Table 2). Paradoxically, HSD decedents in the 0-RBC subgroup would have extended median hours to death relative to controls (Figure 2). In our analysis of the combined data set, although the median hours to death did not differ between the HSD and control groups overall (p=0.675), in the 0-RBC subgroup the HSD group had more median hours to death than controls (p=0.048) as predicted by Figure 2.

Figure 2. Theoretical effect of HSD vs. control crystalloid (Cntrl) on delayed transfusion with no effect on survival.

If HSD-induced physiologic changes and delayed the timing of the initial RBC transfusion (arrows) with no effect on the occurrence or timing of deaths (thunderbolts), then more HSD than control decedents would be classified in the 0-RBC subgroup (left of arrow) and fewer in the alternate subgroups (right of arrow), especially the MT subgroup requiring more time to receive ≥10 transfusions.

Discussion

The results from our causal diagrams, secondary analysis^11,12 and meta-analysis across seven independent study populations with different distributions of injury and bleeding severity^11–17 emphasize that, regardless of the HSD-related transfusion delays, deaths among HSD patients would likely have occurred at roughly the same points in time had the HSD patients been randomized to standard-of-care crystalloid instead. The protective effect of HSD on survival observed in both trials’ MT subgroups was interpreted in the pilot trial report¹¹ as evidence that prehospital HSD may reduce mortality in the highest risk patients. In contrast, the report of the pivotal trial underscored the adverse HSD effect in the presumed lowest-risk patients receiving 0 RBCs to explain the trial’s early termination.¹² Our findings do not support these interpretations, but in no way detract from the decision to terminate the pivotal trial early, given the planned interim trial analyses uncovered potentially adverse HSD effects (e.g., delayed RBC transfusion). They do, however, urge caution in trauma research design and selection of covariates. Assigning patients at the opposite extremes of bleeding severity to the same 0-RBC subgroup reveals how the 24 hour RBC classification actually defeats the purpose of stratification, namely, to form more homogeneous patient subgroups.¹

The pivotal trial investigators and expert reviewers considered an HSD-induced delay in initial RBC transfusion as one possible explanation for the adverse findings in the 0-RBC subgroup.^{12, 22} However, the possibility that HSD could have delayed transfusion without affecting survival, and that the heterogeneous HSD effects on mortality across the different RBC subgroups were artifacts, due solely to bias, was not discussed in the trial reports,^11,12 in the accompanying commentary²² or in subsequent reviews.^{18, 20}

If the suspected HSD-related delay in initial RBC transfusion had truly increased mortality in the HSD trials,^11,12,27 the overall analyses would have revealed between-group differences not only in mortality, but also in the time to death and the count of RBC transfusions over 24 hours (Table 1). Had there been a true increase in HSD mortality in the subgroup of patients with the lowest bleeding severity (expected to be the largest patient subgroup in either trial), a consistent excess in overall mortality would have been expected in the time period between 1–30 days after enrollment. Conversely, had there been a true decrease in HSD mortality in the subgroup of trial patients with the worst bleeding severity (contributing the majority of deaths and in the earliest time period after injury), a consistent deficit in overall mortality would have been expected in the enrollment to 6 hour time period. Such time-interval specific and opposite patterns of overall mortality were not reported in the original trials^11,12 or evident from our secondary analyses. Primary cause of death was not captured in the pivotal trial. In the pilot trial, over 53% of the deaths within 24 hours were attributed to hypovolemic shock. In the time interval between 1–30 days, 60% of deaths were attributed to brain injury, but none to hypovolemic shock.

Because no single, universally accepted indicator of the amount of blood loss or bleeding severity exists for injured patients,³⁶ MT has become entrenched in the trauma research literature as a surrogate marker for the highest severity.^{1,3,30,37,38,39} Alternatives have been proposed, for example, the “critical administration threshold” or CAT, defined as a rate of 3 RBC unit transfusions per hour anytime within the first 24 hours after emergency medical service activation.³⁰ Like MT, the CAT surrogate covariate also sums blood product transfusions over time periods during which study patients may die before reaching a rate of 3 RBC transfusions per hour within 24 hours. Thus the use of the CAT alternative to MT for restriction, stratification or covariate adjustment is also likely to introduce collider bias.

Like the recent HSD trials,^11,12 two previous HSD trials^{14, 16} reported statistically significant effects in a subgroup defined by a post-randomization collider covariate (patients undergoing surgery¹⁴ or assigned a score >4 on the Abbreviated Injury Score for the head¹⁶). Clearly, the potential for collider bias in trauma research is not limited to subgrouping based on the conventional definition of MT or to observational study designs. Distinguishing between a valid and invalid covariate can be difficult because a collider covariate can have a dual role as both a consequence and contributing cause of either the treatment or the outcome. On the one hand, trauma patients must survive long enough to undergo the diagnostic procedures or interventions like MT that may be used as surrogate markers of severity, and on the other, the surrogate markers may also affect survival. Unfortunately, ambiguity in the causal directions linking covariates like MT with treatment or outcome does not preclude collider bias. The accurate and complete ascertainment of injury severity is an evolving process for many severely injured patients. The process can entail a series of definitive diagnostic procedures based on technologies available only in appropriately equipped centers. Clinical correlates, markers and surrogates of injury severity, like massive transfusion (MT), injury severity score and surgical intervention are easily ascertained by retrospective review of patient records. These surrogates are considered “validated” by their strong associations with injury mortality, and have become standard covariates for stratification or adjustment by injury severity. Because of time-honored conventions, trusted validation, lack of reliable alternatives, and the subtlety of collider bias, the widespread research use of these surrogate biomarkers of injury severity is likely to persist and frustrate systematic reviews^40,41 due to the unpredictable occurrence and variable impact of collider bias (e.g., conflicting findings) on trauma resuscitation research.

We acknowledge that there may be alternative explanations for the observed null overall HSD effects on mortality, but heterogeneous effects across strata defined by the 24-hour sum of RBC transfusions.^11,12 However, we believe the available evidence and clinical plausibility support our hypothesis of a survival-neutral HSD-induced delay in initiating transfusion. We interpret the meta-analysis results of all seven prehospital HSD trials as evidence that patients resuscitated with prehospital HSD survived unanticipated transfusion delays^11–17 to mirror the survival of their control counterparts. What remains unanswered is whether patients receiving prehospital standard-of-care crystalloid would have survived similar transfusion delays without the physiologic changes induced by HSD.²³ To determine whether prehospital HSD is superior or equivalent²³ to standard-of-care crystalloid in promoting trauma patients’ survival beyond the highest-risk interval and long enough to ensure definitive diagnosis and interventions, the following study conditions would be required: 1) a trauma setting in which prolonged emergency transport or unavoidable delays to definitive diagnosis and therapy are the norm, 2) a protocol with reliable indications for ED interventions that (unlike SBP) remain detectable regardless of prehospital fluid resuscitation, and 3) sufficient resources to capture accurate data on the nature and timing of all relevant injury characteristics, interventions and outcomes.

Extreme variation in the extent, severity and duration of serious injury sets off highly dynamic, complex and idiosyncratic sequences of trauma resuscitation interventions. The surge of diagnostic and therapeutic procedures occurs over a compressed timeline with the greatest threat to patient survival immediately following injury³⁴ and declining precipitously a few hours thereafter.^34,35,42 Thus, some of the challenges to the scientific integrity of trauma comparative effectiveness research may be unique. Standard clinical research training and tools for data analysis are so attuned to the typical increase in patients’ mortality rates with the passage of time¹ that even trauma’s strikingly opposite pattern can fail to attract attention. Compounding these challenges^39,43 has been the resistance of imminent early trauma death³⁴ to increasingly swift and sophisticated systems of care.^44–46

The validity and reproducibility of future trauma comparative effectiveness research depends on well-informed leadership and may benefit from 1) explicit a priori diagramming of the credible causal pathways among the key covariates and determinants of outcome, 2) avoiding collider bias by replacing any wrongly-ordered covariates with more logical substitutes, and 3) planning data capture and analysis strategies to account for the timing and sequence of key events and covariates.