Coagulopathy and transfusion strategies in trauma. Overwhelmed by literature, supported by weak evidence

The ultimate goal of medical literature is to offer clinicians indications to improve efficacy in treating their patients. Readers, however, often focus on the conclusions of articles and less on design and methods, without verifying whether the study has or has not met two fundamental requirements, i.e. internal and external validity. Ideally, a study meets the internal validity criteria when it eliminates any source of bias. Thus, only random variation and, for example, the treatment under study are responsible for the outcome. On the other hand, a study reaches external validity when its findings can be reliably applied to other contexts.

Unfortunately, serious internal and external validity issues have often been reported in medical literature^1,2. For this reason the publication of articles in which results are excessively emphasised may lead to practices and treatments which, though based on plausible hypotheses, are not fully legitimised by evidence³.

In addressing the issue of coagulopathy and transfusion practices in trauma for a recent consensus conference, I was flooded by an enormous amount of articles and after the review process was over I was left with very little evidence to deal with. Here I would like to highlight the many shortcomings that afflict the literature I have reviewed.

The (mis)use of multivariable models

The incorrect use of statistics determines internal validity issues. The great majority of studies dealing with coagulopathy and transfusions are observational. Studies that limit their analyses to crude comparisons of outcomes without controlling for confounders do not provide reliable evidence. Most studies, however, use either a multivariable approach or matching to try to minimise the effect of confounding. In the era of computers these statistical tools are within the reach of any researcher, but their correct application requires specific knowledge. Since studies dealing with trauma are usually focused on survival, logistic regression or Cox proportional hazard models that consider dichotomous outcomes are the most suitable statistics.

Of 18 studies investigating the causal link between coagulopathy and mortality in trauma using either one of these methods, 12 violated a fundamental rule of thumb that says that at least ten outcomes should be available for each variable included in the model⁴. This finding was consistent with the literature in other fields^5,6. Empirically, but convincingly, it has been demonstrated that unreliable regression coefficients are generated in these cases⁴.

Another common shortcoming concerned variable selection for multivariable models. A common approach was to submit the candidate variables to bivariate analysis using a predefined alpha value. This method is not recommended because it can lead to potentially important predictors being discarded⁷.

Although formal recommendations as for randomised controlled trials do not exist, some useful indications for reporting have been provided for observational studies using logistic regression⁸. Nevertheless, many studies dealing with coagulopathy in trauma and its treatment, underreported statistics. For example, very few studies reported statistical significance tests for the full multivariable model and goodness-of-fit measures. This and other omissions, although underreported methodology is not a marker of low quality, make quality evaluation impossible in many instances.

It is not necessary to be a statistician to assess internal validity, which can sometimes be done simply by looking at the results of multivariable models with attention. For example, in one study investigating the prognostic relevance of hypofibrinogenemia, five variables entered the logistic regression model⁹. Intuitively, five variables are too few to predict a complex event such as death. Indeed, many more variables are included in prognostic models^10–12. In our example, comorbidities or physiological parameters at admission were omitted from the logistic regression. When a model that has explanatory purposes is underfitted (i.e. includes too few variables) there is a risk of generating misleading results¹³. Moreover, in the study mentioned above 517 patients were included and there were 62 deaths⁹. When the sample size is so small, the chance that coefficients of the variables turn out to be statistically significant is also low (i.e. there is low statistical power), and important variables may be lost. Not accounting adequately for confounding may be a source of bias. On the other hand not more than six variables could have been selected without running the risk of overfitting, which we have mentioned previously⁴. Most studies dealing with coagulopathy in trauma enrolled a few hundred patients with a low event rate, so the authors fell between the risk of generating a poor model or an overfitted one.

Another easy way to spot potential problems in a model is to look at the quantitative effect of study variables on the outcome. In a study investigating the effect of plasma on the mortality of massively transfused patients, the fresh-frozen plasma (FFP)/packed red blood cell (PRBC) odds ratio was 0.02 indicating an (unrealistic) 50-fold decrease of deaths for each point of increase in the FFP-PRBC ratio¹⁴. Exaggerated effects are particularly frequent in small studies that often lack reproducibility of their results¹⁵. Thus, similar findings should raise doubts on reliability of the models.

The immortal time bias

A common problem in observational studies investigating treatment is related to the time gap between enrolment in the study and treatment administration. In 2006, this issue was nicely investigated in the “Annals of Internal Medicine” in a critique of a previous article that attributed, at least in part, a 3.9-year survival increase of Oscar winners to their success^16,17. The objection was that in the Cox hazard proportional model used for survival analysis with covariate adjustment, the entire winners’ life span was considered while the exposure to the benefits of winning the Oscar prize was obviously limited to the period of life following their success. Thus, the “immortal time” period during which the actors had not yet been exposed to the influence of having won the Oscar prize and had not experienced the outcome was a prevalent part of the overall survival time. Repeating the analysis accounting for immortal time bias the survival advantage was about 1 year and statistically non-significant¹⁶. This is a specific design issue that hinders internal validity and perfectly fits severe trauma transfusion strategies (e.g. FFP:PRBC ratios, fibrinogen administration) in the literature. Indeed, patients who die within the first few hours are less likely to receive the treatment (e.g. FFP or fibrinogen). Thus, at least in part, patients receive the treatment because they survive and not the opposite. Strategies aimed at neutralising the immortal time bias by excluding deaths that occur within the first hour up to 24 hours^18–21 cause a large loss of information²².

Of three studies that used statistics to account for immortal time bias, one showed that high FFP:PRBC ratios were ineffective and the other two reported positive results^22–24. However, two studies were potentially biased because the variable to event ratio was 3.9 and 7.4, far from the minimal 10 to 1 ratio recommended in the literature^23,24. Interestingly, in the study that claimed to have debunked “the survival bias myth”, coma and age, well-known mortality predictors, turned out to be statistically non-significant in the Cox proportional hazard regression analysis²³. In my opinion, when a model generates such paradoxical results it is legitimate to question its reliability seriously.

The third was a high-quality study carried out in ten level-1 trauma centres, with about 900 patients included in the statistical model. The FFP:PRBC ratio turned out to be protective both as a continuous variable and as a three-level categorical variable. However, only nine variables were left in the model after screening a list that did not include potentially important predictors, such as the existence of serious comorbidities or the use of anti-coagulants. My main concern, however, was that the model did not include a propensity score.

Removing treatment selection bias

Reading the literature regarding different treatments for coagulopathy in trauma, what really caught my attention was that very few observational studies dealt with treatment selection bias. This omission is likely to have a negative impact on the internal validity of the studies.

Randomisation is needed in clinical trials in order to distribute all known and unknown factors that may influence the outcome evenly between the treatment and the control group²⁵. This way, each patient that is enrolled in the study has the same chance of receiving either the study or the control treatment (the placebo, for example). In observational studies investigating the effectiveness of a treatment, instead, there is a high risk of selection bias due to confounding by indication. Actually, physicians may be more prone to administer treatment to some patients and not to others. For example, patients with a poor prognosis or with a low chance of responding to treatment, both factors that could influence prognosis independently of treatment, may not be considered. To compensate for this bias the probability of receiving the treatment is usually estimated with a propensity score²⁶, and its values included in the mortality analysis (e.g. logistic regression or case-control studies). Obviously, the propensity score incorporates only known information, while important influential elements may be impossible to measure directly, such as physicians’ attitude to prescribe or not. This may lead to possible biases if the score is not developed and applied with skill²⁷.

When addressing the efficacy of FFP:PRBC high ratios, fibrinogen administration, and blood transfusions in general it is likely that selection bias risk is high. However, of the 11 studies that passed the final selection for the consensus, only three adopted a propensity-score based approach^19,20,28. Of these, two studies dealt with high FFP:PRBC ratios and both failed to demonstrate a positive effect on survival, but had serious statistical issues^19,20. The third study used the CRASH-2 trial dataset²⁹ to investigate blood transfusion efficacy in trauma and was of high quality²⁸. The propensity score adjustment was applied to four classes of risk stratified on the basis of a severity score also developed from the CRAH-2 sample³⁰. It turned out that blood transfusions increase the adjusted risk of death for the first two lowest risk classes, are uninfluential for the third class and are protective for the most severe. Although, this result is reasonable, the study did not account for survival bias. Moreover, propensity scores are indicated when treatment and the variables that determine its choice are measured at baseline. When patients are treated after observations starts, as in the studies we are dealing with, the “immortal time” bias may be incorporated, providing results that are difficult to interpret³¹. Other choices are available to account for time-dependent variables (e.g. transfusion approaches in trauma) and treatment selection bias³².

Design issues

Both internal and external validity are closely linked with the study design. In the field of trauma the design is often retrospective which may generate some problems. Indeed, patients are frequently included a posteriori in studies on the basis the number of PRBC units that have been administered within a specific time frame. Massive transfusions, used as a proxy of marked severity, for example, are defined as the administration of at least ten PRBC units within 24 hours of injury. In my review two studies using this inclusion criterion claimed the effectiveness of high FFP:PRBC ratios^14,33. The paradox is that any physician who would like to apply these transfusion strategies in her/his reality, should be able to predict at the patients’ admission who will and will not receive ten PRBC units in the next 24 hours to select the ones for whom treatment is indicated. Interestingly, of the nine observational studies dealing with the prognostic effect of FFP:PRBC ratios which I selected for final assessment in my review, seven used this or similar definitions^{14,18–22,33–35}.

Another problem when treatment is used to define the inclusion criteria is that it is influenced by local policies, potentially causing a generalisability issue, particularly when studies are carried out in a limited number of centres as frequently happens in this field.

Other generalisability issues

The objective of a study is to extrapolate its findings to other settings, what we call external validity or generalisability³⁶. There is plenty of literature dealing with this issue for clinical trials, but the subject is less debated for observational studies. However, in the specific field of prognostic models this subject has been widely discussed. Indeed, external validity has been considered a requisite for their application to other contexts, and must always be formally investigated³⁷. When the contexts in which prognostic models had been developed in were very different from those in which they were applied, not only in terms of case-mix but also of context variables such as socio-economic conditions and health-care organisation, they turned out to fail external validation³⁸. This indicates the importance of homogeneity in observational studies between the study and the application setting. Nevertheless, in the field of trauma many hypotheses concerning post-traumatic coagulopathy and its treatment were initially developed and tested in military settings. Although, physicians took advantage of the information provided by these studies it is glaringly obvious that it would be unwise to extend findings from these studies automatically to civilian contexts. For example, in one study carried out in a combat support hospital in Baghdad, of 246 patients only three were females, the average age was 24, 94% had penetrating injuries, and the median stay before transfer to a hospital in Germany was 1 day which hampered a meaningful assessment of in-hospital mortality rate³⁹.

Generalisability issues, however, also regarded one of the very few randomised controlled trials that have been conducted in the field of trauma. The CRASH-2 trial, which investigated the effect of tranexamic acid on survival, was based on the theory that hyper-fibrinolysis underlies coagulopathy in trauma and that breaking this mechanism could improve prognosis²⁹. However, the study was carried out mainly in South America, Africa, Asia, and Eastern Europe where the health-care organisation and socio-economic conditions are very different from those in western countries. The findings from this study as well those from the observational study based on the CRASH-2 dataset²⁸ should, hence, be looked at with caution.

Another obstacle to generalisability is that many studies in the field of trauma are carried out in single centres. In this case, specific case-mixes, therapeutic approaches, and level of health-care delivery may provide non-replicable results⁴⁰.

Coagulopathy and mortality: what evidence do we have?

Treatments are aimed at contrasting the noxious effect of diseases. Thus, for the aim of my review it was crucial to establish what post-traumatic coagulopathy is and how it is related to mortality in trauma, the prerequisite for understanding the potentials for treatment. In a study with a statistical analysis that seemed to be more accurate than the average, a non-linear relation between fibrinogen plasma concentrations and survival was found⁴¹. We would expect, within low fibrinogen plasma concentrations, that increases in fibrinogenaemia would be associated with a reduction of mortality and to find a plateau for higher plasma levels. Unfortunately, in the high concentrations levels, instead, any increase in fibrinogen concentration caused an increased risk of death. This paradoxical result was replicated for the injury severity score (ISS), with a biphasic effect characterized by increased risk of death with increased scores within ISS low values (i.e. <25.7) and risk reduction as the score increased in the high range of ISS values. As mentioned earlier, I think it is legitimate not to trust results from models providing paradoxical results. Besides this, for each gram/Litre of fibrinogen increase within low concentrations, mortality decreased by 92%, reasonably an exaggerated result. Other studies did not seem to provide higher quality evidence.

When dealing with post-traumatic coagulopathy I believe that one of the research goals should be to discriminate between patients dying because they are bleeding and those who are bleeding because they are dying, which is not yet possible with available evidence. I think that as long as studies do not account for main bleeding source control with surgery or interventional radiology (and I could find no such study in the literature), we will fail to discriminate between coagulopathy as a cause and coagulopathy as the consequence of haemorrhage, and thus to identify the target of possible treatments.

Randomised controlled trials

Besides the CRASH-2²⁹, with its generalisability issues mentioned above, the only other randomised controlled trial retrieved in my search was the PROPPR trial⁴². In this study a 1:1:1 ratio of FFP:platelets:PRBC was compared to a 1:1:2 ratio, with the former providing a non-significant 3.7% reduction in 30-day mortality. The sample size was calculated assuming a 35% mortality rate in the control group and a 12% reduction in the study group. However, when the study was concluded mortalities were respectively 26.1% and 22.4%. The minimum sample size required to detect a 12% mortality reduction with a 92% power was 580 patients. However, for the purpose of sample size calculations the minimal clinically relevant therapeutic difference should be chosen⁴³. In the case of mortality from severe trauma I think that, say, a 5% mortality reduction would have been a clinically relevant objective. This would, however, have required about 4,000 participants and about 8 years to carry out the study, an unacceptable timespan for researchers I suppose. In the end the study was concluded but the answer to a clinically meaningful query based on a minimal relevant reduction in mortality was not obtained.

Conclusions

The paucity of evidence I encountered going through the literature dedicated to coagulopathy and transfusion strategies for trauma is confirmed by the absolute prevalence of very-low to low quality evidence reported in recent guidelines⁴⁴. This is a discouraging situation if we consider the hundreds of articles that have been published. For this reason, the standards of the review system of journals should be raised to filter and exclude poor research.

In my opinion, permanent trauma registries designed to investigate crucial questions, providing high-quality data, and using statistical tools correctly for analyses, are a valuable way to shed light on this field of medicine⁴⁵.

When evidence is lacking, the opinions of experts become the main source of indications for physicians. In this context expression of a plurality of opinions should be granted and interdisciplinarity should be wished for, while constantly trying not to mythologise our own research⁴⁶.