New considerations in the design of clinical trials for traumatic brain injury

Abstract

Randomized controlled trials in traumatic brain injury (TBI) pose several complicated methodological challenges related to the heterogeneity of the population. Several strategies have been proposed to deal with these challenges. Recommendations presented by the International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) study group include the use of relatively broad enrollment criteria combined with covariate adjustment for strong predictors of outcome in the analysis phase, rather than the use of strict enrollment criteria. Furthermore, an ordinal rather than a dichotomized analysis of the Glasgow Outcome Scale – the outcome measure in most TBI trials – will increase the statistical power significantly. This review discusses the issue of heterogeneity in TBI trials and summarizes the value of different innovative methods for the design and statistical analysis of randomized controlled trials in TBI. Future directions highlight the opportunities offered by alternative strategies, such as comparative effectiveness research, to investigate the clinical benefits of established and novel therapies in TBI.

Traumatic brain injury

Worldwide, many millions of people suffer from traumatic brain injury (TBI) each year [1]. Despite all efforts from researchers and clinicians to improve outcome after TBI, it remains a major cause of both death and permanent disability [2].

Rather than a single event, TBI is considered to be a continuous process initiated by the initial impact (primary injury) evolving over subsequent hours and days (secondary injury). Different pathophysiological mechanisms are triggered by the initial injury, which are responsible for further neuronal and glial cell death in the brain following the primary impact of TBI, and may result in clinical deterioration of TBI patients. These mechanisms include excitotoxicity, ischemia, edema, oxidative damage, mitochondrial dysfunction, apoptosis and many others [3,4]. These are considered to be possible targets for therapeutic interventions, aiming to limit the disastrous consequences of secondary injury to the brain.

Randomized controlled trials (RCTs) are considered to be the gold standard for evaluating the efficacy of new treatments. However, the acute TBI guidelines, covering many aspects of in-hospital treatment, only contain three level 1 recommendations based on the results of clinical trials. Nevertheless, many multicenter Phase III clinical trials have been performed to test different pharmacological agents and therapeutic strategies affecting different pathophysiological mechanisms that are active in TBI. Investigated neuroprotective agents have targeted, in particular, calcium-mediated damage, lipid peroxidation, glutamate excitotoxicity and edema. These agents have mostly targeted single mechanisms, but some were thought to target a number of different pathophysiologic mechanisms. Despite very promising results in strictly controlled in vitro and in vivo laboratory experiments, none of these clinical trials have convincingly proven clinical efficacy in the treatment of TBI patients [5,6]. Clearly, a major gap exists in translating experimental findings to the clinical situation, for which explanations can be found on both sides of this gap [7].

Preclinical studies

The classical pathway of the development of a new drug starts in the laboratory by defining molecular and cellular pathways that are active in TBI. The next step is to test efficacy of candidate drugs in the many available animal models of TBI [8]. However, none of these models adequately represents the complex picture of TBI seen in human patients. In the experimental animal models, the type and degree of injury can be standardized, moreover pretreatment is possible, and study end points are often mechanistic.

In the clinical situation, wide variability exists in the type of pathology and severity of injury. Furthermore, pretreatment is impossible and interventions within short therapeutic windows can be challenging. For optimal translation to the clinical situation, experimental studies should preferably be performed in more than one model, in more than one species and have effects on both mechanistic and behavioral end points. A workshop organized by the National Institute of Neurological Disorders and Stroke (NINDS) in May 2000 concluded that preclinical studies and early clinical (Phase II) trials have not always been performed with sufficient rigor. As stated by Hall in the proceedings of this workshop: “we simply have not done adequate therapeutic window studies in our animal models in most cases. Furthermore, when we do have such data, we tend to ignore it when we go to the clinic” [9].

Differences may exist in the onset and duration of pathophysiologic mechanism between lissencephalic animals (rats/mice) and gyrencephalic species such as humans. Uncertainty will, therefore, remain when translating experimental time windows obtained in a rodent model to human pathophysiology. General requirements for initiating Phase II studies include:

• Mechanism demonstrated in animal models;

• Drug/agent reverses damage in animal models;

• Mechanism shown to be active in human TBIs;

• Neuroprotective agent that passes the brain–barrier;

• Safety/tolerability in humans with TBI;

• Drug-sensitive end points.

Phase III clinical trials

In the past 30 years, more than 20 large multicenter Phase III trials have been performed to investigate the efficacy of novel neuroprotective agents in TBI [5,6]. However, almost none showed an overall significant treatment effect. Besides deficiencies in the preclinical work-up, substantial limitations in study design have been revealed that have contributed to failures in clinical studies [3,9,10]. The problems include inadequate sample sizes, insensitive outcome measures, inappropriate selection of the study populations, over-optimistic expectations of new therapies and heterogeneity of TBI patient populations. To investigate possible solutions for these problems, the International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) study group was initiated in 2003 as a collaborative venture supported by the US NIH [11]. The study group included clinical, epidemiological and statistical investigators from Belgium, The Netherlands, the UK and the USA. The IMPACT investigators were initially granted access to individual patient data of eight randomized control trials and three observational studies, including a total of 9205 patients [12]. During the continuation funding period (2007–2011), the number of studies was expanded providing access to data of over 40,000 patients. Relevant variables from the individual studies were extracted and merged to form a culture medium for exploring concepts to improve the design of clinical trials in TBI. The focus was on methodological approaches for dealing with the heterogeneity inherent to the TBI population. Identification of robust covariates and the development of prognostic models formed the cornerstone for explorations on how best to deal with the heterogeneity. These explorations involved extensive simulation studies, addressing different approaches to patient selection on enrollment, covariate adjustment and ordinal outcome analysis.

This review discusses the issue of heterogeneity, and summarizes the value of the different innovative methods for the design and statistical analysis of Phase III trials to investigate safety and efficacy of new neuro-protective drugs or other therapeutic interventions in patients with moderate-to-severe TBI. We also address alternative strategies for investigating the clinical benefit of established and novel therapies.

Challenges posed by heterogeneity of the population to the design of trials in TBI

TBI is a heterogeneous disease with respect to cause, pathology, severity and prognosis [13]. This heterogeneity poses methodological challenges to the design and analysis of clinical trials in TBI. The general perception of the problems related to heterogeneity of populations in clinical trials focuses mainly on the increased risk for imbalances between treatment groups. The problems incurred by heterogeneity, however, are much greater and relate to the methodology for efficacy assessment and sample size calculations.

Imbalances

Imbalances in the distribution of baseline characteristics may influence the results of a study. This is especially true if the association between a baseline characteristic that is imbalanced and the outcome is strong. It should be noted that even in the absence of (severe) imbalances in individual parameters, their cumulative risk may result in a significant imbalance. It is therefore recommended to report the prognostic risk estimate for treatment and placebo groups separately. This risk can be calculated with existing prognostic models [14,15].

In TBI subpopulations with smaller sample sizes, risks for imbalances are even higher. Indeed, various TBI studies have reported imbalances in subgroups. For example, in the international Tirilazad study there were imbalances between treatment groups in male patients whose computed tomography (CT) scan showed traumatic subarachnoid hemorrhage [16]. These imbalances were related to the CT lesion type, the occurrence of pretreatment hypoxia or hypotension and the presence of subarachnoid hemorrhage on the pretreatment CT scan favoring outcome in patients treated with placebo. Similar imbalances had also been noted in the North American Tirilazad study with regard to the Glasgow Coma Scale, the pattern of brain injury as demonstrated by the pretreatment CT scan and the frequency of bilaterally unreactive pupils.

Heterogeneity & efficacy analysis

Ideally, efficacy analysis should include mechanistic end points to allow the detection of drug-specific effects. The use of early mechanistic end points has advanced the field of oncology, HIV/AIDS research and cardiovascular medicine, for example. Such measures are, however, not yet available for TBI. In the absence of early mechanistic end points, TBI investigators and regulatory authorities have both adopted the Glasgow Outcome Scale (GOS) or its extended version as the standard for primary efficacy analysis [17,18]. The GOS is an ordinal scale for functional outcome and consists of five categories. In the extended GOS, the categories of severe disability, moderate disability and good recovery are each subdivided into a lower and upper category (Box 1). In Phase III clinical trials in TBI, the GOS has traditionally been collapsed to a binary outcome of ‘unfavorable outcome’ (death, vegetative state, severe disability) versus ‘favorable outcome’ (moderate disability, good recovery), irrespective of the baseline prognostic risk. This prognostic risk can vary greatly, and setting an arbitrary threshold which patients must cross to demonstrate clinical improvement is not reflective of the clinical situation and, in fact, substantially reduces chances of showing benefit. A patient’s prognosis can be so good that no matter how poor the treatment, they may be expected to achieve a favorable outcome, whilst conversely a patient’s prognosis may be so poor that it would become very unlikely for even a highly effective intervention to improve that outcome to an extent that it would change from unfavorable to favorable. Such patients will not contribute to the efficacy analysis. Although, perhaps intuitively attractive because it is so simple, the traditional approach to dichotomize the GOS is counterproductive and disregards potentially valuable information contained in the ordinal scale [19].

Box 1. Glasgow Outcome Scale and its extended version.

Glasgow Outcome Scale

1. Death, mortality from any cause

2. Vegetative state, unable to interact with environment, unresponsive

3. Severe disability, conscious but dependent

4. Moderate disability, independent but disabled

5. Good recovery, return to normal occupation and social activities, may have minor residual deficits

Glasgow Outcome Scale Extended

1. Death

2. Vegetative state

3. Lower: dependant on others for activities of daily life

4. Upper: dependant on others for some activities

5. Lower: unable to return to work or participate in social activities

6. Upper: return to work at reduced capacity, reduced participation in social activities

7. Lower: minor social or mental deficits which do not impair normal functioning

8. Upper: full recovery, no residual complaints or deficits

Heterogeneity & sample size calculation

Sample size calculations for TBI trials have commonly aimed to detect an improvement in favorable outcome by an absolute value of 10%, presuming that there will be an approximately 50:50 distribution of outcome in the patient population. These calculations show that approximately 800 patients will be required to detect the postulated treatment effect with a statistical power of 80%. However, these calculations are potentially flawed; the assumption of a 50:50 outcome distribution underpinning these calculations does not pertain to the group level, but rather to every individual patient. The required sample size is strongly dependent on the outcome distribution and, as this deviates more from the 50:50 level, required sample sizes increase exponentially (Figure 1). Consequently, when patients are included with a more extreme prognosis sample size, calculations need to be adjusted. Even in trials with relatively strict enrollment criteria, up to approximately 40% of patients may have an extreme prognosis (low or high risk). The confounding effect of prognostic heterogeneity will inevitably reduce statistical power to such an extent that chances of demonstrating benefit are greatly reduced.

Figure 1. Required sample size in relation to outcome distribution.

The x-axis displays the (expected) proportion of patients with unfavorable outcome in the study population. The y-axis displays the required number of patients to detect an absolute difference in an unfavorable outcome of 10% between treatment groups with a power of 80%.

Solutions to the challenges posed by heterogeneity of the TBI population

Reducing the risk of imbalances

Three approaches to avoid imbalances can be considered: decrease heterogeneity, increase sample size or stratify randomization. Strict enrollment criteria have commonly been employed in TBI trials with an aim to decrease heterogeneity. The disadvantage of this approach; however, is that the generalizability of findings is decreased and that it is inefficient, as it leads to longer study duration (discussed below). Conversely, increasing the sample size and using less restrictive enrollment criteria (accepting the consequent heterogeneity), will increase generalizability and reduce the risk of occurrence of imbalances. In so called ‘mega trials’, these risks are considered to be very low. The CRASH trial was the first ‘mega trial’ in the field of TBI [20,21] and investigated the efficacy of methylprednisolone in patients with mild to severe TBI (1999–2004). The trial planned to enroll 20,000 patients, but after inclusion of 10,008 it was halted because an interim analysis showed a higher 14-day mortality rate in the active treatment arm (21.1 vs 17.9% deaths; p = 0.0001). Liberal enrollment criteria were used to stimulate recruitment, reduce the risk of imbalances and increase the generalizability of results. The large heterogeneity was seen more as an asset than a problem. Although the approach of selection of the study cohort is substantially different from previous clinical trials in the field of TBI, this trial was one of the first multicenter trials in TBI that demonstrated an overall statistically significant (but negative) treatment effect. Imbalances between treatment groups were not found. Stratifying the randomization on important prognostic factors avoids imbalances in the treatment groups, but has some practical disadvantages, such as the need for more advanced (electronic) randomization schemes.

Dealing with prognostic heterogeneity

Conceptually, prognostic heterogeneity can be reduced by the use of stricter enrollment criteria (also reducing the risk of imbalance) or by the application of covariate adjustment in the analysis phase. Both approaches aim to increase statistical power of a trial.

Simulation studies within the IMPACT database have shown that stricter enrollment criteria can indeed increase statistical power by 33% in observational surveys and 5% in RCTs [22]. However, a disadvantage of selection is the reduction of a trial’s recruitment rate (number of patients that can be recruited in a certain period of time) and, therefore, prolongation of the trial duration. Stricter enrollment criteria led to a reduction in recruitment of approximately 65% in the observational studies and 41% in the trial populations included in the IMPACT database [23]. These studies showed that the benefits of selection in terms of statistical power were outweighed by adverse effects on recruitment rate and trial duration, and therefore make this approach inefficient (Table 1) [22].

Table 1.

Effects of selection and targeting on statistical power (required sample size) and recruitment.

	Required sample size (statistical power)		Recruitment
	Surveys	RCTs	Surveys	RCTs
Original selection	Ref	Ref	Ref	Ref
Strict enrollment criteria	−33%	−5%	↓ 65%	↓ 41%
Prognostic targeting	−28%	−17%	↓ 45%	↓ 43%

RCT: Randomized controlled trial; Ref. Reference category.

Adapted from [22,23].

Alternatively, one may consider selecting patients not based on separate enrollment criteria, but on their summarized prognostic risk (i.e., targeting patients with an intermediate prognostic risk). This approach is now feasible following the development of validated prognostic models for predicting outcome in TBI [14,15]. The concept of prognostic targeting was previously proposed by Machado et al. [24] and confirmed in the study by Roozenbeek et al. (Table 1) [22]. Although the use of a baseline prognostic risk score would seem more efficient than the use of separate inclusion and exclusion criteria, and its benefit in terms of statistical power proven, the exclusion of patients will reduce recruitment and consequently lengthen study duration. The increase in relative study duration, as summarized in Table 2, with the use of both strict enrollment criteria and prognostic targeting render these approaches inefficient. Rather than attempting to decrease heterogeneity on enrollment, one may consider the application of covariate adjustment in the analysis phase. This will not affect recruitment or study duration. Covariate adjustment is a procedure to control for unbalanced prognostic factors in order to obtain less biased estimates of the treatment effect. Stratified randomization is often combined with covariate adjustment, but covariate adjustment does not require stratified randomization.

Table 2.

Net effect of selection, targeting and covariate adjustment on study efficiency.

	Relative study duration
	Surveys	RCTs
Original selection	Ref	Ref
Strict enrollment criteria	+95%	+60%
Prognostic targeting	+5%	+11%
Covariate adjustment	−30%	−16%

RCT: Randomized controlled trial; Ref. Reference category.

Adapted from [22].

Simulation studies with TBI trial data showed that logistic regression analysis of the treatment effect, with covariate adjustment for seven strong predictors of outcome, resulted in a 25% gain in statistical power, compared with the unadjusted analysis [25]. These results were confirmed across the individual patient populations of the IMPACT database, demonstrating an increase in statistical power of up to 30% in more heterogeneous populations of observational surveys and up to 16% in trial populations with stricter criteria [22].

Different approaches can be used to decide which covariates should be used for adjustment. One approach is the use of knowledge on predictive factors of outcome from previous studies. The prognostic analysis performed by the IMPACT study group on long-term mortality and functional outcome after TBI have confirmed the predictive value of multiple covariates, including age, the Glasgow Coma Scale motor score, pupillary reactivity, brain CT scan characteristics and second insults (hypoxia and hypotension) [23]. The prognostic models based on these variables have now been extensively externally validated [14,15]. The models have been shown to be applicable in moderate and severe TBI populations of many different settings, and therefore can be recommended to be used for covariate adjustment in trials in moderate and severe TBI. Prognostic models for mild TBI have also been published, but not externally validated [26].

Another approach is to use the data of the trial to investigate which covariates may best be used. However, the use of this approach has the risk of subjective selection of variables. Simulation studies have shown that this approach indeed carries the risk for biased estimates of the treatment effect, especially in trials with small sample sizes [27]. The use of a prespecified adjustment model is recommended. As a minimum, prespecification of the covariates in the statistical analysis plan should be required.

Covariate adjustment in the analysis phase does not carry the disadvantage of reducing recruitment. Consequently, the IMPACT study group recommends a relatively broad inclusion criteria combined with covariate adjustment in the analysis phase. This approach is expected to yield a reduction of sample size of 20 to 25%, which is much greater than has previously been observed in cardiovascular trials, for example [28]. An additional advantage of broad inclusion criteria is that it will increase the generalizability of the findings in the study to less selected TBI populations.

Ordinal approaches in the primary efficacy analysis

The traditional approach of dichotomizing the GOS has the disadvantages of discarding potentially valuable information and that the point of dichotomization is not related to the initial prognostic risk [19]. Approaches that would either take advantage of the full ordinal nature of the GOS and/or relate the individuals’ outcome to the initial baseline prognostic risk may be expected to be more efficient. In addition, clinically relevant shifts in outcome, for example from moderate disability to good recovery, are taken into account in an ordinal approach. Two possible approaches for this are proportional odds analysis and the application of the sliding dichotomy. Superficially, the two approaches have much in common. They both exploit the ordinal nature of the GOS, but conceptually there are major differences.

In the proportional odds model, the pooled odds ratios are based on the treatment effect calculated for each of the possible splits for collapsing an ordinal scale. In this way, the study sample is not subdivided, but rather every patient contributes to the estimate of a so-called ‘common odds ratio’. Hence, an overall estimate of the shift in outcome across the GOS is obtained for better perception in a clinical audience. The use of the proportional odds model has been referred to as a ‘shift analysis’ [29]. This approach was used in the efficacy analysis of several stroke trials [30,31] and in one recently published TBI trial [32].

In the sliding dichotomy approach, the point of dichotomy of the GOS is differentiated according to the baseline prognostic risk. For patients with a poor prognosis, survival may be relevant, whilst in those with a good prognosis, any outcome worse than good recovery may be considered unfavorable. The concept of the sliding dichotomy approach is intuitively attractive for clinicians and has been adopted for the primary analysis of some Phase III trials in TBI, stroke and intra-cerebral hemorrhage [33–35]. In contrast to the traditional dichotomous analysis of the GOS, in which patients are required to cross a prespecified, fixed and artificially determined boundary, the sliding dichotomy approach takes other transitions, for example that from moderate disability to good recovery into account. Use of the sliding dichotomy approach (and covariate adjustment) requires the identification of robust prognostic models to reliably provide a baseline risk estimate in individual patients, which are available for moderate and severe TBI [14,15].

Extensive simulation studies have been performed in the IMPACT database to explore the benefits of ordinal approaches to the outcome analysis [36]. Both application of proportional odds analysis and the sliding dichotomy yielded considerable benefits, increasing statistical power by 23 to 30%. Applying covariate adjustment in addition to the ordinal analysis further increased power to a total of up to 40 to 49% (Table 3). The benefits of this approach were further confirmed in new datasets and when applying both approaches to the data of the MRC CRASH study. The expected benefits were confirmed in the ‘real life’ situation of a clinical trial [37]. These findings strongly support adopting an ordinal approach to outcome analysis in TBI trials.

Table 3.

Increasing trial efficiency by ordinal analysis and/or covariate adjustment.

Statistical approach	Reduction in sample size (%)
	Median	Interquartile range	Range
Conventional dichotomy	Ref	Ref	Ref
Conventional dichotomy + covariate adjustment	26	20–29	14–29
Proportional odds analysis	23	19–24	18–24
Proportional odds + covariate adjustment	49	45–53	41–57
Sliding dichotomy	30	29–37	16–45
Sliding dichotomy + covariate adjustment	40	34–44	25–51

Ref. Reference category.

Adapted from [36].

The choice between proportional odds analysis and the sliding dichotomy is based more on a value judgment than any scientific motivation. From a statistical perspective, the proportional odds model appears more efficient, but is perhaps less appealing for a clinical audience. Thus, we consider both approaches appropriate for the analysis of TBI trials. Whichever ordinal approach is chosen, the evidence strongly indicates that the conventional dichotomous analysis should be discarded from the trialists’ toolbox, unless one is exclusively interested in one particular outcome.

Future perspective

The reappraisal of trial methodology in TBI and the recommendations proposed by the IMPACT study group have provided us with tools to conduct trials more efficiently in the field of TBI, providing better chances to detect treatment effects. We see three important directions for further improvements: more comprehensive approaches to outcome assessments, standardization of data collection/coding and alternative approaches to exploring efficacy.

More comprehensive approaches to outcome assessments

More than three decades after Jennett and Bond described the GOS, it is still the most commonly used outcome measure in TBI trials. However, by definition the GOS is a global assessment and insufficiently recognizes the complex aspects of outcome following TBI. Furthermore, despite the use of the structured interview [38], misclassification of the GOS assessment can occur with adverse effects upon statistical power [39]. We see a clear need to further develop a multidimensional approach to outcome assessment and classification [40], including neuropsychological measure, and to include the patient’s perspective on quality of life. We should also consider adding in a time dimension. In the field of health economics, the use of quality-adjusted life years is common practice. A similar approach, in which outcome assessments are measured at multiple time points and integrated with mortality, may offer a new and more comprehensive way to approach outcome assessments in TBI. This would offer opportunities to additionally capture the speed of recovery.

A major limitation in the conduct of clinical trials in TBI is the striking lack of early mechanistic end points. Different surrogate, early mechanistic end points for TBI have been suggested in the past, such as intracranial pressure, therapy intensity level, jugular venous oxygen saturation, metabolic measurements and neuroimaging characteristics [9]. However, none of these measures have proven to be directly related to functional outcome [3]. Consequently, in TBI it is impossible to target existing or novel therapeutic approaches appropriately. Mechanistic targeting – the ideal for clinical trials – would necessitate the identification of occurrence and time course of pathophysiologic mechanisms in individual patients. Advances in neuroimaging and the emerging field of biomarkers offer hope for the future. With these techniques, disease processes can be tracked and patients could function as ‘their own controls’. Thus, we may determine if the therapy under investigation might reduce incremental tissue injury – the first principle of neuroprotection.

Standardization of data collection

A high quality observational study, combined with comparative effectiveness research (CER) as well as meta- analysis of individual patient data across existing studies, requires standardization of data collection and coding. Initial steps towards the development of proposals for standardization have been integrated into an internationally oriented interagency initiative towards, ‘an integrated approach to research in psychological health and TBI’. This initiative involving the NINDS, the National Institute on Disability and Rehabilitation Research, the Department of Veterans Affairs, the Defense and Veterans Brain Injury Center and the Defense Centers of Excellence included four working groups in the field of TBI addressing four domains:

• Demographics and clinical assessment;

• Biomarkers;

• Neuroimaging;

• Outcome.

General recommendations have been published [34–37,41]. The diversity and specific characteristics of the topics addressed by the working groups resulted in a different emphasis in the recommendations. In the biomarkers and imaging groups, emphasis was placed on standardization of techniques and procedures, whilst in the outcomes group, the main emphasis was on the selection of instruments. For demographics and clinical assessments, the standardization of coding of the variables was the most important. The process for developing recommendations for clinical data elements was consensus-driven, with multidisciplinary and international input from experts across a broad range of disciplines. A general consensus on selection and coding of data elements was achieved and templates were produced that summarized coding formats, motivation of choices and recommendations for procedures. Some recommendations concerned novel approaches, for example towards assessing the intensity of therapy in severely injured patients. It was recognized that the selection and the required level of detail in data collection would vary greatly with the design and aim of a specific study. Three levels of detail for coding elements were developed:

• A basic version;

• An advanced version;

• An extended format with the greatest level of detail in the extended version.

The coding of these versions was such that in every case the more detailed coding could be collapsed into the basic version, thus facilitating comparisons across studies. The proposed data elements and their coding are intended as ‘building blocks’ for designing a case report form and can be used as ‘plug-in’ elements used multiple times in various sections of a case record form. The overall structure of the Common Data Elements (CDE) recommendations for coding and the templates can be viewed on the IMPACT [101] and CDE websites [102].

The process for standardization of data collection in TBI is considered crucial for advancing the care for TBI patients and has been well received in the field. Endorsements have been obtained from the AANS/CNS section on neurotrauma and critical care, the International and National Neurotrauma Societies and the European Brain Injury Consortium. It should be recognized, however, that the process of standardization is and will remain an ongoing process, for which continuous feedback, refinement and updating is required. This could best be overseen by a noncommercial, scientific authority such as the NIH in the USA.

Alternative approaches to exploring efficacy analysis

RCTs are considered the gold standard for proving efficacy of new treatments. Classical clinical trials are characterized by a strong reductionist approach and suffer from some limitations. RCTs address efficacy rather than effectiveness. Efficacy reflects the extent to which an intervention provides benefit under carefully controlled conditions chosen to maximize the likelihood of observing an effect, whereas effectiveness relates more to the benefits and harms of an intervention in ordinary settings and broader populations. RCTs may, therefore, not always address the effect of different clinical practices in the ‘real world’. Moreover, they are costly and we should recognize that we will never be able to adequately execute powered trials to address all the existing uncertainties in clinical TBI management. Alternative approaches to exploring efficacy should, therefore, be considered in addition to the gold standard of clinical trials. The existing heterogeneity in TBI populations, management approaches and outcome, offers opportunities for exploring causes for these differences and identifying underlying reasons for a given outcome or individual patient response to a selected intervention. This represents CER, the concept of CER in TBI is not new. In 1983, Gelpke et al. [42] analyzed differences and outcome between two centers from The Netherlands and in 1989 Colohan et al. [43] performed comparative analysis of treatment results between Charlottesville (USA) and New Delhi (India). TBI may be considered particularly suited for application of CER for a number of reasons. First, the strong heterogeneity with large differences in both management and outcome between centers and countries provide a major opportunity to compare alternative interventions and managements [44]. Second, the availability of robust and validated risk adjustment models for both mortality and functional outcome permit approaches to adjust for prognostic heterogeneity. Third, sophisticated statistical approaches such as random effect analysis are available to support CER.

A recent workshop ‘Promoting effective traumatic brain injury research,’ held during the National Neurotrauma Symposium in 2010 organized by the EU and the NIH, stated that improved clinical care of TBI patients will likely depend on a range of research approaches, including CER and systems biology. An urgent need for standardization of treatments and validation of effective clinical guidelines in TBI was identified. It was proposed that these goals could best be achieved through a large observational study coupled to CER. Subsequent to this workshop, an ‘international initiative for TBI research’ has been formalized by the EU and NIH/NINDS [45].

Executive summary.

Background

• Clinical trials in traumatic brain injury are particularly challenging, due to the inherent heterogeneity of the patient population, the lack of early mechanistic end points and relative insensitivity of outcome measures.

Solutions to heterogeneity

• Solutions for dealing with the heterogeneity of the patient population have been developed by the IMPACT study group.

• The use of strict enrollment criteria is not recommended, as this is inefficient. Rather, broad enrollment criteria are preferred, combined with covariate adjustment in the analysis phase.

• Dichotomization of the Glasgow Outcome Scale is not recommended. Ordinal approaches to analysis of treatment effects offer greater statistical power and better sensitivity. To this purpose, the use of proportional odds methodology or the sliding dichotomy approach may be considered.

Future directions

• Advances in the fields of neuroimaging and biomarkers, as well as the currently available techniques for multimodality monitoring in the neurointensive care unit, offer hope for the future to better detect, quantify and track pathophysiologic mechanisms, which may be used as mechanistic end points.

• Although randomized controlled trials remain the gold standard for determining efficacy, we should recognize that they have limitations, and that we will never be able to resolve all existing uncertainties on treatment by trials. Comparative effectiveness research may be considered an alternative approach.

• Consensus on standardization of data collection and coding of variables is essential to future high-quality studies.