Pre-Clinical Testing of Therapies for Traumatic Brain Injury

Abstract

Despite the large number of promising neuroprotective agents identified in experimental traumatic brain injury (TBI) studies, none has yet shown meaningful improvements in long-term outcome in clinical trials. To develop recommendations and guidelines for pre-clinical testing of pharmacological or biological therapies for TBI, the Moody Project for Translational Traumatic Brain Injury Research hosted a symposium attended by investigators with extensive experience in pre-clinical TBI testing. The symposium participants discussed issues related to pre-clinical TBI testing including experimental models, therapy and outcome selection, study design, data analysis, and dissemination. Consensus recommendations included the creation of a manual of standard operating procedures with sufficiently detailed descriptions of modeling and outcome measurement procedures to permit replication. The importance of the selection of clinically relevant outcome variables, especially related to behavior testing, was noted. Considering the heterogeneous nature of human TBI, evidence of therapeutic efficacy in multiple, diverse (e.g., diffuse vs. focused) rodent models and a species with a gyrencephalic brain prior to clinical testing was encouraged. Basing drug doses, times, and routes of administration on pharmacokinetic and pharmacodynamic data in the test species was recommended. Symposium participants agreed that the publication of negative results would reduce costly and unnecessary duplication of unsuccessful experiments. Although some of the recommendations are more relevant to multi-center, multi-investigator collaborations, most are applicable to pre-clinical therapy testing in general. The goal of these consensus guidelines is to increase the likelihood that therapies that improve outcomes in pre-clinical studies will also improve outcomes in TBI patients.

Introduction

The Centers for Disease Control and Prevention estimates that there were about 1.7 million new traumatic brain injuries (TBIs) per year in the United States from 2002 to 2006.¹ These TBIs resulted in 1.4 million emergency department visits, 275,000 hospitalizations, and 52,000 fatalities (for a review, see Roozenbeek and colleague²). In addition to new TBIs, there are about 5.3 million people in the United States³ and 7.7 million people in Europe⁴ living with disabilities from TBI. Unfortunately, many of these individuals suffer from the long-term consequences of TBI for many years post-injury.⁵ Although estimates of the economic costs of TBI vary (for a review, see Humphries and associates⁶), the Centers for Disease Control and Prevention estimates medical costs of more than $76 billion annually in the United States alone.⁷ In view of the staggering economic and psychosocial costs of TBI, it is not surprising that considerable amounts of time and money have been devoted to finding treatments for TBI.

To adequately model the heterogeneous nature of TBI in humans, pre-clinical studies have been conducted using experimental models that replicate the important pathophysiological features of different types of human TBI.⁸ Despite the large number of promising neuroprotective agents identified in experimental TBI studies, none have shown meaningful improvements in long-term outcome in clinical trials.^8–10 A number of factors may contribute to these translational failures. For example, Agoston and colleagues⁹ observed that the types of data collected (e.g., histopathological, behavioral, imaging) and the collection times (hours, days, or weeks post-TBI) differ markedly between clinical and experimental TBI studies. It is also apparent that pre-clinical testing should involve multiple experimental TBI models,¹¹ preferably in multiple species.¹²

To develop recommendations and guidelines for the pre-clinical testing of pharmacological/biological therapies for TBI, the Moody Project for Translational Traumatic Brain Injury Research sponsored a symposium attended by investigators with extensive experience in pre-clinical TBI testing to identify potential limitations of current experimental models and protocols. This group also suggested strategies to improve the likelihood of future pre-clinical studies identifying therapies that rapidly and successfully translate to clinical trials.

State of the Art in TBI Clinical Trials

Burke and co-workers¹³ identified 142 randomized, controlled trials (RCTs) as of October 2013 and Bragge and colleagues¹⁴ reported 207 RCTs in moderate-to-severe TBI patients through 2015. Although many of these RCTs tested non-pharmacological therapies (e.g., cerebral perfusion pressure or intracranial pressure [ICP] control), Kabadi and Faden⁷ reported more than 30 trials of “neuroprotective strategies.” As of 2006, 130 therapies had proved successful in pre-clinical testing^15,16 and it is likely that the number now is much higher.

Were “positive” trials really positive?

Unfortunately, most of the beneficial effects observed in clinical trials to date have been transient, not translated into a desirable clinical benefit (e.g., they reduced mortality with a corresponding increase in the vegetative state rate), not reproducible, or offset by negative effects of the therapy. Yet, as summarized in Table 1, there have been several “statistically significant positive results.”¹⁷

Table 1.

Randomized Clinical Trials with Positive Results in TBI Patients, 1990–2012

Therapy	N	Type	Results	References
Pharmacological therapies
Nimodipine	123 TBI w SAH	RCT, multi	Significantly fewer unfavorable GOS scores at 6 months in nimodipine group	Harders et al.28
Progesterone	159 sev TBI	RCT	Significantly more favorable outcomes (GOS & FIM) and lower mortality in progesterone-treated pts 3 and 6 months post-TBI	Xiao et al.30
Amantadine	184 sev TBI	RCT, multi	Significantly faster rate of functional recovery (DRS) 4 wks post-TBI in amantadine-treated pts	Giacino et al.27
Phenytoin	404 sev TBI	RCT, single	Significantly fewer seizures in phenytoin-treated pts 1-7 days post-TBI	Temkin et al.26
Surgical/Management therapies
Hypothermia	215 sev TBI	RCT, multi	Significantly lower ICP and more favorable GOS in sev TBI pts with 5 days vs. 2 days of hypothermia	Jiang et al.20
	47 sev TBI	RCT, single	Significant reduction in number of seizures in hypothermic patients	Clifton et al.23
	396 sev TBI	RCT, single	Significantly lower mortality and more favorable outcome in hypothermia group	Zhi et al.21
	82 sev TBI	RCT, single	hypothermia hastened neurological recovery; trend toward improved outcome	Marion et al.24
Craniectomy	486 sev TBI	RCT, multi	Significantly more favorable GOS at 6 months in pts with larger (15x15 cm) vs. smaller (6x8 cm) craniectomy	Jiang et al.156
Craniectomy	408 sev TBI	RCT, multi	Significantly lower mortality in decompressive craniectomy group	Hutchinson et al.157
CBF targeted therapy	189 sev TBI	RCT, single	Jugular venous desaturation significantly lower in CBF- vs. ICP-targeted therapy	Robertson et al.18
Hyperbaric O2	168 sev TBI	RCT, single	Mortality rate was significantly lower in pts treated with hyperbaric O2	Rockswold et al.19

CBF, cerebral blood flow; DRS, disability rating scale; FIM, functional independence measure; GOS, Glasgow Outcome Scale; multi, multi-center RCT; pts, patients; RCT, randomized clinical trial; SAH, subarachnoid hemorrhage; single, single-center RCT; sev, severe; TBI, traumatic brain injury.

Robertson and associates reported that although cerebral blood flow (CBF)-targeted therapy reduced the incidence of jugular venous desaturation substantially compared with ICP-targeted therapy, neurological outcome was the same in the two groups, perhaps because the frequency of adult respiratory distress syndrome was significantly higher with CBF-targeted therapy.¹⁸ Hyperbaric oxygen (HBO) therapy in severe TBI patients was associated with significantly lower mortality, but the number of favorable outcomes was no higher in the HBO patients than in the controls.¹⁹ In a three-center study of severe TBI patients with intracranial hypertension, Jiang and colleagues²⁰ observed that 5 days of mild hypothermia (33–35°C) resulted in significantly more favorable outcomes than 2 days of hypothermia, but the study design did not include a normothermic control group.

One initially promising technique was reported by Zhi and associates,²¹ who compared mortality and 6-month Glasgow Outcome Scale (GOS) scores in 198 normothermic patients with severe TBI, and an equal number in whom rectal temperature was maintained at 32–35°C for 1 to 7 days. Mortality was significantly lower and the rate of “good” recovery was significantly higher in the hypothermia group. Additionally, CBF and jugular venous O₂ saturation were higher during hypothermia in a small (15) subset of patients in whom those variables were measured prior to and during the period of hypothermia. But in contrast to Zhi and associates, Clifton and colleagues,²² in a multi-center study of 392 patients with severe TBI, observed no significant effect of hypothermia (33°C) on mortality or GOS scores (6 months). Further, the complication rate and length of hospitalization were significantly higher in the hypothermic group. In addition to the study by Zhi and associates,²¹ the results of Clifton and colleagues²² differ from the results of two earlier studies, at least to some extent. In a single-center RCT of 46 patients with severe TBI, Clifton and collegaues²³ observed that hypothermia (32–33°C) was associated with a statistically significant reduction in seizures and a trend toward better outcome (GOS, 3 months) that was not statistically significant. Based on another single-center RCT of 82 patients with severe TBI, Marion and co-workers²⁴ reported that hypothermia (32–33°C) resulted in a significant improvement in GOS (3 and 6 months). Unfortunately, there were no significant differences in GOS scores in the hypothermic compared with the normothermic groups at 12 months post-injury.

Despite the encouraging results of these positive studies, other research failed to confirm these effects. In a multi-center study, Clifton and colleagues²² reported no significant improvement in outcome in patients treated with hypothermia (33°C). But they also noted that,²² although hypothermia may not be beneficial in patients who are normothermic on admission, maintaining hypothermia may be neuroprotective in patients who are hypothermic on admission. In another multi-center RCT of hypothermia (32–35°C) in 384 TBI patients with intracranial hypertension (ICP >20 mm Hg) from 18 trauma centers, Andrews and colleagues²⁵ reported no difference in outcome (6 months, GOS-Extended) between the normothermic and hypothermic patients, and the study was stopped early because of concerns about patient safety in the hypothermic group. Together, these results indicate that hypothermia does not improve long-term outcome and may be associated with a higher rate of complications.

Temkin and associates²⁶ reported significantly fewer seizures in the first 8 days post-injury among patients with severe TBI treated with phenytoin compared with a placebo-treated group. Unfortunately, these positive therapeutic effects were transient, and there were no significant differences in the number of post-traumatic seizures between the two groups from 8 days to 2 years post-TBI. Giacino and colleagues, in a RCT of 184 patients with severe TBI,²⁷ reported that amantadine improved the rate of functional recovery 2 weeks after severe TBI. Although the effect was transient and there were no significant differences between the treated and placebo groups 6 weeks post-TBI, even transient accelerated recovery is a somewhat positive effect. Harders and co-workers²⁸ reported that a group of TBI patients with subarachnoid hemorrhage treated with nimodipine (intravenous and oral for 3 weeks) had significantly fewer unfavorable outcomes (GOS, 6 months) than matched patients treated with placebo. Although these results were encouraging, subsequent studies failed to confirm the positive findings.²⁹ The remaining positive RCT was a single-center trial of progesterone in patients with severe TBI.³⁰ Xiao and colleagues³⁰ reported that progesterone, administered within 8 hours of injury, resulted in significantly better dichotomized GOS scores 3 and 6 months post-injury. Functional Independence Measure (FIM) scores also were significantly better in the progesterone-treated compared with placebo-treated patients 3 and 6 months after injury. Although subsequent studies have failed to confirm the positive results of Xiao and colleagues³⁰ (for a review, see Ma and colleagues,³¹ and Schumachera and associates³²), there is an ongoing debate about the limitations of some of these studies (see Goldstein and associates,³³ and Stein and co-workers³⁴). In summary, few, if any, RCTs have demonstrated unequivocal, lasting benefits of surgical/management or pharmacological therapies for TBI.

Why Have Promising Therapies Failed in Clinical Trials?

Potential reasons for the failure of promising candidate therapies to translate to successful RCTs and clinical benefits in TBI patients are legion. A thorough discussion of these issues is beyond the scope of this article, but they have been addressed in detail in several excellent reviews.^{9,10,13–15,32,35,36} Briefly, the reasons why promising therapies may have failed in clinical testing can be divided into the limitations inherent in pre-clinical testing, and those associated with the clinical trials themselves.

Limitations in pre-clinical testing

TBI in humans is complex and heterogeneous and patients with many different types of TBI (e.g., epidural or subdural hemorrhage, focal contusions, diffuse swelling, diffuse axonal injury [DAI]) may have similar Glasgow Coma Scale (GCS) scores on admission.^8,31 It is likely and reasonable that therapies that show pre-clinical efficacy in experimental models replicating focal TBI might not necessarily prove equally effective in patients with, for example, diffuse TBI. For this reason, it would seem necessary that pre-clinical studies be conducted in models that replicate diffuse TBI (e.g., impact acceleration), focal TBI (e.g., controlled cortical impact, [CCI]), and other models that produce a more mixed phenotype (e.g., fluid percussion injury, [FPI]). (For a review of experimental TBI models, see Johnson; Xiong; Crnak; DeWitt; and Morales,and colleagues, respectively^37–41.) The STAIR (Stroke Therapy Academic Industry Roundtable) criteria recommend that therapies be tested in pre-clinical stroke studies in multiple animal models, including non-human primates, as well as in both sexes.^42,43 However, despite representing a seemingly logical strategy, implementation of the STAIR criteria in pre-clinical stroke research has not led to translational success in clinical pharmacological neuroprotection.

Doppenberg and colleagues⁴⁴ suggested the following requirements for RCTs for therapies for TBI:

• 1.The mechanism on which the drug or procedure acts should be demonstrated in animal models of TBI.
• 2.This mechanism should be blocked in these models by the studied drug or intervention.
• 3.The mechanism should be demonstrated in human TBI.
• 4.The brain penetration of the drug must be adequate to affect the mechanism under study.
• 5.The safety and tolerability of the drug must be shown in human head injury.

To meet the first three requirements, pre-clinical and clinical studies must be designed to include measurements to confirm that the same targets of therapy are present, and the same mechanisms are active. Unfortunately, this is rarely the case in either clinical or experimental studies (see Tables 1 and 2 in Agoston and colleagues,⁹ and Box 1 in Stocchetti and colleagues³⁶).

Table 2.

Potential Peasons for the Failure of Clinical Trials for Therapies for TBI: Reasons Related to Pre-Clinical Testing

Reason	Summary	References
Translational issues	Inadequate understanding of differences in therapeutic window, pharmacokinetics, dosages, etc., between humans and experimental animals	Hawryluk et al.15; Margulies and Hicks121
Multiple injury models	Limited evidence of efficacy in different species, genders, ages, etc.	Hawryluk et al.15; Margulies and Hicks121
Injury mechanisms	Mismatch between primary injury mechanisms in humans and experimental animals (e.g., diffuse vs. focal, DAI)	Hawryluk et al.15; Margulies and Hicks121; Saatman et al.8
End-points	Differences in types of end-points in clinical vs. experimental studies	Agoston et al.9
Sample size/power	Inadequate sample size to produce adequate statistical power	Button et al.77
Time intervals	Experimental studies measure acute effects (<1 month); RCTs measure long-term (>1 month) outcomes	Agoston et al.9
Inadequate scientific rigor/Inadvertent bias/Confounders	Inadequate blinding, selective analyses, reporting bias, randomization, attrition	Holman et al.158; Fanelli et al.159; Vogt et al.160; Tsilidis et al.161; O'Collins et al.162

Modified from: Hawryluk et al.¹⁵; Margulies and Hicks¹²¹.

DAI, diffuse axonal injury; RCT, randomized clinical trial.

In addition to these limitations related to mechanisms and pharmacological properties, there are large discrepancies between the clinical and pre-clinical outcomes of the studies. Imaging (e.g., computed tomography [CT] and, to a lesser extent, magnetic resonance imaging [MRI]) are widely used in clinical studies as a source of diagnostic information but, until recently, imaging has been relatively rare in experimental TBI studies.⁹ However, an increasing number of pre-clinical studies have successfully implemented serial in vivo imaging to track progressive changes in rats after TBI with or without treatment.^45,46

In human TBI studies, injury severity is defined by the GCS, a scale of neurological function based on levels of consciousness.^47,48 Neurological function scales for rodents^49,50 are used in some experimental studies but, more frequently, injury level is defined by the impact parameters (e.g., pressure pulse magnitude, in atmospheres, for FPI; impact depth and velocity for CCI). In addition to basic neurological function, clinical studies typically involve assessments of neuropsychological/ cognitive/behavioral outcomes. Cognitive function (typically spatial memory) testing has become relatively common in experimental TBI studies. Other behavioral changes relevant to TBI, including anxiety, sociability, impulsivity, and depression, that were once seldom measured in pre-clinical TBI studies now are recognized as important outcomes. However, there is still a large gap in the frequency of behavioral testing between human and animal TBI studies.⁹ Post-traumatic epilepsy is another common and serious consequence of TBI that is rarely incorporated into pre-clinical TBI studies, which is likely related to the long-term recovery and extensive electrophysiological monitoring required for the study of post-traumatic epilepsy.^51–53

Moreover, in most RCTs, behavioral outcomes are measured 3 to 12 months post-TBI, whereas long-term (> 1 month) assessments of behavioral or cognitive function are seldom included in experimental TBI studies (Table 2).⁹ The effects of TBI on memory function have been studied for up to 1 year after TBI in rats^54–56 (for a review of chronic behavioral studies, see Osier and associates⁵⁷). Bramlett and Dietrich^58–61 and Smith and colleagues⁶² reported histopathological changes up to 12 months after FPI, and Dixon and colleagues⁵⁴ for up to 12 months post-CCI. Kochanek and associates reported changes in CBF up to 12 months after CCI.⁶³ These expensive and labor-intensive studies contributed important information about the long-term changes associated with experimental TBI, but they are unusual because most pre-clinical research focuses on the acute, short-term effects of TBI.⁹ Considering the recognition that even mild TBI often has long-term consequences,⁵ it is important that pre-clinical TBI studies include evaluations of the chronic behavioral and histopathological effects.

Finally, another important potential factor that may contribute to failure of translation is insufficient rigor in pre-clinical studies. This can contribute to overstated effects and has been shown in other fields such as cancer research to generate irreproducible results and failed clinical translation.^64,65 Estimates of the numbers of irreproducible pre-clinical studies have ranged from ∼40% to as high as 80%.⁶⁶

Limitations in clinical testing

One of the potential reasons for the failure of RCTs is inadvertent bias/confounders (Table 3). Corrigan and colleagues⁶⁷ observed that follow-up bias can occur in patient populations with lower socioeconomic status or in those patients who sustained a TBI due to violence or had a history of drug abuse. Additionally, based on a survey of 24 meta-analyses of RCTs, Page and co-workers⁶⁸ observed that the effects of interventions may be exaggerated by the lack of adequate blinding of investigators, especially those assessing outcomes, and inadequate randomization sequence generation. However, this would not explain the failure of translation—rather, it would produce false-positive clinical trials. It is important to note that inadvertent bias is a significant potential contributor to inaccuracy in pre-clinical studies as well.

Table 3.

Potential Reasons for the Failure of Clinical Trials for Therapies for TBI: Reasons Related to Clinical Trial Design and Methods

Reason	Summary	References
Single center	RCTs conducted in single centers may be less reliable than multiple center RCTs	Bafeta et al.69; Bellomo et al.70
Follow-up bias	Patient selection bias may occur due to loss of follow-up in some patient populations (low socioeconomic status, substance abuse, violent injury mechanisms)	Corrigan et al.67
Randomization/Blinding	Intervention effect estimates may be exaggerated due to inadequate or unclear random sequence generation or blinding, especially with subjective end-points	Bragge et al.14; Page et al.68
Sample size	Inadequate sample size to produce adequate statistical power	Burke et al.13; Bragge et al.14; Button et al.77
Therapy dose selection	Translation of doses used in pre-clinical studies should be based on physiological, pharmacokinetic, and toxicology data	Blanchard and Smoliga79
Patient selection	Select patient populations most likely to be helped by therapy	Hawryluk et al.15; Narayan et al.35
Outcome errors	Misclassification of outcomes (e.g., GOS score) can significantly reduce effect size	Lu et al.85
Inadvertent bias	Pressures to publish, industry bias, positive results, government vs. industry support	Fanelli et al.159; Dwan et al.163; Fanelli et al.164; Easterbrook et al.165

DAI, diffuse axonal injury; GOS, Glasgow Outcome Scale; RCT, randomized clinical trial.

Modified from: Hawryluk et al.¹⁵; Margulies and Hicks¹²¹.

Another potential explanation for the disappointing results of TBI RCTs is the frequency of single-center trials. In a report of the results of meta-analysis of RCTs of therapies in patients with moderate to severe TBI up to 2015, Bragge and colleagues¹⁴ observed that three quarters of the 207 RCTs analyzed were single-center trials. Although single-center RCTs are less expensive and easier to organize and coordinate than multi-center trials, and data collection and analysis are more straightforward, the results often prove hard to replicate.^14,69,70 Maas and associates⁷¹ observed that most of the trials that reported beneficial effects of the therapy tested were single-center RCTs. One example is the single-center RCTs that found hypothermia to be beneficial in patients with severe TBI.^21,23,24 Although these studies appeared to be appropriately designed, randomized, and blinded, the results could not be confirmed in follow-on, multi-center RCTs.^22,25 Similarly, two single-center RCTs showed decreased mortality and improved functional outcomes with progesterone,^30,72 results that were not confirmed in large multi-center clinical trials.^73,74

A related issue is that many RCTs may be underpowered. A clinical trial with fewer than 100 patients may overestimate⁷⁵ or underestimate⁷⁶ treatment effects. Although the direction of the effect may vary, these and other reports^13,77 indicate that small RCTs may yield different results than larger studies of the same therapies.

Therapy dose selection is another important issue. Scaling drug doses from small experimental animals to humans is not as simple as multiplying the mg/kg dose in an animal by the patient's body weight. The U.S. Food and Drug Administration (FDA) recommends that drug doses for humans be scaled using a calculation based on body surface area (BSA)⁷⁸:

where HED = human equivalent dose, AD = animal dose (mg/kg), Km = weight (kg) divided by BSA (m²). The use of a simple body-weight conversion would result in a dose increase of 200-fold for a 60-kg human, whereas dose conversion using BSA would increase the dose by about 65-fold. Although the BSA conversion has been widely used for many years, Blanchard and Smoliga⁷⁹ recently reviewed the evidence that scaling based on BSA also is inadequate due to the evidence that body structures and function that affect drug outcome do not scale only by BSA or body mass. Use of computational modeling of the key factors affecting drug response (known as allometric scaling) across more than one species is the more accurate method for estimating human equivalent dosing.^80,81 Although there is added expense in the assessment of multiple species, such methods improve the accuracy of estimation of key factors of drug disposition.

Granted the majority of RCTs have been conducted in patients with moderate to severe TBI,¹⁴ as of 2013 there have been at least 71 RCTs testing therapies for concussion (i.e., mild TBI)¹³ and several RCTs included patients with mild TBI along with moderate and severe cases.^25,82,83 The inclusion of patients with multiple levels of TBI severity facilitates recruitment and increases the likelihood of acquiring a large patient population. However, one of the postulated reasons for RCT failure is the inclusion of patients who are “too good” or “too bad” to be helped by the therapy tested.^15,35 In some cases, post hoc sub-group analysis in negative RCTs revealed sub-populations of patients that appeared to have benefited from the therapy tested. As noted above, Clifton and colleagues²² suggested that maintaining hypothermia may be beneficial in TBI patients who were hypothermic on admission. To date, this hypothesis has not been tested.

One of the most important aspects of clinical trial design and execution is the selection and accurate assessment of outcomes. A widely used outcome in TBI RCTs is the GOS.¹⁴ The GOS divides outcomes into five categories: Good Recovery (GR); Moderate Disability (MD); Severe Disability (SD); Vegetative; and Death.⁴⁷ To more accurately define recovery, Jennett and colleagues⁴⁸ expanded the original five-category GOS to the eight-category GOS-Extended (GOSE or GOS-E) by adding two levels each to GR, MD, and SD for conscious patients (e.g., MD ≥ MD upper, MD lower; GR ≥ GR upper, GR lower, etc.). The GOS-E provides better differentiation of recovery levels, especially in patients with mild or moderate TBI. Even though the GOS-E may be more subjective, it correlates more closely with other disability ratings, neuropsychological tests, etc., than the original GOS.⁸⁴ However, there have been significant inter-rater differences and misclassifications with the GOS and GOS-E.^84–87 Lu and associates⁸⁵ observed that errors in classifying patients with the GOS could reduce the power of detecting a treatment effect, thereby underestimating the true efficacy of therapy. Further, neither of these scales is adequate to assess more subtle symptoms associated with mild TBI. These limitations inspired the Traumatic Brain Injury Endpoints Development (TED) project that aims to develop and validate novel biomarkers and outcome measures for clinical TBI monitoring.⁸⁸

Operation Brain Trauma Therapy (OBTT): A Novel Approach to Pre-Clinical Testing

OBTT is a consortium of investigators at the Safar Center at the University of Pittsburgh, the Walter Reed Army Institute of Research (WRAIR), the Miami Project to Cure Paralysis at the University of Miami, Virginia Commonwealth University, the University of Florida, Messina University, and Banyan Biomarkers that is supported by the U.S. Department of Defense.^11,89,90 Although OBTT is novel in pre-clinical TBI research, the Multi-center Animal Spinal Cord Injury Study (MASCIS) was a similar multi-center collaboration decades earlier to test the efficacy of therapies for spinal cord injury.⁹¹ Financial support for MASCIS was provided by the U.S. National Institute of Neurological Diseases and Stroke (NINDS). Whereas personnel from each of the participating MASCIS centers were trained to use the same weight-drop spinal cord injury model and the Basso, Beattie, Bresnahan (BBB) locomotor function scale,⁹² the OBTT centers used different TBI models, each center using the model with which it had the most experience (i.e., FPI, Miami; CCI, Pittsburgh; penetrating ballistic-like brain injury [PBBI], WRAIR)^11,89 for the express purpose of addressing the marked anatomical heterogeneity of clinical TBI. All of the OBTT centers used the same drug doses and treatment regimen, assessed the cognitive effects of therapy using the Morris water maze (MWM), collected serial blood samples for biomarker measurements, and harvested the brains for histopathological analyses 21 days post-injury. However, there were some differences in procedures among the OBTT centers. For example, vestibular/motor function was measured using cylinder and grid walk tasks by the Miami group, beam balance and walking tasks by Pittsburgh, and a rotating rod task by WRAIR.⁸⁹ Another significant difference among centers was the level of injury. Pittsburgh and WRAIR used severe levels of CCI and PBBI, respectively, whereas the Miami group used a moderate level of FPI. Details of these and other similarities and differences among the OBTT center have been published.^11,89,90

To date, the five therapies that have been tested by OBTT are erythropoietin,⁹³ cyclosporine,⁹⁴ levetiracetam,⁹⁵ simvastatin,⁹⁶ and nicotinamide.⁹⁷ All of the drugs tested by OBTT were FDA-approved for human use, and all had been effective in experimental TBI models in which therapy was initiated post-injury. Of the five drugs tested, only levetiracetam significantly improved outcome in more than one model. Both doses of levetiracetam improved MWM performance after FPI, whereas the lower dose (54 mg/kg) improved MWM performance after CCI. In the CCI model, the higher dose (170 mg/kg) improved performance on the beam balance task and reduced hemispheric tissue loss. Levetiracetam therapy had no significant effect after PBBI, the most severe TBI model used by OBTT.⁹⁰ Based on this demonstration of efficacy in two TBI models, levetiracetam is being tested in a micro-pig FPI model with the second round of therapy testing now in progress. Among other therapies, OBTT is testing glibenclamide and Kollidon VA64.⁹⁰ Glibenclaide (a.k.a. glyburide), a sulfonylurea used to treat diabetes, improved spatial memory and reduced hippocampal injury after CCI in rats.⁹⁸ Kollidon VA64, a poloaxmer that aids in cell membrane resealing, reduced neuronal degeneration, BBB damage, and brain edema and improved motor function in mice after CCI.⁹⁹ These studies have been completed but the results have not yet been published. Other ongoing studies by OBTT include testing of amantadine, the aquaporin antagonist AER-271, minocycline, the protease inhibitor E64d, and the nicotinamide phosphoribosyltransferase activator P7C3-A20.

Alhough the results of the first set of therapy testing are somewhat disappointing, they do not diminish the remarkable achievement of the OBTT investigators, including unique elucidation, characterization, and comparison of multiple, gold standard pre-clinical TBI models using conventional outcome metrics and novel candidate biomarkers of brain injury as well as setting standards for methodological rigor.^90,94,97,100

Despite its substantive strengths, the OBTT approach has some limitations that should be acknowledged. The time of drug administration (15 min post-TBI) would be impractical in TBI patients. Also, the drug doses and times and routes of administration may not have been optimal, and the OBTT studies weren't designed to test mechanisms or to confirm that the therapies acted on their intended targets, if known. These and other potential flaws have been thoughtfully considered by the OBTT investigators⁹⁰ The work of OBTT also suggests that some of the reasons for failure of translation of therapies may lay within the pre-clinical realm, further emphasizing the need for rigorous, multi-center, pre-clinical approaches to therapy development. OBTT is an excellent example of the type of pre-clinical testing using multiple TBI models and clinically relevant outcome variables that is likely to identify therapies that will prove effective in clinical trials in patients with different types of TBIs.

Moody Project TBI Symposium

The Moody Project TBI Symposium, held in Galveston, TX, in September 2016, was intended to identify potential limitations and gaps in current experimental TBI therapy testing and suggest guidelines to overcome some of these limitations and improve the translation of promising therapies to successful clinical trials in TBI patients. Investigators with extensive experience with pre-clinical testing were invited to participate and address a series of questions (Table 4) related to model and therapy selection, outcomes, and data collection and analysis. The invitees included physicians, pharmacologists, physiologists, biomedical engineers, and animal behaviorists from the United States, Europe, and Australia.

Table 4.

Moody Project Symposium Discussion Topics and Questions

Topic	Questions
Model and therapy selection	Models
	1. Should multiple models be used for therapy testing? If so, how many and which ones?
	2. Should models be standardized across sites?
	3. Injury severity? Mild, moderate, severe? Repetitive mild?
	4. Should studies include both genders?
	5. Should anesthetics be constant across sites?
	6. Should secondary (post-traumatic) insults (e.g., hypotension, hypoxemia) be included?
	Therapies
	1. Therapy selection? FDA approved? Human use approval? BBB permeability?
	2. Should PD/PK be established prior to efficacy testing?
	3. Time/route of administration? How many doses?
	4. Should the same doses and routes and times of administration be used in all models?
Outcomes	1. Which ones?
	2. How should outcomes be weighted in “efficacy score”?
	3. Should outcomes be standardized across sites?
	4. At what time after injury should outcomes be assessed?
	5. Long-term (> 21 days) outcomes?
	6. Serum biomarkers? Which ones? Should biomarker data be included in the efficacy score?
	7. Should mechanistic outcomes be included to confirm/refute theoretical mechanisms(s) of action?
	8. Should outcomes be standardized and/or normalized across sites?
	9. Which outcomes must be positive to recommend a therapy for clinical testing?
Data collection, analysis, sharing and intersite communication	1. Should common data elements be “common” across all sites?
	2. How should the outcome data be analyzed?
	3. Should the data be shared outside of the consortium?
	4. Should data be analyzed by each site or at a data analysis core?
	5. Should studies include Go/No-Go preliminary analyses or short-term outcome screening?
	6. Should studies include an interim analysis?
	7. Should data be shared among sites while collection is in progress?
	8. What types of intersite communication should be permitted/encouraged?

BBB, blood–brain barrier; FDA, Food and Drug Administration; PD/PK, pharmacodynamics /pharmacokinetics.

The participants were divided into three groups and each group discussed one of the broad topic areas at each of the three topic discussion sessions, thereby providing all groups the opportunity to discuss all of the topics and questions (Table 4). Although overall study design was not one of the topics, it was discussed by all of the groups and so it is included as a separate section in this consensus summary. Many of the study design recommendations are specific to multi-model, multi-center studies such as OBTT, whereas others would be applicable to pre-clinical therapy testing in general.

Study Design Recommendations

Large, multi-center pre-clinical studies could be patterned after RCTs, with Stage 1 studies designed for relatively simple, efficient screening using multiple, well-established TBI models. Stage 1 studies would be relatively short (2–3 weeks) with simple behavioral and histopathological outcomes. Stage 1 studies would be preceded by preliminary measurements of pharmacokinetics/pharmacodynamics (PD/PK) and brain penetrance of the therapy to aid in dose selection and route and time of administration. If reliable, species-specific information was available from previous studies, this step could be omitted.

Therapies that improved outcome in Stage 1 screening could advance to Stage 2 trials that might include both sexes, young and old animals, and more numerous, mechanistic outcomes. Stage 2 studies also could include physiological measurements (e.g., CBF, ICP, BBB permeability, microdialysis measurements of transmitters, metabolites), detailed assessments of PD/PK and toxicity, and post-traumatic insults such as hypotension, hypoxia, anemia, and/or polytrauma. An alternative approach would limit the Stage 1 screening studies to single sexes, ages, and injuries, and encourage the formation of additional multi-center consortia that would focus on more complex issues such as the chronic effects of TBI, TBI model development and characterization, mild TBI and/or repetitive mild TBI, combination therapies, etc.

One important requirement of OBTT is the use of a detailed manual of standard operating procedures (MSOP).⁸⁹ The symposium participants agreed unanimously about the importance of the creation and constant updating and revising of the MSOP. The diligent use of accurate, detailed MSOPs ensures that all methods and procedures are consistently and rigorously used among all participating centers and, more importantly, facilitates the replication of the research by other investigators. This is especially important in view of the doubt about the reliability of much scientific data.^77,101–103 When the biotechnology firm Amgen tried to replicate 53 “landmark” pre-clinical cancer studies, it was able to confirm the results in only six (11%).⁶⁴ Begley and Ellis⁶⁴ noted that the authors of the studies that were confirmed provided detailed information about controls, reagents, blinding, and investigator bias.

Additional reasons for unreliable data or questionable results include inadequate sample size⁷⁷ and errors in data collection and analysis.¹⁰² In a study that supported the importance of a detailed MSOP, Carp reported¹⁰³ that, of the 241 functional MRI studies examined, fewer than half provided important information about data collection (e.g., number of subjects not analyzed, the reasons for rejection, whether or how the subjects were compensated, etc.) or methods (e.g., corrections for differences in slice acquisition timing or co-registration to high-resolution scans, whether temporal filtering was conducted, etc.). For a pre-clinical therapy study to be appropriately replicated, transparent reporting and accurate details about the TBI models, surgical procedures, outcomes and data collection, and analysis are required, and those details would be available only with a meticulous, up-to-date MSOP. Further, pre-clinical screening studies, either individual or multi-center, often take months to years to complete and during that interval personnel and/or facilities may change, increasing the likelihood that minor procedural details will vary, either intentionally or inadvertently. Regular, routine updates of the MSOPs will help to minimize the effects of these variations on data reproducibility. These and other recommendations about study design and conduct are included in the RIGOR guidelines from the NINDS.¹⁰⁴

Go/No-Go decision points

Go/No-Go decision points are built into many screening studies to limit futile testing of therapies that prove unlikely to produce positive results. The Go/No-Go decision point in the OBTT studies occurred at the end of first-stage testing when levetiracetam, which improved MWM performance in two TBI models, was approved for further study (Go) but erythropoietin, cyclosporine, simvastatin, and nicotinamide were not (No-Go) because they failed to improve outcomes in more than a single TBI model.⁹⁰ Because the survival interval for the OBTT experiments was 3 weeks, the screening of the five agents took years. A test that predicted the 3-week outcomes but could have been conducted 24 h, or even a week post-injury, would have saved a considerable amount of time, money, and effort. To date, no early test (or method, procedure, task, etc.) that accurately predicts outcome exists but there are some that show promise.

Fluoro-Jade (FJ) is a fluorescent stain that identifies degenerating cells as early as 4 h after injury,^105–107 although there is evidence that FJ-positive cells may be injured but not necessarily dying,¹⁰⁸ and FJ may stain non-neural cells (e.g., activated microglia, astrocytes) under some circumstances.¹⁰⁹ FJ is a potentially valuable screening tool to assess the neuroprotective effects of therapies soon after injury. However, it is unclear whether cell death represents the key target for therapies even in patients with severe TBI, or should be seen as the “one target” for therapies. For example, mechanisms such as axonal injury, cerebral edema, or even cell signaling, among others could be critically important. The key target may also depend on the TBI phenotype, injury severity, timing, and/or presence of secondary extracerebral insults.

Perhaps the most promising early indicators of injury and therapeutic efficacy are serum biomarkers. A component of the OBTT consortium, Mondello and colleagues¹⁰⁰ demonstrated that serum levels of glial fibrillary acidic protein (GFAP) sampled 4 h after all three OBTT TBI models (i.e., FPI, CCI, PBBI) correlated with lesion volume and cortical tissue loss 21 days after injury. Four-hour serum levels of GFAP also correlated significantly with MWM latency to goal platform after FPI but only weakly with MWM performance after CCI or PBBI.¹⁰⁰ These data provided clear evidence that serum GFAP level measurements sampled 4 h after TBI could represent a valuable surrogate histological and/or theranostic end-point that might also serve an ideal tool for screening of therapies and, perhaps, Go/No-Go decisions very soon after injury. That approach (or other biomarkers) clearly warrants future studies of higher throughput screening in pre-clinical TBI models. GFAP was recently “approved” by the FDA as a biomarker for mild TBI.

Despite the promise of methods such as FJ and serum biomarkers for early decisions about therapeutic efficacy, the desirability of very early screening is unclear. As noted above, the time courses of the pathophysiological mechanisms of TBI vary widely over hours to days post-TBI. Early screening would markedly underestimate the efficacy of agents that required repeated or delayed administration. Restorative therapies would not be expected to act within hours or even days or weeks post-TBI. The inclusion of Go/No-Go decision points and the timing and methods used for those decisions should include consideration of the time courses of the pathophysiological processes of the study injury models and the PD/PK and putative mechanisms of action of the test therapies in the test species.

Intersite communications

The OBTT consortium design includes monthly teleconferences among the site investigators. These calls provide opportunities for progress updates and discussions about problems and/or questions about models (e.g., excess mortality, unexpected morbidity, outcomes [unexpected behaviors or injury or drug effects]) and other procedural issues. To minimize the risk of inadvertent bias, results are not discussed until all centers have completed outcome data collection. Although questions and concerns of the laboratory personnel are addressed in the monthly calls, the personnel conducting the experiments are not included to preserve blinding.

Experimental Models of TBI

Evaluations of the pro and cons of myriad existing TBI models and recommendations for the “best” TBI models were beyond the scope of the symposium discussions, and also of this review article. These issues have been thoroughly covered in numerous other review articles.^37–41,110 It is worth noting that all animal models have limitations and only replicate certain features of clinical TBI, which is heterogeneous in nature.

Chronic traumatic encephalopathy (CTE)

Although TBI model development and characterization were not goals of OBTT and Stage 1 therapy testing studies are not well suited to these tasks, the need for an experimental TBI model that reliably replicated significant histopathological and behavioral features of CTE was recognized. The correlation between repeated (usually mild) TBI and chronic or late-developing cognitive dysfunction has been recognized for decades. Punch drunk and dementia pugilistica were terms used in the 1920s and 1930s to describe the long-term effects of TBI.^111,112 The dementia related to repeated mTBI (i.e., CTE) is believed to be due to an excessive accumulation of the microtubule-associated protein tau.^113–115 Goldstein and colleagues¹¹⁶ observed histopathological evidence of phosphorylated tau deposition in the brains of military personnel who had sustained blast-induced TBI, suggesting that blast injury may contribute to CTE. Currently, there is no single experimental model that produces all CTE-like behavioral and pathological effects, but some of these effects can be replicated in rodent TBI models. Goldstein and colleagues¹¹⁶ reported histological evidence of CTE-like neuropathology including tau protein–linked immunoreactivity, persistent perivascular pathology, degeneration of cortical and hippocampal neurons, and chronic inflammation (i.e., astrocytosis and microgliosis) in the brains of mice 2 weeks after simulated blast (compressed gas-driven shock tube). Shock-wave–exposed mice also exhibited deficits in spatial learning and memory, as measured using the Barnes maze.¹¹⁶ However, it is important to note that most of the CTE-like histopathological and behavioral effects of shock-wave–exposure in mice were prevented by immobilizing the animals' heads, thereby preventing the rapid oscillations produced by the shock wave. Thus, rather than a model of blast-induced CTE, Goldstein and colleagues¹¹⁶ may have created a model of CTE produced by rapid, repetitive head acceleration/deceleration.

Certain CTE-like neuropathological changes also can be replicated by CCI. Glushakova and associates¹¹⁷ reported BBB damage, perivascular microbleeds, and white-matter damage that progressed for at least 3 months after CCI. BBB damage,¹¹⁸ perivascular microbleeds, and deep white-matter damage often are observed in humans after repeated, mild TBI.¹¹³ Briggs and co-workers¹¹⁹ reported CTE-like neuropathological changes (e.g., increases in phosphorylated tau, TDP-43, neuroinflammation, reductions in white-matter volume) 53 days after the last of 30 mild, weight-drop TBIs (over 6 weeks) in mice. They also observed deficits in spatial memory acquisition (Barnes maze) and rotarod performance. The authors stated that the TBI levels used were mild, based on duration of righting reflex suppression, but they reported mortality rates of about 20%, a degree of lethality that would not be expected in mild TBI models. In an excellent recent review of CTE and tau, Ojo and colleagues,¹²⁰ observed that, of the 29 studies of tau levels after TBI in rodents, 23 reported increased tau and six reported no change in tau. Of the 16 tau-positive murine studies, eight used genetically modified mice. The most substantive limitation of most current rodent models of CTE is that they fail to replicate typical neuropathological characteristics of CTE in humans such as neurofibrillary tangles, astroglial tangles, neuropil threads, or peri-vascular or peri-ventricular p-tau.¹²⁰ An important consideration in the development of rodent models of mild TBI and CTE is the potentially confounding effects of anesthetics on neuronal injury, CBF, neurotransmitter signaling, etc.¹¹⁰

Multiple rodent models

The OBTT consortium used moderate FPI, severe CCI, and severe PBBI TBI models because these models replicated many important features of diffuse, focal, and penetrating TBI, respectively, in humans.^11,89 An alternative would be the use of the same model by multiple centers, an approach attempted in the MASCIS spinal cord injury study.^90,91 One of the primary advantages of the MASCIS approach was inherent intersite replication of results. However, because human TBI is a heterogeneous insult,⁸ testing of therapies in rodent models that replicate multiple features of different types of human TBI increases the likelihood that therapies that show promise in different types of experimental TBI models also will do so in heterogeneous human TBI. This is the rationale for the previous recommendations for therapy testing in multiple models in stroke (i.e., STAIR criteria)⁴² and TBI.^9,15,121

Large animal models

Compared with rodents, large-animal TBI studies are resource-intensive in terms of money, space, personnel, etc. Despite these considerations, large-animal studies provide unique contributions to TBI research. The cerebral cortices of rats are lissencephalic, lacking gyri and sulci, whereas those of humans, pigs, and other higher mammals are gyrencephalic and divided into gyri by sulci. The use of experimental animals with sulci may be especially important in modeling CTE because the tau pathologies appear to be preferentially localized in the depths of sulci in human CTE.^113,122 There also are significant differences in brain weights, volumes, and surface areas between rats,¹²³ pigs,¹²⁴ and humans.^125,126 The ratios of white to gray matter differ considerably among species and, in general, the volume of white matter increases with the size of the brain due to the need for longer fibers with increasing brain size.¹²⁷ There are major differences in the anatomy of the skull between pigs and rats.¹²⁸ These differences in the anatomy of the brains and skulls in the different species used in research should be considered in the interpretation of experimental results, especially in studies of blast TBI.

The OBTT design included plans to test therapies successful in multiple rodent models using the FPI model in micro-pigs.^11,89 To date, only levetiracetam has improved cognitive outcome in multiple rodent models (FPI and CCI), and further evaluations of levetiracetam in FPI in micro-pigs are ongoing.⁹⁵ As noted above, the importance of large animals in therapy testing is well recognized.^{8,15,42,110,121} but an additional consideration is that FDA approval for a new drug (i.e., Investigational New Drug; IND) or repurposing a previously approved agent may require proof of efficacy in large animals as well as rodents. Cell-based therapies almost invariably require data from large-animal studies (W. Dalton Dietrich, personal communication). Symposium participants who serve or had served on clinical trial study sections for the National Institutes of Health (NIH) commented that proof of efficacy in large-animal studies was crucial for NIH support for clinical trials. Finally, there is an expectation that, in view of the numerous failures of clinical trials for TBI therapies, the NIH is reassessing guidelines for NIH-supported clinical trials with a goal of aligning NIH guidelines with those of the FDA.

Therapies

Therapy selection

The therapies selected by OBTT were FDA-approved agents that had been shown to improve behavioral and/or histopathological outcomes in experimental TBI studies.⁸⁹ Although a higher priority was assigned to therapies supported by a large body of experimental evidence or those that demonstrated very impressive results in a few or even a single study, novel therapies with significant theoretical potential but limited track records also were considered.¹¹ Several other considerations were suggested by symposium participants (Table 5).

Table 5.

Potential Therapy Selection Criteria

Criterion	Rationale
FDA approval	Therapies approved for human use more readily translate to clinical studies
Efficacy	Therapies with efficacy in previous experimental TBI studies are more likely to prove successful in further pre-clinical testing
PD/PK in species	Facilitates dose and time and route of administration selection
BBB permeability	Increases likelihood of achieving therapeutic levels in brain
Human use	Therapies with long histories of use in humans are less likely to have significant negative side effects
Known targets/Mechanisms	Facilitates monitoring of dose and time/route of administration efficacy; therapies with multiple targets/mechanisms may act on multiple deleterious pathways
Novelty	High risk/high reward

BBB, blood–brain barrier; FDA, Food and Drug Administration; PD/PK, pharmacodynamics/pharmacokinetics.

Doses and routes of administration

Ideally, pre-clinical studies would use a range of drug doses/concentrations to generate complete dose/response information. This approach often is impractical for in vivo studies, especially in a large, multi-model consortium such as OBTT. Most investigators use a single dose or, like OBTT, low and high doses based on information from previous studies in the same species and consultation with faculty experts in PD/PK.¹¹ Similarly, decisions about route of administration typically are based on previous research and/or the BBB permeability of the therapeutic agent. Pre-clinical screening studies should use routes of administration that translate easily to patient studies (e.g., intravenous). In the case of agents with low brain penetrance, direct injections into the cerebral ventricles might be possible in patients with moderate/severe TBI with intraventricular ICP monitors/drains but not in those without such devices in place. Information about PD/PK in more than one study species is particularly valuable in determining doses, routes of administration, and times and durations of administration (see below). Such evaluations allow for estimation of the relationship between dose and both blood and brain tissue concentrations as they relate to observed neuroprotection and potential toxicity. Further, the relationship between concentration and a biomarker of target engagement with respect to the resultant measure of neuroprotection should be evaluated. Such studies should also evaluate different degrees of brain injury severity to ultimately determine the therapeutic window as a function of the severity of brain injury. However, these relationships are complex and may vary depending on myriad factors such as the therapy, the amount of BBB damage, and toxicity, among others.

Time and duration of administration

Decisions about the post-injury time of therapy initiation primarily are based on two considerations. Initial administration several hours post-TBI is more readily translatable to human TBI because of the time required for patient transport, admission, early diagnostic testing (e.g., examination; imaging, usually CT) and the acquisition of consent from the patient or the patient's family. In theory, safe, stable, easily administered therapies could be started by first responders but rarely is this feasible. In contrast, administration as soon as possible after injury is better suited to interrupt pathophysiological mechanisms that begin soon after injury. For example, poly(ADP-ribose) polymerase (PARP) activation increased as early as 30 min after FPI in rats¹²⁹ and excitatory amino acid levels increased within minutes after TBI in rats.¹³⁰ Neuronal injury due to these and numerous other secondary injury processes could be interrupted only by therapies administered within minutes of TBI. To increase the likelihood of detecting a positive effect, therapy was initiated in the OBTT trials 15 min after injury. The symposium consensus opinion was that therapies that showed promise when administered early could then be re-tested with delayed initiation of therapy. However, some deleterious secondary injury mechanisms may only operate in delayed periods after the injury, and thus the timing of therapy administration should be tailored to the mechanistic target. For example, in OBTT, testing of the drug simvastatin used a more delayed and chronic approach to dosing⁹⁶ based on prior literature and targeting of mechanisms such as regeneration and repair.^131,132 Indeed, a therapy that targets regeneration or rehabilitation might optimally be administered days, weeks, or even months after the injury.

Decisions about duration of therapy are best based on PD/PK information about the therapy in the study species and the time course of the pathophysiological mechanism(s) targeted. Therapies with short plasma or tissue half-lives may have to be administered repeatedly to be effective. Even if administered early, drugs with slow BBB penetrance might not reach therapeutic brain tissue concentrations for hours after administration. Knowledge of the time course of targeted mechanisms also is important to ensure adequate drug concentrations during the relevant time intervals. For example, lipid peroxidation and mitochondrial oxidative damage started at 24 h and peaked 72 h after CCI in rats.¹³³ Therapies started immediately after injury might be ineffective unless adequate tissue concentrations persisted for days after administration. Additionally, the duration of therapy should be related to injury processes in each experimental model. Both FPI⁶² and CCI⁵⁴ result in progressive neuronal damage for at least a year post-injury. Early treatment might interrupt the processes that result in chronic neurodegeneration, but it is also possible that long-term or delayed administration would be necessary.

Outcomes

As in every other aspect of experimental TBI research, perhaps the most important consideration in outcome assessments is the need to document all outcome measurement methods/procedures in meticulous detail. As evidenced by the OBTT consortium, there are different ways to conduct and analyze outcome measures, particularly behavioral tasks. The Miami group assessed spatial memory performance on post-injury days (PID) 13–16, and tested for retention of the hidden platform location with a probe (missing platform) trial on PID17 and for working memory on PID20–21, whereas the Pittsburgh group conducted spatial memory hidden platform testing on PID14–18, a visible platform test on PID19–20, and a probe trial on PID20.⁸⁹ A cursory examination of the experimental TBI literature revealed substantive differences in MWM procedures. Smith and colleagues¹³⁴ measured spatial memory function using an elegant system of overlapping ovals to measure the time to the goal platform in 20 trials over 2 days. Dash and colleagues¹³⁵ assessed spatial memory using the MWM with 2 days of training followed by a single day of probe trial testing. Dixon and colleagues⁵⁴ measured spatial memory acquisition and retention with r days of hidden platform testing, a probe trial to measure place memory, and 2 days of visible platform trials. The MWM procedures in these studies were described in appropriate detail, but without such careful descriptions, it would be impossible to replicate these studies. As noted above, scientists doing TBI research, as do all investigators, have a responsibility to document all methods and procedures, especially those related to models and outcomes, in sufficient detail to permit accurate replication.

Primary outcomes selection

The OBTT organizers elected to allow each center to use the outcome measures with which they had the most experience. All centers included primary outcomes that assessed biomarkers, motor and cognitive functions, and neuropathology, but the specific tasks in each category varied among centers.⁸⁹ Outcome variables that were the same across centers were the timing of blood sampling for biomarker analysis (4 and 24 h post-injury and at euthanasia), time of euthanasia (21 days), and an analysis of time to the goal platform and a probe trial in the MWM task.⁸⁹ The standardized procedures enabled comparisons across models and centers, whereas the individual methods provided the opportunity to detect treatment effects specific to the method of analysis. The primary consensus recommendation related to outcome selection was that the clinical relevance of the outcome be established. For example, the GCS is the primary determinant of state of consciousness and level of injury in TBI patients. The duration of righting reflex suppression, a reasonable measure of the level of consciousness in experimental animals,¹¹⁰ is a method currently used by several TBI laboratories.⁴⁰ The righting reflex is a mesencephalic reflex that returns before thalamocortical function during recovery from unconsciousness due to anesthesia or brain injury.¹³⁶ The return of the righting reflex is considered to be analogous to the return of consciousness in humans.^137–139 Spatial and working memory are aspects of cognitive function that are frequently tested in experimental TBI studies. Additionally, other types of higher cortical functioning that are assessed in TBI patients (e.g., inhibition/disinhibition, attention, executive functioning, problem solving) can be measured in rodents.⁴⁰ (For an excellent review of human outcomes and their animal equivalents, see Shultz and colleagues.¹¹⁰)

Secondary/supplemental outcomes

For pre-clinical therapy screening studies, secondary outcomes were defined as those that provide additional information but wouldn't be given high priority in decisions about the efficacy of the test therapies. The consensus recommendation was that screening studies should be as “stripped down” as possible to provide as much rigorous therapy testing as financial resources permitted. In-depth explorations of mechanisms, pathways, etc., would be reserved for further studies in successful agents. Imaging is frequently a component of clinical TBI studies but has been relatively rare in past experimental TBI research.⁹ However, as noted above, imaging is becoming more common in experimental TBI research.^45,53 Although MRI and other types of imaging may be a component of TBI clinical research studies, imaging typically is limited to CT in the acute care of TBI patients. Imaging may have a role in Stage 2 studies but was not considered to be especially useful in the determination of the success or failure of test drugs.

Common data elements (CDEs)

Recently, the NINDS undertook the development of a set of CDEs for TBI research that were intended to facilitate reporting, data sharing, comparison of results, and collaboration by standardizing the definitions and protocols for clinical TBI research.¹⁴⁰ This was followed by the formation of a set of CDEs for experimental TBI research.¹⁴¹ Symposium participants recommended the consideration of the NINDS CDEs in the selection of primary and supplemental outcome variables. This recommendation was made with the understanding that the current CDE lists¹⁴¹ should not be interpreted as excluding potential outcome measures. The CDE lists are evolving and variables not currently included can be added later.

Outcome weighting

The goal of pre-clinical screening is to determine whether a therapy warrants further study and/or translation to clinical studies. It is conceivable that screening studies, especially multi-center, multi-model studies, would yield mixed results with some outcome measures showing positive results but others showing no evidence of efficacy. OBTT uses a point system in which outcome categories contributed part of the total score. The motor (4), cognitive (10), neuropathology (4), and serum biomarker (4) categories are assigned scores that, if the test therapy improves outcome in all categories, total 22 points.⁸⁹ When designed, this scoring system provided a mechanism to assign priorities for further testing if all five of the therapies in the initial round of screening proved efficacious. Alas, prioritizing proved unnecessary but the system was felt to be reasonable. The consensus opinion was that assigning cognitive outcome a disproportionately high value is appropriate. In fact, the consensus opinion was that improvements in cognitive outcome, even in the absence of positive effects in any other outcome, would be sufficient to qualify a therapy for further testing.

Long-term outcomes

In experimental animals, TBI resulted in long-term deficits in histopathological and behavioral outcomes.^11,54,62 (For a review, see Osier and colleagues⁵⁷ and Bramlett and Dietrich. ⁶⁰) Patients sustaining a TBI, even a mild TBI, may suffer from neurological and behavior effects for years after injury^142,143 as well as a reduced life expectancy.¹⁴⁴ (For a review, see Masel and DeWitt.⁵) Despite this recognition of the lasting consequences of TBI, relatively few experimental studies include long-term end-points. An analysis of 193 “long-term” experimental TBI studies⁵⁷ yielded only 18, nine, and one with durations of 6 months or more, 1 year or more, and more than 1 year, respectively. These data suggest that less than 10% of all “long-term” experimental TBI studies lasted longer than 6 months and the percentage of all experimental studies with long-term outcome assessments is likely a fraction of a percent. Although there is a clear need for more experimental research on the long-term effects of TBI, assessments of the chronic effects of TBI are ill-suited for Stage 1 screening studies because of the extensive time, money, and personnel required. The consensus opinion is that the best approach is multi-center consortium studies designed specifically to characterize the chronic effects of TBI and identify potential therapies that might prove effective when administered weeks to months post-injury.

Data Analysis and Sharing

Data analysis

The consensus recommendation is to involve statisticians in all aspects of study design and data collection, collation, analysis, and sharing. A central data core is desirable in any multi-center effort. Individual center investigators and their statistician colleagues are capable of analyzing their own data, but a central core is well suited for data analysis across all sites. A relatively autonomous data core reduces the likelihood of the inadvertent bias that could occur if the overall data analysis was conducted by one of the participating centers. A central data core facilitates blinding during the across-center analysis. However, central data cores are an additional expense that might be viewed initially by the source of research support as an unnecessary luxury, making it necessary to establish a compelling rationale that justifies the expenditure.

Data sharing/dissemination

Data dissemination is one of the bedrock principles of research. In the neurotrauma field, data repositories have been created to facilitate the sharing of the results from multiple TBI studies. The International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT) database, which was supported by the NIH, included data from eight RCTs and three observational studies from three TBI centers (later expended to four) in Europe and the United States.^145,146 The Federal Interagency TBI Research Informatics System (FITBIR) is a database created by the U.S. Department of Defense and the NIH that comprises TBI patient data.^147,148 Presently, FITBIR is limited to clinical data but eventually will expand to include CDEs from pre-clinical studies.¹⁴¹ When FITBIR has the capability of accepting pre-clinical information, data from individual studies as well as multi-center consortia can be added. However, data sharing and dissemination might be delayed by the reluctance of investigators to share unpublished data. Typically, data are sequestered for 6 to 12 months before being widely disseminated.

In an important recent article about data sharing, Ferguson and colleagues,¹⁴⁹ observed that in addition to “big data” datasets, there exists a great deal of data collected in small datasets that the authors refer to as “ long tail” data, usually collected by individual investigators as small datasets. Although these small datasets are difficult to collect and disseminate, there is a clear need to do so because the long tail data together represent the majority of data collected by the neuroscience research community. Widespread access to long tail data is useful for discovery purposes but also for issues related to transparency and study replication. Reasons for not disseminating long tail data include concerns about the quality of these data and their use by non-experts, the cost of collecting and maintaining the database, and the potential negative impact of sharing unpublished data on the researchers' careers,¹⁴⁹ as well as the commercial interests of pharmaceutical companies. Whether these perceived limitations can be overcome remains to be seen.

A second and perhaps more challenging component of long tail data is so-called dark data from failed pre-clinical studies as well as ancillary data such as animal care records.¹⁴⁹ Virtually all TBI researchers have data from studies with negative results and most remain unpublished because publishing negative results is difficult. Researchers have a finite amount of time and most prefer to focus that time on data that are more readily reported. Although this is a reasonable preference, unpublished negative experiments might be repeated by others, thereby wasting the time and money of other researchers. In addition, publication bias toward positive studies, particularly in the absence of adequate rigor, can mislead other investigators to pursue unfruitful paths.^150,151 Because many of the negative studies and their unnecessary duplication were supported by government grants, this represents potentially billions of dollars of wasted public funds.¹⁵² Ferguson and colleagues¹⁵³ offer several options to facilitate the collection and dissemination of long tail data and their dark component. There was consensus agreement with Dr. Ferguson's observations, concerns, and recommendations that mechanisms be developed to collect and share all pre-clinical TBI data.

To facilitate data sharing, Ferguson and colleagues proposed that data should be discoverable, accessible, intelligible, assessable, and usable.¹⁴⁹ Discoverable accessible data can be readily located through a search and then easily interrogated. Intelligible data use standard nomenclature that can be read and understood by humans and machines. Datasets should be complete with sufficiently detailed information to allow assessments of reliability. Usable data would be formatted to facilitate use without extensive conversion. (For details, see Box 2 in Ferguson and colleagues.¹⁴⁹) A similar set of recommendations was incorporated in the FAIR (Findable, Accessible, Interoperable, and Reusable)^154,155 guidelines. Among other features, findable includes the assignment of a unique, persistent identifier and the registration of data in searchable resource. Accessible refers to the data being retrievable by their identifier using a standardized communications protocol that is open, free, and universally implementable. Interoperable means that the data use a formal, accessible, shared, and broadly applicable language for knowledge representation. The reusable principle means that the data are described with accurate and relevant attributes. (For details, see Table 2 in Wilkinson and associates.¹⁵⁴)

Concluding Statement

The major conclusion from the workshop was that we have identified and drawn attention to several needs, issues, and biases that plague the current experimental TBI research and literature, and we suggested potential steps to deal with them. We hope that the development of multi-institutional networks and consortia, similar to OBTT, will minimize many of these biases while optimizing resources and exploiting the different expertise. Such collaborations would allow easy, standardized, and harmonized data collection and access to information on experimental TBI studies. Further, rigor in designing and performing scientific research will ensure reproducibility and validated results facilitating unbiased evaluation of tested therapies, fewer false-positives, and, hopefully, more rapid identification of effective drugs. Support from funding agencies, regulatory bodies, and journal editors is essential and would greatly improve the chances for success.

Consensus Summary

Below are the consensus recommendations of the participants of the Moody Project TBI Symposium for pre-clinical testing of therapies for TBI.

Study design

• Multi-center pre-clinical screening could be categorized as Stage 1 or Stage 2 studies. Stage 1 studies would use a defined species, age, and sex and one or two doses of test drugs.

• Therapies that improved outcome in more than one TBI model would advance to Stage 2 testing that could include multiple drug doses and/or times and durations of administration, combined injury models such as TBI and hemorrhage, male and female animals, and mechanistic variables such as imaging, electrophysiology, etc.

• All experimental procedures, methods, outcomes, etc., should be described in detail in Manuals of Standard Operating Procedures (MSOP), which is updated regularly. This recommendation is applicable to single-center and single-investigator studies as well as multi-center, multi-investigator pre-clinical studies of therapies for TBI.

• Appropriate randomization of drug administration and blinding of key research personnel is recommended to ensure robust and unbiased experimental design. The inclusion of complete details of models and outcome assessments, randomization, blinding, etc., in the operating procedures and resultant publications would facilitate replication.

• If included in the study design, Go/No-Go decisions should take into account the time-courses of the pathophysiological processes of the injury models, the PD/PK, and putative mechanisms of action of the test therapies in the test species.

• Intersite communications provide the opportunity for progress updates and discussions of unexpected events related to models or outcomes. Questions or concerns of laboratory personnel conducting the experiments would be included in these discussions.

Models

• The use of well-characterized experimental TBI models that replicate important features of TBI in humans increases clinical relevance and limiting Stage 1 studies to one level of injury in each model enhances efficiency and feasibility. Additional injury levels could be investigated in Stage 2 experiments.

• Therapies that show positive results in Stage 1 and 2 rodent studies should be tested in TBI studies in large animals, such as pigs, with gyrencephalic brains.

Therapies

Therapy selection

• Optimally, the selection of potential test therapies for pre-clinical screening studies would be based on the likelihood of eventual use in humans by careful vetting by expert collaborators in pharmacology to ensure that they have potential druggability (low toxicity, BBB permeability, etc.). Ideally, test therapies would have a wide therapeutic window between neuroprotective efficacy and side effects/toxicity to increase the likelihood of eventual translation to patient studies.

• Additional criteria for therapy selection could be positive results in previous TBI or other central nervous system (CNS) injury models or novel and/or multiple targets known to be important in the pathophysiology of TBI.

• Given the experience of OBTT—where the positive effects of therapies were less than anticipated from the published literature—there would appear to be value in defining therapies showing cross-model efficacy, or potent efficacy in a single model, even if that specific drug could not be advanced. Such findings could help direct the field toward a key mechanism and/or other derivatives that might be able to be synthesized.

• Particularly for repurposed drugs, doses used in pre-clinical animal models should be evaluated for the likelihood of human equivalent dosing in future clinical studies through dose estimation methods.

Doses and route of administration

• Symposium participants recommended that doses and routes of administration be selected based on known PD/PK and brain penetrance information in the test species, efficacy against a defined mechanistic target viewed as important to TBI, and/or by preliminary studies in the test species. It is especially important to include studies to confirm that the therapy produced the desired effect on the target mechanism, if known.

• It is important to include considerations of the feasibility of route in the target patient population in the selection of routes of administration.

Time and duration of administration

• If available, PD/PK information would guide the decisions about the most effective time of therapy initiation and duration.

• Although an early time of initiation of therapy may reduce the clinical relevance of the study, early administration is appropriate for some Stage 1 screening, depending on the therapy and therapeutic target. Later times of administration could be tested in either Stage 1 or Stage 2 studies of successful therapies.

• Considerations of the time courses of the pathophysiological targets of the test therapy in time-of-administration decisions would increase the likelihood of positive results.

Outcomes

Primary outcomes selection

• The clinical relevance of the primary outcomes is an important consideration in outcome selection. Outcomes that have no human equivalent are of limited value in pre-clinical therapy screening.

• For multi-center screening studies, the inclusion of standardized outcome measures would facilitate comparisons among models and centers.

• The selection of a limited number of measurements directly relevant to behavioral or pathological outcomes would improve the efficiency of Stage 1 studies. Supplemental/secondary outcomes such as imaging, physiology (e.g., CBF, ICP) or those intended for detailed explorations of pathophysiological pathways could be reserved for Stage 2 studies. However, in outcome selection, consideration should be given to special cases, such as drugs targeting a reduction in ICP or mechanisms relevant to the link between TBI and chronic neurodegenerative diseases.

Common data elements

• The NINDS CDEs should be considered in primary outcome selection with the inclusion of outcomes not currently specified in the CDE list.

Long-term outcomes

• Although the importance of the measurements of the chronic effects of TBI is recognized, long-term outcomes are not considered to be practical in Stage 1 pre-clinical screening studies.

Data analysis and sharing

• If funding permits, a central data core is an asset to a multi-center pre-clinical screening study. Multi-center data analysis by a data core is more easily blinded and less susceptible to inadvertent bias than data analysis at individual sites.

• Rapid dissemination of information generated by any TBI study is important and data moratoria should be as short possible. If available and appropriate, data sharing via large international TBI databases such as IMPACT and FITBIR would facilitate dissemination.

• Reporting data and results from unsuccessful/negative pre-clinical screening studies in peer-reviewed journals, and/or disseminated using a widely available database is recommended to reduce unnecessary duplication.

Acknowledgments

We thank Andrew Hall for editorial assistance with this manuscript. The Moody Project Symposium and manuscript submission fees were supported by funds from The Moody Project for Translational Traumatic Brain Injury Research. We would also like to thank the important work performed by pre-clinical and clinical team science efforts such as The Multi-center Animal Spinal Cord Injury Study (MASCIS), Operation Brain Trauma Therapy (OBTT), Transforming Research and Clinical Knowledge in TBI (TRACK-TBI), International Initiative for Traumatic Brain Injury Research (InTBIR), Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI), International Mission for Prognosis and Analysis of Clinical Trials in TBI (IMPACT), and Federal Interagency Traumatic Brain Injury Research (FITBIR) of which we collectively amassed previous experience contributing to these discussions.

Author Disclosure Statement

No competing financial interests exist.