A Quantitative Analysis of Clinical Trial Designs in Spinal Cord Injury Based on ICCP Guidelines

Abstract

Clinical studies of spinal cord injury (SCI) have evolved into multidisciplinary programs that investigate multiple types of neurological deficits and sequelae. In 2007, the International Campaign for Cures of SCI Paralysis (ICCP) proposed best practices for interventional trial designs, end-points, and inclusion criteria. Here we quantitatively assessed the extent to which SCI trials follow ICCP guidelines and reflect the overall patient population. We obtained data for all 288 SCI trials in ClinicalTrials.gov. We calculated summary statistics and observed trends pre-2007 versus 2007 onward. To compare the trial population to the overall SCI population, we obtained statistics from the National SCI Statistical Center. We generated tag clouds to describe heterogeneous trial outcomes. Most interventional studies were randomized (147, 73.1%), and utilized active (55, 36.7%) or placebo controls (49, 32.7%), both increasing trends (p=0.09). Most trials were open label (116, 53.5%), rather than double- (62, 28.6%) or single-blinded (39, 18.0%), but blinding has increased (p=0.01). Tag clouds of outcomes suggest an emphasis on assessment using scores and scales. Inclusion criteria related to American Spinal Injury Association (ASIA) status and neurological level allowed inclusion of most SCI patients. Age inclusion criteria were most commonly 18–65 or older. Consistent with ICCP recommendations, most trials were randomized and controlled, and blinding has increased. Age inclusion criteria skew older than the overall population. ASIA status criteria reflect the population, but neurological lesion criteria could be broadened. Investigators should make trial designs and results available in a complete manner to enable comparisons of populations and outcomes.

Introduction

Spinal cord injuries (SCI) can have devastating consequences, including impaired motor, bladder, bowel, sexual, and social function, along with co-morbidities such as chronic pain, skin ulcers, bone loss, and muscle spasticity. Early hopes for a “magic bullet” cure have long since been replaced by broad-based, multidisciplinary investigations into the amelioration of individual sequelae. Clinical trials in these areas have attained varying degrees of success. Early pharmacological trials of therapies to improve neurologic outcome in acute SCI began with the first National Acute SCI (NASCIS) Studies trial (Bracken et al., 1984). Results from later trials of methylprednisolone, sodium succinate, and GM-1 ganglioside, have contributed to current management practices for acute SCI (Lammertse, 2004). However, levels of evidence have been inconsistent. The NASCIS 2 and 3 trials (Bracken et al., 1990,1997) are often cited as evidence that methylprednisolone may be efficacious in patients treated within 8 h of injury, but observers have highlighted instances of borderline statistical significance, unbalanced treatment arms despite randomization, and subjectivity and difficulty in assessment of end-points (Nesathurai, 1998).

The SCI research community has until recently lacked clear guidelines for the design of clinical trials. In 2006, Blight and Tuszynski discussed considerations for the design, analysis, and reporting of human trials (Blight and Tuszynski, 2006). Then, in a series of four papers in 2007, the International Campaign for Cures of SCI Paralysis (ICCP) proposed best practices for clinical trials related to:

• Methodology, given that spontaneous sensory and motor recovery in incomplete patients can be substantial and variable (Fawcett et al., 2007);

• End-points, which should provide an anatomical or neurological assessment of cord connectivity, categorize functional ability to engage in activities of daily living, and measure quality of life (Steeves et al., 2007);

• Inclusion and exclusion criteria, to help control for confounding variables such as stages after SCI, level, type of injury, and other clinical factors (Tuszynski et al., 2007); and

• Phases of a clinical program, including protocols for trial designs, ideally prospective, double-blind, randomized trials utilizing placebo controls (Lammertse et al., 2007).

Clinical trial designs also attempt to reflect the patient populations they are intended to serve. Global estimates of the number of new SCI cases annually range from 15–40 per million (Chinnock and Roberts, 2005). Subjects are primarily males injured by vehicular collisions, falls, sports, and violence (Marino et al., 2011). Of note, there has been a shift in the demographics of SCI due to an aging population. This trend has implications for future clinical trials of therapies for adult patients with acute SCI, and for management strategies of elderly individuals with SCI (Furlan et al., 2009). Epidemiologic data suggest that in the future, the needs of elderly patients with spinal cord injuries will pose a significant health care burden (Fisher et al., 2006).

Detailed descriptions of clinical trial designs have become increasingly available. The ClinicalTrials.gov registry was constructed in response to ethical and scientific concerns that protocol information and results should be publicly available free of charge and in a timely manner, regardless of whether they are published. The bias towards publication of positive studies and the corresponding safety risks to patients have been extensively described (Krleza-Jeric et al., 2005). Early experience suggested that a registry should be comprehensive and adequately funded, with participation required and enforced. Along those lines, the 2004 International Committee of Medical Journal Editors statement requiring registration before publication represented a significant incentive (Rennie, 2004).

In this study, we quantitatively assessed the extent to which SCI clinical trials reflect the best practice guidelines put forth by the ICCP and represent the SCI population. As with previous broad-based efforts in neurotrauma (Saatman et al., 2008), neurology (Chang et al., 2007), and other fields (Scher et al., 2004), we hypothesized that collaborative guideline development efforts could improve the design of trials. We address consistency with the ICCP recommendations by analyzing characteristics of trials from ClinicalTrials.gov, with a focus on design features and outcomes. We also compare inclusion criteria and clinical trial populations with epidemiological data from the National Spinal Cord Injury Statistical Center (NSCISC) (DeVivo et al., 2002), and from the NASCIS trials. We find that among interventional studies, randomization and use of controls is high, while blinding is less common but increasing. Inclusion criteria vary, but age, American Spinal Injury Association (ASIA) status, and neurological lesion level generally reflect the SCI population as described by the NSCI database (NSCIDB).

Methods

We downloaded data for all trials categorized under “spinal cord injury” from ClinicalTrials.gov on 3/11/2011. While the ICCP guidelines focus on trials of protection and repair of the injured spinal cord, we included those not targeting neurological recovery. Many of the guideline recommendations, such as randomization and blinding, are well-accepted design principles, though other features such as inclusion criteria are more situation-specific.

Parameters included National Clinical Trial (NCT) ID, title, recruitment status, study results, conditions, interventions, gender, age group categories, trial phases, enrollment count, study type, study designs, start and completion dates, and outcome measures (see Supplementary Table 1 for definitions; see online supplementary material at http://www.liebertonline.com). Of these variables, only enrollment count was numeric. Enrollment figures could represent either the targeted or actual number of subjects. Therefore, assuming that enrollment totals divisible by 10 may represent estimates, we assessed the extent of this possible bias by comparing the number of trials that had enrollment totals divisible by 10 according to recruitment status, that is, past (completed, suspended, terminated, or withdrawn) versus current trials (active, enrolling, not yet recruiting, or recruiting).

Of the other variables, title, conditions, and outcome measures were free text; recruitment status, availability of study results, gender, age group, trial phases, and study type were categorical; and interventions and study designs were a combination of categorical and free text. We derived trial duration from the start and completion dates, which may also be estimated, and we derived intervention types and design features from the categorical parts of the intervention and study design data. Because the nomenclature used to describe conditions is heterogeneous, we manually aggregated similar concepts. The condition field is self-reported and not standardized. It includes some broad and some narrow categories, and many trials list multiple conditions.

We first summarized demographic and clinical characteristics for SCI trials. Since the electronic publication date of the ICCP papers was 12/19/2006, we tested whether categorical differences between pre-2007 trials and trials from 2007 onward, based on start date, were statistically significant by chi-square testing. To compare trial enrollment and duration between time periods, we log transformed the data and performed t-tests. We next examined trends over time of key SCI trial features. Because there were few trials before 2000, and because ClinicalTrials.gov began accepting data that year, we focused on trends since then.

We then performed an analysis of design features for all trials. Also, since Phase I studies typically focus on the safety of interventions and enroll healthy volunteers, and since the ICCP guidelines are predominantly related to cell-based and drug treatment studies, we performed a subgroup analysis on post-Phase I interventional trials. Because randomized, double-blind, placebo-controlled trials were highlighted as optimal designs, we focused on these characteristics. We calculated summary statistics for design features and compared trends over time to the ICCP guidance.

Because outcomes, methods, and aims (as determined from study titles) of the trials were so heterogeneous, we attempted to develop a semi-quantitative assessment of them. We therefore generated tag clouds, that is, graphical images composed of words or phrases proportionally sized based on the frequency of their occurrence, using a free web-based tool (http://www.tagcrowd.com).

We next assessed trial inclusion criteria and compared them with epidemiological data. While age categories are reported as structured data, exact age and other inclusion criteria are reported as free text. We therefore manually extracted and analyzed key inclusion criteria, with a focus on Phase II\III and III studies. Exclusion criteria were highly specific to the studied conditions and interventions, so we performed no detailed analysis on them. Because inclusion criteria were heterogeneous, we focused on age, duration from injury to enrollment, ASIA classification, and neurological level. To compare the clinical trial populations to the overall SCI population, we obtained statistics for 2009 from the NSCISC. The NSCIDB is one of the world's largest spinal cord injury databases. NSCISC represents a network of 14 federally-sponsored and 3 regional model spinal cord injury systems located at major medical centers throughout the United States. We also compared age ranges from the trial inclusion criteria with the age distributions for the arms of the three NASCIS trials.

Results

Overview of SCI trials and trends

We first summarized demographic and clinical trends for the 288 SCI trials in ClinicalTrials.gov (Table 1), including 115 trials started before 2007 and 166 started from 2007 onward. Dating back to the late 1980s, the number of trials initiated each year has increased over time, with rapid recent increases seen from 2001 to 2002 (+300%), and 2006 to 2007 (+80%), particularly for drugs and novel intervention types including behavioral treatments (Fig. 1A). Nearly four times as many interventional studies (225, 78.1%) have been conducted as observational studies (63, 21.9%), though the number of observational studies doubled from before to after 2007 (21 versus 42). The majority of interventional trial arms have been composed of drug treatments (123, 34.8%), followed by devices (70, 19.8%), and procedures (63, 17.6%). There has been a significant increase (p<0.001) in the relative proportion of behavioral and other interventions, particularly as the number of device trials has fallen since 2007.

Table 1.

Summary of Demographic and Clinical Characteristics for Spinal Cord Injury Trials in ClinicalTrials.gov

Feature	Total count	%	Pre-2007 count	%	2007+ count	%	% Change	p Value
Trials	288	100.0	115	100.0	166	100.0
Gender								0.34
Both	262	91.3	102	88.7	153	92.7	50.0
Male	24	8.4	12	10.4	12	7.3	0.0
Female	1	0.3	1	0.9	0	0.0	−100.0
Age groups								0.004
Adult, senior	152	52.8	55	47.8	94	56.6	70.9
Adult	85	29.5	28	24.3	54	32.5	92.9
Child, adult, senior	32	11.1	20	17.4	11	6.6	−45.0
Child, adult	15	5.2	10	8.7	5	3.0	−50.0
Child	4	1.4	2	1.7	2	1.2	0.0
Status								n\a
Recruiting	112	38.9	21	18.3	91	54.8	333.3
Completed	90	31.3	65	56.5	23	13.9	−64.6
Active, not recruiting	40	13.9	19	16.5	20	12.0	5.3
Not yet recruiting	20	6.9	2	1.7	18	10.8	800.0
Terminated	10	3.5	4	3.5	4	2.4	0.0
Withdrawn	7	2.4	1	0.9	4	2.4	300.0
Enrolling by invitation	5	1.7	1	0.9	4	2.4	300.0
Suspended	4	1.4	2	1.7	2	1.2	0.0
Phase								0.77
Phase II	47	32.9	18	30.0	26	33.3	44.4
Phase I	26	18.2	10	16.7	15	19.2	50.0
Phase III	21	14.7	12	20.0	9	11.5	−25.0
Phase IV	20	14.0	7	11.7	12	15.4	71.4
Phase II, Phase III	14	9.8	7	11.7	7	9.0	0.0
Phase I, Phase II	13	9.1	5	8.3	8	10.3	60.0
Phase 0	2	1.4	1	1.7	1	1.2	0.0
Intervention								<0.001
Drug	123	34.8	44	34.9	76	34.4	72.7
+Biological	4	0.9	0	0.0	4	1.8	n\a
+Dietary supplement	3	0.9	0	0.0	3	1.4	n\a
Device	70	19.8	36	28.6	32	14.5	−11.1
Procedure	63	17.6	31	24.6	32	14.5	3.2
Other	44	12.5	4	3.2	39	17.6	875.0
+Behavioral	44	12.5	10	7.9	32	14.5	220.0
+Radiation	4	1.1	1	0.8	3	1.4	200.0

Counts are numbers of trials for all features except interventions, where counts are trial arms. p Values calculated by chi-square test, and percentage changes indicate differences between counts of pre-2007 trials and trials from 2007 onward. Categories with counts <10 were not included in p-value calculations; interventions with a “+” prefix were rolled up. p Values were not calculated for trial status due to inherent temporal bias. Percentage changes were not calculated for zero denominators. Pre-2007 and post-2007 sums may not match, since some trials did not report start dates.

FIG. 1.

Trends over time of key spinal cord injury (SCI) trial features in ClinicalTrials.gov. (A) The number of SCI trials has increased rapidly since 2000, particularly for drugs and novel intervention types including behavioral treatments. (B) Average trial enrollment (dashed line) has shown a recent increase after holding steady since 2000, while average trial duration (solid line) has fallen steadily. (C) Since 2002, there have been increases in the numbers of Phase I (Ph I), II (Ph II), and IV (Ph IV) trials, relative to Phase III (Ph II) trials. Proportions that do not total 100% exclude Phase 0 trials. (D) The age group distributions in clinical trials have consistently emphasized studies of adults and seniors. Proportions that do not total 100% exclude child-only trials.

We next calculated summary data for the clinical trials. Because of the predominance of recent trials in ClinicalTrials.gov, a considerable number of trials were still actively recruiting (112, 38.9%), compared to the number of completed studies (90, 31.3%). The majority of trials (262, 91.3%) included both genders. Average trial enrollment has increased recently after holding steady since 2000, while average trial duration has fallen steadily (Fig. 1B). The average number of enrolled subjects for all trials was 114, with a median of 46 and an interquartile range of 20–100. Approximately half of enrollment figures may be estimates: 69 of 133 past trials (51.9%) and 96 of 155 current trials (61.9%; p=0.09) had enrollment totals divisible by ten. Trial enrollments have increased over time, but the difference between time periods was not statistically significant (p=0.44). Trial durations have been significantly shorter (p<0.001) from 2007 onward compared with before 2007. Since 2002, there have been increases in the number of Phase I, II, and IV trials relative to Phase III trials (Fig. 1C). Most trials were Phase II (47, 32.9%), with about half as many Phase I (26, 18.2%), Phase III (21, 14.7%), and Phase IV (20, 14.0%) studies. This distribution has not significantly changed from before versus after 2007 (p=0.77), though the fraction of Phase III studies has fallen from 20.0% to 11.5%. The age group distributions in clinical trials have consistently and increasingly emphasized studies of adults and seniors (Fig. 1D). Most trials included adults and seniors (152, 52.8%), or adults only (85, 29.5%), and the representation of these age groups significantly increased (p=0.004) from before to after 2007.

The conditions studied in SCI clinical trials have been extremely heterogeneous (Table 2), even considering the lack of standard terminology. Among listed conditions besides spinal cord injury itself, pain has been the most commonly studied (78 trials, 27.1%). Paralysis and motor deficits (39 trials, 13.5%), and urinary or bladder complications (26 trials, 9.0%), have also been frequently studied. Common complications seen in the NSCIDB that have been less frequently listed include depression, pneumonia, pulmonary emboli, and heterotopic ossification.

Table 2.

Top Conditions Studied in Spinal Cord Injury Trials as Reported in ClinicalTrials.gov

Condition	Count
Spinal cord injury\compression	245
Pain	78
Paralysis	39
Urinary\bladder	26
Nervous system disease	21
Sexual dysfunction	9
Spasticity	9
Bone loss	8
Neurogenic detrusor overactivity	8
Pressure ulcers	7
Stroke	7
Central cord syndrome	6
Cancer	5
Constipation\incontinence	5

Counts are numbers of trials. In some cases, trials studied more than one condition.

Trends in clinical trial designs and outcomes

We next examined trial design features for all interventional studies (Table 3). We observed that randomization has increased steadily since 2000 (Fig. 2A), though the difference after versus before 2007 was not statistically significant (p=0.09). Overall, most interventional studies have been randomized (147, 73.1%) There has been an increase in single-blind studies relative to open label studies (Fig. 2B), and the proportion of blinded studies has increased significantly (p=0.01) since 2007. Most trials have been open label (116, 53.5%), compared with double-blinded (62, 28.6%), or single-blind (39, 18.0%). There has also been a slight increase in controlled studies, particularly using placebo (Fig. 2C), though again the difference after versus before 2007 was not significant (p=0.09). Most studies have utilized active (55, 36.7%) or placebo control (49, 32.7%) compared to uncontrolled studies (30, 20.0%). In terms of study assignment, most trials have implemented parallel (96, 44.9%) or single group assignment (79, 36.9%).

Table 3.

Summary of Design Features for Spinal Cord Injury Trials in ClinicalTrials.gov

Feature	Total count	%	Pre-2007 count	%	2007+ count	%	% Change	p Value
Study type								0.16
Interventional	225	78.1	94	81.7	124	74.7	31.9
Observational	63	21.9	21	18.3	42	25.3	100.0
Allocation								0.09
Randomized	147	73.1	57	66.3	91	77.1	59.6
Non-randomized	54	26.9	29	33.7	27	22.9	−6.9
Masking								0.01
Open label	116	53.5	56	63.6	63	47.4	12.5
Double-blind	62	28.6	24	27.3	39	29.3	62.5
Single-blind	39	18.0	8	9.1	31	23.3	287.5
Control								0.34
Active control	55	36.7	29	40.3	23	34.3	−20.7
Placebo control	49	32.7	20	27.8	27	40.3	35.0
Uncontrolled	30	20.0	16	22.2	10	14.9	−37.5
Dose comparison	13	8.7	5	6.9	7	10.4	40.0
Historical control	3	2.0	2	2.8	0	0.0	−100.0
Design								0.09
Parallel assignment	96	44.9	33	37.9	64	48.9	93.9
Single group assignment	79	36.9	40	46.0	42	32.1	5.0
Crossover assignment	36	16.8	12	13.8	24	18.3	100.0

After study type, all features relate to interventional studies. Counts are numbers of trials. p Values calculated by chi-square test indicate differences between pre-2007 trials and trials from 2007 onward. Categories with counts <10 were not included in p-value calculations. Pre-2007 and post-2007 sums may not match totals since some trials did not report start dates.

FIG. 2.

Trends over time of key spinal cord injury (SCI) trial design features relative to International Campaign for Cures of SCI Paralysis (ICCP) guidance, excluding Phase I and I\II trials. (A) Randomization has increased steadily since 2000. (B) There has been a slight increase in single-blind studies relative to open label studies. (C) There has been a slight increase in controlled studies, particularly using placebo.

In a subgroup analysis, we examined design features for interventional, post-Phase I studies. (Features were similar for Phase I versus I\II studies, so they were grouped.) We excluded studies for which phase or design information was unavailable. Among these studies, there was an increase in the proportion of randomized versus non-randomized studies from the period before (35 of 42, 83.3%) versus after 2007 (44 of 48, 91.7%). There was also an increase in the proportion of blinded versus open-label studies, from 21 of 41 (51.2%) to 34 of 51 (66.7%). Finally, there was an increase in the proportion of controlled versus uncontrolled studies, from 29 of 34 (85.3%) to 27 of 29 (93.1%). However, none of the three differences were statistically significant over time (p=0.34, 0.14, 0.44). Nonetheless, all trends matched the overall data set in directionality, and proportions of desired features were higher in post-Phase I studies.

We then quantitatively and qualitatively assessed trial outcomes. Most interventional trial end-points have been related to efficacy (92, 48.2%), or a combination of safety and efficacy (85, 44.5%). The majority of studies have been treatment-oriented (166, 76.5%). Using tag clouds, we observed that among outcomes (Fig. 3A), pain, motor function, and quality of life were expected themes, but there was also considerable emphasis on assessment, measuring change, and using scores and scales, highlighting the challenges of evaluating SCI as a complex phenomenon. Among methods and interventions (Fig. 3B), electrical stimulation, locomotor training, and intermittent catheterization emerged as common topics. For example, 16 electrical stimulation trials were identified. Among aims (Fig. 3C), general themes such as safety and efficacy as well as pain and function were as expected.

FIG. 3.

Tag clouds of spinal cord injury (SCI) clinical trial features. (A) Outcomes. (B) Interventions. (C) Aims.

Clinical trials and the SCI population

As Phase II\III and III trials are considered pivotal for demonstrating the efficacy of an intervention, they are of particular interest. As with the overall data set, most Phase II\III and III trials were completed (15, 42.9%) or recruiting (9, 25.7%). While most trials included both sexes (30, 88.2%), four enrolled only males, and none enrolled only females. Most trials enrolled adults and seniors (21, 60.0%), or adults only (9, 25.7%). Trials enrolled an average of 121.5 patients (median, 78; interquartile range, 36–180).

We compared inclusion criteria for Phase II\III and III trials in ClinicalTrials.gov with the overall SCI population. Age ranges from the inclusion criteria were most commonly 18–65 years or greater (Fig. 4A). The age distribution from the NSCIDB from 2009 shows a sharp peak of subjects in their late teens and early twenties (Fig. 4B), and the age distributions for the arms of the three NASCIS trials showed a similar though slightly older peak (Fig. 4C). We further analyzed the age distribution in the 26,851 subjects included in the NSCIDB, and found that almost exactly 80% of patients were between 18 and 58 years old. Specifically, 2698 (10.05%) were younger than 18 years, and 2688 (10.01%) were older than 58 years.

FIG. 4.

Comparison of trial age inclusion criteria with the overall spinal cord injury (SCI) population. (A) The age distribution from the National Spinal Cord Injury (NSCI) database (2009) shows a sharp peak in the late teens and early twenties. (B) Age ranges from the inclusion criteria for Phase II\III and III trials in ClinicalTrials.gov are shown as horizontal lines. Lines extending to age 100 indicate no upper age limit was listed. (C) The age distributions for the arms of the three National Acute SCI Studies (NASCIS) trials showed a similar though slightly older peak. Bars depict data for the 1997, 1990, and 1994 trial publications. Data are for methylprednisolone for 24 or 48 h (24MP and 48MP), or tirilazad mesylate for 48 h (48TM); methylprednisolone (MP), naloxone (NLX), or placebo (PBO); and high or standard dose of methylprednisolone (HI and LO).

Among 24 Phase II\III and III trials reporting other key inclusion criteria, 23 trials reported durations between the time of injury and enrollment ranging from less than 8 h (two trials) to over 1 year (11 trials) (Table 4). Fourteen trials included ASIA classification, mostly ASIA A subjects, and 11 trials included neurological lesion level criteria. C6 injuries were most commonly included. Data from the NSCIDB indicate that 46.9% of subjects were classified as “Complete, A” at admission according to the ASIA Impairment Scale, compared to 12.6%, 14.4%, and 17.7% as “Sensory Incomplete, B,” “Non-functional Motor Incomplete, C,” and “Functional Motor Incomplete, D,”, respectively. Other NSCIDB data indicate that the most frequent injury level is C5 (15.4%), followed by C4 (14.5%), C6 (10.8%), T12 (6.7%), and L1 (5.0%).

Table 4.

Summary of Key Inclusion Criteria for Phase INII and III Spinal Cord Injury Trials in ClinicalTrials.gov

NCT ID	Duration	ASIA classification	Neurological level
NCT00561067	<8 hours	A or B	C5 to T12
NCT00004759	<8 hours
NCT00844480	<8 weeks	A or B
NCT00150696	<100 days
NCT01292811	<6 months	Incomplete	C4 to C7
NCT01086930	<6 months	Motor complete or incomplete	C2 to T1
NCT01244594	>6 months	A or B	Cervical or thoracic
NCT00654680	>6 months
NCT01217047	>6 months	A
NCT01236976	>6 months	Complete or incomplete	C6 to T12
NCT00421083	>6 months
NCT00652262	>6 months
NCT00434018	>1 year		C6 and below
NCT00753948	>1 year		Tetraplegia
NCT01095380	>1 year	C or D
NCT00148239	>1 year	Complete or incomplete
NCT00656149	>1 year	Complete or incomplete	C5 to C7
NCT00755079	>1 year	All ASIA classifications	C2 to C8, T1 to T6
NCT00237770	>1 year		Tetraplegia
NCT00138866	>18 months
NCT00041717	>18 months	Incomplete
NCT00059553	1–3 years	C or D	Cervical or thoracic
NCT00266864	Chronic
NCT00879021		Complete or incomplete

Duration, function, and neurological level were manually extracted. Eleven trials with no data reported for any of the three criteria are not listed.

ASIA, American Spinal Injury Association; NCT, National Clinical Trial.

Discussion

In this study, we quantitatively assessed the extent to which SCI clinical trials adhere to the best practice guidelines put forth by the ICCP and reflect the SCI population. We found that recent studies have trended towards larger, shorter, earlier-stage drug trials. Among interventional studies, randomization, blinding, and use of active and placebo controls has increased. Inclusion criteria vary widely. The most consistent and consistently reported criterion, age, reflects the population described by the NSCIDB, but allows inclusion of more patients at the upper age range. The diversity of conditions, interventions, and outcomes studied in SCI clinical trials conveys a sense of both the complex nature of the injury, and also of the challenges that have been encountered in developing treatments.

The increase we observed in the number of clinical trials of drugs has been described in several recent reviews (Baptiste and Fehlings, 2008). The growing number of trials of behavioral interventions has included treadmill training (NCT00004812), peer mentoring (NCT00205205), exercise (NCT00461474), telehealth (NCT00624806), mental practice (NCT01302522), and supported employment (Ottomanelli et al., 2009). The increases in the number of Phase I and II trials relative to Phase III trials, as well as the breadth of conditions being studied, appears to reflect a deeper appreciation of the complexity of SCI, as well as the limited success of many early approaches. It is unclear whether the trend toward shorter trials is by design or is a statistical artifact. Adequate trial enrollment is critical to achieve sample sizes that can demonstrate statistical significance.

Overall, the increases in randomization, blinding, and placebo control in interventional studies are promising. While the proportion of blinded studies has increased significantly (p=0.01) since 2007, most trials have been open label (116, 53.5%), compared with double- or single-blind (101, 46.6%). Better progress has been made in terms of the number of randomized (147, 73.1%) and active or placebo-controlled studies (104, 69.4%), though there is still debate regarding the relative merit and feasibility of active versus placebo controls in various clinical contexts (Eddicks et al., 2007). These rates are even higher in post-Phase I studies. Notably, while the majority of interventional studies have been treatment-oriented (166, 76.5%), there have been a considerable number of prevention studies (30, 13.8%) in conditions such as urinary tract infections (NCT00309114), pressure ulcers (NCT00763282), bone loss (NCT00844480), and secondary neurologic damage (Hu et al., 2010). It has been recognized for some time that lifelong resources and services, as well as prevention efforts, are needed to minimize consequences from SCI (Dollfus, 1993). The use of tag clouds illustrated the heterogeneity of trial outcomes. Among outcomes, pain, motor function, and quality of life were expected themes, but there was also considerable emphasis on assessment, measuring change, and using scores and scales, highlighting the challenges of evaluating SCI as a complex phenomenon. A considerable amount of further work will be needed in the coming years to analyze these data and develop frameworks for comparisons among trials.

As suggested by the ICCP guidelines, we focused on specific inclusion criteria such as age, duration of injury before enrollment, ASIA classification, and neurological lesion level. It was surprising to us that the inclusion criteria of several Phase II\III and III trials did not specify any of these factors besides age. Age ranges for the inclusion criteria for Phase II\III and III trials in ClinicalTrials.gov were most commonly 18–65 years or greater. The exact age distributions for many recent trials are difficult if not impossible to determine, since patient level data are seldom reported, and even category summaries or averages are often not reported. Furthermore, averages of skewed distributions can be misleading, and they are often listed without related measures such as medians or standard deviations. The age distribution from the NSCIDB shows a sharp peak of subjects in their late teens and early twenties, and we found that almost exactly 80% of patients were between 18 and 58 years old. The age distributions for the arms of the three NASCIS trials showed a similar though slightly older peak. The overall SCI patient age distribution indicates that the standard lower threshold inclusion criterion used in trials, often selected for convenience since 18 year olds are considered adults for reasons of informed consent, may in fact appropriately reflect the patient population. On the other hand, commonly used upper thresholds, perhaps defined for the sake of recruiting more patients and because 65 year olds are considered seniors for retirement benefits eligibility, may currently be too high despite aging trends in the chronic population.

Inclusion criteria seek to maximize the potential response to treatment while maintaining external validity, that is, confidence that results can be extended to other populations. This trade-off is not unique to SCI; for example, a study of amyotrophic lateral sclerosis patients enrolled in clinical trials found significant differences with epidemiologic cohorts in age, sex, and other clinical features (Chio et al., 2011). Age and other distributions may be influenced by correlated exclusion criteria such as presence of co-morbidities, by clinical issues such as ethics and safety, and by pragmatic considerations such as speed and cost of enrollment. Given the significant number of SCI patients in their early teens, the research community might consider the potential scientific and clinical benefits, as well as the ethical and practical challenges related to including this population in clinical trials, as has been discussed in the ICCP guideline series (Tuszynski et al., 2007). For neuroprotection studies, researchers may want to reconsider inclusion criteria that are open-ended at the upper ranges of age, since inclusion of more elderly patients might involve inclusion of patients with various co-morbidities, concomitant medications, and differences in pathophysiology. Studies have repeatedly shown a negative correlation between age and recovery in animal models (Siegenthaler et al., 2008), and more complications in clinical studies (New and Epi, 2007). It may be worth investigating whether, by making study populations more heterogeneous and less like the overall SCI population, these inclusion criteria may actually be counter-productive. However, the issue of age-dependent plasticity applies primarily to protection and repair trials. In contrast, co-morbidities such as skin sores may be more appropriately studied in older populations.

Limitations

In this study, we analyzed trials of heterogeneous conditions, interventions, and outcomes as a set, but we propose that the principles of good study design remain similar. We made practical decisions regarding which types of interventions to combine for statistical analysis (e.g., Behavioral with “Other”). The analysis of observational and Phase I trials as well as of exclusion criteria, and the comparison of trends in SCI trials relative to trials in other indications, were beyond the scope of this study. However, it is noteworthy that the number of observational studies doubled from the period before 2007 to the period from 2007 onward, reflecting the utility of these types of studies for understanding disease progression and heterogeneity. In general, we chose to analyze trials initiated before 2007 to the period from 2007 onward based on the publication date of the ICCP papers; while many studies initiated after that date were likely conceived before the date, it is similarly true that the guidelines were being widely discussed before publication.

There were limitations related to the coverage and comparability of data in the ClinicalTrials.gov database. A recent survey found that although trial registration has increased, 39% of trials were still registered late, and only 12% of completed studies reported results within a year (Law et al., 2011). While the database captures structured information on the age categories studied in trials, other data are inconsistently captured in the inclusion\exclusion criteria and in other free text. Uninformative entries have been found for industry trials (DeAngelis et al., 2004). Nonetheless, while there are acknowledged limitations to the use of any registry, this database has supported a wide variety of analyses (Bourgeois et al., 2010; Subramanian et al., 2010). Alternative approaches include the use of other registries or literature reviews, but these approaches also pose challenges. For example, since 2004, trials conducted in the European Union have been registered in the EudraCT database, but this registry is open only to regulatory agencies and organizations that provide funding for research (Haug et al., 2005).

In contrast to ClinicalTrials.gov, the NSCIDB captures etiology, neurologic level, and ASIA impairment scale scores, as well as other outcomes such as functional independence measure scores, respirator use, patient health questionnaire, pain, and ambulation. On the other hand, because the NSCIDB includes a small fraction of SCI patients in the United States from large, urban medical centers, it may not be representative of patients in the rural U.S. or from other countries. Finally, the ICCP represents only one of several potential sources of guidelines for trial designs; other groups include the North American Clinical Trials Network (NACTN) and the International Spinal Research Trust (ISRT).

Future work

The transition from basic science to clinical applications requires continuous feedback to efficiently translate, evaluate, and optimize scientific findings and clinical treatments (McDonald and Becker, 2003). Partnerships among clinical trial sponsors, professional societies, industry, and federal agencies should facilitate identification of priorities and uniformity of measurement standards (Ditunno, 2010). The SCI Model Systems have contributed to various fields of knowledge, including basic science, clinical trials, relevant outcome measures, and quality of life issues (Tate et al., 2011). These centers and other efforts have worked to define what it would mean to develop “cures” (Hulsebosch et al., 2000).

To facilitate future analyses of clinical trials, several steps could be taken. First, it is disheartening that of 288 trials, only four have made results available in ClinicalTrials.gov, and only 11 NCT IDs can be found in PubMed abstracts. At the very least, trials funded with public money should make summary results available in a timely manner. Second, care should be paid to ensure the quality of data submissions, including the use of standard terminology for study conditions and common data elements (Whyte et al., 2010), particularly those related to predictors of spontaneous recovery. While applying a categorization scheme retrospectively to the investigator-reported conditions would introduce further bias, in the future, it would be possible to prospectively enforce a systematic categorization scheme based on the National Library of Medicine's Medical Subject Headings (MeSH), as recommended by the registry, or another system such as ICD-9 or SNOMED. Third, other common parameters available in the NSCIDB could also be included in ClinicalTrials.gov at a categorical or summary level, such as race, marital status, level of education, occupational status and job census code, veteran status, place of residence, and alcohol use. Fourth, consideration should be given to including younger patients and patients with C4–C5 injuries in more trials. Fifth, ClinicalTrials.gov should implement, where possible, the use of controlled vocabularies for conditions, interventions, and outcomes. Sixth, opportunities to implement adaptive study designs should be explored further (Tator, 2006).

Conclusions

In this study we quantitatively assessed the nature of SCI trials in ClinicalTrials.gov, the extent to which SCI trials adhere to the best practice guidelines put forth by the International Campaign for Cures of SCI Paralysis, and the extent to which trial populations reflect the overall SCI population. We found that good design practices such as randomization, blinding, and use of active and placebo controls are increasing. More consistent, detailed, standardized reporting of inclusion criteria and outcomes is needed, and further work is needed to enable comparisons across trial conditions, interventions, and outcomes.

Acknowledgment

We thank Michael G. Fehlings, M.D., Ph.D., for helpful comments.

Author Disclosure Statement

M.D.S. is an employee of Genentech, Inc. No competing financial interests exist for the other authors.