Abstract
Objective
To assess whether trials in head injury are large enough to avoid moderate random errors and designed to avoid moderate biases.
Design
All randomised controlled trials on the treatment and rehabilitation of patients with head injury published before December 1998 were surveyed. Trials were identified from electronic databases, by hand searching journals and conference proceedings, and by contacting researchers. Data were extracted on the number of participants, quality of concealment of allocation, use of blinding, loss to follow up, and types of participants, interventions, and outcome measures.
Results
279 reports were identified, containing information on 208 separate trials. The average number of participants per trial was 82, with no evidence of increasing size over time. The total number of randomised participants in the 203 trials in which size was reported was 16 613. No trials were large enough to detect reliably a 5% absolute reduction in the risk of death or disability, and only 4% were large enough to detect an absolute reduction of 10%. Concealment of allocation was adequate in 22 and inadequate or unclear in 25 of the 47 (23%) in which it was reported. Of 126 trials assessing disability, 111 reported the number of patients followed up, and average loss to follow up was 19%. Of trials measuring disability, 26 (21%) reported that outcome assessors were blinded.
Conclusions
Randomised trials in head injury are too small and poorly designed to detect or refute reliably moderate but clinically important benefits or hazards of treatment. Limited funding for injury research and unfamiliarity with issues of consent may have been important obstacles.
Introduction
Worldwide, many millions of people are treated each year for severe head injury. A substantial proportion die, and many more are permanently disabled.1 Road traffic accidents alone account for an estimated five million head injuries each year.2 Reliable assessment of the net benefits and hazards of various interventions for the treatment and rehabilitation of head injuries could be of considerable importance to public health.
If, for such a common problem, a widely practicable treatment could be shown to reduce the absolute risk of death and disability by “just” a few per cent, this might affect the treatment of hundreds of thousands of patients each year and protect thousands from death or long term disability. To detect reductions of this magnitude, however, both moderate random errors and moderate biases must be avoided.3 This means that randomised controlled trials of treatments for head injury should be large enough to avoid moderate random errors and designed in such a way that moderate biases are also avoided.3 To assess the extent to which randomised controlled trials in head injury meet these criteria we conducted a survey of their size and quality.
Methods
Inclusion criteria
—We included all randomised controlled trials of interventions in the treatment and rehabilitation of head or brain injury contained within the specialised register of the Cochrane Injuries Group as of May 1999 and published before December 1998. A randomised controlled trial was defined as a trial in which the patients followed were assigned to one of two (or more) interventions with random allocation or some quasi-random method of allocation. There was no language restriction.
Identification of studies—The injuries group maintains a specialised register of randomised controlled trials in the prevention and treatment of and rehabilitation after traumatic injury. The register contains both published and unpublished reports that have been found by searches of electronic databases, hand searching of key journals and conference proceedings, and direct contact with trialists. The Cochrane Controlled Trials Register (Cochrane Library 1999, issue 2) was searched to update the specialised register. Medline (OVID) and PubMed, the most up to date version of Medline, were searched in June 1999 to identify further studies published up to December 1998. All reports of randomised controlled trials in systematic reviews prepared by the injuries group that referred to head or brain injury trials were also included. A total of 605 records describing studies of head or brain injury were identified. One collaborator (IR) scanned this list, and 370 records referring to possible randomised controlled studies were identified. Full copies of all these reports were obtained and examined by one investigator (KD) to assess eligibility and for data extraction. Excluded trials were checked by two investigators to ensure that none were eligible.
Data extraction and data analysis—Data were extracted on the date and language of publication, number of randomised participants, method of concealment of allocation, use of blinding, percentage loss to follow up, and types of participants, interventions, and outcome measures. The dimensions of methodological quality assessed were those shown or suspected to be associated with bias in estimates of treatment effects.4 Reports published in languages other than English were translated before data extraction. To test the accuracy of data extraction a 10% random sample of reports was re-examined by FB and the two sets of data were compared.
Statistical analysis—Trials were categorised according to the number of randomised participants and a histogram was plotted. We estimated the size of the intervention effect that a trial of a given number of participants would be able to detect at the 0.05 level of significance (α=0.05) with 80% power (β=0.2), assuming a baseline risk (as estimated by control group event rates) of 0.2.
Results
The combined search strategies identified 370 reports, of which 279 were reports of randomised controlled trials of interventions for head or brain injury. After taking into account multiple reports of the same randomised controlled trial we identified 208 discrete randomised controlled trials. Most of the reports (86%) were published as full journal articles, with the remainder published as abstracts in conference proceedings (13%) or book chapters (2%). Most reports were published in English (92%), with the remainder in French (2%), Italian (2%), Chinese (1%), German (1%), and Spanish (1%).
There was 90% agreement between assessors for the number of randomised participants and for method of concealment of allocation. In the trials in which there was disagreement about the number of randomised participants the numbers differed by only one participant in each trial.
The number of randomised participants was reported in 203 of the 208 trials. The total number of randomised participants in the 203 trials was 16 613. The average number of participants per trial was 82, with no evidence of a trend towards larger trial size over time. Interventions were classified into categories, and table 1 shows the number of trials and number of randomised participants for each category. The largest trial was of additional information, advice, and support to patients after head injury and included 1156 participants. The largest trial of a pharmacological agent was of the aminosteroid tirilazad mesylate and included 1120 participants. The category of agents for which there was the largest number of randomised participants was corticosteroids with 2515 participants.
Table 1.
Number of trials and number of randomised participants by intervention
| Category | No of trials* | No of participants |
|---|---|---|
| Aminosteroid | 3 | 1331 |
| Anaesthetics | 27 | 1010 |
| Antidepressant | 3 | 26 |
| Antiseizure | 8 (1) | 1212 |
| Antispasticity | 2 | 12 |
| Antiulcer | 5 | 354 |
| β blockers | 2 | 135 |
| Calcium antagonists | 5 | 1405 |
| Cognitive therapy | 8 | 406 |
| Corticosteroids | 23 (1) | 2515 |
| Dimethyl sulfoxide | 1 | 35 |
| Endocrine | 5 | 258 |
| Fluid therapy | 4 (1) | 127 |
| Hypothermia | 9 | 364 |
| Management and information | 7 | 2280 |
| N-methyl-D-aspartate antagonists (eliprodil, cerestat) | 2 | 485 |
| Nootropics (piracetam, pramiracetam sulphate) | 3 | 123 |
| Nutrition | 24 (2) | 854 |
| Osmotic diuretics | 3 | 140 |
| Other | 20 (1) | 665 |
| Polyethylene glycol-superoxide dismutase | 2 | 567 |
| Phospholipids (CDP-Choline) | 4 (1) | 350 |
| Pyritinol hydrochloride | 3 | 370 |
| Physical therapy | 8 (1) | 364 |
| Stimulant (methylphenidate, amphetamine) | 10 | 215 |
| TRIS | 4 | 411 |
| Vasopressin | 3 | 36 |
| Ventilation | 10 | 563 |
Number of trials that included patients with mild head injury on Glasgow coma scale given in brackets.
The figure shows the size of trials with a curve giving the magnitude of intervention effect (absolute risk reduction) that a trial of that size would be able to detect at the 0.05 level of significance (α=0.05) with 80% power (β=0.2), with an assumed baseline risk of 0.2. None of the existing trials could be expected to detect reliably a difference in event rates of less than 5%, and only 4% of trials would have been large enough to detect a difference in event rates of less than 10%.
The method of concealment of allocation was reported in only 47 (23%) of the trials. Of these, concealment was judged to be adequate in 22 (11%) and inadequate or unclear in 25. Table 2 shows the methods of allocation used in the trials.
Table 2.
Method of allocation in trials in which it was reported
| Method | No of trials |
|---|---|
| Centralised randomisation by telephone* | 1 |
| Numbered/coded identical containers administered sequentially* | 11 |
| Randomisation scheme controlled by pharmacy* | 8 |
| Sequentially numbered, sealed, opaque envelopes* | 2 |
| Other† | 18 |
| Date of birth | 1 |
| Day of week | 3 |
| Alternation | 3 |
Methods of allocation judged to be adequately concealed.
Ten of 18 trials involved use of envelopes but did not specify if they were sequentially numbered, sealed, or opaque.
The Glasgow coma scale was used as an inclusion criterion in 56 trials. Of these, 51 included patients with severe head injury, 17 included patients with moderate head injury, and eight trials included patients with mild head injury. A total of 126 trials assessed the effect of the trial intervention on the extent of disability. The most commonly used outcome measure was the Glasgow outcome scale, used in 45% of trials assessing disability. Of the 126 trials assessing disability, 111 reported the number of patients followed up. In those trials, the average loss to follow up was 19%. Of the trials measuring disability, only 26 (21%) reported that the outcome assessors had been blinded.
Discussion
There is a growing recognition that large absolute reductions in death or disability cannot realistically be expected in the treatment of head injury.5 Changes in outcome of “only” a few per cent would nevertheless be important, especially if achievable with widely practicable treatments. To detect such moderate treatment effects, both moderate random errors and moderate biases must be avoided.
Failures of existing research
The results of our review suggest that currently available trials in head injury fail on both counts. Most trials of head injury are small. The average number of randomised participants was 82, and the largest trial included only 1156 participants. None of the trials would have been large enough to detect reliably the difference between a 20% and a 15% risk of death or disability. If baseline risk was higher than 20%, then existing trials would have even lower power to detect any given absolute reduction in the risk of death or disability.
The concern about the small size of trials is as relevant for minor head injury as it is for the more severe injuries. Three months after sustaining minor head injury, 79% of patients have persistent headaches, 59% have memory problems, and 34% of previously employed people are still unemployed.6 If a treatment reduced the risk of memory loss, headache, and unemployment by 5% then this would be an important discovery. Most trials, however, would miss this. Moreover, for any given category of therapeutic agent the total number of randomised participants was at most a few thousand. Thus, even in aggregate, existing trials are too small to support or refute the possibility of moderate harm or moderate benefit. This is of particular importance given that most, if not all, of the pharmacological interventions currently used in the management of severe head injury are of unproved benefit.7 The paucity of trials in head injury is underscored by the fact that the total number of randomised participants in all of the available trials in head injury combined (16 613) is less than that in some individual trials in heart disease and stroke.8–10
What is already known on this topic
Millions of people are treated each year for head injury, a substantial proportion of whom die and many more are permanently disabled
If a widely practicable treatment could be shown to reduce the absolute risk of death and disability by just a few per cent, then, because this might affect the treatment of hundreds of thousands of patients, it could protect thousands from death or disability
To detect reliably such modest treatment effects, head injury trials must be large and well designed
What this study adds
The results of this survey of randomised controlled trials in head injury show that currently available trials are too small and too poorly designed to be able to detect or refute reliably realistically modest but clinically important benefits or hazards
Large randomised controlled trials of widely practicable treatments for head injuries are needed
The effect of poorly concealed random allocation can be as large as or larger than the effect of the treatment under investigation and can operate in either direction,11 yet only 23% of the trials reported the method of allocation concealment, and concealment was judged to be adequate in only half of these. We did not contact authors for further information in trials in which concealment of allocation was unclear, as experience with systematic reviews in this topic has shown that additional information can be obtained for only a small proportion of trials. Furthermore, there is evidence to suggest that it is rare for information to be omitted that would change the classification.12
Despite extensive efforts to include as many trials as possible, those that we identified are best considered as a sample of the currently available head injury trials. Because the sample was obtained by systematically searching mainstream medical databases and medical journals there is no reason to suspect that large or good quality trials would have been underrepresented. The chances that we have overlooked a substantial body of large, high quality trials is remote. Although it is inevitable that we have missed some trials, we would not expect health professionals caring for patients with head injury to have ready access to more trials than we have found.
Problems leading to failure
Why is it that those who provide care for patients with head injury have such a meagre evidence base on which to draw? Two factors in particular may be important. Firstly, in comparison with the burden of disability, funding for injury research is less than for almost any other cause of human misery,13 perhaps because there are fewer advocates for research in head injury than for cardiovascular disease or cancer. Secondly, people with head injury and impaired consciousness are unable to give informed consent. They present an important exception to the general requirement to obtain informed consent in clinical research.14 Nevertheless, many ethics committees and investigators are unfamiliar with the idea of randomisation without consent, and the resulting confusion has been an important obstacle to enrolment. It was concern that safer and more effective interventions for life threatening emergencies were not being developed because of the need to obtain informed consent, however, that stimulated the US Department of Health and Human Services to announce in 1995 a waiver of requirements for informed consent in certain emergency research.15 It will be interesting to see whether, over the next decade or so, this has the hoped for effect on trial size.
Road traffic accidents account for most serious head injuries, and with increasing motorisation the global burden of head injuries can only be expected to increase. It is estimated that by 2020 road traffic accidents will be the third leading cause of disability- adjusted life years lost worldwide and the second leading cause in developing countries. Currently, many millions of patients are treated each year for severe head injury, and there are few treatments of established effectiveness. If a widely practicable treatment reduced the risk of death or disability by 5%, then treatment of one million patients would protect 50 000 people from death or disability. Large scale randomised controlled trials have the potential to make an important contribution to the prevention of death and disability from this growing epidemic.16,17
Figure.
Trial size and smallest absolute risk reduction detectable
Acknowledgments
We thank Simona Fiore, Olivier Duperrex, and Leah Li for translating the Italian, Spanish, French and Chinese trial reports; and Iain Chalmers, Doug Altman, and Richard Peto for their comments on the manuscript.
Footnotes
Funding: The Cochrane Injuries Group is funded by the NHS Executive.
Competing interests: None declared.
References
- 1.Murray CJL, Lopez AD. Global health statistics: a compendium of incidence, prevalence and mortality estimates for over 200 conditions. Boston: Harvard University Press; 1996. [Google Scholar]
- 2.World Bank. Global road safety partnership. Washington, DC: World Bank; 1999. [Google Scholar]
- 3.Peto R, Collins R, Gray R. Large-scale randomised evidence: large simple trials and overviews of trials. J Clin Epidemiol. 1995;48:23–40. doi: 10.1016/0895-4356(94)00150-o. [DOI] [PubMed] [Google Scholar]
- 4.Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–412. doi: 10.1001/jama.273.5.408. [DOI] [PubMed] [Google Scholar]
- 5.Teasdale G. The treatment of head trauma: implications for the future. J Neurotrauma. 1991;8(suppl 1):53–60. [PubMed] [Google Scholar]
- 6.Royal College of Surgeons of England. Report of the working party on the management of patients with head injury. London: Royal College of Surgeons; 1999. [Google Scholar]
- 7.Roberts I, Schierhout G, Alderson P. Absence of evidence for the effectiveness of five interventions routinely used in the intensive care management of severe head injury: a systematic review. J Neurol Neurosurg Psychiatry. 1998;65:729–733. doi: 10.1136/jnnp.65.5.729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.CAST (Chinese Acute Stroke Trial) Collaborative Group. CAST: randomised placebo-controlled trial of early aspirin use in 20 000 patients with acute ischaemic stroke. Lancet. 1997;349:1641–1649. [PubMed] [Google Scholar]
- 9.ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both or neither among 17187 cases of suspected acute myocardial infarction. Lancet. 1988;ii:349–360. [PubMed] [Google Scholar]
- 10.International Stroke Trial Collaborative Group. International stroke trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke. Lancet. 1997;349:1569–1581. [PubMed] [Google Scholar]
- 11.Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ. 1998;317:1185–1190. doi: 10.1136/bmj.317.7167.1185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Liberati A, Himel HN, Chalmers TC. A quality assessment of randomised controlled trials of primary treatment of breast cancer. J Clin Oncol. 1986;4:942–951. doi: 10.1200/JCO.1986.4.6.942. [DOI] [PubMed] [Google Scholar]
- 13.Ad Hoc Committee on Health Research Relating to Future Intervention Options. Investing in health research and development. Geneva: World Health Organization; 1996. [Google Scholar]
- 14.Smith R. Informed consent: the intricacies. BMJ. 1997;314:1059–1060. doi: 10.1136/bmj.314.7087.1059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Department of Health and Human Services; Food and Drug Administration. Protection of human subjects: informed consent. Fed Reg. 1995;60:49086–49103. [PubMed] [Google Scholar]
- 16.Yates D, Farrell B, Teasdale G, Sandercock P, Roberts I. Corticosteroids in head injury—the CRASH trial. J Accid Emerg Med. 1999;16:83–90. doi: 10.1136/emj.16.2.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.www.crash.ucl.ac.uk (accessed 10 Mar 2000).