Treating nerves: from anecdote to systematic review

The principle that systematic reviews are the best evidence on which to base clinical practice seems so obvious that it is surprising that their methodology is so recent. It was only in 1979 that Archie Cochrane berated the medical profession for not constructing such reviews. In 1991 the UK Cochrane Centre was founded to respond to this challenge.¹ In 1998 we founded the Cochrane Neuromuscular Disease Review Group to apply the principles and methods of the Cochrane Collaboration to peripheral nerve and muscle disease. In this article, examples principally from peripheral nerve diseases illustrate how our knowledge of the effectiveness of treatments has progressed from anecdotes or observational studies through randomized controlled trials to systematic reviews.

SCURVY: AN ANECDOTE, THE FIRST CONTROLLED TRIAL AND THE VALUE OF LARGE EFFECT SIZES

In 1600 Admiral James Lancaster set out for the East Indies with four ships. They met with contrary winds and took more than six months to reach the west coast of South Africa. By that time the crews of three of the ships were so weak with scurvy that they were unable to man their anchors. His own ship was not affected because the admiral had 'brought to see with him certaine Bottles of the Juice of Limons, which hee gave to each one, as long as it would last, three spoonfuls every morning fasting'.² A marginal note labels this regimen 'The best remedy for scurvy', but this prescient remark did not receive the attention it deserved.

In 1747 a naval surgeon, James Lind, undertook a carefully controlled trial of oranges and lemons. He took 12 sailors with scurvy of about the same stage, divided them into pairs and gave each pair one of the following: a quart of cider; 75 drops of vitriol elixir (sulphuric acid); 6 spoons of vinegar; half a pint of seawater; three nutmegs; or two oranges and a lemon.³

'The consequence was, that the most sudden and visible effects were perceived from the use of the oranges and lemons; one of those who had taken them, being at the end of six days fit for duty... The other was the best recovered of any in his condition and being now deemed pretty well was appointed nurse to the rest.'

With such a large effect size no further trials were needed to conclude that citrus fruits are effective for treating scurvy. The 350 000 references to randomized trials in the Cochrane Library do not include any trials (not even Lind's trial) or systematic reviews of treatment for scurvy. 20 of the 2500 systematic reviews mention vitamin C and one is of particular interest. It concludes that a daily dose of 1 g or more of vitamin C does not prevent the common cold but shortens symptoms by 12 hours (95% confidence intervals 4 to 22 hours),⁴ sufficient in my view to make it worthwhile.

BELL'S PALSY: AN ANECDOTE, A RANDOMIZED TRIAL AND TWO CONFLICTING REVIEWS

Acute idiopathic facial palsy, named after the 19th century physician-scientist Sir Charles Bell, affects 20 per 100 000 of the population each year. Most patients make a good recovery but 15% remain concerned by residual disfigurement, either asymmetry due to contractures or synkinetic movements.⁵ In the first anecdotal report of steroid therapy a woman woke with a complete facial palsy, was given cortisone 100 mg four times daily, and recovered in seven days. The author concluded 'It is indeed difficult, though tempting, to draw a conclusion from a single satisfactory case'.⁶ The first randomized trial was a high-quality study of 26 patients and showed no difference between cortisone and placebo.⁷

There have been two systematic reviews of steroid treatment for Bell's palsy. One included low-quality trials and concluded that steroids are probably beneficial.⁸ However, the Cochrane review, confined to high-quality trials, judged that the combined evidence does not show significant benefit.⁹ In my opinion this is the safest conclusion. It is an indictment of our profession that we have not done adequate trials to discover whether steroids really work in Bell's palsy. Time and again the conclusion of a systematic review is that more research is needed. Furthermore, it has become unethical to embark on a trial without conducting a systematic review first to discover whether a trial is really necessary.

CARPAL TUNNEL SYNDROME: ANECDOTES, RANDOMIZED TRIALS AND UPDATED SYSTEMATIC REVIEWS

Carpal tunnel syndrome affects at least 3% of women and 2% of men at any one time.¹⁰ It causes tingling and pain in the fingers and forearm, which may wake the patient at night and is sometimes worsened by manual activity. The cause is compression of the median nerve where it passes through the carpal tunnel. Marie and Foix in 1913 described an advanced case with wasting and weakness of the thenar eminence. In 1947 Brain, Dickson Wright and Wilkinson described relief of symptoms following division of the flexor retinaculum.¹¹ Nocturnal tingling in the fingers had been recognized since the late 19th century, labelled as 'acroparaesthesiae' and attributed to compression of the brachial plexus at the thoracic outlet. MJ McArdle, from Guy's and the National Hospitals, was the first to propose, in an unpublished paper to the Association of British Neurologists in 1951,¹² that this symptom was due to median nerve compression and could be relieved by operation.

Two randomized controlled trials have found that patients do better one year after operative division of the flexor retinaculum than with conservative treatment (splinting).^13,14 Figure 1 updates the Cochrane review meta-analysis to include the second of these trials.¹⁵ The meta-analysis shows considerable heterogeneity, reflecting probably the higher quality of the second and larger trial, but both trials showed benefit. Attempts have been made to improve on the standard operation by additional procedures such as neurolysis or by endoscopic release through smaller incisions. A recent Cochrane systematic review concluded that these modifications have not yielded significant extra benefit.¹⁶

Figure 1.

Meta-analysis of randomized trials of surgery compared with conservative treatment for carpal tunnel syndrome: relative rate of clinical improvement after 1 year. After Verdugo et al (Ref. ¹⁵). The reader should consult The Cochrane Library for the latest version of a Cochrane Review. Information on The Cochrane Library can be found at www.update-software.com

In 1957 Phalen reported improvement of carpal tunnel syndrome symptoms in 16 out of 20 patients after corticosteroid injections, usually repeated, into the carpal tunnel.¹⁷ In a parallel-group randomized trial in 60 patients the proportion of patients improved four weeks after injection (proximal to the carpal tunnel) was 3.4 times greater in the injected patients than in the controls (95% confidence intervals 1.8 to 8.0).¹⁸ However, the evidence was inadequate to decide whether the benefit from steroids lasts longer than a month.¹⁹ Moreover, in another trial the benefit of local steroids was no greater than that of antiinflammatory treatment and splinting. The systematic reviews highlight our ignorance about the duration of benefit from steroids and how they compare with surgical treatment.

GUILLAIN–BARRÉ SYNDROME: ANECDOTES, RANDOMIZED TRIALS AND SYSTEMATIC REVIEWS

Guillain–Barré syndrome (GBS) affects about 2 per 100 000 of the population each year. It causes progressive flaccid areflexic tetraparesis with sensory loss and in 25% of patients a requirement for artificial ventilation. It is usually due to acute inflammatory demyelinating polyradiculoneuropathy and in the UK less than 10% of patients have acute axonal neuropathy. It is thought to be an autoimmune condition triggered by one of several different infections.²⁰ About 5% of patients die in the acute stage, 15% are left requiring aid to walk at the end of a year and as many as 80% are left with persistent fatigue.²¹

During the 1970s plasma exchange became popular for treating autoimmune conditions and in 1978 Brettle and colleagues described its use in a patient with GBS who recovered quickly. They drew the cautious conclusion that plasma exchange warranted further assessment.²² The first randomized trial did not reveal significant benefit but it included only 29 participants and was too small to detect modest effects.²³ By contrast the systematic review of all trials comparing plasma exchange with no treatment concluded that plasma exchange significantly hastens recovery (Figure 2).²⁴ The favourable results of the trials established plasma exchange as the standard with which other treatments had to be compared.

Figure 2.

Guillain–Barré syndrome: meta-analysis of randomized trials comparing mean improvement on a 7-point disability grade scale 4 weeks after randomization to IVIg or plasma exchange. After Hughes et al. (Ref. ²⁷)

After intravenous immunoglobulin (IVIg) was found useful for treating autoimmune thrombocytopenia, it was tried and seemed successful in chronic inflammatory demyelinating polyradiculoneuropathy. Its many mechanisms include down-regulation of antibody production, blocking of autoantibodies through anti-idiotypes, inhibition of Fc receptors and modulation of T cells. Kleyweg et al.²⁵ first reported its use in GBS. 4 of 6 consecutive bedbound patients improved after its use. The 2 who did not had severe axonal degeneration. The authors suggested that IVIg might be comparable in efficacy to plasma exchange but recommended that 'before clinical application these results should be confirmed in a large trial'. They duly undertook the large trial in 150 patients and showed that the outcome after IVIg was at least as good as that after plasma exchange.²⁶ This was subsequently confirmed by further trials and the Cochrane systematic review.²⁷ Because it is more convenient, IVIg has replaced plasma exchange as the preferred treatment for GBS in most centres.

In the first report of the use of steroids for GBS, a man with GBS began to improve within hours of starting corticotrophin (ACTH) but then worsened. He was thought to have exhausted his adrenal reserves. When given cortisone he improved again and recovered by 7 weeks.²⁸ Debate continued about whether steroids really do benefit patients with GBS. The first randomized trial, not published until 1978, suggested that any effect from oral prednisolone was certainly marginal and possibly harmful.²⁹ The current Cochrane systematic review shows no significant benefit from steroids.³⁰ However, an unpublished Dutch trial reports slightly more rapid recovery in patients treated with the combination of intravenous methylprednisolone and IVIg than with IVIg alone. This new report will need to be included in the Cochrane systematic review and with updating of the meta-analysis. Whether the combined evidence including the new trial will alter the present conclusion remains to be seen.

CONCLUSIONS

It is scarcely surprising that the first published reported use of each of these interventions suggested a favourable effect (Table 1). The corollary is that initial anecdotal reports provide unreliable evidence although in the past they have heavily influenced clinical practice. Even the first randomized controlled trial is an unreliable indicator of where the balance of evidence now lies as to whether an intervention is beneficial or not. The trial has to be free from bias and adequately powered to prove or disprove the efficacy of the agent. Failure to appreciate this point has allowed many treatments, such as corticosteroids for Bell's palsy, to be recommended on the basis of evidence that systematic reviews show to be wholly inadequate. Conversely we have almost certainly rejected many treatments which are in reality effective.

Table 1.

Summary of the conclusions from the first anecdotes and randomized trials and the current Cochrane systematic reviews

Condition	Intervention	First anecdote	First controlled trial	Current Cochrane systematic review
Scurvy	Citrus juice	+ve	+ve
Bell's palsy	Steroids	+ve	-ve	-ve
Carpal tunnel syndrome	Steroid injection	+ve	+ve	+ve*
Carpal tunnel syndrome	Surgery	+ve	+ve	+ve
GBS	Plasma exchange	+ve	-ve	+ve
GBS	IVIg	+ve	+ve	+ve
GBS	Steroids	+ve	-ve	-ve

GBS=Guillain-Barré syndrome; IVIg=intravenous immunoglobulin; +ve=positive; -ve=negative

^*After 1 month: there is inadequate evidence after 3 and 12 months

It might be argued that some treatments are so obviously effective, such as surgical decompression for severe carpal tunnel syndrome, that randomized trials are unnecessary. However, effect sizes are rarely so large that non-randomized studies are adequate to draw conclusions. Even when effect sizes are large, a small randomized trial such as that of James Lind may still be the most efficient way of showing efficacy. Single, large randomized trials are rarely powerful enough to establish the efficacy of an intervention beyond reasonable doubt. They leave unanswered questions about reproducibility and about generalizability to other patient populations and treatment settings. Well conducted up-to-date systematic reviews should be the best evidence on which to base clinical practice—a conclusion which now seems banal but has yet to pervade the whole of medicine.