A systematic review of systematic reviews of homeopathy

Abstract

Homeopathy remains one of the most controversial subjects in therapeutics. This article is an attempt to clarify its effectiveness based on recent systematic reviews. Electronic databases were searched for systematic reviews/meta-analysis on the subject. Seventeen articles fulfilled the inclusion/exclusion criteria. Six of them related to re-analyses of one landmark meta-analysis. Collectively they implied that the overall positive result of this meta-analysis is not supported by a critical analysis of the data. Eleven independent systematic reviews were located. Collectively they failed to provide strong evidence in favour of homeopathy. In particular, there was no condition which responds convincingly better to homeopathic treatment than to placebo or other control interventions. Similarly, there was no homeopathic remedy that was demonstrated to yield clinical effects that are convincingly different from placebo. It is concluded that the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice.

Introduction

Homeopathy is a therapeutic method using preparations of substances whose effects when administered to healthy subjects correspond to the manifestations of the disorder (symptoms, clinical signs, pathological states) in the individual patient. The method was developed by Samuel Hahnemann (1755–1843) and is now practised throughout the world [1]. Homeopathy is based on two main principals [1–3]. According to the ‘like cures like’ principle, patients with particular signs and symptoms can be helped by a homeopathic remedy that produces these signs and symptoms in healthy individuals. According to the second principle, homeopathic remedies retain biological activity after repeated dilution and sucussion even when diluted beyond Avogadro's number.

Few therapies have attracted more debate and controversy than homeopathy. Throughout its 200-year history, critics have pointed out that its very principles fly in the face of science, while proponents have maintained that it is narrow minded to reject an overtly helpful approach to healing only because one cannot explain how it might work [2]. Similarly, proponents have quoted seemingly rigorous trials that suggest efficacy, while critics had little trouble citing equally rigorous studies that implied the opposite.

The existence of contradicting evidence is not unusual in therapeutics. One solution to resolve such contradictions is to conduct systematic reviews and meta-analyses of rigorous studies. In 1997, Linde et al.[3] did just that. The conclusions of this technically superb meta-analysis expressed the notion that homeopathic medicines are more than mere placebos. The authors also stated that no indication was identified in which homeopathy is clearly superior to placebo. Despite this and other caveats, homeopaths worldwide celebrated this publication as the ultimate proof of their treatment. Since then, a flurry of interest in homeopathy has emerged, and several further systematic reviews have been published. This article is an attempt to critically evaluate all such papers published since 1997 with a view to defining the clinical effectiveness of homeopathic medicines.

Methods

Literature searches were carried out in the following databases: Medline (via Pubmed), Embase, Amed, CISCOM (from inception to October 2001). The search terms used were homeopath . . . , homoeopath . . . , clinical trial, meta-analysis, systematic review, efficacy, effectiveness. In addition, other experts in the field (n = 5) were consulted and my own, extensive files were studied. The bibliographies of all articles thus located were scanned for further relevant references. No language restrictions were applied.

Only systematic reviews (including meta-analyses) of controlled clinical trials of homeopathy with human patients or volunteers were included. Non-systematic reviews, overviews, clinical trials and reviews of non-clinical investigations were excluded. All articles were evaluated by the present author. The following information was extracted from the original articles: inclusion/exclusion criteria, total sample size, assessment of methodological quality, results of meta-analyses, overall conclusion of the authors.

Results

Six re-analyses of Linde et al.'s original meta-analysis [3] were located [4–9]. Table 1 summarizes key data from these publications. The results of these re-analyses demonstrate that the more rigorous trials are associated with smaller effect sizes which, in turn, render the overall effect insignificant [5, 6, 8]. One re-analysis suggests that the initial positive meta-analytic result [3] was largely due to publication bias [9], a notion that had been considered by the original authors but was rejected by them. Most notably, perhaps, the authors of the original meta-analysis [3] concluded that their re-analysis ‘weakened the findings of their original meta-analysis’[6]. Collectively these re-analyses imply that the initial conclusions of Linde et al.[3] was not supported by critical evaluation of their data.

Table 1.

The systematic review by Linde et al.[3] and its subsequent re-analyses.

Reference	Included trials (number)	Total patient number	Assessment of methodological quality	Meta-analysis	Overall conclusion*	Comment
Linde (1997) [3]	All double-blind and/or randomized placebo-controlled trials of any clinical condition (n = 186)	2588	Yes	Of 89 trials which could be submitted to meta-analysis: OR = 2.45; of 26 ‘good quality trials’: OR = 1.66 (both in favour of homeopathy)	Clinical effects of homeopathy are not completely due to placebo	Review was criticised for 1) including different remedies 2) including different conditions 3) including nonrandomized trials
Ernst (1998) [4]	All studies from Linde et al.[3] which received 90 (of 100) points in at least 1 of the 2 quality ratings, using highly dilute remedies, following the principles of ‘classical’° homeopathy (n = 5)	587	Yes	OR = 1.0 (no evidence in favour of homeopathy)	Homeopathic remedies are associated with the same clinical effects as placebo	This analysis specifically tested the efficacy of highly diluted remedies (other remedies could still work via conventional pharmaceutical effects)
Linde (1998) [5]	All trials from Linde et al.[3] which tested ‘classical’° homeopathic remedies against placebo, no treatment or another treatment (n = 32)	1778	Yes	19 placebo-controlled trials were submitted to meta-analysis; OR = 1.62; however, when this analysis was restricted to the methodologically best trials the effect was no longer significant	Individualized homeopathy has an effect over placebo; the evidence, however, is not convincing	Not all of the included trials were randomized and many had other serious methodological weaknesses
Linde (1999) [6]	All trials from Linde et al.[3] which could be submitted to meta-analysis (n = 89)	n.d.p.	Yes	The mean OR of the best studies was not in favour of homeopathy	There was clear evidence that studies with better methodological quality tended to yield less positive results	The authors felt that these results ‘weaken the findings of [their] original meta-analysis’
Morrison (2000) [7]	26 trials classified by Linde et al.[3] as high quality (n = 26)	n.d.p.	Yes	None	No significant trend was seen when correlating security of randomization and trial result	Large multicentre trials were recommended
Ernst (2000) [8]	All trials from Linde et al.[3] that received quality ratings between 1 and 4 on the Jadad score (n = 77)	n.d.p.	Yes	None	There is a . . . strong linear correlation between OR and Jadad score (n =  0.97, P < 0.05); homeopathic remedies are, in fact, placebos	Extrapolation from this correlation implies that the most rigorous studies yield an effect size of zero
Sterne (2001) [9]	89 trials of Linde et al.[3] review compared with 89 trials of allopathic medicines	n.d.p.	Yes	Strong evidence for publication bias causing a false positive result in favour of homeopathy	When adjusting high quality trials [of homeopathy] for publication bias, the OR changed from 0.52 to 1.19 but remained unchanged for allopathy	Paper probably not peer-reviewed, adjusting for bias nullified the effect of homeopathy but not for allopathy

RCT = randomized clinical trial, OR = odds ratio,

^*= verbatim quotes, n.d.p. = no details provided.

^°Classical homeopathy = approach where remedies are individualized according to patient characteristics deemed important by homeopaths.

In addition, 11 independent systematic reviews were located [10–20]. Table 2 summarizes key data from these publications. Collectively the findings do not provide strong evidence in favour of homeopathy. With the exception of postoperative ileus [10] and influenza [17] (see below) there is no condition for which homeopathy is convincingly effective [10, 11, 13, 18–20]. Arnica, the most frequently tested homeopathic remedy, is not demonstrably different from placebo [12, 15]. One homeopathic remedy (oscillococcinum) was found to be superior to placebo as a treatment and prevention of influenza but the effect size was small and therefore of debatable clinical relevance [17]. Moreover, the volume of the evidence for oscillococcinum is small and therefore not fully conclusive. Our systematic review of various homeopathic medicines for postoperative ileus produced an overall positive result [10]. Yet several caveats need to be taken into account, most importantly the fact that the definitive study designed as a multicentre trial to replicate several of smaller studies failed to demonstrate a positive effect [10]. One independent review of all homeopathic RCTs regardless of indication or type of remedy yielded a positive result [16]. Yet the statistical approach to generate this result was of debatable validity and the authors are keen to point out that their overall result is weak and not sufficient for definitive recommendations.

Table 2.

Independent systematic reviews of homeopathy.

Reference	Included trials (number)	Total patient number	Assessment of methodological quality	Meta-analysis	Overall conclusion*	Comment
Barnes (1997) [10]	All placebo-controlled trials of homeopathy for postoperative ileus (n = 6)	776	Yes	Weighted mean difference to time until first sign of peristalsis was in favour of homeopathy (−7.4 h)	Homeopathic treatment can reduce the duration of postoperative ileus, however, several caveats preclude a definitive judgement	The methodologically best trial was convincingly negative
Ernst (1998) [11]	All placebo-controlled trials of homeopathy for delayed onset muscle soreness (DOMS) (n = 8)	311	Yes	No meta analysis possible, all randomized trials were negative	The evidence does not support the hypothesis that homeopathic remedies are more efficacious than placebo for DOMS	DOMS was chosen because it was submitted to clinical trials more often than any other condition
Ernst (1998) [12]	All placebo-controlled trials of homeopathic arnica (n = 8)	338	Yes	No meta-analysis possible, no clear trend in favour of homeopathy	The claim that homeopathic arnica is efficacious beyond a placebo effect is not supported by rigorous clinical trials	This analysis set out to test the remedy that had been most frequently submitted to clinical trials, i.e. arnica (see also Lüdtke below)
Ernst (1999) [13]	All RCTs of homeopathy for migraine prophylaxis (n = 4)	284	Yes	No meta-analysis possible; 3 of 4 trials were negative (including the methodologically best)	The trial data . . . do not suggest that homeopathy is effective in the prophylaxis of migraine or headache beyond a placebo effect	This analysis tested the efficacy for a condition that homeopaths often treat in clinical practice
Ernst (1999) [14]	All controlled clinical trials of ‘classical’° homeopathy vsconventional treatments (n = 6)	605	No	No meta-analysis possible	No clear trend in favour of homeopathy	Nonrandomized studies were also included
Lüdtke (1999) [15]	All controlled clinical trials of homeopathic arnica (n = 37)	n.d.p.	Yes	No meta-analysis possible	No clear evidence in favour of homeopathic arnica was found	Paper probably not peer-reviewed, trials that used arnica in combination with other remedies and those which were not placebo controlled were also included
Cucherat (2000) [16]	All RCTs of homeopathy vs placebo with clinical or surrogate endpoints (n = 16)	2617	Yes	Combined 2-tailed P value was highly significant (P = 0.000056) in favour of homeopathy	There is some evidence that homeopathic treatments are more effective than placebo	Strength of evidence was estimated to be low by the authors
Vickers (2000) [17]	All RCTs of homeopathic oscillococcinum vs placebo for influenza (n = 7)	3459	Yes	RR = 0.64 for influenza prevention RR = 0, 28 for influenza treatment	Treatment reduced length of illness significantly by 0.28 days	The authors stated that ‘the data are not strong enough to make a general recommendation’
Linde (2000)	All RCTs of homeopathy vs placebo for chronic asthma (n = 3)	154	Yes	No meta-analysis possible	No clear trend in favour of homeopathy	Not enough evidence for reliable assessment
Jonas (2000) [19]	All controlled clinical trials of homeopathy for rheumatic conditions (n = 6)	392	Yes	Combined OR = 2.19	Homeopathic remedies work better than placebo	Not enough trials for any specific condition to allow reliable assessment
Long (2001) [20]	All RCTs of homeopathy for osteoarthritis (n = 4)	406	Yes	No meta-analysis possible	No clear trend in favour of homeopathy	Not enough evidence for reliable assessment

RCT = randomized clinical trial, OR = odds ratio, RR = relative risk.

^°Classical homeopathy = approach where remedies are individualized according to patient characteristics deemed important by homeopaths.

Discussion

Collectively these data do not provide sound evidence that homeopathic remedies are clinically different from placebos. However, the present analysis has several limitations that should be kept in mind when interpreting its conclusions. Even though a thorough search strategy was adopted, there is no absolute guarantee that all relevant articles were located. Many of the included reviews are from the present author's team, and this could have introduced bias. Finally the validity of conducting a systematic review of systematic reviews has its limitations; most importantly it does not create any information that was not available before.

The clinical evidence summarized above is not dissimilar from the preclinical data. Vickers recently conducted a systematic review of preclinical investigations of homeopathy [21]. Even though 120 papers could be included in the evaluation, this author found that lack of independent replications, serious methodological flaws, and contradictory results precluded any firm conclusion. This systematic review therefore casts considerable doubt on one of the main assumptions of homeopathy, namely that homeopathic remedies retain biological activity even when diluted beyond Avogadro's number (see above).

Perhaps the most recent trial evidence, not yet included in systematic reviews, helps clarify the question whether homeopathic remedies are more than placebos. Since the publication of the systematic reviews, both positive,  e.g.  [22–24].  as  well  as  negative  clinical  trials e.g. [25–27] have emerged. It seems therefore unlikely that these new findings would substantially change the results of any of the systematic reviews were they to be up-dated.

The recent observation of solute clusters in highly diluted water has been interpreted by several homeopaths as increasing the plausibility of homeopathy [28]. This novel finding requires independent replication. Furthermore, this observation (if confirmed) does not lend itself to explaining how solute clusters could have any effects on human health. Thus both the clinical evidence and the basic research underpinning homeopathy remain unconvincing.

If one accepts this conclusion, one might ask what its implications for future research may be. Two opposing views exist. One holds that the definitive trial of homeopathy should be conducted to once and for all settle the question [29]. The other states that ‘new trials . . . are no longer a research priority’ and advocates ‘outcome studies to evaluate the individual treatment decisions . . . and compare outcomes to orthodox treatment’[30]. Such outcome studies exist. They are burdened with a myriad of methodological weaknesses, most importantly a proneness to selection bias, and usually report findings which are convincingly in favour of the homeopathic approach [31]. This could imply that the individualized, empathetic and time-intensive approach most homeopaths adopt to healthcare yields good clinical results. This emphasizes the importance of the therapeutic encounter and is in accordance with a wealth of information in this area [32]. It does not, however, answer the ‘placebo question’. I insist that this question does require an answer – for the sake of scientific honesty and possibly in the name of clinical progress.

In conclusion, the hypothesis that any given homeopathic remedy leads to clinical effects that are relevantly different from placebo or superior to other control interventions for any medical condition, is not supported by evidence from systematic reviews. Until more compelling results are available, homeopathy cannot be viewed as an evidence-based form of therapy.

Conflict of interest: The author is a trained homeopath; he has no financial interests in this area.