The Challenges of Evaluating Specific and Nonspecific Effects in Clinical Trials of Acupuncture for Chronic Pain

Abstract

Background: The Acupuncture Trialists' Collaboration has updated its individual patient data meta-analysis of acupuncture for chronic pain originally published in 2012. The updated meta-analysis, published in 2018, now includes raw trial data from 39 trials and 20,827 patients. The overall effect of acupuncture, and the effect of sham acupuncture controls, was evaluated.

Results: For 4 conditions, acupuncture has statistically significantly better effects than sham acupuncture (effect sizes 0.16–0.19 [small]). When compared with usual care controls, effect sizes are larger (0.44–0.63 [moderate]). Sham acupuncture has a considerable therapeutic effect; true acupuncture compared with usual care has an effect size of around 0.5, of which 60% is ascribed to nonspecific context effects plus sham, and the remaining 40% to the specific benefit of true acupuncture. Investigators also determined no significant variation in effect related to any acupuncture characteristic; that acupuncture's effect size drops against a high-intensity control; and that only 10%–15% of acupuncture's benefit is lost at 12 months post-treatment.

Conclusions: Acupuncture is more than a placebo for chronic pain, and both specific and nonspecific effects can be distinguished in a meta-analysis of appropriate size.

Introduction

The Acupuncture Trialists' Collaboration, which consists of numerous research investigators, has focused on 2 primary questions: (1) Is acupuncture better than sham (placebo) acupuncture? and (2) Is acupuncture better than standard care, usual care, or waitlist? The answers to these questions are pursued by using an individual patient data meta-analysis; raw randomized controlled trial data with chronic pain conditions (e.g., headache/migraine, osteoarthritis, and musculoskeletal back and neck pain); and high-quality trials only (adequate allocation concealment related to randomization). There is also a focus on variations of effects related to the acupuncture characteristics, choice of control group, and trajectories of longer-term outcomes. H.M. has been an integral member of the Collaboration's research team and in this study focuses on findings related to both the specific and the nonspecific effects of acupuncture for chronic pain.

Based on review of raw data up to 2008, a study by Vickers et al. found significant difference between true and sham acupuncture, indicating that acupuncture is more than a placebo, and that acupuncture is effective for the treatment of chronic pain.¹ An update of this study, which included raw trial data from 39 trials and 20,827 patients through to the end of 2015, evaluated the overall effect of acupuncture as well as the effect of sham acupuncture controls.² For headache/migraine, osteoarthritis, shoulder, and musculoskeletal low back and neck pain, the data suggest that acupuncture has statistically significantly better effects than sham. The effects sizes were found to range from 0.16 to 0.19 when excluding outliers, an effect size that can be described as small. When effects of acupuncture are compared with usual care controls, the effect sizes of acupuncture are larger, ranging from 0.44 to 0.63, a moderate effect size. Furthermore, sham outperforms usual/standard care with an effect size of ∼0.3; thus, sham acupuncture has a considerable therapeutic effect, well beyond what would be expected of an inert placebo. The effect of true acupuncture over and above usual care comprises a clinically relevant effect size of around 0.5, of which 60% can be ascribed to the nonspecific effect of context plus sham needling. The remaining 40% of this effect size can be ascribed to the specific benefit of true acupuncture.

The investigators then determined the variations in impact associated with the acupuncture characteristics reported in the trials. They sorted trials by style of acupuncture, point prescription, whether electrical stimulation was allowed, whether moxibustion was allowed, whether deqi was attempted, and the acupuncture-specific patient–practitioner interactions.^2,3 They also sorted the data by the number and frequency of sessions, their duration, the number of needles used, and the years of experience of the practitioner as well as his/her age and sex. When comparing acupuncture with sham acupuncture they found no significant variation in effect related to any acupuncture characteristic. When they compared acupuncture with usual care, they found marginally better outcomes when more acupuncture treatment sessions were provided, suggesting a dose–response effect.^2,3

The investigators then split the trials with usual care control groups into low- or high-intensity care and the sham control groups into those with penetrating or nonpenetrating needle use.^2,4 Acupuncture does not perform as well when there is a high-intensity control, for example, multiple sessions of physical therapy; that is, the effect size drops. In addition, when sham acupuncture involves using penetrating needles there is a markedly lower effect size of the acupuncture comparison, that is, nonpenetrating sham needling appears to have less of a therapeutic effect than penetrating sham needling. When looking at long-term effects, they found that only 10%–15% of the beneficial effects of acupuncture are lost at 12 months after treatment.^2,5

In sum, acupuncture is an evidence-based treatment for chronic pain, with a moderate effect size of ∼0.5 (P < 0.001), which is clinically relevant with 40% of this benefit contributed by the true acupuncture. These studies have shown that high-intensity control interventions reduce the relative effect of acupuncture when compared with usual/standard care, with 85%–90% of the benefit of acupuncture sustained at 12 months. These data have major implications for cost-effectiveness.

Analysis of existing trial data can yield useful information. For example, a network meta-analysis conducted by Corbett et al. compared acupuncture with other physical therapies and found it to be one of the more effective treatments for alleviating osteoarthritis knee pain in the short term.⁶ Moreover, the effect sizes found when comparing pharmacologic intervention against placebo control are much the same as those found with acupuncture against sham, with the exception of opioids, which have an effect size of 0.78.⁷

Conclusions

To better understand the benefit of acupuncture, more comparative effectiveness research is needed ideally with pragmatic trial designs reflecting the real-world context. In a field littered with too many small trials, fully powered clinical trials are needed with adequate sample sizes. For example, seeking a small effect size of say 0.2 in a 2-arm sham-controlled trial will require the participation of over 1000 patients. It is increasingly clear that there is no need for more sham controlled trials replicating existing data regarding chronic pain of the types already extensively studied. Well-designed sham trials should include a sham intervention to be as close to an inert placebo as possible (e.g., avoid penetrating needles). The priority of future research is to evaluate acupuncture against usual care controls, with a focus on conditions that are widespread and problematic, and for which the evidence base on acupuncture is underdeveloped. What is needed is to identify the conditions that acupuncture, as it is routinely delivered, might provide important clinical benefits for patients.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.