The Acupuncture Trialists’ Collaboration: individual patient data meta-analysis of chronic pain trials

Abstract

The Acupuncture Trialists’ Collaboration was established to synthesize data from high quality randomized trials on acupuncture for chronic pain. Trialists joining the collaboration provide raw data from their trial. Each data set is converted into a standardized format and then combined into a single data set for meta-analysis. The primary question addressed by the collaboration is the effect size of acupuncture in comparison to both sham acupuncture and to usual care control. A number of secondary analyses will be conducted, including evaluating variations in the effects of acupuncture by indication, acupuncture characteristics, and types of sham, as well as an assessment of the time course of acupuncture effects. Analyses will also be conducted to see if the type of acupuncture used, traditional Chinese or Western, affects outcome. At the time of writing, members of the Acupuncture Trialists’ Collaboration have conducted a total of 25 chronic pain trials including over 18,000 patients. We hope that our approach can serve as a model for future studies in acupuncture and other complementary therapies: we strongly believe that it is only by breaking down the oppositional culture of competing trialists, and sharing data in a robust scientific collaboration, that we can best translate clinical trial findings into patient benefit.

The purpose of clinical trials of acupuncture is to help clinicians and patients make decisions about treatment. But using acupuncture trials to aid clinical decisions is not straightforward. Take the case of a patient consulting an evidence-based physician for chronic low back pain and asking whether acupuncture might be of value. The doctor searches MEDLINE, and, as of May 2009, finds 65 English-language randomized trials. Even if the doctor were to obtain copies of all of these trials, he or she would find that the results were inconsistent. Some report acupuncture to be superior to sham (placebo) acupuncture¹ while others show evidence that acupuncture is superior to no treatment but not sham², and still others conclude that acupuncture is no better than usual care³.

Clearly what is needed is a meta-analysis to synthesize the results from the different studies. Indeed, many meta-analyses of acupuncture for chronic pain have been published. These studies have tended to come to somewhat indeterminate conclusions, such as that “there is limited evidence that acupuncture is more effective than no treatment for chronic pain; and inconclusive evidence that acupuncture is more effective than placebo”.⁴ This appears to be because, until recently, acupuncture research was dominated by small trials of questionable quality.

The landscape of clinical acupuncture research has recently been dramatically altered by the publication of several large, high quality trials. These include the two NHS funded trials of acupuncture chronic headache disorders (n=401)⁵ and back pain(n=241)⁶ and the German ART (Acupuncture Randomized Trials) studies^{2, 7–9} which randomized 300 patients each in four separate trials on osteoarthritis, chronic low back pain, migraine and chronic tension headache. But even these trials are dwarfed by the GERAC (GERman ACupuncture) trials which accrued close to 1000 patients each on trials on osteoarthritis, chronic low back pain and migraine and 400 on chronic tension headache^10–12. The ARC (Acupuncture in Routine Care) trials have even larger sample sizes: 3000 patients on three separate trials of back pain, neck pain and chronic headache, and 600 patients on a trial of arthritis^13–15.

Most of these data have yet to be synthesized in any meta-analyses. The Acupuncture Trialists’ Collaboration was established to do just that. We are an international group comprised of 29 physicians, clinical trialists, biostatisticians, practicing acupuncturists and other specialists who have come together in an effort to synthesize data from high quality randomized trials of acupuncture for chronic pain. At the time of writing, members of the Acupuncture Trialists’ Collaboration have conducted a total of 25 trials including over 18,000 patients. The indications are diverse including low back pain (6 trials), neck pain (6 trials), migraine (5 trials), tension-type or chronic daily headache (5 trials) and osteoarthritis (6 trials). There is also desirable heterogeneity among control groups: 15 trials include sham and 14 usual care.

Traditional meta-analysis uses the summary data published in study report, such as the mean pain scores in acupuncture and control groups. The Acupuncture Trialists’ Collaboration will instead conduct individual patient data meta-analysis using raw data from these trials. This has long been recognized as the ideal method of analyzing research data. In the words of Iain Chalmers, one of the founders of the Cochrane Collaboration, using individual patient data in a meta-analysis is the “yardstick” by which all meta-analyses should be measured¹⁶. The advantages of using individual patient data compared to the published summary data are as follows:

Standardization between different analytic approaches

Some trials of acupuncture have reported mean change in pain, others have reported “response rates” of the proportion of patients who experienced a threshold reduction in pain (e.g. 33%). These results cannot be combined without access to raw data, which allows conversion from one type of analysis to another.

Application of statistical methods with greater power

In a typical meta-analysis, the investigator records mean and standard deviations for acupuncture and control groups separately. This does not allow the application of techniques, such as analysis of covariance, that have greater statistical power than unadjusted analysis^{17, 18}.

Association between patient-level characteristics and outcome

Individual patient data analyses have far greater power to investigate questions such as whether age or baseline symptom severity influence outcome. As an example, if there were four trials with 250 patients each, analysis of published data would attempt to correlate four values of a predictor (e.g. mean age in each trial) with four values of an outcome (e.g. difference between mean pain scores). Analysis of individual patient data would be able to create a model with 1000 data points.

Data quality

The process of combining data from different sources requires careful data scrutiny by an independent investigator. This provides an opportunity to identify and correct errors in the data set.

The first step of the Acupuncture Trialists’ Collaboration is a search for published trials. To be included in the collaboration, trials must unambiguously meet the strictest standards for randomization: there must be clear, documented procedures in place to prevent researchers guessing a patient’s allocation before they are registered on trial, or changing it afterwards. Typical procedures we look for are randomization at a separate statistical center implemented by a secure computer system, or sequentially numbered, opaque, sealed envelopes where these are held by a person not otherwise involved in the study. Trials must also investigate one of four pain conditions - osteoarthritis; chronic headache; non-specific back, neck or shoulder pain; shoulder pain associated with specific pathology (e.g. rotator cuff tendonitis) – and, as we are interested in chronic pain, outcome must be assessed more than four weeks after the first acupuncture treatment.

Once a study is determined to meet all of the Acupuncture Trialists’ Collaboration’s criteria, trialists are invited to send de-identified raw data to the collaboration’s statistical center at Memorial Sloan-Kettering Cancer Center. Data are then subject quality assurance checks, the most important of which is independent replication of all analyses described in the published report.

Phase two of the collaboration will begin after the systematic review is complete and data from all eligible trials is collected, checked, replicated, and combined. Well-annotated statistical code will be written for the meta-analysis by the principal statistician for the collaboration (Andrew Vickers) and research biostatistician (Angel Cronin) and distributed to all collaborators for comment before implementation. The code will also be published alongside the analyses.

The primary analysis will be to determine the effect size of acupuncture. Each trial will be evaluated by analysis of covariance with the principal endpoint as the dependent variable and, as covariates, the baseline score for the principal endpoint and the variables used to stratify randomization. The effect size for acupuncture from each trial (i.e. the coefficient and standard error) will then be entered into a meta-analysis. We will compute effect sizes for comparisons of acupuncture with usual care and acupuncture with sham. These analyses will be conducted separately for each pain condition (non-specific musculoskeletal pain, specific shoulder conditions, osteoarthritis, headache) and then within pain condition (e.g. separately for chronic tension headache and migraine).

Access to a large, individual patient-level dataset will allow a number of secondary questions to be addressed. For instance, it will let us examine variations in the effects of acupuncture by indication or by acupuncture characteristics, such as the duration or frequency of sessions. We will also be able to evaluate the effects of different types of sham and study the time course of acupuncture effects. The theory that certain people are predisposed to be acupuncture “responders” can be evaluated by analysis of the distribution of treatment effects, on the grounds that the presence of “responders” would lead to a skewed distribution. Using the same approach that meta-analysts use to investigate whether different studies have different results - what are known as “heterogeneity statistics” - we will determine if the effects of acupuncture vary between practitioners.

A key question in acupuncture research is the degree to which the type of acupuncture used affects outcome. For example, some practitioners claim that only traditional Chinese acupuncture is effective, whereas other say that acupuncture based on Western medical principles can also be of benefit. Accordingly, a group of both traditionally and Western trained acupuncturists, all members of the Acupuncture Trialists’ Collaboration, will describe the type of acupuncture used in each trial along various dimensions. Analyses will then be conducted to see which aspects of acupuncture are associated with outcome.

Since the Acupuncture Trialists’ Collaboration’s conception in 2007, we have made considerable progress. We have received start-up funds from the Samueli Institute and later received a substantial grant (R21 AT004189) from the National Institutes of Health. By June 2009, we expect to have complete data sets from 25 trials, including all of the recent large trials from Germany, UK, Spain and the US. We tentatively project that our first meta-analyses will be conducted early in 2010.

We believe that the findings of the Acupuncture Trialists’ Collaboration will have important implications for both clinical practice and research. Individual patient data meta-analysis of high quality trials will provide the most reliable basis for treatment decisions about acupuncture. Analyses as to the impact of different sham techniques, styles of acupuncture or frequency and duration of treatment sessions will no doubt guide future clinical trials.

Above all, however, we hope that our approach can serve as a model for future studies in acupuncture and other complementary therapies. In the Acupuncture Trialists’ Collaboration, a group of trialists, statisticians and other researchers has come together to share raw data and develop, in partnership, a set of research questions and associated analytic strategies. We strongly believe that it is only by breaking down the oppositional culture of competing trialists, and sharing data in a robust scientific collaboration, that we can best translate clinical trial findings into patient benefit.