Methodology guideline for clinical studies investigating traditional Chinese medicine and integrative medicine Executive summary

Abstract

This guideline aims to provide a methodological guidance for clinical studies in TCM and integrative medicine in terms of study design, execution, and reporting. The commonly used methods including experimental and observational methods were introduced in this guideline such as randomized clinical trials, cohort study, case-control study, case series, and qualitative method which can be incorporated into above quantitative methods. The guideline can be used for the evaluation of therapeutic effect of TCM therapies or their combination with conventional therapy. TCM therapy refers to one of the followings or their combination: herbal medicine, acupuncture, moxibustion, cupping, Taichi/Qigong, and Guasha, Tuina (therapeutic massage). It is also suitable for research and development of ethnopharmaceuticals or folk medicine.

1. Background and scope

The system of Traditional Chinese Medicine (TCM) is one of the whole- systems complementary and alternative medicine approaches. Clinical practice is typically based on pattern differentiation, prescription of herbal formulations or acupuncture regimen. Thus, clinical research of TCM should reflect its characteristics as a therapeutic system. Currently, there are three clinical research models. The first one is called ‘disease-pattern model’, where the international classification of diseases (ICD-10) in conjunction with TCM pattern differentiation is used for diagnosis. The second one is the ‘TCM defined disease or symptom model’ and the third one is the ‘pattern model’, which involves the targeting of a specific pattern rather than a disease or a symptom. TCM clinical practice, is based on a holistic approach, and the intervention includes dietary advice, behavior/life style change, as well as the herbal and/or acupuncture treatment. It is considered to be a complex intervention. Furthermore, in China, different TCM therapies are often integrated with conventional therapies and medications in what is also referred to as integrative health care.

This guideline aims to provide a methodological guidance for clinical studies of TCM and integrative healthcare including TCM in terms of study design, execution, and reporting. The role of commonly used experimental and observational methods are discussed in this guideline. These methods include randomized clinical trials, cohort studies, case-control studies, case series, as well as qualitative methods which can also be ‘nested’ into the abovementioned quantitative methods. This guideline can be used for the evaluation of the therapeutic effects of TCM, either on a standalone basis or as used in conjunction with conventional therapy. TCM refers to one of the following treatment modalities or their combination: herbal medicine, acupuncture, moxibustion, cupping, Taichi/Qigong, Guasha, and Tuina (therapeutic massage).

2. Essentials for design of clinical studies in traditional Chinese medicine and integrative medicine

2.1. Clinical question formulation

The most important aspect of a clinical research project is the formulation of a clear and answerable research question. The research question should be formulated based on a comprehensive review of the literature as well as on clinical experience, and it should be clinically relevant, feasible, innovative, ethical and of scientific research value.¹

2.2. Well defined objective

The objective should be well defined in a structured way, and include reference to the subjects with the disease or condition, the intervention, reference treatment(s) if appropriate, and outcome. Secondly, it should be clarified if the study is exploratory or confirmatory.

2.3. Design

The research design is determined by the research question(s)²: Questions relating to the efficacy of a TCM therapy in treating a particular condition can adopt an design such as placebo controlled, randomized clinical trial (RCT).

Questions relating to the effectiveness of TCM therapy in clinical practice can adopt a pragmatic design such as comparative effectiveness research.

Questions relating to the efficacy of TCM (including individual treatment) will require observational studies and/or a small sample exploratory pilot trial (feasibility study) before the conduct of larger confirmatory trials.

2.4. Population

The selection of subjects should be based on research purpose. In clinical trials, we should determine uniform diagnostic criteria (including western medicine disease and TCM syndromes), develop inclusion and exclusion criteria for the subjects. Well defined inclusion and exclusion criteria can help to recruit eligible participants and to avoid confounding bias.³

2.5. Sample

In clinical trials, a minimum sample size is required to test the hypothesis by a predefined statistical power. For exploratory studies in early stages of clinical research, sample size calculation may not be required. The sample size is calculated mainly based on the primary outcomes (depending on efficacy and/or safety endpoint), and other factors such as design, comparisons, testing hypotheses, type I and type II error parameters, should be considered as well.⁴

2.6. Intervention

In protocol of clinical trials, the definition of tested intervention should include name and definition of the tested intervention, dosage, administration, course of treatment, and any co-intervention. The dosage of TCM therapy is generally determined based on past experience of clinical use and preclinical research. Treatment course should be determined considering the development of the disease and the characteristics of the intervention. Co-interventions (if any) should be predefined; otherwise it will affect the efficacy and safety evaluation.^3,5

2.7. Comparator/reference treatment

The reference treatment should be selected in accordance with the study objective. In a comparative effectiveness study, we usually choose therapies that are commonly used in clinical practice as control to determine comparative effect for the subjects. In equivalence trials, standard western medicine therapy or medication could be used as control. In a dose–response study, different doses are compared. When investigating the specific effects of TCM therapy, a placebo should be used if the ethical approval for this is aquired.^3,5

2.8. Outcome measure

Outcome assessment includes the effect and safety, and sometimes cost- effectiveness. The effectiveness evaluation of TCM involves the specific outcomes related to disease and change of TCM syndromes. Depending on the study objectives, primary outcomes can be clinical endpoints, patient- related outcomes, and the secondary outcomes can be quality of life, or other surrogate outcomes such as biomarkers for disease. For evaluation of TCM syndromes, clinical symptoms scales or instruments with validated reliability and validity are preferred. Safety should be evaluated considering the target indications of the intervention, characteristics of the target population, treatment duration, administration, known target organ toxicity, TCM theories (herb matching) and previous experience of clinical use. For rare adverse events, long-term exposure data and sufficient numbers of participants are needed.⁶

2.9. Follow-up

TCM is commonly used in chronic diseases; hence a sufficient follow-up time should be ensured for evaluating the outcomes of TCM. The follow-up length and intervals depend on the purposes of the trials and on the characteristics of different TCM interventions.

3. Randomized clinical trials for TCM and integrative medicine

3.1. Introduction

RCTs are commonly used to evaluate the effect of medical interventions, health education or management.⁷ Since known or unknown confounding factors are controlled in RCT design, a cause-effect association can be established by comparing the outcomes between groups post-treatment. Consequently, the RCT is accepted as a ‘gold standard’ for evaluating therapeutic effects of specific drugs or procedures.⁸

However, limitations exist when conducting classical placebo-controlled RCTs for TCM therapies. Therapeutic effects can be influenced by factors other than the efficacy of the interventions themselves; such factors may include patient preference, practitioner preference, and the patient-practitioner relationship among others. Also, the absence of an ideal placebo control for most TCM treatments limits the application of the classical RCT model. Thus, some modified research models of RCT such as N of 1 trial, pragmatic trials, add-on design, expertise-based trials were introduced for assessing therapeutic effect of TCM therapies (including herbal medicine, acupuncture, moxibustion, cupping therapy, etc.).

3.2. Types of RCTs

3.2.1. Explanatory RCT

Explanatory RCTs test the efficacy in a research setting with highly selected participants and under highly controlled conditions.⁹ This design maximizes the balancing of any confounders, both observed and unobserved, with a view to evaluating the efficacy of the intervention compared to placebo or active drug. Explanatory RCTs are designed to test causal hypotheses, and evaluate the efficacy of the intervention which is strictly enforced in “ideal” setting. Standardized herbal extracts can be tested in this type of design. However, when interventions are complex or flexible, explanatory RCT may result in lower external validity therefore limiting its application.

3.2.2. Pragmatic RCT

Pragmatic RCT should meet the essentials of randomized study design. And the details of its design are more close to the “real world” clinical circumstances and conditions. Since it focuses on the comparison of the effectiveness between different interventions, it can be classified in the category of comparative effectiveness research.¹⁰

Despite randomization which is essential for a RCT, the use of blinding to treatment allocation depends on the studied treatments and conditions. Due to difficulties with this, there is no ideal placebo control for most of the TCM therapies. Especially for non-pharmaceutical therapies, blinding to participants and/or practitioners is almost impossible. However, we recommend that outcome assessors and statisticians are blinded in these studies in order to reduce the potential detection bias. Pragmatic RCTs are more likely to use equivalence or non-inferiority test hypotheses, the sample size is therefore often higher than that of explanatory RCTs.

Reporting of this type of study should follow the extension of the CONSORT statement for pragmatic RCTs.⁹

3.2.3. Crossover trial

A crossover trial is a type of longitudinal study in which subjects receive a sequence of different interventions. In a crossover trial, the influence of confounding covariates is reduced because each crossover patient serves as his or her own control.¹¹ Optimal crossover trials are statistically efficient and therefore require fewer subjects than non-crossover trials. However, “carry-over” effects between interventions may occur, and this can confound the estimates of the intervention effects. Although “carry-over” effects can be avoided by a sufficiently long “wash-out” period based on knowledge of the dynamics of the intervention, the latter is often unknown. Therefore, crossover trials are usually applied in trials which investigate the short-term efficacy of interventions in stable diseases.

3.2.4. N-of-one trial

A number of one (N-of-one) trial is a special type of crossover trial which involves multiple treatment crossovers within a single subject. This experimental research design can be repeated several times to confirm the effectiveness of a particular treatment.¹² Compared to a case report (which is an observational study of an individual patient who was exposed to a certain intervention that produced an outcome of interest), a N-of-one trial is a rigorous, experimental research design. It can be used to detect the response of an individual rather than a group of participants to a treatment, based on changes in the type and dosage of the drug. It can be used for optimizing the treatment within an individual participant, and thus limits its generalization.

Reporting of the N-of-1 trial should also follow the extension of the CONSORT statement.¹³

3.2.5. Add-on design

Add-on designs are used to evaluate the effect of TCM therapy in addition to current standard therapy. When the standardized therapy is available for a disease, it is unethical to interrupt this therapy when giving participants the interested intervention, such as herbal remedy.¹⁴

3.2.6. Dose–response study

A dose-response study is a valid research design to evaluate the effectiveness of traditional Chinese herbal medicine, and to determine the best dose of the herbal product.¹⁵ Dose response studies can generally be classified into parallel groups dose response studies, cross-over dose response studies, and dose titration therapy. A dose–response curve could be drawn with the fitting method based on data from each dose group. The relationship between dose and response could be tested through the regression analysis of the curve, rather than the effectiveness comparison between each two different dose groups.

3.2.7. Factorial design

A factorial design allows investigators to evaluate more than one intervention in a single experiment, whether testing the treatment effects independently or investigating the treatment interactions.¹⁶

3.2.8. Cluster randomized trial

If the individual subject is not suitable to be randomly assigned to the intervention or control group, or in case large samples are required, a cluster randomized trial can be considered. Also, if patients in control group may have risk of being exposed to the intervention, the cluster trial can be considered. For example, in the study of Chinese herbal medicine, this method can be used to mask the difference between the real herbal product/decoction and the placebo, avoid loss of blinding to treatment allocation. An extension of the CONSORT statement is recommended for reporting a cluster RCT.¹⁷

3.2.9. Expertise-based randomized trial

If the interested interventions are skill-based, an expertise based randomized trial will allocate subjects to clinicians with expertise in intervention A or clinicians with expertise in intervention B, so clinicians perform only the procedure they are expert in.¹⁸

3.2.10. Zelen’s design

To minimize the participants’ non-compliance after randomization, Zelen’s design allocates participants to either the treatment or control group before obtaining informed consent. In Zelen’s design, when participants giving their inform consents, the treatments they will accept in this trial are known, which is different from other design types with randomization methods.¹⁹

3.2.11. Adaptive trial

In adaptive trials the protocol is revised according to the preliminary test result under the premise of not destroying the integrity and validity of the study.²⁰ This design aims to modify the protocol during trial, in order to reduce the research cost and shorten the trial duration. Examples of adaptive designs include the re-calculation of sample size based on a pre-planned interim analysis, response adaptive randomization, and selective intervention according to the principle of dropping-the-loser.^21,22

3.2.12. Partially randomized patient preference trial

A partially randomized, patient preference (PRPP) trial has been recommended for use in trials with potential performance bias.²² This model was first applied for evaluating the therapeutic effect of surgery as compared with drugs, in which blinding methods could not be used due to the obvious inconformity between the intervention and control. Given the limitations of a classical RCT, the PRPP model may be more suitable for evaluating non-placebo controlled TCM therapies. In this type of study, patients who do not have preference to the treatments are randomly divided into intervention or control groups; and those who have strong preference to the modalities receive the treatment they choose. Thus, the therapeutic effect of two or more target treatments could be evaluated whilst accounting for patient preference.

To calculate the sample size and analyze the data for nonrandomized arms in a PRPP trial should use the related methods for observational studies (such as cohort study).²³

4. Cohort study

A cohort study, also known as follow-up study, is a major type of observational analytic design. The term “cohort” in modern epidemiology means “a group of people with defined characteristics who are followed up to determine incidence of, or mortality from, some specific disease, all causes of death, or some other outcome.”²⁴ In a cohort study, an outcome or disease-free study population is first identified by the exposure or event of interest and followed in time until the disease or outcome of interest occurs.²⁵ Because exposure is identified before the outcome, cohort studies have a temporal framework to assess causality between exposures and diseases and thus have the potential to provide stronger scientific evidence.²⁶ In clinical research investigating TCM, cohort studies can be used for comprehensive intervention such as integrative care to estimate the outcome in different cohorts.

4.1. Study design

Data collection in cohort studies can be prospective and/or retrospective. Prospective cohort study is carried out from the present time to future. When the study is initiated, the investigator enrolls participants and then follows-up for the outcomes (events of interest) to occur. The limitation is inefficient for investigating outcomes with long latency periods and low incidence rate, and is vulnerable to a high loss to follow-up for long period of follow-up.²⁵ Retrospective or historical cohort studies look to the past to examine medical records for exposure events. As all the relevant events have already occurred when the study is initiated, this study is quicker and cheaper than prospective cohort studies. However, the existing data may be incomplete, inaccurate, or inconsistently measured between subjects.²⁷ The choice of retrospective or prospective cohort is based on scientific and logistic considerations. For example, it might be better to use prospective cohort studies to assess the acupuncture effectiveness because of the short induction period and high outcome occurrence rate. However, retrospective cohort studies may be more suitable to observe the adverse events of herbal medications, due to the long period follow up and low incidence rate.

A registry study is an organized system to collect medical data to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.²⁸ A registry database is a file (or files) derived from the registry. Registry study can facilitate the conduct of cohort studies and help with determining the clinical effectiveness and cost-effectiveness in real world clinical practice.

4.2. Exposure definition and grouping

No “one size fits all” solution exists for exposure measurement. Each intervention (e.g., herbal medicine, acupuncture, integration medicine) requires a unique and thoughtful exposure definition and measurement.²⁹ Indeed, the cumulative dose, or total amount of exposure over a specified time period, is often optimal for adequately defining exposure. For example, based on the cumulative dosage or intensity during the herbal medicine treatment, the participants can be categorized into multi-levels exposure groups (high/middle/low groups). In addition, exposure can be characterized dichotomously (e.g., ever/never) or categorically during the measurement time periods.

Usually, a cohort study may include a comparison group, but it is not always the case. Cohort study without comparison is easy confused with case series. The main difference is that a cohort study sampled the participants based on exposure, and follow-up is part of the cohort study, however, a case series is a study which sampled the participants only based on the outcome/disease, either those who have an exposure or not.³⁰

4.3. Selection of subjects

In cohort studies, the groups being compared should be as similar as possible with respect to factors related to the disease except for the exposure factor. Therefore, the subjects both in the exposure and non-exposure groups should be selected from the same source population.³¹ Subjects may be effectively sampled from the hospital, be members of a community, or from a doctor’s individual practice.²⁵

4.4. Outcome definition and measurement

The selection of outcomes in cohort studies should consider the viewpoints of multiple stakeholders (provider, patient, payer, regulatory, industry, academic and societal) and the intended use for decision making of resulting evidence.²⁹ The outcome may focus on clinical outcomes, such as cancer survival time, disease mortality; or general health-related quality of life measures, such as the SF-36; and/or health resource utilization or cost measures. Intermediate or surrogate outcomes are also used in cohort studies, such as biological marker, which may reflect the follow-up end point outcome. It is important that researchers should pay equal attention and use the same measurements across different exposure groups.

4.5. Statistic analysis

Descriptive and univariate analysis can be used to assess the baseline balance between different observational groups and to identify covariates associated with exposure or outcome. Linear and logistic multivariable regression models are the traditional strategies to adjust for confounding. Propensity scores are increasingly used in cohort study as a multi-covariates adjusting technique, and particularly useful in studies with a common exposure and rare outcome.³² Disease risk scores estimate the probability or rate of disease occurrence as a function of the covariates and are preferred in studies with a common outcome and rare exposure. Instrumental variables is an alternative analytic strategy to estimate the causal effect for which has incomplete information on potential unmeasured confounders.²⁹

5. Case-control study

5.1. Study design

A case-control study is an observational study in which two existing groups differing in outcome are identified and compared on the basis of some supposed causal attribute. In comparative effectiveness research, case-control study can be used to evaluate therapeutic effects. The cases and controls are identified according to outcomes (i.e., cured or not cured, improved or not improved), and the exposure(s) are identified as past treatment(s). By comparing the different treatments of the case group with the control group, the association of treatments with outcomes can be determined. Usually, case-control studies are nested within an underlying cohort; the case-control design identifies all incident cases that develop the outcome of interest and compares their treatment history with the treatment history of controls sampled at random from everyone within the cohort still at risk for developing the outcome of interest.³³ Given proper sampling of controls from the risk set, the estimation of the odds ratio in a case-control study is a computationally more efficient way to estimate the otherwise identical incidence rate ratio in the underlying cohort.

5.2. Important considerations

5.2.1. Selection of cases

When case-control study is used to evaluate therapeutic effects, the criterion for the classification of the ‘outcome group’ and the ‘control group’ is whether a defined clinical outcome occurred. Clinical outcomes can be qualitative or quantitative. To determine the outcome, it is recommended to use international or national criteria.

5.2.2. Selection of control

Controls need to be sampled from all patients from the underlying cohort who remain at risk for the outcome at the time a case occurs. Sampling of controls from all those who enter the cohort (i.e., at baseline) may lead to biased estimates of treatment effects if treatments are associated with loss to follow up or mortality. Matching on important confounders, e.g., age, sex and race, should be considered to improve the efficiency of estimation of treatment effects. However, overmatching (matching on factors strongly associated with treatment) should be avoided; otherwise, we could lose the ability to estimate the effect of the matching variable on the outcome.

5.3. Comment

In western medicine, this study is used to evaluate therapeutic effect. For example, Martinez et al. conducted a case-control study comparing venlafaxine and other antidepressants and risk of sudden cardiac death or near death.³³ An existing cohort of new users of antidepressants was identified. Nested within the underlying cohort, cases and up to 30 randomly selected matched controls were identified. Potential controls were assigned an “index date” corresponding to the same follow up time to event as the matched case. Controls were only sampled from the “risk set”. That is, controls had to be at risk for the outcome on their index date, thus ensuring that bias was not introduced via the sampling scheme.

Case-control study could also be used to evaluate therapeutic effect in TCM. However, in this case, one of the challenge problems is the identification of ‘exposure’, that is, the past treatment, especially when the past treatment is comprehensive intervention. Expert consensus could be used to determine the exposure, when taking into account the initiation, duration and intensity of the past treatment.

6. Case series

6.1. Introduction

Case series are collections of individual case reports, which may occur within a fairly short period of time. The collection of a case series rather than reliance on a single case report can mean the difference between formulating a useful hypothesis and merely documenting an interesting medical oddity.³⁴

Case series can be used for a variety of purposes: to observe patients who would not meet the inclusion criteria of clinical controlled trials, and provide potential evidence for those populations; or target on the special disease (tumor, AIDS, atypical pneumonia, etc.), rare or chronic diseases, complications and adverse reactions; For the study of patients in which controlled trial data is unavailable for ethical reasons or due to an unwillingness to be randomized due to a strong preference of patients for a specific therapy.³⁵

6.2. Key points in study design

Without a control group, case series study only has one group of participants with a specific outcome (a disease or disease-related outcome).³⁶ Some case series also sample the patients with a specific exposure (beside an outcome). However, comparing to a small cohort study, the specific outcome had already happened before carrying out the case series.³⁰ Though its retrospective nature in selecting the participants and reviewing the data, a case series could be conducted in a planned way. The plan should be developed according to the objectives, clearly defined diagnostic, inclusion and exclusion criteria of participants, and the data analysis procedures. Reporting of case series study could be in accordance with the CARE.³⁶

6.3. Limitations

Though it is easy and relatively inexpensive to carry out case series studies, it is generally considered to have many potential confounding factors, and likely to overestimate the effect of the observation. Furthermore, it cannot be used to test for the presence of a valid statistical association or to make an internally valid statement about the effectiveness of treatment.

7. Qualitative research

Qualitative research focuses on the understanding of certain interactions or actions in the natural environment. The commonly applied methods including (1) observational study, where the researcher obtain direct information by observing events, phenomenon or people; (2) in-depth interview, where the researcher as the interviewer go through in depth interview with the participant as the interviewee, in order to get the whole story related to the research question; (3) focus groups; where the researcher records the discussion of a group of people who might have same or different opinions on a series of questions. There are many more methods, e.g., Delphi process, the normal group, consensus method, case study and documentary source. All the methods should be based on the understanding of background knowledge as well as the study setting in order to decide on proper research question and methodology.

Differing from quantitative research, qualitative research answers the “why”, “how” or “what” questions rather than the “how many” or “whether” questions. It can provide deeper, more individualized and also more culture-sensitive information,.

In TCM, qualitative research is suitable for (1) exploring an unexplored/complex area/phenomenon, e.g., to identify the key elements during the TCM diagnosis/treatment process³⁷; (2) studying patients/practitioners/family members’ attitudes, beliefs, expectations and/or experiences of a certain treatment or phenomenon^38,39; (3) identifying people’s different opinions about conflicting phenomena⁴⁰; (4) to provide in depth information in addition to quantitative data collected in clinical trials.⁴¹

In conclusion, since TCM has a typical culture and philosophy feature, qualitative methods can be incorporated into quantitative research methods at different stages. This could become a mixed method for a complicated intervention such as TCM.

Appendix A

Members of the guideline development team: Jian-Ping Liu¹, Ke-Ji Chen², Yu-Tong Fei¹, Rui Gao², Wei Chen¹, Hui-Juan Cao¹, Ying Zhang¹, Zhao-Lan Liu¹, Xun Li¹, Mei Han¹, Li-Qiong Wang¹, Mei Wang³, Guo-Yan Yang¹, He Yu¹, Yun Xia¹, Ni-Ni Chen¹, Shuo Feng¹, Jin Sun¹, Andrew Flower⁴, Claudia Witt⁵, George Lewith⁴, Gudrun Ulrich-Merzenich⁶, Kenneth Muire⁷, Martina Prude⁴.

1 Beijing University of Chinese Medicine, China; 2China Academy of Chinese Medical Science, China; 3 Liaoning University of Traditional Chinese Medicine, China; 4 Southampton University, UK; 5Charité Universitätsmedizin, Germany; 6 Rheinische Friedrich- Wilhelms-University of Bonn, Germany; 7 University of Warwick, UK.