Many randomised controlled trials have been conducted in China to evaluate the effectiveness of traditional Chinese medicine, but much of the information is inaccessible to Western doctors. We estimated the total number of randomised controlled trials published in China and identified problems in applying such methodology to the evaluation of traditional Chinese medicine, which would serve as preparatory work for systematic review and dissemination of the randomised evidence for such medicine.
Methods and results
We randomly selected 28 journals using stratified sampling from a total of 100 Chinese journals of traditional Chinese medicine (4 national, 10 university, 10 provincial or regional, and 4 specialist journals). After special training, eight fifth year medical students (working in pairs) hand searched all the issues of the journals published before 1 January 1997 to identify randomised controlled trials. Discrepancies were settled by one of the principal investigators (S-YZ). Data on methodological quality of randomised controlled trials were extracted from 414 full length articles in the Chinese Journal of Integrated Traditional and Western Medicine. Ten times as many randomised controlled trials appeared in that journal as in the other journals, and those published in that journal were of a higher quality.1,2
Altogether, 2938 randomised controlled trials were identified in the 28 selected journals. The first trials were published in the early 1980s. The number of trials had doubled every two to three years over the past 15 years. The number of randomised controlled trials published in all 100 journals by the end of 1996 was estimated to be around 7500 (95% confidence interval 6000 to 9000). Comparison of hand searched trials with trials of traditional Chinese medicine found in electronic databases (which hold journals of conventional medicine too) shows that journals of conventional medicine in China published about a quarter of the number randomised controlled trials published in journals of traditional Chinese medicine. Thus, almost 10 000 randomised controlled trials were published in China before 1997.
In most of the trials, disease was defined and diagnosed according to conventional medicine; trial outcomes were assessed with objective or subjective (or both) methods of conventional medicine, often complemented by traditional Chinese methods. Over 90% of the trials in non-specialist journals evaluated herbal treatments that were mostly proprietary Chinese medicines. The 10 most common diseases in the trials were ischaemic heart disease, stroke, chronic viral hepatitis, peptic ulcer, childhood diarrhoea, hyperlipidaemia, primary hypertension, upper digestive tract bleeding, diabetes mellitus, and pneumonia. They accounted for a fifth of the trials.
Although methodological quality has been improving over the years, many problems remain.1,2 The method of randomisation was often inappropriately described. Blinding was used in only 15% of trials. Only a few studies had sample sizes of 300 subjects or more. Many trials used as a control another Chinese medicine treatment whose effectiveness had often not been evaluated by randomised controlled trials. Most trials focused on short term or intermediate rather than long term outcomes. Most trials did not report data on compliance and completeness of follow up. Effectiveness was rarely quantitatively expressed and reported. Intention to treat analysis was never mentioned. Over half did not report data on baseline characteristics or on side effects. Many trials were published as short reports. Most trials claimed that the tested treatments were effective, indicating that publication bias may be common; a funnel plot of the 49 trials of acupuncture in the treatment of stroke confirmed selective publication of positive trials in the area, suggesting that acupuncture may not be more effective than the control treatments (figure).3
Comment
The quality of trials of traditional Chinese medicine must be improved urgently. Large and well designed randomised controlled trials on long term major outcomes should be funded.4 Subsequently, such studies may serve as models for future trials in the area. Treatments to be tested should be selected so that potentially effective and important treatments are evaluated first. The best evidence should be systematically reviewed, summarised, and disseminated, which in turn would lead to evidence based decision making in traditional Chinese medicine.5
Figure.
Funnel plot of overall efficacy of acupuncture in treatment of stroke (49 trials), according to trial size
Acknowledgments
We thank Dr Li-Ming Lee for help in project management and Drs Tao Wu, Dong Bei, Shelly Tse for help in data collection and management.
Footnotes
Funding: Chinese University of Hong Kong (grant No 2040518).
Competing interests: None declared.
References
- 1.Yu GP, Gao SW, Li Y, Gong W. A study of the quality of clinical trials in traditional Chinese medicine. Chinese Journal of Integrated Traditional and Western Medicine. 1994;14:50–53. [PubMed] [Google Scholar]
- 2.Xie ZF, Li N. Methodological analysis of clinical articles on therapy evaluation published in Chinese Journal of Integrated Traditional and Western Medicine. Chinese Journal of Integrated Traditional and Western Medicine. 1995;1:3016. [Google Scholar]
- 3.Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Controlled Clin Trials. 1998;19:159–166. doi: 10.1016/s0197-2456(97)00150-5. [DOI] [PubMed] [Google Scholar]
- 4.Collins R, Peto R, Gray R, Parish S. Large-scale randomised evidence: trials and overviews. In: Weatherall DJ, Ledingham JGG, Warrell DA, editors. Oxford textbook of medicine. 3rd ed. Oxford: Oxford University Press; 1996. pp. 21–32. [Google Scholar]
- 5.Chalmers I. The Cochrane Collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann N Y Acad Sci. 1993;703:156–172. doi: 10.1111/j.1749-6632.1993.tb26345.x. [DOI] [PubMed] [Google Scholar]