Abstract
Objective
To identify redundant clinical trials evaluating statin treatment in patients with coronary artery disease from mainland China, and to estimate the number of extra major adverse cardiac events (MACEs) experienced by participants not treated with statins in those trials.
Design
Cross sectional study.
Setting
2577 randomized clinical trials comparing statin treatment with placebo or no treatment in patients with coronary artery disease from mainland China, searched from bibliographic databases to December 2019.
Participants
250 810 patients with any type of coronary artery disease who were enrolled in the 2577 randomized clinical trials.
Main outcome measures
Redundant clinical trials were defined as randomized clinical trials that initiated or continued recruiting after 2008 (ie, one year after statin treatment was strongly recommended by clinical practice guidelines). The primary outcome is the number of extra MACEs that were attributable to the deprivation of statins among patients in the control groups of redundant clinical trials—that is, the number of extra MACEs that could have been prevented if patients were given statins. Cumulative meta-analyses were also conducted to establish the time points when statins were shown to have a statistically significant effect on coronary artery disease.
Results
2045 redundant clinical trials were identified published between 2008 and 2019, comprising 101 486 patients in the control groups not treated with statins for 24 638 person years. 3470 (95% confidence interval 3230 to 3619) extra MACEs were reported, including 559 (95% confidence interval 506 to 612) deaths, 973 (95% confidence interval 897 to 1052) patients with new or recurrent myocardial infarction, 161 (132 to 190) patients with stroke, 83 (58 to 105) patients requiring revascularization, 398 (352 to 448) patients with heart failure, 1197 (1110 to 1282) patients with recurrent or deteriorated angina pectoris, and 99 (95% confidence interval 69 to 129) unspecified MACEs.
Conclusions
Of more than 2000 redundant clinical trials on statins in patients with coronary artery disease identified from mainland China, an extra 3000 MACEs, including nearly 600 deaths, were experienced by participants not treated with statins in these trials. The scale of redundancy necessitates urgent reform to protect patients.
Introduction
When investigators overlook existing evidence, clinical trials might be initiated to deal with treatment uncertainty that has already been solved by previous studies.1 Failing to establish equipoise, such clinical trials are deemed redundant and inappropriate by the research community as they waste resources and put patients at risk of harm. This unnecessary replication is highly problematic in the context of placebo controlled trials, which are legitimate only if no known treatment option exists.2 3 Otherwise, patients who only take placebo in the control group are denied a known effective treatment, violating the ethical principles of conducting clinical trials.4
Cumulative meta-analysis can be used to show the chronological change of the overall estimate of a treatment effect as individual trials are added to the evidence pool.5 Cumulative meta-analysis has been adapted to evaluate when sufficient evidence has accrued to reach a conclusion and, subsequently, to identify the redundancy of additional trials.5 On the one hand, it is challenging for investigators to decide the adequacy of existing evidence solely based on cumulative meta-analysis owing to the varied interpretation of the results, as well as the possibility of the overall estimates at early stages being overridden by subsequent trials.6 On the other hand, clinical practice guidelines, which ideally are both consensus oriented and based on systematic reviews, take into account the balance of harms and benefits as well as feasibility and patient values. Therefore, clinical practice guidelines are more comprehensively adopted by the research community to deal with treatment uncertainty on the basis of cumulative meta-analysis.7
The research community has witnessed a recent proliferation of scientific publications from mainland China.8 However, there are concerns over the redundancy of those research activities.9 We evaluated the potential redundancy of clinical trials from mainland China. Specifically, we identified randomized clinical trials evaluating statins for the treatment of coronary artery disease that were conducted after the benefits of statins were affirmed by clinical practice guidelines and cumulative meta-analysis. We estimated the number of major adverse cardiac events (MACEs) that were experienced by patients who did not receive statins in the redundant trials.
Methods
The data collected and analyzed in our study were retrieved from publications of clinical trials. We followed the strengthening the reporting of observational studies in epidemiology (STROBE) reporting guideline.10
Eligibility criteria
We conducted a literature review to identify eligible trials, defined as randomized clinical trials comparing statins with placebo or no treatment among patients recruited from mainland China who were diagnosed as having coronary artery disease, including stable angina pectoris and acute coronary syndrome. Acute coronary syndrome consisted of unstable angina pectoris and myocardial infarction (eg, acute myocardial infarction, history of myocardial infarction, non-ST elevation myocardial infarction, ST elevation myocardial infarction). We excluded ischemic heart failure because the benefits of statins had not been confirmed among patients with this condition.11 Seven types of statins were included: lovastatin, simvastatin, atorvastatin, rosuvastatin, pravastatin, fluvastatin, and pitavastatin. We also included clinical trials that compared statins and other drug treatments as a combination with placebo or no treatment. Clinical trials were excluded if surgical procedures, such as percutaneous coronary intervention and coronary artery bypass grafting, were conducted as part of the treatment.
We included clinical trials published as journal articles in Chinese or English until December 2019. Protocols, systematic reviews, and meta-analyses were excluded.
Literature search
We searched three English bibliographic databases (PubMed, Embase, and Cochrane Controlled Register of Trials (CENTRAL)), plus four Chinese bibliographic databases (SinoMed (formerly known as Chinese Biomedical Database), the China National Knowledge Infrastructure (CNKI), Wanfang Data, and the China Science and Technology Journal Database (VIP, http://qikan.cqvip.com)).12 The supplemental file presents the search terms. We performed the initial search on 1 January 2020 and completed an updated search on 19 October 2020. Two authors (YJ and JW) independently screened the titles or abstracts, or both, and full text articles. Discrepancies were resolved through discussion with a third author (XW).
Definition of redundant trials
In March and April 2007, two clinical practice guidelines developed by the Chinese Society of Cardiology of the Chinese Medical Association were published that strongly recommended (benefits outweigh harms based on “grade A” evidence) statin treatment for patients with stable angina pectoris and acute coronary syndrome, respectively.13 14 We defined redundant trials as those that were either initiated or continued recruiting new patients at least one year after the clinical practice guidelines were published. Therefore, March 2008 was the cut-off point for stable angina pectoris and April 2008 was the cut-off point for acute coronary syndrome or a mixture of stable angina pectoris and acute coronary syndrome. The one year lag was added to allow trialists to adopt the clinical practice guidelines and take action.
The rationale behind the choice of March and April 2008 as cut-off points was further supported by two assumptions. First, cumulative meta-analysis based on eligible trials had established the benefits of statins for coronary artery disease by March and April 2008, and, second, the findings of eligible trials published before March and April 2008 were consistent, undermining the treatment uncertainties to justify subsequent clinical trials.
Correspondingly, we conducted cumulative meta-analyses with random effect to establish the time points when statins were shown to statistically significantly reduce the incidence of a combination of death, new or recurrent myocardial infarction, and revascularization. We conducted three cumulative meta-analyses for patients with stable angina pectoris, unstable angina pectoris, and myocardial infarction.15 The conclusion of eligible trials on the efficacy and safety of statins was abstracted, for example, whether statin treatment was effective or safe based on the findings.
Statistical analysis
Descriptive analysis—Among eligible trials, we report the distribution of coronary artery disease by type, type of statins, language of publication, funding source, and whether approval from an ethics committee was reported. We also calculated the number of patients enrolled, the number of patients treated in the control group, and the person years of patients treated in the control group.
Primary outcome—The primary outcome was the number of extra MACEs experienced by patients who did not receive statins in the redundant trials. To accommodate the wide range of clinical events reported by individual trials, we defined MACE broadly to include all cause mortality or cardiac related mortality, new or recurrent myocardial infarction, stroke, heart failure, revascularization, and recurrent or exacerbated angina pectoris. We estimated the extra MACEs by risk difference between the statin and control groups in individual trials—that is, the difference between the observed MACEs in the control group and the expected MACEs if patients in the control group were treated as in the statin group. When a redundant trial was initiated before the cut-off time point, we only included the patients recruited after the cut-off point, assuming a constant recruiting process. Bootstrapping was used to construct the 95% confidence intervals of extra MACEs. Supplemental file 2T reports the method in detail. The analysis was conducted in SAS 9.4.
Sensitivity analysis—We explored multiple cut-off time points to define redundant trials, including immediately, six months, two years, and five years after publication of the clinical practice guidelines, as well as the ones established by cumulative meta-analyses. When the patients in a trial were covered by multiple cumulative meta-analyses, we used the most recent time point.
Patient and public involvement
Patients or the public were not involved in the design, implementation, interpretation, or writing up of the study. The study was based on published literature without direct contact of patients or the public.
Results
Characteristics of eligible trials
In total, 2577 eligible trials had been published by December 2019 (fig 1). Of 250 810 enrolled patients, 121 722 were treated without statins in the control group for 32 428 person years. Of the 2577 trials, 1022 (39.7%) recruited patients with unspecified types of coronary artery disease, and more than a half (1474, 57.2%) evaluated atorvastatin (table 1). Most of the trials (2560, 99.3%) were published in Chinese; only 98 (3.8%) trials reported funding sources, mainly from government agencies (86, 3.3%); 272 (10.6%) reported approval from an ethics committee; and 548 (21.3%) reported MACEs. None of the eligible trials were registered in trial registries. Figure 2 presents the number of eligible trials and the number of patients treated in the control group by year of publication.
Fig 1.
Selection of eligible trials. CENTRAL=Cochrane Controlled Register of Trials
Table 1.
Characteristics of eligible trials
| Characteristics | No (%) of eligible trials | ||
|---|---|---|---|
| Considered redundant* | Not considered redundant* | Total | |
| No of trials | 2045 (79.4) | 532 (20.6) | 2577 (100.0) |
| No of patients enrolled | 207 317 (82.7) | 43 493 (17.3) | 250 810 (100.0) |
| Control group: | |||
| No of patients treated | 101 486 (83.4) | 20 236 (16.6) | 121 722 (100.0) |
| Person years of patients treated | 24 638 (76.0) | 7791 (24.0) | 32 428 (100.0) |
| No of trials reporting MACEs | 360 (65.7) | 188 (34.3) | 548 (100.0) |
| Disease condition: | |||
| Unspecified coronary artery disease | 962 (47.0) | 60 (11.3) | 1022 (39.7) |
| Angina pectoris | 607 (29.7) | 221 (41.5) | 828 (32.1) |
| Acute coronary syndrome | 225 (11.0) | 202 (38.0) | 427 (16.6) |
| Myocardial infarction | 219 (10.7) | 43 (8.1) | 262 (10.2) |
| Other | 32 (1.6) | 6 (1.1) | 38 (1.4) |
| Type of statins: | |||
| Atorvastatin | 1291 (63.1) | 183 (34.4) | 1474 (57.2) |
| Simvastatin | 357 (17.5) | 232 (43.6) | 589 (22.9) |
| Rosuvastatin | 265 (13.0) | 1 (0.2) | 266 (10.3) |
| Other or unclear | 132 (6.4) | 116 (21.8) | 248 (9.6) |
| Funding: | |||
| Government | 74 (3.6) | 12 (2.3) | 86 (3.3) |
| Other | 11 (0.5) | 1 (0.2) | 12 (0.5) |
| Not reported | 1960 (95.9) | 519 (97.5) | 2479 (96.2) |
| Ethics committee approval: | |||
| Reported | 265 (13.0) | 7 (1.3) | 272 (10.6) |
| Not reported | 1780 (87.0) | 525 (98.7) | 2305 (89.4) |
| Language of journal article: | |||
| Chinese | 2039 (99.7) | 521 (97.9) | 2560 (99.3) |
| English | 6 (0.3) | 11 (2.1) | 17 (0.7) |
MACEs=major adverse cardiac events.
March 2008 was cut-off point for stable angina pectoris and April 2008 was cut-off point for acute coronary syndrome or a mixture of stable angina pectoris and acute coronary syndrome.
Fig 2.
Number of eligible clinical trials on statin treatment in patients with coronary artery disease and number of patients treated in control group by year of publication
Characteristics of redundant trials
Among 2577 eligible trials, 2045 (79.4%) were considered redundant because they were either initiated or continued recruiting after March or April 2008. In total, 207 317 (82.7%) patients were enrolled in redundant trials, of whom 101 486 were treated in the control group without statins for 24 638 person years. Overall, 962 (47.0%) of 2045 redundant trials recruited patients with unspecified types of coronary artery disease, and more than a half (1291, 63.1%) evaluated atorvastatin (table 1). Most redundant trials (2039, 99.7%) were published in Chinese; only 85 (4.1%) reported funding sources, mainly from government agencies (74, 3.6%); 265 (13.0%) reported approval from an ethics committee; and 360 (17.6%) reported MACEs.
Validation on March and April 2008 as cut-off points
The assumptions supporting March and April 2008 as the cut-off points were validated. First, cumulative meta-analyses showed that statins could statistically significantly reduce the incidence of a combination of death, new or recurrent myocardial infarction, and revascularization among patients with unstable angina pectoris by 2002 and among patients with stable angina pectoris or myocardial infarction by 2004 (supplemental files 3-5), several years earlier than March and April 2008.
Second, 375 (14.6%) of 2577 eligible trials were published before March or April 2007, all of which consistently reiterated that statins were beneficial and safe for patients with coronary artery disease. Nearly all (2573, 99.8%) the eligible trials reached this conclusion.
Primary outcome
In 360 redundant trials that reported MACEs, 22 380 patients were treated in the control group for 6766 person years. In total, 3470 (95% confidence interval 3323 to 3619) extra MACEs were reported, including 559 (95% confidence interval 506 to 612) deaths, 973 (95% confidence interval 897 to 1052) patients with new or recurrent myocardial infarction, 161 (132 to 190) patients with stroke, 83 (58 to 105) patients requiring revascularization, 398 (352 to 448) patients with heart failure, 1197 (1110 to 1282) patients with recurrent or deteriorated angina pectoris, and 99 (95% confidence interval 69 to 129) unspecified MACEs (table 2).
Table 2.
Number of extra major adverse cardiac events (MACEs)
| Item | Primary analysis | Sensitivity analysis | |||||
|---|---|---|---|---|---|---|---|
| 1 year lag | Based on CMA | No lag | 6 month lag | 2 year lag | 5 year lag | ||
| Cut-off point: | |||||||
| Stable angina pectoris | Mar 2008 | Jan 2003 | Mar 2007 | Sep 2007 | Mar 2009 | Mar 2012 | |
| Acute coronary syndrome | Apr 2008 | Jan 2005 | Apr 2007 | Oct 2007 | Apr 2009 | Apr 2012 | |
| No of redundant trials: | |||||||
| All | 2045 | 2356 | 2143 | 2090 | 1919 | 1478 | |
| Those reporting MACEs | 360 | 458 | 393 | 371 | 326 | 207 | |
| No of redundant trials initiated after cut-off point: | |||||||
| All | 1731 | 2147 | 1864 | 1816 | 1602 | 1158 | |
| Those reporting MACEs | 272 | 390 | 309 | 294 | 230 | 156 | |
| No of redundant trials continued recruiting after cut-off point: | |||||||
| All | 314 | 209 | 279 | 274 | 317 | 320 | |
| Those reporting MACEs | 88 | 68 | 84 | 77 | 96 | 51 | |
| No of participants treated in control group: | |||||||
| All | 95 892 | 111 419 | 101 250 | 98 743 | 89 288 | 67 786 | |
| Those reporting MACEs | 22 380 | 27 271 | 23 845 | 23 149 | 20 486 | 14 555 | |
| No of person years treated in control group: | |||||||
| All | 17 318 | 22 049 | 18 760 | 18 083 | 15 508 | 10 480 | |
| Those reporting MACEs | 6766 | 9659 | 7530 | 7153 | 5840 | 3518 | |
| No of extra MACEs (95% CI): | |||||||
| Death | 559 (506 to 612) | 711 (652 to 772) | 609 (555 to 667) | 585 (530 to 640) | 491 (442 to 539) | 275 (238 to 310) | |
| New or recurrent myocardial infarction | 973 (897 to 1052) | 1292 (1198 to 1382) | 1085 (1000 to 1164) | 1029 (950 to 1110) | 845 (774 to 920) | 584 (524 to 648) | |
| Revascularization | 83 (58 to 105) | 130 (100 to 158) | 100 (73 to 125) | 93 (66 to 117) | 60 (40 to 78) | 16 (6 to 25) | |
| Heart failure | 398 (352 to 448) | 495 (442 to 546) | 422 (374 to 473) | 410 (362 to 459) | 364 (321 to 412) | 215 (183 to 251) | |
| Stroke | 161 (132 to 190) | 203 (170 to 236) | 176 (145 to 206) | 168 (137 to 198) | 148 (121 to 175) | 110 (88 to 132) | |
| Recurrent or deteriorated angina pectoris | 1197 (1110 to 1282) | 1560 (1459 to 1665) | 1305 (1216 to 1397) | 1253 (1164 to 1341) | 1050 (969 to 1131) | 683 (609 to 752) | |
| Unspecified MACEs | 99 (69 to 129) | 144 (111 to 177) | 100 (70 to 130) | 100 (70 to 130) | 93 (63 to 120) | 28 (9 to 45) | |
| Total MACEs | 3470 (3323 to 3619) | 4535 (4367 to 4702) | 3795 (3651 to 3955) | 3638 (3487 to 3789) | 3054 (2916 to 3185) | 1913 (1801 to 2014) | |
CMA=cumulative meta-analysis.
Sensitivity analysis
Even allowing for a five year lag after the clinical practice guidelines were released (ie, after March or April 2012), 1478 redundant trials were identified, of which 207 reported 1913 (95% confidence interval 1801 to 2014) extra MACEs, including 275 (95% confidence interval 238 to 310) deaths, 584 (95% confidence interval 524 to 648) patients with new or recurrent myocardial infarction, 110 (88 to 132) patients with stroke, 16 (6 to 25) patients requiring revascularization, 215 (183 to 251) patients with heart failure, 683 (609 to 752) patients with recurrent or deteriorated angina pectoris, and 28 (9 to 45) unspecified MACEs (table 2).
Using 2002 and 2004 as the cut-off points established by cumulative meta-analyses, 2356 redundant trials were identified, of which 458 reported 4535 (4367 to 4702) extra MACEs, including 711 (95% confidence interval 652 to 772) deaths, 1292 (95% confidence interval 1198 to 1382) patients with new or recurrent myocardial infarction, 203 (170 to 236) patients with stroke, 130 (100 to 158) patients requiring revascularization, 495 (442 to 546) patients with heart failure, 1560 (1459 to 1665) patients with recurrent or deteriorated angina pectoris, and 144 (95% confidence interval 111 to 177) unspecified MACEs (table 2).
Discussion
Our study identified 2045 redundant clinical trials conducted in mainland China, with 3470 extra MACEs experienced among participants with coronary artery disease who were not treated with statins in the trials. The unexpected scale of redundancy raises concerns over the ethical foundation of clinical research in mainland China.
Such a large scale of redundant trials might arise when multiple system failures occur. First, investigators might not be trained to consider existing evidence before initiating clinical trials. We did not formally assess how the authors justified their trials, but, as others have reported,16 17 we noticed that few redundant trials cited systematic reviews or previous trials, suggesting a lack of appreciation about previous evidence.18 19 Second, clinical practitioners are under pressure to produce publications.20 21 This could also explain why only 20% of included trials reported clinical events, because it is easier and faster to conduct clinical trials on surrogate laboratory outcomes. Third, when ethics approval was reported, the committees reviewing the trial protocols failed to check the scientific foundation and protect participants from enrolling in harmful trials.3 22 Currently, ethics committee approval required by the China State Food and Drug Administration only covers clinical trials for the marketing license of drugs.23 Moreover, many redundant trials were conducted in primary care settings where approval by an ethics committee was not feasible. Therefore, some redundant trials might fail to obtain ethical approval. Fourth, some journal editors fail to evaluate the scientific value of the publications adequately. By accepting manuscripts from such trials, those journals provided a means for redundant trials to be published, thereby validating the redundancy as acceptable. Those journals might be more interested in pursuing profits rather than scientific merits.24 Fifth, only a small proportion of trials reported the funding source, most of which were either central or local government agencies, which failed to evaluate the scientific value of the redundant trials.25 Last, none of the included trials were registered in trial registries, a function of which is to reduce resource waste by declaring and presenting what has been conducted and what is currently being done in the clinical trial community.26
We also confirmed a large gap between Chinese and English literature.27 Most eligible trials were published in Chinese and only indexed in Chinese bibliographic databases. This might explain why systematic reviews that fail to search Chinese bibliographic databases often fail to include Chinese trials, even when a large amount of Chinese trials on the clinical question exist.28
Our findings might only cover a fraction of the problem. First, nearly 80% of eligible trials did not report clinical events, even with follow-up of many years. It is unclear whether no clinical events occurred in those trials, the events were not collected, or the events were simply not reported. Second, clinical trials with limited follow-up might not capture the long term benefits of statins, leading to underestimated treatment effects and extra MACEs.29 Third, the evidence is scarce on the publication rate of clinical trials in mainland China, and it is challenging to estimate how many trials might have been conducted but not published. Fourth, our study was limited to only one drug class, one disease condition, one type of comparator, and randomized clinical trials. Future studies are needed to evaluate the existence of research redundancy over the entire clinical trial community and the general biomedical research in mainland China.
Comparison with other studies
Previous studies have suggested that external evidence, including from systematic reviews, might be overlooked by researchers before new clinical trials on similar topics are initiated.30 31 32 However, even if systematic reviews confirm the benefits of a treatment in the early stages of clinical research, it is challenging to know whether future trials could modify, or even reverse, that early conclusion because early trials tend to exaggerate positive findings whereas later trials often report reduced effect sizes.33 Consequently, it might not be appropriate to label some clinical trials as redundant or as unethical. In our study, these concerns may be alleviated because the eligible trials consistently reaffirmed the benefits of statins among patients with coronary artery disease. This is unusual and differs from other studies using cumulative meta-analyses, in which a proportion of subsequent trials were in favor of the control group.5 31 The consistency of findings undermines the treatment uncertainties required to support the clinical equipoise to initiate subsequent clinical trials on similar topics.
In general, five arguments support our classification of 2045 trials as redundant. First, the release of clinical practice guidelines strongly recommending statins to all patients with coronary artery disease based on “abundant evidence” should disrupt the clinical equipoise required to justify starting new trials; second, the results of our cumulative meta-analyses suggested that the cut-off points of March and April 2008 were conservative; third, the consistency of the results from eligible trials undermined the treatment uncertainties; fourth, all eligible trials were conducted in mainland China so the redundancy could not be justified by the unsatisfactory representativeness of Asian populations recruited in clinical trials in Western countries34; and fifth, it is noticeable that the landmark clinical trials outside China, such as the 4S trial (Scandinavian Simvastatin Survival Study) published in 1994,35 were overlooked by redundant trials.
Limitations of this study
Our study has several limitations. First, we did not conduct a risk of bias assessment because our primary goal was to estimate the number of extra clinical events rather than the precise treatment effect at the population level. In case of information bias (eg, lack of masking), the extra clinical events might still be attributable to the treatment assignment—for example, patients in the control group did not receive as much attention as patients in the treatment group, therefore this would not invalidate our findings.36 On the other hand, selection bias, which might arise from inappropriate randomization,37 was not addressed. The extent to which this bias might impact our findings is unclear. Second, there are three reasons why we might have underestimated the extra clinical events: we only included journal articles in our study (more redundant trials could have been identified if we expanded our search to other publications, such as gray literature); we excluded trials that were not clearly specified as randomized clinical trials, but there are likely many non-randomized clinical trials that also assigned participants to statins or placebo; and it is likely that some redundant trials failed to report clinical events, but we believe it is inappropriate to extrapolate the number of events by assuming a similar incidence between the trials reporting and not reporting such events. Third, we did not evaluate the quality of the two clinical practice guidelines as the anchor to define redundancy. It is possible that some researchers do not consider those guidelines as trustworthy, even if they were developed by a leading national organization and published in a reputable Chinese journal.
We found redundant clinical trials initiated as recently as 2018. More redundant trials will be published in the near future unless actions are taken by stakeholders in mainland China, which might include, but are not limited to, altering the method to evaluate the academic performance of researchers, legislating the responsibilities and requirements of ethic committees, reaching consensus about publishing requirements for journals, adapting the funding system, and mandating trial registration. Our findings should enlighten researchers in other countries, especially developing countries that share certain characteristics with mainland China. However, the discussion should be on a nation specific, case-by-case basis because of the varied incentives behind research redundancy.
Conclusions
More than 2000 redundant clinical trials on statins among patients with coronary artery disease were identified from mainland China. More than 3000 extra MACEs, including nearly 600 deaths, were experienced by participants not treated with statins in these trials. The scale of redundancy necessitates urgent reform to protect patients.
What is already known on this topic
Redundant clinical trials waste resources and potentially harm patients who might be denied effective treatment, especially in the setting of placebo controlled trials
A concern is that redundancy has become a serious challenge in clinical trials from mainland China, the biggest producer of scientific publications
What this study adds
In mainland China, more than 2000 redundant clinical trials have been initiated since clinical practice guidelines recommended statins to all patients with coronary artery disease in 2008
More than 3000 extra major adverse cardiac events, including nearly 600 deaths, were experienced by participants not treated with statins in these trials, including 973 patients with myocardial infarction and 161 patients with stroke
The redundancy of clinical trials necessitates urgent reform to protect patients
Web extra.
Extra material supplied by authors
Supplementary information: files 1-5
Contributors: YJ and JW conceived the study. They contributed equally to this manuscript as joint first authors. YJ, JW, and KAR designed the study. YJ, JW, LR, XW, and YW acquired and analyzed the data, which was interpreted by YJ, JW, RQ, SE, DDC, XW, LR, YM, and KAR. YJ drafted the manuscript. JW, RQ, SE, DDC, XW, LR, YM, and KAR revised the article critically for important intellectual content. YJ, JW, RQ, SE, DDC, XW, LR, YM, and KAR read the final version of the manuscript and gave approval for it to be published. KAR had access to the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. YJ is the guarantor.
Funding: None.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: Technical appendix, statistical code, and dataset available from the corresponding author at krobin@jhmi.edu.
The lead author (YJ) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Dissemination to participants and related patient and public communities: We did not include members of the public mainly because this study was not funded. However, the idea of how redundant research harms patients was well received by our friends who did not work in the specialty of biomedical research. Upon publication, we will disseminate this research through social media such as WeChat to encourage further discussion in the public.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Borgerson K. Redundant, secretive, and isolated: when are clinical trials scientifically valid? Kennedy Inst Ethics J 2014;24:385-411. 10.1353/ken.2014.0029 [DOI] [PubMed] [Google Scholar]
- 2. Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987;317:141-5. [DOI] [PubMed] [Google Scholar]
- 3. Garattini S, Bertele V, Li Bassi L. How can research ethics committees protect patients better? BMJ 2003;326:1199-201. 10.1136/bmj.326.7400.1199 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.US Department of Health and Human Services. National commission for the protection of human subjects of biomedical and behavioral research. (1979). The Belmont report: Ethical principles and guidelines for the protection of human subjects of research 2013; 45.
- 5. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 1992;327:248-54. 10.1056/NEJM199207233270406 [DOI] [PubMed] [Google Scholar]
- 6. Clarke M, Brice A, Chalmers I. Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One 2014;9:e102670. 10.1371/journal.pone.0102670 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Institute of Medicine. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Graham R, Mancher M. Clinical practice guidelines we can trust: National Academies Press Washington, DC; 2011. [PubMed] [Google Scholar]
- 8. Tollefson J. China declared world’s largest producer of scientific articles. Nature 2018;553 10.1038/d41586-018-00927-4. [DOI] [PubMed] [Google Scholar]
- 9. Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta‐analyses. Milbank Q 2016;94:485-514. 10.1111/1468-0009.12210 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Vandenbroucke JP, von Elm E, Altman DG, et al. STROBE Initiative Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med 2007;4:e297. 10.1371/journal.pmed.0040297 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Ponikowski P, Voors AA, Anker SD, et al. 2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC) Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur Heart J 2016;37:2129-200. [DOI] [PubMed] [Google Scholar]
- 12. Jia Y, Huang D, Wen J, et al. Assessment of Language and Indexing Biases Among Chinese-Sponsored Randomized Clinical Trials. JAMA Netw Open 2020;3:e205894. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Chinese Society of Cardiology of Chinese Medical Association, Editorial Board of Chinese Journal of Cardiology [Guideline for diagnosis and treatment of patients with unstable angina and non-ST-segment elevation myocardial infarction]. Zhonghua Xin Xue Guan Bing Za Zhi 2007;35:295-304. [PubMed] [Google Scholar]
- 14. Chinese Society of Cardiology, Chinese Medical Association. Editorial Board, Chinese Journal of Cardiology [Guideline for diagnosis and treatment of patients with chronic stable angina. Zhonghua Xin Xue Guan Bing Za Zhi 2007;35:195-206. [PubMed] [Google Scholar]
- 15. Kip KE, Hollabaugh K, Marroquin OC, Williams DO. The problem with composite end points in cardiovascular studies: the story of major adverse cardiac events and percutaneous coronary intervention. J Am Coll Cardiol 2008;51:701-7. 10.1016/j.jacc.2007.10.034 [DOI] [PubMed] [Google Scholar]
- 16. Robinson KA, Goodman SN. A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med 2011;154:50-5. 10.7326/0003-4819-154-1-201101040-00007 [DOI] [PubMed] [Google Scholar]
- 17. Sawin VI, Robinson KA. Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 2016;69:174-8. 10.1016/j.jclinepi.2015.03.026 [DOI] [PubMed] [Google Scholar]
- 18. Robinson KA, Brunnhuber K, Ciliska D, Juhl CB, Christensen R, Lund H, Evidence-Based Research Network Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important? J Clin Epidemiol 2021;129:151-57. 10.1016/j.jclinepi.2020.07.020 [DOI] [PubMed] [Google Scholar]
- 19. Lund H, Juhl CB, Nørgaard B, et al. Evidence-Based Research Network Evidence-Based Research Series-Paper 2:Using an evidence-based research approach before a new study is conducted to ensure value. J Clin Epidemiol 2021;129:158-66. 10.1016/j.jclinepi.2020.07.019 [DOI] [PubMed] [Google Scholar]
- 20. Yuan H-F, Xu W-D, Hu H-Y. Young Chinese doctors and the pressure of publication. Lancet 2013;381:e4. 10.1016/S0140-6736(13)60174-9 [DOI] [PubMed] [Google Scholar]
- 21. Qiu J. Publish or perish in China: the pressure to rack up publications in high-impact journals could encourage misconduct, some say. Nature 2010;463:142-3. 10.1038/463142a [DOI] [PubMed] [Google Scholar]
- 22. Grady C. Institutional Review Boards: purpose and challenges. Chest 2015;148:1148-55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.China State Food and Drug Administration. Good Clinical Practice. 2003. http://www.chictr.org.cn/uploads/documents/20150602153036.pdf (accessed Oct 26 2020).
- 24. Clark J, Smith R. Firm action needed on predatory journals. British Medical Journal Publishing Group, 2015. 10.1136/bmj.h210. [DOI] [PubMed] [Google Scholar]
- 25. Shi Y, Rao Y. China’s research culture. American Association for the Advancement of Science, 2010. [Google Scholar]
- 26. Dickersin K, Rennie D. Registering clinical trials. JAMA 2003;290:516-23. 10.1001/jama.290.4.516 [DOI] [PubMed] [Google Scholar]
- 27. Cohen JF, Korevaar DA, Wang J, Spijker R, Bossuyt PM. Should we search Chinese biomedical databases when performing systematic reviews? Syst Rev 2015;4:23. 10.1186/s13643-015-0017-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Vale N, Nordmann AJ, Schwartz GG, et al. Statins for acute coronary syndrome. Cochrane Database Syst Rev 2014;(9):CD006870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Long-Term Intervention with Pravastatin in Ischaemic Disease (LIPID) Study Group Prevention of cardiovascular events and death with pravastatin in patients with coronary heart disease and a broad range of initial cholesterol levels. N Engl J Med 1998;339:1349-57. 10.1056/NEJM199811053391902 [DOI] [PubMed] [Google Scholar]
- 30. Habre C, Tramèr MR, Pöpping DM, Elia N. Ability of a meta-analysis to prevent redundant research: systematic review of studies on pain from propofol injection. BMJ 2014;348:g5219. 10.1136/bmj.g5219 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Fergusson D, Glass KC, Hutton B, Shapiro S. Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding? Clin Trials 2005;2:218-29, discussion 229-32. 10.1191/1740774505cn085oa [DOI] [PubMed] [Google Scholar]
- 32. Clarke M, Hopewell S, Chalmers I. Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. J R Soc Med 2007;100:187-90. 10.1177/014107680710011415 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Ioannidis JPA, Trikalinos TA. Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol 2005;58:543-9. 10.1016/j.jclinepi.2004.10.019 [DOI] [PubMed] [Google Scholar]
- 34. Minocher Homji RS, Lakhoo S, Ray JG. Recruitment of immigrant and ethnic minorities in primary prevention trials of cardiovascular disease. QJM 2011;104:469-76. 10.1093/qjmed/hcr027 [DOI] [PubMed] [Google Scholar]
- 35. Group SSSS. Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S). Lancet 1994;344:1383-9. [PubMed] [Google Scholar]
- 36. Schulz KF, Grimes DA. Blinding in randomised trials: hiding who got what. Lancet 2002;359:696-700. 10.1016/S0140-6736(02)07816-9 [DOI] [PubMed] [Google Scholar]
- 37. Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials 2009;10:46. 10.1186/1745-6215-10-46 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary information: files 1-5