Abstract
Objective To compare treatment effects from randomised trials conducted in more developed versus less developed countries.
Design Meta-epidemiological study.
Data sources Cochrane Database of Systematic Reviews (August 2012).
Data extraction Meta-analyses with mortality outcomes including data from at least one randomised trial conducted in a less developed country and one in a more developed country. Relative risk estimates of more versus less developed countries were compared by calculating the relative relative risks for each topic and the summary relative relative risks across all topics. Similar analyses were performed for the primary binary outcome of each topic.
Results 139 meta-analyses with mortality outcomes were eligible. No nominally significant differences between more developed and less developed countries were found for 128 (92%) meta-analyses. However, differences were beyond chance in 11 (8%) cases, always showing more favourable treatment effects in trials from less developed countries. The summary relative relative risk was 1.12 (95% confidence interval 1.06 to 1.18; P<0.001; I2=0%), suggesting significantly more favourable mortality effects in trials from less developed countries. Results were similar for meta-analyses with nominally significant treatment effects for mortality (1.15), meta-analyses with recent trials (1.14), and when excluding trials from less developed countries that subsequently became more developed (1.12). For the primary binary outcomes (127 meta-analyses), 20 topics had differences in treatment effects beyond chance (more favourable in less developed countries in 15/20 cases).
Conclusions Trials from less developed countries in a few cases show significantly more favourable treatment effects than trials in more developed countries and, on average, treatment effects are more favourable in less developed countries. These discrepancies may reflect biases in reporting or study design as well as genuine differences in baseline risk or treatment implementation and should be considers when generalising evidence across different settings.
Introduction
The predominant share of the global burden of disease is concentrated in less developed countries.1 However, until recently relative few trials were being conducted in these nations.2 3 Evidence on the management of many diseases affecting less developed countries often had to be tentatively extrapolated from studies performed in more developed countries with a longer standing tradition of conducting clinical research. This situation is currently changing. As participation rates in clinical trials decrease and cost increases in Western countries,4 5 6 many organisations providing contracts for research are focusing on eastern Europe,7 Asia,8 and South America,9 where the cost of recruiting participants is low. By 2015, for example, 15% of clinical trials are expected to be conducted in India.8 10 11 With the globalisation of international health,12 results of studies done in countries without a longstanding tradition of clinical research are becoming important to clinical practice in more developed nations.2
Trials carried out in less developed countries may differ in important aspects from those done in countries with stronger traditions in clinical research.13 Firstly, publication dynamics and biases may differ.14 Investigators in less developed countries may face a higher barrier against publication of “negative” results. Trials may remain unpublished or appear in domestic or local journals,15 16 and language biases may exist.17 18 19 Some national literatures on specific disciplines contain only significant results (for example, Chinese and Russian studies on acupuncture).20 Secondly, treatment effects may genuinely vary between countries owing to differences in study populations, baseline risk, concomitant diseases, background management, and clinical settings.
We performed a large scale assessment of meta-analyses on topics with randomised evidence from more developed and less developed countries. We assessed how often randomised trials performed in these countries with different levels of development and different traditions in clinical research give different results, whether treatment effects are systematically larger in one setting than the other, and whether discordant effects are the result of bias or genuine differences.
Methods
Definition of countries
There are varying definitions and no perfect consensus on what countries should be included in the lists of “more developed” and “less developed,” so their separation is not absolute.21 22 These categorisations try to take into account the per capita income but also other factors such as the composite human development index. For research purposes it is also helpful to know whether a country has a longstanding tradition in modern clinical research, uses critical scientific thinking, and in general applies empirical modern methods. We considered more developed countries to be those with both longstanding established market economies and longstanding traditions in clinical research.22 Such countries included the United States, Canada, Australia, New Zealand, Israel, and Japan, and western European countries. All other countries except for those in eastern Europe were considered as less developed.
Our definition is consistent with the list of less developed countries of the International Monetary Fund, except that Israel (considered less developed until 2001 according to the International Monetary Fund) is classified among the more developed countries, given its strong longstanding research tradition, and eastern European countries (less developed according to the International Monetary Fund, except for Slovenia after 2007, the Czech Republic and Slovakia after 2009, and Estonia after 2011) are excluded. Eastern European countries were excluded from our analyses as they may have unique differences7 and are considered to be a separate group of countries in transition.
We also performed sensitivity analyses where we excluded four Asian “tigers” (Hong Kong, Taiwan, Singapore, and South Korea) that evolved from less developed countries into advanced economies according to the International Monetary Fund in 1997, although their tradition of clinical research is not as longstanding as the main more developed countries. Sensitivity definitions excluding nations with a high per capita income but no tradition of clinical research (for example, several Arabian nations) yielded similar results, since few trials were identified that had been done in these countries.
Eligible meta-analyses
We identified meta-analyses that included data from one or more randomised trials conducted in a less developed country and one or more randomised trials conducted in a more developed country. Trials were classified on the basis of the countries in which participants were recruited; countries were considered with the names (for example, United Kingdom) or geographical indicators (for example, North America) as these were reported in the eligible Cochrane review. For consistency we focused on mortality, the most serious outcome.
We searched the Cochrane database of systematic reviews (last update 27 August 2012) using terms for mortality (death OR mortality OR survival) in the title, abstract, or keywords. The reviews included in this database are considered to be thorough in searching for eligible studies.23 We excluded protocols and reviews that had been withdrawn, had no statistical synthesis on mortality, and had no country information of individual trials. Only reviews including randomised and pseudorandomised trials were eligible. Whenever a systematic review contained two or more different pertinent intervention comparisons we considered these separately.
We also excluded reviews in which all the randomised trials from less developed or more developed countries had zero deaths. Multicentre international trials were eligible if all the countries were either less developed or more developed.
Data extraction
From each eligible trial we extracted the publication year, country of origin, number of participants, and number of deaths in each trial arm. We selected deaths from all causes; if, however, there were no data for all cause mortality we used cause specific mortality. Whenever there were several forest plots on mortality, we selected the one that reported overall data rather than subgroups. Whenever separate forest plots pertained to non-overlapping events for the same participants (for example, stillbirths and neonatal deaths), we selected whichever analysis included more deaths. For many conditions deaths are uncommon, conferring low power to show differences in effect sizes for mortality. Therefore, for each eligible topic we also examined separately the meta-analysis on the primary binary outcome (mortality or other). Whenever several eligible binary outcomes existed, we selected whichever had the largest number of studies regardless of whether this was mortality. Only meta-analyses with data from one or more trials from a less developed country and one or more trials from a more developed country were eligible. For each eligible trial we recorded the publication year, country, characteristics of participants, and events per arm.
Two investigators (OAP, DGCI) independently extracted data. Any disagreements were resolved after discussion with the third investigator (JPAI).
Statistical analysis
As a metric of relative risk, we used the odds ratio for our analyses when this could be estimated from available 2×2 tabular data for each trial. When this estimation was not possible we used the hazard ratio or risk ratio estimates as provided in the forest plots. Effect estimates for each trial were coined consistently to represent the odds of death or unfavourable primary outcome for the experimental (newer) intervention versus control. When survival or favourable primary outcomes were reported, we took the complementary mortality or unfavourable primary outcome event counts.
Whenever two or more trials per country group were included in a forest plot, we synthesised them by fixed effects and random effects models.24 Fixed effects assume a common effect across the combined studies, whereas random effects assume that the true treatment effect may differ in each trial, and the summary aims at identifying an average treatment effect. Subsequently, for each topic we calculated the relative relative risk, with corresponding 95% confidence intervals, by dividing the summary relative risk from trials in more developed countries by that in less developed countries on the same topic. A relative relative risk >1.00 means that the experimental intervention has more favourable outcomes in trials from less developed countries versus more developed countries. Furthermore, we calculated the summary relative relative risk across all topics of more developed versus less developed countries, by synthesising the relative relative risks of more developed versus less developed countries from each individual topic using a random effects model.25 Heterogeneity was probed using the Q statistic and I2 metric with corresponding 95% confidence intervals.26 27 In each eligible forest plot we also examined whether estimated intervention effects in smaller studies differed beyond chance from those estimated in larger studies (small study effects) using the Harbord’s test28 when 2×2 data were available and the Egger’s test29 otherwise; both test are considered to be significant for P<0.10.
For each topic where the results of trials from less developed countries differed beyond chance from trials from more developed countries, we examined whether there was evidence for small study effects. We also evaluated the constituent trials to examine whether there was any reason for anticipating ceiling effects related to the standard of care and the mode that an intervention was implemented in less developed or more developed countries—for example, whether an intervention was difficult to apply or required other concomitant interventions or background care to be effective. Moreover, we examined for each topic whether the baseline event risk in the control arms differed significantly between the two country groups by synthesising the baseline risks per country group by random effects using the Freeman-Tukey arcsin transformation.30 Such differences may mean that patients in these two settings vary in severity of disease, concomitant care, or other risk factors that can influence outcomes. Finally, for all topics where there were significant differences in treatment effects in more developed versus less developed countries, we also examined the risk of bias in the reported study design (mode of randomisation, allocation concealment, blinding, intention to treat, losses to follow-up). Quality deficits for these characteristics are associated with potential inflation in treatment effects in randomised controlled trials.31
In estimating the summary relative relative risk, sensitivity analyses were performed limited to topics where a nominally significant treatment effect for mortality had been found, when all trials were combined. Moreover, old trials may have different characteristics and less relevance to current practice.32 Therefore, we also performed sensitivity analyses excluding meta-analyses with any trials published before 1970. Finally, we performed sensitivity analyses excluding trials from countries that evolved from less developed to more developed countries.
All analyses were done in Stata version 11.2. P values are two sided.
Results
Eligible meta-analyses for mortality outcomes
The electronic search identified 2025 reviews. After exclusions (see supplementary fig A1), 131 eligible systematic reviews with 139 meta-analyses for mortality outcomes were considered (see supplementary table A1).
The 139 meta-analyses included 1297 eligible trials (312 were conducted in less developed countries and 985 in more developed countries, see supplementary fig A2). The median publication year of eligible trials was 1997 (interquartile range 1990-2002). Each meta-analysis included a median of 13 trials (interquartile range 8-19) and 2856 participants (interquartile range 1355-11593). Trials from more developed countries did not have substantially larger sample sizes (median 117; interquartile range 54 to 319) than trials from less developed countries (105; 54 to 365): P=0.93, Mann-Whitney U-test.
By fixed effects synthesis, 31 meta-analyses favoured (P<0.05) the experimental or new intervention, five the control, and 103 showed no statistically significant difference. By random effects synthesis, the respective numbers were 27, 1, and 111. Significant evidence was found for small study effects in a total of 16/139 (12%) meta-analyses: 15/133 (11%) meta-analyses with Harbord’s test and 1/6 meta-analyses with Egger’s test.
Significant differences in effect sizes for mortality
By using fixed effects to combine the relative risks from individual trials within the same country group, we identified 11 topics where the treatment effects between trials from more developed and less developed countries differed beyond chance (95% confidence intervals for relative relative risk excluded 1.00). For all these topics the experimental intervention had significantly less favourable results in the more than less developed countries (relative relative risk >1.00, table 1). Antenatal corticosteroids33 noticeably reduced fetal and neonatal deaths when given to women at risk of preterm birth in trials conducted in less developed countries, but had a modest, non-nominally significant benefit in trials conducted in more developed countries (fig 1). A similar pattern was observed for corticosteroids in the treatment of sepsis or septic shock (fig 2),34 systemic antifungals in non-neutropenic critically ill patients (fig 3),35 calcium antagonists in aneurysmal subarachnoid haemorrhage (fig 4),36 intravenous immunoglobulin for preventing infection in preterm or low birthweight infants (fig 5),37 and transarterial embolisation in unresectable hepatocellular carcinoma (fig 6).38 Moreover, antioxidants, given for diverse conditions (fig 7),39 or specifically for prevention of gastrointestinal cancers (fig 8),40 and postoperative radiotherapy for non-small cell lung cancer (fig 9)41 conferred a significantly increased risk of mortality in trials from more developed countries but not in trials from less developed countries (fig 1). Additionally, admission to hospital for bed rest for women with multiple pregnancy (fig 10)42 tended to increase mortality in trials from more developed countries and decrease mortality in trials from less developed countries. Finally, altered fractionation radiotherapy compared with conventional radiotherapy resulted in nominally significant decreases in total mortality from oral cavity and oropharyngeal cancer in both less and more developed countries, although this was larger in trials from less developed countries (fig 11).43
Table 1.
Statistically significant differences in treatment effects in less developed countries versus more developed countries
Topic | Experimental intervention* | Outcome | Summary relative risks (95% CI) | Relative relative risk (95% CI) for more v less developed countries | P values | ||
---|---|---|---|---|---|---|---|
Trials from more developed countries† | Trials from less developed countries† | Small study effects | Differences in baseline risk | ||||
Antenatal prevention in preterm birth33 | Corticosteroids | Fetal and neonatal deaths | 0.83 (0.68 to 1.02) | 0.40 (0.26 to 0.61) | 2.08 (1.30 to 3.33) | 0.103 | <0.001 |
Antioxidant supplements for prevention39 | Antioxidants | Mortality‡ | 1.06 (1.03 to 1.10) | 0.94 (0.85 to 1.05) | 1.13 (1.01 to 1.27) | 0.434 | <0.001 |
Multiple pregnancy42 | Admission to hospital for bed rest | Perinatal death | 3.15 (0.88 to 11.23) | 0.71 (0.35 to 1.46) | 4.42 (1.03 to 18.99) | 0.951 | 0.21 |
Treatment of sepsis and septic shock34 | Corticosteroids | All cause mortality at 28 days‡ | 0.89 (0.74 to 1.08) | 0.35 (0.14 to 0.87) | 2.58 (1.01 to 6.63) | 0.03 | <0.001 |
Prevention of gastrointestinal cancers40 | Antioxidants | Mortality | 1.07 (1.04 to 1.11) | 0.93 (0.84 to 1.04) | 1.15 (1.03 to 1.29) | 0.152 | <0.001 |
Non-neutropenic critically ill patients35 | Systemic antifungals | Mortality‡ | 0.76 (0.58 to 1.01) | 0.24 (0.08 to 0.68) | 3.18 (1.08 to 9.40) | 0.019 | <0.001 |
Treatment of non-small cell lung cancer41 | Postoperative radiotherapy | Mortality‡ | 1.37 (1.12 to 1.68) | 0.85 (0.57 to 1.28) | 1.61 (1.03 to 2.53) | 0.758 | 0.028 |
Aneurysmal subarachnoid haemorrhage36 | Calcium antagonists alone | Case fatality | 0.86 (0.64 to 1.13) | 0.15 (0.03 to 0.76) | 5.73 (1.13 to 28.3) | 0.302 | 0.99 |
Prevention of infection in preterm or low birthweight infants37 | Intravenous immunoglobulin | All cause mortality | 0.95 (0.77 to 1.17) | 0.49 (0.27 to 0.91) | 1.93 (1.01 to 3.66) | 0.137 | <0.001 |
Unresectable hepatocellular carcinoma38 | Transarterial chemoembolisation or transarterial embolisation | All cause mortality‡ | 0.88 (0.72 to 1.08) | 0.50 (0.31 to 0.81) | 1.76 (1.05 to 2.97) | 0.683 | NP |
Oral cavity and oropharyngeal cancer43 | Altered fractionation radiotherapy | Total mortality‡ | 0.91 (0.85 to 0.98) | 0.57 (0.37 to 0.88) | 1.60 (1.03 to 2.48) | 0.790 | NP |
Cirrhosis with upper gastrointestinal bleeding44 | Antibiotics | Bacterial infections‡ | 0.30 (0.21 to 0.42) | 0.13 (0.07 to 0.25) | 2.30 (1.10 to 4.82) | 0.088 | <0.001 |
Peripheral arterial disease of leg45 | Lipid lowering regimens | Cardiovascular events‡ | 0.77 (0.70 to 0.85) | 0.06 (0.01 to 0.46) | 13.28 (1.68 to 104.97) | 0.411 | 0.081 |
Pneumococcal infection in adults46 | Vaccination | All cause pneumonia‡ | 0.85 (0.77 to 0.95) | 0.52 (0.43 to 0.63) | 1.64 (1.32 to 2.03) | 0.864 | <0.001 |
Prevention of pre-eclampsia47 | Antioxidants | Pre-eclampsia‡ | 0.96 (0.80 to 1.16) | 0.38 (0.20 to 0.73) | 2.54 (1.29 to 5.01) | 0.071 | 0.60 |
Vitamin E supplementation in pregnancy48 | Vitamin E | Clinical pre-eclampsia‡ | 0.55 (0.30 to 1.01) | 0.06 (0.01 to 0.45) | 9.52 (1.11 to 81.74) | 0.858 | 0.48 |
Prevention of gastrointestinal cancers40 | Antioxidants | Incidence of gastrointestinal cancers‡ | 1.05 (0.97 to 1.14) | 0.86 (0.74 to 0.99) | 1.23 (1.04 to 1.45) | 0.004 | <0.001 |
Hypertension during pregnancy49 | β blockers | Caesarean section‡ | 1.03 (0.80 to 1.32) | 0.36 (0.17 to 0.77) | 2.83 (1.28 to 6.29) | 0.302 | 0.37 |
Hepatic encephalopathy52 | Probiotics | No recovery from hepatic encephalopathy‡ | 0.03 (0.01 to 0.53) | 0.61 (0.32 to 1.18) | 0.04 (0.01 to 0.93) | 0.429 | 0.09 |
Prevention of pre-eclampsia51 | Antiplatelet agents | Proteinuric pre-eclampsia‡ | 0.74 (0.65 to 0.86) | 0.93 (0.79 to 1.09) | 0.80 (0.65 to 0.99) | <0.0001 | 0.13 |
Tuberculosis prevention in non-HIV infected people53 | Isoniazid | Active tuberculosis‡ | 0.34 (0.28 to 0.42) | 0.59 (0.40 to 0.86) | 0.58 (0.38 to 0.90) | 0.811 | <0.001 |
Prophylactic antifungal agents in very low birthweight infants54 | Fluconazole | Invasive fungal infection‡ | 0.23 (0.11 to 0.49) | 1.09 (0.48 to 2.48) | 0.21 (0.07 to 0.64) | 0.833 | 0.07 |
Prevention of infection in preterm or low birthweight infants37 | Intravenous immunoglobulin | Sepsis‡ | 0.89 (0.74 to 1.06) | 0.31 (0.16 to 0.62) | 2.84 (1.40 to 5.79) | 0.059 | 0.99 |
Antenatal care for low risk pregnancy50 | Reduced number of visits or goal oriented visits | Preterm birth‡ | 1.24 (1.01 to 1.52) | 0.99 (0.91 to 1.08) | 1.26 (1.01 to 1.57) | 0.787 | NP |
Diarrhoea prevention55 | Rotavirus vaccine | Episodes of rotavirus diarrhoea‡ | 0.48 (0.42 to 0.54) | 0.61 (0.52 to 0.72) | 0.78 (0.63 to 0.96) | 0.611 | 0.006 |
NP=not pertinent because number of events in each arm was not available.
*In all cases, the experimental intervention has been compared with placebo or no treatment, except for surgery plus postoperative radiotherapy, which was compared against surgery alone; altered fractionation which was compared with conventional radiotherapy; reduced number of antenatal care visits or goal oriented visits, which were compared with standard care visits.
†Individual trial relative risks from trials within the same country group were combined with a fixed effect model.
‡The primary binary outcomes of the respective systematic reviews.
Fig 1 Fetal and neonatal deaths with antenatal corticosteroids in women at risk of preterm birth
Fig 2 All cause mortality at 28 days with corticosteroids for treating sepsis and septic shock
Fig 3 Mortality with systemic antifungals in non-neutropenic critically ill patients
Fig 4 Case fatality with calcium antagonists in aneurysmal subarachnoid haemorrhage
Fig 5 All cause mortality with intravenous immunoglobulin for preventing infection in preterm or low birthweight infants
Fig 6 All cause mortality with transarterial embolisation in unresectable hepatocellular carcinoma. Only the effect estimates from each trial were retrievable for the corresponding meta-analysis
Fig 7 Mortality with antioxidants for preventing various diseases
Fig 8 Mortality with antioxidants for preventing gastrointestinal cancers
Fig 9 Mortality after postoperative radiotherapy for non-small cell lung cancer
Fig 10 Perinatal deaths with admission to hospital for bed rest for women with multiple pregnancy
Fig 11 Total mortality with altered fractionation radiotherapy for oral cavity and oropharyngeal cancer. Only the effect estimates from each trial were retrievable for the corresponding meta-analysis
Evidence for small study effects was strong in the meta-analyses of steroids and antifungals and possibly also antenatal corticosteroids. The interventions were simple and easy to administer in diverse settings, regardless of the availability of other concomitant interventions and standards of care. The one possible exception was postoperative radiotherapy, where better outcomes might be expected in countries with higher standards of technology, although, if anything, the opposite was seen.
The baseline risk of death was significantly higher in less developed countries in the meta-analyses of antenatal corticosteroids (33% v 16%), corticosteroids for sepsis (76% v 34%), systemic antifungals for non-neutropenic critically ill patients (54% v 28%), and intravenous immunoglobulin in preterm infants (19% v 13%), whereas it was significantly higher in more developed countries in meta-analyses of preventive antioxidants for various conditions (8% v 4%) or for gastrointestinal cancers (12% v 5%), and postoperative radiotherapy for non-small cell lung cancer (39% v 31%) (see supplementary table A2).
Table 2 shows the number of trials from the countries that had unclear or high risk of bias for randomisation sequence generation, allocation concealment, and blinding for the topics where significant differences in the treatment effects for mortality were documented. It is difficult to make comparisons within single topics, given the limited number of trials. However, summing the data across all trials, the proportion of trials with an unclear or high risk of bias was not significantly different in trials from more developed versus less developed countries for sequence generation (27% v 33%, P=0.45), allocation concealment (27% v 30%, P=0.68), or blinding (39% v 33%, P=0.53).
Table 2.
Risk of bias in studies from more developed versus less developed countries based on reported features
Topic | Experimental intervention | Outcome | Randomisation sequence generation | Allocation concealment | Blinding | |||
---|---|---|---|---|---|---|---|---|
Trials from more developed countries* | Trials from less developed countries* | Trials from more developed countries* | Trials from less developed countries* | Trials from more developed countries* | Trials from less developed countries* | |||
Antenatal prevention in preterm birth33 | Corticosteroids | Fetal and neonatal deaths | 7/10 | 0/3 | 7/10 | 1/3 | 5/10 | 1/3 |
Antioxidant supplements for prevention39 | Antioxidants | Mortality† | 14/58 | 2/7 | 12/58 | 1/7 | 6/58 | 0/7 |
Multiple pregnancy42 | Admission to hospital for bed rest | Perinatal death | 1/3 | 0/4 | 1/3 | 1/4 | 3/3 | 0/4 |
Treatment of sepsis and septic shock34 | Corticosteroids | All cause mortality at 28 days† | 4/16 | 0/3 | 4/16 | 0/3 | 5/16 | 0/3 |
Prevention of gastrointestinal cancers40 | Antioxidants | Mortality | 0/8 | 0/5 | 0/8 | 0/5 | 0/8 | 0/5 |
Non-neutropenic critically ill patients35 | Systemic antifungals | Mortality† | 4/9 | 1/1 | 3/9 | 0/1 | 1/9 | 0/1 |
Treatment of non-small cell lung cancer41 | Postoperative radiotherapy | Mortality† | 0/8 | 0/2 | 5/8 | 1/2 | 8/8 | 2/2 |
Aneurysmal subarachnoid haemorrhage36 | Calcium antagonists alone | Case fatality | 0/10 | 1/1 | 3/10 | 1/1 | 5/10 | 1/1 |
Prevention of infection in preterm or low birthweight infants37 | Intravenous immunoglobulin | All cause mortality | 7/10 | 5/5 | 1/10 | 4/5 | 6/10 | 5/5 |
Unresectable hepatocellular carcinoma38 | Transarterial chemoembolisation or transarterial embolisation | All cause mortality† | 1/8 | 1/1 | 2/8 | 0/1 | 8/8 | 1/1 |
Oral cavity and oropharyngeal cancer43 | Altered fractionation radiotherapy | Total mortality† | 3/13 | 1/1 | 3/13 | 1/1 | 13/13 | 1/1 |
Cirrhosis with upper gastrointestinal bleeding44 | Antibiotics | Bacterial infections† | 1/7 | 3/5 | 3/7 | 4/5 | 7/7 | 5/5 |
Peripheral arterial disease of leg45 | Lipid lowering regimens | Cardiovascular events† | 5/6 | 0/2 | 4/6 | 0/2 | 0/6 | 0/2 |
Pneumococcal infection in adults46 | Vaccination | All cause pneumonia† | 4/9 | 2/4 | 7/9 | 3/4 | 5/9 | 3/4 |
Prevention of pre-eclampsia47 | Antioxidants | Pre-eclampsia† | 1/4 | 2/4 | 1/4 | 2/4 | 0/4 | 1/4 |
Vitamin E supplementation in pregnancy48 | Vitamin E | Clinical pre-eclampsia† | 1/2 | 1/1 | 1/2 | 1/1 | 0/2 | 0/1 |
Prevention of gastrointestinal cancers40 | Antioxidants | Incidence of gastrointestinal cancers† | 0/6 | 5/11 | 0/6 | 5/11 | 0/6 | 0/11 |
Hypertension during pregnancy49 | β blockers | Caesarean section† | 7/11 | 1/1 | 7/11 | 0/1 | 7/11 | 0/1 |
Hepatic encephalopathy52 | Probiotics | No recovery from hepatic encephalopathy† | 0/1 | 1/3 | 1/1 | 3/3 | 1/1 | 3/3 |
Prevention of pre-eclampsia51 | Antiplatelet agents | Proteinuric pre-eclampsia† | 13/30 | 5/10 | 20/30 | 7/10 | 26/30 | 8/10 |
Tuberculosis prevention in non-HIV infected people53 | Isoniazid | Active tuberculosis† | 0/6 | 0/4 | 1/6 | 1/4 | 0/6 | 0/4 |
Prophylactic antifungal agents in very low birthweight infants54 | Fluconazole | Invasive fungal infection† | 1/4 | 0/1 | 1/4 | 0/1 | 0/4 | 0/1 |
Prevention of infection in preterm or low birthweight infants37 | Intravenous immunoglobulin | Sepsis† | 5/7 | 3/3 | 0/7 | 2/3 | 3/7 | 3/3 |
Antenatal care for low risk pregnancy50 | Reduced number of visits or goal oriented visits | Preterm birth† | 2/4 | 2/3 | 2/4 | 1/3 | 4/4 | 1/3 |
Diarrhoea prevention55 | Rotavirus vaccine | Episodes of rotavirus diarrhoea† | 7/14 | 1/6 | 10/14 | 1/6 | 4/14 | 0/6 |
*Numbers represent trials with unclear or high risk of bias/total number of trials.
†The primary binary outcomes of the respective systematic reviews.
Summary of comparisons for mortality
When summary relative risks from trials within each country group were synthesised by fixed effects, on average the results from more developed countries were significantly less favourable than those from less developed countries, with a summary relative relative risk of 1.12 (95% confidence interval 1.06 to 1.18, P<0.001, I2=0%, 95% confidence interval 0% to 21%, Q statistic P=0.709). When data were synthesised within each country group by random effects, inferences were similar (1.08, 1.02 to 1.14, P=0.005, I2=0%, 0% to 21%, Q statistic P=0.922), but confidence intervals were wider and only the differences for antenatal corticosteroids, systematic antifungals, calcium antagonists, transarterial embolisation, and altered fractionation radiotherapy were beyond chance. Additionally, summary relative relative risks per fixed effects were 1.10 (1.04 to 1.18, P=0.002, I2=10.5%, 10% to 42%, Q statistic P=0.303) per fixed effects and 1.11 (1.01 to 1.21, P=0.023) per random effects for more developed countries versus China, and 1.21 (1.13 to 1.30, P=0.003, I2=92.3%, 90% to 94%, Q statistic P<0.001) per fixed effects and 1.26 (0.91 to 1.74, P=0.158) per random effects for more developed countries versus India (the two less developed countries with the largest number of trials).
Results were similar when analyses were limited to the 36 meta-analyses that had found nominally significant mortality effects overall at 1.15 (1.08 to 1.23, P<0.001) per fixed effects and 1.17 (1.06 to 1.30, P=0.002) per random effects, I2=17%, 0% to 45%, Q statistic P=0.19, fig 12), the 124 meta-analyses where all trials had been published after 1970 (1.14, 1.08 to 1.21, P<0.001, I2=0%, 0% to 22%, Q statistic P=0.81), and when excluding from calculations the four Asian countries (Hong Kong, Taiwan, Singapore, and South Korea) evolving into more developed countries (1.12, 1.06 to 1.18, P<0.001, I2=0%, 0% to 22%, Q statistic P=0.624).
Fig 12 Relative relative risks and 95% confidence intervals for mortality outcomes between more developed and less developed countries in meta-analyses with nominally significant effects. Relative relative risk estimates and 95% confidence intervals are shown for the 36 topics for which the respective meta-analyses had found nominally significant effects overall per fixed effects synthesis
Eligible meta-analyses for primary binary outcomes
Overall, 127 meta-analyses had primary binary outcomes and available data from at least one more developed country and at least one less developed country (see supplementary table A3); for 58 of those the primary binary outcome was mortality. These 127 meta-analyses included a total of 1312 trials; 319 conducted in less developed countries (median sample size 121, interquartile range 58-318) and 993 in more developed countries (114, 53-310). The median number of trials per meta-analysis overall was 14 (interquartile range 9-21).
By fixed effect synthesis 55 meta-analyses were overall in favour (P<0.05) of the experimental intervention when all trials in the respective forest plot were considered, seven favoured the control, and 65 showed non-significant differences. By random effects, the respective numbers were 46, 5, and 76. In 26/127 (20%) meta-analyses evidence for small study effects was significant: 24/122 with Harbord’s test and 2/5 with Egger’s test.
Significant differences in effect sizes for any primary outcome
Combining by fixed effects model, treatment effects for the experimental intervention in trials from more developed and less developed countries, respectively, the relative differences were beyond chance (relative relative risk and 95% confidence intervals excluding 1.00) in 20 cases, of which six pertained to mortality and 14 to non-mortality outcomes (table 1; also see supplementary fig A3). In 15 of the 20 cases, results were more favourable (or less unfavourable) in trials from less developed countries (relative relative risk >1.00).
Antibiotic prophylaxis for bacterial infections in cirrhotic patients with upper gastrointestinal bleeding,44 lipid lowering regimens for peripheral arterial disease of the leg,45 vaccination for pneumococcal infection in adults,46 and intravenous immunoglobulin for sepsis in preterm or low birthweight infants37 had beneficial effects on the respective outcomes in trials from both less developed and more developed countries, although the benefit was considerably larger in less developed countries. Antioxidants47 and vitamin E48 prevented pre-eclampsia, antioxidants prevented gastrointestinal cancers,40 and β blockers decreased the rate of caesarean sections49 in trials from less developed countries but not in trials from more developed countries. Finally, the reduced number of antenatal visits or goal oriented visits only marginally increased the risk of preterm birth in more developed countries, but showed a small non-significant reduction in trials from less developed countries.50
Evidence of small study effects was strong for the meta-analyses of antioxidants for pre-eclampsia, antioxidants for gastrointestinal cancers, and intravenous immunoglobulin for sepsis. All interventions were simple and easy to apply in diverse settings and background standards of care. The baseline risk of the outcome was significantly higher in less developed countries in the meta-analyses of corticosteroids for treating sepsis (76% v 34%) and systemic antifungals for non-neutropenic critically ill patients (54% v 28%), and it was significantly higher in more developed countries in the meta-analyses of preventive antioxidants in various conditions (8% v 4%) (see supplementary table A2).
For the remaining five non-mortality related primary outcomes, the treatment effects were more beneficial in trials from more developed countries. These included antiplatelet agents to prevent proteinuric pre-eclampsia,51 probiotics in hepatic encephalopathy,52 isoniazid prophylaxis against active tuberculosis in people not infected with HIV,53 prophylactic fluconazole for invasive fungal infections in very low birthweight infants,54 and rotavirus vaccine for the prevention of diarrhoea.55
For the meta-analysis of antiplatelets, evidence for the presence of small study effects was strong and the larger trials showed no clear benefit in either country group. All four interventions were simple and easy to administer in diverse settings. The risks in the control groups at baseline were significantly lower in more developed countries for active tuberculosis (2% v 15%), gastrointestinal cancer (2% v 3%), and rotavirus diarrhoea, whereas the risk was higher in more developed countries for bacterial infections in cirrhotic patients with upper gastrointestinal bleeding (43% v 28%) (see supplementary table A2). The proportion of trials with an unclear or high risk of bias was not significantly different in trials from more developed countries for sequence generation (42% v 45%, P=0.76), allocation concealment (52% v 52%, P=0.95), or blinding (51% v 41%, P=0.22, table 2).
When the relative relative risks were synthesised across all 127 topics, there was some between topic heterogeneity with fixed effects summary relative odds ratio 1.07 (95% confidence interval 1.02 to 1.12, P=0.009) and random effects summary relative relative risk 1.09 (1.01 to 1.18, P=0.034, I2=37%, 95% confidence interval 22% to 49%, Q statistic P<0.001).
Discussion
We evaluated 139 meta-analyses with mortality outcomes, which included trials performed in less developed and more developed countries. In 11 cases, experimental interventions had significantly more favourable results in less developed countries than in more developed ones, whereas the opposite was never seen. On average, trials conducted in less developed countries had 1.12-fold more favourable effect sizes than trials done in more developed countries. When focusing only on interventions with an overall statistically significant impact on mortality the difference was 1.15-fold. Given that even effective interventions rarely achieve more than 1.15-fold to 1.20-fold reductions in the relative risk of mortality,56 relative differences of 1.10-fold to 1.15-fold may confound the presence or not of a genuine effect of many interventions. When we considered any primary binary outcome, in 20 topics treatment effects varied significantly based on the country group of included trials, and in 15/20 results were more favourable in less developed countries. Totally ineffective treatments may spuriously seem effective based on research published from less developed countries. As an increasingly larger share of clinical research is being done in less developed countries without strong research traditions, this may create a flood of spurious evidence.
Possible explanations
Given the systematic preponderance of more favourable results in trials from less developed countries, one potential explanation is that the available randomised evidence from developed countries is more biased. An empirical evaluation of 307 published randomised trials from China, 117 from India, and 304 from Western countries showed that Indian and Chinese trials were of much lower methodological quality.57 Another empirical study highlighted that authors of Chinese trials often mislabelled basic study designs. Among 3137 studies indexed in the China national knowledge infrastructure database and claimed by their authors to be randomised, only 207 were indeed randomised.58 Most Chinese trials do not adhere to the CONSORT guidelines for reporting59 and many trials from less developed countries are not registered in ClinicalTrials.gov or even the composite World Health Organization trials registry.60 Trials from less developed countries tend to report on average more significant results.20 57 For example, such publication bias has been previously seen for Chinese (but not Indian) trials.20 57 61
Publication bias or selective analysis and outcome reporting biases62 63 may be influential in shaping this picture. A higher barrier to publication for authors from less developed countries that do not have a longstanding tradition in clinical research may further boost selective reporting.17 Of course mortality is a hard endpoint and more difficult to manipulate than other endpoints, but even for mortality, selective analysis may achieve inflated effects, as recently shown by corticosteroid trials.64 The presence of patterns showing small study effects is also suggestive (not conclusive) of selective reporting biases.65 Small study effects may also influence the literature in nations with strong traditions of running clinical trials. For example, this may be the case for antiplatelet agents to prevent proteinuric pre-eclampsia,51 where small trials suggest substantial benefits (more so in more developed than less developed countries), but the largest trials66 67 in both more and less developed countries have shown no benefits. Large, well conducted trials are needed to probe the claims for country specific major benefits and they may demonstrate that many of these claims are spurious. For example, after the publication of the examined Cochrane reviews, a recent large trial68 conclusively found no benefit from antioxidants in the prevention of pre-eclampsia (odds ratio 1.00), as opposed to the extremely large benefit that previous small trials had suggested (0.38).
Additionally, in the topics where results between more and less developed countries differed, there was no overall pattern of having a higher or lower proportion of trials with unclear or high risk of bias in sequence generation, allocation concealment, or blinding, when research was compared between more developed and less developed countries. There is some evidence that these quality deficits are associated with inflated treatment effects, although the impact is lesser when the outcome is mortality.31 Although we cannot exclude the possibility that these quality aspects may have played a role in explaining the difference in some specific topics, they do not seem to be the main answer for the discrepancies overall. It should also be acknowledged that reported quality may not necessarily reflect the true quality of trials.69
Differences in treatment effects in less developed versus more developed countries may also be due to genuine differences rather than to biases. Low income and middle income countries face substantial financial barriers to the total healthcare budget,70 which may limit the implementation of expensive interventions.71 This might hold especially true for trials designed by the same body (institute, industry, etc) and conducted in more and less developing countries, as study quality and biases are usually not expected to be different, except maybe for selective reporting of negative results. However, we did not identify any discrepancies where the implicated intervention was expensive or difficult to administer and its efficacy may have depended largely on sophisticated background standards of care. The one exception was postoperative radiotherapy, but then the observed benefit was larger in less developed countries, a paradox that suggests that bias is a more likely explanation than differences in standards of care and technological ability. Trials with different results sometimes studied populations with different baseline risks. For example, mortality in newborns is on average higher in less developed countries72 and we cannot exclude the possibility that corticosteroids may result in a larger benefit in these locations. Similarly, the larger benefit of isoniazid prophylaxis in more developed countries may be explained by an increased burden of multidrug resistant tuberculosis,73 lower rates of treatment compliance,74 and limited access to healthcare74 in less developed countries.
Potential limitations of the study
Some caveats should be acknowledged. Firstly, we worked with available meta-analyses that may have already removed some biases from the primary literature. Meta-analysts may have contacted the authors of primary trials and obtained outcome information not reported in published reports, or they may have standardised outcomes, for example, to include all cause mortality and all available follow-up, whereas primary papers may have focused on subset analyses or other secondary analyses such as cause specific deaths.64 Thus bias may be larger in the reports of primary trials than that seen in meta-analysis based data. Secondly, agreement in treatment effects between more developed and less developed countries does not necessarily mean that both estimates are correct; occasionally both may be equally biased. Some recorded treatment effects in meta-analyses may simply reflect bias.75 For example, some investigators have argued convincingly that pneumococcal vaccination is ineffective in adults and that the apparent benefits in preventing pneumonia in the respective meta-analysis (seemingly larger in less developed countries) are entirely spurious.76 Thirdly, the endpoints of primary trials may not be the same as the respective primary endpoints in meta-analyses, and mortality might not be the primary endpoint for several considered trials. Even so, mortality is a major outcome and trials with favourable mortality results should attract attention regardless of whether this was a primary or secondary endpoint. In fact, an unexpectedly large number of small trials claim significant differences in mortality.77 Fourthly, not all organisations agree on what countries are less developed, and the status of countries has changed over time, with several previously less developed countries adopting market economies. However, these countries still can be separated from countries with longstanding traditions in clinical research, and sensitivity analyses using different definitions yielded similar results. We cannot separate whether per capita income or tradition in clinical trials research is the decisive factor that makes the difference, since few trials were done in countries without strong longstanding traditions in clinical research, where per capita income has increased dramatically in the past decades. Finally, some trials performed in less developed countries may be designed and coordinated by investigators in more developed countries. If anything this would tend to diminish differences between the two groups. Nevertheless, none of the trials implicated in the seven topics with statistically significant differences in mortality had such collaborative patterns. Moreover, it is possible that industry sponsorship may also affect the results of trials, in particular for expensive interventions where large markets are at stake. However, most of the interventions where discrepancies were identified were not expensive and sponsors would not have major invested interests.
Conclusions and implications for future research
Overall, in a globalised world, evidence from less developed countries will increasingly influence decisions in more developed countries and vice versa. It is important to generate randomised evidence in diverse settings including populations with differences in baseline risk, comorbidities, and access to healthcare. It is also important to improve the quality and minimise the biases of randomised trials around the world. Biases could be reduced through more thorough registration of trials from less developed countries, strengthening ethical standards,78 and a global view in the design and interpretation of the overall clinical research agenda.79 Meta-analyses of the available evidence can routinely explore differences and potential explanations thereof for trials performed in countries with different economies and traditions of clinical research. This information should be taken into consideration in guidelines and in the adoption of these interventions.
What is already known on this topic
An increasing number of trials are performed in less developed countries with no longstanding tradition in clinical research
It is unclear whether the results of trials in more developed versus less developed countries are similar for the same intervention and conditions
What this study adds
Randomised trials from less developed countries in a few cases show significantly different treatment effects from randomised trials in more developed countries
The reported treatment effects are larger in the less developed countries
These discrepancies may often reflect biases as well as genuine differences and should be taken into account when generalising evidence across different settings
Contributors: JPAI conceived the original idea. OAP, DGC-I, and JPAI designed the study. OAP and DGC-I identified the eligible reviews and performed the data extraction. OAP and JPAI performed the statistical analyses. OAP, DGC-I, and JPAI interpreted the data and wrote the manuscript. All authors have critically commented on and approved the final version of the manuscript. JPAI is the guarantor. All authors, external and internal, had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding: This study received no funding.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: The statistical code and datasets are available from the corresponding author at jioannid@stanford.edu.
Cite this as: BMJ 2013;346:f707
Web Extra. Extra material supplied by the author
Supplementary figures, legends, and table
Countries in which eligible trials had been done
References
- 1.Murray CJL, Lopez AD. The global burden of disease: a comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Harvard School of Public Health, Harvard University Press on behalf of the World Health Organization and The World Bank, 1996.
- 2.Swingler GH, Pillay V, Pienaar ED, Ioannidis JP. International collaboration, funding and association with burden of disease in randomized controlled trials in Africa. Bull World Health Organ 2005;83:511-7. [PMC free article] [PubMed] [Google Scholar]
- 3.Isaakidis P, Swingler GH, Pienaar E, Volmink J, Ioannidis JP. Relation between burden of disease and randomised evidence in sub-Saharan Africa: survey of research. BMJ 2002;324:702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Murthy VH, Krumholz HM, Gross CP. Participation in cancer clinical trials: race-, sex-, and age-based disparities. JAMA 2004;291:2720-6. [DOI] [PubMed] [Google Scholar]
- 5.Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007;17:643-53. [DOI] [PubMed] [Google Scholar]
- 6.Bell J. Resuscitating clinical research in the United Kingdom. BMJ 2003;327:1041-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Platonov P. Clinical trials in Russia and Eastern Europe: recruitment and quality. Int J Clin Pharmacol Ther 2003;41:277-80. [DOI] [PubMed] [Google Scholar]
- 8.Varawalla N. India’s growing clinical research sector: opportunity for global companies. IDrugs 2007;10:391-4. [PubMed] [Google Scholar]
- 9.Malakoff D. Clinical trials and tribulations. Spiraling costs threaten gridlock. Science 2008;322:210-3. [DOI] [PubMed] [Google Scholar]
- 10.Drabu S, Gupta A, Bhadauria A. Emerging trends in contract research industry in India. Contemp Clin Trials 2010;31:419-22. [DOI] [PubMed] [Google Scholar]
- 11.Bajpai V, Saraya A. Boom in clinical research industry: a dangerous trend. Trop Gastroenterol 2009;30:177-81. [PubMed] [Google Scholar]
- 12.Walt G. Globalisation of international health. Lancet 1998;351:434-7. [DOI] [PubMed] [Google Scholar]
- 13.Lorenzo C, Garrafa V, Solbakk JH, Vidal S. Hidden risks associated with clinical trials in developing countries. J Med Ethics 2010;36:111-5. [DOI] [PubMed] [Google Scholar]
- 14.Smith R. Publishing research from developing countries. Stat Med 2002;21:2869-77. [DOI] [PubMed] [Google Scholar]
- 15.Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994;309:1286-91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Academic medicine: the evidence base. BMJ 2004;329:789-92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pan Z, Trikalinos TA, Kavvoura FK, Lau J, Ioannidis JP. Local literature bias in genetic epidemiology: an empirical evaluation of the Chinese literature. PLoS Med 2005;2:e334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet 1997;350:326-9. [DOI] [PubMed] [Google Scholar]
- 19.Juni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol 2002;31:115-23. [DOI] [PubMed] [Google Scholar]
- 20.Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 1998;19:159-66. [DOI] [PubMed] [Google Scholar]
- 21.Wikipedia. Developing country. 2012. http://en.wikipedia.org/wiki/Developing_country.
- 22.United Nations Statistics Division. Composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings. 2012. http://unstats.un.org/unsd/methods/m49/m49regin.htm.
- 23.Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med 2007;4:e78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-88. [DOI] [PubMed] [Google Scholar]
- 25.Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med 2002;21:1513-24. [DOI] [PubMed] [Google Scholar]
- 26.Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21:1539-58. [DOI] [PubMed] [Google Scholar]
- 27.Ioannidis JP, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ 2007;335:914-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med 2006;25:3443-57. [DOI] [PubMed] [Google Scholar]
- 29.Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629-34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Freeman MF, Tukey JW. Transformations related to the angular and the square root. Ann Math Statist 1950;21:607-11. [Google Scholar]
- 31.Savovic J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012; published online 4 September, 2012. doi:10.7326/0003-4819-157-6-201209180-00537. [DOI] [PubMed] [Google Scholar]
- 32.Patsopoulos NA, Ioannidis JP. The use of older studies in meta-analyses of medical interventions: a survey. Open Med 2009;3:e62-8. [PMC free article] [PubMed] [Google Scholar]
- 33.Roberts D, Dalziel S. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database Syst Rev 2006;(3):CD004454. [DOI] [PubMed] [Google Scholar]
- 34.Annane D, Bellissant E, Bollaert PE, Briegel J, Keh D, Kupfer Y. Corticosteroids for treating severe sepsis and septic shock. Cochrane Database Syst Rev 2004;(1):CD002243. [DOI] [PubMed] [Google Scholar]
- 35.Playford EG, Webster AC, Sorrell TC, Craig JC. Antifungal agents for preventing fungal infections in non-neutropenic critically ill patients. Cochrane Database Syst Rev 2006;(1):CD004920. [DOI] [PubMed] [Google Scholar]
- 36.Dorhout Mees SM, Rinkel GJ, Feigin VL, Algra A, van den Bergh WM, Vermeulen M, et al. Calcium antagonists for aneurysmal subarachnoid haemorrhage. Cochrane Database Syst Rev 2007;(3):CD000277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Ohlsson A, Lacy JB. Intravenous immunoglobulin for preventing infection in preterm and/or low-birth-weight infants. Cochrane Database Syst Rev 2004;(1):CD000361. [DOI] [PubMed] [Google Scholar]
- 38.Oliveri RS, Wetterslev J, Gluud C. Transarterial (chemo)embolisation for unresectable hepatocellular carcinoma. Cochrane Database Syst Rev 2011;(3):CD004787. [DOI] [PubMed] [Google Scholar]
- 39.Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C. Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database Syst Rev 2008;(2):CD007176. [DOI] [PubMed] [Google Scholar]
- 40.Bjelakovic G, Nikolova D, Simonetti RG, Gluud C. Antioxidant supplements for preventing gastrointestinal cancers. Cochrane Database Syst Rev 2008;(3):CD004183. [DOI] [PubMed] [Google Scholar]
- 41.Port Meta-analysis Trialists Group. Postoperative radiotherapy for non-small cell lung cancer. Cochrane Database Syst Rev 2005;(2):CD002142. [DOI] [PubMed] [Google Scholar]
- 42.Crowther CA, Han S. Hospitalisation and bed rest for multiple pregnancy. Cochrane Database Syst Rev 2010;(7):CD000110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Glenny AM, Furness S, Worthington HV, Conway DI, Oliver R, Clarkson JE, et al. Interventions for the treatment of oral cavity and oropharyngeal cancer: radiotherapy. Cochrane Database Syst Rev 2010;(12):CD006387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Chavez-Tapia NC, Barrientos-Gutierrez T, Tellez-Avila FI, Soares-Weiser K, Uribe M. Antibiotic prophylaxis for cirrhotic patients with upper gastrointestinal bleeding. Cochrane Database Syst Rev 2010;(9):CD002907. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Aung PP, Maxwell HG, Jepson RG, Price JF, Leng GC. Lipid-lowering for peripheral arterial disease of the lower limb. Cochrane Database Syst Rev 2007;(4):CD000123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Moberley SA, Holden J, Tatham DP, Andrews RM. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Syst Rev 2008;(1):CD000422. [DOI] [PubMed] [Google Scholar]
- 47.Rumbold A, Duley L, Crowther CA, Haslam RR. Antioxidants for preventing pre-eclampsia. Cochrane Database Syst Rev 2008;(1):CD004227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Rumbold A, Crowther CA. Vitamin E supplementation in pregnancy. Cochrane Database Syst Rev 2005;(2):CD004069. [DOI] [PubMed] [Google Scholar]
- 49.Magee LA, Duley L. Oral beta-blockers for mild to moderate hypertension during pregnancy. Cochrane Database Syst Rev 2003;(3):CD002863. [DOI] [PubMed] [Google Scholar]
- 50.Dowswell T, Carroli G, Duley L, Gates S, Gulmezoglu AM, Khan-Neelofur D, et al. Alternative versus standard packages of antenatal care for low-risk pregnancy. Cochrane Database Syst Rev 2010;(10):CD000934. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Duley L, Henderson-Smart DJ, Meher S, King JF. Antiplatelet agents for preventing pre-eclampsia and its complications. Cochrane Database Syst Rev 2007;(2):CD004659. [DOI] [PubMed] [Google Scholar]
- 52.McGee RG, Bakens A, Wiley K, Riordan SM, Webster AC. Probiotics for patients with hepatic encephalopathy. Cochrane Database Syst Rev 2011;(11):CD008716. [DOI] [PubMed] [Google Scholar]
- 53.Smieja MJ, Marchetti CA, Cook DJ, Smaill FM. Isoniazid for preventing tuberculosis in non-HIV infected persons. Cochrane Database Syst Rev 2000;(2):CD001363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Clerihew L, Austin N, McGuire W. Prophylactic systemic antifungal agents to prevent mortality and morbidity in very low birth weight infants. Cochrane Database Syst Rev 2007;(4):CD003850. [DOI] [PubMed] [Google Scholar]
- 55.Soares-Weiser K, Goldberg E, Tamimi G, Pitan OC, Leibovici L. Rotavirus vaccine for preventing diarrhoea. Cochrane Database Syst Rev 2004;(1):CD002848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, et al. Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? Int J Epidemiol 2009;38:276-86. [DOI] [PubMed] [Google Scholar]
- 57.Zhang D, Freemantle N, Cheng KK. Are randomized trials conducted in China or India biased? A comparative empirical analysis. J Clin Epidemiol 2011;64:90-5. [DOI] [PubMed] [Google Scholar]
- 58.Wu T, Li Y, Bian Z, Liu G, Moher D. Randomized trials published in some Chinese journals: how many are randomized? Trials 2009;10:46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Moher D, Schulz KF, Altman DG, Grp C. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191-4. [PubMed] [Google Scholar]
- 60.International Clinical Trials Registry Platform (ICTRP). 2012. http://apps.who.int/trialsearch/.
- 61.Tang JL, Zhan SY, Ernst E. Review of randomised controlled trials of traditional Chinese medicine. BMJ 1999;319:160-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457-65. [DOI] [PubMed] [Google Scholar]
- 63.Chan AW. Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med 2008;5:e230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Contopoulos-Ioannidis DG, Ioannidis JP. Claims for improved survival from systemic corticosteroids in diverse conditions: an umbrella review. Eur J Clin Invest 2012;42:233-44. [DOI] [PubMed] [Google Scholar]
- 65.Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 2011;343:d4002. [DOI] [PubMed] [Google Scholar]
- 66.Subtil D, Goeusse P, Puech F, Lequien P, Biausque S, Breart G, et al. Aspirin (100 mg) used for prevention of pre-eclampsia in nulliparous women: the Essai Regional Aspirine Mere-Enfant study (Part 1). BJOG 2003;110:475-84. [DOI] [PubMed] [Google Scholar]
- 67.Caritis S, Sibai B, Hauth J, Lindheimer MD, Klebanoff M, Thom E, et al. Low-dose aspirin to prevent preeclampsia in women at high risk. National Institute of Child Health and Human Development Network of Maternal-Fetal Medicine Units. N Engl J Med 1998;338:701-5. [DOI] [PubMed] [Google Scholar]
- 68.Villar J, Purwar M, Merialdi M, Zavaleta N, Thi Nhu Ngoc N, Anthony J, et al. World Health Organisation multicentre randomised trial of supplementation with vitamins C and E among pregnant women at high risk for pre-eclampsia in populations of low nutritional status from developing countries. BJOG 2009;116:780-8. [DOI] [PubMed] [Google Scholar]
- 69.Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol 2011;64:407-15. [DOI] [PubMed] [Google Scholar]
- 70.Samb B, Desai N, Nishtar S, Mendis S, Bekedam H, Wright A, et al. Chronic diseases: chronic diseases and development 4. Prevention and management of chronic disease: a litmus test for health-systems strengthening in low-income and middle-income countries. Lancet 2010;376:1785-97. [DOI] [PubMed] [Google Scholar]
- 71.Pablos-Mendez A, Gowda DK, Frieden TR. Controlling multidrug-resistant tuberculosis and access to expensive drugs: a rational framework. Bull World Health Organ 2002;80:489-95. [PMC free article] [PubMed] [Google Scholar]
- 72.Lozano R, Wang HD, Foreman KJ, Rajaratnam JK, Naghavi M, Marcus JR, et al. Progress towards Millennium Development Goals 4 and 5 on maternal and child mortality: an updated systematic analysis. Lancet 2011;378:1139-65. [DOI] [PubMed] [Google Scholar]
- 73.Sharma SK, Mohan A. Multidrug-resistant tuberculosis—a menace that threatens to destabilize tuberculosis control. Chest 2006;130:261-72. [DOI] [PubMed] [Google Scholar]
- 74.Pecoul B, Chirac P, Trouiller P, Pinel J. Access to essential drugs in poor countries—a lost battle? JAMA 1999;281:361-7. [DOI] [PubMed] [Google Scholar]
- 75.Pereira TV, Ioannidis JP. Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol 2011;64:1060-9. [DOI] [PubMed] [Google Scholar]
- 76.Huss A, Scott P, Stuck AE, Trotter C, Egger M. Efficacy of pneumococcal vaccination in adults: a meta-analysis. CMAJ 2009;180:48-58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Ioannidis JP, Lau J. The impact of high-risk patients on the results of clinical trials. J Clin Epidemiol 1997;50:1089-98. [DOI] [PubMed] [Google Scholar]
- 78.Watson R. Developing countries need stronger ethical guidelines on research. BMJ 2007;334:1076-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Ioannidis JP, Karassa FB. The need to consider the wider agenda in systematic reviews and meta-analyses: breadth, timing, and depth of the evidence. BMJ 2010;341:c4875. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary figures, legends, and table
Countries in which eligible trials had been done