Relation between randomized controlled trials published in leading general medical journals and the global burden of disease

Abstract

Background

More than two-thirds of the world's population live in low-income countries, where health priorities are different from those of people living in more affluent parts of the world. We evaluated the relation between the global burden of disease and conditions or diseases studied in randomized controlled trials (RCTs) published in general medical journals.

Methods

A MEDLINE search identified 373 RCTs that had been published in 6 international peer-reviewed general medical journals in 1999. Manual review excluded non-RCTs, brief reports and trials in which the unit of randomization was not the patient; 286 RCTs remained eligible for analysis. We identified the RCTs that studied any of the 40 leading causes of the global burden of disease. Five of these conditions were considered unsuitable for study with an RCT design and were excluded from subsequent analysis. To provide a practical perspective, we asked 12 experts working with international health organizations to rate the relevance to global health of the articles that studied any of the top 10 causes of the global burden of disease, as measured by disability-adjusted life years (DALYs) and mortality, using a 5-point Likert scale.

Results

Among the 286 RCTs in our sample, 124 (43.4%) addressed 1 of the 35 leading causes of the global burden of disease. Of these, ischemic heart disease, HIV/AIDS and cerebrovascular disease were the most commonly studied conditions. Ninety articles (31.5%) studied 1 of the top 10 causes of the global burden of disease. The mean rating (and standard deviation) for international health relevance assigned by experts was 2.6 (1.5) out of 5. Only 14 (16%) of the 90 trials received a rating of 4 or greater, indicating high relevance to international health. Almost half of the 40 leading causes of the global burden of disease were not studied by any trial.

Interpretation

Many conditions or diseases common internationally are underrepresented in RCTs published in leading general medical journals. Trials published in these journals that studied one of these high-priority conditions were generally rated as being of little relevance to international health.

More than two-thirds of the world's population live in low-income countries,¹ where health priorities are very different from those in more affluent parts of the world. Relative to people living in wealthy nations, the most impoverished 20% of the world's population is 9 times more likely to die of infectious diseases and 10 times more likely to die in childhood.² Measures of the burden of disease have been correlated with the allocation of research funds to explore whether research is being conducted in relation to need. A relation has been documented between studies funded by the National Institutes of Health and the burden of disease in the United States.³ On an international level, the Global Forum for Health Research has estimated that less than 10% of health research spending is directed toward diseases or conditions that account for 90% of the global burden of disease, a phenomenon referred to as the “10/90 gap.”⁴ These numbers suggest that many common conditions or diseases are not being adequately studied.

Globalization, particularly through international travel, means that infectious diseases cross international boundaries.⁵ Increasingly, we are aware that global health, economic development and security are intertwined.^6,7 Participants at the last 4 summit meetings of the G-8 countries are reported to have spent more time debating infectious disease than nuclear safety.⁷ Furthermore, an important component of preventive public health is to create international partnerships across the world's health research communities.⁸ Finally, from a humanitarian perspective, globalization means that we know how people live and die on the other side of the world and recognize some responsibility to improve their health and living conditions.⁹

It is important that clinical research trials highly relevant to global health be published in general medical journals. Studies published in these journals have the potential to improve awareness of global health issues in the general medical community. Some trials highly relevant to global health issues are being published in subspecialty journals. For example, infectious disease subspecialty journals are an appropriate venue for trials about conditions such as malaria. However, these journals are less frequently read by physicians in general practice or by those in subspecialties other than infectious disease, which makes the information less accessible to the general medical community.

We conducted this study to examine the representation of the conditions or diseases that are the leading causes of the global burden of disease in trials published in general medical journals. We selected general medical journals for this systematic review because they are widely read by physicians internationally and may be accessible to physicians working in the developing world.

Methods

We compared 1999 World Health Organization (WHO) data on the international burden of disease¹⁰with the conditions and diseases studied in a sample of RCTs published during a similar time period.

We used 2 measures of burden of disease, disability-adjusted life years (DALYs) and mortality, as defined in the Global Burden of Disease Study^11,12 and used by the WHO. Mortality is based only on the number of deaths, whereas DALY is defined as the sum of life years lost because of premature death and years lived with disability adjusted for severity.¹¹ Data from the WHO's annual World Health Report for the year 1999¹⁰ were used to determine the 30 leading causes of burden of disease worldwide in terms of both mortality and DALYs (see Table 1). There was considerable overlap between the conditions and diseases identified as the 30 leading causes of burden of disease as defined by DALYs and those identified as such by mortality. Combining these 2 top-30 lists yielded an overall list of 40 distinct conditions or diseases. Five of these conditions (war, homicide and violence, self-inflicted injuries, drowning and fires) were considered unsuitable for study with an RCT. Accordingly, they were excluded from subsequent analyses.

Table 1

A systematic review identified all RCTs published in 1999 in 6 international general medical journals: the Annals of Internal Medicine, the BMJ, the Journal of the American Medical Association (JAMA), The Lancet, the New England Journal of Medicine (NEJM) and the Canadian Medical Association Journal (CMAJ).⁹ The first 5 of these journals were selected because, according to the ISI Journal Citation Reports,¹³ they have the highest citation indices for general medical journals that publish RCTS and because they have been described as the leading international general medical journals.^14,15 CMAJ was selected because it is the leading Canadian general medical journal.

We used a careful 2-step process to identify trials for inclusion in our sample. First, we conducted a MEDLINE search limited to these 6 journals and the publication type “randomized controlled trial.” We identified 373 RCTs for potential inclusion. Second, 2 of the authors (D.L., A. Mashari) manually reviewed all trials and excluded non-RCTs (n = 4), commentaries (n = 5) and a cost analysis (n = 1). Research letters (n = 51) and brief reports (n = 13) (i.e., “brief communications,” “short reports” and articles less than 2 pages long) were also excluded, as their format might have restricted description of the study population. We further excluded trials in which the unit of randomization was not the patient (n = 13), as trials with the patient as the unit of randomization would be expected to include more information about their participants. A total of 286 RCTs remained in our sample.

Two reviewers (J.M.D., D.L.) independently reviewed each trial and determined whether it studied a condition or disease that was 1 of the 35 distinct conditions or diseases responsible for the WHO's leading causes of the global burden of disease, as measured by DALYs or mortality. If a trial evaluated more than one leading cause of the global burden of disease, the trial was assigned to the category involving the largest group of participants. For example, if a cancer therapy trial evaluated 100 participants with breast cancer and 50 with liver cancer, the trial was classified as studying breast cancer. Disagreements in classification were resolved in a consensus meeting with a third reviewer (P.A.R.).

To provide a more practical perspective on the international health relevance of the trials, we surveyed 19 experts working with international health organizations. We received 12 responses (63%): 2 from Médecins Sans Frontières/Doctors Without Borders, 3 from the International Clinical Epidemiology Network (INCLEN), 2 from the World Health Organization and 5 from other international health-focused organizations. Eleven of the 12 were physicians. These 12 experts, blinded to journal and author names, independently read a list of the titles of 90 trials, each examining 1 of the 10 leading causes of the global burden of disease, as measured by DALYs or mortality. The experts scored each title to indicate the degree to which they felt the article would be relevant to international health. The scores ranged from 1 (little relevance) to 5 (highly relevant). For example, an article entitled “A randomized controlled trial to evaluate the benefit of carotid endarterectomy” (a fictitious title) would probably receive a low rating, indicating that it was of little relevance to international health. In contrast, an article entitled “A randomized controlled trial to evaluate oral rehydration in children with diarrhoea in Africa” would probably receive a high rating, indicating that it was highly relevant. This rating approach based on article titles has been used successfully in another study.¹⁴

Descriptive statistics were used to describe the number and types of trials identified and the characteristics of their study populations. For the analysis of the relevance of the articles to international health, we calculated the mean rating of the 12 expert reviewers for each of the 90 article titles. Mean scores for each condition or disease were obtained by averaging the scores across all articles addressing that condition or disease. The reliability of the survey information was evaluated with a 2-way random effects intraclass correlation for the absolute agreement of results. The intraclass correlation coefficient for the averaged ratings was 0.98 for absolute agreement. One reviewer did not rate 2 of the article titles. Removing this reviewer or these items from the analysis or using the mean score of the article titles from the other 11 reviewers to replace the missing values did not affect the overall intraclass correlation coefficient.

Results

Of the 286 RCTs in our sample, 103 (36.0%) were published in The Lancet, 76 (26.6%) in NEJM, 47 (16.4%) in BMJ, 43 (15.0%) in JAMA, 14 (4.9%) in the Annals of Internal Medicine, and 3 (1.0%) in CMAJ.

Table 1 lists the 30 leading causes of the burden of disease as measured by DALYs and by mortality and the number of trials that studied these conditions or diseases. A total of 124 (43.4%) of the 286 articles in our sample addressed 1 of the conditions in the combined list of 35 distinct leading causes of global burden of disease. A total of 61 (21.3%) of the trials focused on the 3 most common conditions in our sample: 33 (11.5%) evaluated ischemic heart disease, 18 (6.3%) evaluated HIV/AIDS, and 10 (3.5%) evaluated cerebrovascular disease. Of the combined list of the 35 distinct leading causes of global burden of disease, 7 (20%) were not studied in any RCT. These included conditions or diseases for which RCTs have the potential to provide valuable information (e.g., measles and pertussis).

Table 2 lists the mean ratings assigned to the titles of 90 articles that studied 1 of the 10 leading causes of global burden of disease. Overall, the mean rating (and standard deviation [SD]) assigned by the international health experts was low (2.6 [1.5]), which indicated that the trials were of little relevance to international health. Fourteen (16%) of the 90 trials received a mean score of 4 or greater, which indicated that they were of greater relevance to international health. Of these, 3 trials focused on diarrheal diseases, 2 on malaria and 9 on HIV/AIDS. Ten of these highly rated trials were conducted in countries that were in the low (n = 8) or lower middle (n = 2) income brackets.

Table 2

Of the 18 trials that evaluated HIV/AIDS, only 9 (50%) were rated highly for their relevance to international health. The remaining 9 trials were rated as having some relevance to international health (mean rating 3.1 [SD 1.2]). Two-thirds (12 of 18) of the HIV/AIDS trials were conducted in the United States or Western Europe.

Interpretation

Our findings indicate that there is a mismatch between the global burden of disease and the conditions or diseases that were studied in clinical trials published in general medical journals in 1999. Twenty percent of the 35 most deadly and debilitating conditions or diseases experienced by people internationally were not studied by any trial in our sample.

In addition, even trials that studied a condition or disease ranked among the top 10 leading causes of global burden of disease were frequently rated as being of little relevance to global health. Specifically, less than a fifth of these trials were rated as highly relevant to global health by a group of international health experts. There are a number of potential reasons why such trials were poorly rated. For example, a therapy studied in an RCT might be beneficial but very costly, putting it beyond the reach of most people living in a low-income country.

Horton⁸ wrote to 86 clinical investigators from low-income countries to determine ways to improve the flow of information to the developing world. Four barriers to information exchange were identified: lack of relevant research, difficulties with the publication process (e.g., inability to write in English), editorial bias that may give higher priority to the interests of Western culture and restricted access to scientific information. These barriers may be responsible, in part, for the under-representation among published RCTs of the conditions or diseases that are important in low-income countries.

The current study had certain limitations. First, we focused only on RCTs. RCTs represent the “gold standard” for the evaluation of any new drug therapy, and we therefore hope that ultimately such trials will be performed to study therapies for conditions that are leading causes of global death and disability. For example, RCTs will be necessary to evaluate new drug therapies for the treatment of common infectious diseases such as malaria and tuberculosis. However, other types of study designs also provide opportunities to evaluate conditions or diseases of relevance to global health. For example, timely and important information on the clinical description of SARS from Canada¹⁶ and China¹⁷ and the possible isolation of the causative agent¹⁸ was published within weeks of its occurrence in a country other than the one where the virus originated. None of these publications was an RCT, although as therapies become available, we expect that RCTs will follow. Finally, we evaluated only trials published in leading general medical journals, but we think it unlikely that the results for less widely known general medical journals would be different.

Many conditions or diseases that are common globally are underrepresented in RCTs published in leading international general medical journals. Trials that did study these high-priority conditions were generally rated as being of little relevance to global health. Strategies are required to achieve a better balance between the global burden of disease and the topics of clinical trials published in the leading general medical journals.

β See related article page 1687