Identifying gaps in research prioritization: The global burden of neglected tropical diseases as reflected in the Cochrane database of systematic reviews

Abstract

Background:

Neglected tropical diseases (NTDs) impact disadvantaged populations in resource-scarce settings. Availability of synthesized evidence is paramount to end this disparity. The aim of the study was to determine whether NTD systematic reviews or protocols in the Cochrane Database of Systematic Reviews (CDSR) reflect disease burden.

Methods:

Two authors independently searched the CDSR for reviews/protocols regarding the NTDs diseases. Each review or protocol was classified to a single NTD category. Any discrepancy was solved by consensus with third author. NTD systematic review or protocol from CDSR were matched with disability-adjusted life year (DALY) metrics from the Global Burden of Disease 2010 Study. Spearman's rank correlation coefficient and associated P values were used to assess for correlation between the number of systematic reviews and protocols and the %2010 DALY associated with each NTD.

Results:

Overall, there was poor correlation between CDSR representation and DALYs. Yellow fever, echinococcus, onchocerciasis, and schistosomiasis representation was well-aligned with DALY. Leprosy, trachoma, dengue, leishmaniasis, and Chagas disease representation was greater, while cysticercosis, human African trypanosomiasis, ascariasis, lymphatic filariasis, and hookworm representation was lower than DALY. Three of the 18 NTDs had reviews/protocols of diagnostic test accuracy.

Conclusions:

Our results indicate the need for increased prioritization of systematic reviews on NTDs, particularly diagnostic test accuracy reviews.

Introduction

As a group of medically diverse conditions, neglected tropical diseases (NTDs) impact >1.4 billion people worldwide in low- and middle-income nations with poor access to healthcare facilities.[1,2] The World Health Organization (WHO) has established health-related milestones in the upcoming decade for the control of NTDs.[3] A recent 2015 WHO report on NTDs highlights the importance of embedding systematic and dynamic priority setting processes for NTD research at the national level.[4] For each NTD, priorities are identified in a continuum from basic disease knowledge to novel methodology (treatment, prevention, diagnosis, etc.,) and research implementation. Prioritization of limited funds to endemic countries can aid in lifting and empowering economies, which suffer from billion dollar losses every year from the burden of NTDs.[5,6,7] The availability of synthesized evidence, such as systematic reviews of interventions and diagnostic tests, is crucial for disease control initiatives and optimal resource allocation.

Cochrane is a global partnership of thousands of researchers, healthcare practitioners, and patient advocates producing the “gold standard” of systematic reviews.[8,9] The Cochrane Database of Systematic Reviews (CDSR) houses systematic reviews and protocols (published proposals for future systematic reviews) of interventions and diagnostic test accuracies covering a broad diversity of diseases and cross-cutting topics.[10] Since CDSR reviews are methodologically rigorous and require substantial investment of time and resources, Cochrane has initiated an effort to promote the transparency of priority setting processes by establishing empirical methods for review prioritization.[11,12,13] Systematically analyzed data on burden of disease for disadvantaged groups is important to inform priority setting exercises in the Cochrane Collaboration and other research databases.

Disease burden may be used to prioritize investments and set priorities for health research.[14] The Global Burden of Disease (GBD) Study 2010 is a collaboration of nearly 500 researchers representing 50 countries across the globe. GBD has transformed the global health landscape by creating public access to an objective measure of burden from 291 diseases and injuries, including 18 NTDs, in 187 countries.[15] Disease burden is measured by disability-adjusted life years (DALYs), a metric which uniquely combines a morbidity component (years lost to disability) and a mortality component (years of life lost). The following 18 NTDs were studied by GBD: Chagas disease, leishmaniasis, human African trypanosomiasis, schistosomiasis, cysticercosis, echinococcus, lymphatic filariasis, onchocerciasis, trachoma, dengue, yellow fever, rabies, ascariasis, trichuriasis, hookworm disease, foodborne trematodiases, leprosy, and other NTDs. The other NTDs category contains 27 conditions. Prior studies have assessed the relationship between the burden of broad categories of disease with systematic reviews and randomized trials.[16,17] This study will specifically determine the correlation between CDSR representation of NTDs and respective GBD 2010 DALY estimates.

Methods

NTD search terms were generated using GBD International Classification of Diseases-Tenth Revision codes and disease synonyms, and entered into the CDSR “title, abstract, keywords” search function.[18] Search terms were selected as broad as possible. In addition, since NTDs have many synonymous names, these were added as search terms to increase the sensitivity of the search. Two authors independently performed the CDSR search (updated February 24–25, 2015). A systematic review or protocol was matched to one of the 18 NTDs according to its study objectives and main results by two authors independently. No systematic review was classified into more than one NTD disease category. The particular NTD was required to be a predominant focus of the objectives and main results. Systematic reviews that did not relate substantively to at least one of the 17 NTDs were excluded. Data were collected on type of publication (systematic review or protocol), date of publication, Cochrane review group, type of systematic review (intervention or diagnostic test accuracy [DTA]), funding support, and number and countries of authors. Any discrepancy in the selection and extraction was solved by consensus with third author.

Methodology used by GBD 2010 to generate DALY estimates is published elsewhere.[19,20] GBD data are available for public access through interactive online data visualizations.[21,22] The following metrics were collected for the 18 NTDs: Percent of total 2010 DALYs of all 291 diseases studied by GBD and median percent change in DALY from year 1990 to 2010. Spearman rank-correlation coefficients and associated P values were used to assess for correlation between the number of systematic reviews and protocols and the %2010 DALY associated with each NTD. Rho, a measure of correlation, and the P value, which tests the null hypothesis of no correlation, were determined. A data plot with linear line of best fit was used to visually demonstrate over- or under-representation of NTDs in CDSR in relation to disease burden.

Since the current study is an analysis of data already in the public domain and does not involve living subjects, Institutional Review Board approval was not required.

Results

Search terms yielded a total of 58 CDSR titles, of which 12 were excluded due to lack of abstract objectives and main results focus on the particular NTD [Tables 1 and 2]. Thus, 28 systematic reviews and 10 protocols representing the 17 single NTDs and 6 systematic reviews and 2 protocols representing the other NTDs category were included in the analysis [eTables 1 and 2 for included titles and eTables 3 and 4 for excluded titles].

Table 1.

NTD conditions studied by GBD 2010 with corresponding ICD-10 codes, search terms, number of systematic reviews (R) and protocols (P) in CDSR, and percent of total DALYs (arranged in order of decreasing percentage of total DALY)

Table 2.

“Other NTDs” category studied by GBD 2010 with corresponding ICD-10 codes, search terms, number of systematic reviews (R) and protocols (P) in CDSR, and percent of total DALYs (arranged in order of decreasing percentage of total DALY)

eTable 1.

Titles for all reviewa and protocols included for 17 NTDs studied by GBD 2010 and associatde Cochrance group responsible for publication

eTable 2.

Titles for all reviews and protocols included for the other NTDs studied by GBD 2010 and associatde Cochrance group responsible for publication

eTable 3.

Excluded titles generated from search 17 NTDs studied by GBD 2010 in the CDSR

eTable 4.

Excluded titles generated from search the other NTDs studied by GBD 2010 in the CDSR

These were published by the following Cochrane review groups: Infectious Diseases Group (n = 19), Eyes and Vision Group (8), Skin Group (5), Heart Group (3), Pregnancy and Childbirth Group (3), HIV/AIDS Group (2), Neuromuscular Group (2), Neuromuscular Disease Group (1), Gynecological Group (1), Hepato-Biliary Group (1), Breast Cancer Group (1). More than half (63%) of systematic reviews and protocols were published in 2010 or later.

There was a weak positive, nonsignificant correlation between %2010 DALY and number of Cochrane reviews and protocols associated with each NTD (rho = 0.281, P = 0.259). Overlaying a line of best fit to the scatter plot of our data visually emphasizes that the number of reviews and protocols per NTD was not strongly associated with %2010 DALY in a linear fashion (R² = 0.33)” [Figure 1]. When protocols were excluded from the analysis, correlation was even lower and not statistically significant (rho = 0.13, P = 0.61).

Figure 1.

Comparison between neglected tropical disease representation in the Cochrane Database of Systematic Reviews and burden of disease disability-adjusted life year metrics: There was a weak positive, nonsignificant correlation between %2010 DALY and number of Cochran reviews and protocols associated with each NTD (rho = 0.281, P = 0.259). Overlaying a line of best fit to the scatter plot of our data visually emphasizes that the number of reviews and protocols per NTD was not strongly associated with %2010 disability-adjusted life year in a linear fashion (R² = 0.33)

Schistosomiasis (1 P), leishmaniasis (1 R), and dengue (1 P) were the only conditions with systematic reviews or protocols of DTA. While 204 authors from 31 nations generated the 46 titles representing the NTDs, almost half of reviews (n = 20 or 43%) did not have a single author from geographic regions where NTDs are prevalent, such as Asia, Africa, or South America. In fact, most of the systematic reviews (n = 30 or 65%) were published by a first author from Europe or North America, areas where NTDs are virtually nonexistent. Analysis of funding availability demonstrated that 57% of reviews were funded (n = 26), whereas 43% were unfunded (20).

Of the conditions studied, the other NTDs category accounted for the greatest disease burden. Only six of 27 conditions included in this composite category (Lyme disease, toxoplasmosis, pneumocystis, loiasis, congenital toxoplasmosis, and strongyloidiasis) were represented in CDSR. Thus, CDSR reviews are absent for 21 conditions in the other NTDs category [Table 2].

Discussion

Overall, there was poor correlation between the global burden of NTDs with systematic review and protocol representation in CDSR. There is a fundamental lack of synthesized research conducted in low- and middle-income nations where most NTDs are endemic. Cysticercosis, human African trypanosomiasis, ascariasis, lymphatic filariasis, and hookworm disease are specific NTDs that perhaps warrant increased prioritization by CDSR according to their disease burden. Prioritization of DTA reviews is a potential area for CDSR expansion since diagnostic tests are pivotal components of healthcare decisions for early intervention. In order to accomplish disease eradication, WHO has underlined the need for targeted research to develop accessible new diagnostics for NTDs.[23]

While burden of disease is important, priority setting is a complex process with many considerations, such as equity, cost-effectiveness, intervention availability, interest group advocacy, capacity building, and infrastructure availability. Many Cochrane review groups use particular criteria to guide decision-making processes such as disease burden, problem magnitude, and impact of an intervention on policy or treatment change.[13] As an example, the Cochrane Infectious Diseases Group accords greater priority status for review topics that align with the United Nations Millennium Development Goals to reduce inequalities in income, food, education, sex, child and maternal mortality, and diseases that affect marginalized populations.

Cochrane typically publishes synthesized research evidence available from randomized controlled trials. Paucity of randomized controlled trials covering NTDs may be an important factor contributing to poor CDSR representation, particularly for rabies, trichuriasis, and food-borne trematodiasis, which had no systematic reviews or protocols. A 2012 network analysis of the randomized evidence for first- and second-line NTD treatments found 8 of 16 NTDs with either only one trial or fewer than 100 participants.[24]

An important consideration in priority setting is equipoise, which describes uncertainty in the efficacy of particular treatments. Interventions with well-established efficacy may not be highly prioritized as topics for future systematic reviews. Lack of researchers with training on research synthesis methods from nations where NTDs are endemic is another potential contribution to the mismatch between burden of disease and CDSR representation. Greater than 40% of NTD reviews in CDSR lacked an author from endemic regions, highlighting a need to build synthesized research capacity in low- and middle-income nations. Finally, our investigation of NTD systematic review funding patterns is consistent with the 2010 WHO report that described a lack of funding in primary NTD studies.[23]

Limitations of the GBD study have been previously described.[14] Many systematic reviews include “lumping” of many interventions with a broad scope while others may be “split” into a narrow scope covering a single intervention or condition. Lumping is often used for topic areas with limited number of clinical trials. Thus, “number counting” of reviews and protocols to assess NTD representation may not always be an appropriate measure. There have also been important and well-described limitations to the use of disease burden for NTDs, which are generally considered to be grossly underestimated. Reasons for this include lack of standardized disease definitions, unequal access to medical care, and inadequate assessment of the financial, physical, psychological, and social burden of NTDs.[25,26] These limitations are also inherent to prevalence and incidence estimates for NTDs. Nevertheless, evidence-based public health measures are critical and inherently dependent on the availability of synthesized research. In alignment with the 2015 WHO NTD guidelines, we suggest accordance of greater priority to synthesized research on NTDs in pursuit of future research, capacity building, prioritization processes, and funding decisions. The results of our study provide empirical data to enhance the transparency and guide priority setting of future systematic reviews.

Conclusion

Results of the study indicate the need for increased prioritization of systematic reviews on NTDs, particularly diagnostic test accuracy reviews.

Financial support and sponsorship

The study was funded by the Bill and Melinda Gates Foundation.

Conflicts of interest

SB, MN and RPD are involved with activities of the Cochrane. No other financial or non-financial conflicts of interest.