Abstract
Purpose
To conduct an expert-led process for identifying research priorities for eight areas of adolescent health in low- and middle-income countries. Specific adolescent health areas included communicable diseases prevention and management, injuries and violence, mental health, noncommunicable diseases management, nutrition, physical activity, substance use, and health policy.
Methods
We used a modified version of the Child Health and Nutrition Research Initiative methodology for reaching consensus on research priorities. In a three phase process, we (1) identified research and program experts with wide-ranging backgrounds and experiences from all geographic regions through systematic searches and key informants; (2) invited these experts to propose research questions related to descriptive epidemiology, interventions (discovery, development/testing, and delivery/implementation), and health policy/systems; and (3) asked the experts to prioritize the research questions based on five criteria: clarity, answerability, importance or impact, implementation, and equity.
Results
A total of 142 experts submitted 512 questions which were edited and reduced to 303 for scoring. Overall, the types of the top 10 research questions in each of the eight health areas included descriptive epidemiology (26%), interventions: discovery (11%), development/testing (25%), delivery (33%), and policy, health and social systems (5%). Across health areas, the top questions highlighted integration of health services, vulnerable populations, and different health platforms (such as primary care, schools, families/parents, and interactive media).
Conclusions
Priority questions have been identified for research in eight key areas of adolescent health in low- and middle-income countries. These expert-generated questions may be used by donors, program managers, and researchers to prioritize and stimulate research in adolescent health.
Keywords: Adolescent health, Research priorities, Low-income countries, Middle-income countries, World Health Organization, Health systems, Communicable Diseases, Injuries, Violence, Mental health, Noncommunicable diseases, Nutrition, Physical activity, Substance use, Health policy
Implications and Contribution.
The Department of Maternal, Newborn, Child, and Adolescent Health of the World Health Organization (WHO) conducted an exercise to establish global research priorities for adolescent health in low- and middle-income countries through 2030, building on earlier work that proposed research priorities in adolescent sexual and reproductive health.
In 2014, there were 1.2 billion adolescents aged 10–19 years old, comprising 16.4% of the world's population. Adolescent mortality was estimated at 1.3 million in 2012, with the leading global causes of death being road injury, human immunodeficiency virus (HIV), suicide, lower respiratory infections, and interpersonal violence [1]. The great majority of the world's adolescents live in low- and middle-income countries (LMICs) [1], and 97% of deaths among young people occur in LMICs [2]. In the past 50 years, reductions in early child mortality have been greater than declines in adolescent mortality [3].
In terms of the global burden of diseases, the top three causes of disability-adjusted life years lost among adolescents are unipolar depressive disorders, road injury, and iron-deficiency anemia [1]. However, mortality and disability-adjusted life year data will underestimate the potential disease burden among adolescents because they do not reflect conditions and behaviors that can lead to future disability and mortality later in life, such as tobacco use and dependence or physical inactivity [4]. Health-related risk behaviors adopted or consolidated during adolescence may not always affect the adolescent's health during the second decade of life but will have a substantial effect later in life, and some will affect the health of future generations [4], [5].
Improving the health of adolescents in LMICs will be essential for the world to achieve the United Nations Sustainable Development goals [6], and the specific targets and goals included in the United Nations Secretary General's Global Strategy for Women's, Children's and Adolescents' Health [7]. Although there has been an increased call for research on the health and wellbeing of adolescents and young people to guide these and other global and national initiatives, research from LMICs is still limited [1], [8].
Here, we report the findings from an exercise to identify research priorities for eight areas of adolescent health in LMICs with the aim of stimulating research on the priority questions identified. The specific areas of adolescent health selected for inclusion were communicable diseases prevention and management (including diarrhea, parasites, hepatitis, malaria, meningitis, tuberculosis, influenza, pertussis, pneumonia, and others), injuries and violence, mental health, noncommunicable diseases management (including asthma, diabetes, cancer, hypertension, heart disease, and others), nutrition, physical activity, substance use, and adolescent health: policy, health and social systems. Of note, adolescent sexual and reproductive health and related topics were not included, as they had been the subject a recent similar research prioritization exercise [9].
Methods
The Child Health and Nutrition Research Initiative (CHNRI) developed a method for ranking the relative importance of competing research options to help decision makers to effectively allocate limited resources to reduce morbidity and mortality [10]. The CHNRI approach has previously been applied to more than 50 health areas [11], [12], [13], [14], [15], including adolescent sexual and reproductive health [9].
We implemented a modified version of the CHNRI priority setting method in three phases. In Phase 1, we identified research and program experts through systematic searches of published and gray literature, members of journal editorial boards, and through interviews with key informants at WHO, and invited them to participate in the exercise. In Phase 2, we asked the experts who agreed to participate to propose research questions related to descriptive epidemiology, interventions, and health and social systems research. In Phase 3, we asked the same experts to prioritize the research questions generated in Phase 2 using a scoring scheme based on five criteria.
Phase 1: Identification of Research and Program Experts
Experts were identified through journal publications, membership of journal editorial boards, from lists of participants at WHO meetings and consultations, and by nominations from relevant WHO departments. For journal publications, we identified experts in each health area through a systematic search of PubMed and Web of Science databases from 2005 to 2015. To be included on this preliminary list, authors had to have published at least two relevant articles within a specific health area that explicitly covered adolescents (ages 10–19 years) in LMICs during the 2005–2015 period. If more than 20 experts met these criteria, then the number was reduced to a maximum of 20, based on number of publications, relevance of the titles of the articles, and the position of authorship, with discrepancies resolved through discussion by D.R. and J.F. This resulted in 116 experts.
We searched for peer-reviewed journals related to adolescent health in all six official United Nations languages. Members of the editorial boards of the two peer-reviewed journals related to adolescent health with the highest impact factor (Journal of Adolescent Health—2.75 and Journal of Research on Adolescence—2.51) based on Web of Science Journal Citation Reports for 2013 [16] were included in the adolescent health: policy, health and social systems area. This identified an additional 69 experts.
Since the experts identified through the systematic PubMed and Web of Science search were likely to mainly be researchers, we also identified participants at WHO meetings and consultations held in 2010–2015 and that were relevant to the eight adolescent health areas through reports that were available on the WHO website and the WHO Index Medicus, a database focused on health literature produced by and within LMICs from all regions. Such meetings usually include program implementers and policymakers and researchers. The meetings included several that had participation by young persons themselves. We also invited representatives of the WHO departments relevant to each health area to review the lists and nominate any additional key experts in their respective fields. Overall, this resulted in 265 additional experts.
Combining the list of experts resulted in a total of 450 different individuals (Table 1). All these 450 experts were sent an invitation to participate in the research prioritization process, and 217 (48%) agreed to participate.
Table 1.
Health area | Experts identified through PubMed and WOS searches |
Experts identified through WHO consultation reports and by WHO departments |
Total experts identified |
Agreed to participate |
Submitted questions |
Scored questions by health area |
Other scorersa | |||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
n | % | n | % | N | n | % | n | % | n | % | ||
Formula | a | a/c | b | b/c | c | d | d/c | e | e/d | f | f/d | |
Communicable diseases | 5 | 9 | 51 | 91 | 56 | 28 | 50 | 18 | 64 | 16 | 57 | 1 |
Injuries and violence | 23 | 32 | 50 | 68 | 73 | 22 | 30 | 17 | 77 | 19 | 86 | 1 |
Mental health | 15 | 31 | 33 | 69 | 48 | 31 | 65 | 22 | 71 | 21 | 68 | 5 |
NCDs management | 17 | 37 | 29 | 63 | 46 | 18 | 39 | 11 | 61 | 10 | 56 | 2 |
Nutrition | 20 | 36 | 36 | 64 | 56 | 39 | 70 | 22 | 56 | 18 | 46 | 2 |
Physical activity | 17 | 33 | 35 | 67 | 52 | 19 | 37 | 15 | 79 | 13 | 68 | 0 |
Substance use | 19 | 45 | 23 | 55 | 42 | 23 | 55 | 13 | 57 | 11 | 48 | 0 |
Adolescent health: policy, health, and social systems | 69 | 90 | 8 | 10 | 77 | 37 | 48 | 24 | 65 | 22 | 59 | 19 |
Total | 185 | 41 | 265 | 59 | 450 | 217 | 48 | 142 | 65 | 130 | 60 | 30 |
NCDs = Noncommunicable diseases; WHO = World Health Organization; WOS = Web of Science.
Experts in “adolescent health: policy, health and social systems” were invited to also score in a specific health area of their choice, and experts in a specific health area were invited to also score in “adolescent health: policy, health and social systems.”
Phase 2: Identification of Research Questions
The experts identified in Phase 1 were divided into groups based on their expertise in the eight adolescent health areas. Each expert was asked to propose research questions of the greatest priority for adolescent health within their health area related to descriptive epidemiology, interventions, and health and social systems research:
-
1.
Descriptive epidemiology (1 question): descriptive studies, designed to measure burden of disease, explore risk, and protective factors.
-
2.Intervention research
-
A.Discovery research (1 question): designed to create new interventions.
-
B.Development and testing research (1 question): development, testing, and evaluation of interventions.
-
C.Implementation/Delivery research (1 question): designed to improve the effectiveness, deliverability, affordability, sustainability, and scale-up of existing interventions.
-
A.
-
3.
Health policy/Health and social systems research (1 question): questions related to the overall health and social systems that affect adolescents not necessarily specific to any one health area.
Questions were submitted via a survey tool using SurveyMonkey (Palo Alto). The 512 submitted questions (Appendix A) were synthesized by removing redundancies and questions not relevant to adolescent health, as well as repositioning questions that belonged in different health areas. Some questions were rephrased in an attempt to improve clarity. This resulted in 303 questions that were included in the final scoring (Appendix B).
Phase 3: Prioritization of Research Questions
The same 217 experts were asked to score the final list of research questions generated in Phase 2 in their health area of expertise and in the adolescent health: policy, health and social systems area. Experts in the adolescent health: policy, health and social systems area were asked to score one additional health area of their choosing.
Experts were asked to score questions against five specific criteria:
-
1.Descriptive epidemiology:
-
○Clarity: is the question well framed and are its endpoints clear?
-
○Answerability: can the question generate important new knowledge in an ethical way?
-
○Importance: would the question identify problems that may result in an important intervention?
-
○Implementation: how likely will the question contribute to tailoring of interventions to targeting of specific populations?
-
○Equity: would the answer to this question help to identify inequities (e.g., in disease burden, access to and/or utilization of services)?
-
○
-
2.Intervention research (discovery, development/testing, and implementation/delivery research):
-
○Clarity: is the question well framed and are its end-points clear?
-
○Answerability: can the question generate important new knowledge in an ethical way?
-
○Impact (Discovery): would the answer to this question be likely to result in the identification of an intervention, which, if proved effective, would be very important for the improvement of adolescent health and/or development?
-
○Impact (Development/Testing and Implementation/Delivery): would the answer to this question, if positive, result in an effective intervention?
-
○Implementation: would the answer to this question, if positive, result in an intervention or a strategy with a strong likelihood of being affordable and sustainable in most LMICs?
-
○Equity: would the answer to this question help to identify inequities (e.g., in disease burden, access to and/or utilization of services)?
-
○
-
3.Health and social systems research
-
○Clarity: is the question well framed and are its endpoints clear?
-
○Answerability: can the question generate important new knowledge in an ethical way?
-
○Importance: how likely is this question to change adolescent health?
-
○Implementation: what is the feasibility of this systems change?
-
○Equity: would the answer to this question help to identify inequities (e.g., in disease burden, access to and/or utilization of services)?
-
○
Experts were asked to score each question for each of the criteria based on the standard CHNRI scoring system: yes, no, or undecided.
In October 2015, 15 external experts joined the authors and other WHO staff in a meeting at which the methods and preliminary findings were discussed before they were finalized.
Data Analysis
All answers were converted to a score. A “yes” scored 100; “undecided” 50; and “no” 0 points. Rankings were based on the total Research Priority Score (RPS), which was computed as the mean of the scores for the different criteria, weighted according to published guidelines from CHNRI stakeholders [17] and adjusted to a 100-point scale, according to the formula:
RPS = [(answerability × .86) + (impact × 1.56) + (deliverability × .77) + (equity × .81)]/4. Although clarity has been used as a criterion in previous CHNRI exercises [9], its weight has not been validated by CHNRI methodological guidelines so it was not included in the final RPS [17]. In addition, the Average Expert Agreement (AEA) scores are reported, which represent the average proportion of scorers that agreed on responses for each of the five criteria asked. This was computed as:
Results
Characteristics of the 142 experts who submitted questions are shown in Table 2. Over half were female (57.0%) and were employed in academic institutions (63.1%). There were fewer representatives from governments and donor organizations. Most experts had a postgraduate degree (88.7%). About half of the experts described their primary role as a researcher, whereas about a fifth were program managers and a 10th were clinical health practitioners or policy makers. The experts represented 62 countries from North America (28.2%), South America (10.6%), Europe (21.1%), Africa (14.8%), Asia (13.4%), and Oceania (12.0%). The number of experts who were identified and who participated in each of the stages of the exercise is shown by health area in Table 1. The total number of experts who were approached was 450; the number varied by “health area” from 44 (substance use) to 77 (adolescent health: policy, health and social systems). From these, a total of 217 agreed to participate, 142 submitted questions, and 130 scored the questions.
Table 2.
n | Percentage (%) | |
---|---|---|
Sex (N = 142) | ||
Male | 61 | 43.0 |
Female | 81 | 57.0 |
Age (N = 140) | ||
20–29 | 10 | 7.0 |
30–39 | 20 | 14.1 |
40–49 | 37 | 26.1 |
50–59 | 41 | 28.9 |
60–69 | 26 | 18.3 |
70–79 | 6 | 4.2 |
Country of nationality (N = 142) | ||
North America | 40 | 28.2 |
South America | 15 | 10.6 |
Europe | 30 | 21.1 |
Africa | 21 | 14.8 |
Asia | 19 | 13.4 |
Oceania | 17 | 12.0 |
Highest degree (N = 141) | ||
Postgraduate degree | 125 | 88.7 |
Undergraduate degree | 16 | 11.3 |
Organization (N = 142) | ||
Academic | 87 | 61.3 |
International organization | 22 | 15.5 |
Nongovernmental organization | 13 | 9.2 |
Government | 6 | 4.2 |
Donor organization | 2 | 1.4 |
Consulting | 5 | 3.5 |
Other | 7 | 4.9 |
Primary role (N = 142) | ||
Researcher | 80 | 56.3 |
Program manager | 29 | 20.4 |
Health practitioner | 14 | 9.9 |
Policy maker | 14 | 9.9 |
Donor representative | 4 | 2.8 |
Other | 1 | .7 |
Denominators change slightly because of missing data.
Appendix C presents the full list of 303 questions that the experts were asked to score, the mean scores of each question and of each health area. The top 10–ranked research questions in each of the eight health areas are shown in Table 3. The total RPS for the top 10 questions in the eight health areas ranged from 73 to 100 out of a possible 100. The AEA score for the top 10 questions (as ranked by total RPS) ranged from 61 to 98 out of a possible 100 in the eight health areas.
Table 3.
Health area | Research type | Clarity score | Answerability score | Impact score | Implementation score | Equity score | Total score (Research Priority Score) | Average expert agreement | |
---|---|---|---|---|---|---|---|---|---|
Communicable diseases prevention and management | |||||||||
1 | What are the key barriers faced by adolescents to access TB and TB/HIV diagnostic and treatment services in high- and low-income countries, and how can these be overcome? | Intervention: delivery/implementation | 100 | 97 | 100 | 90 | 97 | 97 | 95 |
2 | What are treatment adherence rates, and what are the risk factors for nonadherence or default, among adolescents on long-term treatment for TB? | Descriptive epidemiology | 100 | 94 | 97 | 94 | 97 | 96 | 93 |
3 | What is the potential contribution of peer-led interventions for improving retention in care among adolescents with TB and/or HIV? | Intervention: delivery/implementation | 97 | 94 | 97 | 91 | 90 | 94 | 87 |
4 | Which programmatic interventions developed to improve adolescent retention in care and treatment adherence for other communicable diseases (i.e., HIV) would be useful for application in TB programs? | Intervention: delivery/implementation | 88 | 93 | 93 | 90 | 93 | 93 | 84 |
5 | What is the incidence and burden of TB among younger (10–14 years) and older (15–19 years) adolescents in the world, by sex, particularly among adolescents with HIV, and what proportion of the adolescents have drug resistant TB? | Descriptive epidemiology | 91 | 100 | 91 | 85 | 91 | 92 | 87 |
6 | What is the best way to help adolescents to adhere to TB medication when they are also taking ARVs? | Intervention: development/testing | 88 | 88 | 94 | 94 | 90 | 92 | 84 |
7 | What are the true rates (based on empirical data not models) of mortality and DALYs lost from diarrheal disease and from lower respiratory tract infections in adolescents (10–14 years and 15–19 years) by sex, SES, rural/urban, by HIC/UMIC/LMIC/LIC, and by world region? | Descriptive epidemiology | 91 | 94 | 90 | 88 | 94 | 91 | 85 |
8 | What are the rates of development of antituberculosis drug resistance in adolescents? | Descriptive epidemiology | 82 | 100 | 91 | 91 | 81 | 91 | 82 |
9 | How effective and cost-effective is the integration of HIV and TB surveillance to enhance early detection and case management in adolescents? | Intervention: delivery/implementation | 100 | 100 | 94 | 88 | 78 | 91 | 85 |
10 | Can overall duration of TB treatment and/or frequency of TB medication dosing be reduced to facilitate adherence and improve rates of treatment completion among adolescents? | Intervention: development/testing | 94 | 83 | 100 | 90 | 80 | 90 | 84 |
Communicable diseases prevention and management (mean) | 93 | 94 | 95 | 90 | 89 | 93 | 87 | ||
Injuries and violence | |||||||||
1 | What are the barriers and facilitators to increasing compliance with motorcycle helmet legislation? | Intervention: delivery/implementation | 100 | 94 | 91 | 91 | 75 | 88 | 86 |
2 | What are the risk and protective factors at various levels (individual, family, peer/social, community) for injuries and violence among adolescents LMICs? | Descriptive epidemiology | 73 | 95 | 90 | 82 | 80 | 87 | 77 |
3 | How best can school-based “safe routes to school” initiatives be scaled up to include larger numbers of schools and to be incorporated with community-based initiatives? | Intervention: delivery/implementation | 97 | 80 | 83 | 83 | 80 | 82 | 80 |
4 | To what extent do strategies that have been shown to reduce one form of violence (e.g., bullying) effectively prevent other forms of violence that youth experience (e.g., partner violence, sexual violence, suicidal behavior)? | Intervention: development/testing | 92 | 83 | 86 | 72 | 75 | 81 | 72 |
5 | What types of communication strategies work best to actually change the key behaviors that put adolescents at increased risk of injuries? | Intervention: delivery/implementation | 88 | 88 | 91 | 78 | 56 | 81 | 75 |
6 | What are the risk and protective factors associated with the increased risk of burn injuries among adolescent girls in many South Asian countries? | Descriptive epidemiology | 65 | 92 | 88 | 76 | 58 | 80 | 66 |
7 | What are the incidence and risk and protective factors associated with fatal drowning among adolescents? | Descriptive epidemiology | 75 | 100 | 80 | 68 | 68 | 80 | 72 |
8 | How best can brief alcohol interventions be combined with brief violence reduction interventions and be effectively delivered through the health system, when adolescents present with injuries? | Health policy, health and social systems | 77 | 83 | 82 | 79 | 70 | 79 | 70 |
9 | What specific behavior modification strategies are effective in reducing risk taking behavior and exposure to injury and violence? | Intervention: Development/Testing | 72 | 83 | 83 | 75 | 64 | 78 | 66 |
10 | Can swimming and water survival training be effectively implemented in late primary or early secondary school in LMICs in high risk/high need communities, and do such programs have an intergenerational effect by also protecting subsequent generations of children by transfer of awareness, skills, knowledge, rescue techniques? | Intervention: development/testing | 83 | 89 | 81 | 69 | 64 | 77 | 66 |
Injuries and violence (mean) | 82 | 89 | 85 | 77 | 69 | 81 | 73 | ||
Mental health | |||||||||
1 | What would be the most cost-effective, affordable and feasible package of interventions for promotion of mental health and prevention of mental health disorders among adolescents? | Intervention: development/testing | 92 | 95 | 100 | 92 | 86 | 94 | 88 |
2 | What are effective interventions to prevent and treat mental health problems of adolescents that can be delivered at primary care level in LMICs? | Intervention: development/testing | 100 | 97 | 89 | 87 | 82 | 89 | 88 |
3 | What are effective interventions addressing self-harm/suicide in adolescent girls in LMICs? | Intervention: development/testing | 97 | 87 | 97 | 79 | 76 | 87 | 82 |
4 | What are the costs and benefits of integrating management of child and adolescent mental disorders with other child and adolescent health care delivery platforms? | Intervention: delivery/implementation | 94 | 92 | 89 | 86 | 78 | 87 | 82 |
5 | How can mental health and psychosocial support (including identification, support and basic management of relevant conditions) be integrated with adolescent friendly services, general health, reproductive health, etc? | Health policy, health and social systems | 91 | 91 | 94 | 79 | 71 | 86 | 74 |
6 | What is the effectiveness of parenting programs in the prevention of mental health disorders in adolescents? | Intervention: development/testing | 95 | 95 | 95 | 79 | 66 | 86 | 81 |
7 | How can mental health promotion interventions in schools be scaled up in LMICs? | Intervention: delivery/implementation | 89 | 92 | 94 | 86 | 61 | 85 | 79 |
8 | What are effective and evidence-based screening, prevention, and treatment interventions for adolescents with neurodevelopmental disorders and intellectual disabilities (and their families) that can be delivered in various settings? | Intervention: development/testing | 79 | 84 | 89 | 84 | 79 | 85 | 72 |
9 | What are the risk and protective factors for mental health problems among adolescents with developmental disorders (e.g., developmental delay, autism) in LMICs? | Descriptive epidemiology | 95 | 89 | 84 | 82 | 84 | 85 | 80 |
10 | What are the models of adolescent mental health services in LMICs that can be delivered to particularly vulnerable adolescents (refugees, out-of-school youth, young people living with HIV, exposed to gender-based violence, youth in armed conflict)? | Intervention: delivery/implementation | 86 | 89 | 81 | 81 | 92 | 85 | 80 |
Mental health (mean) | 92 | 91 | 91 | 83 | 77 | 87 | 81 | ||
Noncommunicable disease management | |||||||||
1 | Can a low-cost rapid antigen test for diagnosis of streptococcal pharyngitis (which can lead to rheumatic heart disease) be developed that is suitable for use in low-resource settings? | Intervention: discovery | 95 | 95 | 86 | 86 | 77 | 86 | 84 |
2 | Can interventions for the management of NCDs that have been shown to be effective in adults be used directly in adolescents? | Intervention: development/testing | 86 | 85 | 95 | 75 | 75 | 85 | 73 |
3 | How do interventions devised for the management of NCDs in high-income countries be used for adolescents in LMICs translate globally? | Intervention: delivery/implementation | 91 | 100 | 86 | 73 | 64 | 82 | 71 |
4 | What are the mortality and morbidity rates and their causes among adolescents with diabetes in LMICs | Descriptive epidemiology | 91 | 77 | 77 | 86 | 77 | 79 | 78 |
5 | What proportion of children born with sickle cell disease survive into and through adolescence? | Descriptive epidemiology | 86 | 91 | 77 | 77 | 68 | 78 | 67 |
6 | What are successful strategies of existing effective programs in LMICs that manage adolescents with diabetes? | Intervention: delivery/implementation | 32 | 80 | 80 | 75 | 75 | 78 | 65 |
7 | What interventions are effective in improving access to the medicines and supplies needed to manage diabetes and other endocrine disorders in adolescents? | Health policy, health and social systems | 80 | 89 | 78 | 72 | 72 | 78 | 65 |
8 | Are there biological changes (markers) heralding NCDs in adolescents, and how early can these be detected? | Intervention: discovery | 64 | 91 | 82 | 55 | 73 | 77 | 65 |
9 | How does the prevalence of NCDs change during adolescence by age and sex? | Descriptive epidemiology | 55 | 75 | 90 | 65 | 60 | 76 | 61 |
10 | Is it possible to develop a low-cost, accurate blood glucose point of care test (end-user cost USD .10 or less per test)? | Intervention: discovery | 82 | 82 | 75 | 75 | 60 | 73 | 63 |
Noncommunicable disease management (mean) | 76 | 87 | 83 | 74 | 70 | 79 | 69 | ||
Nutrition | |||||||||
1 | What are the causes of anemia among adolescent girls and how does this vary by region? | Descriptive epidemiology | 98 | 85 | 93 | 92 | 93 | 91 | 87 |
2 | What are the relationships between early pregnancy and stunting, anemia, and NCD risk (overweight, diabetes, and hypertension)? | Descriptive epidemiology | 89 | 97 | 87 | 89 | 79 | 88 | 84 |
3 | What social and behavior change communication platforms are the most effective to reach adolescents to help them to improve their diet? | Intervention: development/testing | 85 | 98 | 93 | 73 | 73 | 86 | 75 |
4 | How does the burden of disease from nutritional causes for adolescent boys and girls vary by country and within countries, and by socioeconomic status? | Descriptive epidemiology | 85 | 80 | 80 | 89 | 95 | 85 | 83 |
5 | What is the prevalence of adolescent undernutrition and overnutrition by risk/protective factors such as sex, urban/rural residence, schooling, access to green spaces, access to food and socioeconomic strata in different world regions? | Descriptive epidemiology | 88 | 85 | 83 | 74 | 88 | 82 | 78 |
6 | How do nutrition interventions during adolescent antenatal and postnatal visits impact on birth outcome, maternal, neonatal and child health? | Intervention: development/testing | 80 | 88 | 84 | 79 | 70 | 81 | 70 |
7 | How can community-based adolescent obesity prevention programs be better implemented and scaled up? | Intervention: delivery/implementation | 78 | 85 | 80 | 75 | 78 | 80 | 64 |
8 | What antenatal interventions can be developed to help support the specific health and nutritional needs of adolescent pregnant girls in developing countries? | Intervention: discovery | 84 | 79 | 82 | 74 | 82 | 80 | 71 |
9 | What are the most effective interventions for preventing and reducing overweight/obesity in adolescents either in schools or out-of-school? | Intervention: development/testing | 88 | 80 | 78 | 78 | 83 | 79 | 72 |
10 | How do we improve compliance and acceptability of iron supplementation programs among adolescents (e,g., design supplements with lower dose iron, different form such as powders, fewer side effects, etc)? | Intervention: discovery | 95 | 82 | 84 | 79 | 66 | 79 | 74 |
Nutrition (mean) | 87 | 86 | 84 | 80 | 80 | 83 | 76 | ||
Physical activity | |||||||||
1 | Considering comprehensive theoretical models and variables from different levels/systems/contexts (e.g., socioecological model), which variables predict, at an individual or population level, the different patterns of physical activity in adolescents living in LMICs? | Descriptive epidemiology | 91 | 100 | 100 | 100 | 100 | 100 | 98 |
2 | What is the best (feasibility, cost, acceptability, effectiveness, sustainability) design of a school-based intervention that aims to engage and gain the support of students, parents and teachers for young people to take the recommended 60 minutes of physical activity daily, and to ensure that there are at least two physical education (PE) classes within schools per week, with at least 50% of the time for PE classes spent in moderate-to-vigorous intensity physical activity? | Intervention: Discovery | 70 | 100 | 100 | 100 | 100 | 100 | 94 |
3 | What are the policy and/or environmental changes that influence physical activity among adolescents in LMICs? | Health policy, health and social systems | 100 | 100 | 100 | 94 | 94 | 98 | 96 |
4 | How best can the capacity of the education sector be improved to deliver high-quality physical education programs within schools? | Intervention: delivery/implementation | 89 | 100 | 100 | 94 | 94 | 98 | 93 |
5 | How does one best implement a sustainable, structured physical activity program for adolescents in schools and out of schools in LMICs? | Intervention: delivery/implementation | 78 | 100 | 100 | 100 | 89 | 98 | 93 |
6 | How best can parents, teachers, and policymakers be engaged in creating physical activity–friendly school environments for children and adolescents? | Intervention: delivery/implementation | 100 | 89 | 100 | 89 | 94 | 94 | 91 |
7 | What are scalable and sustainable approaches to improve physical activity in children and adolescents, particularly in low-to-middle income countries? | Intervention: delivery/implementation | 89 | 83 | 94 | 100 | 100 | 94 | 91 |
8 | How can effective local interventions for increasing physical activity in adolescents best be scaled up for national and cross-national implementation? | Intervention: delivery/implementation | 100 | 100 | 94 | 94 | 83 | 93 | 91 |
9 | What are the effects of daily physical education and recreation on total physical activity levels, physical fitness, cognitive development, and school performance among children and adolescents? | Intervention: development/testing | 90 | 95 | 95 | 90 | 90 | 93 | 86 |
10 | What are adolescents' preferences for type of physical activities by community and what is the relationship of this to their cultural background? | Intervention: delivery/implementation | 89 | 94 | 94 | 89 | 89 | 92 | 84 |
Physical activity (mean) | 90 | 96 | 98 | 95 | 93 | 96 | 92 | ||
Substance use | |||||||||
1 | What prevention and treatment services related to substance use are acceptable to adolescents? | Intervention: discovery | 95 | 95 | 100 | 100 | 95 | 98 | 95 |
2 | What are the risk factors contributing to adolescents' substance use in the different world regions? | Descriptive epidemiology | 73 | 86 | 91 | 86 | 91 | 89 | 78 |
3 | What is the effectiveness of implementation of youth friendly services interventions on substance use? | Intervention: delivery/implementation | 77 | 91 | 91 | 91 | 82 | 89 | 76 |
4 | What is the efficacy and effectiveness of a screening instrument linked to a brief intervention for alcohol use among adolescents for use in primary care settings? | Intervention: development/testing | 77 | 91 | 86 | 91 | 82 | 87 | 78 |
5 | Are there distinct patterns of and factors leading to substance use (tobacco and other substances) among in- and out-of-school female adolescents and male adolescents? (these include: context of use, preferred substance, and use related practices, among others) | Descriptive epidemiology | 64 | 91 | 91 | 77 | 82 | 86 | 75 |
6 | Is the use of electronic substitutes for nicotine delivery by adolescents a gateway or replacement mechanism for smoking? | Intervention: discovery | 91 | 86 | 91 | 91 | 73 | 86 | 80 |
7 | What are the most effective strategies for evaluating community-based interventions for reducing the burden of substance use among adolescents? | Intervention: discovery | 91 | 86 | 86 | 86 | 82 | 85 | 82 |
8 | How does consumption of alcohol and other substances among adolescents change over time when alcohol and other drug policies change? | Descriptive epidemiology | 85 | 85 | 85 | 90 | 80 | 85 | 78 |
9 | What is the effectiveness of programs assisting parents in the management of adolescents with substance use disorders? | Intervention: development/testing | 77 | 86 | 86 | 86 | 77 | 85 | 78 |
10 | What is the impact of peer education on reducing substance abuse in young people? | Intervention: delivery/implementation | 91 | 86 | 91 | 86 | 68 | 84 | 80 |
Substance abuse (mean) | 82 | 89 | 90 | 89 | 81 | 88 | 80 | ||
Adolescent health: policy, health and social systems | |||||||||
1 | What platforms and strategies are most effective to reach and help the most vulnerable adolescents (e.g., those not in school, slum dwellers, and/or those in poor families)? | Intervention: delivery/implementation | 90 | 91 | 90 | 87 | 93 | 90 | 85 |
2 | What are the most cost-effective interventions to decrease multiple health-risk behaviors and conditions and promote healthy behaviors? | Intervention: development/testing | 82 | 92 | 92 | 91 | 73 | 88 | 80 |
3 | How can primary health care services be designed to most effectively meet the unique health needs of adolescents? | Intervention: delivery/implementation | 84 | 93 | 91 | 82 | 82 | 88 | 80 |
4 | How can new technologies such as cell phones and the Internet be used effectively to provide information, referral and treatment for adolescents? | Intervention: delivery/implementation | 97 | 97 | 93 | 81 | 71 | 87 | 82 |
5 | What is the coverage of primary health care services for adolescents? | Descriptive epidemiology | 79 | 94 | 85 | 76 | 86 | 85 | 79 |
6 | How can we develop health systems to interact with adolescents in both traditional (in person) and innovative (virtual) ways to promote positive health choices and prevent illness? | Intervention: development/testing | 78 | 91 | 89 | 86 | 71 | 85 | 73 |
7 | How do adolescents use information technologies (e.g., web, traditional and social media), and what implications does this have for their health behavior, and for the design of interventions? | Descriptive epidemiology | 80 | 93 | 89 | 84 | 68 | 84 | 75 |
8 | What interventions can be used to facilitate continuity of care for mobile adolescent populations? | Intervention: delivery/implementation | 94 | 91 | 83 | 74 | 88 | 84 | 81 |
9 | What is the effectiveness of different models of provision of primary care by community health workers in settings that are accessible and acceptable to adolescents? | Intervention: delivery/implementation | 81 | 89 | 79 | 82 | 82 | 82 | 78 |
10 | What are the key interventions that should be part of routine school health service provision? | Intervention: development/testing | 83 | 87 | 86 | 81 | 69 | 82 | 73 |
Adolescent health: policy, health and social systems (mean) | 85 | 92 | 88 | 82 | 78 | 86 | 78 | ||
Overall (mean) | 86 | 90 | 89 | 84 | 80 | 87 | 79 |
ARV = Antiretroviral therapy; DALYs = disability-adjusted life years; HIC = high-income country; HIV = human immunodeficiency virus; LIC = low-income country; LMIC = lower-middle income country; LMICs = low- and middle-income countries; SES = socioeconomic status; TB = tuberculosis; UMIC = upper middle-income countries; USD = United States Dollar.
For the top 10 questions in each health area, the overall mean RPS was 87, and the mean AEA was 79. In terms of scoring criteria, answerability had the highest mean score (90), followed by impact (88) and clarity (88). Equity was the criterion that had the lowest mean score (81). The types of research represented in the top 10 questions for the eight health areas were descriptive epidemiology (26%), interventions: discovery (11%), development/testing (25%), delivery (33%), and policy, health and social systems (5%; Appendix D). The top 10–ranked research questions are shown by research type in Appendix E.
Across health areas, the top-ranking research questions highlighted various themes reflecting the diversity of issues affecting adolescent health. Several questions featured delivery of interventions via different platforms, such as schools (N = 14), primary care (N = 5), families/parents (N = 5), and interactive media (i.e., novel communication technologies, mobile phones, internet, social media; N = 4). Other questions addressed integration of health services, for instance between physical health, mental health, and reproductive health services. Finally, key subpopulations of vulnerable adolescents were identified in top-ranked research questions including young sex workers, injecting drug users, refugees, and out-of-school youth. Additional themes are discussed by specific health area.
Discussion by Health Area
Communicable diseases prevention and management
The top 10 communicable diseases prevention and management questions were dominated by tuberculosis (TB; 9 of 10), with six of nine of the TB questions also related to HIV co-infection or linking TB and HIV services (Table 3). Four of the questions were related to adherence, and three were related to retention in care. The only non-TB question that was in the top 10 communicable diseases questions was related to diarrhea and lower respiratory tract infections, whereas questions related to malaria or neglected tropical diseases were absent. This may have reflected the interests of the experts who proposed questions and scored them in this health area. For instance, the systematic searches of the literature for experts in malaria or neglected tropical diseases and adolescence yielded far fewer results than searches for experts in TB or HIV/AIDS and adolescence. Adolescence has been identified as a critical time in HIV and TB treatment and care, with recent studies demonstrating that HIV has risen to become the second-highest cause of adolescent mortality globally [1]. Nonetheless, research in malaria and neglected tropical diseases among adolescents may be a crucial under-represented research area despite the fact that they were not featured in this priorities exercise.
Injuries and violence
The top-ranked injuries and violence question was related to barriers and facilitators of motorcycle helmet legislation. Other specific issues addressed in the top 10 questions related to drowning, bullying, partner violence, sexual violence, and burn injuries. Three of the questions related to applying or combining interventions in one area to other areas (for instance, combining brief alcohol interventions with brief violence reduction interventions or using strategies against bullying to prevent partner violence or sexual violence).
Although one question related to gender-based violence was submitted, it did not rank in the top 10 injuries and violence questions in this exercise. Of note, the previous research priorities exercise on adolescent sexual and reproductive health included an entire area on gender-based violence. Eighteen experts on gender-based violence participated, and five priority questions were featured in the published results for that exercise. Some of these questions addressed underlying issues for gender-based violence. For instance, the top-ranked question was, “how do programs that aim to keep girls in school longer through measures such as conditional cash transfers affect the prevalence of gender-based violence?”
In addition, effective strategies for a responsive health system, empathetic provider behavior, and having a single point of access to multiple different types of care may serve to promote focus on improving service delivery for sexual violence, burns, and other injuries.
Mental health
The top-ranked mental health question addressed the cost-effectiveness of a package of interventions for the promotion of mental health. A question about the effectiveness of parenting programs in the prevention of mental health disorders also featured in the top 10. Three questions were related to integration of management of mental health with primary care or reproductive health care services and other strategies such as adolescent friendly health services. The third-ranked mental health question focused on suicide and self-harm behaviors in adolescent girls. Recent global reports have estimated that suicide surpassed maternal mortality as the leading cause of death among older adolescent (15–19 years old) females globally [1]. However, although ranked third among older adolescent (15–19 years old) males, the actual mortality rates were estimated to be almost identical (11.73/100,000/year in females vs. 11.72/100,000/year in males) in 2012 [1]. Research on self-harm and suicide in adolescent males also remains an under-researched area.
Noncommunicable diseases management
The top-ranked question on noncommunicable diseases (NCDs) management related to developing a low-cost rapid antigen test for streptococcal pharyngitis for the prevention of rheumatic heart disease. This was the only intervention: discovery question that ranked number one in a health area. In addition to rheumatic heart disease, research on other forms of heart conditions may be important during adolescence. For instance, for children born with congenital heart diseases who survive through childhood, loss to follow-up and transitions to adult medical care remain continuing health challenges. Furthermore, acquired heart-related conditions such as hypertension, high cholesterol, and coronary heart disease may begin to develop in adolescence and are an emerging research area. Four of the 10 top-ranked questions were related to diabetes, whereas other specific diseases included rheumatic heart disease and sickle cell disease. Although no questions related to cancers affecting adolescents featured in this exercise, this remains an important area of research in LMICs [18].
Two of the top 10 questions were related to applying existing interventions in one population group to another, for instance applying NCDs management interventions in adults to adolescents or applying NCDs management interventions in high-income settings to low-income settings. Of note, the NCDs management section did not include NCDs prevention because many of these preventive behaviors may be covered by the substance use, nutrition or physical activities areas. However, there are other areas of behaviors including sleep patterns, increased screen time with electronics and social media, social pressures, and stress related to studies, work, or earnings that may contribute to the development of NCDs and other health problems.
Nutrition
The top-ranked nutrition question related to the causes of adolescent anemia and how the causes vary by geographical region. Four of the top five nutrition questions were descriptive epidemiology research questions. Two of the 10 top-ranked questions related to the relationship between overnutrition and undernutrition; three related to differences in nutritional risk factors or problems by region, country, or socio-economic status; and two related to nutritional status or support for pregnant adolescent girls.
Physical activity
Identifying variables that predicted physical activity patterns among adolescents in LMIC was the top-scoring physical activity question. Five of the 10 top-ranked questions related to schooling or school-based physical activity interventions, whereas two related to scaling up physical activity interventions. Overall, most physical activity questions related to interventions, and particularly, their development/testing or implementation/delivery.
Substance use
Three of the top 10 research questions in this health area were intervention: discovery questions, more than in any other health area. The top-ranked question on substance use was related to the most acceptable prevention and treatment services to adolescents. Two questions specifically addressed alcohol and tobacco, respectively. Three of the questions were related to community-based, parent-based, or peer-based interventions.
Adolescent health: policy, health and social systems
The top-ranked question for adolescent health: policy, health and social systems related to platforms to reach the most vulnerable adolescents. To reach the most vulnerable adolescents, research on their health status and needs may also be necessary. Three of the top questions related to primary care, including effectiveness of different models and coverage, and three related to information and communication technology, whether mobile health interventions, the internet, or social media.
General Discussion
Priority questions have been identified for research in eight key areas of adolescent health in LMICs through 2030, extending earlier work that proposed research priorities in adolescent sexual and reproductive health. Using a modified version of the priority setting method developed by the CHNRI, we received input from 142 experts who generated 512 research questions. These expert-generated questions may be used by donors, program managers, and researchers to stimulate and develop research in adolescent health.
A limitation of the exercise is possible nonresponse bias given that not all experts agreed to participate in the exercise (217/450 = 48%) and not all those who agreed to participate actually did so. Only 142/217 (65%) submitted questions and 130/217 (60%) scored questions. This was despite efforts to encourage responses from all. Selection bias may also affect results, as a majority of respondents were researchers from academic institutions, with less representation from program implementers, policy makers, and funders. This potential bias may reflect the lack of questions on estimating numbers for harmful practices among adolescents. Although equity had the lowest overall average score relative to the other criteria among the top questions, the mean was still 80/100. This value was consistent with previous research priority exercises with mean equity scores 84–86 [12], [13], [19]. The creation of eight health areas, which were based on burden of adolescent mortality and morbidity as well as a life course approach to health, inevitably leads to some degree of merging and separation of topics. Furthermore, some specific disease areas may have stronger representation than others based on identified experts and their response rates, despite an effort to include a breadth of expertise by topic. The use of PubMed and Web of Science databases may have identified experts who tended to publish in English language scholarly journals although we searched for publications and experts in all languages and from all regions using the WHO website and WHO Index Medicus, a database focused on health literature produced by and within LMICs.
Furthermore, some of the questions that were submitted spanned two or more types of research question (e.g., both development and delivery types of intervention question) but, for ease of scoring and analysis, were categorized as the category that they were submitted in.
Strengths of the exercise included identification of a large number (450) of experts in adolescent health spanning a diverse range of health areas. The CHNRI methodology is a systematic and transparent process that has become the most common methodology for identifying research priorities since 2001 [20]. It uses independent scoring by experts, avoiding situations where the most vocal or opinionated individuals affect group decisions or priorities [20]. The range of AEA in this exercise (61–98) was consistent with, or higher than, previous research priority exercises, such as for preterm birth (62–83) [13], childhood pneumonia (64–76) [19], and newborn health and prevention of stillbirths (62–77) [12]. Although questions were organized in vertical health areas, during the analysis questions were also classified horizontally by delivery platforms (i.e., primary care, schools, families/parents, and interactive media). Future research may consider qualitative methodology exploring themes of top questions across all adolescent health areas (including questions from the adolescent sexual and reproductive health priorities exercise).
Adolescent health is receiving increasing attention globally. For example, the updated United Nations' Global Strategy for Women's, Children's, and Adolescents' Health 2016–2030 features adolescents for the first time, and the emphasis on going beyond “survive” to “thrive” and “transform” will greatly increase the focus on adolescent health and development [7]. Furthermore, although only one of the 17 Sustainable Development Goals is specific to health, all 17 will directly or indirectly affect adolescent health [6]. Given the need for evidence-based policies and programs to improve adolescent health as part of these new global initiatives, priority questions for research in eight key areas of adolescent health in LMICs have been identified using a transparent process that included experts from multiple disciplines, types of institutions, and countries. These expert-generated questions may be used by donors, program managers, and researchers to stimulate and develop research in adolescent health.
Acknowledgments
The authors thank the 142 experts who actively participated in the exercise by submitting and/or scoring questions; the participants at the WHO Consultation on Adolescent Health Research Priorities who included Drs. Sulafa Ali, Margit Averdjik, Anne Buvé, Bruce Dick, Aoife Doyle, Adesegun Fatusi, Rashida Ferrand, Gwyn Hainsworth, Daniel Hale, Mark Jordans, Ana Menezes, Mahmood Nazar Mohamed, Vikram Patel, Daniel Tobon Garcia, and Daniel Wight; WHO colleagues including Drs. Annabel Baddeley, Valentina Baltag, Paul Bloem, Raschida Bouhouch, Alexander Manu, Nigel Rollins, Chiara Servili; and Claire Ory-Scharer and Margaret Kigundu for administrative support. The authors particularly thank Drs. Sachiyo Yoshida, Michelle Hindin, and Rajiv Bahl for advice related to CHNRI methodology, Joya Banerjee and Jill Kowalchuk for help with the systematic searches, and Tomas Allen for advice on electronic literature database searches.
Footnotes
Conflicts of Interest: The authors have no conflicts of interest or financial disclosures to report.
Disclaimer: The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of WHO.
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jadohealth.2016.03.016.
Funding Sources
Partial funding for this exercise was provided by the US Agency for International Development and the Mary Duke Biddle Clinical Scholars Program, Stanford University (JMN).
Supplementary Data
References
- 1.World Health Organization . World Health Organization Press; Geneva, Switzerland: 2014. Health for the world's adolescents: A second chance in the second decade. [Google Scholar]
- 2.Patton G.C., Coffey C., Sawyer S.M. Global patterns of mortality in young people: A systematic analysis of population health data. Lancet. 2009;374:881–892. doi: 10.1016/S0140-6736(09)60741-8. [DOI] [PubMed] [Google Scholar]
- 3.Viner R.M., Coffey C., Mathers C. 50-Year mortality trends in children and young people: A study of 50 low-income, middle-income, and high-income countries. Lancet. 2011;377:1162–1174. doi: 10.1016/S0140-6736(11)60106-2. [DOI] [PubMed] [Google Scholar]
- 4.Gore F.M., Bloem P.J., Patton G.C. Global burden of disease in young people aged 10-24 years: a systematic analysis. Lancet. 2011;377:2093–2102. doi: 10.1016/S0140-6736(11)60512-6. [DOI] [PubMed] [Google Scholar]
- 5.Patton G.C., Viner R.M., Linh le C. Mapping a global agenda for adolescent health. J Adolesc Health. 2010;47:427–432. doi: 10.1016/j.jadohealth.2010.08.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.United Nations . United Nations Press; New York, NY: 2015. Transforming our world: The 2030 Agenda for sustainable development. [Google Scholar]
- 7.United Nations . United Nations Press; New York, NY: 2015. Every Woman, Every child. Global strategy for Women's, Children's, and adolescents' health (2016-2030): Survive, thrive, transform. [Google Scholar]
- 8.Patton G.C., Coffey C., Cappa C. Health of the world's adolescents: A synthesis of internationally comparable data. Lancet. 2012;379:1665–1675. doi: 10.1016/S0140-6736(12)60203-7. [DOI] [PubMed] [Google Scholar]
- 9.Hindin M.J., Christiansen C.S., Ferguson B.J. Setting research priorities for adolescent sexual and reproductive health in low- and middle-income countries. Bull World Health Organ. 2013;91:10–18. doi: 10.2471/BLT.12.107565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Rudan I., Gibson J.L., Ameratunga S. Setting priorities in global child health research investments: Guidelines for implementation of CHNRI method. Croat Med J. 2008;49:720–733. doi: 10.3325/cmj.2008.49.720. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bahl R., Martines J., Ali N. Research priorities to reduce global mortality from newborn infections by 2015. Pediatr Infect Dis J. 2009;28(1 Suppl.):S43–S48. doi: 10.1097/INF.0b013e31819588d7. [DOI] [PubMed] [Google Scholar]
- 12.Yoshida S., Martines J., Lawn J.E. Setting research priorities to improve global newborn health and prevent stillbirths by 2025. J Glob Health. 2016;6:010508. doi: 10.7189/jogh.06.010508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bahl R., Martines J., Bhandari N. Setting research priorities to reduce global mortality from preterm birth and low birth weight by 2015. J Glob Health. 2012;2:010403. doi: 10.7189/jogh.02-010403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lawn J.E., Bahl R., Bergstrom S. Setting research priorities to reduce almost one million deaths from birth asphyxia by 2015. PLoS Med. 2011;8:e1000389. doi: 10.1371/journal.pmed.1000389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Fontaine O., Kosek M., Bhatnagar S. Setting research priorities to reduce global mortality from childhood diarrhoea by 2015. PLoS Med. 2009;6:e41. doi: 10.1371/journal.pmed.1000041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Thomson Reuters. 2015 journal Citation Reports® Science edition 2013.
- 17.Kapiriri L., Tomlinson M., Chopra M. Setting priorities in global child health research investments: Addressing values of stakeholders. Croat Med J. 2007;48:618–627. [PMC free article] [PubMed] [Google Scholar]
- 18.Bleyer A., Barr R., Hayes-Lattin B. The distinctive biology of cancer in adolescents and young adults. Nat Rev Cancer. 2008;8:288–298. doi: 10.1038/nrc2349. [DOI] [PubMed] [Google Scholar]
- 19.Rudan I., El Arifeen S., Bhutta Z.A. Setting research priorities to reduce global mortality from childhood pneumonia by 2015. PLoS Med. 2011;8:e1001099. doi: 10.1371/journal.pmed.1001099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health. 2016;6:010507. doi: 10.7189/jogh.06.010507. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.