Setting priorities for global mental health research

Abstract

Objective

To set investment priorities in global mental health research and to propose a more rational use of funds in this under-resourced and under-investigated area.

Methods

Members of the Lancet Mental Health Group systematically listed and scored research investment options on four broad classes of disorders: schizophrenia and other major psychotic disorders, major depressive disorder and other common mental disorders, alcohol abuse and other substance abuse disorders, and the broad class of child and adolescent mental disorders. Using the priority-setting approach of the Child Health and Nutrition Research Initiative, the group listed various research questions and evaluated them using the criteria of answerability, effectiveness, deliverability, equity and potential impact on persisting burden of mental health disorders. Scores were then weighted according to the system of values expressed by a larger group of stakeholders.

Findings

The research questions that scored highest were related to health policy and systems research, where and how to deliver existing cost-effective interventions in a low-resource context, and epidemiological research on the broad categories of child and adolescent mental disorders or those pertaining to alcohol and drug abuse questions. The questions that scored lowest related to the development of new interventions and new drugs or pharmacological agents, vaccines or other technologies.

Conclusion

In the context of global mental health and with a time frame of the next 10 years, it would be best to fill critical knowledge gaps by investing in research into health policy and systems, epidemiology and improved delivery of cost-effective interventions.

Introduction

About 14% of the global burden of disease is attributable to mental disorders.¹ Even in sub-Saharan Africa, where communicable diseases are common, mental disorders account for nearly 10% of the total burden of disease.² Mental disorders are linked to many other health conditions¹ and are among the most costly medical disorders to treat.³

Certain treatment and preventive strategies for mental disorders are known to be effective (even in low- and middle income-countries), particularly those for depressive and anxiety disorders and schizophrenia.^4,5 However, health systems around the world face a scarcity of financial resources and qualified staff – a situation that is often compounded in low- and middle-income countries by lack of commitment from public health policy makers and inefficient use of resources.⁶ As a result, measures known to be effective for dealing with mental disorders are often not implemented.

The Global Forum for Health Research has long highlighted the major imbalance between the magnitude of mental health problems (especially in low- and middle-income countries) and the resources devoted to addressing them. This is the so-called “10/90 gap”; that is, only 10% of global spending on health research is directed towards the problems that primarily affect the poorest 90% of the world’s population.⁷ This gap stunts the development of evidence-based health policies and practice in low- and middle-income countries and limits progress in medicine and public health.^8,9 The impact of the gap is particularly evident in the field of mental health, in which the evidence base depends mainly on European and North American cultural norms.¹⁰ Recent studies indicate that up to 94% of the published literature in high-impact psychiatric journals is from North America, Europe and Australia/New Zealand,¹¹ with sometimes as little as 3% originating from low- and middle-income countries.^11–13

One strategy to redress this imbalance is to invest abundantly in mental health research in low- and middle-income countries. The many possibilities for mental health research in such countries are beyond the capability of any one government or agency to fund; therefore, guidelines are needed to help define priorities for mental health research investments. Since 1990, several initiatives have been designed to set such priorities.^14–18 They include the “combined approach matrix” priority-setting tool for health research, which was applied to schizophrenia;¹⁷ a 2001 report from the United States National Institute of Mental Health, which outlined three priority areas of research in child and adolescent mental health;¹⁹ and Rosenheck’s seven principles for resource allocation in the mental health field.^14–20

These attempts to set investment priorities have some limitations: a focus on the generation of new technologies, knowledge and processes, rather than on the implementation of already proven interventions;¹⁸ insufficient transparency around the processes and criteria used to derive the suggested investment priorities;^19,20 and lack of a clear algorithm and method for linking suggested priorities with future investment decisions.¹⁷ The objective of this paper is to address investment priorities in mental health research at the global level and to propose a more rational use of scarce funds in this area by using a systematic method for setting priorities in health research investments recently developed by the Child Health Research Nutrition Initiative (CHNRI).^18,21 The method can be applied in different contexts and for different purposes. It has been used to set priorities for zinc-related health research²² and child health priorities at the national level in South Africa.²³ It is also currently being implemented by WHO to set global research priorities for eight leading causes of child deaths, and by the International Committee on Child Development to address research priorities to improve child development (Rudan I, personal communication).

Methods

Expert group and context

Fig. 1 summarizes the steps involved in applying the CHNRI method. The rationale, conceptual framework and application guidelines for the method have been described in detail elsewhere.^18,21–25 In applying the CHNRI method in this study, the first step was to ask a group of leading experts in mental health (n = 39) – the Lancet Mental Health Group – to form a technical working group (Step 1, Fig. 1). The group comprised mainly psychiatrists (74%) but also included psychologists, epidemiologists, an economist, a primary care physician and an anthropologist. It was largely composed of males (77%), and 46% of the members were from low- or middle-income countries.

Fig. 1.

Process for setting research priorities

^a Text in parentheses indicates the selections made in this study.

List of research options

Members of the technical working group were then asked to generate a list of research questions by research domain (Step 2, Fig. 1). They proposed a total of 290 questions (not all group members provided questions for all disorders). Many of the questions were either identical or sufficiently similar to allow them to be combined, and three of the authors (MT, VP and SS) synthesized them into a final list of 55 questions. Research investment options were then scored according to the five criteria recommended by the CHNRI to discriminate between suggested research investment options (Step 3, Fig. 1).^18,25 Scoring, which was voluntary and took place over a relatively short period, was eventually performed by 24 members (61%) of the working group (other eligible members were unable to complete scoring due to time constraints). The experts who completed scoring had a similar profile (71% psychiatrists, 83% of them male and 38% from low- and middle-income countries) to that of the original larger group.

The experts answered three questions related to each criterion (Box 1), in line with the conceptual CHNRI framework suggested by Rudan et al.²⁵ In this way, the proposed research investment options received five “intermediate scores” (one for each criterion), ranging from 0% to 100%. These values represented a robust measure of the collective view of the experts that the option would satisfy the chosen criterion.

Box 1. Questions used by technical experts to assign intermediate scores to competing research questions^a.

Criterion 1: likelihood that research would lead to new knowledge (enabling development or planning of an intervention) in an ethical way

1. Would you say the research question is well framed and end-points are well defined?

2. Based on the level of existing research capacity in proposed research and the size of the gap from current level of knowledge to the proposed end-points, would you say that a study can be designed to answer the research question and to reach the proposed end-points of the research?

3. Do you think that a study needed to answer the proposed research question would obtain ethical approval without major concerns?

Criterion 2: assessment of likelihood that the intervention resulting from proposed research would be effective

1. Based on the best existing evidence and knowledge, would the intervention that would be developed or improved through the proposed research be efficacious?

2. Based on the best existing evidence and knowledge, would the intervention that would be developed or improved through proposed research be effective?

3. If the answer to either of the previous two questions is positive, would you say that the evidence on which these opinions are based is of high quality?

Criterion 3: assessment of deliverability, affordability and sustainability of the intervention resulting from proposed research

1. Taking into account the level of difficulty with intervention delivery from the perspective of the intervention itself (e.g. design, standardizability, safety), the infrastructure required (e.g. human resources, health facilities, communication and transport infrastructure) and users of the intervention (e.g. need for change of attitudes or beliefs, supervision, existing demand), would you say that the end-points of the research would be deliverable within the context of interest?

2. Taking into account the resources available to implement the intervention, would you say that the end-points of the research would be affordable within the context of interest?

3. Taking into account government capacity and partnership requirements (e.g. adequacy of government regulation, monitoring and enforcement; governmental intersectoral coordination, partnership with civil society and external donor agencies; favourable political climate to achieve high coverage), would you say that the end-points of the research would be sustainable within the context of interest?

Criterion 4: assessment of maximum potential of disease burden reduction

As this dimension is considered “independent” of the others, to score competing options fairly, their maximum potential to reduce disease burden should be assessed as their potential impact fraction under an ideal scenario; that is, when the exposure to targeted disease risk is decreased to 0% or coverage of proposed intervention is increased to 100% (regardless of how realistic that scenario is at the moment – that aspect will be captured by other dimensions of the priority setting process, such as deliverability, affordability and sustainability).

For potential interventions:

Maximum potential to reduce disease burden should be computed as “potential impact fraction” for each proposed research avenue, using the equation:

PIF = [Σ_{(i = 1 to n)} P_i (RR_i − 1)] / [Σ_{(i = 1 to n)} P_i (RR_i − 1) + 1]

where PIF is the potential impact fraction, i.e. the potential to reduce disease burden through reducing risk exposure in the population from the present level to 0% or increasing coverage by an existing or new intervention from the present level to 100%; RR is the relative risk, given the exposure level (< 1.0 for interventions, > 1.0 for risks), P is the population level of distribution of exposure, and n is the maximum exposure level.

For existing interventions:

Maximum potential to reduce disease burden should be assessed from the results of conducted intervention trials; if no such trials have been undertaken, then it should be assessed as for non-existing interventions, above.

The following questions should then be answered:

1. Taking into account the results of conducted intervention trials (i.e. existing interventions) or, for the new interventions, the proportion of avertable burden under an ideal scenario (i.e. potential interventions), would you say that the successful reaching of research end-points would have a capacity to remove 5% of disease burden or more?

2. To remove 10% of disease burden or more?

3. To remove 15% of disease burden or more?

Criterion 5: assessment of the impact of proposed health research on equity

1. Would you say that the present distribution of the disease burden affects mainly the underprivileged in the population?

2. Would you say that either mainly the underprivileged or all segments of the society equally would be the most likely to benefit from the results of the proposed research after its implementation?

3. Would you say that the proposed research has the overall potential to improve equity in disease burden distribution in the long term (e.g. 10 years)?

Reference group

To ensure involvement of the wider society in directing research investment priorities, we collected opinions from a larger reference group, comprising 43 stakeholders’ representatives, including those from low- and middle-income countries. The process of involving stakeholders in the CHNRI process has been detailed elsewhere.²⁴ The reference group comprised nine psychiatrists, four psychologists, two social workers, three government employees, six nongovernmental organization representatives, six researchers, six users of mental health services and seven members of the public. We contacted members of the group by e-mail and asked them to express their opinions through an electronic questionnaire that described the elements of the process. We asked them to rank the five criteria used for setting priorities. The criterion for effectiveness received the highest rank (2.47), followed by maximum potential for disease burden reduction (2.56), deliverability (3.00), predicted effect on equity (3.28) and answerability (3.70). The next step was to define weights for the five scores for each option (Step 4, Fig. 1). These observed average ranks were then used to compute weights by dividing the expected average rank in the hypothetical situation of all five criteria being equally important (which should be 3.00) by the observed average rank. These weights were subsequently applied to compute intermediate scores. Thus, for each scored research investment option, the intermediate score for effectiveness was increased by 21% (i.e. 3.00/2.47 = 121%) and for maximum potential for disease burden reduction, by 17%; the score for deliverability did not change; and the intermediate scores for equity were decreased by 9% and for answerability, by 19%. The weighted mean of the five intermediate scores represented the overall “research priority score”, which also ranged from 0 to 100%.

Computation and assessment

It was not appropriate to use Kappa statistics to assess greatest agreement and greatest controversy because the data sets produced allowed for missing responses, “undecided” responses and different number of experts scoring different criteria.²⁶ Instead, for each research investment option, we reported the average proportion of scorers that agreed on the 15 questions asked. This average expert agreement (AEA) was computed for each scored research investment option as:

where q is a question that experts are being asked in order to rank competing research investment options from 1 to 15.

For each evaluated research investment option, the AEA shows the proportion of scorers who gave the same most frequent answer to an average question (e.g. when the average expert agreement is about 60%, this means that for an average question related to a specific research investment option, three out of five scorers gave the most frequent answer).

Results

Table 1 and Table 2 show the final results (10 highest and 10 lowest priorities) of the scoring process used by the technical experts, broken down by type of disorder. Appendix A (available at: http://academic.sun.ac.za/psychology/english/TomlinsonM.htm) shows the final scores of all proposed research options. The five that scored highest all addressed either health policy and systems research options, epidemiological research or research to inform priority setting. The only exception in the 10 highest-scoring research priority options was the one ranked ninth, which proposed the development of a new intervention. All of the five highest-scoring research options addressed either alcohol and drug abuse or child and adolescent mental disorders.

Table 1. Ten mental health research questions given highest priority by members of the Lancet Mental Health Group, 2008^a.

Rank	Research question			Percentage
				Disorder	Answerable?	Effective?	Deliverable?	Maximum impact?	Equitable?	Weighted score	Agreement
1		What HPSR is needed to determine the most effective intersectoral (social, economic and population-based) strategies for reducing consumption in high-risk groups (particularly men), thus reducing the burden of alcohol abuse?		A/DA	90.8	88.8	85.8	81.0	73.3	85.9	76.0
2		What training, support and supervision will enable existing maternal and child health workers to recognize, and provide basic treatment for, common maternal, child and adolescent mental disorders?		C/AMD	97.2	80.8	89.8	59.8	87.0	83.3	78.5
3		What are the effectiveness and cost effectiveness of school-based interventions, including for children with special needs?		C/AMD	95.2	76.5	91.2	59.2	87.3	82.0	78.6
4		What HSPR is needed to estimate the effects of integrating management of child and adolescent mental disorders with that of other child and adolescent physical diseases, including poor nutrition?		C/AMD	94.4	75.5	84.3	66.3	84.3	81.4	73.8
5		How effective are early detection and simple, brief treatment methods that are culturally appropriate, implemented by non-specialist health workers in the course of routine primary care, and can be scaled up?		A/DA	93.3	78.4	88.3	59.6	83.3	81.1	78.5
6		What is the cost effectiveness of trials of interventions for CMD in primary and secondary care?		CMD	95.5	87.7	79.4	60.0	74.2	80.3	73.4
7		What HSPR is needed to develop effective and cost-effective methods for delivering family interventions in low-resource settings to decrease relapses?		S/PSYCH	96.8	83.9	79.4	61.7	74.2	80.0	77.9
8		What HSPR is needed to develop feasible, effective and cost-effective ways of integrating parenting interventions and social skills in early childhood care?		C/AMD	81.5	81.5	79.6	69.2	80.0	79.8	74.3
9		How effective are new, culturally appropriate community-level interventions (e.g. family therapy) for child and adolescent mental disorders (including mental retardation and epilepsy)?		C/AMD	97.2	77.6	87.0	56.3	79.2	79.7	75.3
10		What is the effectiveness of dispensing antipsychotic medication by general community health workers, to reduce relapse and admission rates in people with psychoses?		S/PSYCH	92.9	72.6	80.3	66.7	80.6	79.1	73.9

A/DA, alcohol and drug abuse; C/AMD, child and adolescent mental disorders; CMD, depression and common mental disorders; HPSR, health policy and systems research; S/PSYCH, schizophrenia and other psychotic disorders.
^a All the listed research options were scored by technical experts in light of five criteria: (1) their potential to generate new knowledge in an ethical way; (2) the likelihood that the intervention resulting from them would be effective; (3) the deliverability, affordability and sustainability of the intervention resulting from them; (4) the resulting intervention’s maximum potential for reducing the burden of disease; and (5) their potential effect on equity in the population.

Table 2. Ten research questions given lowest priority by members of the Lancet Mental Health Group, 2008^a.

Rank	Research question			Percentage
				Disorder	Answerable?	Effective?	Deliverable?	Maximum impact?	Equitable?	Weighted score	Agreement
46		How effective are new, antipsychotic drugs and delivery methods for improved action and fewer side-effects?		S/PSYCH	81.7	70.5	31.7	49.2	53.3	58.0	66.8
47		How effective are new techniques for early detection and management of schizophrenia?		S/PSYCH	77.8	52.7	42.5	46.5	63.9	56.5	59.0
48		What new, innovative and appropriate interventions can be designed to reduce poor mental health outcomes in at-risk children?		C/AMD	64.8	41.3	55.9	46.1	70.8	55.4	55.0
49		What are the benefits of a simpler classification of depressive and anxiety disorders in practice?		CMD	72.1	52.3	58.3	37.9	55.4	55.0	53.5
50		How effective are new drugs for the prevention and treatment of child and adolescent mental disorders such as psychosis?		C/AMD	71.2	43.9	44.8	58.2	56.0	55.0	58.0
51		How can large-scale, efficient and ethically sound drug discovery research be carried out in LAMI countries?		S/PSYCH	61.9	56.9	44.2	47.5	56.6	54.2	52.9
52		How effective are inexpensive, naturally occurring or pharmacological agents that make alcohol intake very unpleasant? (E.g. some antidepressants are used for smoking cessation)		A/DA	78.4	50.0	55.3	27.2	64.9	54.1	59.4
53		What are the benefits of new and innovative promotion and prevention programmes?		CMD	60.8	40.0	55.8	50.8	49.2	51.6	54.1
54		How effective are new, innovative interventions for alcohol and drug abuse that target biological vulnerability or predisposition, and the interplay between genetic and epigenetic mechanisms in causation and recovery?		A/DA	62.1	47.1	42.2	49.0	50.9	50.7	52.9
55		What are the benefits of researching indigenous or local treatments (including non-traditional approaches such as acupuncture and herbal remedies) as potential treatments?		S/PSYCH	61.9	27.4	51.8	18.4	59.5	42.2	59.6

A/DA, alcohol and drug abuse; C/AMD, child and adolescent mental disorders; CMD, depression and common mental disorders; HPSR, health policy and systems research; LAMI, low- and middle-income; S/PSYCH, schizophrenia and other psychotic disorders.
^a All the listed research options were scored by technical experts in light of five criteria: (1) their potential to generate new knowledge in an ethical way; (2) the likelihood that the intervention resulting from them would be effective; (3) the deliverability, affordability and sustainability of the intervention resulting from them; (4) the resulting intervention’s maximum potential for reducing the burden of disease; and (5) their potential effect on equity in the population.

Of the 10 lowest-scoring research options, seven proposed developing new interventions and technologies (Table 2). Several of the options also targeted the development of new interventions and new drugs or pharmacological agents, vaccines or other technologies. Of the top 10 options, the only one that addressed pharmacological agents (Evaluate the effectiveness of dispensation of anti-psychotic medication by general community health workers in order to reduce relapse and admission rates) was actually concerned with health systems and epidemiological research.

When the scores were broken down by type of disorder, a similar pattern emerged, with health systems and epidemiological research predominating in the highest-scoring research options for each disorder. New interventions that scored highly were those predominantly focused on community, social, behavioural and prevention strategies.

The priorities for the competing investment options varied widely – research priority scores ranged from 42.2% to 85.9%. Scores for answerability for the 55 research options were relatively high (the 47 highest priority options all scored above 80%). However, other criteria helped to lower the overall scores. For example, while the research option “to develop new, more efficacious antipsychotic drugs” scored 81.7% on answerability, it scored poorly on deliverability (31.7%). This illustrates the fundamental problem of health system delivery that is characteristic of much of the developing world. The criterion of reducing the burden of disease contributed to low overall scores. Apparently the technical working group felt that, although such a question may be answerable, it is unlikely to have a strong impact in reducing disease.

For the 10 highest priority research investment options, the average expert agreement parameter was 73.4–78.6%. In other words, about three out of four scorers gave the same answer to an average question related to those options. This level of agreement is much higher than expected from random assignment of scores 0 or 1 (less than 50% because an “undecided” answer would also be allowed). This shows that the experts agreed on the priorities overall but not on the research investment options at the bottom of the ranking list, for which the average expert agreement parameter was generally 50–60%.

Discussion

To significantly reduce the burden of disease caused by the four priority categories of mental disorders in low- and middle-income countries within the next 10 years, research funding should focus on three areas: health policy and systems research; where and how to deliver existing cost-effective interventions in a low-resource context; and epidemiological research on the broad categories of child and adolescent mental disorders or those pertaining to alcohol and drug abuse. Epidemiological research is important because of the lack of policy-relevant information in the developing world.²⁷

Priority-setting exercises in the mental health field have shown that, for schizophrenia, further research is needed on its burden to families, the cost effectiveness of therapeutic interventions, ways to bridge the treatment gap, and ways to overcome stigmatization and social isolation.¹⁷ The 2001 report from the US National Institute of Mental Health outlined three priority areas of research in child and adolescent mental health: developing new interventions, moving from efficacy to effectiveness in intervention development, and moving from effectiveness to dissemination in intervention deployment.¹⁹ These exercises all resulted in broad, general recommendations. Although few would disagree with their overall messages, they rarely provide specific guidance on how to distribute resources for health research. In contrast, the CHNRI approach generates specific outcomes and suggestions, since it lists concrete research options and assigns them priority scores. The scores provide information about the risk associated with each specific investment in health research, under the assumption that reducing the disease burden is the main expected end-point. Policy makers can then invest in a mixed portfolio of health research investments with a variety of risk levels while respecting the values of wider society.

A recent review highlighted the scarcity of trials testing interventions for the treatment or prevention of mental disorders in low- and middle-income countries,⁵ especially in the areas of child and adolescent mental disorders and of alcohol and substance abuse disorders. As an example, the review uncovered only 11 trials on alcohol dependence or harmful use and only five trials on mental retardation. In addition, despite the large and increasing burden of substance use disorders in developing countries, 34% of all low- and middle-income countries lack a substance abuse policy.²⁷ According to Rajendram et al., nearly 58% of all research papers on alcohol abuse come from Canada and the United States of America, 30% from Western Europe, and 10% from Australia, Japan, or New Zealand, while the rest of the world, which has 87% of the disease burden, contributes only 8%.²⁸ Furthermore, a recent review showed that of all psychiatric disorders, alcohol and drug abuse had the widest treatment gap (78.1%).

The results of the prioritization exercise described in this paper, in which the top seven research options related to alcohol and substance abuse and to child and adolescent mental disorders, perhaps reflect how these themes have been ignored. They provide evidence of the need to implement health systems research in mental health and carry out epidemiological research in low- and middle-income countries.

The predominance of research on new interventions, particularly against drug-related problems, among the low-priority options is consistent with findings from other exercises based on the CHNRI approach.^22,23 In these studies, research on health policy and systems scored highly because of its perceived ability to achieve the largest equitable gains in reducing disease burden over a reasonably short time frame, a criterion specified before the scoring was undertaken. This finding contrasts with trends in the allocation of most research funding and with the United States National Institute of Mental Health’s stated main priority for research into child and adolescent mental health, which is the development of new interventions.¹⁹ To be successful, even new, highly effective pharmacological treatments need well-functioning health systems to deliver them and psychosocial interventions to accompany them.²³

The expert working group listed few research questions that addressed primary prevention, perhaps due to the difficulty of framing research questions on prevention in a form that could be scored against the five chosen criteria. Also, where such questions were listed, they were not given high priority, perhaps because the context within which scoring was taking place was initially specified as “overall global burden, with the improvements expected over a time frame of the next 10 years”. The CHNRI process highlighted the experts’ general lack of optimism that preventive interventions could be effective or make a real difference to mental health globally.

It is possible that the experts in the working group were systematically more biased against preventive investment options or new interventions than against other categories. However, the group chosen for this exercise was the largest and the most diverse ever to conduct a CHNRI exercise to date. It is thus unlikely that a different group of experts would have arrived at quite different results. Nevertheless, the CHNRI approach could be subject to expert opinion bias because different groups may have different opinions. An advantage of the CHNRI method over previous priority-setting reports^14–18 is that potential biases are made transparent in the scoring process. Furthermore, the larger and more diverse the group of chosen experts, the lower the likelihood that scores would significantly deviate from those assigned by a different group of experts. Another limitation of this study was that the research topics were chosen by a self-selected group of professionals – albeit a diverse and senior one – and that the choice of four disorders is not exhaustive, although it does represent a significant part of the disease burden of mental disorders worldwide.

Despite the limitations, this study clearly shows a need to invest in research on the implementation of existing interventions and ways to overcome health system constraints in developing countries. ■