Using Economic Evidence to Set Healthcare Priorities in Low‐Income and Lower‐Middle‐Income Countries: A Systematic Review of Methodological Frameworks

Abstract

Policy makers in low‐income and lower‐middle‐income countries (LMICs) are increasingly looking to develop ‘evidence‐based’ frameworks for identifying priority health interventions. This paper synthesises and appraises the literature on methodological frameworks – which incorporate economic evaluation evidence – for the purpose of setting healthcare priorities in LMICs. A systematic search of Embase, MEDLINE, Econlit and PubMed identified 3968 articles with a further 21 articles identified through manual searching. A total of 36 papers were eligible for inclusion. These covered a wide range of health interventions with only two studies including health systems strengthening interventions related to financing, governance and human resources. A little under half of the studies (39%) included multiple criteria for priority setting, most commonly equity, feasibility and disease severity. Most studies (91%) specified a measure of ‘efficiency’ defined as cost per disability‐adjusted life year averted. Ranking of health interventions using multi‐criteria decision analysis and generalised cost‐effectiveness were the most common frameworks for identifying priority health interventions. Approximately a third of studies discussed the affordability of priority interventions. Only one study identified priority areas for the release or redeployment of resources. The paper concludes by highlighting the need for local capacity to conduct evaluations (including economic analysis) and empowerment of local decision‐makers to act on this evidence.

1. Introduction

Policy makers are increasingly looking to develop ‘evidence‐based’ priority‐setting frameworks for the health sector that incorporate value for money criteria. This is especially important in low‐income and lower‐middle‐income countries (LMICs) where resources are highly constrained and public provision of a basic package of healthcare services is often beyond reach. The budgeting process in many LMICs requires that government departments prepare their plans and budgets based on prioritisation of interventions to support the achievement of the Millennium Development Goals, Sustainable Development Goals and the pursuit of Universal Health Coverage (Tangcharoensathien et al., 2013). From an economic perspective, priority setting can be seen as an ‘explicit’ process for choosing the optimal portfolio of programmes from a limited national healthcare budget (Hauck et al., 2004). Priority setting can occur at the global, regional, and national or subnational level and involves both the identification and weighting of criteria and a framework for establishing priorities (Oxman et al., 2006). Put simply, it is about who gets what at whose expense (Williams, 1988).

A range of frameworks has been used to set priorities on the basis of value for money. One approach has been to compare the cost‐effectiveness of different interventions using a quality‐adjusted life year (QALY) league table. This involves ranking interventions according to their ‘incremental cost per QALY’ and then allocating funds starting at the top of the list and working down, in theory, stopping once the prevailing budget has been exhausted (Mauskopf et al., 2003; Gerard and Mooney, 1993). Alternatively, such decisions are made with reference to a threshold or ceiling, whereby all interventions below should be funded and those above not (Marseille et al., 2015; Cleemput et al., 2011). For example, generalised cost‐effectiveness ratios, initiated by the World Health Organisation, compare the cost and health benefits of packages of different interventions against a situation in which those interventions did not take place (Murray et al., 2000). Using a threshold based on multiples of gross domestic product per capita, the results are then put in three categories: very cost‐effective, cost‐effective, and not cost‐effective (Hutubessy et al., 2002).

Another approach is program budgeting and marginal analysis (PBMA) (Mooney, 2002; Mitton and Donaldson 2004). PBMA provides an information framework to allow a picture of where resources are currently going (programme budgeting) and thereafter looking at whether a movement of resources from one programme to another might increase total benefits (marginal analysis). There are even broader frameworks that take account of efficiency criteria, including option appraisal, multi‐criteria decision analysis (MCDA) and multiple attribute utility analysis (Mooney et al., 2012). MCDA, in particular, is increasingly being used to evaluate healthcare interventions and involves the systematic definition of criteria relevant to a decision, the performance of options against these criteria and often the weighting of criteria to produce an overall value metric (Baltussen et al., 2006; Marsh et al., 2014). According to Mooney et al. (2012), approaches like these represent ways of allowing often‐difficult‐to‐measure attributes (such as patient reassurance) to be included and weighted alongside efficiency criteria. Guidance is also available on how equity criteria can be considered in addition to cost‐effectiveness analysis for priority setting (Norheim et al., 2014; Cookson et al., 2009).

What is our definition of priority setting? What makes these examples of priority‐setting frameworks and sets them apart from all economic evaluation studies is that a funding decision is central to the analysis as opposed to an intervention. These studies seek to identify priority areas for investment (or disinvestment) often taking into account a range of organisational objectives including (but not limited to) efficiency. While economic evaluation studies in health provide evidence based on the relative cost of achieving units of health gain, priority‐setting studies centre on the ultimate investment decision. In such decisions, there are potentially multiple inputs and influences, one of which is cost‐effectiveness information. Several reviews of priority‐setting approaches that incorporate efficiency criteria are available for high‐income countries (HICs; see, e.g. Mooney et al., 2012; Robinson et al., 2012; Bate and Mitton, 2006; Noorani et al., 2007; Hauck et al., 2004). While less is known about comparable frameworks used in LMICs, recent informal reviews point to a growing level of interest in evidence‐based priority setting by policy makers in these countries (see, e.g. Glassman et al., 2013; Hipgrave et al., 2014). This review builds on the existing priority‐setting literature by concentrating on LMICs and, secondly, by adopting a transparent and explicit approach to the selection and assessment of methodological frameworks (which incorporate economic evaluation evidence) for priority setting in health care. A systematic review of wider issues such as acceptability and stakeholder engagement associated with priority setting was outside the remit of this review.

2. Methods

The review focused on papers describing frameworks for setting healthcare priorities and included some assessment of relative costs and consequences of healthcare interventions.

As the term ‘priority setting’ may not be universally used, we adopted a comprehensive search strategy using a series of broad but related terms and then carried out extensive manual reviews to filter out those papers that did not meet our definition of priority setting (as set out in Section 1). The authors initially reviewed 20 papers that they felt were relevant to the research question. The subject headings of these papers were reviewed in MEDLINE and Embase in order to ascertain the pertinent subject headings and significant text words/keywords from the abstracts. Subsequently, a template search was developed in MEDLINE (Ovid) that included three concepts: priority setting, cost, and low‐income or lower‐middle‐income countries. Each concept was composed of MeSH subject headings and keywords in order to develop a sensitive search strategy. The low‐income and lower‐middle‐income concept included country names based on the World Bank's (2014) defined list. The MEDLINE search was translated into the appropriate terms in Embase (Ovid), PubMed and EconLit (EBSCOhost). The search strategy (including a complete list of search terms) is shown in Web appendix. This was supplemented by a search of relevant papers' reference lists. All references were exported to RefWorks.

Papers were eligible for inclusion if they were published in the peer‐reviewed literature between 2004 and 2014, were complete (e.g. conference abstracts were excluded) and were reported in the English language. All paper types (with the exception of books) were included. Eligible papers also needed to report on at least one ‘low’‐income or ‘lower‐middle’‐income country as defined by the World Bank (World Bank, 2014). Studies that referred to ‘developing countries’ were also included despite not specifying an individual country. Eligible papers needed to report on priority‐setting frameworks or approaches that considered a portfolio of interventions for a particular disease, programme or entire health sector.

Two independent reviewers (V. W. and C. M.) screened the first 83 articles against the inclusion criteria to determine inter‐rater reliability of the reviews. Any disagreement in article selection was discussed until consensus was reached. Agreement was assessed using a simple kappa analysis (Cohen, 1960). Substantial agreement (defined as kappa exceeding 0.6) was desired for a decision to continue with a single reviewer (Landis and Koch, 1977). The calculated kappa score was 0.78, classifying the level of agreement as ‘almost perfect’ (Landis and Koch, 1977). Screening for article selection was then completed by one reviewer (V. W.). A second reviewer (C. M.) was consulted when clarity was needed. Both reviewers read the full text of the selected articles and resolved disagreements by consensus. Reasons for inclusion and exclusion were recorded.

A data extraction form was designed and piloted for this study which included fields for the following descriptive data: date and location of the study; type of paper (modelling, commentary, review, framework development, etc.); country in which the study was undertaken; scale (global, regional, national or subnational); type of health interventions; criteria used for priority setting; technique used to measure efficiency; name of priority‐setting process; and main data sources. The data were initially extracted by one reviewer (V.W.). A second reviewer (C. M., L. C. or T. D.) independently extracted the same data for all papers, and any disagreements were resolved. All variables included in the data extraction form are defined in Table 1.

Table 1.

Description of variables/questions for data extraction

Variable/question	Definition
Overview of peer‐reviewed papers
Author/year	Authors of the article and year of publication.
Country	Location of the priority‐setting exercise.
Paper type	Description of the study by authors – review, economic modelling, exploratory/pilot study, strategic planning document and framework development.
Scale	Global, regional and national/sub‐national.
Methods and data sources
Interventions	Type of health sector interventions to be prioritised.
Criteria	Stated criteria upon which priorities were set.
Efficiency measure	Cost‐effectiveness (including cost‐utility) or cost–benefit analysis. Includes ratio used (e.g. cost per DALY averted or cost per QALY gained).
Priority‐setting approach	Framework into which efficiency (and other) criteria feed (e.g. ranking of interventions based on multi‐criteria decision analysis or generalised cost‐effectiveness approach, programme budgeting and marginal analysis, etc.)
Data source	Literature (peer‐reviewed literature and open‐access databases), expert/stakeholder opinion, primary data collection.
Appraisal
Was the perspective of the economic analysis specified?	Perspective or viewpoint of the economic analysis. Includes society or provider. (Yes/No/Not applicable)
Was allowance made for uncertainty in the estimates of costs and consequences?	Identification and testing of uncertain parameters associated with costs and consequences. (Yes/No/Not applicable/Some)
Was affordability of priority interventions discussed/measured?	Recognition of an explicit budget constraint. (Yes/No/Not applicable/Some)
Did the exercise investigate disinvestment as well as investment in health interventions?	Decommissioning, disinvesting or redeploying resources from currently funded interventions. (Yes/No/Not applicable)
Was the study embedded in the local policy and planning context?	Broad indication of likely impact and sustainability of the priority‐setting framework on decision‐making. (Yes/No/Not applicable)

Uniformly recognised criteria for appraising studies in this field are not available. Guidelines developed for PBMA by Peacock et al. (2010) were adapted for the appraisal of studies in this review. Five key questions were asked of each study: (i) Was the perspective of the study determined? (ii) Was a sensitivity analysis performed? (iii) Was affordability of the health interventions considered? (iv) Did the study consider releasing or redeploying resources (as well as investment)? and (v) Was the study embedded in the local policy and planning context with involvement by local decision‐makers? Table 4 summarizes the reviewer responses to these appraisal questions for each of the included papers.

Table 4.

Appraisal of priority‐setting frameworks

Article	Author(s)	Was the perspective of the economic analysis specified?	Was allowance made for uncertainty in the estimates of costs and consequences?	Was affordability assessed?	Did the exercise investigate disinvestment as well as investment?	Was the study embedded in the local policy and planning context?
#1	Baltussen et al, 2007	Y (societala)	N	N	N	N
#2	Hansen & Chapman, 2008	Y (provider)	Y	N	N	N
#3	Kapiriri & Norheim, 2004b	N	N/A	N/A	N/A	N
#4	Kase, 2006	Y (provider)	N	Y – budget thresholdc	Y	Y
#5	Baltussen, 2006	Y (provider)	N	Y – budget impact analysisd	N	N
#6	Baltussen et al., 2006	Y (societal)	N	Y – budget impact analysise	N	N
#7	Chisholm et al, 2008	N	Y	N	N	N/A
#8	Diaby & Lachane, 2011	Y (provider)	Y	Y – budget impact analysisf	N	N
#9	Evans, Lim et al, 2005	Y (provider)	Y	Y – budget thresholdg	N	N/A
#10	Ginsberg et al, 2012	Y (societal)h	Y	N	N	N/A
#11	Jehu‐Appiah et al, 2008	Y (societal)	N	N	N	Y
#12	Laxminarayan et al., 2006	Y (provider)	N	N	N	N/A
#13	Makundi et al., 2007	N	N	N	N	N
#14	Venhorst et al, 2014	N	N	Y – measure not specified	N	N/A
#15	Marsh et al., 2014	Somei	Somej	Some – budget impact analysisk	N	N/A
#16	Chao et al, 2014	Some	Somel	N	N	N/A
#17	Diaby et al., 2011	Y (provider)	N	Y – budget impact analysism	N	N/A
#18	Simons et al, 2011	Y (providern)	Y	N	N	N/A
#19	Canning, 2006	N	N	N	N	N/A
#20	Whittington et al., 2012	N	Y	N	N	N/A
#21	Madi et al., 2007, o	N/A	N/A	N/A	N/A	Y
#22	Adam et al., 2005	Y (societalh)	N	Y – budget thresholdg	Y	N/A
#23	Baltussen et al, 2005	Y (societalh)	Y	Y – budget thresholdg	N	N/A
#24	Baltussen, 2012	Y (societalh, p)	Y	N	N	N/A
#25	Chisholm, Baltussen et al, 2012	Y (societalh)	N	N	N	N/A
#26	Chisholm, Naci et al, 2012	Y (societalh)	Y	N	N	N/A
#27	Chisholm, Saxena et al, 2012	Y (societalh)	Y	Y – budget thresholdq	N	N/A
#28	Darmstadt et al, 2005	Y (societalh)	Y	Y – budget thresholdq	N	N/A
#29	Edejer et al, 2005	Y (societalh)	Y	Y – budget thresholdg	N	N/A
#30	Morel et al, 2005	Y (provider)	Y	Y – budget thresholdg	N	N/A
#31	Ortegon et al, 2012	Y (societalh)	Y	Y – budget thresholdr	N	N/A
#32	Stanciole et al, 2012	Y (societalh)	Y	Y – budget thresholdq	N	N/A
#33	Hogan et al, 2005	Y (societal)	Y	Y – budget thresholdg	N	N/A
#34	Cecchini et al 2010	N	Y	N	N	N
#35	Chisholm, Doran et al, 2006	N	Somes	N	N	N/A
#36	Gureje et al, 2007	N	N	Y – budget thresholdt	N	N

Y, Yes; N, No; N/A, not applicable; Some.

^a Cost‐effectiveness data are derived from the WHO CHOICE project, which is reported to take a societal perspective (Evans et al., 2005).

^b This study only sought to derive criteria for priority setting.

^c Threshold not specified.

^d Budget impact was one of the criteria for priority setting. Budget impact was not measured.

^e Intervention was defined as having a large budget impact when it costs more than (an arbitrarily defined) $US15 million (i.e. >10% of annual public health expenditure).

^f Budget Impact Analysis (BIA) suggests that the new priority list of reimbursable drugs deemed affordable if the real economic impact of drugs per member is less than $US66.

^g Interventions to be highly cost‐effective if they cost less than the gross domestic product per capita to avert each disability adjusted life years (DALY) and cost‐effective if each DALY could be averted at a cost of between one and three times the gross domestic product per capita. Other interventions are not cost‐effective. According to the authors, this incorporates an element of affordability as regions and countries with lower national income will have lower cut‐off points.

^h These classifications are based on estimates of gross national income (GNI) per capita for the previous year (World Bank, 2014).

ⁱ This review includes a checklist that assesses whether a ‘well‐defined question was posed in answerable form’ (Drummond et al., 2005). This was taken to include a description of study viewpoint.

^j According to the authors, approximately half of the studies in the review conducted a sensitivity analysis.

^k Of the 40 studies, three included budget impact as a criterion for priority setting. Budget impact was not measured.

^l According to the authors, the majority of studies included in the review conducted a sensitivity analysis.

^m Budget impact analysis was recommended as one of the drug selection criterion. Budget impact not measured as part of the proposed framework.

ⁿ Obtained from companion costing paper by Wolfson et al. (2008).

^o This study only sought to derive criteria for priority setting.

^p Authors note that they did not include time costs of people seeking and undergoing care or changes in productivity losses as a result of the interventions.

^q Intervention packages deemed ‘very cost effective’ if below average per‐person GDP, or Intl$1391. Implies an element of affordability as countries with lower national income will have lower cut‐off points.

^r Uses average per capita income (which in both sub‐regions is close to $Int2000) as a threshold for considering an intervention to be highly cost‐effective. Implies an element of affordability as regions and countries with lower national income will have lower cut‐off points.

^s According to the authors, the majority of studies performed a sensitivity analysis.

^t Uses average per capita income in Nigeria (which is $US320) as a threshold for considering an intervention to be highly cost‐effective. According to the authors, the total financial outlay of government for the most highly cost‐effective package of mental health interventions is estimated to be relatively small (less than $US1 per capita).

No ethical approval was required for this systematic review.

3. Results

3.1. Study selection

The search in Embase, MEDLINE, Econlit and PubMed identified 3968 articles, and a further 21 articles were identified through manual searching of reference lists. After the removal of duplicates, 3061 titles and abstracts were screened against the inclusion criteria. Of these, 123 full articles were assessed for eligibility with 36 articles meeting the inclusion criteria and kept for data abstraction (Figure 1). A complete list of the papers can be found in Appendix 1. The main reasons for excluding studies was that despite mentioning priority setting, costs and cost‐effectiveness in the abstract or paper, they did not focus on the health sector or the funding of a package of healthcare interventions.

Figure 1.

Selection of studies flow chart. *Authors contacted to confirm that a full paper was not available. **Priority setting papers that do not focus directly on health. ***Information captured in other papers. PS, priority setting; CE, cost effectiveness

3.2. Study characteristics

Table 2 shows that of the 36 eligible papers, 15 involved studies of priority setting at the national level (11 from African countries), 14 at the regional level and 9 at the global level. The majority of studies (19) took a modelling approach.

Table 2.

Priority‐setting studies in LMICs: overview of peer‐reviewed papers

Article	Author/year	Paper type	Primary aim	Interventions	Scale (location)
#1	Baltussen et al, 2007	Exploratory	Show how multiple criteria can be used to guide the priority‐setting process.	Lung health programme	National (Nepal)
#2	Hansen & Chapman, 2008	Exploratory	Assess feasibility of conducting cost‐effectiveness analyses for a large number of health interventions in a developing country.	65 curative interventions for common health problems and preventative interventions	National (Zimbabwe)
#3	Kapiriri & Norheim, 2004	Exploratory	Explore stakeholders' acceptance of criteria for setting priorities for the healthcare system.	Not specified	National (Uganda)
#4	Kase, 2006	Strategic planning	Describe process for designing Government Medium Term Expenditure Framework.	Essential health services (wages/salaries), basic system support and interventions (malaria, immunisation, safe motherhood, outreach, supervision)	National (Papua New Guinea)
#5	Baltussen, 2006	Exploratory	Identify priority interventions under two assumptions: public spending should be targeted at the whole population or the poor only.	All priority interventions listed in 2002 World Health Report	National (Ghana)
#6	Baltussen et al., 2006	Exploratory	Show how multiple criteria can be used to guide the priority‐setting process.	Set of hypothetical health interventions – taken from 2002 World Health Report	National (Ghana)
#7	Chisholm et al., 2008	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Schizophrenia	Regional Americas, Africa and South‐East Asia + National (Chile,a Nigeria, Sri Lanka)
#8	Diaby & Lachane, 2011	Exploratory	Evaluate the feasibility of developing a new formulary for a health mutual fund.	Formulary for drug reimbursement	National (Cote D'Ivoire)
#9	Evans, Lim et al., 2005	Economic modelling	Summarise key findings from a series of papers on the cost‐effectiveness of strategies to achieve the millennium development goals for health.	Maternal and neonatal health, child health, HIV and Aids, malaria and tuberculosis	Regional (sub‐Saharan Africa and South East Asia)
#10	Ginsberg et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Breast, cervical and colorectal cancers	Regional (sub‐Saharan Africa and South East Asia)
#11	Jehu‐Appiah et al., 2008	Exploratory	Illustrate how multiple criteria can be used to guide the priority‐setting process.	Child health, reproductive health, and communicable diseasesb	National (Ghana)
#12	Laxminarayan et al., 2006	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	94 clusters of interventions – representing 218 interventions covering: tuberculosis, HIV/AIDS, illness and mortality in children, tropical diseases, reproductive health, nutrition, cancer, neurological disorders, cardiovascular disease, injury prevention, surgery, alcohol and tobacco use, delivery of interventions and strengthening health systems.	Regional (South Asia and sub‐Saharan Africa) + Global (LMICs)
#13	Makundi et al., 2007	Exploratory	Test out the ‘balance sheet method’ for priority setting, which incorporates both scientific evidence and public values.	Integrated Management of Childhood Illness (IMCI), safe water, HIV, tuberculosis, malaria	National (Tanzania)
#14	Venhorst et al., 2014	Exploratory	Develop rating tool for policy makers to prioritise interventions based on multiple criteria.	Breast cancer	Global (LMICs)
#15	Marsh et al., 2014	Review	Document studies that have used multi‐criteria decision analysis to set healthcare priorities and lessons learnt.	Pharmaceuticals, public health interventions, screening, surgical interventions, and devices	Regional + national
#16	Chao et al., 2014	Review & economic modelling	Extract and appraise economic assessments for their methodological quality.	Surgery	Regional + national
#17	Diaby et al., 2011	Review & framework development	Review processes used by high‐, middle‐ and low‐income countries, to prioritise medicines for reimbursement.	Formulary for drug reimbursement	National (Canada, US, UK, France, Germany, Brazil, South Korea, Ghana)
#18	Simons et al., 2011	Economic modelling	Explain how a disease‐intervention planning tool can be used to prioritise health interventions and review of preliminary user experience.	Measles	Not applicable
#19	Canning, 2006	Review + exploratory	Explore the economic case for prioritising prevention over the treatment of HIV/AIDS. Compares cost‐effectiveness criterion to other criterion for setting priorities.	HIV prevention and treatment	Regional (Africa)
#20	Whittington et al., 2012	Economic modelling	Illustrate the challenges and uncertainties of setting priorities amongst competing interventions at the global level using economic evidence.	Water, sanitation and preventive health interventions (insecticide‐treated bed nets, cholera vaccination).	Global (LMICs)
#21	Madi et al., 2007	Exploratory	Describe a process for involving key stakeholders to elicit and prioritise health interventions.	Maternal	National (Burkina Faso, Ghana and Indonesia)
#22	Adam et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Maternal and neonatal health	Global (LMICs)
#23	Baltussen et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Tuberculosis	Global (LMICs)
#24	Baltussen & Smith, 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Vision and hearing loss	Regional (sub‐Saharan Africa and South East Asia)
#25	Chisholm, Baltussen, et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Non‐communicable diseases and injuries	Regional (sub‐Saharan Africa and South East Asia)
#26	Chisholm, Naci, et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Road traffic injuries	Regional (sub‐Saharan Africa and South East Asia)
#27	Chisholm & Saxena 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Neuropsychiatric conditions	Regional (sub‐Saharan Africa and South East Asia)
#28	Darmstadt et al., 2005	Review and modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Neonatal	Global (LMICs)
#29	Edejer et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Child health	Global (LMICs)
#30	Morel et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Malaria	Global (LMICs)
#31	Ortegon et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Cardiovascular disease, diabetes, tobacco use	Regional (sub‐Saharan Africa and South East Asia)
#32	Stanciole et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Chronic obstructive pulmonary disease and asthma	Regional (sub‐Saharan Africa and South East Asia)
#33	Hogan et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	HIV/AIDS	Global (LMICs)
#34	Cecchini et al., 2010	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Chronic diseases	National (Brazil, China, India, Mexico, Russia, South Africac)
#35	Chisholm, Doran et al., 2006	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Alcohol, tobacco and illicit drug use	Regional (America, Europe and South East Asiad)
#36	Gureje et al. 2007	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Mental health	National (Nigeria)

LMICs, low‐income and lower‐middle‐income countries; MCDA, multi‐criteria decision analysis.

^a Note that Chile is classed as a high‐income country while Nigeria and Sri Lanka are classified as ‘lower‐middle’‐income countries (World Bank, 2014).

^b These are the final set of interventions identified using MCDA. Original exercise included childhood diseases, communicable and non‐communicable diseases, reproductive health and injuries.

^c Only India is a “lower‐middle‐income” country and meets the eligibility criteria for this review.

^d These regions are defined by WHO as American sub‐region AmrB (countries with low rates of child and adult mortality, e.g. Brazil or Mexico); European sub‐region EurA (countries with very low child and adult mortality, e.g. France or Norway); and South East Asian sub‐region SearD (countries with high child and adult mortality, e.g. India or Nepal).

Table 3 shows that studies covered a broad range of health interventions with only two studies including health systems strengthening interventions related to financing, governance, information and human resources for health (#4 and #12). A little under half of all studies (39%) included at least one other criterion apart from efficiency for priority setting, most commonly ‘equity’ and ‘feasibility’. Most studies (91%) specified a measure of ‘efficiency’, commonly defined as cost per disability‐adjusted life year (DALY) averted (72%). Only one study calculated cost per QALY gained (#8), and one study estimated costs and benefits in monetary terms to derive a benefit–cost ratio (#20).

Table 3.

Priority‐setting studies in LMICs: methods and data sources

Article	Criteria	Efficiency measure	PS approach	Highest priority interventions	Data source
#1	Efficiency, severity of disease, number of potential beneficiaries, age of beneficiaries, individual health benefits, poverty reduction	Cost‐effectiveness (cost per DALY averted)	DCE to derive criteria. Ranking interventions based on MCDAa	TB control, followed by oral rehydration therapy for diarrhoea and case management of pneumonia in child health and several interventions in HIV/AIDS. AIDS control including the provision of antiretroviral therapy.	Literature + expert opinion
#2	Efficiency, important health problems	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE (grouped by health problem)	Curative treatment at health centres and hospital outpatient departments of pneumonia, severe diarrhoeal diseases, peptic ulcer, dysentery, malaria, trachoma, schistosomiasis haematobium and glaucoma.	Literature + local survey data + expert opinion
#3	Severity of disease, benefit of the intervention; cost of the intervention, efficiency; quality of the data on effectiveness; the patients age; place of residence; lifestyle; importance of providing equity of access to health care and the community's viewsb	Cost‐effectiveness (no ratio given)	N/A (identify criteria only)	N/A (identify criteria only).	Local survey data
#4	Benefit the greatest number; impact greatly on morbidity and mortality; prevention focused; accessible and affordable; efficient; impact on performance; improve financial and management sustainability	Not specified	Explicit approach akin to PBMAc – used agreed criteria as basis for prioritising in 3 key areas and identify the necessary resource shifts	Malaria prevention (bed nets and selected spraying in the highlands), safe motherhood and family planning, immunisation, STI/HIV/AIDS.	Expert opinion
#5	Efficiency; poverty reduction; severity of disease; age of target group; budget impact; individual health effect	Cost‐effectiveness (no ratio given)	Ranking based on 2 steps: define who should be targeted (step 1), use DCE to derive weights for relative criteria for PS, then scored/mapped interventions against these to develop rank ordering of interventions to targeted groups (step 2)	Prevention of mother‐to‐child HIV/AIDS transmission and oral rehydration therapy to treat diarrhoea in childhood (whole population), case‐management of pneumonia in childhood (targeting the poor).	Literature + expert opinion
#6	Efficiency; poverty reduction; severity of disease; age of target group; budget impact; health effects	Cost‐effectiveness (cost per DALY averted)	DCE to derive criteria. Ranking interventions based on MCDA	Prevention of mother to child transmission in HIV/AIDS control and treatment of pneumonia and Diarrhoea in childhood.	Local survey data
#7	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEAd – ranking of interventions based on relative CE	Interventions using older antipsychotic drugs combined with psychosocial treatment, delivered via a community‐based service model.	Literature + local survey data + expert opinion
#8	Efficiency; severity of the condition; socioeconomic status; age group of patients	Cost‐effectiveness (cost per QALY gained)	Ranking of interventions based on MCDA	Antimalarials, treatments for asthma and antibacterials for urinary tract infection.	Literature + expert opinion
#9	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – Ranking of interventions based on relative CE	N/A (priority interventions can be found in individual papers from this series (#33–37)).	Literature + local survey data + expert opinion
#10	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Cervical cancer control – screening through cervical smear tests or visual inspection with acetic acid in combination with treatment. Colorectal cancer control – increasing the coverage of treatment interventions.	Literature + expert opinion
#11	Targeting vulnerable populations; efficiency, severity of disease; number of beneficiaries; diseases of the poor	Cost‐effectiveness (cost per DALY averted)	DCE to derive criteria. Ranking interventions based on MCDA	Childhood interventions, most interventions targeting communicable diseases and two reproductive health interventions (supervised deliveries and emergency obstetric care).	Literature + local survey data + expert opinion
#12	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	See full publication for priority interventions for 94 diseases and conditions.	Literature + local survey data + expert opinion
#13	Prevalence; disease burden; coverage; severity of disease; efficacy; efficiency; equity	Cost‐effectiveness (ratio not given)	Balance sheet methode to derive ranking of interventions	N/A (development and testing of a model for incorporating scientific evidence and societal values in priority setting).	Literature + local survey data + expert opinion
#14	Effectiveness; quality of the evidence; magnitude of individual health impact; acceptability; efficiency; technical complexity; affordability; safety; geographical coverage; accessibility	Cost‐effectiveness (ratio not given)	N/A (develop criteria only)	N/A (development of a rating tool to assess the impact of breast cancer interventions on multiple criteria).	Literature + expert opinion
#15	Efficiency; disease severity; treatment access; target population size; curative or preventative; budgetary and other practical constraints; evidence quality; political factors	Not specified	Ranking of interventions based on MCDA	N/A (summarises existing approaches for MCDA of healthcare interventions and lessons learnt).	Literature
#16	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Male circumcision in Mozambique and cataract repair in Nepal.	Literature + expert opinion
#17	Efficiency; severity of disease; capacity of the intervention to reduce poverty; age; anticipated health gains; financial impactf	Not specified	Ranking of interventions based on MCDA	N/A (summarises article #6).	Literature + expert opinion
#18	Efficiency	Cost‐effectiveness (cost per additional case, death, and DALY averted)	Ranking of interventions based on relative CE	N/A (tool for measles strategic planning).	Literature + expert opinion
#19	Efficiency	Cost‐effectiveness (cost per DALY averted and cost per infection averted)	Ranking of interventions based on relative CE	Mass media messages, peer education, condom distribution, treatment of sexually transmitted diseases for commercial sex workers, treatment of tuberculosis is highly cost‐effective in those who are HIV‐positive as well as the general population.	Literature + expert opinion
#20	Efficiency	Cost–benefit analysis (BCR)	Ranking of interventions based on BCRsg	Biosand filter and point of use chlorination interventions offer the largest benefits to a household's well‐being.	Literature
#21	Ability to meet national policy priorities; reduce maternal mortality and/or morbidity; improve services; efficient and financially sustainable	Cost‐effectiveness (ratio not specified)	Not specified	N/A (describes process involving key stakeholders to elicit and prioritise evaluation needs for safe motherhood).	Primary data
#22	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Community‐based newborn care package, antenatal care (tetanus toxoid, screening for pre‐eclampsia, screening and treatment of asymptomatic bacteriuria and syphilis), skilled attendance at birth, offering first level maternal and neonatal care around childbirth, emergency obstetric and neonatal care around and after birth.	Literature + expert opinion
#23	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Treating only smear‐positive cases, treatment for both smear‐positive and smear‐negative and extra‐pulmonary cases at a coverage level of 95%.	Literature + expert opinion
#24	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Treatment of chronic otitis media, extracapsular cataract surgery, trichiasis surgery, treatment for meningitis, and annual screening of school children for refractive error.	Literature + expert opinion
#25	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Disease clusters cover over 500 interventions. A subset of 53 interventions is deemed ‘highly’ cost‐effective. See full article for details.	Literature
#26	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Single most cost‐effective intervention varies by region. Combined intervention strategy that simultaneously enforces multiple road safety laws (e.g. the combined enforcement of speed limits, drink‐driving laws, and motorcycle helmet use).	Literature
#27	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Population‐based alcohol control (Africa), drug treatment of epilepsy in primary care (South‐East Asia).	Literature
#28	Efficacy; effectiveness; feasibility; efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Family care/low birthweight care, Emergency obstetric care, family care/low birthweight care + community‐based case management of pneumonia, Skilled maternal and immediate neonatal care, emergency obstetric care + corticosteroids for preterm labour + antibiotics for preterm premature rupture of membranes.	Literature + expert opinion
#29	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Fortification with zinc or vitamin A.	Literature + expert opinion
#30	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	High coverage with artemisinin‐based combination treatments.	Literature + expert opinion
#31	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Demand reduction strategies of the Framework Convention for Tobacco Control; combination drug therapy for people with a >25% chance of experiencing a cardiovascular event over the next decade, either alone or together with specific multidrug regimens for the secondary prevention of post‐acute ischaemic heart disease and stroke; and retinopathy screening and glycaemic control for patients with diabetes.	Literature
#32	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Low‐dose inhaled corticosteroids for mild persistent asthma.	Literature + expert opinion
#33	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Education and treatment of sexually transmitted infections for sex workers.	Literature + expert opinion
#34	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Health information and communication strategies that improve population awareness about the benefits of healthy eating and physical activity, fiscal measures that increase the price of unhealthy food content or reduce the cost of healthy foods rich in fibre, regulatory measures that improve nutritional information or restrict the marketing of unhealthy foods to children.	Literature + expert opinion
#35	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Nicotine replacement therapy or brief physician advice (individual level); taxation of alcoholic or tobacco products (population‐wide level).	Literature + expert opinion
#36	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	For schizophrenia: community‐based treatment with older antipsychotic drugs plus psychosocial support or case management. For depression, epilepsy, and alcohol use disorders: older antidepressants, with or without proactive case management in primary care, older anticonvulsants in primary care, and random breath testing for motor vehicle drivers.	Literature + local survey data + expert opinion

LMICs, low‐income and lower‐middle‐income countries; DALY, disability‐adjusted life year; PS, priority setting; CE, cost‐effectiveness; N/A, not applicable; BCR, benefit–cost ratios; DCE, discrete choice experiment.

^a MCDA (multi‐criteria decision analysis) involves describing criteria, arranging the criteria on a performance matrix and assigning ratings for each program option to aid transparent and consistent decision making (Baltussen et al., 2007).

^b High‐weight criteria. The authors also identified ‘average’‐weight and ‘low’‐weight criteria.

^c PBMA (programme budgeting and marginal analysis) priority‐setting toolkit that helps decision‐makers maximise the impact of healthcare resources on the health needs of a local population and examines how resources are currently spent and the costs and effects of changing spending patterns (Mitton and Donaldson, 2004).

^d GCEA – generalised cost‐effectiveness approach. Interventions classified into those are very cost‐effective, cost‐ineffective, and somewhere in between (Hutubessy et al., 2002).

^e Balance sheet method = model for incorporating scientific evidence and societal values in priority setting (Eddy, 1990).

^f Authors suggested criteria for LMICs.

^g While authors discuss how BCRs can inform priority ranking of interventions, data limitations prevent them from doing so in this context.

In reviews such as this where the topic and questions being asked are broad, heterogeneity is inevitable both in terms of methods and outcomes. Priorities were derived through a range of approaches including MCDA, generalised cost‐effectiveness approach (GCEA), balance sheet method and an approach akin to PBMA (Mooney, 2002). GCEA and MCDA were the most common approaches used in 40% and 18% of studies, respectively. All of the GCEA studies relied on data from the WHO Choosing Interventions that are Cost‐effective Project. Most studies relied on secondary data for priority setting and noted several gaps in the evidence base especially in relation to effectiveness data, which often resulted in country‐level studies relying on regional data or expert opinion. The priority interventions identified by the different studies were also diverse, representing mixes of curative and preventive actions and of population and individually focused interventions.

3.3. Appraisal

Around half (45%) of the studies took a societal perspective for the economic analysis, and 27% did not specify any perspective (Table 4). It was also revealed that 58% of studies undertook a sensitivity analysis (which was most often a one‐way univariate analysis), and 51% assessed the affordability of priority interventions. This was most often carried out through a threshold analysis designed to reflect the opportunity cost of investment or through a budget impact analysis whereby an intervention is said to have a large budget impact when it costs more than an arbitrarily defined proportion of annual public health expenditure (i.e. more than 10%). Only one study investigated the issue of disinvestment and identified priority areas for reduced healthcare spending (#4). Only three studies were embedded in local appraisal and budgeting processes involving policy makers and ministerial officials responsible for healthcare investments (#4, #11 and #21). It should be noted that the appraisal question concerning local stakeholder involvement was only applicable to 12 studies because the remaining ones were either systematic reviews or designed to test a tool or focussed on priority setting at the global or regional level. Also, any studies for which the appraisal category is recorded as ‘not applicable' were excluded from the total number of studies included in the denominator.

4. Discussion

The need for priority setting and trading‐off between potentially beneficial and life‐saving investments options is a universal problem. It is especially important in highly resource‐constrained settings where the most basic health services are sometimes unaffordable and where there is a heavy reliance on international donor support. As such, policy makers in LMICs and donors are increasingly looking to develop ‘evidence‐based’ approaches for setting investment priorities in the health sector. This review of the peer‐reviewed literature provides a comprehensive overview of the different frameworks designed to take account of the problem of resource scarcity and incorporate the assessment of relative costs and effects of interventions in LMICs.

This review shows that there is widespread acknowledgement that decisions on the allocation of scarce healthcare resources are shaped by a range of criteria and that economic evaluation information must be considered alongside other health system goals. Approximately half of the studies in this review included multiple criteria for priority setting. Some studies also reported different priorities based on efficiency versus non‐efficiency criteria, thereby highlighting the importance of specifying the relevant criteria at the outset of any priority‐setting exercise (Diaby et al., 2011; Johansson and Norheim, 2011). Overlap between different criteria was recognised as an additional methodological issue requiring greater attention by researchers. In the MCDA literature, for example, studies included related criteria such as ‘cost of treatment’, ‘effectiveness of treatment’ and ‘cost‐effectiveness of treatment’ (see, e.g. Marsh et al., 2014; Diaby et al., 2011).

In LMICs, the most common framework in the peer‐reviewed literature for explicit prioritising of healthcare interventions is the league table approach. These tables are typically based on MCDA where simultaneous criteria are used or on GCEA where only efficiency criterion is used. Cost per DALY averted is the dominant measure of cost‐effectiveness, and it is often justified on the grounds that almost 90% of the global burden of disease is accounted for by LMICs (Murray and Lopez, 1997; Zarate, 2007). PBMA, which creates a management process into which results from standard economic evaluations and other evidence can be incorporated (Ruta et al., 2005), has been widely used in HICs including Australia, Canada, New Zealand and the UK (Mitton et al., 2014) but has not established a footing in LMICs. In addition to the widespread problem of a lack of credible information, the legitimacy and capacity of institutions management processes are often weak. Criteria such as ‘feasibility’ and ‘sustainability’ frequently feature amongst priority‐setting criteria in LMICs, which again may reflect the constrained capacity of local priority‐setting institutions.

This review also revealed a distinction between frameworks developed for use at the global and regional level versus those designed to set priorities at the country level. There have been an increasing number of sector‐wide frameworks such as The WHO Choosing Interventions that are Cost‐Effective Project and the Disease Control Priorities Project that have been led largely by the international community seeking to identify the most cost‐effective interventions for achieving targets such as the Millennium Development Goals and Universal Health Coverage. These two projects represented approximately half of all studies included in this review. It has been argued that while these analyses provide an indication of value for money, they do so at the expense of diversity or specificity of individual country contexts (Makundi et al., 2007). In contrast, country‐level priorities are often based on more consultative processes that require significant investment in building relationships and engaging and empowering stakeholders (Madi et al., 2007). To date, there has been little recognition in the priority‐setting literature of the diversity of stakeholders involved in both the funding and implementation of healthcare interventions in LMICs. Most country‐level priority‐setting analyses make the implicit assumption that the national government has over‐arching responsibility for priority setting, but in reality, substantial healthcare funds are already earmarked for specific interventions or disease areas based on the priorities of international or transnational non‐governmental organisations. Consequently, priority‐setting frameworks often fail to reflect the multifaceted nature of priority setting in many LMICs. Global institutions responsible for setting priorities should be lobbied to use and disclose their frameworks especially those receiving public money and be made accountable for how their activities might distort national‐level and local‐level priority setting. Recent initiatives such as the Asia‐Pacific Regional Capacity Building for Health Technology Assessment, a collaboration between the International branch of the National Institute of Clinical Excellence (UK) and countries in the region, are designed to provide technical support to governments in LMICs to help build up their priority‐setting capacity and to support multi‐stakeholder engagement (Teerawattananon et al., 2014; NICE International, 2014).

A lack of cost‐effectiveness evidence for many key health interventions in LMICs is another limitation to explicit priority setting at both the global and country level. The results of this review show that most studies base their analyses on extremely limited efficacy and effectiveness data or on expert opinion. Moreover, extrapolation of best available international evidence – from higher to lower income resource settings is often required but can be problematic due to clear differences in health systems and epidemiology (Adam et al., 2005; Chisholm and Saxena, 2012). However, priority‐setting approaches examined in this review tended to lack any account of uncertainty, utilising single‐point estimates for effectiveness and cost‐effectiveness.

Given these information gaps, priority‐setting studies in LMICs utilise economic evaluation data from many different settings. In doing so, methodological inconsistencies are inevitable. Interventions that are deemed to be cost‐effective in one setting may not be in another especially where coverage rates, disease incidence, relative prices and costs are different. These factors affect the applicability and reliability of evidence underpinning priority‐setting approaches. However, extrapolations are a necessary part of these exercises, and as such, evaluators working with decision‐makers need to adapt to the use of imperfect, ‘broad brushed’ data in a pragmatic way to inform a decision at hand (Mooney, 1994). A bigger problem observed in several studies was the practice of excluding potential interventions from consideration due to a lack of data (Adam et al., 2005). Because ultimately priority setting is about assessing as many competing options as feasible, such omissions undermine the validity of these exercises by introducing clear bias into their recommendations. Future initiatives need to be based more strongly on pragmatic responses to imperfect data.

This review also highlighted that many priority‐setting exercises are based on unrealistic assumptions about the overall functioning of health systems in LMICs. The expected costs and effects of priority health interventions depend on accompanying investments in health systems to support appropriate levels of human resources, timely referral systems, efficient and equitable financing systems and adequate institutional infrastructure (Laxminarayan et al., 2006 ). A common weakness of many priority‐setting initiatives is that investments in such infrastructure are not formally deliberated upon and included amongst the options being considered nor are the potential unintended consequences for health systems associated with shifting resources, such as staff from one programme to another. Priority‐setting exercises should not assume, for example, that a skilled workforce is available to deliver priority health interventions in LMICs nor should it assume that shifting health workers from one programme to another is a costless exercise.

Another important message emerging from this review is the limited attention paid to identifying and prioritising options for redeployment of resources. This is despite mounting evidence from around the world demonstrating missed opportunities to improve health through reallocation of public monies towards more cost‐effective interventions (Glassman et al., 2013). While most studies in this review implied that highly cost‐effective interventions should be prioritised over less cost‐effective ones, only one study provided a ‘real life’ example of where resources were targeted for release (Kase, 2006). This is not a problem confined to LMICs and has been documented in many HICs (Jan, 2003; Elshaug et al., 2007; Mortimer, 2010). It has been argued that problems with translating ‘wish lists’ of priority health interventions into action is typically a consequence of a failure to release resources from elsewhere in the health budget (Mortimer, 2010). It has been recommended that greater attention needs to be paid to creating stronger incentives for change by, for example, linking investment proposals to disinvestment proposals with relatively similar input requirements (Mortimer, 2010). Examples of good practice for resource release or redeployment do exist from HICs through the application of PBMA in which MCDA has been employed as a tool for assessing relative value of services (Mitton et al., 2014); however, again, translating this experience to LMICs has received limited attention to date.

The vast majority of economic evidence used in priority setting comes from cost‐effectiveness analyses where the measure used is typically cost per DALY averted. Only one priority‐setting exercise relied on evidence from a cost–benefit analysis (Whittington et al., 2012). The overall focus on health outcomes often means that broader welfare consequences such as improved production or increased days at school are not taken into account despite being important for many interventions in LMICs (Evans et al., 2005). For example, it has been reported that antiretroviral treatment for HIV and AIDs can help keep health workers and teachers in their posts and prevent a possible breakdown in society (Pufall et al., 2014). Such limitations highlight a wider challenge in adapting top‐down approaches to priority setting to the preferences and values of local constituencies. The use of measures such as cost per QALY in some LMICs has been justified on the grounds of local preferences being more important than expert DALY weights (Chalkidou, Communication).

The findings of this review must be seen in the light of some limitations. First, we acknowledge that some priority‐setting initiatives are documented in the grey literature. We chose to confine our search to the peer review literature given the focus of this review is largely on methodological issues and that these publications are expected to have undergone some basic quality control. Our search was also restricted to papers in the English language published in the past 10 years. Thirdly, it was beyond the scope of this paper to comprehensively assess ‘impact’, defined as the degree to which priority‐setting results fed into an appraisal process that was led or convened by budget holders. Our appraisal, which asked whether a study was embedded in local budgeting processes, revealed that very few studies met this criterion. While this provides some insight into the likely impact and sustainability of a priority‐setting framework, this is not a definitive measure of impact on actual resource allocation decision‐making. Currently, much of the evidence needed to assess policy impact is not routinely published, particularly in instances of failure in implementation. The task of attribution is further complicated by the delay that often occurs between the presentation of evidence and of action, the multiple and complex influences that tend to bear on resource allocation decisions and because the role of priority‐setting evidence is not necessarily explicitly acknowledged when decisions are ultimately made. As a result, the existing literature (peer reviewed or otherwise) is unlikely to be representative of what is really happening on the ground in LMICs. Policy makers should be encouraged to document and share their experiences with priority setting. Initiatives such as the Capacity Building Programme on Universal Health Coverage Academy in Thailand (www.ihpp.thaigov.net/capuhc), coordinated by the International Health Policy Programme and funded by Rockefeller, is likely to prove to be a much better source of priority‐setting approaches than the western literature alone.

Table 5 provides a summary of the different challenges discussed earlier and includes recommendations about what might be done to encourage a more transparent and implementable way of using economic information for priority setting in LMICs.

Table 5.

Examples of factors to support priority setting and the use of economic evidence in LMICs

• Greater encouragement of empirical studies that systematically evaluate real‐world priority setting. Most studies are small‐scale exploratory exercises that are not embedded in local policy and planning context and that rely on regional estimates of costs and effects.

• Interventions are cost‐effective in some settings and not others. Greater effort is needed to derive country and context specific data for priority setting.

• Develop local capacity to conduct evaluations (including economic analysis) and empower local decision‐makers to make decisions based on this evidence.

• Participation of all stakeholders in priority setting from community representatives to high‐level policy makers. Priorities are frequently based on a small group of mid‐level policy makers.

• Greater attention must be paid to identifying areas for the redeployment of resources because many countries are currently funding high‐cost, ineffective interventions, and thereby missing opportunities for health improvement.

• At the country level, budget allocation is typically the responsibility of the Ministry of Finance (MoF) that relies on historical funding priorities. Greater ‘buy‐in’ by the MoF is required if evidence‐based priorities are to be established.

• Health system strengthening needs greater recognition in priority setting. The expected costs and effects of priority health interventions depend heavily on accompanying investments in health systems.

In conclusion, governments and donors are under mounting pressure to demonstrate greater accountability for how limited health resources are used to meet health system goals in recipient countries. Increasing efforts are therefore being made in LMICs to adopt a more explicit approach to priority setting whereby resources are allocated with the intention of maximising value for money as well as other societal objectives. This review points to a number of factors currently constraining the adoption of such an approach in LMICs. Barriers associated with poor data and limited technical capacity are not new and have been noted in other reviews of priority setting (see, e.g. Glassman et al., 2013). Incorporation of the wider practical constraints arising from the political and institutional context, building local capacity, identifying areas for disinvestment and identifying unintended consequences for the wider health system remain some of the least‐developed aspects of the priority‐setting literature and have particular relevance in resource‐constrained settings. Consideration of these constraints will help strengthen priority‐setting frameworks and encourage evidence‐based decisions on healthcare spending in LMICs.

Conflict of Interest

The authors have no conflict of interest.

Supporting information

Supporting info item

Appendix 1.

Papers Meeting Final Selection

MEDLINE M9 LIC

• 1. Baltussen R, Ten Asbroek AH, Koolman X, Shrestha N, Bhattarai P, Niessen LW. Priority setting using multiple criteria: should a lung health programme be implemented in Nepal? Health Policy Plan. 2007 May; 22 (3): 178–85.

• 2. Hansen KS, Chapman G. Setting priorities for the health care sector in Zimbabwe using cost‐effectiveness analysis and estimates of the burden of disease. Cost Eff Resour Alloc. 2008; 6:14.

• 3. Kapiriri L, Norheim OF. Criteria for priority‐setting in health care in Uganda: exploration of stakeholders' values. Bull World Health Organ. 2004 Mar; 82 (3): 172–9.

• 4. Kase P. Prioritization in the Papua New Guinea health sector: progress towards a health medium‐term expenditure framework. P N G Med J. 2006 Sep–Dec; 49 (3–4): 76–82.

MEDLINE M9 LMIC

• 5. Baltussen R. Priority setting of public spending in developing countries: do not try to do everything for everybody. Health Policy. 2006 Oct; 78 (2–3): 149–56.

• 6. Baltussen R, Stolk E, Chisholm D, Aikins M. Towards a multi‐criteria approach for priority setting: an application to Ghana. Health Econ. 2006 Jul; 15 (7): 689–96.

• 7. Chisholm D, Gureje O, Saldivia S, Villalon Calderon M, Wickremasinghe R, Mendis N, et al. Schizophrenia treatment in the developing world: an interregional and multinational cost‐effectiveness analysis. Bull World Health Organ. 2008 Jul; 86 (7): 542–51.

• 8. Diaby V, Lachaine J. An application of a proposed framework for formulary listing in low‐income countries: the case of Cote d'Ivoire. Appl Health Econ Health Policy. 2011 Nov 1; 9 (6): 389–402.

• 9. Evans DB. Lim SS. Adam T. Edejer TT. WHO Choosing Interventions that are Cost Effective (CHOICE) Millennium Development Goals Team. Evaluation of current strategies and future priorities for improving health in developing countries. BMJ. 2005 Dec 17; 331 (7530): 1457–61.

• 10. Ginsberg GM, Lauer JA, Zelle S, Baeten S, Baltussen R. Cost effectiveness of strategies to combat breast, cervical, and colorectal cancer in sub‐Saharan Africa and South East Asia: mathematical modelling study. BMJ. 2012; 344: e614.

• 11. Jehu‐Appiah C, Baltussen R, Acquah C, Aikins M, d'Almeida SA, Bosu WK, et al. Balancing equity and efficiency in health priorities in Ghana: the use of multicriteria decision analysis. Value Health. 2008 Dec; 11 (7): 1081–7.

• 12. Laxminarayan R, Mills AJ, Breman JG, Measham AR, Alleyne G, Claeson M, et al. Advancement of global health: key messages from the disease control priorities project. Lancet. 2006 Apr 8; 367 (9517): 1193–208.

• 13. Makundi E, Kapiriri L, Norheim OF. Combining evidence and values in priority setting: testing the balance sheet method in a low‐income country. BMC Health Serv Res. 2007; 7: 152.

• 14. Venhorst K, Zelle SG, Tromp N, Lauer JA. Multi‐criteria decision analysis of breast cancer control in low‐ and middle‐income countries: development of a rating tool for policy makers. Cost Eff Resour Alloc. 2014; 12: 13.

Embase LIC E1

• 15. Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi‐criteria decision analysis: a review of the literature. Pharmacoeconomics. 2014 April; 32 (4): 345–65.

Embase LMIC E1

• 16. Chao TE, Sharma K, Mandigo M, Hagander L, Resch SC, Weiser TG, et al. Cost‐effectiveness of surgery and its policy implications for global health: a systematic review and analysis. The Lancet Global Health. 2014 June; 2 (6): e334–45.

• 17. Diaby V, Laurier C, Lachaine J. A proposed framework for formulary listing in low‐income countries: incorporating key features from established drug benefit plans. Pharm Med. 2011; 25 (2): 71–82.

• 18. Simons E, Mort M, Dabbagh A, Strebel P, Wolfson L. Strategic planning for measles control: using data to inform optimal vaccination strategies. J Infect Dis. 2011 Jul; 204 Suppl 1: S28–34.

EconLit LIC

• 19. Canning D. The economics of HIV/AIDS in low‐income countries: the case for prevention. Journal of Economic Perspectives. 2006 Summer; 20 (3): 121–42.

• 20. Whittington D, Jeuland M, Barker K, Yuen Y. Setting priorities, targeting subsidies among water, sanitation, and preventive health interventions in developing countries. World Dev. 2012 08; 40 (8): 1546–68.

PubMed

• 21. Madi BC, Hussein J, Hounton S, D'Ambruoso L, Achadi E, Arhinful DK, et al. Setting priorities for safe motherhood programme evaluation: a participatory process in three developing countries. Health Policy. 2007 Sep; 83 (1): 94–104.

Other

• 22. Adam T, Lim SS, Mehta S, Bhutta ZA, Fogstad H, Mathai M, et al. Cost effectiveness analysis of strategies for maternal and neonatal health in developing countries. BMJ. 2005 Nov 12; 331 (7525): 1107.

• 23. Baltussen R, Floyd K, Dye C. Cost effectiveness analysis of strategies for tuberculosis control in developing countries. BMJ. 2005 Dec 10; 331 (7529): 1364.

• 24. Baltussen R, Smith A. Cost effectiveness of strategies to combat vision and hearing loss in sub‐Saharan Africa and South East Asia: mathematical modelling study. BMJ. 2012; 344: e615.

• 25. Chisholm D, Baltussen R, Evans DB, Ginsberg G, Lauer JA, Lim S, et al. What are the priorities for prevention and control of non‐communicable diseases and injuries in sub‐Saharan Africa and South East Asia?. BMJ. 2012; 344: e586.

• 26. Chisholm D, Naci H, Hyder AA, Tran NT, Peden M. Cost effectiveness of strategies to combat road traffic injuries in sub‐Saharan Africa and South East Asia: mathematical modelling study. BMJ. 2012; 344: e612.

• 27. Chisholm D, Saxena S. Cost effectiveness of strategies to combat neuropsychiatric conditions in sub‐Saharan Africa and South East Asia: mathematical modelling study. BMJ. 2012; 344: e609.

• 28. Darmstadt GL, Bhutta ZA, Cousens S, Adam T, Walker N, de Bernis L, et al. Evidence‐based, cost‐effective interventions: how many newborn babies can we save?. Lancet. 2005 Mar 12–18; 365 (9463): 977–88.

• 29. Edejer TT, Aikins M, Black R, Wolfson L, Hutubessy R, Evans DB. Cost effectiveness analysis of strategies for child health in developing countries. BMJ. 2005 Nov 19; 331 (7526): 1177.

• 30. Morel CM, Lauer JA, Evans DB. Cost effectiveness analysis of strategies to combat malaria in developing countries. BMJ. 2005 Dec 3; 331 (7528): 1299.

• 31. Ortegon M, Lim S, Chisholm D, Mendis S. Cost effectiveness of strategies to combat cardiovascular disease, diabetes, and tobacco use in sub‐Saharan Africa and South East Asia: mathematical modelling study. BMJ. 2012; 344: e607.

• 32. Stanciole AE, Ortegon M, Chisholm D, Lauer JA. Cost effectiveness of strategies to combat chronic obstructive pulmonary disease and asthma in sub‐Saharan Africa and South East Asia: mathematical modelling study. BMJ. 2012; 344: e608.

• 33. Hogan DR, Baltussen R, Hayashi C, Lauer JA, Solomon JA. Cost effectiveness analysis of strategies to combat HIV/AIDS in developing countries. BMJ 2005 Dec 17; 331 (7530): 1431‐7.

Other (2)

• 34. Cecchini M, Sassi F, Lauer JA, Lee YY, Guajardo‐Barron V, Chisholm D. Tackling of unhealthy diets, physical inactivity, and obesity: Health effects and cost‐effectiveness. Lancet. 2010 Nov 20; 376 (9754): 1775–84.

• 35. Chisholm D, Doran C, Shibuya K, Rehm J. Comparative cost‐effectiveness of policy instruments for reducing the global burden of alcohol, tobacco and illicit drug use. Drug Alcohol Rev. 2006 Nov; 25 (6): 553–65.

• 36. Gureje O, Chisholm D, Kola L, Lasebikan V, Saxena S. Cost‐effectiveness of an essential mental health intervention package in Nigeria. World Psychiatry. 2007 Feb; 6 (1): 42–8.

Dedication We would like to dedicate this paper to the late Professor Gavin Mooney whose many years of work on priority setting helped pave the way for this review.

Wiseman, V. , Mitton, C. , Doyle‐Waters, M. M. , Drake, T. , Conteh, L. , Newall, A. T. , Onwujekwe, O. , and Jan, S. (2016) Using Economic Evidence to Set Healthcare Priorities in Low‐Income and Lower‐Middle‐Income Countries: A Systematic Review of Methodological Frameworks. Health Econ., 25: 140–161. doi: 10.1002/hec.3299.