Assessing capacity for health policy and systems research in low and middle income countries*

Abstract

Background

As demand grows for health policies based on evidence, questions exist as to the capacity of developing countries to produce the health policy and systems research (HPSR) required to meet this challenge.

Methods

A postal/web survey of 176 HPSR producer institutions in developing countries assessed institutional structure, capacity, critical mass, knowledge production processes and stakeholder engagement. Data were projected to an estimated population of 649 institutions.

Results

HPSR producers are mostly small public institutions/units with an average of 3 projects, 8 researchers and a project portfolio worth $155,226. Experience, attainment of critical mass and stakeholder engagement are low, with only 19% of researchers at PhD level, although researchers in key disciplines are well represented and better qualified. Research capacity and funding are similar across income regions, although inequalities are apparent. Only 7% of projects are funded at $100,000 or more, but they account for 54% of total funding. International sources and national governments account for 69% and 26% of direct project funding, respectively. A large proportion of international funds available for HPSR in support of developing countries are either not spent or spent through developed country institutions.

Conclusions

HPSR producers need to increase their capacity and critical mass to engage effectively in policy development and to absorb a larger volume of resources. The relationship between funding and critical mass needs further research to identify the best funding support, incentives and capacity strengthening approaches. Support should be provided to network institutions, concentrate resources and to attract funding.

Background

Health policy and systems research (HPSR) is increasing in prominence in low and middle income countries, stimulated by social and political pressure towards health system equity and efficiency. Yet the institutional capacity to fund and produce quality research and to have a positive impact on health system development has been little examined and touches mainly on specific areas such as malaria research or the impact of research on health reforms [1-6]. In general, however, there is increasing pressure to direct research investments on the basis of evidence of policy relevance and impact [7-13]. Indeed, in this decade of efforts to link development, health and research world-wide, there is little enquiry into the role of scientific capacity in general [14].

This paper seeks to develop an empirical basis for assisting decisions on what are likely to be good investments to increase capacity in health policy and systems research (HPSR) in developing countries. It presents the results of a survey undertaken by the Alliance for Health Policy and Systems Research (Alliance-HPSR) in 2000 and 2001 to analyse institutional structure and characteristics, engagement with stakeholders, institutional capacity, level of attainment of critical mass and the process of knowledge production by institutions in low and middle income countries.

Low and middle income countries account for 4.9 billion of the world's 6 billion people. Identifying research capacity for a specific field such as HPSR in such a large context is daunting. Yet it is becoming ever more important to assess gaps and resource requirements, particularly for the low income regions of the world that are the targets of initiatives such as the Global Fund for AIDS, Malaria and Tuberculosis. Regional-level analysis is also of importance to WHO and research networks, agencies and donors with a particular interest at this level.

The Alliance for Health Policy and Systems Research http://www.alliance-hpsr.org was established with the collaboration of the World Health Organisation and the Global Forum for Health Research in November 1999 with the aim of contributing to health development and the efficiency and equity of health systems through research on and for policy. Alliance objectives include promoting research capacity on national and international issues, developing the information for policy decisions, stimulating the generation of knowledge, strengthening international research collaboration, identifying global level influences on health systems and promoting appropriate research.

Alliance activities are carried out with the support of WHO and the collaboration of regional networks. Activities focus on low and middle income countries outside WHO's European region, where other programmes supporting HPSR.are in operation (such as the European Health Observatory, http://www.who.dk/observatory). A grants program is in operation and technical analyses have focused in the area of capacity assessment, capacity strengthening and the analysis of the characteristics in the research to policy process. Technical support is provided to partners in proposal development and in the developing dissemination and research impact strategies. The Alliance encourages partnership from institutions producing, supporting or funding HPSR research, as well as from users of evidence for policy in developing countries. To date over 300 institutions have joined as partners, and direct contacts have been established with over 700 institutions.

The Commission on Health Research for Development drew attention to the importance of health research as an "essential link to equity and development" [15]. It proposed that low and middle income countries should review and strengthen the management of health research so as to meet their national needs as well as contributing to the global fund of knowledge. Furthermore, the Commission proposed that governments in low and middle income countries should allocate at least 2% of national health expenditures and 5% of externally funded programmes to research and capacity strengthening. Capacity strengthening, for the Commission, encompassed individual capacity, institutional infrastructure that supports research, the research component of policy formulation and field action, and global health research.

Following these lines, HPSR capacity is here defined as the level of expertise and resources at the researcher, project and institutional levels for the production of new knowledge and applications to improve the social response to health problems. Capacity to engage stakeholders in policy and programme development is included. Institutions are defined as groups of collaborating professionals dedicated to HPSR within a legal entity or a unit of a larger legal entity (such as a health policy research unit within a school of public health or a ministry of health).

HPSR was defined by the Alliance as knowledge generation to improve how societies organise themselves to achieve health goals, including how they plan, manage and finance activities to improve health, as well as the roles, perspectives and interests of different actors in this effort. HPSR contributes to sound, socially relevant and ethically acceptable guidance for more effective, efficient and sustainable health policies and systems. The health system functions of regulation, organisation, financing and delivery of services are the focal subjects of HPSR. Broader determinants directly affecting the health system are also considered within the purview of HPSR, such as social and economic policies affecting key health system structures and processes.

The Global Forum for Health Research classified health research and development (R&D) funding according to per capita income level of the source and recipient countries and by topic, including HPSR [16]. The analysis of HPSR funding in this study floows Global Forum's classifications and focuses mainly on international funding for national health research in low and middle income countries, and on funding from governments and private sources within countries. An important caveat, however, is that these 2 categories are difficult to distinguish in practice, since much donor funding flows through governments.

Together with funding, the question of project initiation was considered, as it reflects on the priorities followed for the research, as well as the degree of autonomy on the part of research institutions. The role of government in initiation is distinguished from that of international donors and contractors, while the role played by the research institution itself is also identified.

Methods

The assessment considers six strategic and interrelated groups of variables: institutional/country context and characteristics, institutional capacity and engagement with stakeholders, attainment of critical mass of researchers to produce quality, sustainable research, and the process of knowledge production (project portfolio characteristics, including external research project funding). While no output data were obtained to assess the attainment of critical mass or quality (such as publications), some indicators are explored as a step towards further research in this area. Most attention is given to describing capacity indicators and their relationships, mostly comparing their distribution across contrasting country contexts as measured by gross national per capita income. Given the lack of knowledge on what constitutes an ideal level or composition of capacity, indicators used should be regarded as tentative. Figure 1 summarises the key capacity variables and the indicators available to assess them.

Figure 1.

Variables and indicators in the assessment

The survey

The Alliance-HPSR recruits institutional partners on the basis of common aims towards producing and using HPSR in developing countries, irrespective of the size or legal status of the institution or unit. Partners are requested to provide a 30 question profile in order to assist the Alliance maintain a continuous assessment of capacity to produce, demand or support HPSR, including countries in both North and South. This study involves all 176 Alliance partner institutions producing HPSR in low and middle income countries. Most of the data (78%) are for 2000, although data for 1999 (8%)and for 2001 (14%) are also included. The questionnaire comprised sections on structure, projects, institutional environment and capacity development. The database was coded and cleaned prior to processing.

A total of 108 questionnaires for the 176 HPSR producers in developing countries were sent in or Web posted (61% response rate) (table 1). Biases could have occurred at two levels: the request for partnership and the response to the questionnaire. Over-representation at both levels could have occurred of more competitive and productive institutions with larger project portfolios and funding, and more interest in international funding. On the other hand, larger institutions may have been discouraged from responding given the larger number of projects to be reported, although they would also have more capacity to respond. Furthermore, the response rate could have been lower among institutions where producing HPSR is not a main function.

Table 1.

Institutions Surveyed, Contacted and Total in the Population, by Income Group

Level of Income	Surveyed				Contacted				Total in Population
	Institutions		Countries		Institutions		Countries		Institutions*		Countries
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
Low (LI) <$756	50	46	21	54	268	44	34	44	294	45	58	44
Lower Middle (LM) $756–$2,995	36	33	10	26	188	31	27	35	193	30	44	33
Upper middle (UM) $2,996–$9265	22	20	8	21	151	25	16	21	162	25	31	23
TOTAL	108	100	39	100	607	100	77	100	649	100	133	100

*Total institutions in the population were estimated through calculating the ratio of contacted institutions per population within each of UM, LM & LI countries and then expanding to the total population in each region.

Estimating the population of HPSR producer institutions

HPSR occurs at many levels and no single assessment will ever capture all the players. For this study the population of HPSR producers in low and middle income countries outside Europe was approximated by focusing on the subset of HPSR institutions that contacted the Alliance between 1999 and 2001 seeking funding opportunities and, to a lesser extent, general information. Close collaboration with 4 regional HPSR networks ensured contact with all the major HPSR producers in Latin America, Africa, South and South East Asia and China.

Between 2000 and 2002 the Alliance launched calls for brief three-page letters of intent for funding of HPSR projects, with a total response of 780 valid applications from developing country researchers. During this period, the Alliance also encouraged institutions to apply for partnership, for which 309 were recruited. As a result of these appeals a total of 607 research-producing institutions in 77 low and middle income countries outside WHO's European region were contacted (table 1), of which 176 are Alliance partners from 39 countries. It is likely that the 609 institutions are very close to the total number of HPSR producers in developing countries given the wide appeal made by the Alliance in collaboration with WHO at country level.

A total of 56 developing countries, contributing as a whole 6.6% of the developing country population, were not reached by the Alliance. Eleven of the 34 countries larger than 1 million inhabitants are in the Middle East or Central Asia and only 3 in Latin America. To ensure these 57 countries are included in the extrapolations, the expected number of institutions was estimated by assuming the same population coverage as in the countries reached within each of the three income groups (table 1). This procedure may have over-inflated the number of existing institutions, as not all countries, particularly the smaller, low-income ones, necessarily have HPSR producers. However, this may be compensated for by the under-representation of smaller HPSR producers in the larger countries that were reached.

Countries surveyed amount to 29% of the total of low and middle income outside Europe, with a somewhat greater inclusion of low income countries (which could reflect their greater interest in international resources). Institutions surveyed correspond to 17% of those expected in the countries considered, with a distribution across income groups which is almost identical. While the proportion of institutions surveyed is lower than would be desirable to reach robust conclusions, the very close match between sampled and contacted institutions across income groups lends support for both comparisons and extrapolations. None-the-less, extrapolations should be considered only tentative at this stage, and need to be confirmed through other approaches. In particular, surveying a higher proportion of institutions would provide more reliable information on the size and make-up of institutions.

Estimations

Project duration was obtained through a categorical question with 3 brackets between 1 and 24 months and one for projects with longer duration. A numerical figure was estimated by using the mid point of each category and capping duration at 2 years for the small fraction of projects at this level. The questions on funding for research enquired about project funding external to the institution, that is, financing for research received from national or international donors or from government or private agencies. Core funding to institutions was not included. Project funding was obtained for each project listed using a categorical question with six funding brackets between $0 and $99,999. Funding for projects below $100,000 was estimated by assigning to each project the middle point in its bracket. The questionnaire asked for projects funded above this figure to be indicated, and the specific funding amount and project duration were obtained through a follow-up telephone or e-mail request. Details could not be obtained for only 5 out of the 26 projects in this category, and these were assessed at $100,000 and 2 years duration.

Annual project expenditure and total annual funding were estimated by annualising total costs for projects over a year and considering all costs for projects under a year. Total human resources, number of HPSR projects and project funding were estimated by scaling up survey results by the share of institutions in the population for each income group of countries. However, this weighting was minimal given the very close match between the proportions of institutions surveyed and those in the population for each income group.

Data grouping and aggregation

Data are analysed for institutions, researcher workforce, projects and total funding. In each case, the countries' per capita gross national income is used to group and aggregate data. The categories used are those from the World Bank: Low Income Countries (LICs) with $755 or less, Lower Middle Income Countries (LMICs) between $756 and $2,995, and Upper Middle Income Countries (UMICs) between $2,996 and $9,265.

Surveyed institutions are mostly distributed in LICs (46%), with 33% LMICs and 20% UMICs (see figure 2 for list of countries). LICs with most sampled institutions are (in order) India, Bangladesh, Pakistan, Indonesia, Kenya, Uganda and Ghana. Those for LMICs are China, Colombia, Philippines, Thailand, Bolivia, Cuba and Sri Lanka; while those for UMICs are Argentina, Brazil, South Africa, Mexico, Korea, Rep., Uruguay and Chile.

Figure 2.

Countries Surveyed by Income Group

Results

Institutional characteristics and capacity

Research institutions provide a basic context for project teams and researchers through the values they embody which will vary according to the public, private or mixed legal status of institutions as well as their national or international mission. The experience of institutions indicates their research capacity, stability and ability to contribute to cross-institutional support. Staff incentives are key for retention while they, together with access to information technology, contribute to the quality of outputs. The overall qualifications of the research workforce, together with the project investment and the size of the portfolio, contribute also to the context in which projects operate and critical mass is attained.

Legal status and scope

Public institutions constitute 64% of the total, private 29% and mixed public-private 7% (table 2). One quarter of private institutions are for profit. The proportion of private institutions in the Americas is greater than in the other regions, with 40%, against only 29% and 19% for Asia and Africa, respectively. The public-private distribution of institutions across income groups is interesting: in UMICs and LICs private and mixed institutions account for as many as 46% of the total, with only 17% in LMICs (where China weighs heavily). Private institutions are significantly larger, with 13.5 researchers on average, against 8.2 for their public counterparts. Both legal types have similar shares of full time resources, although public institutions have a somewhat larger share of PhDs in their research workforce, with 24% against 14%. Half (51%) of institutions are exclusively national in scope of activities, while the other half also consider themselves active at the international level.

Table 2.

Characteristics and Capacity of Institutions Doing HPSR

	TOTAL		Per Capita GNP
			Low		Lower Middle		Upper Middle
	No.	%	No.	%	No.	%	No.	%
Total No.	108	100	50	46	36	33	22	20
Legal status
Private	31	29	17	34	5	14	9	41
Public	69	64	27	54	30	83	12	55
Mixed public/private	8	7	6	12	1	3	1	5
Experience
Less than 1 year	7	6	3	7	2	6	1	5
1 to 2	4	4	2	4	1	3	1	5
3 to 5	22	20	9	17	7	18	7	30
6 to 10	29	27	16	33	8	21	6	25
Over 10 years	46	42	20	39	19	52	8	35
TOTAL	108	100	50	100	36	100	22	100
Information technology
No/rare PC access	8	7	7	14	1	3	0	0
No Internet	5	4	4	7	1	3	0	0
FTE* researcher base
FTE per institution	8.4		9.7		7.0		8.0
No. of institutions with: 2 FTE or less	17	16	8	16	7	21	2	10
>2 to 4	22	20	8	16	10	28	4	19
5 to 10	37	34	13	26	11	31	12	52
More than 10	32	30	21	42	7	21	4	19
PhDs in workforce
Zero PhDs	26	24	7	14	7	21	10	48
1 to 2	53	49	28	57	20	55	6	29
3+	29	26	15	30	9	24	5	24
Project portfolio
Total projects	3.0		2.8		2.9		3.4
Projects <1 year	2.3		2.0		2.3		2.5
Project funding
Total for portfolio **	$155,226		$150,806		$178,636		$126,470
Annually	$80,521		$72,140		$111,879		$81,901
Databases and publications for decision makers
Producer	45	42	23	46	11	31	11	50

* FTE: Full Time Equivalent. Indicates the total number of person/months employed by including full time and part time researchers under a single category. **Includes total project funding for all projects within the institution and their entire duration.

Experience

Institutions with over 10 years of experience account for 42% of the total (table 2). Those newer to the field include 47% with 3 to 10 years and 10% with less than three years of activities. 6% of the total have less than a year of experience. There are marked differences in the experience of HPSR institutions across regions and income groups. In Africa only 21% of institutions have had more than 10 years of experience, against 46% in Asia and 59% in the Americas. LMICs have the highest proportion of institutions with over 10 years of experience (52%) while the proportion of institutions across income regions with less than a year's experience is very similar.

The research experience of the director or person responsible for the unit undertaking HPSR as measured in years of activity closely matches the experience of the research institution (corr = 0.51), suggesting that, in most cases, new institutions are not being formed by experienced researchers, but rather heads become seasoned in HPSR while in the post. The most experienced institutions tend to have larger numbers of researchers (corr = 0.20), though there is no correlation with the proportion of PhDs (corr = 0.09). The more experienced institutions also tend to have a larger funding portfolio (corr = 0.20).

Incentives

Researchers work mainly in universities and other academic environments. It is therefore not surprising that the most common career development incentive consists of opportunities to attend conferences and meetings and support for training and education (95% and 93% of respondents identify these as very or fairly important to retain personnel). Opportunities to engage in consulting are at least fairly important for 89% of respondents, a figure which is very similar for both public and private institutions. Less important, but still highly valued incentives, are bonuses (59%) and pay rises tied to productivity (64%). These pecuniary incentives are more valued in private institutions (39% of private vs. 28% of public institutions consider very important bonuses tied to productivity, while these figures are 31% and 15% for periodic pay rises tied to productivity).

Information technology

Researchers have no or rare access to personal computers in only 5% of institutions, affecting mostly LICs, with 14% of institutions in this situation. Up to 60% of institutions are well connected to the internet (all or most of their computers are linked) while 11% have no or rare access. IT infrastructure is less developed in Africa, where 19% of institutions have no or rare access to PCs for their researchers. Regarding Internet access in this geographical region, this is rare or null in 27% of institutions, against only 2% and 4% for Asia and the Americas, respectively.

Research workforce

Out of the total researcher workforce surveyed, 68% are full time and 19% have a PhD (table 3). In LICs there is a higher concentration of PhDs, with 21% of the total on average, against 18% in LMICs and 14% in UMICs. Interestingly, in Africa PhDs account for as many as 26% of the total workforce, against 20% in Asia and 14% in the Americas. These figures suggest that training is not the main disadvantage in LICs and in Africa relative to other regions.

Table 3.

Researcher Characteristics

	TOTAL	Per Capita GNP
		Low	Low Middle	Upper Middle
Total FTE researchers	909	483	251	175
% full time	68	81	55	51
Total PhD	169	100	44	25
% PhD	19	21	18	14
Project funding per researcher	$15,198	$13,426	$20,637	$12,286
Annual project funding per researcher	$7,618	$5,370	$12,035	$7,488
FTE per project	2.8	3.5	2.4	2.3

In LICs, full time dedication to research is as high as 81%, dropping to the low 50%s in the other groups. The more qualified researchers are more likely to have full time dedication: 95% for Masters and 68% for PhDs, against only 49% for Bachelors.

Total project funding per researcher is $10,300, being lowest in LICs at $8,500 and highest in LMICs at around $12,900. Project funding per researcher expressed on an annual basis is on average $6,500, lowest in LICs at $5,900 and highest at $7,300 in LMICs. These differences are explained not by funding levels per project but a by a lower number of researchers per institution and per project in LMICs relative to other regions (see below).

Research portfolio

Out of the total institutions, 11% report no research projects for the current year, although they are seeking opportunities to undertake research. On average the HPSR project portfolio contains 3.0 projects. Institutions in UMICs have an average of 3.4 projects, against 2.9 and 2.8 for LMICs and LICs, respectively. Most institutions (58%) have 3 to 4, while 29% have 1 to 2. Only 13% have more than 4 projects. As a whole, 76% of the project portfolio is of less than one year's duration, a figure which is as high as 80% for LMICs.

Based on the proportion of projects under one year's duration it can be estimated that institutions initiate on average 2.3 projects per year, a high number when it is considered that there are 3.7 researchers per project initiated and 0.7 PhDs (table 3), and that for each funded project there will be several unsuccessful proposals. In LMICs there are 3.0 researchers per project initiated, as against 4.5 for LICs.

The average institution's research portfolio is worth $155,000 in project funding, of which $80,500 is spent annually. Average portfolio funding is similar across income regions. Most institutions (45%) have a portfolio worth less than $50,000, 15% between this figure and $99,000, 25% between $100,000 and $199,000 and only 15% have a portfolio worth over $100,000. However, two institutions in Colombia and one in Pakistan have extremely high research portfolios, worth over $1 million.

Attainment of critical mass

Attainment of critical mass can be defined in theory as the grouping of a minimum number of researchers within a single institution with the right mix of qualifications and disciplines as to be able to produce quality HPSR in a sustainable way. In practice, there are no clear standards of what constitutes a minimum number or right mix, so analysis here is tentative. Clearly, the institutional and country contexts will influence the attainment of critical mass.

Size of researcher work force

Each institution relies on average on 8.4 researchers, ranging from 9.7 in LICs to 7.0 in LMICs (table 2). While only 19% of institutions in LMICs and 21% in UMICs are larger than 10 researchers, 42% of those in LICs are in this category (figure 3). However, many of the smaller institutions constitute units within larger universities or agencies that may provide other support to achieve critical mass.

Figure 3.

Number of HPSR Researchers per Institution, by Income Group

Looking at the number of PhDs, only 26% of institutions have at least three full time equivalent at this level (figure 4). A total of 49% have between 1 and 2 and 24% have none. While the proportion of institutions with 3 or more PhDs is similar across income groups, that for institutions with no PhDs is higher in UMICs (48%), against LICs with only 14%. The smaller the institution, the larger the proportion of PhDs (Corr = -0.21), raising concerns as to the extent to which those with PhDs are training up a cadre of junior researchers.

Figure 4.

Indicators of Critical Mass in HPSR Producer Institutions

Disciplines available

Institutions report a range of disciplines available, on average 9 with 20% having between 1 and 5, 44% 5 to 10 and 35% more than 10 (figure 5). However, most institutions (54%) report more disciplines available than contracted researchers, indicating that staff are normally considered as qualified in more than one discipline. There is no difference in the average across income regions.

Figure 5.

Availability of Disciplines and of PhDs in HPSR Producer Institutions

As expected, public health is the discipline most frequently available within institutions, with 77% of them having at least one researcher qualified in this field. Interestingly, economics, statistics and management – all disciplines very close to HPSR- are present in approximately two-thirds of institutions. Other highly relevant disciplines such as sociology and public administration are found in 52% and 49% of institutions, respectively. Anthropology, political science and psychology are present only in around one-third of institutions. Better staffed disciplines tend to have better qualified researchers (Corr = 0.17).

Knowledge production process

Out of the total projects surveyed, 44% are in LICs, 33% in LMICs and 23% in UMICs (table 4). Projects have an average duration of 14.3 months, and are of somewhat longer duration in UMICs with 16.6 months on average, against 12.5 in LMICs and 14.5 in LICs. Only 24% of projects have a duration of two years or more, while 34% have a duration of between one and two years. Projects of less than a year amount to 42% of the total. Project duration and amount of funding are fairly correlated, as would be expected (Corr. = 0.37). On average 76% of projects are initiated per year, given their short duration. The total number of researchers per project is 2.8 overall, being highest for LICs at 3.5 and similar in the other two regions.

Table 4.

HPSR Project Characteristics

	TOTAL		Per Capita GNP
			Low		Low Middle		Upper Middle
	No.	%	No.	%	No.	%	No.	%
Total projects	321		140		106		75
Duration average (months)	14.3		16.6		12.7		16.9
FTE per project	2.8		3.5		2.4		2.3
% projects over 1 year	76		71		79		72
Project funding
% with project funding	96%		92%		99%		97%
Average all sources	$47,638		$51,277		$49,338		$37,083
Internationally funded	$43,215		$61,728		$51,455		$35,446
Government funded	$54,148		$10,476		$58,356		$46,250
Privately/other funded	$23,600		$43,250		$5,000		$21,500
Annually per project	$27,030		$24,816		$31,141		$24,725
Project initiation by source, within groups
Donor/contractor/research partner	99	31	45	32	23	21	29	38
Government	78	24	30	22	40	38	11	14
Own institution	108	34	54	38	31	29	23	30
Mix/Other	37	12	11	8	12	12	13	17
Projects funded by source, within groups
International	193	60	101	72	47	44	47	62
Government	96	30	27	19	48	45	20	27
Private & Other	33	10	13	9	11	11	8	11

Project funding

96% of projects report a relationship with a funding agency outside the institution, while 91% provide specific amounts. Of them, 91% had only one source and the rest had a combination, mostly of three sources: international donors, national government, and private.

Total project funding is $47,600 on average, with an annual flow of funds of $27,000. The differences across income regions for total average funding per project is of 29% across the extremes, favouring LICs. Differences for annual project expenditure are less pronounced, with LMICs showing the largest amount. This is mainly due to the longer duration of projects in LIC, against the shorter term in LMIC.

Almost a third of projects (30%) received grants below $10,000, while 27% received between $10,000 and $24,999 (figure 6). Only 7% of projects are funded at or above $100,000, although they account for 54% of total funding. Only 17% of institutions hold these large grants. LICs tend to receive a larger proportion of smaller grants and fewer of the larger ones, although differences are otherwise small across income regions.

Figure 6.

Range of Project Funding by Income Group

Capacity development

Among the activities undertaken by institutions to increase their capacity, raising the awareness of policy issues and processes by researchers is the one most often undertaken and with the lowest rate of failure (figure 7). In contrast, securing stable sources of funding for HPSR, although frequently tackled, is considered unsuccessful by 28% of respondents on average. The figure is higher for LICs and UMICs, at 46% and 50%, respectively.

Figure 7.

Undertaking and Succeeding in Capacity Development Strategies

Engagement with stakeholders

External Boards

The engagement of external boards or advisory bodies is high, being a practice in 70% of institutions. The range of participation of stakeholders varies widely but it is generally narrow: out of 11 key actors, only 29% are represented on average (Key actors are: consumer/community groups, funding agencies, government health providers, international advisors/experts, ministry of finance or equivalent, ministry of health, other government agencies, other stakeholders, own research staff, academic institutions, private/NGO health providers). Health authorities and own staff are most often included in these advisory or governance mechanisms (35% of cases), followed by government, international experts and other government bodies (25% of cases). Financing agencies and NGOs are included in only 8% and 7% of cases, respectively.

External influence on project portfolio

The research institution is the initiator in only 34% of projects, while 31% are initiated by a donor agency, international research partner or by a private contractor. Governments initiate in 24% of cases (table 4). 12% of projects are reported as a mix of the above. Interestingly, in UMICs, governments initiate projects in only 14% of cases, against 22% in LICs and as high as 38% in LMICs. Furthermore, initiation by own institution is higher in LICs, with 38%, against only 30% in UMICs. In this latter group of countries, institutional engagement in private contracting and a mix of modalities is more prevalent than in the other regions.

International donors are the most important influence in terms of number of projects funded, with 60% of the total, while governments fund 30% and private and other sources 10%. However, governments fund at highest levels, with $54,148 per project on average, followed by international donors with $43,215 and private and other sources with $23,600. (As indicated earlier, governments may use donor funds). International donors provide larger grants on average in LICs ($61,728) than in the other two regions, contrasting with the much lower size of government funding. Grants in LICs have longer duration than in other regions and are therefore lower in terms of annual spending. LMICs have the greatest share of projects funded by governments, with 45% of the total, although these are of shorter duration. LMICs appear able to attract larger grants on average from international donors when compared to UMICs, while relying on a larger proportion of government grants. The share of privately funded projects is similarly low across regions, but the amounts vary widely, being lowest in LMICs.

Impact on policy making

With respect to perceptions of the impact of research on policy making, only 34% of respondents consider this below expectations and 2% believe they are not successful in ensuring awareness of research results and recommendations by stakeholders. In spite of the marked client orientation of research projects as evidenced by source of project funding, only 46% of institutions target special databases or publications to policy makers, suggesting a rather academic orientation of final products and a weak provider-client relationship even when projects are government-financed.

Activities to improve stakeholder engagement

Respondents were asked specifically to confirm engagement in a series of activities and to rate their success (figure 8). Ensuring awareness of research results and recommendations by policy stakeholders is the activity most often undertaken and with the lowest reported rate of failure. Assessing the impact of research on policy and gaining community-wide recognition for the institution/unit as producers of high quality, objective HPSR are the activities least undertaken and reported to be least successful. The data suggest that respondents are generally confident of their efforts to strengthen stakeholder engagement, although this view, placed in the context of other findings, suggests rather limitations in strategic planning and an overoptimistic view.

Figure 8.

Strategies to Improve Stakeholder Engagement

Extrapolations and Total Funding

Researchers and projects

With the caveats stated in the methodology, the results were extrapolated to the estimated population of 649 HPSR producer institutions in low and middle income countries identified by the Alliance (table 5). The total workforce dedicated to HPSR can be estimated at 5,471 researchers. The researcher density is lowest in LMICs, with 0.7 researchers per million inhabitants, against 2.2 in UMICs, a threefold difference, and 1.2 in LICs. The total PhDs are 1,009, of which 588 would be in LICs. There would be in total 1,942 projects, of which 1,476 are initiated per year. Out of all active projects, donors or contractors initiate the highest number, at 598.

Table 5.

Total Resource, Project and Funding Extrapolations

	TOTAL		Per Capita GNP
			Low		Low Middle		Upper Middle
	No./ $	%	No./ $	%	No./ $	%	No./ $	%
Total researcher FTEs	5471	100	2841	52	1342	25	1288	24
Researchers per million	1.1		1.2		0.7		2.2
Total PhDs	1009	100	588	58	235	23	186	18
Total projects	1942	100	823	42	568	29	551	28
Total initiated per year	1476	100	625	42	454	31	391	27
Project initiated by
Donor/contractor/research partner	598	31	266	32	121	21	210	38
Government	469	24	177	22	215	38	79	14
Own institution	651	34	317	38	165	29	166	30
Mix/Other	224	12	63	8	66	12	96	17
Number of projects funded, within groups
International	1151	59	575	70	251	44	347	63
Government	594	31	170	21	257	45	143	26
Private & Other	198	10	78	9	60	11	61	11
Project funding*	$91	100	$40	44	$30	33	$21	23
Within country groups
International	$62	68	$36	89	$14	46	$12	60
Government	$24	27	$2	4	$16	53	$7	34
Private & Other	$5	5	$3	7	$0	1	$1	6
By sources
International		100		58		22		20
Government		100		6		65		29
Private & Other		100		64		7		29
Annual funding	$58	100	$23	41	$21	37	$13	23
By sources
International	$39	100	$21	53	$10	25	$8	22
Government	$16	100	$1	9	$11	67	$4	23
Private & Other	$2	100	$1	51	$0	11	$1	37
Annual per capita HPSR funding × 10,000
International	$80		$85		$53		$144
Government	$33		$6		$58		$64
Private & Other	$5		$5		$1		$15
Total	$117		$96		$113		$223
HPSR project funding as % of THE**	0.007%		0.013%		0.007%		0.005%

All funding figures in millions of dollars except where noted. *Includes total funding for the duration of the project. **THE: Total Health Expenditure. Source: WHO National Health Accounts for 1998.

External funding

The research portfolio in the total population of institutions is estimated to be worth $91 million. Of this total, international donors fund 68% or $62 million, governments 27% ($24 million), and private and other (national) sources 5% ($5 million). Direct funding by international donors dominates in LICs, with 89% of the total against 4% from governments and 7% from private sources. International funding is also large in UMICs although more balanced with government funding (60% vs. 34%), while in LMICs governments are the predominant source, contributing 53% of project funds.

Annual project funding can be estimated at $58 million, with international donors accounting for $39 million, governments for $16 million and private and other (national) sources for $2 million. International donors provide 53% of their annual resources to LICs, 25% to LMICs and 22% to UMICs. Of total government funding, 67% is allocated in LMICs.

Equity of funding distribution

LICs spend on average $96 per 10,000 inhabitants, LMICs $113, and UMICs $223. The main explanation for differences between the three regions is government funding, as this source is estimated to contribute $6 per 10,000 inhabitants in LICs, against $58 in LMICs and $64 in UMICs, a 10 fold difference between the extremes. International sources help to redress the balance between LICs and LMICs, spending $85 per 10,000 in the former and $53 in the latter, in spite of the lower average project funding offered to LICs. However, international funding further compounds inequality when UMICs are considered, which receive $144 per 10,000 inhabitants.

The share of research project funding relative to total health expenditure is 0.007% for developing countries in general, with 0.007% and 0.005% for LMICs and UMICs, respectively, and, interestingly, 0.013% in LICs, about three times higher than in UMICs. This is explained mostly by the much lower annual per capita health expenditure in LICs, at $76 as compared to $240 for the other two regions combined. The funding effort in LICs vis-à-vis what would be expected from per capita health expenditure is therefore greater by a magnitude of five.

Discussion

Methods

The findings of this study must be interpreted in the light of the data and methods used. The data were obtained from a questionnaire completed by institutions and followed up only for projects with funding in excess of $100,000. Although the questionnaire was piloted beforehand and simplified to the extent possible, the nature of the information sought on research projects and funding may have been difficult to complete, especially for the larger institutions and those which lack good information systems. It was not possible to distinguish between funding for research by governments derived from national or international sources. The questionnaire focused on key features of the research institutions and of research projects, potentially omitting important aspects of institutions such as external support to capacity development not linked to research projects. Indicators of capacity must be regarded as tentative, given lack of knowledge of what constitutes an ideal level of capacity. Moreover, the data provide a cross-sectional picture, thus limiting the extent to which they can provide evidence of changes over time in institutional size and characteristics. Finally, although the number of countries and institutions represented in the income and regional groupings were reasonably substantial, it is nonetheless possible that some of the country grouping comparisons are affected by the particular characteristics of the countries represented, most notably China and Colombia in the case of LMICs.

Institutional and country context

In spite of the evident activity across a large number of developing countries, HPSR is still at very low levels as apparent from the low researcher density and low spending as a proportion of total health expenditure. However, reference standards are lacking for a more objective assessment. It is of concern that the total annual HPSR funding estimated in this study represents only 0.007% of health expenditure, a ratio that would not be altered significantly even allowing for a large downward error and for investments in the North directly benefiting the South. The Commission for Health Research for Development recommended that total health research expenditure in the South should be at the same level as developed countries with respect to total health expenditure, that is, at about 2%. If HPSR accounts for 5% of this total, a figure very likely below what occurs in practice, this norm would place HPSR at 0.1% of total health expenditure. Current HPSR expenditure at 0.007% is then 14 times below this norm. Even if all of the research funding available through multilateral agencies were to be disbursed and spent within developing countries (see below), actual spending would be 0.012%, that is 8 times below the stated norm of 0.1%.

If core funding to institutions were included, actual spending would increase. While it is not possible to estimate by how much, it is worth considering that project funding is likely to be over 50% of total funding, given that average project funding per researcher is higher than annual salaries.

Institutional characteristics and capacity

HPSR producer institutions are generally small. The proportion of institutions under one year old, about 6%, suggests an expanding field. Private non profit and mixed public private entities have an important role in the constellation of HPSR producers, particularly in upper middle (mainly Latin America) and low income countries, where they account for almost half the total. While a more detailed analysis is warranted, the somewhat larger research workforce of private institutions points to their capacity to attract resources and to be sustainable.

LICs, and particularly African institutions, are the least experienced and thus have the least intra-regional peer support. Furthermore, new institutions are generally led by relatively inexperienced researchers. The longer institutions have been in existence, the greater tends to be both the number of researchers and the size of the project portfolio, though growth appears to involve the recruitment of less qualified researchers rather than researchers with PhDs. Access to computers is a major concern in LICs and reliability of internet access should be the subject of further study.

It is interesting to note the higher proportion of full time and PhD-qualified researchers in LICs relative to higher income countries. Further investigation is required to examine to what extent this is influenced by demand side versus supply side factors. On the one hand there may be a lack of competing job opportunities for researchers trained at the PhD level in this region, meaning that those trained are retained by their institutions; on the other hand there may be greater access to funding for PhD studies in LICs. It is encouraging that LICs do not have the lowest researcher to population ratio. However, project funding per researcher does not reward the greater capacity suggested by the data, being in LICs only 2/3 the value of that of LMICs. The fact that the proportion of PhDs is inversely correlated to the size of the researcher workforce may suggest that PhDs favour smaller institutions, possibly offering better pay and entrepreneurial opportunities. The data suggest a trend where newly qualified PhDs are establishing small yet dispersed research and consulting units in both the public and private sectors.

Attaining critical mass

With only 26% of institutions having 3 or more PhDs (tentatively about 170 in developing countries as a whole) and only 15% with research portfolios over $100,000, most institutions appear to be far from achieving what might be regarded as a critical mass. It is encouraging that institutions in LICs have a significantly larger concentration of researchers and somewhat higher proportion of PhDs and that project teams tend to be larger than in higher-income countries. However, the encouraging findings on the higher proportion of PhDs in LICs relative to other regions has to be tempered with Alliance funding experience that suggests that PhDs across all regions are not more successful in obtaining research funding than researchers with only Masters' training [17]. Furthermore, a large number of institutions in LICs suffer from important information technology restrictions.

It is encouraging that the key disciplines for HPSR of economics, statistics and management are present in at least two thirds of institutions across income groups, and researchers specialized in these subjects tend to be better qualified.

Efficiency in the knowledge production process

In view of the low concentration of key resources, it is troubling that most projects in the research portfolio are of short duration, with an average of just over a year, and that it is therefore likely that much of the research portfolio is being renewed on a yearly basis. Furthermore, there are few researchers per project initiated. The capacity to produce high quality research is likely to be affected by the need to prepare new applications. Indeed, the low salaries in low income countries require researchers to seek a constant flow of projects in order to generate an income sufficient to live on. Projects are not only of short duration, particularly in LMICs, and involving relatively few researchers per project, but also small in terms of funding at $27,000 per project per year on average.

The estimates of total project funding, at $90.5 million for the project portfolio and $58 million on an annual basis, have to be considered as preliminary pending more accurate estimations, although the error is not likely to be too large. International donors account for about 68% of the total. Their higher share of 89% in LICs offsets the lower contributions by governments and pushes funding up to achieve fairly similar average project funding across the three regions and fairly similar per capita funding across LICs and LMICs. Anecdotal evidence suggests that costs for undertaking similar research are higher in Africa and Latin America than in Asia.

Bilateral overseas development assistance is the most likely source for the international funding. While international foundations also play an important role in funding health research, it is likely that much of their funding is on disease or population-group-specific health services research rather than on system wide issues. Multilateral funding, particularly by development banks and other agencies, would be mostly disbursed through national governments and thus would account for an unknown proportion of the amounts reported here as coming from national governments.

Bilateral assistance for health research of all types in developing countries has been estimated in other studies at $350 million for 1998 [16]. The annual sum estimated for international HPSR funding in this study, at $36 million, would account for 10.3% of these sources (if they were maintained at the same level for 2000/01), although this percentage would be somewhat lower if the contribution by foundations was added to the bilateral total.

Project funding by governments to HPSR producer institutions in developing countries may account for only a small part of the funding actually available to them for HPSR. The amount estimated here of $16 million spent annually is much lower than the amounts that development banks earmark for HPSR as part of their health lending. World Bank lending for health research was estimated for 1998 at $55.8 million, or 4.7% of total health lending approved for the year. Most of these funds are earmarked for policy and health systems research by developing country researchers and institutions [[16], p 36]. (While a small amount of the $55.8 million funding would go to European countries as well as for research areas outside HPSR, there are other multilateral funding sources not considered which would tend to compensate for this). Therefore, there is about 3 times as much funding earmarked for HPSR from government sources than is actually identified here as spent through national institutions. If this gap were closed, total HPSR spending at the national level would more than double.

It may be that this multilateral support for HPSR is either spent outside the country through contracting agencies in the North or is not spent in spite of being earmarked within development projects. Indeed, the small size and low critical mass of most national institutions would often not be appropriate to compete with the North nor to meet the administrative requirements associated with such contracts. Anecdotal evidence suggests that a large part of government resources made available by multilateral institutions and earmarked for HPSR may go unspent due to the low priority to research assigned by decision makers, the lack of capacity to undertake the competitive tendering required and the lack of competitive bidders [18,19].

Engagement with stakeholders

In spite of the marked engagement of external actors at the project level, stakeholders are being involved at the institutional level only modestly, with a limited range of actors, little influence at policy levels and low production of tailor-made databases and publications directed to decision-makers. Despite this, institutions are in most cases confident of their success in implementing strategies to relate to stakeholders and to obtain funding. This perception warrants further research as it may rather reflect a low level of strategic planning and of expectations. The near total absence of international and private actors on advisory boards is noteworthy.

The relative influence of donors and contractors on project activity can be appraised in terms of their level of involvement in project initiation. In general, for every one project initiated by international donors or contractors, they fund 1.9 projects. This indicator is similar across the three regions, ranging from 2.2 in LICs to 1.7 in UMICs. Thus local project initiation, especially by the research institution, is more common than might be expected from the pattern of research funding. Furthermore, project initiation by international donors/contractors vis-à-vis other actors is relatively similar across the three regions, suggesting that HPSR in LICs is no more donor-driven than in better-off countries. However, international donors have a higher level of project funding in LICs, helping offset lower contributions by other actors and leading to similar project activity levels between LICs and middle income countries combined.

In spite of the fact that government policy makers and service providers will be the main beneficiaries of research, they fund only a third of total projects, although this proportion is higher in LMICs. This may be possibly due to the larger demand from health sector development projects in LMICs. Private funding is modest but significant and at a similar level across regions.

Notwithstanding the impact of donors on the distribution of research funds by region and their support to government and institution-initiated projects, the predominantly small grants funding is unlikely to encourage the kind of longer duration, quality research conducive to the establishment of specialised programmes of national and international significance and with capacity to relate to policy makers. On the other hand, it needs to be acknowledged that the small size and low capacity of most institutions would not allow larger grants. Partnerships between institutions such as those advocated by the Alliance might overcome part of this problem.

Conclusions

This paper has identified a low volume of funding for HPSR in developing countries. It is likely that the main problem is not availability of funds per se, but rather the constraints imposed by the weak institutional capacity and lack of critical mass of most institutions. Nonetheless, appropriate project funding is crucially related to capacity strengthening, as it stimulates training and institutional development as well as provides the critical experience to consolidate research skills.

Current patterns of project funding, on average characterised by short term efforts and $27,000 per annum funding, mainly from international donors, seem to be associated with small institutions, a good proportion of which are in the private sector. Such small units may be failing both to achieve critical mass for quality, sustainable research and to relate effectively to government policy development needs. Most institutions are further burdened with the imperative of renewing their project portfolio almost on a yearly basis. Further research is required to identify the public-private collaboration and networking which, evidence from Latin America suggests [20,21], may place larger private entities in an advantageous situation.

In spite of substantial funding by governments in LMICs and UMICs, this source has the greatest potential to increase allocations given the resources available from multilateral agencies for HPSR as part of their health sector lending. This source has the greatest capacity to break the vicious circle between small institutional size and small grants, as well as to allocate funds equitably across regions. However, such funding by itself would not be enough to ensure the most effective involvement by the research community in policy development. This requires increasing the capacity of both governments and researchers to demand and provide scientific evidence.

From a methodological perspective, the study points to the usefulness of indicators such as the number of researchers and their qualifications, proportion of PhDs and the range of disciplines. The cut-off points here suggested as indicative of critical mass are very tentative. None-the-less, it is revealing that while the range of disciplines reported is quite broad for many institutions, the concentration of researchers, the number of PhDs and their proportion relative to the total researcher workforce are more helpful in categorising institutions. The impact of these variables on project portfolio funding and particularly on publications and engagement with stakeholders needs to be further analysed. Furthermore, the interaction between these indicators and country context needs to be assessed on the basis of more solid outcome indicators. For example, greater researcher concentration and similar levels of PhD training in LICs as compared to other regions may not be sufficient to ensure similar quality and impact of research, particularly given lower government funding.

Barriers to disbursement of government funding and to effective stakeholder engagement need to be further analysed to understand the role of various factors at play, including the design of research as a part of health sector development projects, the capacity to identify and target research needs, the role of competitive tendering and peer review of research proposals, institutional capacity to submit quality proposals and the role of international technical co-operation. In particular, other evidence suggests the importance of strategically integrating research into the health system functions of stewardship and service delivery to ensure government support for research [18,22,23].

International funding, advocacy and technical support agencies need to identify the best roles and opportunities for collaboration to offer the right incentives and to develop appropriate programmes which support the consolidation of HPSR demand and supply capacity in developing countries.

Competing interests

None declared.

Authors' contributions

MAGB and AM designed the conceptual framework for the project. MAGB designed and implemented the methods and processed results. MAGB and AM interpreted the data and drafted the discussion and conclusions.