Protocol: a realist review of user fee exemption policies for health services in Africa

Abstract

Background

Four years prior to the Millenium Development Goals (MDGs) deadline, low- and middle-income countries and international stakeholders are looking for evidence-based policies to improve access to healthcare for the most vulnerable populations. User fee exemption policies are one of the potential solutions. However, the evidence is disparate, and systematic reviews have failed to provide valuable lessons. The authors propose to produce an innovative synthesis of the available evidence on user fee exemption policies in Africa to feed the policy-making process.

Methods

The authors will carry out a realist review to answer the following research question: what are the outcomes of user fee exemption policies implemented in Africa? why do they produce such outcomes? and what contextual elements come into play? This type of review aims to understand how contextual elements influence the production of outcomes through the activation of specific mechanisms, in the form of context–mechanism–outcome configurations. The review will be conducted in five steps: (1) identifying with key stakeholders the mechanisms underlying user fee exemption policies to develop the analytical framework, (2) searching for and selecting primary data, (3) assessing the quality of evidence using the Mixed-Method Appraisal Tool, (4) extracting the data using the analytical framework and (5) synthesising the data in the form of context–mechanism–outcomes configurations. The output will be a middle-range theory specifying how user fee exemption policies work, for what populations and under what circumstances.

Ethics and dissemination

The two main target audiences are researchers who are looking for examples to implement a realist review, and policy-makers and international stakeholders looking for lessons learnt on user fee exemption. For the latter, a knowledge-sharing strategy involving local scientific and policy networks will be implemented. The study has been approved by the ethics committee of the CHUM Research Centre (CR-CHUM). It received funding from the Canadian Institutes of Health Research. The funders will not have any role in study design; collection, management, analysis, and interpretation of data; writing of the report and the decision to submit the report for publication, including who will have ultimate authority over each of these activities.

Article summary

Article focus

• Demonstrate how a realist review can provide new insights on user fee exemption policies in Africa.

• Highlight the foundations of the realist approach and explicit its key principles.

Key messages

• Traditional systematic reviews fail to provide valuable lessons when it comes to understand how the context and the mechanisms of a policy interact to produce outcomes.

• The realist approach is a promising way for synthesising scientific and grey literature to understand the black boxes of complex social interventions, such as health policies.

Strengths and limitations of this study

• Innovative way to systematically review literature on user fee exemption policies.

• Inclusion of grey literature in the review.

• Geographic area limited to Africa.

Background

User fee exemption in African countries

Several low- and middle-income countries (LMICs), specifically countries in Africa, have removed user fees for health services in order to improve accessibility to healthcare for vulnerable populations. This move follows the setback of the Bamako Initiative (BI), which was launched in 1987. Initiated in the spirit of the Alma Ata Declaration, the BI—promoted by the United Nations Children's Fund (Unicef) and the WHO—aimed to improve access to primary healthcare and essential medicines in countries where the public health sector was facing serious underfunding problems and was characterised by a strong emphasis on hospital-based medicine. Thanks to partial cost recovery through users' financial participation and the involvement of the community, the BI was intended to improve the efficiency and the equity of healthcare services in LMICs.¹

However, utilisation of health services declined sharply in the countries that subscribed to the BI, especially among vulnerable populations.^{2 3} Measures to ensure that the poorest members of society had access to health services were also seldom found to be effective.⁴ The principle of equity in primary healthcare access was therefore severely compromised.^{5 6}

In an attempt to address these failures, some African countries—such as South Africa in 1994 and Uganda in 2001—decided to eliminate user fees. Others, including Burundi, Liberia and Niger, followed suit. Concerned about the Health Millennium Development Goals, many African countries now feel encouraged to provide free care at the point of service,^{7 8} and more than 15 countries had implemented such policies by 2011. Several studies have been conducted on exemption policies. In the most recent literature review, Ridde and colleagues⁹ identify 32 scientific articles on the experiences of seven African countries. In addition, many evaluations have been conducted by non-governmental organisations or on behalf of African governments. Although the recent proliferation of research comes in response to the urgency of the debate on user fees, it does not provide the necessary guidance for decision-makers as they try to adapt these policies to their objectives, target populations and local contexts.

The challenges of evaluating complex social interventions

User fee exemption policies are inherently complex interventions.¹⁰ They are dependent on the context in which they are implemented, and their implementation is not standardised.¹¹ They also have a strong social component, in that they are designed to promote and protect the health of populations and to reduce inequalities.

Evaluating complex social interventions raises further questions about scientific research methods. The experimental approach seems to have reached its limits in understanding these interventions. Because it seeks to control contextual variables and to ensure that ‘all else is equal’, it does not capture the complex nature of interventions whose outcomes, by definition, depend on the context in which they are implemented.¹² By contrast, the constructivist approach perceives social interventions as a complex process of negotiations between different actors. Such a vision seldom recognises the asymmetry of powers between actors in a society and often denies the role of structures in human choices.¹³ In addition, by refusing any principle of causality, this approach does not address the question of the outcomes of an intervention in relation with processes.¹⁴ More and more authors now support a ‘contingent’ approach, implying that the choice of methods should be guided by the research question.^{12 15}

As an alternative to this methodological debate, Chen and Rossi¹⁶ suggest a theory-based perspective. They believe that theorisation allows a better understanding of how interventions, and social phenomena in general, are supposed to work. Such an approach, which focuses on ‘black boxes’ of programs, would capture the complexity of social interventions by studying how the different theoretical elements that compose them are intertwined^{17 18} and by reintegrating the context as a key element in the production of outcomes. In addition, from an Evidence-based Policy (EBP) perspective, shedding light on the multiple logics that underlie policies would be more useful to decision-makers.

By extending the scope of Evidence-based Medicine, EBP meets the growing demand that political decisions be justified through lessons learnt from past experiences. The theory-based approach would allow knowledge of the different contexts in which an intervention works to be broadened,^{18 19} and the accurate level of abstraction with which to generalise the results of research to be achieved.²⁰ Thus, it would ensure the external validity of evaluation studies and make the results transferable to other contexts. By providing insights into how programs operate and can be implemented in different contexts, this type of evaluation would be better able to meet the expectations of decision-makers.²¹

Based on this approach, the realistic evaluation suggests that we set aside the traditional question of the effectiveness of interventions and instead investigate how they work.¹³ The question is no longer: Do interventions work? But also: How? For which populations? And in what contexts?

Syntheses and systematic reviews: the tools of EBP

Two documents have been produced to inform technicians and decision-makers in charge of formulating and/or implementing exemption policies in their countries. In 2008, Save The Children UK (STC-UK) published a guide to help with the planning and implementation of exemption policies.²² In 2009, Unicef published a Policy Guidance Note,²³ based on evaluations of exemption policies conducted in several African countries.²⁴ These documents have limitations, however. First, the document by STC-UK is only based on available data on the Ugandan experience, which limits the scope of its arguments, especially since it specifically states that these data are not homogeneous. In addition, its recommendations are more common sense than the result of a systematic analysis of the variables involved in the exemption processes. Finally, the recommended steps focus on macro-level planning and implementation and leave out key elements for the success of policies of this scale, such as the mechanisms at work or the contextual elements that come into play. The Unicef document partially complements the STC-UK guide because its recommendations come from a more in-depth analysis, addressing the issue of context and process of policy formulation more precisely. However, it is a technical ‘toolbox’ with a relatively normative perspective. While informing decision-makers on what elements to consider in formulating policies, it does not give them any indication on how these elements interact with the context.

Ridde and colleagues conducted two literature reviews on exemption policies, using the scoping study method. The first review gives an outline of the scientific knowledge on this issue and draws attention to research needs.²⁵ The authors attempt to make some recommendations; however, this method presents a risk of overinterpretation of data. It also has methodological limitations: on the one hand, the quality of the studies was not assessed, while on the other hand, by deciding to focus solely on scientific publications with peer review, the authors excluded a large amount of contextual and informal knowledge. The objective of the second literature review is to highlight the pressures exerted by exemption policies on health systems.²⁶ The criticisms to the first review also apply here. Additionally, this second review is not intended to guide decision-makers by responding to their expectations on the operation and implementation of policies. Therefore, these documents are of limited interest for policy-makers and international organisations, especially in a context where the idea of ‘evidence-based’ policy predominates.

Meta-analysis and systematic reviews are the favourite tools of EBP supporters and decision-makers, for the good reason that “[…] good evidence syntheses free them up to concentrate on the other aspects that go into policy-making […].”²⁷ A systematic review was published in 2011 by Lagarde and Palmer²⁸ evaluating the effectiveness of different schemes of healthcare funding. Only randomised controlled trials, interrupted time series studies and controlled before-and-after studies were included, in accordance with the EPOC (Effective Practice and Organisation of Care Group of the Cochrane Collaboration) method which the authors used to assess the quality of research design. Based on these criteria, the researchers only included five studies, all of which were deemed to be of low quality. They explain: “We considered the evidence on the removal of user fees to be at high risk of bias. In particular, the presence of confounding factors (concurrent policy changes), the lack of reliability of routine data and limited sample sizes weaken the evidence base.”²⁸ This systematic review reveals the limitations of the traditional way of synthesising scientific knowledge when it comes to complex social interventions. Considering only knowledge produced through methods limiting bias and random errors,²⁹ studies using so-called less robust designs, including research designs used in social sciences, were excluded, even though they provide valuable scientific data.³⁰ By focusing on the effectiveness of interventions, such systematic reviews do not take into account the process and contextual elements, which allowed the interventions to produce outcomes.

The need to deal with complex social interventions has led to changes in these synthesis tools.^{31 32} Systematic reviews in fact failed to keep their promises in terms of transferability to other more social issues.³³ New forms of reviews, which take into account the qualitative data³⁴ and also the combination of quantitative and qualitative data, are emerging.³⁵ As noted by Jackson and colleagues,³⁶ it is no longer the hierarchy of evidence that must guide the selection of studies to be included in the review, but their usefulness in answering the research question. As such, mixed approaches seem promising. Pope and colleagues³⁷ identify four mixed review approaches: the narrative approach, the thematic approach, the EPPI (Evidence for Policy and Practice Information) approach and the realistic approach. According to Pluye and colleagues,³⁸ only the last two involve a systematic review process. The EPPI approach juxtaposes several reviews to answer different subquestions, which together make up a very broad main research question. The final step of this type of review is to combine the results of the ‘subreviews’ in a meta-synthesis.³³ This process requires a significant amount of time to complete and the availability of several researchers.³⁷ Moreover, it is said to yield limited results in terms of the meta-synthesis. Proposed by Pawson in line with the realistic evaluation,¹³ the realist review aims to develop middle-range theories that take into account how the context (C) influences mechanisms (M) to produce outcomes (O).^{39 40} That is what Pawson calls C-M-O configurations. It is the only review that proposes a systematic integration of contextual analysis in order to better understand how interventions produce outcomes.

Methods

The realist approach

The realist approach provides the possibility of identifying causal patterns underlying complex interventions.¹³ Realism assumes that reality exists independent of human constructions, but that it is only perceptible through our senses.¹⁴ This approach postulates the existence of causal patterns, regardless of our understanding. Thus, an intervention does not work in itself; it is the mechanisms that underlie it which act (or fail to act) to produce the observed outcomes. These mechanisms are influenced by the context in which the intervention is implemented. From this perspective, the replication of experiments that try to control contextual elements is futile. We should instead try to observe patterns in the production of outcomes—what Lawson calls ‘demi-reg’ (quoted by Pawson, page 22)³⁹—and identify causal arrangements. This is called the ‘generative’ vision of causality. Research aims therefore to identify and describe, in a certain context (C), the mechanisms (M) operated by the intervention to produce its outcomes (O).

The approach proposed by Pawson and Tilley¹³ is rooted in a realist perspective of social change. Social phenomena are constructed both by the actions of individuals and by their understanding of such phenomena—individuals who in turn are constrained and enabled by social structures. As social systems, social interventions are built from the interplay between agents and structure. We must therefore understand how the agent and the structure interact to produce what characterises social reality. The manner in which interventions are broken down into context, mechanisms and outcomes should enhance our understanding of these social phenomena. Mechanisms should be understood as elements from the reasoning of actors facing interventions.¹⁰ Realism in social sciences holds that demi-regs are formed from the occurrence of mechanisms: “Realists thus think of the underlying engine of social reality in terms of people's reasoning as well as the resources available to them.”⁴¹ The context is similar to, but not limited to, the structure: it is social, cultural, historical or institutional. It is what allows or, conversely, what constrains the action of agents. Indeed, actions are part of a set of social processes that constitute social reality. Outcomes are the product of the interaction of these mechanisms and the context. Since outcomes are dependent on the context, they are therefore not immutable laws of nature.

C-M-O configurations are the tools that help explain social change by identifying these demi-regs (table 1).¹⁴ The goal is to refine these demi-regs by submitting them to empirical testing. More precisely, the identification of demi-regs must, according to Pawson, allow the development of middle-range theories, defined by Merton⁴² as “theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behavior, social organization and social change.” Middle-range theories enable us to get to the level of abstraction needed to understand the diversity of outcomes produced in different contexts.⁴¹ The C-M-O framework ensures the external validity of the research because it lets us move to the necessary level of abstraction for the theory or theories to be useful in other contexts. Blaise and colleagues¹² thus speak of the ‘plausibility’ of the predictive power of these theories. They explain that middle-range theories, rather than interventions per se, are what one should try to replicate in other contexts by improving the design of interventions based on similar mechanisms.

Table 1.

Key concepts of the realist approach, adapted from Ridde and colleagues¹⁰

Mechanism	Element of the reasoning of the actor facing an intervention. A mechanism: (1) is generally hidden, (2) is sensitive to context variations and (3) produces outcomes.
C-M-O configuration	Conceptual tool to link the elements of context, mechanisms and outcomes of an intervention.
Intervention theory	Set of hypotheses that explain how and why the intervention is expected to produce outcomes. It can be broken down in the form of one or more C-M-O configurations.
Middle-range theory	Level of theoretical abstraction that provides an explanation of semi-regularities in the context–mechanism–outcome interactions of a set of interventions.

The demi-regs are explained through expanding the conceptual vision, that is, moving from a descriptive structure to an explanatory one: “[…] theory-building moves up and down a ladder of abstraction.”⁴¹ By observing the mechanisms underlying user fee exemption policies in different contexts, it should be possible to see the appearance of demi-regs in the outcomes. The demi-regs can then be synthesised in a typology of ‘context–mechanism–outcomes’ (C-M-O) families.¹⁸

Research questions

The following research question, broken down into three specific questions, guides the research: what are the outcomes of user fee exemption policies implemented in Africa? why do they produce such outcomes? and what contextual elements come into play?

Study design

A systematic realist review such as that proposed by Pawson and colleagues^{39 40} will be carried out. This will make it possible to integrate knowledge on the experiences of at least 15 African countries. The study will be conducted in five steps. The process adopted for this research will not be linear but iterative, based on the course of action proposed by Pawson.³⁹

Step 1: identifying mechanisms operated by exemption policies

This step, which is similar to a logic analysis,⁴³ has two specific objectives: (1) to highlight the theory of exemption policies, that is, how they are supposed to work, for which recipients and with what anticipated outcomes and (2) to identify the contextual elements (institutional, organisational, socioeconomic, cultural) that influence the way mechanisms activated by exemption policies are expected to operate and produce outcomes. It is therefore a matter of identifying the mechanisms that form the basis of exemption policies. We will first reconstruct the logic of the issue that these policies aim to solve (namely the financial barrier to healthcare access), and second, reconstruct the intervention theory of these policies. This will be done in an exploratory, non-exhaustive and inductive way. Two sources of information will be sought: (1) official and scientific documentation around the main concepts of exemption policies, including the determinants of healthcare access and equity and (2) key actors' experiences. These actors (decision-makers and health managers from Mali, Burkina Faso and Niger) will participate in developing these models. Then, a framework will be formulated based on the context–mechanism–outcome principle. This framework will be discussed and validated with local and international exemption policy experts.

Step 2: researching primary data

With regard to the scientific data, the following document search strategy will be used: (1) the Ovid Medline, Embase, Popline, HMIC, Web of Knowledge, African Healthline, AJOL (African Journals On Line), EconLit, Business Source Premier databases will be used, as well as the websites of journals that regularly publish articles on health system financing, such as Health Policy and Planning, WHO Bulletin and Social Science and Medicine; (2) combinations of key words in English and French (table 2) and their truncations will be entered in these databases; (3) the relevance of the retrieved documents will be assessed according to exclusion and inclusion criteria (box 1); (4) bibliographic references from the included documents will be reviewed using the ‘snowballing’ technique to identify additional documents; (5) the ISI Web of Science database will be used to identify articles citing the included documents. Articles that address the context, mechanisms or outcomes may be included.

Table 2.

List of key words for the document search strategy

AND			-OR-	Themes and expressions (grey literature)
-OR-	-OR-	-OR-
User fee*	Aboli*	Developing countr*	Free healthcare	Health services accessibility	Healthcare costs
User charge*	Exempt*	Africa*	Free care	Health facilities	Health insurance
Cost-sharing	Waiv*	Low income countr*	Free service*	Health disparities	Health expenditure*
Cost-recovery	Remov*	Middle income countr*	Free healthcare	Health policy	Health financing
Out-of-pocket	End*	LMIC*	Universal access to healthcare	Health equity	National health programs
	Discontinu*		Universal access to healthcare
	Free

The mandated truncation symbol “*” retrieves records that contain the search term and all possible suffix variations of a root word. For example, “user fee*” will retrieve records containing “fee” and “fees”.

Box 1. Inclusion and exclusion criteria.

Inclusion criteria

• The document's main focus is health.

• The document deals with at least one African country.

• The document addresses the issue of user fee exemption in the health sector OR the research takes place in the context of user fee exemption in the health sector.

• The user fee exemption policy dealt with in the document is a national policy.

Exclusion criteria

• The document's main theme is not health.

• The document does not deal with at least one African country.

• The document addresses a different issue than the issue of user fee exemption in the health sector.

• The document focuses on user fee exemption in the context of cost recovery policies (waivers).

• The document focuses on user fee exemption as part of a programme run by a non-governmental organisation.

Because grey literature is a relevant source of information for realist reviews, evaluation reports or policy documents published by African governments, international organisations, non-governmental organisations and consultancy firms, as well as dissertations and theses, may also be included. Our contacts with networks of researchers, decision-makers and other stakeholders in North America, Europe and Africa will facilitate the collection of these documents. The Database on African Theses and Dissertations and Dissertations and Thesis will also be searched.

The search for new documents will end at the point of saturation, that is, when the research yields no more new sources of information. References will be compiled in Mendeley, a reference manager.

Step 3: assessing the quality of studies

Unlike traditional systematic reviews, there is no need to assess study designs based on the hierarchy of evidence in a realist review. Quality assessment is done instead in a heuristic perspective to enrich the C-M-O configurations and should answer the question: “Is this study good enough to provide some evidence that will contribute to the synthesis?”³⁹ Each study should be assessed according to how it clarifies the configurations. Therefore, the unit of analysis is not the study itself, but fragments of information that are produced.⁴⁴ However, this approach does not ensure a transparent selection of articles. Some authors have assessed the methodological quality of the articles in their realist review but failed to specify which tools they used.^{45 46} The Mixed-Method Appraisal Tool proposed by Pluye and colleagues⁴⁷ seems an appropriate compromise between the need for a rigorous and transparent quality assessment and the need to consider the data according to their relevance for the development of middle-range theories. Indeed, this tool makes it possible to describe the methodology of qualitative, quantitative and mixed studies based on 19 separate items. For this review, studies considered to be of too low quality may be excluded (the minimum quality standard will be determined considering the overall quality of the studies). To facilitate this process, a summary table will specify the authors, objectives, type of study, different methodological aspects, conclusions and assessment made according to the Mixed-Method Appraisal Tool.

Step 4: extracting the data

The documents included in the review will be compiled into the QDA Miner^® software to facilitate their organisation and analysis. This software can encode and annotate a large number of documents, extract relevant data and link emerging themes. The analytical framework developed in step 1 will be used to analyse the documents and will be further specified in an iterative manner so as to integrate new explanatory elements. More specifically, data from which new categories are created and differences between categories will be reported.

Step 5: synthesising the data

Each primary study will be “inspected for evidence, according to how it supports, weakens, modifies, supplements, reinterprets or refocuses the preliminary theory”.³⁹ By applying, completing and clarifying the analytical framework, we can take a critical look at the contribution of each study to the initial theory. It will thus be possible to specify configuration elements (C-M-O) and develop a middle-range theory. This process will follow the abductive approach that Blom and Morén, citing Danermark, explain: “Abduction means that single events or occurrences—by means of concepts, theory and models—are described and interpreted as expression of more general phenomena.”⁴⁸ Each study will help clarify or reformulate the C-M-O configurations in order to take into account potential contradictory elements. The middle-range theory will finally be put into words, as well as modelled to highlight the links between contextual elements, mechanisms and outcomes.

Internal and external validity

The internal validity of the research is ensured by the in-depth study of the theoretical articulation of user fee exemption policies. The construction of explanations so as to make ‘sense’ of exemption policies, through an iterative process between empirical data and construction of C-M-O configurations, also contributes to strengthening internal validity. The external validity of the research stems from the principle of explanation discussed above, as well as the approach based on the C-M-O configurations.⁴⁹ Taking the context into account in the production of outcomes increases the generalisation potential of the study. Indeed, developing middle-range theories allows a better understanding of the elements involved in the production of outcomes. A research logbook will document the research approach step by step, along with any adjustments and methodological choices made. This logbook will contain methodological as well as theoretical notes. This will help ensure the reliability of the research results.⁵⁰ A glossary will be prepared to clarify the terminology used in the research.

Discussion

Importance of the research

The results of the proposed research may be useful to African decision-makers who would be willing to implement exemption policies in their countries and to government and international agencies that wish to support them. More than 30 countries have adopted the BI and are thus potential users of the results of this research, especially in the context of an emerging consensus around these issues within the international community.^{8 51} In addition, this study is part of a broader research movement initiated around new practices of user fee exemptions in LMICs, but which mainly focuses on outcomes at the expense of other pieces of knowledge that will be studied within this research. Once the review has been completed, a realistic evaluation could be undertaken to test the new middle-range theory.

Few researchers, including those in the field of public health, have used the realist approach to synthesise the literature. This is doubtless due to both methodological and conceptual challenges.¹⁰ A secondary objective is therefore to pursue the reflections initiated on the realist review in order to facilitate its use for the benefit of research on health policies.⁵²

Knowledge-sharing strategy

To promote the use of the knowledge generated through this research, three activities will be implemented. First, a restitution workshop will be organised at the end of the research with key stakeholders. Then, the RESAO, a West African network of researchers and decision-makers, will be involved in the research. Its role will be to mobilise key informants at step 1 and for the restitution workshop and also to provide technical expertise on exemption policies. Finally, a policy brief in French and in English presenting the research results will be produced and broadcast via the RESAO and the Financial Access to Health Services Community of Practice.⁵³