From favours to entitlements: community voice and action and health service quality in Zambia

Abstract

Social accountability is increasingly invoked as a way of improving health services. This article presents a theory-driven qualitative study of the context, mechanisms and outcomes of a social accountability program, Citizen Voice and Action (CVA), implemented by World Vision (WV) in Zambia. Primary data were collected between November 2013 and January 2014. It included in-depth interviews and focus group discussions with program stakeholders. Secondary data were used iteratively—to inform the process for primary data collection, to guide primary data analysis and to contextualize findings from the primary data. CVA positively impacted the state, society, state–society relations and development coordination at the local level. Specifically, sustained improvements in some aspects of health system responsiveness, empowered citizens, the improved provision of public goods (health services) and increased consensus on development issues appeared to flow from CVA. The central challenge described by interviewees and FGD participants was the inability of CVA to address problems that required central level input. The mechanisms that generated these outcomes included productive state–society communication, enhanced trust, and state–society co-production of priorities and the provision of services. These mechanisms were activated in the context of existing structures for state–society interaction, willing political leaders, buy-in by traditional leaders, and WV’s strong reputation and access to resources. Prospective observational research in multiple contexts would shed more light on the context, mechanisms and outcomes of CVA programs. In addition to findings that are intuitive and well supported in the literature we identified new areas that are promising areas for future research. These include (1) the context of organizational reputation by the organization(s) spearheading social accountability efforts; (2) the potential relationship between social accountability efforts and making ambitious national programs operational at the frontlines of the health system and (3) the feasibility of scale up for certain types of local level responsiveness.

Key Messages

• Social accountability consists of citizen efforts to hold the government to account for the provision of essential services. It is increasingly invoked as a way of improving governmental health services.

• Citizen Voice and Action (CVA), a social accountability effort combining community score cards and social audit processes, positively impacted the state, society, state–society relations and development coordination in Zambia. Specifically, sustained improvements in health system responsiveness, empowered citizens, the improved provision of public goods (health services) and increased consensus on development issues appeared to flow from CVA. The mechanisms that generated these outcomes included productive state–society communication, enhanced trust, and state–society co-production of priorities and the provision of services.

• In addition to findings that are intuitive and well supported in the literature we identified new areas that are promising areas for future research. These include (1) the context of organizational reputation by the organization(s) spearheading social accountability efforts; (2) the potential relationship between social accountability efforts and making ambitious national programs operational at the frontlines of the health system and (3) the feasibility of scale up for certain types of local level responsiveness.

Introduction

Greater accountability is increasingly invoked as necessary to improving the coverage and quality of health services in low and middle income countries (Freedman and Schaaf 2013; Joshi 2013; Lodenstein, Dielman, Gerretsen and Broerse 2013). The consensus definition of accountability encompasses answerability and enforceability. Governments are obligated to provide information and justification regarding policy and practice (answerability), and sanctions must exist for failure to comport with policy (enforceability) (Schedler 1999; Goetz and Gaventa 2001; Brinkerhoff 2004). In addition to sanctions, more normative approaches describe how professional or social norms favoring accountability can engender enforceability (O’Connell 2005; Bovens 2010; Freedman and Schaaf 2013). Such focus on norms echoes increasing acknowledgement of the import of health systems “software,” such as norms, values and power in shaping health service delivery (Sheikh, Gilson, Agyepong, Hanson, Ssengooba and Bennett 2011). Social accountability is an area of growing research and programmatic interest in the broader accountability field; it consists of “ongoing and collective effort[s] to hold public officials to account for the provision of public goods which are existing state obligations” (Houtzager and Joshi 2008; Gullo, Galavotti and Altman, 2016). Thus, these collective efforts are intended to engender answerability and enforceability, potentially by ensuring existing rules are followed, by changing the rules, and/or by transforming health systems software.

Though there is widespread acknowledgement that context is key to the implementation and impact of social accountability efforts, and thus that program processes and outcomes are somewhat unique to each setting, broadly applicable lessons on social accountability implementation and impact are emerging. This study contributes to that evidence base by presenting empirical data from a theory-driven qualitative study of a social accountability program implemented by World Vision (WV) in Zambia, and by strongly contextualizing this data in the existing literature on social accountability. The study was undertaken in three districts. The article is intended to inform program implementers, donors and other stakeholders about aspects of context, mechanisms and outcomes that might be considered in the application of a social accountability program theory in Zambia and beyond. We also suggest areas of future consideration and research for program implementers and health systems researchers.

Social accountability and health evidence base

Joshi (2014) has proposed three different domains of impact for social accountability: (1) state, (2) social and (3) state–society relations. We looked at these domains at the local level. First, impact on the state may include outcomes such as reduced corruption and more responsive public officials (Gaventa and McGee 2013; Joshi 2013, 2014). Second, social accountability efforts may enhance social goods, such as improved provision of public goods, empowered citizens and increased social cohesion (Gaventa and McGee 2013; Joshi 2014). Third, impact on state–society relations may include the creation of institutional channels for state/society interaction and an increase in perceived state legitimacy (Gaventa and McGee 2013; Joshi 2014). While the intrinsic value of some of these impacts, such as empowerment and trust, are contested (Gaventa and McGee 2013; Carothers and Brechenmacher 2014), we work from the premise that they are valuable as intermediate outcomes (Topp, Chipukuma and Hanefeld, 2015), as well as development outcomes in their own right, particularly in the context of government health systems in low and middle income countries, where “[n]eglect, abuse and exclusion by the health system … [can be] part of the very experience of being poor” (Freedman 2005).

A diverse array of strategies can be described as “social accountability.” We study Citizen Voice and Action (CVA), which combines aspects of two fairly common approaches, namely ‘community scorecards’ and ‘social audit’.

CVA traces its origins to the World Bank’s Community-Based Performance Monitoring (CBPM) piloted in the Gambia, which was in turn derived from the international non-governmental organization (NGO) CARE’s Community Score Card process as implemented in Malawi (Winterford 2009). CBPM enhanced the traditional Community Score Card approach by including national service delivery standards (e.g. standards for hospital cleanliness) as well as perception-based indicators that are generated by the community (through focus group discussions), such as health facility staff punctuality.

We describe the program theory below. CVA has been used in many domains, including health and education. Our description of the program theory focuses on the assumptions and activities that are most relevant to health.

The CVA program theory is premised on information, voice, dialogue and accountability. The program provides opportunities for citizens to learn what their rights are by facilitating greater transparency and access to national service delivery standards, as well as by supporting citizens to articulate standards (“perception-based indicators”) and generate their own information. The national service delivery standards used differ by setting; in Zambia, the health standards used relate to maternal health care and primary health care. Once they are empowered with greater information, citizens express their voice through channels provided by CVA, and over the long-term, through other channels that are enabled by CVA. As a result of the expression of citizen voice, providers obtain more information about citizen priorities and challenges. Providers learn more about citizens because citizen and service provider voices are expressed in a 2-way dialogue, which builds understanding and trust among those present, and also provides an opportunity for the creation of partnerships to improve service delivery. According to WV’s program theory, citizens demand accountability through their expression of voice, but the accountability emerging from CVA is not just about providers being more accountable to communities. “The primary objective of CVA is to increase dialogue and accountability between three groups: citizens, public service providers and government officials (political and administration) to improve the delivery of public services” (WV 2016). Thus, the program theory appears to target all stakeholders. Yet, the normative document for CVA recognizes the fact that some hold more power than others; a key objective of citizen voice is to “influence[s] government processes and services” … ultimately, “hold[ing] power holders accountable” (WV 2016).

As shown in Figure 1, the program occurs in three, iterative phases. The first phase entails WV-led relationship building with communities and service providers and stakeholder mobilization to inform the community and relevant actors about the goals and components of CVA. Next, WV convenes an open community gathering during which a CVA Committee is formed, usually by a consensus process. About 10–15 people join; membership is voluntary. CVA Committee members are often also members of other community structures, such as village development committees and neighbourhood health committees. Insofar as possible, WV tries to facilitate the creation of a diverse CVA Committee, so that the Committee has widespread legitimacy.

Figure 1.

Overview of the CVA process

Following facilitation from WV, representatives from the government educate communities about relevant legislation and national service delivery standards. Citizens may have preferences and priorities that are not formally enshrined in national standards, thus they also articulate standards (“perception-based indicators”) that they think their local facility should meet. In the second phase, the health facility’s (or other service provider’s, depending on the context) realization of both perception-based indicators and national service delivery standards are assessed. A social audit process is used with service providers and communities to assess performance of the clinics against national service delivery standards. Here, citizens and service providers observe the facility and look at facility data to assess to what extent the facility is compliant with national service delivery standards. Then, citizens and service providers use community score cards to rate their health facilities against the perception-based indicators. Third, citizens, local elected representatives and service providers, convene interface meetings. They discuss the service delivery gaps identified and elaborate action plans to address some of these challenges. Action plans identify individuals and groups responsible for each action. The plans are then implemented and monitored in subsequent interface meetings. The three phases are repeated, as communities and the government tackle increasingly difficult challenges.

In Zambia, WV initiated CVA in the three rural districts covered by this study in 2008, with a focus on two sectors: health and education. The five health facilities directly engaged in CVA in these three rural districts are at the primary care level. WV Zambia is now leading CVA in 16 of Zambia’s 103 districts, across all 10 provinces of Zambia. It is also important to note that as of February 2016, CVA is being implemented in hundreds of sites in 45 countries globally. Several of these sites are the subject of ongoing research.

Study setting

Zambian health system

Demand on primary health services is high, driven partly by improvements in life expectancy, the introduction of HIV care and treatment services at the primary health care level, high rates of fertility, and migration to urban and peri-urban areas (Masiye, Chitah, Chanda and Simeo 2008; National AIDS Council and Government of the Republic of Zambia 2012).

Against this backdrop of growing demographic and epidemiologic pressures, a chronic shortage of human resources for health continues to affect Zambian primary health care. In 2006, the Ministry of Health reported there were only 646 doctors to the country’s then 12 million population (MOH and GRZ 2006). Between 2008 and 2010 staffing levels improved but still fell far below international recommendations (Herbst, Vledder, Campbell, Sjoblom and Soucat 2011). Healthcare workers are also highly unevenly distributed (Schatz 2008).

Due in part to the pressures cited above, numerous concerns about clinical quality and health system responsiveness have been identified in Zambia, including lack of respect for patients, lack of provision of information, inexcusable delays in the provision of care, and burdensome out-of-pocket payments (Phiri, Fylkesnes, Ruano and Moland, 2014).

Five WV CVA sites in three districts were included in this study, namely Chibombo, Mumbwa and Lufwanyama. These sites were chosen because they were the pilot sites for CVA in Zambia, and are thus the locations where CVA had been implemented the longest. The three districts are largely rural. Chibombo and Mumbwa are in Central Province and Lufwanyama is in the Copperbelt Province. The districts have varying levels of socioeconomic development. Each CVA site is comprised of a primary health care facility and the catchment area. It is important to note that the CVA Committee is based in the village where the health facility is located. Committee members travel and hold events in other villages, but they are less regularly present in villages that are formally part of the health facility catchment area, but that are at the edges of the catchment area.

Study aim and methods

Recent literature from interdisciplinary research on health systems explains that health systems are complex adaptive systems, characterized by interdependent relationships, contingent constellations of power and non-linearity (Handler, Issel and Turnock 2001; Adam and de Savigny 2012). These systems reflect and enact dynamics of social and political power, such that the health system is not merely a mechanistic service delivery system (Gilson 2003; Freedman 2005). The delivery of health services depends on the active inputs of individuals, which are embedded in the larger health system context. For example, health care workers provide care, adhere to guidelines, interact with each other and interact with patients according to their personal values, social and professional norms and larger health system infrastructure, among other factors. In brief, providers, patients and others exercise agency according to their reasoning and resources (Pawson, Greenhalgh, Harvey and Walshe 2005; Dalkin, Greenhalgh, Jones, Cunningham and Lhussier 2015). It is in this context that social accountability efforts seek to effect change.

We sought to make tentative, contextualized programmatic and theoretical propositions about how the CVA program theory was realized in the health sector in 3 of Zambia’s 103 districts. The study aimed to answer:

1. How does CVA affect the relationship between citizens and the health sector?

2. How does the health sector respond to CVA?

3. What elements of context facilitate or hinder positive change in the health sector in response to CVA?

Answering these questions required identifying and understanding the program outcomes, and the mechanisms that, in the given context, engendered these outcomes. Context, mechanism and outcome configurations are facets of “realist evaluation”, which is one of several approaches in the broader field of theory-driven research (Pawson 2013). This approach to analysis seeks to understand what works, for whom and under what circumstances, rather than the more common approach of simply examining activities or other easily observable inputs (Lodenstein, Dielman, Gerretsen and Broerse 2013). Among others, there are two principles of realist evaluation that make it particularly apt for examining CVA in a health systems context. First, realism assumes that all systems are “open” systems, meaning that the boundaries among given systems—such as the health system and the community—are permeable and changeable (Westhorp, Walker, Rogers, Overbeeke, Ball and Brice 2014). This facilitates understanding of CVA’s intent to change relationships within societies and across the state/society divide, as well as elements of context that shape the outcomes of such efforts. Second, observable changes, such as more polite health care workers, are generally caused by non-observable processes, termed “mechanisms” (Westhorp, Walker, Rogers, Overbeeke, Ball and Brice 2014). “Mechanisms” are thus distinct from activities; they are the underlying processes that operate in particular contexts to generate outcomes (Astbury and Leeuw 2010; Dalkin, Greenhalgh, Jones, Cunningham and Lhussier 2015). The “causal powers” of a program relate to what resources the program provides, what ‘reasoning’ is induced in response, and what behaviour changes are generated (Westhorp, Walker, Rogers, Overbeeke, Ball and Brice 2014). Some research and evaluations of social accountability have been criticized for focusing on activities, or social accountability “tools”, rather than on how change happens (Joshi and Houtzager 2012; Gaventa and McGee 2013; Fox 2015). In contrast, a focus on mechanisms militates against such widgetization, or taking a tool-focused approach that is blind to human relations and power dynamics. Instead, we seek to understand how social accountability efforts occur in a larger accountability ecosystem (Joshi and Houtzager 2012; Cornish 2015; Halloran 2015). Moreover, close attention to context and underlying change processes facilitates context-sensitive conclusions, ultimately contributing to more educated uptake of research evidence into practice (Marchal, van Belle, van Olmen, Hoeree and Kegels 2012; Reddy, Wakerman, Westhorp and Herring 2016). Given that CVA and similar programs are currently being implemented in hundreds of sites, prudent use of research evidence is key.

A full-fledged realist evaluation would typically require longitudinal engagement with program participants and stakeholders. Moreover, given CVA’s widespread use, a rigorous realist evaluation would entail looking at multiple countries. Thus, we describe this study as a realist informed qualitative study, an approach that has been taken in other contexts where researchers feel that the context, mechanisms, and outcomes framing would add value to extant data (Shankardass, Renahy, Muntaner, O'Campo 2014; Kok, Ormel, Broerse et al. 2016). Drawing from insights in health systems research, this study design highlights the contingent, embedded, and iterative nature of social accountability efforts.

Further, by focusing on underlying change processes and the contexts that affect how they work, it offers a different way of assessing whether findings are portable to other situations.

Data collection

To answer the questions above and to ensure the practical and theoretical relevance of the study, we used both primary and secondary data.

Secondary data were used iteratively. Secondary data included WV program documents, score cards and action plans generated by CVA activities, and materials WV developed summarizing health entitlements. More importantly, we also reviewed articles regarding social accountability in all domains (not just health), as well as health systems and policy research articles relating to relationships within health systems and between communities and the health system. Because a significant amount of research on social accountability has not been published in peer-reviewed literature, we included some grey literature in our search. These articles were identified through an initial literature search on “social accountability” in Google and Google Scholar, but the list was augmented iteratively as new resources were identified through follow up on the citations of those on the initial list, or, as new articles were published. All articles identified (n = 63) were hand coded thematically. These secondary data sources were used to inform the development of interview tools and of deductive codes, and, iteratively, to contextualize findings from the primary data (though only 45 had been identified and coded before tool development, such that only 45 were used for this purpose). The authors continuously moved back and forth between the peer-reviewed literature and the data during the analysis phase, assessing to what extent findings from other studies were supported and contradicted in our data, as well as identifying which findings and questions arising from our data had seemingly not arisen in previous studies. In this way, this study is part of a larger process of aggregating knowledge on program theory for social accountability (Manzano-Santaella 2011).

Primary data were collected between November 2013 and January 2014. CVA had started in these communities in 2008. At the time the research was conducted, the program was ongoing in all of them. Methods used included in-depth interviews with district health officials (n = 5), traditional community leaders (n = 2), rural health centre staff from one facility in each of the three sites (n = 4), WV staff based in the districts under study (n = 8) and WV staff based in Lusaka (n = 1). Focus groups were also conducted with CVA members in each of the three sites (n = 27). The interview and focus group discussion guides were developed based on our research questions, as well as findings from other studies on social accountability and/or health systems in low and middle income countries that we felt may be apt. Specifically, to try to address the research questions in the given context, we developed the tools to explore elements of context, mechanisms, and outcomes that were found to be relevant in other social accountability programs.

Recruitment was carried out via verbal invitation, issued by WV staff to members of the CVA Committee. The Committee members were asked to report to a nominated location on a particular day; all individuals doing so on the assigned day were eligible to participate. They were given information about the study and its goals, and asked to provide written informed consent. The Chibombo, Lufwanyama and Mumbwa focus groups included 7, 15 and 6 participants, respectively, including a mix of men and women.

This study focuses on the health sector, though we did not stop participants when they spoke about relevant changes in the education sector or other domains. Participants reported on their experiences with CVA from its initiation to the present.

The study team visited each site for 2–3 days. Rural health clinic staff were recruited with the assistance of WV, who informed the staff when the research team would visit. All interviews and focus groups were led by trained research assistants in the local language of the interviewee’s choice. They were recorded and transcribed. Where needed, the transcripts were translated into English. All were imported into QSR NVivo 10.

The study received ethical approval from the Eres Converge Institutional Review Board (IRB) in Zambia and from the Columbia University Medical Center IRB in New York City, USA. Relevant officials from Mumbwa, Lufwanyama and Chibombo Districts provided written permission for study activities to take place.

Data analysis

Initial deductive codes were developed based on the literature. These codes related to context, mechanisms and outcomes. We did not categorize the codes as relating to (or comprising) context, mechanism, or outcomes until later in the analysis process, as many could fall under more than one rubric. For example, mutual trust between the community and health providers might be considered to be a mechanism generating increased health service utilization, or, it could be considered to be an intrinsically important outcome on its own. MS and ST each examined five transcripts to refine the deductive codes and to develop inductive codes in a focused coding process (Charmaz 2006). We went back and forth between the literature and the data several times, and consulted with MN in a continual process of assessing and finalizing the codes. As we iteratively developed literature-based codes that accommodated emerging findings, our analytic process can be described as deductive, indicative, and abductive (Timmermans and Tavory 2012).

Once the codes were finalized, we re-coded all transcripts. To ensure consistent coding, MS and ST coded 10 of the transcripts jointly. They divided the remaining 13 transcripts, with an overlap of 3 transcripts. The percentage agreement on these was three transcripts was 97.4%. We judged percentage agreement to be the most appropriate indicator of consistent coding since the likelihood of guessing was low (McHugh 2012).

After looking at the coded data in its totality, we decided to define outcomes as the most downstream consequence for which we had adequate data to at least partially attribute to CVA. We relied on the Joshi typology, though we ultimately added a category. Finally, we also considered what study participants described as outcomes. We categorized the underlying processes for which we had adequate data to conclude had engendered the outcomes as mechanisms. We were especially likely to label such processes mechanisms (rather than outcomes) if the data indicated they contributed to multiple outcomes.

Codes were grouped into larger themes, with each code being included in multiple themes. The themes addressed aspects of the context, potential mechanisms, and outcomes of CVA. Brief analyses were written on each theme. In addition, a data display describing all putative outcomes of CVA was made. This display included outcomes falling under Joshi’s three domains of impact as they occurred at the local level (state, society and state–society relations). Over time, we added a fourth category, impact on development coordination. We then expanded this data display to include related findings in a few other key reviews and studies on social accountability for health. This allowed us to firmly ground our findings in the literature.

Results and discussion

The following section describes and contextualizes the results of our study. After discussing the process of CVA itself, we apply Joshi’s domains of impact to our local level outcomes. Consistent with our realist orientation, we then hypothesize mechanisms that underlie the outcomes observed, and, finally, the elements of the context that facilitated activation of the mechanisms. As noted, we did not conduct a full-fledged realist evaluation. For this reason, we did not attempt to elaborate context, mechanism, and outcome configurations. We felt that the contextual variation among the sites was too limited—or our ability to ascertain this variation was inadequate—as the interviews were too few and we lacked the prospective observation data we would have preferred. Thus, we did not feel confident making assertions about relationships within context, mechanism and outcome configurations. For example, we had ample evidence to assert that “existing structures facilitating state/society collaboration” enabled multiple mechanisms, but did not have sufficient evidence to say that it was not a contributing aspect of context for other mechanisms. Moreover, some of the mechanisms identified are inter-related and synergistic; they can be described as emergent properties of a well-functioning health system (Topp and Chipukuma 2015; Topp, Chipukuma and Hanefeld 2015). In other words, they are not independent variables with additive impact on health system functioning, but attributes that are nurtured by CVA that in turn nurture each other and further engender health system improvements. Several of the mechanisms work in concert. Again, given the somewhat circumscribed nature of the research, we did not feel it appropriate to propose configurations that ignored these complexities, or to propose complex configurations without sufficient empirical basis. Thus, we propose context, mechanism and outcome factors for further empiric investigation.

Process of CVA

Respondent descriptions of CVA matched the WV program model; program fidelity was high. As designed, the program allows for different timeframes and adaptation according to context. For example, moving through the three phases—enabling citizen engagement, engagement via community gathering, improving services and influencing policy—may require multiple meetings in a short timeframe, or it may require many meetings with various stakeholders over a long timeframe.

We also assessed to what extent the process was characterized by elite capture. Some researchers have found that individuals engaged in social accountability are wealthier and more educated than the average citizen, and thus may not advocate for issues that affect the most excluded (Mansuri and Rao 2004; Fox 2015; Grandvoinnet, Aslam and Raha 2015). To some degree, this capture may be inevitable, as social accountability addresses collective goals, and engagement of those with power, such as traditional leaders and Health Committee members, is both unavoidable and necessary. Review of the FGD transcripts suggests that while there were members of CVA Committees who were among the village elite, not all were. Issues such as adequate drugs or health worker absenteeism seemingly affect most, if not all, users of the health facility.

Outcomes of CVA

Below are the outcomes that came through in our data. We put them into four broad domains, and we also provide more specific descriptions within each domain. Domains 1–3 (state responsiveness, social, state–society relations) occurred largely at the local level, while the fourth domain (development coordination) occurred largely at the district level.

Domain of outcome 1: state responsiveness

Most priorities identified in CVA action plans require at least a one-time state response, if not a consistent change in practice by governmental employees. We understood state responsiveness to include improved health worker response. Interviewees in all sites described more polite and timely treatment by health care workers as flowing from CVA. A few referred to decreased absenteeism. Many community members also described feeling that they had more access to district officials. These findings are all common in the literature, although enhanced access to government officials is somewhat less frequent (Joshi 2010; Bjorkman, De Walque and Svensson 2014; Ho, Labrecque, Batonon, Salsi and Ratnayake 2015; Wild, Wales and Chambers 2015).

This quote from FGD participants in Mumbwa is illustrative of their general satisfaction with CVA and its health sector responsiveness outcomes:

I can say in the health system there is a great change […] nowadays the health service providers have changed their work culture in a positive way. In the past they used to report late for work and you will find that they used to rebuke or ridicule us a lot.

Health providers, too, acknowledged these changes. Some explained these changes were reciprocal; they described being more inclined to be responsive in part because they felt the community was more understanding of their challenges, limitations, and mandate.

It is not easy working in a [Health] centre like here because you are doing everyone’s job, so you are tired. [A] person will come and approach me and if I am very rude they will […] shy away. But now it is like they understand. When they approach you they will explain and you know even you yourself will be bad to turn away someone like that. (Health provider, Chibombo)

This improved responsiveness became an expectation. Once behaviours improved, CVA Committee members were not afraid to demand that service providers answer for perceived transgressions of this “new normal”. In these cases, enforceability was realized.

In the past [service providers] used to report late for work and you would find that they used to rebuke or ridicule us a lot. But nowadays if we see a health service provider is rebuking a patient we are quick to take action and remind them of their responsibilities and also that patient’s rights to be attended to. (FGD Participant, Mumbwa).

Domain of outcomes 2: social

Society level outcomes included empowered citizens, the improved provision of public goods and increased consensus on development issues.

Citizen empowerment began in part with increased knowledge. In all three districts, interviewees reported that CVA positively impacted individual and community knowledge, particularly in relation to understanding health entitlements and minimum standards for local health facilities. This finding is ubiquitous in peer reviewed literature on social accountability and service delivery, which shows that while information is usually not sufficient, it can play a determinative role in affecting change through social accountability campaigns (Bjorkman and Svensson 2007; Reinikka and Svensson 2011; Gaventa and McGee 2013; Joshi 2013; Papp, Gogoi and Campbell 2013; Bjorkman, de Walque and Svensson 2014; Fox 2015; Ho, Labrecque, Batonon, Salsi and Ratnayake 2015).

CVA has changed a lot of things. [For example] helping us learning about government policies. Without knowledge of government policies you wouldn’t even know what’s happening […] We were taught and now the fear has gone. (FGD Participant, Lufwanyama)

CVA members in the three districts additionally reported that CVA activities improved their own and their communities’ understanding of decision-making processes. This too, has been found in earlier studies (Papp, Gogoi and Campbell 2013) but it is a less common finding. CVA Committee members linked their newfound policy knowledge and understanding of government processes to empowerment.

So this time we really know how to approach the government[…]and tell them: Here there is a mistake. Me [I] am a citizen and [I] have rights in such a way. (Headman, Mumbwa)

[I]n the past we were very ignorant. But when CVA was brought to us and they explained […] I truly felt that I was a citizen, a national citizen. One who is given powers to speak out when I realize that things are not going in the manner they are supposed to be. (FGD Participant, Chibombo)

The collective aspect of CVA also seemed to engender collective empowerment. This is important since expressing voice requires individuals to sometimes take social risks in settings where they have traditionally lacked power. Respondents referred to a ‘strength in numbers’ phenomenon.

Finally, citizens stated that they felt empowered in part because they saw concrete results in the improved provision of public goods (health and education), a virtuous circle that is part of the program theory of CVA and that has been identified in other settings (Joshi 2008). Table 1 summarizes specific descriptions of improved provision of health services, as well as the resources used to effect the improvements.

Table 1.

Improved provision of public goods in three sites with CVA activity

Site	Improved provision of public goods	Resources
Lufwanyama	Mothers’ shelter built (accommodation near to clinic for pregnant women who are close to term)	Foundation bricks contributed by community; funding from Constituency Development Funds (CDF) and WV
	Four new beds in labour ward	WV and Save the Children
	Medical licentiate and nurse hired, one of whom was since transferred	Government health budget
	Increased availability of essential drugs (allocation of essential drug kids to health centre increased)	Government health budget
Chibombo	Environmental Health Technician and Midwife hired	Government health budget
	Increased availability of essential drugs (allocation of essential drug kids to health centre increased)	Government health budget
Mumbwa	Bore holes repaired	Inputs from NGOs that initially constructed bore holes, with some money and time offered by the community
	New rural clinic constructed Houses for rural clinic staff constructed	District gave some cement; community contributed labour and materials. WV contributed materials for roofs of 4 houses.
	Health centre staff now working on weekends, as per policy	NA
	New clinic wing for maternity care, postnatal care and mother’s shelter constructed	Community bought crushed stones and sand; NGO contributed additional funds
	Health care workers that were disliked by community transferred out and new ones were posted	NA

These outcomes can be grouped into several broad categories. Table 2 delineates these categories and indicates whether or not other studies of social accountability and health have had similar findings.

Table 2.

Categories of service provision improvements

Type of change	Similar findings in the literature
Infrastructure improvement	CARE’s CSC facilitated infrastructure improvements in health facilities in four countries studied. Community and the government contributed (Wild and Harris 2011)
Reduced drug stock outs	Reduced drug stock outs in multiple countries (Wild and Harris 2011; Bjorkman, de Walque and Svensson 2014; Ho, Labrecque, Batonon, Salsi and Ratnayake 2015)
Hiring new staff and transferring unpopular staff	Wild and Harris (2011)
Improved staff adherence to policy	Bjorkman, de Walque, Svensson 2014; Joshi 2010; Papp, Gogoi and Campbell 2013

In addition to these improvements in the provision of public goods, interviewees and FGD participants explained that CVA promoted social consensus on key development priorities within communities. These priorities are shared by World Vision, the Government of Zambia and other development partners. Participants explained that Government of Zambia policies on practices such as early marriage were included in score card standards, ensuring that they were discussed at CVA-related awareness raising and interface meetings. Table 3 summarizes these putative changes. Those changes marked with an asterisk were mentioned by only one interviewee; the rest were mentioned by more than one.

Table 3.

CVA promotion of social consensus on key development priorities within communities

Site	Social cohesion and consensus on development issues
Lufwanyama	CVA Committee mediated between husband and wife*
	More meetings within the traditional leadership structure than pre-CVA*
	Community more comfortable approaching traditional leadership than pre-CVA*
	Decrease in child marriage
Chibombo	More men come for first ANC appointment
	Increase in activity of neighbourhood health committees
Mumbwa	Decrease in early marriage*
	Increased reporting of “child defilement”, suggesting increased awareness*
	Decrease in traditional methods of addressing child defilement (exchange of cattle)*
	Increased acceptance of vaccination within communities that had many refusers (such as certain religious sects)*
	Higher educational enrolment as parents are convinced about importance of education, particularly for girls, including enabling girls to reenrol in school after ‘falling pregnant’*

These shifting dynamics of power are explained by a Mumbwa FGD participant:

Equally as parents in the past we used to accept and obey whatever the headman could say to us even where we could see that things were wrong. But this time around we have the power to reject things which are not good for the community and reason with the headman in matters affecting us.

These specific findings relate to CVA in Zambia, but a few researchers in other settings have described similar changes in gender dynamics stemming from social accountability projects, such as community willingness to discuss gender-based violence (Datta, Jones and Febriany et al. 2011). In general, however, shifting community norms is an under-explored area in evaluations and studies of social accountability.

Domain 3 of outcome: state-society relations

State–society relations outcomes included improved functioning of pre-existing channels for state–society interaction and greater quantity and quality of interactions between citizens and elected leaders.

Participants explained that the CVA interface meetings were a structured but informal political space for state–society interaction. As identified in other contexts, interface meetings provide a platform for gathering and aggregating citizen voice, filling a “representation” role (Wild and Harris 2011; Gaventa and McGee 2013; Fox 2015). The community seemed to understand the interface meeting as a new channel that enters the “institutional repertoire”, rather than as a one-time component of a particular project (Joshi and Schultze-Kraft 2014).

So whenever there is a problem at the health centre or if there is something we don’t understand we will always call for an interface meeting between us the community, and the health service providers (Community member, Chibombo).

It also appeared that CVA strengthened the functioning and transparency of pre-existing channels for state–society interaction, namely the Neighborhood Health Committees (NHCs). NHCs are intended to strengthen service accountability by facilitating health system answerability to an elected group of citizens. CVA Committees often include members of the local NHC. NHCs reportedly met more frequently once CVA started. They were tasked with implementing and monitoring some of the Action Plan items, including infrastructure improvements. Through CVA, NHC members became more aware of what obligations membership entailed and they were more likely to fulfil these obligations.

Moreover, multiple interviewees stated that NHCs became more transparent as CVA shifted norms and expectations around access to information. As explained by a FGD participant in Lufwanyama:

The relationship improved after CVA was introduced. When the money was allocated to the [NHC] nobody knew [what happened to it.] But ever since CVA was introduced the [NHC] has been more transparent.

Other studies have similarly found that social accountability efforts strengthened existing channels for state-society interaction (Wild and Harris 2011; Westhorp, Walker, Rogers, Overbeeke, Ball and Brice 2014; Ho, Labrecque, Batonon, Salsi and Ratnayake 2015).

Finally, there were indications that CVA increased and deepened interactions between citizens and elected leaders. Members of Parliament (MPs) and local councillors attended interface meetings to learn what the constituency wanted, and/or because they had been explicitly engaged by the CVA Committee to play a role in realizing action plan priorities.

CVA has helped us in that these days we are able to talk to the counsellors, MPs … and to see whether they are capable of running the community well once elected. Before CVA came we were unable to do that for we were afraid to question them.

Similarly, Tembo and Chapman (2014) and Westhorp, Walker, Rogers, Overbeeke, Ball and Brice (2014) found that local politicians developed a better understanding of local needs through social accountability efforts. This finding is shared by social accountability research in some contexts, but not others (Wild and Harris 2011; Joshi 2013; Ho, Labrecque, Batonon, Salsi and Ratnayake 2015).

Domain 4 of outcomes: development coordination (DC)

We added development coordination as an additional outcomes area to the three identified by Joshi. Development coordination could be categorized as a route to improved provision of public goods, but given the important financial and administrative role development support plays in Zambia, it is appropriate to describe this phenomenon separately.

CVA enhanced development coordination through the interface meetings and the production of Action Plans. NGO representatives attended interface or other CVA-related meetings as “stakeholders” in the issues being discussed. The meetings served as an opportunity for governmental and non-governmental actors with human and financial resources (district health and education officials, community, NGOs, churches, elected officials, donors) to coordinate their inputs.

As explained by a District Health Officer from Chibombo:

[T]hrough CVA, we are able to strengthen the communication between the cooperating partners and government departments … we can work as a team.

In addition, the Action Plans were described as an expression of community and local government priorities that had been articulated through a transparent process of consultation and negotiation. This provided a framework for input from various development partners. This addition to the typology of social accountability outcomes is novel in the literature. We are unaware of any research that has identified social accountability programs as a mechanism for donor coordination.

Challenges

Unsurprisingly, given its objective to transform relationships and improve service delivery, CVA faced a range of challenges spanning political, work culture, and operational issues. The central and recurring challenge described by interviewees and FGD participants was the inability of CVA to address problems that required central level input. Indeed, there are very few examples in the literature of visible national level change stemming from local level social accountability efforts (Wild and Harris 2011).

WV Zambia intends to support CVA Committees to aggregate common challenges and advocate vis-a-vis central level authorities. However, the challenge of central level contribution likely goes beyond the question of what districts are permitted to do by policy, to deeper problems of administrative and financial management across levels of the government. For example, in Zambia, local level authorities often lack the financial capacity to deliver on their mandate because funds transfers from the central level are delayed (Chikulo 2014). It also raises important questions about the domains of potential outcomes that exclusively local level social accountability efforts can have in decentralized settings (Cleary, Molyneux and Gilson 2013).

These challenges may undercut the community’s faith in CVA and in their district officials, particularly if community members do not fully understand the chain of de facto and de jure decision-making. As one official explained:

Most of these [actions] are supposed to be supported from […] central government […] But the community because they know us they will say the District Health Office didn’t do anything. [District Official, Chibombo]

Interviewees and FGD participants also described challenges surrounding long entrenched social and cultural norms. This, too, has been the object of little study in the published social accountability literature. While these practices—such as early marriage and corporal punishment of children—have already been thrown into question by national policy changes and ongoing national and community level discussions, the program and those involved in it were nonetheless perceived by some community-members as a direct challenge to sacrosanct norms.

[W]e have been abusing the rights of children by forcing them into sexual cleansing. In the beginning most elders thought that this programme was there to destroy the cultural norms and beliefs considering the fact that they have been practising these beliefs from time immemorial. [FGD Participant, Chibombo]

Finally, FGD participants and interviewees described several aspects of CVA operations that they found constraining. These included lack of funds for local travel (Committee members travel to other villages); lack of adequate funds for telephone air time; the time required to participate, particularly for health providers and district officials; and high turnover rates in the government, which resulted in successive government officials needing to be sensitized and trained by CVA officials and/or community members. Implicit in some community members’ descriptions was the problem of ‘volunteer fatigue’ indicative of the high opportunity costs of participating in volunteer activities in settings of endemic poverty (Maes 2015; Topp, Price, Nanyangwe-Moyo, Mulenga, Dennis and Ngunga 2015). With the exception of high turnover rates in government, these challenges are little documented in the peer-reviewed literature, though they affect the probable sustainability of the project (Wild and Harris 2011).

Program mechanisms

Several mechanisms contributed to the outcomes above. The nuance of how these mechanisms are activated in the context of CVA in Zambia may be unique, but they are broadly similar to mechanisms that have been identified in other studies of social accountability.

Productive state–society communication

CVA contributed in two distinct ways to strengthening communication among community members, service providers, district officials, and elected representatives. First, the project trained CVA committee members in non-confrontational negotiation and facilitation skills, with a focus on listening and ‘dialoguing’ to support productive, respectful communication among all participants, including with district officials and elected representatives. Respondents described this approach as differing from the more confrontational approach—such as enumerating and publicizing problems with health service quality—to engaging service providers that had prevailed in the past. Second, CVA entailed interface meetings that promoted bi-directional information sharing. Community and government representatives from all three sites emphasized the mutually constitutive nature of the inter-face meetings, where community members aired their grievances (supported by data collected via the community score-card exercise) and asked questions. At the same time, government officials noted that the meetings provided an important space for them to be able to explain government policy, respond to community concerns, and describe the challenges they faced in trying to strengthen health services.

Enhanced trust

Better and more frequent communication via interface meetings and the resultant improvements in transparency, answerability and enforceability provided a self-reinforcing basis for strengthening the interpersonal trust between citizens and government officials. In relation to health care, trust has been theorized to be dependent on assessments of competency but also on judgements of reliability, sincerity, generosity and fairness (Wuthnow 2004). Respondents described the central role that interface meetings—a low-risk environment which enabled all parties to raise concerns—played in building precisely these perceptions of sincerity and fairness among all stakeholders.

Reflecting relevant theory, trust was built over time (Mayer, Davis and Schoorman 1995). Repeated and generally positive interactions between CVA members and government officials fortified trust. CVA members, district officials and healthcare workers, described the accumulation of positive experiences that enabled more frequent and less formal interactions, such as ‘drop-in’ visits to by CVA members to the District Medical Officer.

Co-production

The trust and communication channels built between communities and the health system were leveraged in the co-production of health system improvements. “Co-production” refers to goods that are jointly produced by citizens and the government (Ostrom 1996; Joshi and Moore 2004). We include: (1) the elaboration of priorities and (2) the provision of services and their necessary inputs, as goods that can be co-produced. First, in all three sites, priorities were coproduced to develop action plans. Respondents explained that steps toward action plan objectives were often defined with input from the community and the government:

“We can discuss our concerns in regards to the services provided by the health workers … From their explanation together we will put our heads together and chart the way forward”(Chibombo, FGD).

Consistent with empirical findings on social accountability, community member articulation of priorities at times empowered allies within the health system to be part of the co-production process (Fox 1996). Government staff may have lacked the incentive or the political space to effect change otherwise.

Second, similar to findings in other contexts and as seen in Table 1, some infrastructure improvements were also coproduced (Papp, Gogoi and Campbell 2013; Bjorkman, de Walque and Svensoon 2014; Ho, Labrecque, Batonon, Salsi and Ratnayake 2015; Wild, Wales and Chambers 2015). The co-production that we learned about was temporary, although it seems that in certain circumstances, institutionalized co-production could potentially arise from CVA.

The co-production of action plans and monitoring was described by various respondents as an important step in generating a sense of mutual accountability amongst all stakeholders. These findings are reflective of those from a realist review of accountability and empowerment interventions conducted by Westhorp, Walker, Rogers, Overbeeke, Ball and Brice (2014), who found that ‘mutual accountability’ is strengthened when “relevant stakeholders establish common goals […], an agreed action plan with clear responsibilities for each stakeholder group, and a monitoring process […] building mutual accountability.”

Contextual factors that enable the activation of mechanisms

Realist evaluations can identify mechanisms that are activated to greater or lesser degrees in different contexts (Dalkin, Greenhalgh, Harvey and Walshe 2015). We identified four particular contextual factors—outlined in Table 4—that appeared to be of critical importance to the mechanisms above.

Table 4.

Contextual factors contributing to CVA success

Contextual factor	Explanation	Significance to this study	Similar findings in other contexts
Existing structures promoting state/society collaboration	CVA interacted with—and dovetailed—government-created mechanisms for community participation in development (e.g. Area Development Coordination Committees, Safe Motherhood Action Groups)	Existence of these groups reflected stated governmental commitment to inclusive community participation Groups provided scaffolding for the conduct and realization of CVA activities and goals, as they were sometimes charged with conducting or monitoring the implementation of some CVA action plan activities CVA enhanced efficacy of these groups by reducing social risks for community members and/or health centre managers who used these structures complain	Evans (2012), Zulu, Michelo, Msoni, Hurtig, Byskov and Blystad (2014), Chikulo (2014), Ensor, Green, Quigley, Badru, Kaluba and Kureya (2014). [example only]
Willing political leaders	Political leadership both facilitated—and was strengthened by – CVA	Locally elected councillors were generally happy to participate in interface and other meetings, albeit with some wariness regarding the CVA agenda Respect for the material and moral authority of elected positions promoted community engagement, particularly in early meetings	Westhorp, Walker, Rogers, Overbeeke, Ball and Brice (2014), McGee and Gaventa (2010), Papp, Gogoi and Campbell (2013), Joshi (2010)
Traditional leader buy-in	Traditional leaders were important “interlocutors”, or intermediaries, who used community trust and legitimacy to facilitate “relationships, conditions and spaces” for accountability coalitions	Boosted attendance by citizenry at interface meetings Promoted accountability, insofar as community members were more likely to follow through on commitments made in traditional leaders’ presence	Papp, Gogoi and Campbell (2013), Tembo and Chapman (2014)
WV Reputation and Access to Resources	Long-term presence in Districts; construction of visible organizational and physical infrastructure (e.g. schools and participatory committees) Financial and organization flexibility to make 15-year commitments and build relationships carefully and slowly	Viewed as a neutral party vis-à-vis the health system and thus capable of acting as a trusted interlocutor/advisor Assisted community to navigate governmental agencies including relationship building, and to think through action plan priorities and follow up Well positioned to come up with resources for action plan implementation	To the authors’ knowledge, this has not been closely investigated in social accountability and health literature

Some of these contextual factors have been found to be relevant in other settings; others, namely the reputation of the implementing organization, are less explored. These factors provided a scaffolding for the conduct of CVA activities, while also being strengthened by CVA, as part of a virtuous cycle of enhanced communication and trust.

Outstanding questions

This research was conducted retrospectively in three districts of one country. The outcomes and mechanisms hypothesized above require further study, ideally in multiple sites and countries, using observational techniques and prospective approaches so that mechanisms and context can be further elaborated, and context, mechanism, outcome configurations can be proposed. These findings could help to further specify the CVA program theory, and potentially create somewhat different program theories for different contexts. Moreover, more research is needed to gather direct accounts of the experiences and perceptions of patients and community members who are not members of the CVA Committee.

Our case study did not explore two important issues that are under-addressed in the peer-reviewed literature.

The first relates to the scalability of social accountability. When CVA communities obtain an increase in a scarce commodity that cannot be easily produced at the local level—such as health workers—these gains may come at the expense of another site. For example, it is not clear if the transfer of new staff from other rural areas to CVA communities is the optimal outcome from a public health perspective. Moreover, transfers of poorly performing workers outside of CVA communities may only transfer the problem; another community may suffer at the hands of an inadequate or abusive provider. It could be that one of the mechanisms of CVA is a “squeaky wheel” phenomenon. District level officials may be willing to spend extra time and effort to placate a squeaky wheel, but they may lack the ability to do this in a scaled-up context. Also, at least in its initial stages, CVA relies on WV’s reputation as well as the curation and accompaniment they provide. To what extent can this be scaled up by WV and/or replicated by other actors?

Second, there is mixed evidence—and opinions—regarding whether externally induced social accountability projects can transform power relations. We saw indications that some transformations were beginning in Zambia: members of the community overcame entrenched norms of passivity to demand answerability, and health providers appeared to feel obligated to maintain “the new normal”. Moreover, CVA can potentially be a game changer insofar as it fosters implementation of national level strategies at the local level. The discrepancy between evidence-based national health objectives and on the ground implementation is much-lamented, but we lack adequate knowledge of how to ensure “the rubber hits the road” (Adams, Sedalia, McNab and Sarker 2015). As evidenced with enhanced compliance with opening hours and enhanced community consensus on development goals, CVA has the potential to push implementation of programs that the community can easily monitor. This in itself is transformative.

Conclusion

This study comprised a first step in a realist-informed assessment of CVA. We identified contextual factors, mechanisms, and outcomes that were salient in the sites we assessed in Zambia and that responded to ongoing discussions in the social accountability field. Our findings reaffirmed the idea that CVA and similar approaches should not be plucked from a menu of “ways to foster development”. Rather, they should be part of a long-term, integrated, iterative, and partnership-based approach to social change.

In addition to findings that are intuitive and well-supported in the literature—such as the facilitating role traditional leaders can play—we identified new areas that are promising areas for future research. These include: (1) the context of organizational reputation; World Vision’s perceived track record and existing relationships in the communities appeared to facilitate cooperation from community members and health facilities. (2) The potential relationship between social accountability efforts and the “last mile” of global development program implementation (i.e. making ambitious national programs operational at the frontlines of the health system). (3) The potential relationship between social accountability efforts and the capacity of communities, health providers, and even district officials to advocate and operate effectively in a newly de-centralized context. (4) The feasibility of scale up for certain types of local level responsiveness. Could district authorities handle CVA in all of the villages in their remit? Together, these findings offer theoretical propositions and empirical questions to be explored in future social accountability research.