Commentary on “Promising careers? A critical analysis of a randomised control trial in community health worker recruitment in Zambia,” by James Wintrup

Abstract

“Promising careers? A critical analysis of a randomised control trial in community health worker recruitment in Zambia” (Wintrup, 2021) raises important questions about the uses of randomized controlled trials (RCTs) and uses our RCT embedded in Zambia’s National Community Health Assistant (CHA) Program (Ashraf et al., 2020a) as a case study to illustrate the pitfalls of the RCT methodology and especially its potential to do harm. This commentary clarifies the misunderstandings at the heart of Wintrup (2021)’s critique.

In 2010 we collaborated with the Ministry of Health (MOH) of the Government of Zambia to evaluate the professionalization of community health work through the country’s new (2010) Community Health Assistant (CHA) program. Our evaluation was embedded in the national pilot of the program and found evidence that advertising the career benefits associated with the new position at the time of recruitment drew in individuals who had significantly greater health impact in their communities. This allayed concerns that making community health workers (CHWs) part of the civil service with formal salaries, benefits, and career opportunities would backfire by attracting less prosocial individuals. The evaluation results became an important input into national policy that ensured that CHAs remained professionalized workers.

In this commentary, we (1) describe the history of our engagement with CHW policy in Zambia and how, in particular, the distinction between CHAs and existing CHWs informed both the design and the framing of the RCT, (2) clarify misunderstandings in Wintrup (2021) about the RCT’s advertisement of career opportunities, (3) show that the criticisms of the RCT’s outcomes are based on omissions of major health impacts documented in Ashraf et al. (2020a), and (4) discuss how the foregoing, as well as leading questions asked to a convenience sample in Wintrup (2021), undermine the author’s claim of harm to vulnerable CHAs.

1. Policy engagement

In 2008, the Zambian Ministry of Health began discussions to create within the national health workforce a new, formalized cadre of CHWs, which would later be called Community Health Assistants. Based on qualitative fieldwork conducted in 2008 and 2009, one of us published a case study that provides a contemporaneous account of the genesis of the CHA program and policy questions raised by it (Ashraf and Kindred, 2010). This case study discusses all of the “longstanding debates about CHWs” noted in Wintrup (2021). Within the MOH, these debates were nuanced, as reflected, for example, by one MOH official’s reflections balancing the importance of supporting CHWs against the potential adverse consequences of reframing a volunteer position as a paid job in the civil service:

“They have families, they need to survive, they need to feed their children and send them to school. We have some very poor communities that cannot raise any form of support for the CHWs. So there has to be another way of supporting them …. But the moment you link [the job] to money you begin to commercialize it, which makes a greater problem of ‘unless you give me this, I cannot do that’ …. We also don’t want to create allegiance to the government system at the expense of the community. We want the CHWs’ identity to remain as part of the community so that they’re not viewed as government workers.” (Ashraf and Kindred, 2010)

In 2010, the MOH formally announced a national strategy for this new cadre (Ministry of Health, 2010). In doing so, the MOH conceived of CHAs as distinct from informal CHWs in many respects—e.g., their job qualifications, training requirements, scope of work, supervision, and, in what would prove to be the impetus of our RCT, formal employment within the civil service. The distinctions and relationship between the two cadres were discussed at length in the original CHA strategy (Fig. 1) and summarized as follows in the MOH’s most recent National Community Health Strategy:

Fig. 1.

Diagram of the relationship between Community Health Assistants (CHAs) and community health volunteers as depicted in the original CHA strategy (Ministry of Health, 2010). The CHA title had not been chosen at the time of publication of the strategy, and thus the document refers to them as “formalized CHWs.”

At the community level the Zambian health system relies on two main cadres of health workers. The first cadre are the Community Health Assistants, who receive 12 months of comprehensive training in primary health care and are formally employed as civil servants by the Ministry of Health. The second cadre are community-based volunteers, who tend to work directly with implementing partners and are trained over shorter periods, usually with a focus on specific disease verticals. (Ministry of Health, 2019)

In short, CHAs must be understood as distinct from traditional CHWs, with formal employment in the civil service sharply delineating the former from the latter in fundamental financial, political, and cultural respects. Since Wintrup (2021) does not use the CHA title anywhere in the paper, we do not know whether the author or his interview respondents are sometimes conflating the two roles. Reflecting this ambiguity, Wintrup (2021) states, “CHWs [sic] are not classic ‘bureaucrats’ and describing them as such does not help to convey the distinctive work that they do.” This may be true of informal CHWs, but not CHAs. CHAs are quintessential street-level bureaucrats (Lipsky, 1980), and researchers are increasingly examining professionalized cadres such as CHAs from this perspective (see, for example, in this journal, Atinga et al. (2018) and Krieger et al. (2021), as well as Nunes and Lotta (2019), Roy (2020), Sudhipongpracha and Poocharoen (2021), and Weaver (2021)).

Regarding the criticism in Wintrup (2021) of our “framing of the RCT as a contribution to academic debates, rather than as a study aimed to produce context-specific policy advice about CHWs [sic] in Zambia,” we first note that a single research study can generate multiple publications, some more oriented toward academic knowledge gaps and others more policy-oriented. Our academic paper was written to shed light on a question that encompasses, but also transcends, Zambia’s CHA program: whether adding private benefits to public service delivery leads to “losing prosociality in the quest for talent” (the title of the paper).

That said, as we will describe in Section 2, the actual RCT was explicitly designed in close collaboration with the MOH to “produce context-specific policy advice” about the CHA program. In fact, throughout our engagement with the CHA program, we have assisted the MOH with its policy-making needs in ways that conferred no academic benefits. For example, at the request of the MOH, we carried out two large-scale pro-bono policy research projects to support the development and implementation of the CHA strategy: (1) in 2008–2009, a survey of more than 100 CHWs across three rural districts, which served as an important data input in the design of the CHA strategy and (2) in 2011–2012, in preparation for the deployment of the first cohort of CHAs, a survey of 75 of the 162 communities participating in the pilot, in which we collected data from district health officials, health facility staff, and volunteer CHWs. Both surveys contained qualitative and quantitative components, and neither was published in an academic journal but rather was submitted to the MOH as an internal policy report. Similarly, in 2017, we conducted a nationwide household survey that provided many of the outcome measures for our RCT (see Section 3) but that we also synthesized in a report focused on broader policy issues such as overall staffing levels at rural health posts; data from this report was used and cited extensively in the MOH’s 2019 National Community Health Strategy (Ministry of Health, 2019). Finally, we presented our research findings to the MOH and relevant stakeholders annually between 2009 and 2019. In all of this policy engagement, we drew on our other research related to CHWs and community health in Zambia (see, for example, Ashraf et al., 2020b, 2014b; 2014c, 2014a, and 2010; as is customary in economics, the authors are ordered alphabetically, with no lead author) as well as CHW programs in other countries (see, for example, DeRenzi et al. (2016), Lee (2018), Lee and Vedanthan (2019), Palazuelos et al. (2013), Porter et al. (2009), and Whidden et al. (2018)).

2. Career opportunities

The central thesis of Wintrup (2021) is that our RCT recruited the pilot cohort of CHAs with false promises of a career path that has “never existed,” and that the results of the RCT led the Government of Zambia to continue to promise this non-existent career path in subsequent recruitment waves. This reconstructed narrative reverses causality. Career benefits for CHAs did not emerge on the policy agenda due to the RCT; rather, the RCT arose because career benefits, within the broader goal of professionalizing a cadre of CHWs as described above, were an integral part of the CHA strategy from its inception. As early as 2009, before the RCT was conceived, the MOH stated that the new formalized CHA cadre “will be appointed on probation and later confirmed as per civil servant regulations” and, under the heading of “staff progression,” “will be considered for training opportunities upon completion of a minimum period of two years [sic] service in any desired field in the health profession” (Ministry of Health, 2009).

As we recount in our paper, when we met with MOH leadership in the months leading up to the CHA program launch in 2010 to discuss evaluation needs related to the program, the director of the program articulated the MOH’s primary policy preoccupation: “What is going to happen now that [CHWs] will see themselves as civil servants? Will they be connected to the community?” The MOH officials asked if we as researchers could use the CHA pilot to shed light on this question, and it is through this invitation that we and the MOH collaboratively designed the recruitment RCT. The salary attached to the new position was considered for study, but the wage scale had not been finalized. Instead, the MOH officials felt that advertising an in-kind benefit would be more appropriate and that, among these, opportunities for career progression most strongly distinguished a civil-service health care position from one in the informal sector.

In light of the reality of how the recruitment RCT arose, we hasten to emphasize that the MOH officials with whom we had the privilege to collaborate were highly competent professionals committed to stewarding the new CHA program faithfully. As anyone who conducts research in Zambia is aware, health research in Zambia must obtain scientific and ethical approval from the MOH, as well as from an academic Institutional Review Board within Zambia. In our experience, it is inconceivable that the MOH would allow a team of foreign academic researchers to unilaterally devise and impose a nationwide experiment advertising false information to hundreds of communities.

In addition to the argument that we as researchers directed the MOH to advertise career opportunities during the inaugural CHA recruitment drive, the other claim in Wintrup (2021) regarding career opportunities is that they never materialized. We believe this reflects different understandings of what constitutes a career path. In Wintrup (2021), a career path is conflated with automatic promotion to higher-level cadres. Because this is not the case for CHAs, the article states that CHAs’ career benefits are identical to what “any other Zambian citizen would have to do in order to become a nurse or clinical officer.” This is false in three respects: (1) unlike other training courses, the one-year CHA training, which entails significant acquisition of transferable knowledge and skills, is fully financed by the government along with room and board, thus mitigating typical financial barriers to entry into civil service, (2) CHAs granted study leave may continue to receive salary, and (3) they are assured the higher-level position upon completing training. By any standard definition, this is a career ladder. Importantly, due to uncertainty about the future trajectory of the CHA program, the recruitment poster explicitly offered only the first of these benefits: a one-year "training opportunity" to gain skills and boost your career”. In short, what distinguishes the CHA cadre from other CHW positions is the possibility, not the guarantee, of career advancement. Given that no policy of guaranteed promotion exists for any other MOH cadre, we venture to speculate that applicants would have found such a promise highly incredible.

Over the past decade since the CHA program launched, there have been varying views within the Government of Zambia about fostering career opportunities for CHAs within their home communities, and we would caution against assuming that the opinions of one government official, as presented in Wintrup (2021), represent a consensus view. Nevertheless, through three major government transitions over the past decade, CHAs have remained civil servants on government payroll, and the MOH’s official policy remains to “aim to upgrade those CHA who worked diligently for at least 3 years into nursing, environmental health and clinical officer programs” (Ministry of Health, 2017). In addition, we would add that fostering higher-level career opportunities within home communities is not a contradictory objective. In fact, as the MOH states, “While each of the country’s health posts should be overseen by a nurse, currently 34% of nurse posts at health posts are vacant. As a result, 23% of all CHA are acting as facility in-charges” (Ministry of Health, 2019). Given these vacancies, a natural career path for CHAs would be to up-skill them into nursing and other roles within their health post, with new CHAs recruited to replace them. Because CHAs, unlike nurses and other health cadres, are chosen by their home communities, it is not appropriate to assume that CHAs-turned-nurses will migrate to urban areas at the same rate as those who become nurses through the standard govrenment hiring process in which local communities play no role.

As Wintrup (2021) notes, 15 out of the original 307 CHAs were in higher-level training or positions as of 2020. Whether this rate of promotion is adequate is a complex normative question that implicates not only the welfare of CHAs, but also the Government of Zambia’s financial constraints and competing policy objectives. As such, adopting only the CHAs’ perspective in arguing that career opportunities are inadequate, and therefore they are being harmed, misses the larger picture. But we are concerned that this is not even the CHAs’ perspective, as we describe in Section 4 below.

3. Impact of CHAs

According to Wintrup (2021), Ashraf et al. (2020a) claims that the “best” health workers are those who “visit the greatest number of households and conduct the greatest number of meetings.” In fact, Ashraf et al. (2020a) uses home visits and community meetings as key inputs, and presents primary data from a nationwide household survey, as well as clinical data from local health facilities, to demonstrate sizable impacts on health outcomes. This includes significant improvements in facility-based childbirth, well-child facility visits, breastfeeding, deworming treatment, nutrition, immunization, and sanitation practices. On one metric alone—the share of children under 5 who are underweight—we find a reduction of 25 percent. In sum, using rigorous methods including anthropometric measurement of nutritional status and transcription of household and clinic medical records, our experiment found that the career-oriented recruitment intervention improved the lives of thousands of vulnerable, impoverished children in underserved settings. We are puzzled as to why this positive impact on the health of the most vulnerable in these communities is mostly ignored in Wintrup (2021), which instead critiques us for not adequately attending to the “relationships that CHWs [sic] formed with people.” This again leads us to ask whether there is a full appreciation of the distinction between CHAs and traditional CHWs. While relationships are integral to all health care, CHAs are, first and foremost, medical providers, trained to prevent, diagnose, and manage medical conditions. We stand by our focus on objective health outcomes in our analysis of the RCT.

4. Harm

Wintrup (2021) argues that our research harmed thousands of CHAs. While we cannot observe what would have happened had we not conducted our study, let us suppose that, without the RCT, the Government of Zambia would have discontinued the CHA program, and the roughly 3,000 Zambians currently employed as CHAs would not have been hired. Would these individuals have been better off as traditional CHWs? Several factors indicate otherwise. First, we have described above how the one-year CHA training is unique among the MOH’s professional courses in that tuition, room, and board are fully financed by the MOH; this training is far more extensive than the self-financed courses in community health typically available to traditional CHWs. Second, CHAs are paid up to 6.5 times the average per capita income in rural Zambia, and indeed, the CHA salary increased from USD 3480 to USD 6480 per year between 2012 and 2017; CHWs, in contrast, are mostly volunteers or periodic employees of NGOs. Finally, retention in the CHA program has been exceptionally high; for example, in our 2017 survey of the 307 original CHAs, after five years on the job, none had quit.

How can we reconcile this with the evidence in Wintrup (2021)? This is challenging for two reasons. First, we could find no information on the study’s sample, or sampling strategy, such that we read, “Many of the CHWs [sic] who were promised a future career are deeply frustrated with their situation,” but are not told how “many” CHAs the author spoke with, and among these, how many felt they were “promised” career promotions and how many were “deeply frustrated” by not having obtained them. To support his argument, the author cites a study that “found that many CHWs [sic] were frustrated in their current positions” (Zulu et al., 2014), suggesting that this frustration was due to thwarted career ambitions. This study, however, was based on interviews with 12 CHAs in a single district who had been on the job for only 1 year at the time of the fieldwork, and therefore were not eligible for promotions; indeed, the study does not once mention the CHAs’ feelings with regard to career opportunities.

Finally, and in contradiction to best practice, Wintrup (2021) uses leading questions, such as, “There is a problem because there isn’t really a career ladder for [the CHWs] is there?” This manner of interviewing leads to contradictory responses, such as, “Even though there is no career ladder, the [CHWs]...are eligible for training to become nurses. And this training can be paid for by the government.” This is precisely a career ladder.

We do not doubt that, as Wintrup (2021) reports, some CHAs are unhappy with aspects of their jobs. However, we do not believe that this proves that our study made thousands worse off by facilitating the creation of a professional, well-trained, adequately paid health cadre.

5. Conclusion

There is nothing more vital to a country’s health system than the quality of its human resources. The CHAs that provide the foundation for Zambia’s community health service delivery have shown great skill and dedication, with measurable health impact. In a world in which most opportunities for salaried work within the civil service require leaving one’s community, providing a career for talented individuals in rural, remote areas to be able to both develop their skills and remain in their underserved home communities, to which they are so committed, is one of the most powerful benefits to come out of Zambia’s CHA program. This should inspire even more opportunities and paths that can allow individuals to stay in their communities of origin and develop themselves at the same time.

In addressing the misunderstandings raised in Wintrup (2021), we hope we have been able to shed light on the complexity and depth of the context in which our research evolved and interacted with policy. We strongly support critical scholarship, especially when serious considerations such as research ethics and the welfare of vulnerable populations are at stake. RCTs do not have a monopoly on knowledge; in particular, we value qualitative and quantitative research, which is why, as we have described, we conducted both during the lead-up to the RCT, as well as throughout our engagement with public health-focused research in Zambia more broadly. Moreover, rigorous peer critique before and after publication remains a bedrock of scientific research—one that is aided by transparency and good will. We are grateful to the Journal for this exchange.