Collapsing the Vertical–Horizontal Divide: An Ethnographic Study of Evidence-Based Policymaking in Maternal Health

Abstract

Using the international maternal health field as a case study, we draw on ethnographic research to investigate how public health researchers and policy experts are responding to tensions between vertical and horizontal approaches to health improvement. Despite nominal support for an integrative health system approach, we found that competition for funds and international recognition pushes professionals toward vertical initiatives. We also highlight how research practices contribute to the dominance of vertical strategies and limit the success of evidence-based policymaking for strengthening health systems. Rather than support disease-and subfield-specific advocacy, the public health community urgently needs to engage in open dialogue regarding the international, academic, and donor-driven forces that drive professionals toward an exclusive interest in vertical programs.

Debates about vertical versus horizontal approaches to health improvement have a lengthy unresolved history in public health.1 Vertical approaches are generally disease specific and promote targeted clinical interventions delivered by a specialized service. Horizontal approaches, by contrast, tackle several interrelated health issues by strengthening health systems and developing integrated delivery systems.1^–3

Despite prolonged efforts to combine vertical and horizontal approaches, vertical programs have dominated and are often found competing with one another for funds and professional recognition.1^,4^,5 Authors have warned that disproportionately concentrating funds into disease-based initiatives in developing countries may compromise health systems and fragment complex interventions.6^,7 An increasingly popular compromise approach is to devote general health resources to a limited package of interventions prioritized on the basis of cost-effectiveness. This approach has thus far failed to enable synergy between vertical and horizontal approaches.1

One reason for continuing tensions relates to the difficulty of producing evidence of effectiveness for evidence-based policymaking in a wide array of social, political, and health system contexts.8 Critics claim experimental research, originally developed to assess the effect of targeted clinical interventions on a measurable outcome, is unsuitable for investigating the population-level, nonclinical, and context-specific health system domains.9^–12 In response, authors have begun calling for non-experimental epidemiological methods and an interdisciplinary approach.13^–17

Professionals in the international maternal health subfield are currently grappling with how to improve vertical and horizontal synergy.18^–20 This has lead to lively debate on the role of evidence production. Whereas some call for the scientific rigor of randomized controlled trails,21^–24 others claim that using randomized controlled trials is misplaced because of the complex health systems needs of maternal health interventions.25^–28

Using the subfield of maternal health as a case study, we explore 2 questions: What are the main challenges faced in bringing vertical and horizontal approaches together? What are the social and epistemological factors that constrain researchers from producing evidence for synergistic vertical and horizontal policymaking?

METHODS

Our research focused on debate at the international level because of our interest in developments that are critical to the field’s overarching reputation and identity. Conceptually, we explored how the maternal health field has emerged as a coherent and recognizable network of specialist actors, technologies, and ideas.29^,30

We triangulated 3 methods—open-ended, in-depth interviews; participant observation; and review of published and gray literature documents—to improve the validity of findings and explore diverse perspectives.31 We interviewed a total of 67 professionals (Table 1 ▶), identified opportunistically through professional networking, publications, and conference proceedings. Of 67 informants, 19 were from developing countries. Many informants had experience working in multiple domains of public health.

TABLE 1—

International Interview Participants: October 2004–June 2007

	No. of Participants
International academic researchers	19
UN agency representatives	10
Donor body representatives	8
International NGO representatives	16
National-level researchers	8
National-level policy experts and program managers	6
Total	67

Note. UN = United Nations; NGO = nongovernmental organization.

Interviews followed a semistructured guide to explore definitions of evidence and evidence-based policymaking, professional experiences with production and use of evidence for policymaking, historical shifts in policy, debates around integration and health systems strengthening, and relationships with donors. Using an inductive process, we modified the interview guide to reflect concerns that emerged during data collection.31

We conducted participant observation within academic settings, as well as at 15 research meetings, academic conferences, and policy meetings. Of these, 8 were not specific to maternal health but focused on general public health or child, neonatal, or reproductive health. All informants were interviewed formally once, although participant observation enabled repeated contact with many of the informants. Because key players in the field often represent their views via publications, we used published and gray literature as forms of ethnographic data.

We transcribed interviews verbatim and stored and organized them in NVivo7 version 7.0 (QSR International, Cambridge, Massachusetts). Both authors read the interviews and notes taken during participant observation and carried out thematic analysis independently. No significant discrepancies in identified themes and analytic conclusions were found.

RESULTS

Horizontal Versus Vertical Programmatic Approaches

Over the past 20 years, the maternal health field has undergone 2 significant conceptual shifts, first toward and then away from vertical approaches. In 1987, the launch of the Safe Motherhood Initiative aimed to separate maternal health from child health to highlight the much neglected issue of maternal mortality.32 Although the field of maternal health is still considered by some to be weak, this initiative has succeeded in rallying support for maternal mortality and in garnering support for vertical interventions, such as antenatal risk screening, training traditional birth attendants, and providing emergency obstetric care.19^,33

Maternal health specialists have also increasingly recognized that vertical interventions cannot be delivered without a functioning health system. By definition, this implies integration of vertical interventions used within maternal health and greater collaboration with other subfields.14^,20^,27^,33 Support for this position has resulted in widespread interest in coordinating initiatives, reflected most recently in the merging of 3 separate partnerships into the Partnership for Maternal, Neonatal, and Child Health.13^,34^–36

In interviews, we asked informants to reflect on the implications of this history for improving vertical–horizontal synergy. The most prominent issue informants mentioned was the pressure to support vertical approaches because of an intense sense of competition between subfields. As one informant stated:

The maternal health field really competes against other fields for money. And other fields, like the big spenders—malaria, HIV/ AIDS, even child health—have a better record of promoting evidence-based interventions. Maternal health might be at risk of being left behind, because if you miss the target too often, with traditional birth attendant training, then risk screening, you create donor fatigue.

Anxieties around how donors view the relative importance of health problems were paramount. “All fields have that anxiety,” said one policy expert. “Maternal health had its heyday, and newborn health is now having its heyday. They’re all scared they won’t get the attention and money they had before.” Several informants claimed that integration, although theoretically sensible, would in actuality divert funds and policy attention from maternal health. As one academic stated, “I think the jury is out on whether [the fields will integrate] or whether one will get sucked into the other’s agenda and get lost.” The lack of funds for strengthening comprehensive health systems added to the view that strategies being promoted in related subfields such as child and neonatal health counter those needed in maternal health.

In general, the greater the sense of competition and threat, the more liable informants were not only to reject integration but also to endorse the view that a focused vertical approach is more effective in capturing the attention of funders and policymakers.

Informants demonstrated academic support for this position by making reference to policy studies (in particular, Schiffman37) that have highlighted the importance of maternal health-specific “focusing events” and “political champions.” Making comparisons with global programs such as Integrated Management of Childhood Illnesses, these informants highlighted the need to establish a simple and unifying set of policies that is easy to market to politicians and donors. One such policy expert argued that maternal health’s “very sad history” could be attributed to “a failure of strategy” and that “the question [now] is whether this constituency can get its act together and push more effectively.”

By contrast, a minority of informants felt that the “attention-seeking strategies of vertical initiatives such as [Global Alliance for Vaccines and Immunisation and] Roll Back Malaria” were disempowering because they alienated subfields from one another and fragmented initiatives within each subfield. These informants explained that maternal health experts have attempted to bolster the field’s reputation by searching for a single targeted vertical intervention, or “magic bullet,” that would appear to be globally applicable and feasible to donors and governments.

The search for a single intervention was not only reductionistic, some argued, but contributed to infighting and the constant shifting of proposed vertical interventions, from training traditional birth attendants to antenatal care to emergency obstetric care, each vying for policy attention. Such dynamics resulted in the splintering of what could be a comprehensive community and facility-based health systems approach into specific targeted subcomponents, or, as one policy expert described, isolated “bits of the jigsaw puzzle.” These informants claimed that the search for new, targeted vertical solutions ironically had the opposite effect than originally intended. Rather than boost the field’s reputation, the picture that emerged to donors and governments was that of an uncoordinated and divisive group.

The Role of Researchers and the Limitations of Current Evidence-Based Models

Polarization of academic researchers and policy experts.

Our second research question examined the factors that constrain researchers from producing evidence that enables synergistic vertical and horizontal policymaking. Our results indicate that researchers were hindered by a detrimental polarization that positions the academic community in stark opposition to a group we termed policy experts. In broad terms, this group includes professionals from UN agencies, international nongovernmental organizations, and developing country governments.

Our informants’ attention to this polarization reinforced opposing views regarding the relative importance of advocacy and program development versus research for ensuring the field’s survival. In general, researchers felt policy experts were more deeply involved in the process of advocating for political and financial investment in maternal health. Researchers reluctantly accepted the need for such advocates, even if what they espoused was empirically unfounded. One informant claimed,

There would not be a penny of funding if people listened to me. . . . I’m too negative. Some people are good spokespersons for Safe Motherhood. [They] will stand up and say things; they know there is no data behind it, but they will keep saying it. And it gets the work done.

Other researchers went further, claiming that the field’s failures relate directly to an insufficient “evidence-based approach” that was partially caused by advocates’ “militant” style. As one interviewee noted,

When people became aware of the M in MCH [Maternal and Child Health], the field was dominated by people on a mission, and while it is good to have such people, because they are the ones who attract attention and bring money, if it is too exclusive, you will miss the scientific rigor and skepticism.

In contrast to this critique, policy experts frequently held researchers responsible for paralyzing action and political will by emphasizing the scientific uncertainty of the current evidence base. These informants claimed that research often directly contradicted policy experts’ need to persuade donors of the importance of maternal mortality and suitability of a particular programmatic approach. As one informant highlighted,

The big challenge is that there’s uncertainty no matter what. And policymakers have to deal with uncertainty. When it’s uncertain, the retrospectoscope is going to prove that you were wrong in your efforts to be certain. Policymakers can’t sit on the fence. Researchers can.

Some policy experts even claimed that investing resources in effectiveness research would undermine the field by diverting attention and funds from much-needed programs. As one senior policy expert described,

This field has been so contentious because there hasn’t been enough money. If [only] there had been money to do both research and [develop] programs in the way that child health has had money. . . . This contentiousness causes donors to turn around and run in the opposite direction, so it’s a vicious cycle.

At the same time that informants put forth such dichotomizing statements, several researchers were well aware that tensions between research, advocacy, and policymaking needed to be assuaged for the sake of the field’s professional coherence and future success. In response, some researchers explicitly devoted considerable attention to what they termed “advocacy research,” such as estimating the global magnitude of maternal health problems compared with other diseases. Researchers highlighted the political importance of this work, even if some claimed this type of research does not answer analytic questions relating to programmatic development and evaluation.

Policy experts and researchers are clearly in a mutually interdependent, if tumultuous, relationship. When asked to reflect critically on this relationship, informants often made reference to the rapidly expanding body of literature on communication problems between academic researchers and policy experts.38 Indeed, several respondents felt that these difficulties were at the core of failed effectiveness for evidence-based policymaking and argued for improved communication channels, more effectively disseminating new evidence, and capacity building for each respective group.

Diverting attention from questions of epistemology.

Although important to elucidate, the intensive focus on improving communication diverted our informants’ attention from engaging with epistemological questions relating to evidence-based health system policymaking. Despite growing debates regarding the limitations of current epidemiological methods for health systems questions, few informants spontaneously engaged in discussions about research models. Rather, several repeatedly espoused the superiority of the randomized controlled trials design for providing definitive proof of the causal relationship between intervention and outcome, irrespective of the type of intervention being evaluated. With the randomized controlled trial, said one statistician, “you don’t need to understand how the interventions work” to establish its relative advantage. Another claimed that

no design can [control confounding] as the randomized controlled trials. One should probably always aim at doing randomized controlled trials. If you want to deviate from this rule you should have very good reasons.

It was only after prompting informants to describe specific instances involving the use and interpretation of evidence for policy that issues relating to the limitations of experimental research emerged. Core to these discussions were the logistical, ethical, and analytic difficulties of conducting effectiveness research on horizontal approaches. As one informant explained,

Designing a study for skilled attendance at delivery is [very difficult] because how the hell do you do a trial of a midwife versus no midwife or a midwife versus a traditional birth attendant? It becomes a very difficult medical and organizational dilemma. Do you get women to deliver at home and women to deliver at hospital?

Informants interested in health systems questions (e.g., budget support and human resource strengthening) expressed frustration at the scientific method’s inability to adequately research these topics. As one informant argued,

It’s really hard to measure the impact, you know, what are you measuring? And the line of attribution [from budget support] down to improvements in maternal health outcomes is also difficult.

According to many informants, resistance to the Partnership for Maternal, Neonatal, and Child Health’s promotion of the continuum of care approach is based largely on difficulties relating to affect evaluation:

If you want to say the continuum of care is the answer, how do we validate and monitor that? How do we say it was proven to work, what are the outcomes, how many lives are saved?

Despite such frank discussions, most informants rarely questioned their own epistemological positions or ventured into new methodological and disciplinary arenas. Rather, they modified their research questions—specifically, the types of interventions being tested and the units of analysis used—to suit an experimental or quasi-experimental design. Most often this meant avoiding questions relating to health systems strategies and focusing on vertical clinical interventions, such as the effect of calcium supplementation or oxytocin administration. These informants explained that clinical research will always be relevant to policy and that such research allows them to carve out their own area of expertise and publish successfully.

Other informants more committed to studying health systems issues attempted to overcome the limitations of experimental study designs by testing only a single subcomponent of a larger health systems package. Examples include the effect of road construction or introducing mobile phones and ambulances on health utilization rates. As one informant explained, conclusively evaluating complex multi-component interventions is such a challenge that “people are avoiding those kinds of studies and instead proposing studies like ‘what if we put an ambulance in the villages? Will that do it?’” However, as another informant aptly summarized, the practical implications of using the randomized controlled trials for multicomponent interventions are tremendously complex:

To do a [sic] good randomized controlled trials, you have to ask a very narrow question. There isn’t enough money in the world to answer all the questions with randomized controlled trials. So people say, ‘we’ll put three things together that we think work and then we’ll test that against no change.’ But it’s highly unlikely that all of [the components] are equally cost effective or that you need all to be synergistic. You could take a few and get the same amount of change. . . . Your hypothesis could be that it’s any one or the combination of factors or even some synergy about using certain ones together. To test all those combinations is impossible!

Reasons for the predominant research focus on vertical interventions.

The normative power of scientific values surely persuades researchers to abide by experimental designs. However, informants highlighted other important reasons for the predominant focus on experimental studies of vertical interventions. Results from studies that clearly demonstrate the effectiveness of a single specific subcomponent were said to generate consensus, to be easier to disseminate to policy experts, and to have more straightforward applications in policy development. Vertical studies were also allegedly preferred by donors, who demanded to see a return on their investments by encouraging governments to implement policies for which both intervention and outcome could easily be monitored. Informants felt mounting pressure to use evidence about the relative cost-effectiveness of different subcomponents to help governments in developing countries with resource allocation.

A less explicit reason for informants’ reticence to deviate from experimental designs relates to the field’s low status and to the issues of competition reviewed in the “Results” section. Referencing a recent publication,23 a number of informants claimed that because the lack of an evidence-based approach in maternal health has compromised the field’s standing, only the highest research standard should now be accepted. Contextual, observational epidemiology, and multidisciplinary research were not viewed as proper academic research and were often relegated to the less scientific realm of operations research. As one international policy expert described, “Health systems research can’t really ever tell us much, other than at a highly contextualized level.” One researcher stated that only those in well-established subfields who are “starting from the top” can afford to take on the professionally risky activity of pushing the limits of epidemiological theory and methods. Maternal health, by contrast, is starting from the bottom and, therefore, needs more-rigorous experimental studies to be able to provide conclusive recommendations and secure its status.

Other informants were more critical of this position, stating that the scientific community’s insistence on using randomized controlled trials has created a dogmatic and detrimental donor demand for experimental evidence. As a couple researchers stated, the indiscriminate use of the randomized controlled trials often provides very rigorous answers to irrelevant questions. However, being bold and diverting from experimental designs means opening oneself up to criticism and potentially losing publications, funds and political credibility. As another epidemiologist stated,

I am so convinced of the argument. . . . But what makes policymakers shift? Do we need another beautiful trial showing that traditional birth attendants make no difference? I hope not. It’s not whether in the perfect circumstances you can train traditional birth attendants and supervise them. Of course that can make a difference. But then you’re talking about an expensive system; you might as well train skilled providers. . . . Quite a few people are calling for trials of community health workers . . . and the donors are taking note. If we’ve gone that far . . . what a waste of money. Maybe we have to play the game; I don’t know.

As this comment suggests, informants sometimes felt that reducing the focus of the research question to conduct randomized controlled trials was scientifically unnecessary but politically and professionally indispensable. At the same time, informants also frustratingly acknowledged that this approach reinforced the dominance of vertical approaches and compromised a health systems approach. As one researcher aptly summarized, the scientific attempt to discern if a particular community- or facility-based strategy has a greater effect on mortality over another “is just trappings, and feels like a waste of time and money. . . . I wouldn’t say one is better than the other, I would say if you neglect the nuts and bolts of the system, you risk getting nothing done.”

DISCUSSION

Many policy experts support the agenda to integrate subfields and wish to work toward health systems strengthening. In practice, however, the competitive playing field pressures policy experts to support subfield-specific initiatives and funding in an effort to bolster the field and advocate for resources and political will. These findings indicate that a distinction exists between what can be termed policy-relevant approaches and advocacy-sensitive approaches. The former respond to policymaking and program implementation needs, be they vertical or horizontal. The latter, by contrast, are used to advocate for the survival and status of the maternal health professional community and tend to be vertical approaches. Under the pressures of subfield competition, our results show that key experts are being pushed toward advocacy-sensitive practices, and because they are more vertical by nature, this is happening at the expense of practices that could more adequately respond to synergistic vertical–horizontal policymaking.

Maternal health researchers, in turn, respond to the pressures for financial support and professional prestige by aiming to produce evidence that is politically expedient, useful for securing their academic reputations, and able to ensure the survival of the maternal health community. For many, this means the use of experimental research to evaluate either clinically targeted interventions or vertical subcomponents of larger health systems packages. These dynamics impede researchers from following the lead of recent literature13^–17^,25^–28 that scrutinizes the suitability of an experimental clinical research model for questions relating to complex health systems interventions. As a result, the production of useful evidence for horizontal policymaking, as well as for vertical–horizontal synergy, is sorely lacking.

The issues of rivalries over funding, diverse donor-driven agendas, and what informants describe as the “false and damaging” dichotomies between maternal and child health, as well as between community versus facility-based interventions, have received considerable attention in the literature.19^,39^–41 The recent Lancet series19 on maternal survival had as one of its main aims to “provide an opportunity to mark a shift [away] from unhelpful dichotomies that slow action in countries [and] stifle funding.”4^(p9) Given the results of our analysis, we must question whether such high-profile statements will have the desired effect of joining diverse factions if they do not (or cannot) address the factors that drive vertically oriented programs and research.

Policy researchers have argued that significant obstacles to a coherent policy agenda on vertical–horizontal synergy include weak health systems, current priority-setting mechanisms based on uncritical support for traditional disease ranking and cost-effectiveness measures, and uncoordinated and conflicting donor agendas on which many developing countries are reliant.42^–45 In addition to these obstacles is the crucial issue of how to improve evidence-based policymaking practices.

The literature suggests that poor communication between researchers and policymakers is a key constraint to improving evidence-based policymaking.46^,47 Our informants have clearly been influenced by this literature, yet our findings suggest that the limited ability of experimental methods to provide evidence about integration and complex health systems is a more important impediment. This is being increasingly recognized in the public health and sociological literature.17^,48^,49 As we have shown, when it comes to actually changing evidence-based practices, the messages ensuing from this body of research remain theoretical and difficult to operationalize.

CONCLUSIONS

On the basis of our findings, we suggest modifying evidence-based policymaking practices in 2 main ways. First, it is important to create institutional environments that actively promote the development of new research models for investigating complex and context-specific interventions. As we and other authors have shown, context-specific health systems research contradicts the need in public health for a generalizable and marketable evidence-base of vertical programs that are easy to evaluate and show a measurable impact on outcomes.10^,50^,51 A major challenge for public health lies in prioritizing context-specific horizontal initiatives even where impact cannot be as precisely shown as in the case of vertical interventions.52^,53 This is particularly the case in developing countries, where vertical initiatives to reduce mortality quickly are vital and, yet, where progress in general development requires active intersectoral collaboration and wide-ranging social initiatives.54

Second, it is important to examine the larger international, donor-driven, political, and academic factors that persuade policy experts and researchers to adhere to current normative models of vertical programming and evidence production. Otherwise, competition will continue to encourage subfield-specific advocacy, give preeminence to vertical and subcomponent interventions, and push researchers toward the uncritical application of experimental methods. Continuation of the status quo may also lead to the marginalization of one of epidemiology’s primary contributions to public health: that of identifying interrelated determinants of disease patterns and mechanisms of change.