Systematic Review of the Effectiveness of Mass Media Interventions for Child Survival in Low- and Middle-Income Countries

Abstract

Through a systematic review of the literature, this article summarizes and evaluates evidence for the effectiveness of mass media interventions for child survival. To be included, studies had to describe a mass media intervention; address a child survival health topic; present quantitative data from a low- or middle-income country; use an evaluation design that compared outcomes using pre- and postintervention data, treatment versus comparison groups, or postintervention data across levels of exposure; and report a behavioral or health outcome. The 111 campaign evaluations that met the inclusion criteria included 15 diarrheal disease, 8 immunization, 2 malaria, 14 nutrition, 1 preventing mother-to-child transmission of HIV, 4 respiratory disease, and 67 reproductive health interventions. These evaluations were then sorted into weak (n = 33), moderate (n = 32), and stronger evaluations (n = 46) on the basis of the sampling method, the evaluation design, and efforts to address threats to inference of mass media effects. The moderate and stronger evaluations provide evidence that mass media-centric campaigns can positively impact a wide range of child survival health behaviors.

Beginning in the 1960s, ministries of health in low- and middle-income countries partnered with international organizations to implement interventions addressing maternal and child health. Although global child mortality rates have declined by 70% in the past 50 years, more than 7 million children under the age of 5 years still die each year. In 2012, the Child Survival Call to Action (http://apromiserenewed.org/A_Call_to_Action.html), a renewed international commitment to ending preventable child deaths by 2035, prompted a series of systematic reviews of the evidence for enhancing child survival and development in low- and middle-income countries through population-level behavior change (Fox & Obregón, 2014). As part of the Evidence Summit on Enhancing Child Survival and Development in Lower- and Middle-Income Countries by Achieving Population-Level Behavior Change (Evidence Summit), evidence review teams were tasked with synthesizing the evidence in six major domains: supporting children and caregivers, empowering communities, sustainable health systems and policy supports, gender dynamics, stigma and discrimination, and advances in science, technology, and innovation (Balster, Levy, & Stammer, 2014). This article aimed to supplement the work of the evidence review teams by focusing specifically on mass media interventions. Toward that goal, this article summarizes and evaluates the evidence for the effectiveness of mass media interventions for child survival–related health topics including diarrheal diseases, immunization, malaria, nutrition, preventing mother-to-child transmission of HIV, reproductive health, and respiratory infections.

The potential advantages of mass media (e.g., radio, television, newspapers) include their reach and frequency, control over message content and delivery, consistency, ease of translation into multiple languages, and relatively low cost per person exposed. Potential disadvantages include the difficulty of capturing audience attention in an increasingly cluttered media environment, the oft-criticized one-way flow of information, and the inability to individually tailor messages. The use of mass media to address public health concerns is not intended as a magic bullet, but rather as one of a number of potential intervention strategies that may be more or less appropriate under given circumstances.

Prior Published Reviews

The majority of published reviews to date on health topics relevant to child survival in low- and middle-income countries do not mention mass media interventions at all. Others cover mass media only tangentially, noting it as a possible strategy (Briscoe & Aboud, 2012; Chopra, Sharkey, Dalmiya, Anthony, & Binkin, 2012; Pegurri, Fox-Rushby, & Damian, 2005). Of the reviews that do address mass media health communication in low- and middle-income countries, many highlight HIV/AIDS and reproductive health interventions because they comprise the bulk of the published literature in this area (HIV: Bertrand, O'Reilly, Denison, Anhang, & Sweat, 2006; Myhre & Flora, 2000; Noar, Palmgreen, Chabot, Dobransky, & Zimmerman, 2009; reproductive health: Basten, 2009; Hornik & McAnany, 2001; Snyder, Nafissatou, & Badiane, 2003; Storey et al., 2011). HIV/AIDS interventions, with the exception of campaigns focusing specifically on preventing mother-to-child transmission (PMTCT), are beyond the scope of this review, but reproductive health interventions are relevant, since child timing and spacing is related to child survival.

Four prior reviews include substantial discussion of mass media campaigns promoting reproductive health in low- and middle-income countries. In a meta-analysis of integrated family planning campaigns that received technical assistance from the Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs (JHU/CCP) between 1986 and 2001, Snyder, Nafissatou, and Badiane (2003) identified an average effect size of .07 for mass media campaigns addressing the use of modern methods (on the basis of data from 15 of the 39 campaigns). In a review of mass media effects on fertility, Hornik and McAnany (2001) dedicated a section to the evaluations of family planning interventions in twelve low- and middle-income countries between 1976 and 1999. They found that although there is evidence for the effectiveness of mass mediated programs in increasing demand for contraceptive services at clinics, there is much less evidence for an effect on population-level behavior and even less for long-term effects. Once the intervention ceases, clinic demand tends to rejoin the preintervention trend. Basten (2009) reviewed edutainment interventions for reproductive health and found evidence suggesting that soap operas can be effective in changing attitudes and behaviors related to reproductive health. A report by Storey and colleagues (2011) summarized social and behavior change interventions in the area of reproductive health, maternal health, neonatal health, and child health and nutrition. After searching PubMed, Scopus, and Popline for studies published between 2000 and 2010, they identified 19 articles with mass media interventions in low- and middle-income countries. Of those, seven also meet our inclusion criteria as described below (six family-planning campaigns and one iron deficiency anemia campaign), and the rest either deal with HIV/AIDS or do not have a behavioral outcome. The authors concluded that mass media campaigns that are theory-based, targeted, and use multiple media channels can affect reproductive health behavior through ideation/social diffusion.

Two reviews, both written by Green (1989, 1999), that evaluate the effects of mass media on breastfeeding between 1980 and 1999 are also relevant. Although mass media-centric programs have been effective in improving breastfeeding behaviors, Green (1989, 1999) found that many of the intervention and evaluation designs do not permit causal inferences; other program components may have been responsible for some or all of the observed effect.

Wakefield, Loken, and Hornik (2010) covered a range of child survival health topics in a systematic examination of reviews and notable studies published after 1998 and retrieved from Medline, PsycINFO, Embase, Soclit, ERIC, and Communication and Mass Media Complete. They found moderate evidence for the effectiveness of mass media campaigns on family planning, immunization, and diarrheal disease, and weak evidence for mass media breastfeeding interventions. They concluded that programs in which mass media is part of a multifaceted intervention strategy are more likely to be successful than mass media alone and that one-off or episodic behaviors are more amenable to mass media effects than habitual or ongoing behaviors.

These reviews highlight the potential for mass media to impact child survival, but underscore the reality that health communication campaigns are almost always implemented in conjunction with other interventions making it difficult to isolate independent effects. Although randomized controlled trials are the gold standard for establishing cause and effect, they generally are not appropriate for mass mediated public health campaigns. For those campaigns, interventions with mutually reinforcing program components are widely considered best practice. Furthermore, policy interest is in campaigns that operate on a large scale and over time rather than in the tightly controlled and focused interventions that are well studied with randomized controlled designs. As a result, evaluators use a variety of methods to link mass media messages to behavior change and often, as with the intervention itself, the most successful evaluation is one that combines several methods.

Method

Search Strategy

This article provides a systematic review of the literature containing evidence about the effects of mass media interventions on child survival–related behavior change in low- and middle-income countries. It attempts to address what has been done, what has been successful, and what knowledge gaps remain. The article expands on the work of previous reviews by capturing a broader range of child survival health topics over a longer period and with more rigorous inclusion criteria. To be included, studies had to (1) describe a mass media intervention using the radio, television, or newspapers; (2) address a child survival health topic including diarrheal diseases, immunization, malaria, nutrition, PMTCT, reproductive health, or respiratory infections; (3) present quantitative data from a low- or middle-income country (a country designated by the World Bank as a low-income, lower-middle income, or upper-middle income economy); (4) use an evaluation design that compared outcomes using (a)preintervention and postintervention data, (b) treatment versus comparison groups, or (c) postintervention data across levels of exposure; and (5)measure and report a behavioral or health outcome. Descriptive studies, studies on general mass media effects as opposed to purposeful interventions, and studies reporting only outcomes in knowledge, attitudes or intentions were excluded.

We conducted a two-phase search strategy to locate as many full-text English-language evaluations as possible between 1960 and May 2013 in both the published and gray literature. First, we searched electronic databases including Communication and Mass Media Complete, Embase, PsychInfo, and Medline/PubMed for articles mentioning a communication related term in the title in conjunction with the name of a low- or middle-income country in the title or abstract and any of a set of terms associated with the seven health topics relevant to child survival (for an example, see online supplemental archives). The search returned 3,861 titles and abstracts that were screened for duplication and relevance (Figure 1). Of those, 119 full-text articles were considered relevant and were assessed for eligibility. In the second phase of the search, bibliographies of the retrieved articles, the search results of the Evidence Summit (Balster et al., 2014), and the website of the Center for Communication Programs were examined for relevant material resulting in 106 additional titles. The authors used a variety of methods to locate promising abstracts and recover the full-text documents which were then assessed for eligibility.

Figure 1.

PRISMA flow diagram.

A total of 120 full-text articles including 13 diarrheal disease, 15 immunization, 25 malaria, 16 nutrition, 5 PMTCT, 2 respiratory infections, and 44 reproductive health studies were excluded after careful review. In general, the excluded studies failed to meet several criteria: 52% percent did not have a purposeful mass media intervention, 3% did not deal with one of the stated child survival health topics, 7% did not present quantitative data from a low- or middle-income country, 41% did not incorporate at least one comparison group, and 54% did not measure and report a behavioral or health outcome.

The search process resulted in a total of 106 articles that met the inclusion criteria including 11 diarrheal disease, 6 immunization, 2 malaria, 14 nutrition, 1 PMTCT, 4 respiratory infection, and 68 reproductive health studies (Table 1). All together, the studies describe 111 campaigns in 46 low- and middle-income countries. Some of the articles describe multiple campaigns (Hornik et al., 2002, for example, describes eight campaigns), whereas some campaigns are described in multiple papers. In addition, some campaigns address multiple child survival topics and were included in each of the health topics for which they provided behavioral outcome measures (e.g., the Smiling Sun Campaign in Bangladesh was included in both the immunization and reproductive health sections).

Table 1. Summary of systematic review search results.

Child survival health topic	Full-text articles assessed for eligibility	Full-text articles excluded with reasons	Studies included in synthesis	Campaigns
Diarrheal disease	24	13	11	15
Immunization	21	15	6	8
Malaria	27	25	2	2
Nutrition	30	16	14	14
PMTCT	6	5	1	1
Respiratory disease	6	2	4	4
Reproductive health	112	44	68	67
Total	226	120	106	111
Note. Some articles describe more than one campaign, and some campaigns are described by more than one article. PMTCT = Preventing mother-to-child transmission.

For each of the included studies, descriptive tables were created to capture important features of the mass media campaigns and their corresponding evaluations (accessible in online supplemental archives). These details were then transferred to an excel spreadsheet to facilitate a summary description of the interventions (accessible in online supplemental archives). As described in more detail below, evidence for the success of the mass media campaigns was assessed by categorizing the evaluations as weak, moderate, or stronger and drawing conclusions from the moderate and stronger evaluations about the potential for mass media campaigns to impact behavioral and health outcomes related to child survival.

Results

What Has Been Done?

We attempted to code the included campaigns on a number of features that the literature suggests may be relevant to the success of mass media campaigns. These include use of (a) formative research, (b) theory, (c) audience segmentation, (d) targeted messages, (e) message pretesting, (f) number of channels, and (g) duration and frequency of the campaign messages (Noar, 2006). Unfortunately, it is often not clear from the written evaluation whether each of these components was incorporated in the campaign design, rendering the information difficult to code. Details about formative research and message pretesting, although surely underreported, were relatively straightforward to code. Use of theory, however, was difficult to code because it was often unclear when authors claimed to use theory whether it was employed in the design of the campaign, the evaluation, or simply post hoc in the written article. Consequently, the results can more accurately be understood as reflecting the details provided by campaign evaluators rather than what the campaigns actually did or did not do. We will review the interventions by major health topics affecting child survival. For a summary of campaign regions, decades, and communication channels see Table 2.

Table 2. Summary of campaign region, decade, and communication channels.

Health topic	n	East Asia and the Pacific	Europe and Central Asia	Latin America and Caribbean	Middle East and North Africa	South Asia	Sub-Saharan Africa	1960s	1970s	1980s	1990s	2000	2010	Television	Radio	Newspapers	Interpersonal communication
Diarrheal disease	15	2	0	2	1	2	8	0	0	10	2	3	0	5	12	1	13
Immunization	8	1	0	3	0	2	2	0	0	3	4	1	0	6	8	3	6
Malaria	2	0	0	0	0	0	2	0	0	0	0	1	1	1	1	0	1
Nutrition	14	4	1	6	1	0	2	0	1	4	4	4	1	8	11	4	10
PMTCT	1	0	0	0	0	0	1	0	0	0	0	1	0	0	1	0	0
Respiratory disease	4	1	0	2	0	1	0	0	0	0	3	1	0	4	4	4	3
All child survival	44	8	1	13	2	5	15	0	1	17	13	11	2	24	37	12	33
		18%	2%	30%	5%	11%	34%	0%	2%	39%	30%	25%	5%	55%	84%	27%	75%
Reproductive health	67	8	1	15	8	11	24	4	10	11	28	13	1	37	60	19	25
		12%	1%	22%	12%	16%	36%	6%	15%	16%	42%	19%	1%	55%	90%	28%	37%
Note. PMTCT = preventing mother-to-child transmission.

Diarrheal Disease

A total of 11 studies describing 15 diarrheal disease campaigns met our inclusion criteria. The campaigns took place primarily in Sub-Saharan Africa (Burkina Faso, Burundi, Democratic Republic of the Congo [formerly Zaire], The Gambia, Ghana, Kenya, Lesotho, and Swaziland) with additional campaigns in Ecuador, Egypt, India, Indonesia, Mexico, and Pakistan. Ten campaigns took place in the 1980s, two in the 1990 s, and three after the year 2000. Eleven of the campaigns promoted the use of oral rehydration therapy, either a homemade water-sugar-salt solution or manufactured packets of oral rehydration solution, to prevent dehydration during diarrheal episodes. The diarrheal disease campaigns that did not promote oral rehydration therapy addressed proper stool disposal and/or handwashing (Curtis et al., 2001; Scott, Schmidt, Aunger, Carbrah-Aidoo, & Animashaun, 2008), water purification (JHU/CCP, 2008), or increased fluid intake (but not specifically oral rehydration therapy) during episodes of diarrhea (Hornik et al., 2002). Of the studies, 80% described formative research, 80% mentioned a behavior change theory (most commonly the theory of reasoned action, health belief model, and social marketing), and 53% pretested campaign messages.

All of the campaigns targeted caretakers of children under the age of 5 years. One also targeted community leaders and important others (mothers-in-law, fathers, neighbors; Curtis et al., 2001). In general, the campaigns aimed to increase knowledge of the causes and consequences of diarrhea in young children and change beliefs and behaviors regarding the treatment of diarrhea. Only one campaign tried to increase social network discussion (Curtis et al., 2001) or change social norms (JHU/CCP, 2008). In terms of channels, 80% of the campaigns used the radio and 33% used the television, most often in the form of radio and television spots (47% and 27%, respectively). One campaign also used a television drama and television programs (JHU/CCP, 2008), one used a song played on the radio (Scott et al., 2008), and one used radio programs (Curtis et al., 2001). Pamphlets (60%), posters (40%), and billboards (33%) were widely used, whereas only one campaign used newspapers (JHU/CCP, 2008), two used mobile cinema (Hornik et al., 2002; Kenya et al., 1990), two used promotional materials (Hornik et al., 2002; Kassegne, Kays, & Nzohabonayo, 2011), and one used flipcharts (Hornik et al., 2002). Of the diarrheal disease campaigns, 87% included a complementary interpersonal communication component either in the form of training of service providers (40%), community health workers (27%), or retailers (7%), or community events (20%) and school-based activities (20%). Most of the campaigns lasted fewer than 2 years, with five campaigns lasting longer (Curtis et al., 2001; Fox, 1998; Gutiérrez et al., 1996; Hornik et al., 2002; Miller & Hirschhorn, 1995). Only three campaigns provided any information about the frequency of message delivery, indicating that messages were aired daily (Fox, 1998; Kassegne et al., 2011; Miller & Hirschhorn, 1995; Scott et al., 2008).

Immunization

Six studies describing eight immunization campaigns met our inclusion criteria. The campaigns took place in Bangladesh, Democratic Republic of the Congo, Ecuador, Lesotho, Mexico, Peru, and the Philippines in the 1980s and 1990s. Only one campaign took place after the year 2000 (Guilkey & Hutchinson, 2011). The campaigns generally focused on more than one of the routine childhood vaccinations including polio (75%); diphtheria, pertussis, and tetanus (63%); measles (88%); and tuberculosis (50%). They aimed to increase utilization of vaccination services by informing parents of (a) the role vaccinations play in preventing common childhood illnesses and (b) when and where vaccinations are offered free of charge. One campaign also mentioned engaging in advocacy (Guilkey & Hutchinson, 2011). Of the studies, 75% described formative research and 75% mentioned a behavior change theory including the theory of reasoned action, health belief model, entertainment-education, social cognitive theory, and applied behavioral analysis. Sixty-three percent claimed to have pretested messages.

In general, the immunization campaigns targeted caretakers of children under age 5 years using radio (100%), television (75%), and newspapers (38%). The most common format was radio and television spots, but one campaign also used a television drama and quiz show (Guilkey & Hutchinson, 2011) and one campaign used a radio song (Pérez-Cuevas et al., 1999). Pamphlets and posters were used by half of the campaigns, whereas only two used promotional materials (Hornik et al., 2002), and one each used billboards (Guilkey & Hutchinson, 2011), flipcharts (Hornik et al., 2002), or mobile loudspeakers (Quaiyum, Tunon, Baqui, Yum, & Khatun, 1997). All but two of the immunization campaigns included an interpersonal component, most commonly immunization days (Hornik et al., 2002; Pérez-Cuevas et al., 1999; Quaiyum et al., 1997), the training of service providers (Hornik et al., 2002), community events (Guilkey & Hutchinson, 2011; Quaiyum et al., 1997), and activities in schools (Hornik et al., 2002). Most of the campaigns lasted less than 1 year, with only two extending for longer periods of time (Hornik et al., 2002). None of the campaigns provided information about the frequency of message delivery.

Malaria

Only two studies describing malaria campaigns met our inclusion criteria. One campaign in The Gambia was not really a mass media campaign, but was included in this review because it had the potential to be scaled up and because evaluations of mass media campaigns addressing malaria are rare. In this campaign, audio messages in the form of songs encouraging people to repair holes in their bed nets were disseminated on cassette tapes in a Gambian village between August and November 2003 (Panter-Brick, Clarke, Lomas, Pinder, & Lindsay, 2006). The Knock Out Malaria campaign in Cameroon comprised two 3-month campaign waves (July to October 2011 and April to July 2012). The campaign engaged famous local artists to promote consistent use of bed nets through catchy radio songs and spots and television music videos that quickly went viral (Bowen, 2013). The Knock Out Malaria campaign also used billboards and mobile phone texts to disseminate messages. Both studies described formative research and a theoretical framework, but neither explicitly mentioned pretesting campaign messages. The messages were directed to the general public, encouraging everyone to sleep under well-maintained mosquito nets.

Nutrition

Fourteen studies describing fourteen nutrition campaigns met our inclusion criteria. The campaigns took place in Bolivia, Brazil, China, Honduras, Indonesia, Jordan, Kazakhstan, Mali, Mexico, Trinidad and Tobago, and Uganda. The interventions addressed a wide variety of nutrition topics including breastfeeding, complementary feeding, and/or adequate nutritional intake. Breastfeeding campaigns often focus on the importance of early initiation, giving colostrum, breastfeeding exclusively for the first 6 months, continued breastfeeding for 2 years, and the timely introduction of complementary foods (Ferreira Rea & Berquo, 1990; Gueri, Jutsum, & White, 1978; Gupta, Katende, & Bessinger, 2004; Hornik et al., 2002; Huffman, Panagides, Rosenbaum, & Parlato, 1991; Monterrosa et al., 2013). Interventions addressing complementary feeding of children between 6 and 24 months of age emphasized not giving food and water until 6 months of age, continued breastfeeding, hands-on feeding practices, meal frequency, and meal diversity (Bonvecchio et al., 2007; Huffman et al., 1991; Monterrosa et al., 2013; Sun et al., 2011). Adequate nutritional intake interventions promoted the consumption of supplements/fortified foods (Bonvecchio et al., 2007; Sun et al., 2011; Sun, Guo, Wang, & Sun, 2007; Warnick et al., 2004) or foods naturally rich in essential nutrients such as vitamin A (De Pee et al., 1998; Hornik et al., 2002; Monterrosa et al., 2013; Parvanta, Gottert, Anthony, & Parlato, 1997) or iron (Baizhumanova et al., 2010; Monterrosa et al., 2013; Sun et al., 2007). Of the studies, 57% described some formative research, 71% mentioned a theoretical framework, most commonly social marketing, the theory of reasoned action, and the health belief model, and 36% pretested messages.

The nutrition campaigns targeted primarily women of reproductive age. Two campaigns each also targeted men of reproductive age (Gupta et al., 2004; Parvanta et al., 1997), health workers (Hornik et al., 2002; Sun et al., 2011), or caregivers of children 6–24 months old (Bonvecchio et al., 2007; Sun et al., 2011). Radio was used by 79% of the campaigns, television by 57%, and newspapers by 29%. Radio and television spots were the most common format used by 57% and 43% respectively, but television programs (Sun et al., 2011), radio programs (Huffman et al., 1991, Monterrosa et al., 2013), radio dramas (Parvanta et al., 1997), and radio songs (Huffman et al., 1991) were also used by some campaigns. In addition, 57% of campaigns used posters and/or pamphlets and a couple used flipcharts (Bonvecchio et al., 2007; Parvanta et al., 1997), promotional materials (Sun et al., 2007; Warnick et al., 2004), billboards (De Pee et al., 1998), or mobile loudspeakers and/or videos shown in health care facilities (Bonvecchio et al., 2007). Of the nutrition campaigns, 71% included an interpersonal communications component in the form of training of service providers or community health workers (Gupta et al., 2004; Hornik et al., 2002; Sun et al., 2011) and two held community events (Sun et al., 2007; Warnick et al., 2004). One of the nutrition campaigns lasted less than a month (Monterrosa et al., 2013) and two ran, intermittently, for more than 12 months (Ferreira Rea & Berquo, 1990; Huffman et al., 1991), but the majority of the campaigns were between 1 and 12 months long. Only three studies mentioned that campaign messages aired daily; the rest provided no frequency information.

PMTCT

Only one PMTCT campaign met our inclusion criteria. The study described a twice-weekly serial drama aired on national radio in Botswana between 2001 and 2003 (Kuhlmann et al., 2008). The drama included characters and storylines that illustrate both the negative consequences of not getting tested for HIV during pregnancy and the benefits of PMTCT services. The study did not describe any formative research or message pretesting but mentioned social cognitive theory and the Modeling and Reinforcement to Combat HIV (MARCH) framework.

Respiratory Diseases

Although the search for mass media campaigns addressing respiratory diseases included both pneumonia and tuberculosis, only four tuberculosis campaigns (and no pneumonia campaigns) met our inclusion criteria. The campaigns took place in Colombia, Peru, and Vietnam in the 1990s and in India in 2001. Messages informed the general public about the symptoms of tuberculosis and encouraged anyone with a cough lasting more than 15 days to seek treatment. The messages emphasized the treatability of the disease, often free at local clinics, and the severity if left untreated. Two campaigns also tried to reduce stigma towards people with tuberculosis (Jaramillo, 2001; Llanos-Zavalaga, Poppe, Tawfik, & Church-Balin, 2004). One campaign specifically targeted community leaders in addition to the general public (Thuy, Huong, Tawfik, & Church-Balin, 2004). Three of the four studies described formative research, but only one mentioned theory (Jaramillo, 2001), and none mentioned pretesting messages. All of the tuberculosis campaigns used radio, television, and newspapers to disseminate campaign messages through a wide variety of formats including spots, talk shows, and dramas. A few campaigns also used videos shown in health care facility waiting areas (Llanos-Zavalaga et al., 2004), posters (Sharma et al., 2005; Thuy et al., 2004), and pamphlets (Thuy et al., 2004). Three out of the four campaigns supplemented the mass media with interpersonal communication, primarily community events and training of service providers (Llanos-Zavalaga et al., 2004; Thuy et al., 2004). The duration of the tuberculosis campaigns varied from 6 weeks (Jaramillo, 2001) to intermittently over the course of 13 years (Llanos-Zavalaga et al., 2004) and none of the campaigns provided information about the frequency of emissions.

Reproductive Health

Sixty-eight reproductive health studies describing 67 campaigns across the globe between the 1960s and 2013 met our inclusion criteria. For the most part, the campaigns were general family planning/birth spacing campaigns that aimed to increase uptake of modern contraceptive methods and services among men and women of reproductive age. Often campaigns targeted specific sub-groups including women (72%), men (63%), married people (28%), people of low (Bailey, Janowitz, Solis, Machuca, & Sauzo, 1989; Kincaid et al., 1996; Singhal & Rogers, 1999), middle (Agha & Beaudoin, 2012; Agha & Meekers, 2010), or high (Foreit, de Castro, & Duarte Franco, 1989) socioeconomic status or people of urban (Agha & Beaudoin, 2012; Agha & Meekers, 2010; Babalola & Brown, 2001; Babalola, Vonrasek, Brown, & Traore, 2001) or rural (Singhal & Rogers, 1999; Sypher, McKinley, Ventsam, & Valdeaellano, 2002) residence. A few campaigns also targeted health workers (Basten, 2009; Boulay, Storey, & Sood, 2002; Kim, Kols, Nyakauru, Marangwanda, & Chibatamoto, 2001; Palmer & Sood, 2004; Shefner-Rogers & Sood, 2004), community leaders (Basten, 2009; Kim & Marangwanda, 1997), and religious authorities (Blake & Babalola, 2002). In addition to general family planning campaigns, there were three vasectomy campaigns, five adolescent/young adult reproductive health campaigns, two maternal mortality campaigns, and a radio program in Nepal that covered a wide range of child survival behaviors. The vasectomy campaigns were designed to improve attitudes towards vasectomy in Brazil and Guatemala and encourage married men with completed families to undergo the procedure (Bertrand, Santiso, Linder, & Pineda, 1987; Foreit et al., 1989; Kincaid et al., 1996). The reproductive health campaigns for adolescents, youth, and young adults each targeted a slightly different age range between 12 and 24 years (Agha, 2002; Babalola, Folda, & Babayaro, 2008; Kim et al., 2001). In general, they aimed to change social norms surrounding the use of reproductive health services and products by young people, encourage young people to frequent family planning clinics, to talk with parents, educators, and service providers about sexual health, and to protect themselves from pregnancy and sexually transmitted infections. The maternal mortality campaigns both took place in Indonesia between 1999 and 2002 in an attempt to increase male and community involvement in preparing for emergencies during pregnancy and delivery (Palmer & Sood, 2004; Shefner-Rogers & Sood, 2004). BBC Media Action's radio program, aired in Nepal between 2009 and 2011, addressed a host of child survival topics including antenatal care, safe delivery, postnatal care, newborn care, breastfeeding and complementary feeding, diarrhea, and acute respiratory infection (Basten, 2009).

Of the studies evaluating reproductive health mass media campaigns, 48% discussed formative research and 21% mentioned message pretesting. Seventy-eight percent mentioned a theoretical framework, most commonly entertainment-education (39%), diffusion theory (24%), social marketing (22%), social cognitive theory (16%), the theory of reasoned action (15%), the health belief model (12%), stages of change (10%), the ideation model (9%), and/or steps to behavior change (9%). Radio was used by 90% of the campaigns, television by 55%, and newspapers by 28%. Although radio and television spots were the most common format, used by 42% and 36% of the campaigns respectively, 27% used radio dramas, 18% used television dramas, and 25% used radio talk shows. In addition, campaigns disseminated messages via pamphlets (33%), posters (30%), films (12%), and promotional materials (10%). Only a few campaigns used music videos (Babalola & Brown, 2001; Babalola et al., 2001), television talk shows (Guilkey & Hutchinson, 2011; Hess, Meekers, & Storey, 2012; Yassa & Farah, 2003), radio songs (Babalola & Brown, 2001; Babalola et al., 2001; Kane, Gueye, Speizer, Pacque-Margolis, & Baron, 1998), cassette tapes (Blake & Babalola, 2002; Jato et al., 1999), mobile sales units (Sweeney, 1977), mobile cinema (Kabir & Islam, 2000), mobile loudspeakers (Guilkey & Hutchinson, 2011; Lieberman, 1972; Schellstede & Ciszewski, 1984), billboards (Bailey et al., 1989; Kabir & Islam, 2000; Kincaid et al., 1996; Schellstede & Ciszewski, 1984; Singhal & Rogers, 1999; Sweeney, 1977), or flipcharts (Blake & Babalola, 2002; Hess et al., 2012). Several campaigns used authority figures or famous people to bolster their messages including ministers of health in Bolivia and the Philippines (Kincaid & Do, 2006; Valente & Saba, 2001; Valente & Saba, 1998), religious leaders and the royal family in Jordan (Yassa & Farah, 2003), a popular Indonesian singer (Palmer & Sood, 2004; Shefner-Rogers & Sood, 2004), a famous Egyptian actress (Robinson & Lewis, 2003), famous Nepali artists (Basten, 2009), and professional Zimbabwean soccer players (Kim & Marangwanda, 1997). In addition to mass media, 38% of the campaigns included an interpersonal communication component, most commonly in the form of community events (29%) or training of community health workers (11%) or service providers (8%). Of the campaigns, 15% also engaged in franchise development efforts, attempting to associate a logo with quality services at participating clinics (Babalola & Brown, 2001; Babalola et al., 2001; Babalola & Vonrasek, 2005; Do & Kincaid, 2006; Guilkey & Hutchinson, 2011; Gupta, Katende, & Bessinger, 2003; Jato et al., 1999; Kane et al., 1998; Kim et al., 2001; Olaleye & Bankole, 1994; Robinson & Lewis, 2003). Only three campaigns explicitly engaged in advocacy efforts (Basten, 2009; Guilkey & Hutchinson, 2011; Yassa & Farah, 2003). Of the campaigns, 40% lasted less than 1 year, 30% lasted between 1 and 2 years, and 12% lasted for more than 2 years. Eighteen percent of the campaigns did not give any information about the duration of the campaign. Of the campaigns, 61% did not provide details on the frequency of message emissions. Of those that did provide frequency information, half aired messages on a weekly basis and half more frequently.

Evaluation of the Evidence

To draw conclusions about the effectiveness of mass media interventions for child survival, the evaluations were sorted into stronger, moderate, and weak categories on the basis of how thoroughly they addressed threats to inference of mass media effects. Typically, intervention evaluations can be considered as tests of efficacy (does the intervention work under controlled conditions), tests of effectiveness (does the intervention work under realistic conditions or in multiple settings) or tests of sustainability (does the intervention continue to be effective after its early period of introduction; Balster et al., 2014). Almost all of the evaluations reviewed here fall into the middle category, of effectiveness studies; all included studies were operating under reasonably realistic conditions and almost none of them included longer term implementation or data collection which would permit claims of sustainability. The descriptions below point to any exceptions.

Methods for establishing the validity of claims of mass media effects consist of both design elements and statistical techniques. In this section, we present the criteria for characterizing the primary components of the evaluation method including sampling method, timing of data collection, and use of statistical controls or advanced statistical methods to address threats to inference. Decisions about each of these components affect confidence in the inferences regarding campaign effectiveness.

Sampling Method

We distinguish between samples likely to be biased or unrepresentative of the population to which evaluators wish to generalize, and samples less likely to be biased. Samples considered biased may be very small, homogeneous, have a high nonresponse rate or are in other ways at substantial risk of being unrepresentative of the target population. These samples do not permit generalization to the larger population of interest and therefore are of limited value to this review.

Timing of Data Collection

The evaluations included in this review all compare groups either (a)across individuals who have different levels of exposure postintervention, (b) across time before and after the intervention, or (c) across outcomes between treatment and control sites. Each of these comparison approaches has strengths and weaknesses for supporting claims of intervention effects. Some studies use more than one of these comparison strategies to strengthen claims of mass media effects.

In the case of postintervention only designs, analysts use individual exposure data to compare groups who vary in their exposure levels. In most cases, variation in exposure is defined by self-reports of individual respondents. Although useful, this design is threatened by self-selection and causal order challenges. Self-selection, as opposed to random assignment to condition, is a concern because exposure to mass media messages is rarely the only difference between exposed and unexposed groups. Therefore, observed effects may not be due to exposure but rather to confounding variables that affect both exposure and the outcome of interest. This threat can be reduced by controlling statistically for as many known potential determinants of exposure and the outcome behavior as possible. Nevertheless, unmeasured confounders could still pose a threat. Studies that rely entirely on this postintervention comparison of individuals with more or less self-reported exposure, but that do not make statistical adjustments for confounders, are considered to have weak designs.

Ambiguous causal order is also a threat to inference with postintervention only designs. Because both exposure and outcome measures are collected at a single point in time, often both through self-report, it is impossible to know whether exposure drives the behavior or whether the behavior drives recall of exposure. Three ways to address this threat are to show that greater levels of exposure are associated with incrementally greater values on the outcome behavior (dose-response), to observe campaign-related change on the beliefs targeted by the mass media messages and not on other related beliefs, or to trace a pathway from exposure through the theoretically based mediators targeted by the mass media campaign (e.g., attitudes, self-efficacy, social norms, intentions) to the outcome behavior. Researchers may be able to argue that evidence consistent with these elaborations of the basic association will be consistent with one causal order (intervention on outcome) but not the reverse.

Another common method of campaign evaluation, the pre-/postintervention survey design, examines whether outcome measures gathered from similar samples before and after an intervention are changing. This design is particularly threatened by history or other interventions, events or natural (secular) changes that occur simultaneously with the intervention and may be responsible for the observed effects. This simple comparison is generally considered to be a weak design. The value of the evidence from such a design is strengthened if it includes some additional information. For example, the threat of history can be somewhat reduced by also showing that levels on the outcome variable vary across levels of exposure postintervention, thereby linking observed effects to the mass media campaign. Another method for minimizing the threat of history is to collect time-series data (often in the form of clinic service statistics or sales data) that show that outcomes vary in expected ways with the presence or absence of a time-varying mass media intervention. The pre-post intervention survey design is also threatened by the possibility of nonequivalent samples. Using a panel design (with checks for attrition) or randomly selected samples following a consistent sampling method can attenuate this threat.

The third typical design compares treatment versus control geographic areas. When used for evaluating mass media campaigns, this comparison is most often a quasi-experimental design in which the treatment and control sites are not randomly assigned. This design may be threatened by non-comparable baselines and differential natural rates of change over time in addition to history (if other events only in the treatment site could explain observed differences). These threats can be mitigated by increasing the number of matched treatment and control sites to permit analyses using the site, rather than the individual, as the unit of analysis. If that is not possible, it may be helpful to collect multiple pre-intervention measurements to establish comparable baseline trends or to measure exposure and establish differential effects by exposure levels, thus combining the several types of designs. We consider designs that use more than one of these design elements to be stronger than those that use only one.

All of the evaluations that draw conclusions from a substantially biased sample (or that do not report the sampling method) are classified as weak because their findings are not generalizable to the target population. In addition, studies with only one comparison group (before/after, low versus high exposure, or treatment versus control) that do not use statistical controls to adjust for potential a priori differences between the treatment and comparison group are considered weak because no effort is made to address those threats to inference. The evaluations classified as moderate use an unbiased sample and make some effort to address threats to inference either by employing two comparison group approaches (a combination of before/after, low versus high exposure, and/or treatment versus control), or one comparison group and basic statistical controls. Stronger evaluations are those that have an unbiased sample and make a substantial effort to address threats to inference through a combination of multiple comparison groups, statistical controls, and, in some cases, advanced statistical methods like propensity score matching, bivariate probit models, or fixed effects analysis.

Of the 111 evaluations that met the inclusion criteria, 33 were classified as weak, 32 as moderate, and 46 as stronger (see Table 3). Of the weak evaluations, 88% made no attempt to address threats to inference, 30% did not report the sampling method, and 21% suffered from a biased sample. The weak evaluations are not described further because they do not present convincing evidence for the effects of mass media interventions for child survival. The results section describes the 78 campaigns with moderate and stronger evaluations divided into a section for reproductive health (46 campaigns) and a section for all other child survival health topics (32 campaigns). For a summary of the evaluation design for moderate and stronger campaigns, see Table 4.

Table 3. Summary of assessment of evaluation strength.

Child survival health topic	Strength of evaluation
	Weak	Moderate	Stronger
Diarrheal disease	2	10	3
Immunization	1	3	4
Malaria	1	0	1
Nutrition	5	3	6
Preventing mother-to-child transmission	0	1	0
Respiratory disease	3	1	0
Reproductive health	21	14	32
Total	33	32	46

Table 4. Summary of evaluation design for the moderate and stronger campaign evaluations.

Health topic (number of campaigns per topic)	n	Moderate evaluation	Stronger evaluation	Low exposure: 0–30%	Moderate exposure: 31–60%	High exposure: 61–100%	Probability sample	Sample size: 1,000+	Statistical controls	Advanced statistical methods	Effect of exposure	Change over time	Difference between treatment/control	Observed effect on intermediary outcomes	Observed effect on behavior	Observed effect on health outcome
Diarrheal disease	13	10	3	5	2	5	13	11	8	1	3	12	2	6	10	0
Immunization	7	3	4	3	2	2	7	6	6	2	2	6	0	2	5	0
Malaria	1	0	1	0	0	1	1	1	1	1	0	1	0	0	1	0
Nutrition	9	3	6	1	5	1	5	3	6	3	3	9	2	5	8	1
PMTCT	1	1	0	0	0	1	0	0	1	0	1	0	0	0	1	0
Respiratory diseases	1	1	0	0	1	0	0	0	0	0	0	1	1	0	1	0
All child survival	32		14 44%	9 28%	10 31%	10 31%	26 81%	21 66%	22 69%	7 22%	9 28%	29 91%	5 16%	13 41%	26 81%	1 3%
Reproductive health	46	14 30%	32 70%	9 20%	16 35%	20 43%	41 89%	33 72%	36 78%	13 28%	31 67%	28 61%	9 20%	28 61%	41 89%	0 0%
Note. PMTCT = preventing mother-to-child transmission.

What Has Been Successful? Evaluation of the Evidence

Child Survival

Adequate exposure is a key component of campaign success; the campaign must reach substantial proportions of the target audience with enough frequency to be recalled. Of the child survival campaigns, 31% achieved high exposure defined through various techniques for eliciting self-reported exposure (61–100%), 31% moderate exposure (31–60%), 28% low exposure (0–30%), and 9% did not report exposure. Six of the studies asked about exposure by channel (Bowen, 2013; Curtis et al., 2001; De Pee et al., 1998; Gupta et al., 2004; Quaiyum et al., 1997; Scott et al., 2008), but only two provided any sort of indication of frequency of exposure to the campaign (Kuhlmann et al., 2008; Monterrosa et al., 2013).

There appears to be a publication bias in the literature toward successful campaign evaluations. Of the 32 child survival evaluations, only 6 found no evidence of effect on a behavioral outcome: two on oral rehydration therapy and two on vaccination coverage (Hornik et al., 2002), one on water purification (JHU/CCP, 2008), and one on exclusive breastfeeding (Gupta et al., 2004). Four of these six evaluations were published as part of a chapter reviewing the effectiveness of 10 HealthCom programs in eight countries. The HealthCom project attributed the failure of diarrheal disease and immunization campaigns in Democratic Republic of the Congo, Lesotho, and Indonesia-West Java to weak mass media interventions and low exposure levels (Hornik et al., 2002). Exposure to a marketing campaign in Pakistan to promote the use of water purification tablets was successful initially and then sales decreased because people did not like how the product changed the taste of the water (JHU/CCP, 2008). The evaluation of a mass media campaign to promote exclusive breastfeeding in Uganda found effects of exposure on knowledge of exclusive breastfeeding, but not on behavior, perhaps because the evaluation was conducted prematurely, only 2 months after the campaign began (Gupta et al., 2004).

Of the 32 child survival evaluations, 26 show positive effects of mass media interventions on a wide range of self-reported behaviors including antenatal care during the most recent pregnancy (Hutchinson, Lance, Guilkey, Shahjahan, & Haque, 2006), vaccination coverage (Hornik et al., 2002; Hutchinson et al., 2006; Quaiyum et al., 1997; Zimicki et al., 1994), last night bed net use (Bowen, 2013), early initiation of breastfeeding (McDivitt, Zimicki, Hornik, & Abulaban, 1993; Sun et al., 2011), minimum dietary diversity (Monterrosa et al., 2013; Sun et al., 2011), consumption of iron-rich foods (Sun et al., 2011) and foods rich in vitamin A (De Pee et al., 1998; Monterrosa et al., 2013), handwashing with soap (Scott et al., 2008), and oral rehydration therapy (Gutiérrez et al., 1996; Hornik et al., 2002; Kassegne et al., 2011; Kenya et al., 1990; McDowell & McDivitt, 1990). Three evaluations show effects on somewhat more objective behavioral outcomes including observed handwashing after cleaning a baby's bottom (Curtis et al., 2001), number of tuberculosis tests performed (Jaramillo, 2001), and vitamin A serum status (De Pee et al., 1998). ¹

In addition, two campaigns showed dose-response effects on behavior (JHU/CCP, 2008; Hornik et al., 2002) and one traced a mediation pathway from exposure to knowledge to vaccination coverage (Hornik et al., 2002). In addition to behavioral effects, twelve evaluations found positive effects on awareness/knowledge and less than four each on beliefs/attitudes (JHU/CCP, 2008; Kassegne et al., 2011; Monterrosa et al., 2013; Sun et al., 2007), self-efficacy (Kassegne et al., 2011), social norms (JHU/CCP, 2008; Monterrosa et al., 2013), intentions (Monterrosa et al., 2013; Sun et al., 2007), and ideation (JHU/CCP, 2008).

All of the moderate and stronger evaluations relied on face-to-face interviews except for one that used clinic service statistics (Jaramillo, 2001). The majority of the face-to-face interviews were administered to probability samples, most commonly multistage cluster samples, of greater than 1,000 respondents. Used in 88% of the studies, the pre-/postintervention survey design allows campaign evaluators to examine change over time. Seven evaluations gathered data at three or more time points (Curtis et al., 2001; De Pee et al., 1998; Gutiérrez et al., 1996; Hornik et al., 2002; Kenya et al., 1990; McDowell & McDivitt, 1990), one of which was paired with a quasi-experimental design (Kenya et al., 1990). The rest of the pre-/postevaluations are single before-/after-measurement designs combined with either statistical controls (Bowen, 2013; Hornik et al., 2002; Sun et al., 2011), propensity score matching (Hornik et al., 2002), a quasi-experimental design (Hornik et al., 2002), a panel sample (Bonvecchio et al., 2007; Quaiyum et al., 1997; Sun et al., 2007), a quasi-experimental design and panel sample (Monterrosa et al., 2013) or additional analyses across levels of exposure (Gupta et al., 2004; Hornik et al., 2002; Kassegne et al., 2011; Scott et al., 2008; Warnick et al., 2004). One quasi-experimental study employed time series data from before, during, and after a tuberculosis mass media campaign (Jaramillo, 2001). Three evaluations used postintervention-only data across levels of exposure. The evaluators attempted to compensate for the weak after-only design by using extensive statistical controls (Kuhlmann et al., 2008), propensity score matching (JHU/CCP, 2008), or bivariate probit modeling (Guilkey & Hutchinson, 2011).

Reproductive Health

Exposure to reproductive health campaigns was measured through various self-report techniques by all but one of the campaigns with 43% achieving high exposure levels (61–100%), 35% moderate exposure levels (31–60%), and 20% low exposure levels (0–30%). Thirty-five percent also provided exposure information by channel, but only 7% provided information on frequency of exposure (Do & Kincaid, 2006; Lettenmaier, Krenn, Morgan, Kols, & Piotrow, 1992; Rogers et al., 1999).

Four evaluations had mixed results. In Zimbabwe, evaluators found effects of exposure on contraceptive use, but no significant changes between baseline and follow-up (Kim & Marangwanda, 1997). In St. Lucia, differences in current modern contraceptive use between baseline and endline were not statistically significant, but differences between radio soap opera listeners and nonlisteners were significant at one time point (Vaughan, Regis, & St. Catherine, 2000). The evaluation of a male motivation campaign in Guinea detected significant effects of exposure on contraceptive ideation and intention to use a modern method within a panel of men and women, but not on contraceptive behavior (Blake & Babalola, 2002). A campaign to promote midwives and community preparedness for birth emergencies in Indonesia found effects of exposure on the use of a skilled birth attendant, but no significant increases in that indicator over time (Palmer & Sood, 2004). It is possible that the evaluations were conducted too soon to detect population-level effects on behavior even though campaign exposure influenced intentions and behavior.

Two evaluations found no evidence for effects. In Peru, observed increases in new family planning acceptors at clinics did not exceed predicted levels on the basis of precampaign trends even though exposure to the mass media campaign was high (>60%; Hornik et al., 2002). In South Africa, comparisons over time between a treatment and control area registered a negative net effect on condom use for pregnancy prevention among women and no net effect on other contraceptive use behaviors (Agha, 2002). The authors attributed the result to mass media messages aired on a newly established community radio station with low coverage levels.

Of the 46 evaluations, 40 showed positive effects of mass media-centric reproductive health interventions on a behavioral outcome, most commonly, use of modern contraceptives (30 campaigns). Of the evaluations that found effects on contraceptive use, 28 were based on self-report as were evaluations that found significant effects on number of sexual partners (Agha, 2002), ever use of condoms (Meekers, Van Rossem, Silva, & Koleros, 2007), and delivery preparedness (Shefner-Rogers & Sood, 2004). ² On the basis of time series data from clinics, rather than self-report, evaluations found significant effects on number of vasectomies performed (Bertrand et al., 1987; Foreit et al., 1989; Kincaid et al., 1996), number of new family planning acceptors (Bailey, 1973; Lieberman, 1972; Piotrow et al., 1990), and on modern contraceptive use (Agha, 2002; Boulay et al., 2002). An evaluation that used condom sales data found a positive effect on condom purchasing behavior (Schellstede & Ciszewski, 1984).

In addition, evaluators found positive effects on beliefs/attitudes (41%), interpersonal discussion (26%), awareness/knowledge (24%), intentions (13%), self-efficacy (Agha, 2002; Sweeney, 1977), and social norms (Boulay et al., 2002). Eight campaigns demonstrated dose-response effects (Babalola & Vonrasek, 2005; Blake & Babalola, 2002; Gupta et al., 2003; Hindin, Kincaid, Kuma, Morgan, & Kim, 1994; Jato et al., 1999; Kim & Marangwanda, 1997; Kincaid, 2000; Lettenmaier et al., 1992) and one traced a mediation pathway through exposure to ideation to current use of modern contraceptives (Kincaid, 2000).

Five campaigns relied on clinic statistics (Foreit et al., 1989; Kincaid et al., 1996; Hornik et al., 2002), distribution data (Schellstede & Ciszewski, 1984) or a quota sample (Singhal & Rogers, 1999). The rest administered face-to-face interviews to probability samples, most commonly multistage probability samples, of greater than 1,000 respondents. The evaluators used a wide range of evaluation designs that permit comparisons across levels of exposure (67%), across time (61%), and across treatment and control sites (20%). The most common designs were the single pre-/postintervention survey (46%) and the postintervention across levels of exposure design (41%). Of the evaluations, 78% used at least some statistical controls and 20% or less used panel data, a quasi-experimental design, time series data from clinic/sales statistics, propensity score matching, or bivariate probit models.

What Knowledge Gaps Remain?

In general, the existing evaluations of health topics related to child survival fail to consistently provide all the details about the campaign design, exposure, and evaluation that are necessary to conduct a meta-analysis and to better understand what features of the campaign are most associated with success. Future campaign evaluations should be sure to include explicit information about formative research, use of theory in the campaign design and the evaluation, audience segmentation, message development and pretesting, channels used, duration and intensity of the campaign, population exposure levels broken down by channel and detailing how exposure was measured, frequency of exposure, sampling method, sample size and population, and analyses performed, including control variables and statistical significance of differences between comparison groups.

More specifically, with the exception of reproductive health, there are relatively few evaluations of mass media campaigns for child survival that permit conclusions to be drawn about the effectiveness of mass media as an intervention strategy. More rigorous and recent evaluations across a more diverse range of target behaviors are needed for all of the child survival health topics, but especially in the areas of malaria, PMTCT, and respiratory diseases (which each only have one rigorous evaluation, to date). Studies evaluating the cost-effectiveness of mass media would also be useful. Only two of the moderate and stronger studies provided cost-effectiveness data, concluding that mass media interventions can be very cost-effective: $0.05 per additional antenatal care user, $0.30 and $0.36 for each additional child vaccinated for measles and diphtheria, pertussis, and tetanus, respectively, and $1.62 per additional person protected by a mosquito net (Bowen, 2013; Guilkey & Hutchinson, 2011). It is also important to better understand the duration of mass media effects. Only one study examined whether or not the positive effects of mass media on behavior persisted over time (McDowell & McDivitt, 1990). The evaluators found that 3 years after a campaign to promote the use of a homemade water-sugar-salt solution in The Gambia, both knowledge and use had decreased sharply.

With respect to reproductive health, the evaluations reviewed here suggest that mass media-centric reproductive health interventions can affect the use of modern contraceptive methods in a variety of contexts. Nevertheless, more research is necessary on a wider range of reproductive health outcomes that are important to child survival, for example, antenatal and perinatal service utilization. Cost-effectiveness studies would also be valuable as none of the studies provided cost-effectiveness data. In addition, evaluators should test theoretical mechanisms of effect. Often, the proximate determinants of behavior (beliefs, attitudes, intentions) are measured, but mediation analyses are not performed. These additional analyses can both help strengthen claims of effect and contribute to the theoretical understanding of how mass media campaigns change behavior. More long-term evaluations should be conducted to better understand how quickly the effects of mass media interventions dissipate across different behaviors.

Discussion and Conclusions

There are three classes of conclusions that can be drawn from this systematic review of the evidence for the effects of mass media campaigns on child survival. Although overlapping, it is useful to distinguish between conclusions about the available literature, substantive conclusions about mass media effects, and conclusions about the quality of existing evaluations.

First, the 111 evaluations reviewed here describe mass media campaigns for child survival that took place between 1960 and 2013. Of the moderate and stronger campaign evaluations, 34 (52%) took place before the year 2000, so truncating the search within the past ten or even 20 years would neglect a large part of the relevant literature. In addition, there are surprisingly few rigorous evaluations of communications campaigns for most of the focal child survival health topics with the exception of reproductive health. Furthermore, the evaluations that are available reflect a publication bias: Weaker campaigns are less likely to be evaluated at all, and evaluations of campaigns that show poor results are less likely to be written up and to find publication. As a result, the high proportion of successful campaigns reviewed here (81%) surely overestimates the proportion of all mass media campaigns that are successful.

This publication bias makes it difficult to draw substantive conclusions about mass media effects because we lack information about failed communications campaigns. From the few unsuccessful campaigns that have been published, we can glean several insights. First, there must be room to move the population on the target behavior. In Democratic Republic of the Congo and Lesotho, for example, where baseline vaccination rates were already high, it was difficult to detect effects of the mass media campaign (Hornik et al., 2002). In addition, exposure to campaign messages must be high enough to reasonably expect an effect; low exposure to immunization and diarrheal disease campaigns may have been responsible for the failure of mass media campaigns in Democratic Republic of the Congo, Indonesia, and Lesotho (Hornik et al., 2002). If the evaluation does not respect the model of effect for a given behavior in a given context, the campaign will be deemed ineffective even though it might have eventually produced a positive result. Conducted only two months after the campaign launch, the evaluation of an intervention to promote exclusive breastfeeding in Uganda did not take into account the time-frame required to influence a complex behavior and found no effects on exclusive breastfeeding (Gupta et al., 2003).

From the successful campaigns with moderate and stronger evaluations, we conclude that interventions with mass media campaigns can positively impact a wide range of child survival health behaviors in low- and middle-income countries around the world. These include one-off behaviors such as tuberculosis testing or vasectomy, episodic behaviors such as vaccinations, use of oral rehydration therapy, and early initiation of breastfeeding, and habitual behaviors such as nightly bed net use, handwashing, consumption of iron and foods rich in vitamin A, and use of modern contraceptives. In addition, evaluations show effects across theoretical frameworks, channels, target audiences, message types and styles, and evaluation designs.

The quality of the evaluations reviewed here leaves room for improvement. Of the evaluations that met our inclusion criteria, 33 were too weak to draw conclusions about the effectiveness of mass media. Many of the weak evaluations did not provide enough information to inspire confidence in the results, omitting essential details about sampling method, sample size, statistical controls, analyses performed, and the statistical significance of differences between groups. Others used biased samples and/or did not make any effort to address threats to inference.

In the future, evaluators should address threats to inference of mass media effects by using unbiased samples, multiple comparison groups across time, levels of exposure, and treatment and control sites, statistical controls and advanced statistical methods, and data triangulation. The written report should reflect the measures taken to mitigate threats to inference. Published evaluations of mass media campaigns should provide detailed information about the campaign, exposure, and the evaluation to permit meta-analyses as the literature base grows.

In sum, there is evidence that mass media interventions can be effective for addressing a wide range of health behaviors related to child survival. There is further evidence that a number of approaches can be effective. The publication of additional rigorous evaluations of both successful and unsuccessful communications campaigns is necessary to better understand which components of campaign design are associated with successful campaigns. Studies should strive to better understand the duration of mass media effects and to evaluate the cost-effectiveness of mass media campaigns as compared with other types of health interventions.

Supplemental Material

Supplemental data for this article (Appendix: Descriptive Tables and Spreadsheets Summarizing Studies Reviewed) can be accessed on the publisher's website at http://dx.doi.org/10.1080/10810730.2014.918217.