Abstract
Given the high morbidity and mortality among children in low- and middle-income countries as a result of preventable causes, the U.S. government and the United Nations Children's Fund convened an Evidence Summit on Enhancing Child Survival and Development in Lower- and Middle-Income Countries by Achieving Population-Level Behavior Change on June 3–4, 2013, in Washington, D.C. This article summarizes evidence for technological advances associated with population-level behavior changes necessary to advance child survival and healthy development in children under 5 years of age in low- and middle-income countries. After a rigorous evidence selection process, the authors assessed science, technology, and innovation papers that used mHealth, social/transmedia, multiplatform media, health literacy, and devices for behavior changes supporting child survival and development. Because of an insufficient number of studies on health literacy and devices that supported causal attribution of interventions to outcomes, the review focused on mHealth, social/transmedia, and multiplatform media. Overall, this review found that some mHealth interventions have sufficient evidence to make topic-specific recommendations for broader implementation, scaling, and next research steps (e.g., adherence to HIV/AIDS antiretroviral therapy, uptake and demand of maternal health service, and compliance with malaria treatment guidelines). While some media evidence demonstrates effectiveness in changing cognitive abilities, knowledge, and attitudes, evidence is minimal on behavioral endpoints linked to child survival. Population level behavior change is necessary to end preventable child deaths. Donors and low- and middle-income countries are encouraged to implement recommendations for informing practice, policy, and research decisions to fully maximize the impact potential of mHealth and multimedia for child survival and development.
Despite the overall decline in under-5 mortality rates from 12.6 million in 1990 to 6.6 million in 2012, 21,000 under-5 child deaths occur daily (You, Bastian, Wu, & Wardlaw, 2013). Many of these under-5 child deaths could be averted with wider deployment of existing prevention and treatment interventions. Healthy behaviors on the part of families, communities, and health systems (e.g., completing vaccinations, using oral rehydration, breast feeding, spacing pregnancies, handwashing, and home compliance with treatment guidelines) could dramatically accelerate reductions in under-5 mortality (Black, Morris, & Bryce, 2003; Lozano et al., 2011; Rajaratnam et al., 2010). Health professionals, governments, and researchers have implemented a number of interventions to address the health of children in low- and middle-income countries (LMICs), often focusing on improving the coverage and quality of interventions of known efficacy. To date, there has been no systematic review of the science, technology, and innovation-based support related to the effectiveness of evidence-based interventions that could aid in the elimination of under-5 child morbidity and mortality in LMICs.
Communication and media platforms have been used to promote change in health behaviors for decades. Printed pamphlets, billboards, radio, television, the Internet, and now mobile devices have all been used to communicate information to a range of populations (Free et al., 2013; Naugle & Hornik, 2014). Rapid changes have occurred in technologies and their availability, as well as in the ways individuals and groups use and interface with these technologies. As with technological developments of the past, the modern emergence of mobile health (mHealth) and new social/transmedia platforms affords another opportunity for reaching, teaching, connecting, motivating, and empowering individuals and groups to address specific health concerns.
Despite the relatively nascent state of these fields, at present, globally thousands of activities are underway across the reproductive, maternal, newborn and child health continuum that leverage mHealth and social/transmedia strategies (Labrique, Vasudevan, Chang, & Mehl, 2012). These activities have the potential to impact maternal, neonatal, and child health survival by changing caregiver behavior and catalyzing and improving the delivery of interventions. These strategies also have the ability to improve access to information and enhance community-driven demand for quality services (Labrique et al., 2012).
The Evidence Summit on Enhancing Child Survival and Development in Lower- and Middle-Income Countries by Achieving Population-Level Behavior Change was intended to review critical evidence to inform LMIC governments, donors, and development practitioners on the interventions effective in changing behavior to reduce under-5 mortality and enhance healthy child development (Pablos-Méndez & Fox, 2012). Six evidence review teams (ERTs) were recruited to consider various research areas related to behavior change and child survival (Balster, Levy, & Stammer, 2014). The broad goal for the science, technology, and innovation (STI) ERT was to (a) review the evidence and determine which STI behavioral interventions result in population-level behavior change that could contribute to child survival and healthy development; and (b) make informed recommendations to LMICs, donors and development practitioners on practice, programs, policy, and research. A related review was prepared for this Summit which focused on mass media interventions (Naugle & Hornik, 2014). Our article reports results from the STI ERT, which focused on four areas: mHealth, social/transmedia, health literacy, and devices. For health literacy and devices, there were an insufficient number of studies to support causal attribution of interventions to outcomes or perform a substantial evidence review; thus, we focused our efforts on mHealth, social/transmedia, and other media. The evidence review included areas such as use of mHealth for compliance with antiretroviral treatment in adults with HIV, which has both direct implications (e.g., perinatal transmission of HIV and transmission via breast feeding) and indirect implications (e.g., therapeutic compliance) for child survival. We present these results in detail to ensure (a) visibility on the adequacy of study design for intended area of evidence generation and (b) reporting of critical context (e.g., rural vs. urban) and implementation elements that may be helpful when considering adaptation or adoption. We also provide insight into the challenges to conducting high-quality research on the effectiveness of technology platforms and gaps that these types of interventions can fill to improve child survival.
Method
The STI ERT set out with the goal of reviewing the literature and making recommendations on four STI areas and their ability to contribute to behavior change relating to child survival and healthy child development: mHealth, health literacy, medical devices, and social/transmedia. The literature was tiered using a systematic, structured approach to prioritize studies that enable causal attribution of the STI intervention on behavior resulting in desired health outcomes, the behavior itself, or on knowledge attitudes and practice (see Appendices A and B in the supplemental online appendices for additional information). After conducting an assessment of the relevance, quality, and strength of evidence on the effective utilization of mHealth and social/transmedia platforms to advance behavior changes necessary to prevent or avert child deaths, we make specific recommendations for practice, policy, and research. Further detail regarding the evidence selection strategy can be found in this special issue of the Journal of Health Communication (Balster et al., 2014).
mHealth Evidence Selection Strategy
mHealth, for the purposes of this review, was defined as mobile technologies used to address the health needs of individuals, communities, and societies (Istepanian, Laxminarayan, & Pattichis, 2006). A wide range of mHealth strategies was considered for this review. However, only those strategies that directly influenced or modified behaviors contributing to child health outcomes were included. Interventions that might influence behavior indirectly, such as supply chain management strategies, were excluded from the analysis. Researchers from the U.S. Agency for International Development (USAID) conducted a preliminary literature review in January 2013. Only three mHealth studies were found that met the inclusion criteria (i.e., peer-reviewed research literature from LMICs since 1990 examining interventions that included a comparison group and targeted health behavior outcome data relevant to child survival); therefore, the ERT collected additional mHealth evidence. Through an internal “call for evidence” (which enabled ERT members to identify and submit additional peer-reviewed and gray literature not identified during the initial systematic review), both gray and peer-reviewed literature were identified. The mHealth evidence reviews were manually screened to identify additional studies that met the inclusion criteria. A total of 15 studies were found to meet criteria for assessing causal attribution to health outcomes. Papers were reviewed by ERT members for quality (see Balster et al., 2014, for method), sorted by intervention type such as appointment reminders, medication adherence messages, and maternal education (see Appendix A in the supplemental online appendices), and synthesized to inform recommendations related to population-level behavior change.
Social Media and Transmedia Evidence Selection Strategy
Although definitions of social/transmedia vary, for the purpose of this article, we used the following operational conceptions: Transmedia is a related group of media content with linkages across media forms and platforms, which may or may not have a narrative component, including traditional (e.g., radio, TV dramas, street theater) and new media technologies (e.g., SMS, social media; Jenkins, 2006). For example, a transmedia intervention could consist of a health education intervention disseminated through television and print media, complemented by SMS messages and group discussion through social media. This more inclusive definition of transmedia is used in order to capture the broader evidence relevant to the child health field (Weinreich, 2013) and includes multiplatform media efforts that are not part of some definitions of transmedia. Social media refers to the interaction and exchange among and between individuals and virtual communities through a variety of different platforms, including blogs, wikis, podcasts, forums, message boards, review and opinion sites, and social networking (Chou, Hunt, Beckjord, Moser, & Hesse, 2009; Hawn, 2009).
Similar to the mHealth evidence selection strategy, the STI ERT attempted to identify evidence on social/transmedia interventions that supported behaviors that accelerated reductions in under-5 mortality as well as optimized health and protective child development. However, of the 61 studies from the preliminary literature search conducted by USAID in January 2013, none was deemed relevant. Because no literature was found using the original criteria, the STI ERT worked with the National Institutes of Health Library to conduct a second literature search inclusive of the Cochrane Library, the Cochrane Central Register of Controlled Trials, a resource derived from regular systematic searches of bibliographic databases including CENTRAL, MEDLINE, EMBASE, PsycINFO, and Science Citation Index. An additional cross-check of all relevant literature included a range of online directories, including PubMed, EMBASE, Scopus, the Web of Science, and POPLINE. A comprehensive list of the key concepts, keywords, Relevant Medical Subject Headings were iteratively developed and tested in PubMed and subsequently adapted to other database searches (see Appendix B in the supplemental online appendices). We also took steps to capture evidence in the gray literature. This included searches on relevant organizations and websites, including the USAID Clearinghouse, determined through consultation with key stakeholders and content experts. Additional citations were sought through the reference lists of relevant documents.
We gathered 26 pieces of useable evidence though some of the literature focuses on multiple-media platforms and not transmedia or social media per se (Appendix C, in the supplemental online appendices, depicts the distribution of this literature by publication type). The references fall into one of four basic study categories: (a) experimental and quasi-experimental reports in peer-reviewed publications; (b) published literature reviews or other reports, or gray literature; (c) reports of experimental and quasi-experimental studies; and (d) more generalized synthesis.
Basis of Recommendations
Ideal evidence for recommendations to scale global health activities or interventions included the following: efficacy (does it work?), effectiveness (does is work in a variety of populations and contexts?), and sustainability data (cost-effectiveness, demand, adaptation into the health system, can supply meet demand at scale?; Higgs, Zlidar, & Balster, 2012; Luoto et al., 2013). Study designs vary necessarily by the question of interest and the target application of the study (e.g., individuals, communities, populations, health systems). For example, a randomized control trial is an appropriate design to address efficacy but not necessarily for optimal implementation or to determine cost effectiveness or other elements of sustainability.
Recommendations are based on strength of expert opinion—strongly (I), moderately (II), or weakly (III) recommended—and adequacy and quality of evidence—A, B, or C—as demonstrated in Figure 1. This two-part score is assigned to all practice recommendations for the use of mHealth and social/transmedia (see Table 1). Specific research recommendations for necessary next steps in evidence acquisition are provided as well and are based on the sequential development of evidence from efficacy, effectiveness, followed by sustainability with all three being ideal for global public health decision making. For example, if there are quality data to support effectiveness, but no sustainability data exist, the ERT will recommend implementation science studies or operational research to gain this information as a next step. The letter grade for the research recommendation simply refers to the ERT assessment of the current status of the strength of the evidence.
Table 1. Topic-specific practice and research recommendations.
Topic | Practice | Research |
---|---|---|
Topic 1: mHealth and Provider Behaviors | Scale mHealth to provide guidance to health care workers administering malaria treatments to children under-5 years of age to enhance compliance with malaria treatment guidelines. (IA) | Commence additional efficacy and effectiveness studies using mHealth to comply with treatment guidelines for other acute childhood and maternal diseases such as pneumonia, IMCI, and treatment of malaria in pregnancy. (C)Consider exploratory efficacy study on enhancing compliance with treatment guidelines on more chronic conditions such as managing severely undernourished children or pediatric tuberculosis. (C) |
Topic 2: mHealth Use for Adherence to Medications | Integrate mobile messaging into compliance strategies for HIV+ patients initiating antiretroviral therapy in conjunction with additional implementation science studies to optimize (e.g., two-way vs. one-way messaging) and scale mobile text messages to HIV+ patients taking antiretroviral therapy. (IIB) | Undertake studies to determine efficacy and effectiveness of mobile texting to mothers for provision of antiretroviral therapy to their HIV+ children using viral load as an endpoint. (C)Undertake studies to determine efficacy of mobile texts to populations using pre-exposure prophylaxis to prevent HIV acquisition where compliance is directly correlated with efficacy. This may be particularly effective in adolescent girls in Sub-Saharan Africa who have a much greater incidence of HIV acquisition. (C)Undertake studies to determine efficacy and effectiveness of mobile texting to tuberculosis infected mothers and adolescents and mothers of children under five with tuberculosis for compliance with tuberculosis therapy using microbiologic endpoints when possible. (C) |
Topic 3: mHealth Use for Demand Creation | Mobile texts to enhance attended skilled births should be integrated into antenatal care programs. (IIB) | Undertake effectiveness research to explore the combination of mobile texts for compliance with attendance at antenatal care visits and attended skilled birth. (B)Undertake effectiveness research on using mobile testing to enhance HIV testing and counseling leveraging the knowledge of the threshold effect and efficacy of motivational messaging over informational messaging. (C) |
Topic 4: mHealth Use for Patient Compliance with Health Care Appointments | Mobile texting should be integrated into antenatal care programs under the guidance of implementation science to increase compliance with antenatal visit schedules. (IIB)Mobile texting should be integrated into expanded program on immunization schedules for children and may be particularly helpful in mobile communities or refugee camp settings. (IIB) | Undertake implementation science for optimal scaling of mobile messages to women entering antenatal care to ensure demand creation, cost-effectiveness, integration into the health system, and adequate supply of quality maternal health services. (B)Undertake implementation science for optimal scaling of mobile messages to women for the expanded program on immunization schedules of their infants and children to ensure demand creation, cost-effectiveness, integration into the health system, and adequate supply and access to child immunization services. (B) |
Topic 5: Social/Transmedia for Healthy Cognitive Development and Reducing Stigmatization | Provide access to and encourage children under-5 to gain exposure to localized high-quality educational media programming such as Sesame Street. a (IA) | Pursue studies which evaluate the effectiveness of media programs such as Sesame Street on health outcomes in under-5 population (e.g., immunization rates, BMI, bed net use). (C)Pursue a longitudinal cohort study to evaluate the efficacy of Sesame Street, Soul Buddyz and other high-quality media programming to educate young children and their families on HIV and reduce stigmatization with an endpoint of decreased HIV incidence. b (C) |
Topic 6: Social/Transmedia and Other Media for Healthy BMI in Young Children | Integrate educational and playful activities related to healthy nutrition and active lifestyles into pre-school classrooms and family workshops. c (IIIC) | Conduct additional research to determine which components of the intervention (pre-school education and activities, parental workshops, and/or teacher training) results in positive effect on BMI. (C)Conduct rigorous implementation science to learn more about critical sustainability and implementation elements. (B) |
Note. Recommendations are based on strength of expert opinion—strongly (I), moderately (II), or weakly (III) recommended—and adequacy and quality of evidence—A, B, or C. BMI = body mass index. IMCI =Integrated Management of Childhood Illness. |
aEvidence suggests that this is particularly helpful to children in low socioeconomic settings and is scalable to remote settings as well.
bHIV incidence in young women is eight times that of their male counterparts (Abdool Karim & Dellar, 2014). Finding scalable sustainable interventions to change their behaviors is critical to reaching global goals of ending AIDS and creating an AIDS-free generation.
cStrong findings in the cluster randomized cross over study suggest that this approach to early childhood obesity should be pursued further. It is particularly interesting that absent change in habits, young children had a dramatic change toward healthy BMI in both the initial intervention group and the crossover group, which was sustained at 12 months.
Results
Overall, the systematic review provided valuable insight concerning the strengths and limitations of the current evidence in the use of mHealth's and social/transmedia technology's effect on maternal, neonatal, and child health survival and development. Our systematic review produced 26 total articles related to understanding the role of mHealth and social/transmedia in the promotion of maternal and child health. Many of the research studies reported in the mHealth section are randomized clinical trials (RCTs) that provide strong evidence for the efficacy and effectiveness of mHealth interventions while sustainability evidence is weaker. The evidence for social/transmedia technology on child and maternal health is limited by challenges in and demonstrating the counterfactual (see summary in Appendix E, in the supplemental online appendices). Although our social/transmedia search produced evidence describing the effect of social/transmedia on the social, emotional, and cognitive development of children in LMIC, there was a paucity of evidence demonstrating behavioral changes. Studies are described in enough detail to enable the reader to gain insight into design, intervention, target population, context, and rigor of results.
mHealth
All 15 mHealth studies used text message interventions, a service that is widely available, and can be cost-effective over large geographic areas. The studies fell into five intervention categories: (a) provider behaviors, (b) patient medication compliance, (c) maternal education and behaviors, (d) patient compliance with health care appointments, and (e) creating demand for health care services. A summary of critical aspects of the mHealth studies, such as target evidence, study design, study endpoints, and impact, can be found in Figure 1. More detailed results are subsequently provided.
Provider Behaviors
Two mHealth RCTs evaluated the efficacy of phone text message interventions to health workers (Chang et al., 2011; Zurovac et al., 2011). Chang and colleagues (2011) conducted an exploratory substudy within a larger cluster randomized study to evaluate the effect of public health care workers on compliance of HIV patients with their antiretroviral therapy as assessed by the patients' HIV virologic response over the 26-week intervention. Four public health care workers were responsible for 446 patients in the mHealth intervention group, and six were responsible for 524 patients in the control group. No significant differences were found in the primary outcome of virologic failure between the control group and the intervention group (19.4% vs. 16.4%), although the outcomes slightly favored the intervention condition.
The second study, a cluster RCT of 107 rural government health facilities in Kenya, assessed an mHealth intervention targeting 119 rural health care worker compliance with malaria treatment guidelines for severe malaria in children under 5 years of age from 2006 to 2009 (Zurovac et al., 2011). The behavioral endpoint was compliance by the health workers with malaria treatment guidelines for outpatient treatment of pediatric malaria. Health care workers in the intervention group were sent two daily text messages on their personal mobile phone about malaria case management for 6 months. The messaging aimed to reinforce correct counseling, prescription, dosing, and dispensing of antimalarial drugs. Case management practices were assessed for 1,157 children in the intervention group and 1,112 children in the control group. The study found that daily text messages, as compared with no daily text messages, significantly improved correct malaria case management by 24% (p = .004) both immediately after intervention and 6 months after intervention.
Jones and colleagues (2012) conducted a follow-up qualitative study to Zurovac and colleagues' (2011) cluster RCT to explore why health care workers receiving the instructional text messages on malaria treatment changed their behaviors and showed high acceptance for the intervention. The authors found the intervention was acting at the action and maintenance levels of the 5 Stages of Change model (Prochaska & DiClemente, 1991). Thus, receiving daily text messages created an enabling environment that prompted health care workers to take action by implementing correct malaria case management practices. Second, the authors found that the intervention had created demand by the participating health care worker for additional mHealth capabilities for managing severe malaria in children and in pregnant women.
Another follow-up study to the Zurovac and colleagues' (2012) mHealth RCT focused on assessing a sustainability element—the cost-effectiveness—of the mobile text messaging intervention. The cost per additional child correctly managed was US$0.50 under study conditions. The estimated cost would be US$0.36 if implemented by the Ministries of Health in the same area, and estimated at only US$0.03 per child if implemented nationally. The total cost of implementing the mobile text messaging intervention on a national scale was estimated to be US$98,000, which would represent only 1% of the US$10 million awarded by The Global Fund to the Kenyan Ministry of Health to strengthen malaria case management between 2011 and 2015. The authors concluded that the overall cost seemed to be a reasonable and affordable expenditure.
Antiretroviral Therapy Adherence
Our search yielded three mHealth RCTs that examined adherence to antiretroviral therapy for persons living with HIV/AIDS (Lester et al., 2010; Mbuagbaw et al., 2012; Pop-Eleches et al., 2011). Two demonstrated a positive effect, and one did not. The first study was an RCT of HIV-infected persons initiating antiretroviral therapy at three Kenyan clinics between 2006 and 2008 (Lester et al., 2010). Participants were randomized to either a reminder text message once a week that required them to confirm receipt of the message within 48 hours (two-way texting) or to a control (standard of care without messaging). At 12 months, the text message group, compared with the standard of care group, had significantly improved antiretroviral therapy adherence, relative risk for nonadherence (0.81, p = .006), and plasma HIV-1 viral RNA load suppression (<400 copies per ml of 57% vs. 48%, p = .04). The estimated number needed to treat was 11 patients for each additional patient with viral suppression in the text message group over the standard care group. From the number needed to treat information, the investigators concluded that, hypothetically, weekly text messaging applied to the 297,800 people initiating antiretroviral therapy in Kenya could result in 26,354 additional people with fully suppressed viral loads, which we now know would contribute to decreasing HIV incidence overall in the population, including mothers and children under 5 years of age.
The second study was an RCT of 431 HIV-infected individuals who initiated antiretroviral therapy in the past 3 months at a rural health clinic in Kenya (Mbuagbaw et al., 2012). Participants were randomly assigned to the control group or one of four intervention groups: either short or long daily reminder text messages or short or long weekly reminder text messages. The primary outcome was antiretroviral therapy adherence greater than 90% at 12 and 48 weeks. The secondary outcome assessed differences in treatment interruptions between the intervention groups and the control group. The investigators found that 53% of individuals receiving weekly reminder text messages (either short or long) reached the threshold of 90% antiretroviral therapy adherence compared with 40% of individuals in the control group (p = .03) during the 48-week follow-up period. In addition, only 81% of individuals in the intervention groups experienced treatment interruptions, while 90% of individuals in the control group experienced treatment interruptions during the 48-week follow-up period (p = .03). There were no statistically significant differences between the intervention groups who were administered daily text message reminders (either short or a long) and the control group on medication adherence or treatment interruption.
The third study was an mHealth RCT of 201 HIV-infected individuals who had been on antiretroviral therapy for at least 1 month at a hospital in Yaoundé, Cameroon (Pop-Eleches et al., 2011). Participants in the intervention group were sent short weekly text messages including a motivational note, a reminder to take their medication, and a number to call back in case the participant had questions. The control group was provided usual standard of care. The primary outcomes of the trial were to assess medication adherence using a visual analogue scale, number of doses missed (in the week preceding the interview), and number of antiretroviral therapy prescriptions refilled at the pharmacy. The investigators found no significant difference on adherence between the intervention group and the control group on the visual analogue scale.
Maternal Health
We identified two mHealth RCTs addressing maternal knowledge, emotional well-being, and pregnancy health outcomes (Jareethum et al., 2008; Sharma, Hebbal, Ankola, & Murugabupathy, 2011). One involved 68 healthy pregnant women who attended and delivered at a antenatal hospital in Siriraj, Thailand (Jareethum et al., 2008). Women in the intervention group were sent weekly text messages concerning potentially abnormal symptoms from 28 weeks of gestation until delivery. Endpoint assessments included maternal satisfaction, confidence, and anxiety during the antenatal and perinatal periods and pregnancy outcomes (i.e., gestational age at birth, infant birth weight, preterm labor). Mothers in the intervention group had significantly higher satisfaction than did mothers in the treatment as usual control group during the antenatal period (9.25 vs. 8.00, p < .001) and during labor (9.09 vs. 7.90, p = .007). Likewise, participants in the text message group had significantly higher confidence (8.91 vs. 7.79, p = .001) and less anxiety (2.78 vs. 4.93, p = .002). There were no differences in pregnancy outcomes, satisfaction, anxiety, or confidence during the postnatal period.
The second study was an mHealth study of mothers and their children focused on assessing the differences in text message and paper pamphlets on oral health in Karnataka, India (Sharma et al., 2011). Mothers and their children were randomly assigned to either control group (pamphlets) or the mHealth intervention group (text message). A total of 21 oral health messages were sent via text message or pamphlets (depending on group assignment) three times a day, every day for 7 days. This procedure was repeated for 4 weeks. The primary outcomes were oral health knowledge, oral health attitudes, and oral health practice of the mothers and plaque visibility of their children. There were significant improvements in oral health knowledge (p < .001), positive oral health attitudes (p < .001) oral health practice (p < .001), and visible plaque scores (for the children; p < .001) preintervention versus postintervention in both groups. There was a significant increase in scores of oral health knowledge (p < .001), and oral health practice in the text message group compared to the pamphlet group but surprisingly was no significant difference between groups on visible plaque scores.
Patient Compliance with Health Care Appointments
Six studies reported on interventions using text message reminders to improve health care appointment attendance (Chen, Fang, Chen, & Dai, 2008; da Costa, Salomão, Martha, Pisa, & Sigulem, 2010; Kaewkungwal et al., 2010; Leong et al., 2006; Liew et al., 2009; Prasad & Anand, 2012). Of these six studies, five were designed as RCTs (Chen et al., 2008; da Costa et al., 2010; Leong et al., 2006; Liew et al., 2009; Prasad & Anand, 2012). Although the types of appointments differed across studies (outpatient care, chronic disease care, oral health, antenatal care), all six studies demonstrated significant improvements in service uptake and attendance when text messages were used. Although Chen and colleagues (2008), Leong and colleagues (2006), and Liew and colleagues (2009) reported no significant difference between the phone call and text message arms, two of these studies concluded that text messaging was a more cost-effective strategy. One non-RCT study assessed before and after rates of compliance with antenatal care and expanded program on immunization schedules. This study took place along the Thai-Myanmar border in areas with high numbers of refugees and displaced persons (Kaewkungwal et al., 2010). The two components of the program included were from an existing Thailand Ministry of Health module. The intervention included (a) automated reminders via text message to mothers with children with antenatal care/expanded program on immunization visit due dates and (b) updated information regarding the antenatal care and child's immunization status via text message when performing antenatal care/expanded program on immunization activities. The total average number of antenatal visits significantly increased after the intervention, as did the percentage of women who came on time. On-time immunization increased (OR = 2.13, p < .05) pre- and postintervention for child vaccinations. Overall, the intervention increased antenatal care/expanded program on immunization coverage in the study area for both Thai and non-Thai mothers and children who were either permanent resident or migrants.
Creating Demand for Health Services
Our search yielded two results in the area of demand creation, one for HIV testing and counseling (de Tolly, Skinner, Nembaware, & Benjamin, 2010) and the second for skilled birth attendants (Lund et al., 2012). The first study, conducted in South Africa, specifically investigated the effectiveness of different text message content and frequency for HIV testing and counseling uptake (de Tolly et al., 2012). A total of 2,533 randomly selected participants were randomized into one of five groups: (a) control group (no text messages), (b) informational text message (3 times), (c) informational text message (10 times), (d) motivational text message (3 times), or (e) motivational text message (10 times). Text messages were sent every 3 days (over 9 days for participants in the 3 times group, and over 30 days for participants in the 10 times groups). The study found that the group receiving 10 motivational messages had the most impact on uptake of HIV testing and counseling (OR = 1.7, p = .0036). There was no difference between the control group and the 3 times group in HIV testing and counseling uptake, suggesting that there may be a threshold effect. Furthermore, neither the 3 nor 10 informational text message interventions messages significantly affected HIV testing and counseling compared with controls. The cost per test of using the 10 motivational text messages was US$2.41. These results show the potential of text messaging to influence the uptake of HIV testing and counseling and the importance of appropriate content and adequate dose (or threshold) to achieve this outcome.
The second study (Lund et al., 2012) was a cluster RCT among 2,550 women across 24 primary health care facilities in two urban and four rural districts in Zanzibar to examine the effect of text messaging on skilled delivery attendance. The intervention consisted of two components: (a) a one-way text message to the mothers and (b) a mobile voucher so that mothers could communicate directly with primary health care providers. The text message component was to provide simple health education and appointment reminders to encourage attendance at antenatal clinics, skilled delivery attendance, and postnatal care. The major outcome measure was skilled delivery attendance. The authors found that women in the intervention group from an urban setting were significantly more likely to have skilled delivery attendance than were women in the intervention group from a rural setting (OR = 5.73, p < .01).
Neonatal Health
Although our search did not yield any RCT evidence, we decided it was important to include information concerning burgeoning results from observational studies regarding the mHealth technology and maternal and neonatal health. Given that the field is still nascent, a limited but growing body of evidence exists to support the role of mobile technologies in improving neonatal health outcomes. mHealth interventions that affect neonatal survival are, at present, focused on (a) improving health system accountability and promoting equity of service delivery during the antenatal period, (b) compressing response times during emergency obstetric and neonatal crises, and (c) bolstering surveillance and facilitated referral during the first month of life. Upstream, preconception, and antenatal strategies focus on reducing maternal risk factors for improved neonatal outcomes primarily through education or medical intervention. Studies have shown that mobile phones have been effectively used to identify women exhibiting signs of obstetric complications and facilitate timely obstetric care, and improve facility-based delivery and skilled attendance at birth. The use of mobile phones to monitor and document birth weight within 7 days of delivery in rural Kenya has shown significant improvements in timely infant weight monitoring at the community level (Gisore et al., 2012). Seidenberg and colleagues (2012) reported that text messages may be a valuable tool to alert remote diagnostic laboratories to expedite checking of newborn HIV status. Text message technology is also being used successfully since 2011 for birth registration across public facilities in Nigeria (37 states, 774 local government authorities, and 3148 registration centers), and since 2009 to register, track, and identify infant malnutrition across Millennium Villages Program populations in Kenya (Kanter et al., 2009).
Social/Transmedia
Overall, our search produced 10 articles related to social/transmedia and child health yet most of the literature focuses on multiple-media platforms and not strictly transmedia or social media per se. Robust controlled efficacy studies showing the direct influence of multimedia platforms on healthy behaviors with known impact on child survival are limited. However, studies showing positive impact on changes in attitudes, awareness, and knowledge were more plentiful. The evidence we subsequently review outlines the burgeoning evidence in four categories: (a) healthy practices; (b) vaccinations, maternal health, and neonatal health; (c) malaria and HIV/AIDS; and (d) nutrition.
Healthy Practices
Briscoe and Aboud (2012) performed a systematic review of 24 interventions and programs that assessed the role of media in improving four central health practices—the use of bed nets, handwashing, face washing, and complementary feeding—related to child health low-income countries. The authors found that the interventions with the greatest effect used a multilevel approach, addressing behavior change on an individual and structural level. However, many of the studies included in the review neglected to specify behavior change techniques used in the comparison and control groups. Although the review found that behavior change interventions were most effective when participants were engaged on multiple levels—including behavioral (performance-based techniques); social (interpersonal and small media); sensory (mass media, materials); cognitive (problem solving)—a lack of information on the overall quality and effectiveness of the programs on the health outcomes made it difficult to extrapolate the extent to which one technique was more effective than another.
While primarily examining the impact of a single platform (television) and not the effect of social/transmedia per se, a recent meta-analysis of the impact of localized Sesame Street education programming is worth mentioning because it provides some of the most systematic and comprehensive evidence available on the value of a large-scale children's media initiative on health knowledge (and other outcomes). The study included data from 24 studies, 15 countries, and more than 10,000 children (Mares & Pan, 2013). Its aim was to determine the program's effectiveness across the three main categories: cognitive outcomes (e.g., letters, numbers), learning about the world (e.g., knowledge of culture, health/hygiene), and social reasoning and attitudes (e.g., positive attitude toward social outgroups). The authors found that children who watched Sesame Street were significantly more likely to have better social attitudes (p < .05) and a greater affinity for learning about the world including health practices (p < .05) compared with themselves at an earlier time point or compared with other children. Children who were exposed to Sesame Street in the low socioeconomic status sample were more likely to show greater improvement across all outcome measures (p < .05). The researchers found no statistically significant average effect on all outcomes for boys, but a significant, positive effect for girls (p < .05). Limitations notwithstanding, the authors concluded that the size of the Sesame Street intervention was comparable to that of other early childhood interventions, with one key difference: its media intervention reached more children than any other program in a cost-effective and scalable manner.
Vaccinations, Maternal Health, and Neonatal Health
Our search yielded five review and intervention studies examining the combined effect of social/transmedia on vaccination use, maternal health, and neonatal health (Hornik et al., 1991; Ir, Horemans, Souk, & Van Damme, 2010; Larson, Saha, & Nazrul, 2009; Majdzadeh et al., 2008; Perreira et al., 2002). Researchers conducting a study in southwestern Guatemala implemented three programs, including a clinic-based program that trained health care workers in prenatal counseling and provided educational media to clients, radio messages regarding obstetric complications, and educational sessions conducted through women's groups (Perreira et al., 2002). The investigators conducted three different surveys across three years (1997, 1998, and 1999) to assess whether there was a difference in the percentage of women who heard of danger signs during pregnancy before and after intervention. Women who were surveyed in 1999, postintervention implementation, were three to five times more likely to have heard about danger signs in pregnancy than were women surveyed in 1997, preintervention implementation.
The gray literature search produced an early evaluation from the USAID-sponsored program Health Communication for Child Survival (HEALTHCOM) that described the effect of a mass media campaign in the Philippines on improving measles vaccination coverage over a 5-year period (Hornik et al., 1991). The authors found that overall vaccination levels increased and were sustained following a mass media campaign using simple messaging about malaria through TV, radio, and print media channels. In addition, a study from Iran found that a mass media campaign with targeted TV commercials yielded statistically significant increases in community awareness of the measles-rubella vaccination, increasing from 81% at the beginning of the intervention to 100% by the end of the campaign (Majdzadeh et al., 2008). Although both of these studies showed a linkage between mass media and vaccination-related outcomes or awareness, neither included a discussion of study limitations, making it difficult to assess the full strength of the findings.
Larson and colleagues (2009) assessed the effect of a zinc awareness media campaign targeting caregivers for addition of zinc treatment to oral rehydration solution for diarrhea in Bangladesh. The intervention combined five major components, including the zinc formulation, research in support of zinc scale-up, product promotion among providers and caregivers, health care delivery systems, and knowledge transfer regarding the project. Seven repeated ecological surveys were conducted in four representative population areas—urban slum, urban non-slum, municipal, and rural—among 3,200 households of children with active or a recent case of diarrhea from September 2006 to November 2006. In all four of the areas sampled, the rates of zinc treatment and awareness increased with the introduction of the intervention (p < .001). Zinc treatment awareness increased to about 65% (from between 5 and 10% before intervention) in the urban slum and urban non-slum areas to 55% (from under 5% preintervention) in the municipals and rural areas by the 23-month follow-up visit. Nearly 2 years after the media campaign, 25% of urban non-slum areas, 20% of municipal and urban, and 10% of rural children under 5 years of age were receiving zinc as treatment for diarrhea. Boys were more likely to receive zinc as a treatment for diarrhea than were girls at the 18-month follow-up (21% versus 16%, p = .024).
HIV/AIDS and Malaria
Our search produced two articles related to multimedia interventions and HIV/AIDS and malaria prevention (Borzekowski & Macha, 2010; Health and Development Africa, 2005; Khulisa Management Services, 2005). Borzekowski and colleagues (2010) used a systematic, quasi-experimental evaluation to identify the positive impact of Kilimani Sesame in 223 children's cognitive skills, emotional and social attitudes and behaviors, and health and hygiene knowledge, attitudes, and behaviors, with a focus on malaria and HIV/AIDS in Tanzania. The study further examined the correlation between positive impact and children's recognition of program characters. Kilimani Sesame was a 6-week, multimedia intervention consisting of 13 radio shows (15 minutes each), 13 TV shows (30 minutes each), 3 storybooks, and a guide for caregivers (parents, teachers). Children in the high-exposure group engaged 6 hours every week with intervention materials, whereas children in the low-exposure group spent 2 hours every week with intervention materials. There were no differences in behavioral outcomes (i.e., HIV- and malaria-prevention behaviors); however, the lack of a significant finding might be influenced by limited access to prevention resources (e.g., bed nets).
A central component of the Takalani Sesame intervention was to reduce HIV/AIDS related stigma and discrimination in South Africa. A protagonist Muppet character, Kami, is HIV positive and serves as a model for communicating age-appropriate messaging about HIV/AIDS. An evaluation conducted by Khulisa Management Services (2005) found that children who were exposed to Takalani Sesame experienced measurable gains in HIV/AIDS knowledge and attitudes, including basic knowledge of the disease, blood safety, destigmatization, and coping with illness. Adult viewers of Takalani Sesame's companion television special Talk to Me were twice as likely to talk with their children about HIV as nonviewers (Health Health to Enhance Child Survival 177 and Development Africa, 2005). The influence of such iconic characters and their linkage to promoting behavior change is hinted at in the literature, but this is an intervention whose power should be further and more systematically examined.
Nutrition
A cluster RCT nutrition study targeting preschoolers, parents, and teachers conducted in Colombia involved 1,216 children 3 to 5 years of age, 928 parents, and 120 teachers of 14 preschools (Céspedes et al., 2012). The primary outcome was the change from baseline of the mean score (standardized on a scale from 1 to 100) of knowledge, attitudes, healthy habits, and living an active lifestyle and the children's body mass index at the end of the 5 months and 12 months. The children in the intervention groups (seven preschools) received educational and playful activities in a classroom setting related to good nutrition and healthy lifestyle, including daily Sesame Workshop Healthy Habits storybooks, posters, videos, games, and songs, weekly health notes and one workshop called Healthy Family Day. Parents also received weekly health notes and participated in three workshops focusing on healthy nutrition and active lifestyles. Teachers received three training sessions, personalized working sessions, and a teacher's guide. The children in the control group (seven preschools) engaged in their usual curriculum.
The body mass index change score was 10.9% for children in the intervention group and only 5.4% in the control group (p < .001). There was evidence of significant changes in attitudes (p < .001) but not in knowledge or habits. Similar results were found in parents attending the intervention group, scoring significantly higher in knowledge and attitudes but not in habits as compared with parents in the control group. The researchers could not identify any baseline difference in body mass index measures across intervention and control groups, and there was no association between children's baseline body mass index and their knowledge, attitudes, and habits at follow-up. The follow-up survey at 12 month showed maintained or improved results and equally strong results were obtained after the control group received the delayed intervention (absolute weighted difference in body mass index = 6.38, p < .001), providing evidence that interventions targeting young children can be effective and sustainable to reduce the risk of chronic disease later in life.
Discussion
Although availability of high-quality evidence of mHealth interventions (primarily text message–based interventions) is still limited, rigorous peer reviewed studies and meta-analyses included in this review have generated promising results regarding the positive impact of these interventions on child survival and development. The evidence toward social/transmedia interventions in the reduction of child mortality is burgeoning and markedly less advanced than are mHealth interventions. mHealth interventions have been effective in improving adherence to medication, uptake of service, education of caregivers, and clinical provider compliance with protocols. mHealth studies suggest that investments in mHealth can effectively improve child health by connecting caregivers to the health system, improving quality of services provided by health workers, and facilitating adherence to recommended treatments. Even though the evaluation of transmedia and social media interventions still face methodological challenges, particularly in assessing their direct effect on behavioral outcomes, these interventions have shown large-scale results in cognitive, social, and emotional development of children, as well as effective support to health education interventions in maternal and child health.
The evidence suggests that mHealth interventions can be effective and sustainable in achieving health care provider compliance with treatment and care protocols. Zuvorac and colleagues' studies (Zurovac et al., 2012; Zurovac et al., 2011) demonstrated effectiveness and sustainability of mHealth on changed provider behavior as measured by enhanced compliance with complex malaria treatment guidelines. Although there was an intervention effect with statistically significant increase in compliance with malaria treatment guidelines, only half the children were well managed overall. Thus, there is room for improvement. Although the study did not assess health outcomes, we can assume that compliance with appropriately updated treatment guidelines would confer mortality benefits. If this intervention was scaled, overall effect on under-5 malaria mortality should be assessed at the population level. This sequence of studies is admirable for the demonstration of efficacy, followed by qualitative work and cost effectiveness, which provide both the evidence needed to recommend scaling to improve the quality of malaria treatment for children under 5 and a model for building an evidence base for a novel technology intended for public health impact.
Although Chang and colleagues' (2011) study did not find an effect of mobile use by health workers on their patient's virologic response, the study had some significant limitations rendering it unlikely to do so. First, the parent study was unable to find a difference in virologic response between the patients cared for by public health care worker versus the no public health care worker group. Therefore, it was improbable that the mHealth substudy, with a smaller sample size, would find a difference in virologic response between mHealth versus no mHealth (Chang et al., 2011). Second, since virologic response is primarily a function of patient compliance with antiretroviral therapy it is not necessarily related to, nor tightly linked to, ease of public health care workers' communication; thus, given the study design, a positive effect on patients' viral loads was unlikely. It is a lesson in the importance of aligning the measured outcome directly to the intervention and should not undermine the strong findings in Zurovac's studies (Zurovac et al., 2012; Zurovac et al., 2011).
Remarkable progress in prevention of maternal to child transmission of HIV and demonstration that antiretroviral treatment prevents HIV infection in HIV-negative partners has, in conjunction with other HIV-prevention modalities such as circumcision, resulted in bold calls to “End AIDS” and create of an “AIDS-free generation” (Abrams & Myer, 2013; Davies, 2012; Fauci & Folkers, 2012; Montaner, 2013). Adherence to antiretroviral therapy correlates with HIV viral suppression, reduced resistance, and enhanced survival (Chesney, 2006); however, compliance with antiretroviral therapy must be in the range of 90–95% to reduce the risk of developing resistance and enable the drugs to work effectively. An overall reduction in HIV incidence and compliance with antiretroviral therapy, by both mothers and children, would dramatically increase the survival of children under 5 years of age (e.g., reduce mother to child transmission, increase the health of HIV-infected mothers who care for their children, and reduce mortality of children living with HIV) and reduce HIV incidence.
Adherence to antiretroviral therapy will help facilitate reductions of HIV incidence in high-burden countries enabling such countries to reach a tipping point where new persons on HIV therapy exceed those with new infections. Mbuagbaw and colleagues' (2012) results support the efficacy of once a week mobile text messages to enhance antiretroviral therapy compliance while daily messages extinguish the adherence behavior. Likewise, Lester and colleagues' (2010) study found that weekly two-way texting significantly enhances compliance with antiretroviral therapy as measured by a validated endpoint compared to controls. Furthermore, Lester's group offered promising sustainability estimates with the number needed to treat being 11 enabling the authors to project the effect of this intervention at scale. Additional research is need to optimize the intervention (e.g., Is two-way texting more effective than one-way adherence reminders?), although it appears that a once-a-week frequency is correct. The negative results of the third antiretroviral therapy adherence study could be from the smaller sample size or the less rigorous endpoint. Future studies applying mobile messaging to enhance HIV adherence should use the hard endpoint of HIV viral load, which is a sensitive reliable measure of HIV adherence.
Studies demonstrating the efficacy and effectiveness of mobile messaging on maternal behaviors such as greater attendance at antenatal appointments, enhanced attended skilled deliveries, and enhanced compliance with their children's vaccination schedules strongly support the need to precede to implementation science for mHealth in these areas to obtain sustainability evidence. There are interesting context-specific findings in the aforementioned research such as enhanced attended skilled deliveries in urban vs. rural centers. Although there may be myriad reasons for this, it is nonetheless encouraging that mobile messaging shows promise in changing the behaviors of women living in urban centers. By 2050, more than 70% of the world's population is projected to live in urban centers in LMICs (Cohen, 2003); thus, effective sustainable means of creating demand for essential health services will be critical to reaching and maintaining global health goals.
Using mHealth to create demand for service appears to hold promise. The evidence revealed important distinctions for the use of mobile texting for maintaining a behavior (e.g., compliance with long-term medication vs. initiation of a new behavior such as undertaking HIV testing and counseling). Whereas daily messages extinguish medication compliance behaviors, more messages of a motivational (verses informational) appear to be necessary to initiate the new behavior, perhaps with scary consequences, of HIV testing and counseling. The effective use of mobile texting for maternal use of attended skilled births also holds much promise and could easily be integrated into a mobile texting program aimed at enhancing compliance with antenatal care appointments.
Social and transmedia have often been cited as significant drivers of the social and political revolutions such as the “Arab Spring.” The potential scale of these platforms to influence behavioral and social norms is attractive, yet a paucity of literature was found on the utilization of social/transmedia interventions for behavioral outcomes leading to child survival. However, the results identified provide robust impact of programming such as Sesame Street on cognitive development, knowledge, and attitudes (including knowledge and attitudes around health care and practices). As the global community strives toward global health goals such as an “AIDS-free generation” and ending extreme poverty overcoming obstacles such as stigmatization and discrimination will be critical to achieving results for the lowest quartile and in high-risk groups such as men who have sex with men, changes in children's knowledge and attitudes may translate into critical health impact. While it would be unthinkable that progress on addressing the HIV/AIDS epidemic and other maternal child health issues would have been possible without the contribution of communication strategies, fiscal stewardship of limited global health resources requires rigorous evidence to guide global health investments. Therefore, additional research in this area is urgently needed especially given the broad coverage and scalability of programs such as Sesame Street, Soul Buddyz (Goldstein, Anderson, Usdin, & Japhet, 2001; Goldstein, Usdin, Scheepers, Anderson, & Japhet, 2003) and other high-quality media programming to educate young children and their families on HIV and reduce stigmatization.
Studies showing change in body mass index and attitudes but not knowledge or practices challenge the notion that knowledge and practices must precede behavioral changes and support the notion that behavioral changes and health impact should be measured as endpoints in studies of behavior change for child survival and development.
In the next section, we put forward a series of recommendations for research, policy, and practice. Recommendations for research focus on critical issues that research teams, agencies, and donors should pursue to further advance our knowledge and evidence. Recommendations for policy and practice draw on our review of existing literature with a view to contextualizing the implications of these findings in broader efforts for population-level behavior change for child survival and development. These research recommendations can guide governments, development agencies, civil society organizations, and other stakeholders involved in child survival and development efforts in generating new and robust evidence that can be integrated into programmatic work.
Topic-Specific Practice and Research Recommendations
Given fiscal constraints and the urgent need for evidence based global health practice to end preventable child deaths, the donor and research community must be cognizant of existing knowledge and evidence implementing efficient sustainable practices and building upon what is already known, rather than implementing nonsustainable or ineffective programs, repeating studies, or advancing research on a failed intervention or activity. Thus, the topic specific practice and research recommendations in Table 1 are based on the following: the experts' systematic review and assessment of the evidence during the evidence summit process; the premise that global public health decisions should be based on quality evidence demonstrating efficacy, effectiveness, and sustainability (see Figure 1); and the need for research to build on existing knowledge resulting in the evidence base necessary for allocation of global health investments into specific interventions and activities. The ERT 6 (STI ERT) experts suggest the specific practice recommendations in Table 1 be advanced to policy and integration into national and/or regional implementation plans. Specific research recommendations follow the practice recommendations and are based on existing knowledge (the level of which is indicated according to Figure 1). More general policy and research recommendations are made in the following section.
Policy Recommendations and General Recommendations on Approaches to Future Research
Improvements of maternal and child health outcomes ultimately depend on the integration of evidence and operationalization of policies into effective practice. mHealth, social/transmedia, and other media interventions in maternal and child health will likely continue to grow in number and scope over the next few years (Kaplan, 2006). Knowing that rigorous research must inform policy and practice decisions, additional recommendations from the ERT on future general approaches to future research are included as well. The following recommendations are not graded but are strongly advised by the experts to enhance the application and impact of practice recommendations and to guide future research.
-
1.
Government stewardship is needed to facilitate mHealth partnerships that leverage the strengths of relevant partners in order to best serve national health needs. mHealth deployment is an inherently cross-sector enterprise. Public-private partnerships may facilitate large-scale interventions bringing together diverse health, technology, and media stakeholders including regulators, development organizations, commercial enterprises, and beneficiaries. Mobile and internet content is unregulated in most countries, posing the risk of health content being disseminated that is inaccurate, misleading, or otherwise of poor quality. Government oversight and coordination with partners can ensure that mHealth content is vetted by appropriate expert bodies, consistent with national guidelines and standards for care, and responsive to the broader media and communication environment that can disseminate information that does not support healthy behaviors.
-
2.
Policies should be adopted to protect the privacy and dignity of citizens affected by mHealth information services. Governments can help prevent the misuse of confidential personal information, protect individual dignity, and reinforce trust in digital information platforms. Several studies examined in this report highlighted some of the security and ethical concerns associated with mobile phone use (Labrique, Kirk, Westergaard, & Merritt, 2013). For example, studies discuss how bringing mobile phones into people's homes and relationships can result in the unintended disclosure of one's HIV status (Mbuagbaw et al., 2012) or loss of privacy on shared phones (Chang et al., 2011).
-
3.
An integrated approach is needed to ensure greater contribution of mHealth, social/transmedia, and other media strategies to child survival priorities at multiple levels (e.g., individuals, community, and health systems). For example, policies to advance the promise of mHealth as a vehicle for population-level behavior change should build upon a framework that positions mHealth as a means by which to improve coverage, quality, or efficiency of proven interventions (Labrique, Vasudevan, Kochi, Fabricant, & Mehl, 2013). Also, health programs should adopt an overarching strategy that integrates mHealth, social/transmedia, and other media into community and health system interventions. In most LMICs, mHealth and social/transmedia interventions have evolved in an ad hoc fashion, driven by commercial interests, disease-specific programs, and uncoordinated donor and research projects. mHealth and social/transmedia policies can encourage network deployment where it is most needed, help ensure interoperable platforms, promote rational utilization within the health system, protect information security, and ensure scalable, sustainable approaches. Policies, including national communication policies, also can facilitate the systematic and mass dissemination of relevant health information and linkages with national health communication plans and strategies.
-
4.
Increase mobile phone ownership and access by women. Mobile phone ownership has been shown to enhance women's personal security, autonomy, mobility and self-efficacy, provide opportunities for income generation, business skills and improved standards of living, and strengthen social networks (Botha et al., 2010). Globally, however, there is a gender gap in phone ownership, with women 20% less likely to own a phone than men. As mobile subscriptions grow, barriers to mobile ownership and use by women may further contribute to their marginalization. mHealth policies should reflect a gender sensitive and gender equity perspective that both helps address this gap.
-
5.
Investments in mHealth and social/transmedia interventions should include a focus on the most vulnerable and marginalized populations.. A systems approach that looks at the multiple social determinants of maternal and child health disparities ought to be developed to ensure that mHealth and social/transmedia media programs reach the most marginalized, do not contribute to increased disparities, and enable conditions for the benefit of the poorest and most vulnerable segments of society. The pervasiveness of transmedia and social media and the increasing portability of devices that facilitate exposure to these platforms will provide numerous opportunities to reach broad segments of the population with increasingly targeted approaches. However, misalignment of technology ownership and availability and understanding where the magnitude of child mortality is greatest—among the poorest, marginalized segments of a population who often do not have phones—is also worth exploring.
-
6.
Research should include a focus on factors that affect adoption of interventions. Although this review was designed to inquire about the effects of mHealth interventions on child survival, there is clearly a need to understand factors that promote or hinder their adoption and use. Research questions could be categorized according to user-level factors (e.g., individual, societal, and structural) and adaptability characteristics (e.g., likelihood of mHealth being adopted in a given culture according to the technology's cost, its availability, the simplicity with which it can be used, and the applicability of the technology in accordance with existing cultural practices).
-
7.
Include the study of underlying mechanisms in research. Research should include a focus on underlying processes that explain how mHealth and social/transmedia affect meaningful validated intermediate variables that, in turn, affect outcomes such as child survival. This calls for validating and/or using surrogate endpoints that link to meaningful health outcomes. For example, mHealth technologies improve case management for health care workers, which, in turn, can have positive effects on survival (Zurovac et al., 2011). In addition, mHealth technologies also affect attendance in clinics, which, in turn, affect other health outcomes, as revealed by a number of studies (Chen et al., 2008; da Costa et al., 2010; Kaewkungwal et al., 2010; Leong et al., 2006; Liew et al., 2009; Lund et al., 2012; Prasad & Anand, 2012).
-
8.
Identify important contextual moderators in research. These questions focus on the conditions under which mHealth and social/transmedia technologies will have either greater or lesser impact. One group of moderators could include, for example, characteristics of behaviors that are being targeted for change. Some behaviors (e.g., early immunization) require only a few discrete episodes of action, whereas other behaviors (e.g., breastfeeding, handwashing, adherence to medications) require a more frequent adoption regimen; some behaviors are enacted by just a single person, whereas others require the joint support of one's family or community; and some behaviors are addictive, making them more difficult to change, whereas others are enacted with much greater volitional control. What is currently unknown is how these behavioral characteristics render the use of mHealth technologies more or less effective.
-
9.
Future mHealth research must use rigorous designs, validated outcome measures, and important implementation information in peer reviewed studies. One of the reasons why our review resulted in such a limited number of studies was that having a comparison group with clearly defined health outcomes was a criterion for inclusion (Balster et al., 2014). Although this is a much lower threshold than requiring studies to have adopted a randomized design, the paucity of studies that met even this lower bar highlights our limited ability to make rigorous assessments about the effects of mHealth technologies. Thus, future studies need to adopt more rigorous designs and endpoints and systematically build the evidence base needed to avoid wasting resources and enable comparisons across studies, modalities, and outcomes. Furthermore, there is a need for data analysis and reporting that systematically articulates basic implementation information, such as the type of media platform used, the content presented, media distribution and reach, consumer exposure, and other key specifics related to media interface. There is also a need for qualitative studies that provide a richer understanding about the nature of the cultural factors that shape the adoption and success of these new technologies. These qualitative studies should examine the ways in which behaviors and social norms interact and how social norms may shift in important ways due to the infusion of new technologies in the community, and differences in role definitions (for example, in notions of what it means to be a mother, a provider, or a patient) that are brought about because of new technologies.
-
10.
Future research should emphasize enhanced understanding of mechanisms of change. While this review primarily focused on evidence of the impact of mHealth and social/transmedia on child survival-related behaviors and outcomes, future research should attempt to understand the mechanisms of change that these technologies trigger in cognitive and decision making processes (Cole-Lewis & Kershaw, 2010; Pingree et al., 2010; Riley et al., 2010; Wakefield, Loken, & Hornik, 2010). To date, for example, most mHealth interventions are driven by a focus on transmission of information as the primary intervening variable without sufficiently exploring change mechanisms (Jones et al., 2012). Most, if not all, of the interventions that met the review criteria did not spell out a clear theory of change. This fact points to the need for more theory-driven mHealth interventions that explore informational, cognitive, normative, and behavioral dimensions of maternal and child health, an area in which there is already a rich body of knowledge in the health communication literature.
-
11.
Enhance capacity and develop methodologies to conduct mHealth, social/transmedia research. Although there is a significant contemporary trend toward the development and execution of social/transmedia (and multiplatform) approaches, including linkages to mHealth platforms, the limited evidence to date of research on the impact of social/transmedia makes it unfeasible to discern its overall effectiveness or potential for population-level behavior change to affect child survival, though the positive effect of media such as Sesame Street on cognitive development of young impoverished children at scale is encouraging. In the future there is a great need to build capacity and methodologies to study multiplatform approaches for behavior change leading to child survival in a rigorous and systematic manner. It is only after the creation of a solid body of research focused on social/transmedia that an assessment of its value can be adequately made. Owing to the very nature of this type of media, studying integrated platforms poses structural, funding, and other challenges that need to be considered and solved. Yet, due to the popularity and potential impact of social/transmedia, seemingly investing in building the capacity and methodologies to conduct such research is money well spent.
Conclusions
The purpose of this article was convey the results of the evidence summit review on effective and sustainable interventions that use technology platforms innovatively to promote and support behavior and social changes that are needed to accelerate reductions in under-5 mortality, and optimize health and protective child development up to age 5. There have been hundreds of peer-reviewed articles published that demonstrate the value of ethical and science-based interventions. Many of these have heralded work characterized by the value of what technology interventions in health can do.
Nonetheless, despite the rich evidence and academic inquiry advancing the field, it was a challenge to sift through and identify quality peer-reviewed evidence and gray literature that could help support future health practice, strategies, and programs. For example, our review showed that while social/transmedia and other media interventions show areas of impact, especially on knowledge, attitudes and cognitive gains, evidence is less strong on behavioral outcomes.
In parallel, it is incumbent upon the academic and development communities to advance the evidence base for STI interventions leading to sustained population behavior changes necessary for accelerated reductions of under-5 child mortality. Social sciences need the full support of research institutions and must become an integral part of multisector development challenges such as child survival. Furthermore, the social science community is encouraged to embrace scientific methods which enable assessments of causal attribution to both behavior and child survival outcomes. A common research agenda and an enabling environment for its implementation in the next few years is an urgent requirement to generate the most compelling evidence of STI interventions, and contribute more effectively to achieving global goals to reduce preventable deaths of children under-5 years of age and promote their healthy development.
Supplemental Material
Supplemental data for this article (Appendices A-E) can be accessed on the publisher's website at http://dx.doi.org/10.1080/10810730.2014.929763.
Acknowledgments and Funding
The authors are grateful to the following members of the Evidence Review Team 6 who were important contributors to the preparation of the original evidence review paper, which was presented at the Evidence Summit, and contributed to this article: Scott Ratzan, Wendy Meltzer, Anand Narasimhan, Christine Saba, David Milestone, Dennis Israelski, John Parrish-Sprowl, Kristian Olson, Leslie Snyder, Pamela Riley, Patricia Mechael, Raj Pannu, Brianna Ferrigno, Rajiv Rimal, Ryan Phelps, Sean Southey, and Shannon Dorsey. The authors also acknowledge the expertise and assistance of the U.S. National Institutes of Health Library and their Biomedical Librarians, Janet Heekin, MLS, and Alicia Livinski, MPH, MA, who provided additional systematic search of the literature to identify relevant papers for the social/transmedia portion of this review.
Dr. Elizabeth Higgs conducted part of this work while detailed to the U.S. Agency for International Development (USAID) Global Health Bureau while being salaried by the Division of Clinical Research, National Institute of Allergy and Infectious Diseases, Health and Human Services, and does not have any reported conflict of interest. Dr. Allison Goldberg participated in this evidence summit while completing her PhD at Columbia University Mailman School of Public Health, New York, and became an employee of Anheuser-Busch during the time this article was finalized. Stephanie Cook received financial support from USAID during the writing of this article. Dr. Rafael Obregón participated in the evidence review summit as a staff member of the United Nations Children's Fund and has no reported conflict of interest. Dr. Alain Labrique has no reported conflict of interest. Carina Schmid reports no conflict of interest. Dr. Charlotte Cole was employed by Sesame Street at the time of the evidence summit.
The views and opinions expressed in this paper are those of the authors and not necessarily the views and opinions of the U.S. Agency for International Development, the U.S. National Institutes of Health or the United Nations Children's Fund.
Footnotes
This article not subject to U. S. copyright law.
References
- Abdool Karim Q., & Dellar R. (2014). Inclusion of adolescent girls in HIV prevention research: An imperative for an AIDS-free generation Journal of the International AIDS Society, 17(1), 19075 10.7448/IAS.17.1.19075 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Abrams E. J., & Myer L. (2013). Can we achieve an AIDS-free generation? Perspectives on the global campaign to eliminate new pediatric HIV infections Journal of Acquired Immune Deficiciency Syndrome, 63(Suppl. 2), S208–S212 10.1097/QAI.0b013e3182986f55 [DOI] [PubMed] [Google Scholar]
- Balster R. L., Levy R., & Stammer E. (2014). Evidence aquisition and evaluation for evidence summit on population-level behavior change to enhance child survivial and development in low- and middle-income countries Journal of Health Communication, 19(Suppl 1), 10–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Black R. E., Morris S. S., & Bryce J. (2003). Where and why are 10 million children dying every year? The Lancet, 361, 2226–2234. [DOI] [PubMed] [Google Scholar]
- Borzekowski D. L. G., & Macha J. E. (2010). The role of Kilimani Sesame in the healthy development of Tanzanian preschool children Journal of Applied Developmental Psychology, 31, 298–305. [Google Scholar]
- Botha A., Makitla I., Tolmay J. P., Ford M., Seetharam D., Butgereit L., … Abouchabki C. (2010, May). Mobile4D platform. Paper presented at the IST-Africa 2010, Durban, South Africa. [Google Scholar]
- Briscoe C., & Aboud F. (2012). Behaviour change communication targeting four health behaviours in developing countries: A review of change techniques Social Science & Medicine, 75, 612–621. [DOI] [PubMed] [Google Scholar]
- Céspedes J., Briceño G., Farkouh M. E., Vedanthan R., Baxter J., Leal M., … Dennis R. (2012). Targeting preschool children to promote cardiovascular health: Cluster randomized trial American Journal of Medicine, 126, 27–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chang L. W., Kagaayi J., Arem H., Nakigozi G., Ssempijja V., Serwadda D., … Reynolds S. J. (2011). Impact of a mHealth intervention for peer health workers on AIDS care in rural Uganda: A mixed methods evaluation of a cluster-randomized trial AIDS and Behavior, 15, 1776–1784. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen Z. W., Fang L. Z., Chen L. Y., & Dai H. L. (2008). Comparison of an SMS text messaging and phone reminder to improve attendance at a health promotion center: A randomized controlled trial Journal of Zhejiang University Science B, 9, 34–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chesney M. A. (2006). The elusive gold standard. Future perspectives for HIV adherence assessment and intervention Journal of Acquired Immune Deficiency Syndrome, 43(Suppl. 1), S149–S155 10.1097/01.qai.0000243112.91293.26 [DOI] [PubMed] [Google Scholar]
- Chou W. Y. S., Hunt Y. M., Beckjord E. B., Moser R. P., & Hesse B. W. (2009). Social media use in the United States: Implications for health communication Journal of Medical Internet Research, 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen J. E. (2003). Human population: The next half century Science, 302, 1172–1175 10.1126/science.1088665 [DOI] [PubMed] [Google Scholar]
- da Costa T. M., Salomão P. L., Martha A. S., Pisa I. T., & Sigulem D. (2010). The impact of short message service text messages sent as appointment reminders to patients' cell phones at outpatient clinics in Sao Paulo, Brazil International Journal of Medical Informatics, 79, 65–70. [DOI] [PubMed] [Google Scholar]
- Davies E. (2012). Hilary Clinton looks to an AIDS free generation BMJ, 345, e8203 10.1136/bmj.e8203 [DOI] [PubMed] [Google Scholar]
- de Tolly K., Skinner D., Nembaware V., & Benjamin P. (2012). Investigation into the use of short message services to expand uptake of human immunodeficiency virus testing, and whether content and dosage have impact Telemedicine and e-Health, 18, 18–23. [DOI] [PubMed] [Google Scholar]
- Fauci A. S., & Folkers G. K. (2012). Toward an AIDS-free generation JAMA, 308, 343–344 10.1001/jama.2012.8142 [DOI] [PubMed] [Google Scholar]
- Free C., Phillips G., Galli L., Watson L., Felix L., Edwards P., … Haines A. (2013). The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review. PLoS Medicine, 10, e1001362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gisore P., Shipala E., Otieno K., Rono B., Marete I., Tenge C., … Esamai F. (2012). Community-based weighing of newborns and use of mobile phones by village elders in rural settings in Kenya: A decentralised approach to health care provision BMC Pregnancy Childbirth, 12, 15 10.1186/1471-2393-12-15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldstein S., Anderson A., Usdin S., & Japhet G. (2001). Soul Buddyz: A children's rights mass media campaign in South Africa Health and Human Rights, 5, 163–173 10.2307/4065370 [DOI] [Google Scholar]
- Goldstein S., Usdin S., Scheepers E., Anderson A., & Japhet G. (2003). The treatment of AIDS in Soul Buddyz: A multimedia campaign for children's health in South Africa In Singhal A. & Howard W. S. (Eds.), The children of Africa confront AIDS: From vulnerability to possibility (pp. 193–210): Athens: Ohio University Press. [Google Scholar]
- Hawn C. (2009). Take two aspirin and tweet me in the morning: How Twitter, Facebook, and other social media are reshaping health care Health Affairs, 28, 361–368. [DOI] [PubMed] [Google Scholar]
- Health and Development Africa (2005). Impact assessment of a new programming component on HIV = AIDS Johannesburg, South Africa: Author. [Google Scholar]
- Higgs E. S., Zlidar V. M., & Balster R. L. (2012). Evidence acquisition and evaluation for a U.S. Government Evidence Summit on Protecting Children Outside Family Care Child Abuse and Neglect, 36, 689–700 10.1016/j.chiabu.2012.09.002 [DOI] [PubMed] [Google Scholar]
- Hornik R., Zimicki S., Lee M. B., Hernandez J. R., de Guzman E. M., Abad M., … Bagasao T. M. (1991). The HEALTHCOM project in the Philippines: The National Urban Immunization Program 1989–1990 Philadelphia, PA: Annenberg School for Communication. Center for International Health and Development Communication, University of Pennsylvania. [Google Scholar]
- Ir P., Horemans D., Souk N., & Van Damme W. (2010). Using targeted vouchers and health equity funds to improve access to skilled birth attendants for poor women: A case study in three rural health districts in Cambodia BMC Pregnancy and Childbirth, 10, 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Istepanian R., Laxminarayan S., & Pattichis C. S. (2006). M-health: Emerging mobile health systems Berlin, Germany: Springer. [Google Scholar]
- Jareethum R., Titapant V., Tienthai C., Viboonchart S., Chuenwattana P., & Chatchainoppakhun J. (2008). Satisfaction of healthy pregnant women receiving short message service via mobile phone for prenatal support: A randomized controlled trial Medical journal of the Medical Association of Thailand, 91, 458–463. [PubMed] [Google Scholar]
- Jenkins H. (2006). Collaboration, participation and the media New Media & Society, 8, 691–698. [Google Scholar]
- Jones C. O. H., Wasunna B., Sudoi R. K., Githinji S., Snow R. W., & Zurovac D. (2012). “Even if you know everything you can forget”: Health worker perceptions of mobile phone text-messaging to improve malaria case-management in Kenya PLoS One, 7, e38636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaewkungwal J., Singhasivanon P., Khamsiriwatchara A., Sawang S., Meankaew P., & Wechsart A. (2010). Application of smart phone in “Better Border Healthcare Program”: A module for mother and child care BMC Medical Informatics and Decision Making, 10, 69–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kanter A. S., Negin J., Olayo B., Bukachi F., Johnson E., & Sachs S. E. (2009). Millennium global village-net: Bringing together Millennium Villages throughout sub-Saharan Africa International Journal of Medical Informatics, 78, 802–807. [DOI] [PubMed] [Google Scholar]
- Khulisa Management Services (2005). Impact assessment of Takalani Sesame season II programme Johannesburg, South Africa: Author. [Google Scholar]
- Labrique A. B., Kirk G. D., Westergaard R. P., & Merritt M. W. (2013). Ethical Issues in mHealth Research involving persons living with HIV/AIDS and substance abuse. AIDS Research and Treatment, 2013, 1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Labrique A. B., Vasudevan L., Chang L. W., & Mehl G. (2012). H_pe for mHealth: More “y” or “o” on the horizon? International Journal of Medical Informatics, 82, 467–469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Labrique A. B., Vasudevan L., Kochi E., Fabricant R., & Mehl G. (2013). mHealth innovations as health system strengthening tools: 12 common applications and a visual framework Global Health: Science and Practice, 1, 160–171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Larson C. P., Saha U. R., & Nazrul H. (2009). Impact monitoring of the national scale up of zinc treatment for childhood diarrhea in Bangladesh: Repeat ecologic surveys PLoS Medicine, 6, e1000175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leong K. C., Chen W. S., Leong K. W., Mastura I., Mimi O., Sheikh M. A., … Teng C. L. (2006). The use of text messaging to improve attendance in primary care: A randomized controlled trial Family Practice, 23, 699–705. [DOI] [PubMed] [Google Scholar]
- Lester R. T., Ritvo P., Mills E. J., Kariri A., Karanja S., Chung M. H., … Najafzadeh M. (2010). Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): A randomised trial The Lancet, 376, 1838–1845. [DOI] [PubMed] [Google Scholar]
- Liew S. M., Tong S. F., Lee V. K. M., Ng C. J., Leong K. C., & Teng C. L. (2009). Text messaging reminders to reduce non-attendance in chronic disease follow-up: A clinical trial The British Journal of General Practice, 59, 916–920. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lozano R., Wang H., Foreman K. J., Rajaratnam J. K., Naghavi M., Marcus J. R., … Atkinson C. (2011). Progress towards Millennium Development Goals 4 and 5 on maternal and child mortality: An updated systematic analysis The Lancet, 378, 1139–1165. [DOI] [PubMed] [Google Scholar]
- Lund S., Hemed M., Nielsen B. B., Said A., Said K., Makungu M. H., & Rasch V. (2012). Mobile phones as a health communication tool to improve skilled attendance at delivery in Zanzibar: A cluster‐randomised controlled trial BJOG: An International Journal of Obstetrics & Gynaecology, 119, 1256–1264. [DOI] [PubMed] [Google Scholar]
- Luoto J., Maglione M. A., Johnsen B., Chang C. E. S. H., Perry T., & Shekelle P. G. (2013). A comparison of frameworks evaluating evidence for global health interventions PLoS Medicine, 10(7), e1001469 10.1371/journal.pmed.1001469 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Majdzadeh R., Moradi A., Zeraati H., Sepanlou S. G., Zamani G., & Zonobi V. (2008). Evaluation of the measles-rubella mass vaccination campaign in the population covered by Tehran University of Medical Sciences East Mediterranean Health Journal, 14, 810–817. [PubMed] [Google Scholar]
- Mares M. L., & Pan Z. (2013). Effects of Sesame Street: A meta-analysis of children's learning in 15 countries Journal of Applied Developmental Psychology, 34, 140–151. [Google Scholar]
- Mbuagbaw L., Thabane L., Ongolo-Zogo P., Lester R. T., Mills E. J., Smieja M., … Kouanfack C. (2012). The Cameroon Mobile Phone SMS (CAMPS) trial: A randomized trial of text messaging versus usual care for adherence to antiretroviral therapy PLoS One, 7, e46909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Montaner J. S. (2013). Treatment as prevention: toward an AIDS-free generation Topics in Antiviral Medicine, 21, 110–114. [PMC free article] [PubMed] [Google Scholar]
- Naugle D., & Hornik R. C. (2014). Systematic review of the effectiveness of mass media interventions for child survival in low- and middle-income countries Journal of Health Communication, 19(Suppl 1), 190–215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pablos-Méndez A., & Fox E. (2012). Enhancing child survival and development in lower- and middle-income countries by achieving population-level behavior change Journal of Health Communication, 17, 1117–1118. [DOI] [PubMed] [Google Scholar]
- Perreira K. M., Bailey P. E., de Bocaletti E., Hurtado E., de Villagrán S. R., & Matute J. (2002). Increasing awareness of danger signs in pregnancy through community-and clinic-based education in Guatemala Maternal and Child Health Journal, 6, 19–28. [DOI] [PubMed] [Google Scholar]
- Pop-Eleches C., Thirumurthy H., Habyarimana J., Graff Zivin J., Goldstein M., de Walque D., … Ngare D. (2011). Mobile phone technologies improve adherence to antiretroviral treatment in resource-limited settings: A randomized controlled trial of text message reminders. AIDS, 25, 825–834. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Prasad S., & Anand R. (2012). Use of mobile telephone short message service as a reminder: The effect on patient attendance International Dental Journal, 62, 21–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Prochaska J. O., & DiClemente C. C. (1991). Stages of change in the modification of problem behaviors Progress in Behavior Modification, 28, 183–218. [PubMed] [Google Scholar]
- Rajaratnam J. K., Marcus J. R., Flaxman A. D., Wang H., Levin-Rector A., Dwyer L., … Murray C. J. L. (2010). Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970–2010: A systematic analysis of progress towards Millennium Development Goal 4 The Lancet, 375, 1988–2008. [DOI] [PubMed] [Google Scholar]
- Seidenberg P., Nicholson S., Schaefer M., Semrau K., Bweupe M., Masese N., … Thea D. M. (2012). Early infant diagnosis of HIV infection in Zambia through mobile phone texting of blood test results Bulletin of the World Health Organization, 90, 348–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sharma R., Hebbal M., Ankola A. V., & Murugabupathy V. (2011). Mobile-phone text messaging (SMS) for providing oral health education to mothers of preschool children in Belgaum City Journal of Telemedicine and Telecare, 17, 432–436. [DOI] [PubMed] [Google Scholar]
- Weinreich N. (2013). Transmedia storytelling for health: An integrated behavior change model. Paper presented at the 2013 National Conference on Health Communication, Marketing, and Media. [Google Scholar]
- You D., Bastian P., Wu J., & Wardlaw T. (2013). Level and trends in child mortality New York, NY: United Nations Children's Fund. [Google Scholar]
- Zurovac D., Larson B. A., Sudoi R. K., & Snow R. W. (2012). Costs and cost-effectiveness of a mobile phone text-message reminder programmes to improve health workers' adherence to malaria guidelines in Kenya PLoS One, 7(1), e52045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zurovac D., Sudoi R. K., Akhwale W. S., Ndiritu M., Hamer D. H., Rowe A. K., & Snow R. W. (2011). The effect of mobile phone text-message reminders on Kenyan health workers' adherence to malaria treatment guidelines: a cluster randomised trial The Lancet, 378, 795–803. [DOI] [PMC free article] [PubMed] [Google Scholar]