Short message service (SMS) reminders for childhood immunisation in low-income and middle-income countries: a systematic review and meta-analysis

Abstract

Introduction

Childhood vaccine delivery services in the low- and middle-income countries (LMIC) are struggling to reach every child with lifesaving vaccines. Short message service (SMS) reminders have demonstrated positive impact on a number of attrition-prone healthcare delivery services. We aimed to evaluate the effectiveness of SMS reminders in improving immunisation coverage and timeliness in LMICs.

Methods

PubMed, Embase, Scopus, Cochrane CENTRAL, CINAHL, CNKI, PsycINFO and Web of Science including grey literatures and Google Scholar were systematically searched for randomised controlled trials (RCTs) and non-RCTs that evaluated the effect of SMS reminders on childhood immunisation and timeliness in LMICs. Risk of bias was assessed using the Cochrane Risk of Bias 2.0 assessment tool for RCTs and Cochrane Risk of Bias in Non-randomised Studies of Interventions tool for non-RCTs. Meta-analysis was conducted using random-effects models to generate pooled estimates of risk ratio (RR).

Results

18 studies, 13 RCTs and 5 non-RCTs involving 32 712 infants (17 135 in intervention groups and 15 577 in control groups) from 11 LMICs met inclusion criteria. Pooled estimates showed that SMS reminders significantly improved childhood immunisation coverage (RR=1.16; 95% CI: 1.10 to 1.21; I²=90.4%). Meta-analysis of 12 included studies involving 25 257 infants showed that SMS reminders significantly improved timely receipt of childhood vaccines (RR=1.21; 95% CI: 1.12 to 1.30; I²=87.3%). Subgroup analysis showed that SMS reminders are significantly more effective in raising childhood immunisation coverage in lower middle-income and low-income countries than in upper middle-income countries (p<0.001) and sending more than two SMS reminders significantly improves timely receipt of childhood vaccines than one or two SMS reminders (p=0.040).

Conclusion

Current evidence from LMICs, although with significant heterogeneity, suggests that SMS reminders can contribute to achieving high and timely childhood immunisation coverage.

PROSPERO registration number

CRD42021225843.

Key questions.

What is already known?

• Childhood immunisation coverage in several low/middle-income countries (LMICs) remains short of the WHO’s Global Vaccine Action Plan 2020 goal of 90% childhood immunisation coverage.

• LMICs have witnessed exponential increase in mobile phone usage in the last decade and mobile phone short message service (SMS) reminders have also shown positive impact on attrition-prone healthcare delivery services.

What are the new findings?

• Meta-analysis shows that SMS reminders significantly improved childhood immunisation coverage and timeliness in the intervention groups compared with control groups.

• However, SMS reminders are significantly more effective in improving childhood vaccination coverage in low-income countries and lower middle-income countries than in upper middle-income countries, mainly due to ceiling effect.

• Sending more than two SMS reminders at least 24 hours before the appointment date ensures timely receipt of childhood vaccines than sending one or two SMS reminders.

What do the new findings imply?

• SMS reminders could be effective in improving overall vaccination coverage and timeliness in LMICs. This is especially significant in lower middle-income and low-income countries where the highest numbers of unvaccinated children live, and children are most likely to miss out on lifesaving vaccines.

Introduction

Despite significant improvements made in the global childhood immunisation coverage in the last two decades, 14 million children worldwide still missed out on lifesaving vaccines in 2019.¹ While global childhood immunisation coverage reached 85%, and 125 countries reached at least 90% childhood immunisation coverage in 2019,¹ childhood immunisation coverage in several World Bank-defined low/middle-income countries (LMICs) remained short of the WHO’s Global Vaccine Action Plan 2020 goal of 90% childhood immunisation coverage.^2–4 Sixty-three of the 69 WHO member countries that are yet to reach at least 90% DPT-3 (third dose of diphtheria, pertussis and tetanus) vaccine coverage in 2019 are LMICs.^{1 2} Countries in the World Bank-defined lower middle-income category and low-income category had DPT-3 coverage of 84% and 74%, respectively, in 2019.^{1 5} With this current trend, none of the LMICs will meet the sustainable development goals (SDG) 2030 childhood vaccination coverage targets for DPT and measles.⁶ In particular, countries in sub-Sahara Africa (SSA) account for approximately 25% of the annual global births, but SSA children contribute to 45%–50% of severe morbidity and fatalities worldwide from leading vaccine preventable diseases.⁷ One in five SSA children goes without lifesaving vaccines.⁸

SDG 2030 priorities include eliminating vaccine preventable diseases and improving access to new lifesaving vaccines.⁶ Achieving these goals requires continuously functioning childhood immunisation programmes,⁹ as shown in online supplemental file 1. Childhood immunisations are often delayed or missed either due to caregivers’ lack of awareness about the vaccines or their due dates.¹⁰ Many LMICs do not have functioning primary healthcare systems and routine well childcare services, making a somewhat complicated primary childhood vaccine series (with multiple appointments at various ages) difficult for caregivers to remember.^{8 9} Besides missing an opportunity to protect the child with lifesaving vaccines and the immense benefits thereafter,^{11 12} missing scheduled childhood immunisation appointments has financial and human resource implications leading to inefficiency in healthcare delivery.¹⁰ Immunisation reminders have been shown to improve compliance and timeliness, but studies suggest that traditional reminders have a low impact.^13–16 Reminder systems work through a variety of mechanisms including phone calls, letters, postcards and email meant to prompt the patient.¹⁷ Although most types of reminder systems are effective, mobile phone reminders have been found to be most effective.^{14 18}

Recent explosion of mobile phone usage transcends age, gender and state boundaries, and has significant potential for improving health.^17–19 Mobile phones have become integral parts of daily life. Wireless technologies cover over 95% of the global population.²⁰ LMICs have witnessed an exponential increase in the number of mobile phone users.²¹ Among the regions, with over 90% of the population covered by 2G networks at the end of 2017,²² and over 770 million mobile cell subscriptions in 2018,²³ SSA is the fastest growing mobile region in the world. Mobile phone penetration (defined as the number of SIM cards or mobile phone numbers per 100 people in a region) in SSA was 82% in 2018.²³ India and Pakistan, together, have over a billion mobile phone subscribers: over 870 million and over 130 million, respectively.^{23 24}

Short message service (SMS) reminders are also important utilities in a new system of healthcare delivery called mobile health (mHealth) which arguably is one of the key factors shaping the future of healthcare.^{25 26} A number of recent systematic reviews have demonstrated positive impact of SMS reminders on a variety of attrition-prone healthcare delivery, including antenatal care,²⁷ healthcare appointment,^{10 28} adherence to HIV medications²⁹ and adherence to chronic disease medications.³⁰ Although SMS reminders have also been shown to have a positive impact on vaccination in children, adolescents and adults in high-income countries,^31–34 trials in LMICs have not been comprehensively pooled and analysed.

Given the increasing ubiquitousness of mobile phones in LMICs, the demonstrated utility of SMS reminders in reducing attrition in other healthcare delivery services, the documented success of SMS reminders in improving vaccination coverage in high-income countries and the pressing need for interventions to boost childhood immunisation coverage and timeliness in LMICs, it is important to evaluate interventional studies on SMS reminders in LMICs. Therefore, this study aims to evaluate the effectiveness of SMS reminders in improving childhood immunisation coverage and timeliness in LMICs, thus generating evidence to support governments and development partners in strengthening existing vaccine delivery structures in LMICs.

Methods

The protocol for this systematic review was registered on PROSPERO database (CRD42021225843), and the review findings were reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.³⁵

Inclusion criteria

We included articles published in any language that were peer reviewed and that met the eligibility criteria based on the PICOS strategy (online supplemental file 2—inclusion criteria):

• Population (P): the study used mothers of children less than 24 months from LMICs, as classified by the World Bank in 2020,³⁶ as the population.

• Intervention (I): interventions in which SMS/text messages provide reminders related to vaccinations. Interventions were included even if the effectiveness of SMS reminders on childhood immunisation coverage or timeliness was not the primary intervention assessed or was assessed in conjunction with other interventions to improve demand for childhood immunisation (such as conditional cash transfers or other forms of reminder interventions).

• Comparison (C): the study compared the intervention with usual care in which mothers are reminded verbally at the health centre or the next appointment was written on the child’s health card.

• Outcome (O): the study evaluated the effectiveness of SMS reminders on DPT-3, Penta-3 or overall childhood immunisation uptake and/or timeliness. We accepted whichever method by which the outcome was assessed in the included intervention trials/studies, including by mothers self-report, home-based health records or facility-based childhood immunisation register.

• Study design (S): the study was either a randomised controlled trial (RCT) or a non-RCT interventional study.

Search strategy and study selection

We searched PubMed, Embase, Scopus, Cochrane CENTRAL, CINAHL, CNKI, PsycINFO and Web of Science for RCTs and non-RCTs of the effects of mobile phone SMS reminders on childhood immunisation published from 1 January 2000 to 30 November 2020 in 135 World Bank-defined LMICs,³⁶ but later updated the search to 31 December 2020. We used search terms covering short message services reminders (SMS, texts, text message), childhood immunisation and LMIC (online supplemental file 3—search strategy). We also searched trial registries (eg, Pan African Clinical Trial Registry, ClinicalTrials.gov and Chinese Clinical Trial Registry) for relevant registered trials; grey literature websites (eg, New York Academy of Medicine Grey Literature and Open Grey); prepublication server deposits (eg, medRxIV and bioRxIV) and Google Scholar. We also sought for relevant articles from the references of studies identified through the database search. There was no language restriction and non-English studies were translated into English using a translation service.

The search was independently conducted by two authors (PE and LOL) and duplicate articles from different database searches were excluded. The two authors first underwent a moderation exercise to ensure uniform application of inclusion criteria. Then, titles and abstracts were assessed applying the inclusion criteria. Discrepancies were resolved by discussion. Finally, full text of each remaining articles was assessed against the inclusion criteria.

Data extraction

Two authors (PE and LOL) extracted information from the included studies including the country in which the study was conducted, study design, intervention details, study participant characteristics and setting, sample size and outcome(s) relative to childhood immunisation coverage. Where studies reported both intention-to-treat and per-protocol analyses, we used the intention-to-treat data. Where studies reported data for both overall immunisation and DPT-3 data, we extracted data for overall immunisation. However, we also included studies that reported only DPT-3 outcomes, given the conventional use of DPT-3 to monitor progress of interventions aimed at improving vaccine delivery services.^{12 37} DPT-3 coverage—defined as the proportion of children receiving complete (three) doses of diphtheria, pertussis and tetanus—is a particularly valuable measure of the childhood immunisation coverage and countries’ vaccine delivery effectiveness.^{1 37}

Authors of eligible unpublished concluded trials identified from trial registries were contacted to provide results for inclusion in the study. Studies that employed cluster sampling were reduced to their effective sample size using the reported design effect and intracluster correlation coefficient before entry into statistical software.³⁸ Microsoft Excel was used to organise extracted data from included studies. Disagreements were resolved through discussion until there was 100% agreement.

Risk of bias assessment

Two authors (PE and LOL) independently used the Cochrane’s Risk of Bias 2.0 tool for assessing risk of bias³⁹ to assess included RCTs and their respective protocols and trial registry records for risk of bias in five domains: (1) bias arising from the randomisation process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in measurement of the outcome and (5) bias in selection of the reported result. If any of the five domains was found to be associated with some concerns of risk of bias or high risk of bias, the overall risk of bias was rated as ‘some concern’ or ‘high risk’, respectively. Otherwise, RCTs were rated as ‘low risk’. Disagreements were resolved by discussion.

Likewise, two authors (PE and LOL) independently assessed risk of bias in non-RCTs across seven domains using the Cochrane Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I) tool.⁴⁰ Overall risk of bias was rated as ‘low risk’, ‘moderate risk’ or ‘serious/critical risk’. Disagreements were also resolved by discussion. Risk of bias assessment for both randomised and non-randomised studies were then presented graphically using the Risk-Of-Bias VISualization tool.⁴¹

Measures of treatment effect and heterogeneity

Risk ratio (RR) was used as a measure of treatment effects and reported with 95% CIs. The primary outcome was change in childhood immunisation coverage while the secondary outcome was change in timelessness of childhood immunisation vaccines. Clinical heterogeneity (ie, variability in participants, interventions, outcomes studied) and methodological heterogeneity (ie, variability in study design and risk of bias) of included studies were characterised using descriptive statistics. Statistical heterogeneity (ie, variability in the intervention effects) was reported using the I² statistic.

Data synthesis

We performed data analysis according to the guidelines specified in the Cochrane Handbook for Systematic Reviews of Interventions.³⁸ Pairwise meta-analysis using the conventional random effects (DerSimonian-Laird) model was performed to pool data from individual trials and reported as pooled RR with 95% CI.^{38 42} Analyses were conducted using Stata V.16.1 (STATA Corp). Predictive interval was calculated using Prediction Intervals programme provided by BIOSTAT (Englewood, New Jersey, USA).⁴³ An α of 0.5 was used as the cut-off for statistical significance.

Sensitivity analysis was first performed using fixed-effect (Mantel-Haenszel) model which offers ‘the best estimate of the intervention effect’,³⁸ as it attributes more weight to more precise studies.⁴² Then, sensitivity analysis was also performed using the random effects restricted maximum likelihood (REML) model.⁴⁴ Robustness of pooled estimate was also assessed for the influence of studies with less than 200 participants in either arm—as studies with small sample size are more likely to exaggerate the intervention’s effect size.³⁸ Pooled estimate was also assessed for influence of studies with sample size outliers. Lastly, sensitivity of pooled estimate was assessed for influence of studies that reported only DPT-3 data.

Subgroup analyses were performed by the countries’ income status, study setting (urban vs rural), study design, number of SMS reminders sent per scheduled visit, timing of last SMS reminder and study quality. Meta-regression was performed to assess the modifying effect of country’s income status, study setting, study design, number of SMS reminders sent and time last SMS reminders was sent on the intervention effect. Finally, evidence of publication bias was assessed by examining the symmetry of the funnel plot and performing Harbord test.

Assessment of certainty of evidence

The certainty of evidence was assessed using Grading of Recommendations, Assessment, Development and Evaluation on the GRADEPro platform.⁴⁵ For each outcome, the quality of the body of evidence was assessed as high, moderate, low or very low; based on the following criteria: risk of bias, heterogeneity, imprecision, indirectness and publication bias.^{38 45}

Results

Selection of studies

The study selection process is illustrated in a PRISMA flow diagram (figure 1). The database search yielded a total of 5480 studies, and we included seven additional studies after manual searches in Google Scholar, grey literature, clinical trial registries and tracking references of selected primary articles. A total of 759 duplicates were removed and the titles and abstracts of the remaining 4728 studies were screened based on the exclusion criteria. Most studies (n=4689) were excluded. After eligibility and critical appraisal of the full texts of 39 records, 18 studies met the inclusion criteria for data extraction, qualitative synthesis and meta-analysis,^46–63 while 21 studies were excluded for the following reasons: intervention was not SMS reminder (n=9),^64–72 study setting was not an LMIC (n=2),^{73 74} conference abstract (n=2),^{75 76} literature reviews (n=2),^{77 78} could not extract data on specific effect of SMS reminder (n=2),^{79 80} observational case–control study (n=1),⁸¹ no comparator/control arm (n=1),⁸² control arm not usual care (n=1)⁸³ and outcome was not DPT-3, Penta-3 or overall immunisation (n=1).⁸⁴

Figure 1.

PRISMA flow diagram showing studies selection process. DPT-3, third dose of diphtheria, pertussis and tetanus; LMICs, low/middle-income countries; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; SMS, short message service.

Characteristics of included studies

Table 1 summarises the main characteristics of the 18 studies included in the analysis. Of the 18 studies, most (n=13) were RCTs,46–50 53 54 58–63 while the remaining (n=5) were non-RCTs.51 52 55–57 Most studies were peer-reviewed and published (n=17) and one was an unpublished doctoral thesis.⁵⁷ A total of 32 712 infants (17 135 in intervention groups and 15 577 in control groups) were included in this review. Studies were undertaken in 11 different countries, including Nigeria (n=5),49 52 57 61 62 Guatemala (n=3),^{47 59 60} Kenya (n=2),^{48 63} and one each in Bangladesh,⁵⁵ Burkina Faso,⁵³ Cote D’ivoire,⁵⁸ India,⁵⁴ Pakistan,⁵⁰ South Africa,⁵⁶ Vietnam⁵¹ and Zimbabwe.⁴⁶ Included studies reported 19 different interventions: 9 in urban settings, 7 in rural settings and 3 were in mixed settings. All 18 studies reported outcome data on childhood immunisation coverage, out of which 12 studies reported outcome data on childhood immunisation timeliness. Included studies reported outcome data based on either home-based health records/EPI Immunisation cards (n=7) or facility-based immunisation registers (n=11). The median sample size of included studies was 720 children and IQR is 1724.

Table 1.

Description of included studies and reported outcomes

Study, publication status	Study location and setting	Study population	Intervention description	Study methods	Study outcomes
Bangure et al, 2015, published46	Zimbabwe Urban	Mothers or caregivers who recently delivered or during third or seventh day visit in Kadoma City Clinic in Mashonaland West province. Children <7 days	One-way SMS reminders sent 7 days, 3 days and 1 day before immunisation appointment	Study design: RCT Sample size: 304 children (intervention: 152; control: 152) Length of follow-up: 3 months	Receipt of DPT-3 vaccines (coverage) and delay in immunisation (timeliness)
Ceballos et al, 2020, published47	Guatemala Rural	Households with children less than 2 years old and pregnant women, have access to a cellphone, and have at least one literate member in the municipalities of Santa Maria Nebaj and San Miguel Uspantan. 1162 households were randomised into two groups: 610 received SMS reminders (intervention) and 552 did not (control)	One-way SMS reminders sent to the intervention group about a week in advance to the date in which the child was due to receive vaccination	Study design: cluster RCT Sample size: 658 children (intervention: 340; control: 318) Length of follow-up: 6 months	Receipt of routine vaccines (coverage)
Coleman et al, 2020, published56	South Africa Urban	Mothers–child pairs receiving ANC and PNC/EPI care in six public healthcare facilities in the Mobile Alliance for Maternal Action (MAMA) intervention in inner city Johannesburg. Children <12 months	One-way maternal health SMS reminders sent twice weekly for each vaccination in the first year	Study design: non-RCT Sample size: 356 children (intervention: 181; control: 175) Length of follow-up: 12 months	Receipt of first-year infant vaccines (coverage)
Dipeolu et al, 2017, unpublished PhD thesis57	Nigeria Rural	Mothers–child pairs that delivered and mothers attending immunisation clinic at 10 primary healthcare facilities in Kajola and Ibarapa North LGA in Oyo State. Children <4 weeks.	One-way SMS reminders sent 2 days, 1 day before and on the day of immunisation appointment	Study design: non-RCT Sample size: 366 children (intervention: 179; control: 187) Length of follow-up: 6 months	Receipt of all infant vaccines (coverage)
Dissieka et al, 2019, published58	Cote d’Ivoire Mixed (includes rural, urban and peri-urban locations)	Mothers–child pairs recruited at time of BCG immunisation visit in 29 health facilities in Korhogo district, Children <5 weeks	One-way SMS reminders sent 2 days before immunisation appointment	Study design: RCT Sample size: 1596 children (intervention: 798; control: 798) Length of follow-up: 12 months	Receipt of all infant vaccines (coverage)
Domek et al, 2016, published59	Guatemala Urban	Parents–child pairs who owned a mobile phone and brought their children for vaccination at two public health clinics in Guatemala City clinics. Children aged between 8 and 14 weeks	One-way SMS reminders sent at 6, 4 and 2 days before the next scheduled immunisation appointment date	Study design: RCT Sample size: 321 children (intervention: 160; control: 161) Length of follow-up: 10 weeks	Receipt of DPT-3 vaccines (coverage)
Domek et al, 2019, published60	Guatemala Mixed (includes rural and urban locations)	Parents–child pairs who owned an active phone capable of receiving SMS who brought their children at two clinics in urban Guatemala City and two clinics in rural southwest region (Colomba & Coatepeque, and Quetzaltenango). Children aged between 6 weeks and 6 months	Automated one-way SMS reminders sent at 3 days, 2 days and 1 day before the next scheduled immunisation day	Study design: RCT Sample size: 720 children (intervention: 358; control: 362) Length of follow-up: 10 weeks	Receipt of DPT-3 vaccines (coverage) and timely receipt (timeliness)
Ekhaguere et al, 2019, published61	Nigeria Urban	Parturient mother–child pairs in Mother & Child Hospital Ondo and Akure, Ondo State. Newborn infant	One-way SMS reminders sent 2 days and 1 day before immunisation appointment	Study design: RCT Sample size: 600 children (intervention: 300; control: 300) Length of follow-up: 12 months	Receipt of all infant vaccines (coverage) and timely receipt (timeliness)
Eze et al, 2015, published62	Nigeria Urban	Caregiver–child pairs in eight health facilities in Egor LGA, Edo State Children due for first or second schedule of vaccines	One-way SMS reminders sent 1 day before immunisation appointment. Follow-up messages were sent in cases of missed appointments	Study design: RCT Sample size: 1001 children (intervention: 501; control: 500) Length of follow-up: 18 weeks	Receipt of infant vaccines (coverage) and timely receipt of vaccines (timeliness)
Gibson et al, 2017, published63	Kenya Rural	Mother–child pairs in 76 randomly assigned villages in Gem or Asembo districts. Children less than 35 days old	One-way SMS reminders sent 3 days and 1 day before immunisation appointment*	Study design: cluster-RCT Sample size: 748 children (intervention: 388; control: 360) Length of follow-up: 12 months	Receipt of infant vaccines (coverage) and timely receipt of vaccines (timeliness)
Haji et al, 2016, published48	Kenya Mixed (includes rural and urban locations)	Mother–child pairs in Langata, Machakos and Njoro districts. Children <6 weeks	One-way SMS reminders sent 2 days before and on the day of scheduled immunisation day*	Study design: RCT Sample size: 744 children (intervention: 372; control: 372) Length of follow-up: 14 weeks	Receipt of Penta-3 vaccines (coverage) and timely receipt (timeliness)
Kawakatsu et al, 2020, published49	Nigeria Urban	Caregiver–child pairs attending immunisation clinic at 33 primary healthcare centres (PHCs) across 20 LGAs in Lagos State between 25 March and 27 June 2019. Children <6 weeks	One-way SMS reminders sent 2 days before the scheduled immunisation day	Study design: RCT Sample size: 8337 children (intervention: 4893; control: 3444) Length of follow-up: 3 months	Receipt of all infant vaccines (coverage) and timely receipt (timeliness)
Kazi et al, 2018, published50	Pakistan Urban	Parents–child pairs who owned an active phone in Ibrahim Haidry (IH) union council in Karachi. Children less than 2 weeks of age	Four one-way SMS reminders sent within the week of the scheduled routine immunisation day	Study design: RCT Sample size: 300 children (intervention: 150; control: 150) Length of follow-up: 4 months	Receipt of DPT-3 vaccines (coverage)
Nguyen et al, 2017, published51	Vietnam Rural	Two cohorts include all children born in September and October 2013 (control) and in September and October 2014 (intervention) in Ben Tre province	Multiple one-way SMS reminders sent before the scheduled routine immunisation day	Study design: non-RCT (pre-post intervention design) Sample size: 8075 children (intervention: 4078; control: 3997) Length of follow-up: 12 months	Receipt of infant vaccines (coverage)
Oladepo et al, 2020, published52	Nigeria Rural	Mother–child pairs attending immunisation clinics in Primary Health Centres in 14 LGAs across six states and the Federal Capital Territory (FCT), Abuja. Children ≤2 months	Multiple one-way SMS reminders sent three times a week before the next immunisation appointment	Study design: non-RCT Sample size 3500 children (intervention: 1750; control: 1750) Length of follow-up: 10 months	Receipt of all infant vaccines (coverage) and timely receipt (timeliness)
Schlumberger et al, 2015, published53	Burkina Faso Urban	Mother–child pairs attending Centre de Santé et de Promotion Social (CSPS) in Colma 1 (medical district of Do). Do is one of the urban regions in Bobo-Dioulasso, Burkina Faso. Children <1 month	One-way SMS reminder sent before next due EPI vaccination sessions	Study design: RCT Sample size: 523 children (intervention: 255; control: 268) Length of follow-up: 5 months	Receipt of DPT-3 vaccines (coverage) and timely receipt (timeliness)
Seth et al, 2018, published54	India Rural	Pregnant mothers and children less 24 months in rural community in Mewat region in Haryana State from 10 July 2016 and 20 July 2017 were prospectively enrolled.	One-way SMS reminders sent before day of scheduled immunisation day*	Study design: RCT Sample size: 405 children (intervention: 201; control: 204) Length of follow-up: 13 months	Receipt of all infant vaccines (coverage) and timely receipt (timeliness)
Uddin et al, 2016, published55	Bangladesh Rural (A) and urban (B)†	Pregnant women and children aged less than 11 months in two separate areas. Rural: two upazilas (sub-districts) in Sunamgonj district. Urban: two zones in Dhaka City with the most street dwellers	Three one-way SMS reminders sent 1 day before scheduled EPI immunisation day, at the opening time on the day of the scheduled EPI immunisation, and 2 hours before closing time on the day of the scheduled EPI immunisation day	Study design: non-RCT (pre-post intervention design) Rural: Sample size: 2080 children (intervention: 1040; control: 1040) Urban: Sample size: 2078 children (intervention: 1038; control: 1040) Length of follow-up: 12 months	Receipt of infant vaccines (coverage)

*Multi-arm trial includes other interventions not relevant to this study.

†Study reported separate quasi-experiment data for urban and rural settings.

ANC, Antenatal care; DPT-3, third dose of diphtheria, pertussis and tetanus; EPI, Expanded Programme on Immunization; LMICs, low/middle-income countries; PNC, Postnatal care; RCT, randomised controlled trial; SMS, short message service.

Risk of bias assessment

Of the 13 RCTs, about half (n=6, 46%) were rated as having a low risk of bias, 5 were rated as having some concerns and 2 were rated as having high risk of bias (online supplemental file 4A). Most of the included RCTs (n=10; 77%) were rated as having low risk of bias arising from the randomisation process whereas the remaining RCTs were rated as having some concerns for this domain. Most RCTs (n=12; 92%) had a low risk of bias in the selection of the reported result. Based on weighted risk using trials’ sample size, 30% of the included RCTs were rated as having a low risk of bias, about 60% as having some concerns and about 10% high risk of bias (online supplemental file 4B).

Of the five non-RCTs, most (n=4; 80%) were rated as having serious or critical risk of bias whereas only one (20%) was rated with a low risk of bias in all domains (online supplemental file 5A). Of note, all included non-RCTs were rated as low risk of bias for classification of participants, deviation from intended interventions, measurement of outcomes and selection of reported results. Based on weighted risk using trials’ sample size, 15% of the included non-RCTs were rated as having a low risk of bias, 82% as having serious risk of bias and about 3% critical risk of bias (online supplemental file 5B). In general, the main causes of serious overall bias risk according to ROBINS-I assessment for non-RCTs were weaknesses in the confounding bias, selection of participants and missing data bias domains.

Childhood immunisation coverage

Twelve studies46 48–53 55 57 58 61 62 showed that SMS reminders significantly improved childhood immunisation coverage in children in the intervention group compared with those in the control group with usual care. Three studies showed that childhood immunisation coverage in the control group was relatively high (compared with intervention group).^{47 56 59} Two other studies evaluated the utility of SMS reminders in one arm of a three-arm study including compliance-linked monetary incentives in other arms.^{54 63} Both studies reported insignificant effectiveness for SMS reminders alone but statistically significant effect when coupled with monetary incentives.^{54 63} Finally, in another study, countrywide vaccine shortages precluded the evaluation of SMS reminders on overall childhood immunisation coverage.⁶⁰ However, SMS reminders demonstrated statistically significant improvement in childhood immunisation timeliness.⁶⁰

Meta-analysis of data from included interventions (n=19, sample size=32 712 participants) showed that SMS reminders significantly improved childhood immunisation coverage; RR=1.16; 95% CI: 1.10 to 1.21; I²=90.4%—figure 2A. However, the predictive interval for this effect overlaps the null (0.96, 1.41), indicating some uncertainty about the distribution of effects in comparable populations. Pooled estimates using the fixed-effect model (RR=1.13; 95% CI: 1.12 to 1.15; I²=90.9%) and the random-effects REML model (RR=1.17; 95% CI: 1.09 to 1.25; I²=95.6%) were similar. Meta-analysis conducted by excluding the studies with fewer than 200 participants in either arm also produced similar results: RR=1.16; 95% CI: 1.10 to 1.23; I²=92.6%. Also, pooled estimates of studies excluding two studies with outlying sample size^{49 51} produced similar results: RR=1.17; 95% CI: 1.10 to 1.25; I²=90.9%. Lastly, pooled estimates excluding studies that reported only DPT-3 data showed similar results: RR=1.17; 95% CI: 1.10 to 1.23; I²=93.4%.

Figure 2.

Forest plot of pooled estimates on childhood immunisation coverage (A) and timeliness (B). Uddin et al, 2020 reported two separate interventions: Uddin, 2020 (A) for rural and Uddin, 2020 (B) for urban settings.

In subgroup analysis, we found substantial differences in intervention effect size by country’s income status and the study’s quality and a marginal difference by study design. We found no difference in effects by study setting, outcome measure, number of SMS reminders sent or the timing of the SMS reminder—table 2. While SMS reminders were marginally effective in upper middle-income countries, they were significantly more effective in lower middle-income and low-income countries (p<0.001). Meta-regression analysis shows that only countries’ income status was a statistically significant intervention effect modifier (online supplemental file 6). A change from upper middle-income status to lower middle-income status corresponds to an increase of intervention effect of 18.1% (p=0.002) while a change from upper middle-income status to low-income status corresponds to 27.1% increase of intervention effect (p=0.006). Funnel plot presents graphical diagnostics of small study effects based on subjective visual inspection. Symmetric location of individual plots in the funnel plots indicates absence of publication bias. Graphical assessment of the funnel plot suggests absence of publication bias (online supplemental file 7). Objective assessment of publication bias using the Harbord test also indicated there is no evidence of publication bias (p value=0.2088) (online supplemental file 8).

Table 2.

Subgroup analysis of SMS reminders effectiveness on improving childhood immunisation coverage

Subgroup	No of studies*	Sample	Pooled RR	95% CI	I2 statistic (%)	P value†
All studies	19	32 712	1.16	1.10 to 1.21	90.4
Country’s income status						<0.001
Low-income country	1	523	1.39	1.19 to 1.62	–
Lower middle-income country	14	30 134	1.19	1.13 to 1.26	89.9
Upper middle-income country	4	2055	1.01	0.96 to 1.06	58.4
Study design						0.060
Randomized Controlled trials	13	16 257	1.13	1.06 to 1.20	89.6
Non-randomised controlled trials	6	16 455	1.22	1.12 to 1.32	92.2
Study setting						0.625
Rural settings	7	15 832	1.11	1.03 to 1.20	92.0
Mixed settings	3	3060	1.22	0.99 to 1.51	96.7
Urban settings	9	13 820	1.17	1.09 to 1.26	83.3
Outcome measured						0.270
DPT-3 coverage	8	4513	1.14	1.06 to 1.22	80.4
Overall immunisation coverage	11	28 199	1.17	1.10 to 1.25	93.4
Number of SMS reminders sent						0.441
1 or 2 SMS reminders	9	10 683	1.15	1.08 to 1.22	89.2
>2 SMS reminders	10	22 029	1.17	1.07 to 1.27	92.0
Timing of last SMS reminder						0.124
Sent on scheduled immunisation day	4	5307	1.27	1.08 to 1.49	91.9
Sent 1 or 2 days before scheduled day	15	27 405	1.13	1.07 to 1.19	89.9
Risk of bias (quality) of included studies						<0.001
Low risk of bias	7	7625	1.38	1.22 to 1.55	86.7
Moderate and high risk of bias	12	25 087	1.07	1.03 to 1.12	84.6

*Uddin et al, 2020 reported two separate interventions: Uddin, 2020 (A) for rural setting and Uddin 2020 (B) for urban setting.

†χ² test for subgroup difference.

DPT-3, third dose of diphtheria, pertussis and tetanus; SMS, short message service.

Childhood immunisation timeliness

Of the 12 included studies that evaluated the effect of SMS reminders on childhood immunisation timeliness, 10 studies demonstrated that SMS reminders significantly improve timely receipt of vaccines in children in the intervention group compared with those in the control group with usual care. Meta-analysis of data from included interventions (n=12, sample size=25 257 participants) showed that SMS reminders significantly improved timely receipt of childhood vaccines; RR=1.21; 95% CI: 1.12 to 1.30; I²=87.3%—figure 2B. However, the predictive interval for this effect overlaps the null (0.93, 1.56), indicating some uncertainty about the distribution of effects in similar populations.

Although pooled meta-estimates using the fixed-effect model showed similar statistically significant results (RR=1.22; 95% CI: 1.19 to 1.25; I²=87.8%), the pooled estimates obtained using the random-effects REML model were not statistically significant (RR=1.33; 95% CI: 0.98 to 1.81; I²=99.4%). Meta-analysis conducted by excluding the studies with fewer than 200 participants in either arm also produced similar results: RR=1.18; 95% CI: 1.13 to 1.23; I²=65.2%. Also, pooled estimates of studies excluding two studies with outlying sample size^{49 51} equally produced similar results; RR=1.23; 95% CI: 1.10 to 1.38; I²=89.3%. Lastly, pooled estimates excluding studies that reported only DPT-3 data showed similar results; RR=1.20; 95% CI: 1.12 to 1.28; I²=72.9%.

Subgroups analysis showed minor variation in the effectiveness of SMS reminders in improving timely receipt of childhood vaccines across different country income status, settings, study designs, outcome measure, timeliness cut-offs and study quality (table 3). However, the timely receipt of childhood vaccines was significantly improved when more than two SMS reminders were sent for childhood immunisation appointment versus when one or two SMS reminders were sent, and this difference was statistically significant (p=0.040). Also, SMS reminders significantly improved timeliness for scheduled immunisation day versus for later appointment days (p=0.024). Meta-regression analysis shows that only the number of SMS reminders sent was a statistically significant modifier of the intervention effect—online supplemental file 9. Increasing the number of SMS reminders to more than two SMS reminders improves the intervention effect by 24.0% (p=0.008). Funnel plot presents graphical diagnostics of small study effects based on subjective visual inspection. Symmetric location of individual plots in the funnel plots indicates absence of publication bias. Graphical assessment of the funnel plot suggests absence of publication bias (online supplemental file 9). Objective assessment of publication bias using the Harbord test also indicated that there is no evidence of publication bias (p value=0.7125) (online supplemental file 9).

Table 3.

Subgroup analysis of SMS reminders effectiveness on improving childhood immunisation timeliness

Subgroup	No of studies	Sample	Pooled RR	95% CI	I2 statistic (%)	P value*
All studies	12	25 257	1.21	1.12 to 1.30	87.3
Country’s income status						0.554
Low-income country	1	523	1.22	1.01 to 1.47	–
Lower middle-income country	10	24 014	1.20	1.10 to 1.30	89.5
Upper middle-income	1	720	1.38	1.08 to 1.75	–
Study design						0.560
Randomised controlled trials	10	13 682	1.21	1.09 to 1.34	87.8
Non-randomised controlled trials	2	11 575	1.25	1.19 to 1.32	52.4
Study setting						0.604
Rural	4	12 728	1.20	1.09 to 1.31	74.9
Peri-urban (mixed urban and rural)	2	1464	1.21	1.05 to 1.40	45.7
Urban	6	11 065	1.32	1.11 to 1.58	92.3
Outcome measured						0.259
DPT-3 coverage	7	4192	1.32	1.13 to 1.54	91.1
Overall immunisation coverage	5	21 065	1.20	1.12 to 1.28	72.9
Time cut-off for timeliness						0.024
Scheduled vaccination day	6	13 684	1.44	1.19 to 1.74	92.5
1–28 days after scheduled day	6	11 573	1.14	1.08 to 1.22	70.5
Number of SMS reminders sent						0.040
1 or 2 SMS reminders	8	20 433	1.16	1.11 to 1.21	59.3
>2 SMS reminders	4	4824	1.91	1.18 to 3.07	94.9
Timing of last SMS reminder
Sent 1 or 2 days before scheduled day	10	24 108	1.25	1.14 to 1.36	85.1	0.228
Sent on scheduled immunisation day	2	1149	0.98	0.68 to 1.43	88.3
Risk of bias (quality) of included studies						0.071
Low risk of bias	4	1871	1.76	1.14 to 2.70	95.2
Moderate and high risk of bias	8	23 386	1.18	1.11 to 1.25	71.3

*χ² test for subgroup difference.

DPT-3, third dose of diphtheria, pertussis and tetanus; SMS, short message service.

Body of evidence

In this review, the body of evidence for the effectiveness of SMS reminders for improving childhood immunisation coverage was graded as moderate (online supplemental file 12). The risk of bias was serious and there was high heterogeneity among studies which downgraded the quality of the body. The body for evidence for the effectiveness of SMS reminders for improving childhood immunisation timeliness was graded as high considering several factors including strong association and dose response gradient (online supplemental file 12).

Discussion

Vaccination is a critical priority for LMICs. Existing systems for delivery of childhood vaccines in several LMICs have struggled to reach the critical target required to forestall outbreaks of vaccine preventable diseases.^{37 85} Efforts to raise global childhood immunisation levels require a strong focus on the countries where the highest numbers of unvaccinated children live while not neglecting countries where children are most likely to miss out on childhood immunisation.⁸⁶ This meta-analysis of 18 articles presents the best available evidence on the effectiveness of SMS reminders on improving childhood immunisation coverage and timeliness in children less than 2 years in LMICs.

Our pooled estimate showed that SMS reminders have the potential to improve childhood immunisation coverage in children. Even though the included studies showed substantial heterogeneity, the magnitude of the effects are large and uniformly in the positive direction. We did not find any change in level of heterogeneity when we assessed the intervention effect by the number of SMS reminders sent, the timing of the last SMS reminder, the study setting and the study design. However, subanalysis of intervention effect by countries’ income status showed decreased heterogeneity. The precise effect of SMS reminders on vaccination rates is likely to be influenced by income status of the country. We found relatively lower effectiveness of SMS reminders in upper middle-income countries although these countries have better childhood immunisation infrastructure than lower middle-income and low-income countries. Upper middle-income countries are likely to have high vaccination rates and thus limited potential for SMS reminders to improve the childhood immunisation rates further (ceiling effect).47 56 59 60 63 However, our finding contradicts a small number of studies that have shown SMS reminders to be effective in high income countries.^{13 34} Our study findings are generally comparable with findings from similar studies,^{13 87–89} though Yunusa et al suggests that both phone call reminders and SMS combined with voice message reminders are more effective than SMS reminders alone. However, compared with recent studies, our study specifically focused on SMS reminders,^{87 89} exclusively focused on LMICs,^{13 88} is a comprehensive update on SMS reminders,^{13 88} and employed meta-analysis to obtained an estimate of overall pooled effect.⁸⁸

We also found that SMS reminders can improve childhood immunisation timeliness, although included studies showed substantial heterogeneity. Observed heterogeneity could be due to the methodological and clinical differences between the studies, although the magnitude and direction of the intervention effects mitigates this concern.³⁸ Substantial heterogeneity remained when we assessed the effect by country income status, study design, study setting, number of SMS reminders sent and timing of last SMS reminder, but it dropped when we limited the analysis to non-RCTs and in participants that received one or two SMS reminders. The precise effect of these interventions is likely to be greatly influenced by the number of SMS reminders sent. Sending more than two SMS reminders and sending the last SMS reminders at least 24–48 hours before the scheduled immunisation day was found to be most effective. These findings suggests that habituation—the propensity to ignore messages because of high frequency—is not a problem and that a reminder sent at least 24 hours before the scheduled immunisation day allows sufficient time for the mother/caregiver to prepare for the appointment. The effectiveness of SMS reminders in improving timely childhood vaccination was consistent across income status indicating the value for this intervention across multiple settings. Similar to our study findings for childhood immunisation coverage, our study findings are highly comparable to similar studies that specifically explored effectiveness of SMS reminders on childhood immunisation timeliness in LMICs.⁸⁷

Our findings are consistent with findings of other reviews that have assessed the effectiveness of SMS reminders on the childhood vaccination uptake,13 18 88 90 91 adherence to tuberculosis treatment,⁹² focused antenatal care visit and skilled birth attendance²⁷ and attending clinic appointments.^{10 28} Likewise, improvement in timely vaccination due to SMS reminders is consistent with the findings of other reviews.⁸⁸ A recent Cochrane systematic review comparing different types of patient reminders, however, noted that telephone reminders were more effective than SMS reminders.¹⁴ While telephone reminders may be more advantageous in reaching a population with limited or no education, the debate on which reminder is more cost-effective is inconclusive.^{47 93–98} Given that several studies have demonstrated mothers’ preference for phone call reminders over SMS reminders, particularly in populations with low literacy and resource-constrained settings,17 19 21 47 it is important to further explore in future studies if and how SMS reminders can be combined with mobile phone reminders (multiple modes of reminders) to obtain optimal outcomes.

Although this study demonstrated that SMS reminder is an effective tool for increasing vaccination uptake and timeliness in LMICs, a number of critical factors highlighted from included studies must be considered before implementing this intervention. Several mothers in included studies did not have personal mobile phones.^{52 62 63} Compared with urban setting, rural settings have high proportion of families sharing a single phone, which proves difficult to know if the message is going to the right person.^{17 21} This could considerably impact the roll-out of this intervention at scale. Furthermore, implementing this intervention could exacerbate existing health inequalities due to education and wealth.¹⁷ Additionally, mobile phone interventions require extensive infrastructure for mobile communication and mothers to have the facilities for charging their phones which might not always be the case, especially in rural communities.^{49 57} Also, SMS reminders depend on mothers being able to read SMS messages. Mothers in some studies have indicated preference for mobile phone calls to SMS as this allows two-way communication for questions and clarifications.⁴⁶ Furthermore, it is difficult to know if the SMS reminders sent were received and/or read, as one-way texting cellular companies may only record whether a message was sent, not received or read.^{59 60} Finally, forgetting vaccination appointment or ignoring child vaccination schedules are often not the main hurdle to accessing care. SMS reminders do not help address other prominent challenges such as accessibility (transportation, transport costs), attitude of healthcare workers, availability of vaccines, awareness of the importance of vaccination and outright rejection of vaccination.⁴⁹

Strengths and limitations

Our approach to this review has several strengths. We included RCTs and non-RCTs, searched multiple databases including grey literature and trial registries, and considered studies in any location or language. Therefore, we believe that we have assembled the widest possible body of relevant knowledge. In addition, we rigorously adhered to widely accepted guidelines for conducting and reporting systematic reviews. However, our findings are not without limitations. Risk of bias assessment indicated some concerns for risk of bias from the randomisation process in about a quarter of included RCTs. Likewise, we cannot rule out the possibility of confounding in the included non-RCTs. Furthermore, patients and/or clinical assessors in over half of included RCTs were not blinded, although a recent study showed no difference in estimated treatment effect between trials with and without blinded patients, healthcare providers or outcome assessors.⁹⁹ We included studies using two outcome measures; DPT-3 coverage and overall childhood immunisation coverage, which could have had disparate impact on the overall pooled estimate. Additionally, although we explored source of heterogeneity and differences in effect size, subgroup analyses must be interpreted with caution due to small sample size.¹⁰⁰

Implications for practice and research

Consistent with a growing body of literature, our study suggests the potential of mobile interventions including SMS reminders to boost the stagnating childhood immunisation coverage in LMICs as well as the timeliness of vaccination. More generally, digital communication technologies, including mHealth, have promising impact on healthcare. Hence, it is essential to support further innovative mHealth initiatives from all stakeholders. As more LMICs prioritise core e-government systems to facilitate social, health and economic activities,²³ governments and development partners need to consider building vaccination e-registries to leverage this digitalisation trend. Our review indicates that more qualitative studies are needed to understand the nuanced social and cultural details such as the optimal number of reminders, ideal time for sending reminders and content of reminder message for operational optimisation of SMS reminders. Finally, our review also suggests the need for more qualitative and quantitative studies to evaluate the effectiveness of a combination of multiple modes of reminders (eg, phone call and SMS reminders; or voice message and SMS reminders) on childhood immunisation coverage and timeliness especially in rural areas or low-literacy populations.

Conclusion

Meta-analysis of several LMIC studies has shown that SMS reminders can be effective at improving childhood vaccination uptake and timeliness. It is reasonable that in resource-constrained settings, simple and cost-effective mHealth interventions such as SMS reminders should be implemented in the healthcare system to improve child health outcomes. This is even more important in LMICs where increasing childhood immunisation coverage is a public health priority.

Data availability statement

Extracted data are available, on request, from the corresponding author.

Ethics statements

Patient consent for publication

Not required.