The Effect of Text Message Reminders to Health Workers on Quality of Care for Malaria, Pneumonia, and Diarrhea in Malawi: A Cluster-Randomized, Controlled Trial

Abstract.

The use of mobile technologies in medicine, or mHealth, holds promise to improve health worker (HW) performance, but evidence is mixed. We conducted a cluster-randomized controlled trial to evaluate the effect of text message reminders to HWs in outpatient health facilities (HFs) on quality of care for malaria, pneumonia, and diarrhea in Malawi. After a baseline HF survey (2,360 patients) in January 2015, 105 HFs were randomized to three arms: 1) text messages to HWs on malaria case management; 2) text messages to HWs on malaria, pneumonia, and diarrhea case management (latter two for children < 5 years); and 3) control arm (no messages). Messages were sent beginning April 2015 twice daily for 6 months, followed by an endline HF survey (2,536 patients) in November 2015. An intention-to-treat analysis with difference-in-differences binomial regression modeling was performed. The proportion of patients with uncomplicated malaria managed correctly increased from 42.8% to 59.6% in the control arm, from 43.7% to 55.8% in arm 1 (effect size −4.7%-points, 95% confidence interval (CI): −18.2, 8.9, P = 0.50) and from 30.2% to 50.9% in arm 2 (effect size 3.9%-points, 95% CI: −14.1, 22.0, P = 0.67). Prescription of first-line antibiotics to children < 5 years with clinically defined pneumonia increased in all arms, but decreased in arm 2 (effect size −4.1%-points, 95% CI: −42.0, 33.8, P = 0.83). Prescription of oral rehydration solution to children with diarrhea declined slightly in all arms. We found no significant improvements in malaria, pneumonia, or diarrhea treatment after HW reminders, illustrating the importance of rigorously testing new interventions before adoption.

INTRODUCTION

Mobile health, known commonly as mHealth, involves using portable electronic devices to support medical and public health practice.¹ mHealth can be used to target various aspects of health, including disease surveillance, monitoring availability of commodities, improving health services delivery, encouraging patient adherence to medications, and patient appointment reminders, among others.² Within mHealth, mobile phone text messaging is a relatively simple strategy to improve health-care delivery in low-income countries. Mobile phones are widespread in Asia and Africa, with an estimated 93% of Africans having access to cell phone services,³ and dissemination of health-related messages via mobile phones is an inexpensive way to reach geographically dispersed recipients.

Multiple studies from high-income countries have examined the use of mobile phones to improve patient diagnosis, care, and treatment; provide reminders to encourage healthy behaviors in patients; and facilitate patient appointment scheduling.^4–7 Evidence from developing countries, however, is scarce and largely descriptive, particularly for improving health service delivery.^8,9 The existing evidence from high-quality studies in developing countries is mixed. One cluster-randomized trial in Kenya showed that sending daily text message reminders to health workers (HWs) for 26 weeks led to a 24–percentage-point (%-point) improvement in correct management of uncomplicated malaria, an improvement that was sustained 6 months after discontinuing the messages.¹⁰ Messages were well received¹¹ and cost effective.¹² Another cluster-randomized trial from China found significant improvements in HW prescribing knowledge (17%-points) and practices (26%-point decrease in steroid prescriptions) after receiving text message reminders on the management of acute respiratory illness (ARI).¹³ Other text message interventions targeting HW behaviors in developing countries, however, have found little to no impact.^14,15 Several other recent articles on mHealth to improve health-care delivery, including one systematic review, concluded that additional rigorous trials are needed and that, such studies should be carried out in developing countries.^8,16,17

Most text message interventions have targeted a single disease or condition, whereas patient management requires an integrated approach. Two important questions regarding the potential for mHealth to improve case management, identified as part of a cost analysis of the pediatric malaria trial in Kenya, are the following: 1) how effective are text messages for improving malaria case management under a “test and treat” (versus clinical diagnosis) approach? and 2) how effective are text messages for improving the quality of care for multiple health conditions?¹² These questions are particularly relevant today, as many patients seeking care for a febrile illness will have a negative malaria test result. We conducted a study to assess the potential of text message reminders to HWs to improve case management of malaria (among patients of all ages) and of pneumonia and diarrhea (among children less than 5 years of age), in a three-arm cluster-randomized trial in Malawi.

MATERIALS AND METHODS

Study context.

Malaria remains endemic in all 28 districts of Malawi, a landlocked country in southeast Africa with an estimated population of 18 million. In 2014, malaria parasite prevalence among children aged 6–59 months was 33% nationally.¹⁸ In 2016, malaria accounted for an estimated 30% of outpatient visits and 6.2 million cases (both confirmed and presumed) at health facilities (HFs) and in communities.¹⁹ Malawi adopted guidelines for universal testing for malaria and scaled up rapid diagnostic tests (RDTs) in 2011. In 2013, Malawi updated its malaria guidelines to include injectable artesunate for the treatment of severe malaria and began refresher training on these guidelines in late 2014. According to a HF survey in 2015, malaria diagnosis and treatment practices were found to be reasonably good in Malawi; however, management of severe malaria and adherence to recommendations to give patients the first dose of antimalarials at the HF, a practice associated with better patient adherence,^20,21 were extremely poor.²²

Along with malaria, pneumonia and diarrhea are the leading causes of child death in Malawi, accounting for an estimated 14% and 8% of under-five deaths, respectively.²³ There is evidence that case management of these diseases at HFs, assessed based on Integrated Management of Childhood Illness (IMCI) guidelines, is poor.²⁴ Clinicians measured respiratory rates among children with cough and difficult breathing only 1.8% and 18.7% of the time, according to two studies,^25,26 and gave oral rehydration solution (ORS) to only 42.3% of children with diarrhea.²⁶ Malawi’s latest IMCI guidelines are from 2013, although the last nationwide training on IMCI took place in 2009–2010. Because of funding constraints, only seven of Malawi’s 28 districts, including Chikwawa, a district included in our study, received IMCI training in 2014. In addition, selected districts, including three in our study (Chikwawa, Nsanje, and Phalombe) received training on ARI case management in 2012–2013. All districts were trained on provision of zinc for diarrhea in 2011–2012.²⁴

Study area.

This trial took place in southern Malawi, where in 2014, the rate of malaria parasitemia in 6- to 59-month olds was 33%, similar to the national prevalence.¹⁸ Malaria transmission is perennial but peaks from January to April.²⁷ Seven districts, including Blantyre, Chikwawa, Chiradzulu, Mulanje, Nsanje, Phalombe, and Thyolo, were selected because HWs at these facilities had received a planned national malaria case management refresher training in late 2014 (see Figure 1). Within these districts, HFs were selected if they had a functional outpatient department (OPD), were accessible by road, and were managed by the Government or the Christian Health Association of Malawi, which together manage 89% of HFs in Malawi.²⁸ During the baseline survey, six facilities in Chikwawa and Nsanje could not be accessed because of unusually heavy rains and were, therefore, replaced by six facilities in Phalombe district. Facilities were the unit of randomization for the text message intervention, which was directed at HWs within facilities. Data collection at baseline and endline included facility-level assessments, HW interviews, and patient exit interviews (see details in the following paragraphs).

Figure 1.

Map of southern Malawi indicating health facilities included in the study.

Intervention description.

The cluster-randomized trial had three arms: 1) malaria-only messages, 2) malaria, pneumonia, and diarrhea messages, and 3) a control arm. In arms 1 and 2, unidirectional, twice-daily message reminders on key aspects of recommended management of malaria (all patients), pneumonia, and diarrhea (latter two only among children less than five years of age) were sent to HWs providing outpatient care and to drug dispensers involved in patient counseling. The intervention and study design were modeled after the study in Kenya that found significant improvements in malaria case management after 6 months of twice-daily, one-way text messages.¹⁰ Messages were sent in English at 8:00 am and again at around 1:30 pm, using the Telerivet platform (www.telerivet.com, San Francisco, CA) and the Twilio messaging service (www.twilio.com, San Francisco, CA). The 10 unique malaria messages were repeated each week in the malaria-only arm; in the malaria, pneumonia, and diarrhea arm, these malaria messages were sent for 1 week, and then followed by pneumonia and diarrhea messages the next week (see Table 1). Messages were based on the Malawi malaria case management²⁹ and IMCI guidelines³⁰ and designed to highlight key actions for improving patient diagnosis or treatment. All messages were a maximum of 160 characters and included a unique proverb in English or Chichewa, the predominant local language, to maintain HW interest in reading the messages.

Table 1.

List of malaria and pneumonia messages along with sample proverbs

Timing	Arm 1: Malaria only	Arm 2: Malaria, pneumonia, and diarrhea
Monday am	“Check ALL patients with fever or history of fever for signs of severe malaria! if any severe sign TREAT and REFER urgently!” Better be safe than sorry	“Check ALL sick children for any DANGER sign-if unable to drink OR lethargic OR vomit everything OR convulsions TREAT and REFER!” Well begun is half done
Monday pm	“Ask ALL patients about fever, take temperature, and check other malaria signs and symptoms.” Where there is smoke there is fire	“Check ALL sick children for danger signs! check ALL for fever, cough, difficult breathing, diarrhea, pallor, ear, and other problems!” The seeker is the finder
Tuesday Table 2: am	“Test for malaria ALL children < 5 and preg women with fever. Also, test ALL patients ≥ 5 with fever + one symptom.” Look before you leap	“For ALL children with cough or difficult breathing COUNT BREATHS in 1 minute and look for chest indrawing and stridor!” Actions speak louder than words
Tuesday pm	“For febrile patients without severe signs, TREAT for malaria ONLY if test is positive—do NOT treat negatives for malaria!” A word to the wise is sufficient	“Child has PNEUMONIA IF breath count is FAST: Over 60 if less than 2 months, over 50 if 2–12 months. Or over 40 if 12–59 months.” Never too old to learn
Wednesday am	“When malaria test is NEGATIVE, check for other causes; if none found, give antipyretic and ask patient to return if fever persists!” Persistent work triumphs	“Child with cough or difficult breathing has SEVERE PNEUMONIA if any danger sign, chest indrawing, or stridor present–TREAT and REFER!” Knowledge is power
Wednesday pm	“For uncomplicated malaria 1st line Rx is LA, 2nd line is ASAQ. For children < 5 kg and in 1st preg trimester, give quinine + clindamycin.” Do the right thing	“Child does NOT have pneumonia if breath count is NOT fast and NO danger/severe sign–treat for cold; do NOT give antibiotic!” Things don’t change; we change
Thursday am	“New pre-referral Rx for severe malaria is IM artesunate; if not available, use IM quinine; if quinine O/S, use rectal artesunate.” Never too old to learn	“For pneumonia, 1st line Rx is amoxicillin and 2nd line is erythromycin. For SEVERE pneumonia, treat with IM X-pen and REFER!” It works if you work it
Thursday pm	“Prescribe LA based on WEIGHT: 1 × 6 for 5–14 kg; 2 × 6 for 15–24 kg; 3 × 6 for 25–34 kg; 4 × 6 for ≥ 35 kg.” A goal without a plan is just a wish	“Give ALL children FIRST antibiotic dose at facility, explain dosing at home, and advise to finish all doses even if feel better!” A little late is too late
Friday am	“Give FIRST LA DOSE to ALL patients with uncomplicated malaria at FACILITY even if on an empty stomach!” To be willing is to be able	“Give ALL children with diarrhea ORS, zinc and advice on extra fluids; give antibiotics ONLY if blood in stool!” First things first
Friday pm	“Advise ALL patients to take 2nd LA dose after 8 hours, then every 12 hours to complete all six doses even if they feel better!” Say little but say it well	“Assess dehydration in child with diarrhea; if some dehydration, give ORS + 1st dose zinc at FACILITY; if severe, give IV fluid or REFER!” Willingness is the key

ORS = oral rehydration solution; Rx = prescription; IM = intramuscular; ASAQ = artesunate-amodiaquine. Messages displayed as they appeared on health workers’ mobile phones; some spacing was compressed to accommodate character count restrictions. “LA” is the local terminology for AL.

After baseline data collection in early 2015 (see in the following paragraph), a stakeholder workshop was held in late February 2015, where representatives from the Malawi Ministry of Health (National Malaria Control Programme and the IMCI Department), the study team, district health teams, and HF clinicians reviewed preliminary baseline results and discussed the most important messages to include in the intervention. All HWs providing outpatient care and all staff dispensing drugs in the pharmacy, who are responsible for patient counseling, in HFs randomized to intervention arms were asked for consent to participate and for all relevant mobile phone numbers (HWs in our study had an average of 1.4 mobile numbers each). Text messages were sent to consenting HWs twice daily on weekdays for 6 months between April and September 2015.

Data collection.

Trained survey teams composed of nurses and clinical officers collected baseline data at all study HFs from January to February 2015. At each HF, one OPD was randomly sampled (if more than one) and patients were systematically sampled in the waiting area based on patient volumes 1 week earlier for a target of 20 patients/facility; if eligible (presenting to the HF for the first time for their current illness) and consented (or their caregiver consented if < 18 years), patients were enrolled and an exit interview was conducted before they left the HF. Surveyors asked patients about their symptoms, HW interactions and their clinical encounter, any laboratory tests performed, medications prescribed, and how to take antimalarials or antibiotics if prescribed. Surveyors also checked patients’ health passport (a medical record kept by patients) and performed a brief reexamination, including a blood smear. At the end of the day, interviews with all clinicians providing care in the sampled OPD were conducted to assess HWs’ background, training, supervision, and access to relevant guidelines; HWs were also asked to complete a brief set of clinical case scenarios. An assessment of HF staffing, equipment, and drug stocks was also conducted through interviews and visual inspection. Additional details on the data collection methodology can be found elsewhere.^22,26 The same data collection procedures, including assessment of HWs’ exposure to the study intervention, occurred at the endline, when survey teams visited all HFs in November–December 2015.

Sample size determination.

The primary outcome for sample size determination was the proportion of patients with suspect uncomplicated malaria who were managed correctly (see Table 2). To detect a difference in correct management of suspected malaria of 16%-points between each intervention arm and the control arm, assuming a power of 80%, alpha of 5%, a design effect of three, and 66% correct management of patients in the control arm at follow-up, 573 patients with suspected malaria were needed per arm. Assuming that 85% of patients coming to HFs have suspected malaria,³¹ 700 patients per arm were needed for a total of 2,100 patients across the three arms, at both baseline and endline. Assuming that a survey team could interview 20 patients per day on average at a HF, 35 HFs were required per arm, or 105 in total for the study.

Table 2.

Definitions of outcomes

Definitions of conditions and diseases
Disease/condition	Definition	Correct case management steps
Suspected uncomplicated malaria	< 5 years: Fever* AND no signs of severe malaria/febrile illness†	RDT or blood smear performed (if RDT in stock or facility able to do microscopy)
	5+ years: Fever* AND no signs of severe malaria AND at least one other sign/symptom suggestive of malaria, including chills, joint/muscle pains, headache, nausea, vomiting, diarrhea, abdominal pain, profuse sweating, or general malaise/weakness‡	If microscopy/RDT positive (or presumed malaria if facility unable to do diagnostic test): All of the following prescribing and counseling tasks:
		First-line antimalarial prescribed§
		If AL, prescribed in recommended weight- or age-specific dose
		If AL, tablets dispensed to patients if AL in stock
		First dose of AL given at facility
		Patient/caregiver knows how to take drug (correct number tablets/dose, number of doses/day, and number days to take)
		Patient/caregiver recalls being told to complete all doses even if feel better
		If microscopy/RDT negative:
		No antimalarial prescribed or dispensed
Suspect severe malaria/severe febrile illness	Fever* AND one or more general danger signs or signs of severe malaria†	Referral to inpatient unit
		Administration of IV, IM, or rectal artesunate (or IV/IM quinine if artesunate not available at facility)
Uncomplicated pneumonia (< 5 years)	Cough or difficult breathing AND fast respiratory rate (≥ 60 if age < 2 months; ≥ 50 if aged 2–11 months or ≥ 40 if 12–59 months)	HW measured respiratory rate
		First-line antibiotic prescribed‖
		First dose of antibiotic given at facility.
Diarrhea (< 5 years)	Diarrhea reported by HW or patient	Oral rehydration solution administered or dispensed

AL = artemether–lumefantrine; HW = health worker; IM = intramuscular; IV = intravenous; RDT = rapid diagnostic test.

* Self-reported history during exit interview, measured temperature during re-examination, or measured and documented by HW.

† Signs/symptoms of severe malaria/febrile illness include at least one of the following: vomiting everything, convulsions, unconsciousness/lethargy, jaundice, extreme pallor, little to no urine output, dark urine, stiff neck, and unable to drink/breastfeed (last two only for < 5 years)

‡ As per Malawi Malaria Case Management guidelines 2013.

§ In most cases, AL (first-line) or artesunate–amodiaquine (second-line), but quinine plus clindamycin for pregnant women in their first trimester and children weighing < 5 kg.

‖ Cotrimoxazole, erythromycin, or amoxicillin.

Randomization and analytical approach.

As patient-level randomization is not feasible for interventions directed at HWs, such as text messages, facilities were randomized to intervention arms after baseline data collection. After stratifying by facility type (district hospital, rural/community hospital, and health center) and then by district, facilities were rank-ordered by their score at baseline on correct management of patients with suspected uncomplicated malaria. Triplets of facilities were then randomized, using a random number generator in Microsoft Excel, to each of the three arms. As all facilities were allocated at once, there was no need to conceal allocation.

We used a difference-in-differences (DiD) analysis approach—([endline–baseline])_intervention minus [endline–baseline]_control]—to assess the effects of text messages in each intervention arm compared with the control arm on key malaria, pneumonia, and diarrhea case management indicators. We assessed the relevance of potential confounding variables, drawn from Table 3 and previous literature on case management in Malawi,^22,26 by adding them to the DiD model one at a time and assessing their effect on the coefficients of interest. Confounders were defined as variable that changed the effect sizes of interest by 20% or more if the effect sizes were significant in the model (or rendered significant by the addition of the confounder). The primary analysis was intention to treat, including all relevant patients at the 105 facilities; a sensitivity analysis was carried out using a per-protocol (PP) population, whereby only patients seen by HWs in either intervention arm who reported receiving the messages at endline (and all patients to whom a given indicator applied at these selected facilities were included at baseline) were included, along with all relevant patients in the control arm.

Table 3.

Characteristics of facilities, HWs, and patients surveyed at baseline and endline

	Baseline				Endline	P-value*
	Arm 1	Arm 2	Control	All
Facilities	N = 35	N = 35	N = 35	N = 105	N = 105	–
Malaria diagnostics available	82.9	82.9	91.4	85.7	92.4	0.1219
Microscopy	22.9	22.9	28.6	24.8	19.0	0.4520
Rapid diagnostic tests	73.4	77.1	85.7	79.0	91.4	0.0043
Artemether–lumefantrine in stock (any dose)	88.6	100.0	86.7	91.4	87.6	0.3674
Any first-line antibiotic in stock	100.0	100.0	97.1	99.1	96.2	0.1745
Amoxicillin	60.0	85.7	62.9	69.5	73.3	0.5412
Cotrimoxazole	97.1	100.0	91.4	96.2	90.5	0.0969
Erythromycin	71.4	74.3	74.3	73.3	69.5	0.5412
Oral rehydration solution in stock	80.0	82.9	80.0	81.0	84.8	0.4639
HWs	N = 50	N = 52	N = 48	N = 150	N = 131	–
Cadre
Medical officer/doctor	0.0	0.0	0.0	0	1.5	0.0489
Clinical officer (3 years training)	10.2	21.6	15.2	15.8	13.7
Medical assistant (2 years training)	73.5	64.7	67.4	68.5	76.3
Nurse	12.2	11.8	10.9	11.6	8.4
Other	4.1	2.0		4.1	0.0
Female	26.0	6.5	29.2	26.7	29.0	0.6620
Access to malaria treatment guidelines	46.0	51.9	41.7	46.7	26.7	0.0006
Access to Integrated Management of Childhood Illness guidelines	20.0	26.9	37.5	28.0	25.2	0.5954
2014 Malaria case management training	70.0	73.0	68.8	70.7	75.6	0.3557
Supervision last 6 months	62.0	65.4	75.0	67.3	58.8	0.1376
Patients	N = 774	N = 794	N = 792	N = 2,360	N = 2,536	–
Fever (history of or temperature ≥ 37.5°C)	76.6	76.7	73.5	75.6	66.1	< 0.0001
Suspected uncomplicated malaria	75.0	73.3	70.4	73.0	59.7	< 0.0001
Suspected severe malaria	4.3	2.4	3.1	3.3	1.7	0.0204
< 5 years of age	N = 221	N = 261	N = 243	N = 725	N = 784	–
Cough/difficult breathing	54.8	54.5	46.7	51.7	61.3	0.0021
Pneumonia (uncomplicated)	23.4	22.4	15.8	20.3	25.3	0.1047
Diarrhea	42.2	37.4	25.8	34.7	21.6	0.0004

HW = health worker.

* P-values assess differences between baseline and endline. P-values from chi-square tests, except for HW cadre (Fisher’s exact test). Chi-square tests for patients use the Rao-Scott correction to account for sampling.

Patient-level analyses were weighted to account for sampling probability (of OPDs and patients) and the response rate, and analyses took into account clustering at the facility triplet level by using generalized estimating equations.³² Difference-in-differences regression models were run using the GENMOD procedure in SAS, specifying a binomial distribution, identity link, and an independent correlation structure to generate risks and risk differences.³³

Definitions of outcome variables.

Outcomes were based on Malawi case management guidelines^29,30 and definitions are included in Table 2. Our primary outcome was correct management of suspected uncomplicated malaria, which involved diagnostic testing (if the facility had the ability to test), or presumptive treatment (if unable to test), and treatment according to test results, including correct dosing of first-line antimalarials if positive, along with key counseling messages and no antimalarials if test-negative (Table 2).

RESULTS

Among the 105 HFs included in the study, the ability to diagnose malaria on the day of the survey team’s visit either by microscopy or RDT was 85.7% at baseline and 92.4% at endline (Table 3). Stocks of essential medicines for malaria, pneumonia, and diarrhea were high and not significantly different between the baseline and endline surveys. More than 87% of facilities had at least one type of artemether–lumefantrine (AL), the first-line antimalarial, in stock, and more than 99% had at least one of the antibiotics appropriate for pneumonia treatment in stock.

At most facilities, only one HW providing outpatient care was interviewed, and the majority (68.5% at baseline and 76.3% at endline) were medical assistants, a position with 2 years of formal training (Table 3). About three-quarters of HWs had received malaria case management refresher training in 2014. Nearly half (46.7%) of HWs at baseline had the latest malaria case management guidelines on site, although only 26.7% did at endline, P = 0.0006.

Among the 2,360 patients with complete exit interviews at baseline (see study flow chart in Supplemental Figure 1), in January–February 2015 (rainy season), 75.6% had fever, according to self-report or HWs’ documentation in their health passport compared with 66.1% of the 2,536 patients interviewed at endline in November–December 2015 (drier season) P < 0.0001 (Table 3). More children less than 5 years of age at endline (61.3%) were reported to have cough or difficult breathing compared with children at baseline (51.7%), P = 0.0021, and diarrhea was more prevalent among children at baseline (34.7%) than at endline (21.6%), P = 0.0004 (Table 3).

The primary outcome of correct case management of suspected uncomplicated malaria increased in all arms but the change was not significantly greater in the intervention arms: the proportion of patients managed correctly increased from 42.8% to 59.6% in the control arm, from 43.7% to 55.8% in arm 1 (effect size −4.7%-points, P = 0.50), and from 30.2% to 50.9% in arm 2 (effect size 3.9%-points, P = 0.67). No confounding was identified.

Comparing baseline and endline measurements, there were no differential changes in secondary malaria case management indicators by intervention arm, with the exception of patient knowledge of dosing if they were prescribed AL, which improved significantly more in the malaria, pneumonia, and diarrhea intervention arm compared with control (DiD = 18.2%-points, P = 0.012) (Table 4, secondary outcome 6). Testing of suspected malaria patients (secondary outcome 1) was reasonably good, with around three-quarters of suspected malaria patients tested at both baseline and endline. Giving the correct antimalarial in the correct dose (secondary malaria outcome 3) was also high (> 90% at endline) but did not significantly improve as a result of the intervention. Providers rarely gave an antimalarial for patients with a negative RDT, with only 3.8% of patients at baseline and 0.9% at endline treated if they had a negative RDT (secondary malaria outcome 8). At both baseline and endline, most of the patients reported being told to complete all doses, even if they felt better (secondary malaria outcome 7).

Table 4.

Malaria, pneumonia, and diarrhea case management results, by time point and study arm

	Baseline			Endline			Effect size (difference in differences)
	Arm 1	Arm 2	Control	Arm 1	Arm 2	Control	Arm 1, % change (95% CI)	P-value	Arm 2, % change (95% CI)	P-value
Primary outcome: Correct case management of suspect uncomplicated malaria
Overall correct malaria case management*	237/573 (43.7%)	186/565 (30.2%)	225/559 (42.8%)	288/518 (55.8%)	247/462 (50.9%)	285/540 (59.6%)	−4.7 (−18.2, 8.9)	0.501	3.9 (−14.1, 22.0)	0.670
Secondary outcomes: Malaria
1. RDT/Blood smear performed	327/428 (81.4%)	339/440 (73%)	384/490 (79.6%)	365/463 (75.9%)	324/411 (76.6%)	410/522 (78.1%)	−4.0 (−14.0, 6.0)	0.435	5.2 (−15.5, 25.8)	0.623
2. First-line AM†	212/281 (76.0%)	227/283 (82.0%)	238/272 (88.5%)	156/181 (87.6%)	131/147 (89.6%)	177/190 (94.0%)	6.1 (−16.2, 28.5)	0.590	2.2 (−15.9, 20.3)	0.812
3. If AL, correct dose	175/221 (79.6%)	240/257 (94.2%)	210/265 (82.3%)	109/113 (97.2%)	108/111 (98.9%)	136/139 (98.3%)	−1.7 (−19.5, 22.9)	0.862	−11.3 (−28.0, 5.4)	0.186
4. If AL, tablets dispensed	182/188 (97.1%)	246/257 (95.7%)	214/222 (97.2%)	113/114 (99.1%)	109/125 (87.2%)	135/136 (99.8%)	−0.6 (−6.0, 4.8)	0.831	−10.5 (−27.2, 5.0)	0.178
5. First dose AL given	23/182 (11.9%)	19/245 (5.4%)	39/213 (27.8%)	13/113 (13.7%)	25/109 (20.1%)	15/135 (12.2%)	17.2 (−14.8, 49.1)	0.292	30.4 (−5.7, 65.7)	0.099
6. If AL, patient knows dose	134/182 (74.3%)	171/246 (68.3%)	173/214 (86%)	84/113 (71.7%)	85/109 (78.2%)	99/135 (77.7%)	5.7 (−10.2, 21.7)	0.482	18.2 (4.0, 32.4)	0.012
7. If AL, patient told to complete	108/182 (60.8%)	135/247 (54%)	119/214 (60.1%)	50/113 (41.1%)	62/109 (54.7%)	71/135 (53.7%)	−13.3 (−38.0, 11.4)	0.292	7.2 (−14.5, 29.0)	0.515
8. IF RDT neg, no AM	185/191 (97.2%)	168/181 (94.8%)	168/181 (95.9%)	238/239 (99.7%)	226/228 (99.5%)	232/238 (98.3%)	0.0 (−5.1, 5.0)	0.992	2.3 (−4.6, 9.1)	0.517
9. Correct treatment severe malaria	0/33 (0%)	3/25 (7.3%)	0/32 (0%)	3/16 (11.6%)	0/10 (0%)	0/6 (0%)	11.6	–	−7.3	–
Secondary outcomes: Pneumonia and diarrhea
1. Health worker measured respiratory rate if child had cough/difficult breathing	5/115 (3.1%)	4/137 (2.3%)	0/120 (0%)	5/185 (3.9%)	0/118 (0%)	5/161 (1.9%)	−1.1	–	−4.2	–
2. Prescription of first-line antibiotic for children with pneumonia‡	31/45 (63.4%)	39/56 (68.3%)	29/42 (59%)	57/80 (73.5%)	39/51 (78.1%)	48/68 (73%)	−3.8 (−38.4, 30.7)	0.828	−4.1 (−42.0, 33.8)	0.831
3. First dose of antibiotic given at facility	1/31 (2.2%)	0/39 (0%)	1/29 (4.4%)	0/57 (0%)	1/39 (3.1%)	1/48 (0.7%)	1.5	–	6.8	–
4. Prescription of oral rehydration solution to children with diarrhea§	57/93 (58.7%)	73/98 (76.8%)	48/81 (58.6%)	35/64 (52.3%)	22/39 (59.7%)	39/69 (55.7%)	−3.5 (−24.7, 17.6)	0.743	−14.2 (−39.5, 11.1)	0.272

AL = artemether–lumefantrine; CI = confidence interval; RDT = rapid diagnostic test; AM = antimalarial. Data are n/N (%) unless otherwise stated. Artemether–lumefantrine-related indicators for patients with suspected malaria. Suspect uncomplicated malaria: 72% at baseline and 60% at endline. Test positivity rate (facility malaria test): 47.5% at baseline and 35.1% at endline. Ninety percent patients attended facilities with AL in stock at baseline, 84% at endline.

* All applicable case management actions performed for patient with suspected uncomplicated malaria (Table 2).

† Among patients testing positive at facility for malaria or suspect malaria cases not tested at facilities unable to test.

‡ Among children with clinically diagnosed pneumonia, according to re-examination during exit interviews.

§ An alternative definition for correct diarrhea treatment including both ORS and zinc was used, but was lower overall: 38.2% and 35.0% at endline, with no significant differences by arm. This indicator was not used for diarrhea case management as zinc stocks were not systematically measured during the survey.

Administering the first dose of AL at the facility (secondary malaria outcome 5) was very low in all arms at both time points and was not significantly improved by the intervention; in the malaria, pneumonia, and diarrhea message arm, this practice improved from 5.4% at baseline to 20.1% at endline (secondary pneumonia and diarrhea outcome 3), but this increase was not significant, despite a 15.6%-point reduction in the control arm (P = 0.099). Suspected severe malaria was present in 3.3% of patients at baseline and 1.7% at endline (Table 3), and almost none of these patients were correctly managed with pre-referral intramuscular artesunate and referral or admission to the inpatient setting (Table 4, secondary malaria outcome 9).

Pneumonia and diarrhea indicators also did not significantly improve as a result of the text message intervention. Health workers’ measurement of respiratory rate remained extremely low at less than 5% in all arms at both time points (secondary pneumonia and diarrhea outcome 1). Although prescription of the first-line antibiotic to children with clinically diagnosed pneumonia (according to the surveyors’ re-examination) improved by 10–14%-points in all arms from baseline to follow-up, the improvements did not differ by arm (secondary pneumonia and diarrhea outcome 2). Giving the first dose of antibiotic at the HF remained extremely low in all arms, and prescription of ORS to children with diarrhea decreased slightly in all arms from baseline to endline (secondary pneumonia and diarrhea outcome 4).

At the endline, 39.5% of HWs interviewed in the malaria-only arm and 46.5% in the malaria, pneumonia, and diarrhea arm reported receiving text messages, P = 0.514 (see Table 5). Delivery reports from the software used to send the messages, indicated that approximately 4% of the messages sent were not delivered or failed to send, whereas the rest were either delivered or sent, indicating they should have been received (the software could not detect whether messages had actually been opened). Among the HWs receiving the messages, according to the endline survey, most reported receiving the messages twice per day (65–77%) and reading them twice per day (65–71%); overall, 39.5% of HWs in either intervention arm received and read messages at least once per day. Analysis of the PP population did not differ much from the intention-to-treat population (Supplemental Table 1). Changes in two prescribing indicators—prescription of the correct dose of AL (outcome 5) and dispensing AL if in stock at the facility (outcome 4)—were actually significantly worse in the malaria, pneumonia, and diarrhea arm compared with the control using the PP population (DiD effect sizes of −15.8%-points, P = 0.0383 and −17.5%-points, P = 0.045, respectively).

Table 5.

Percentage of HWs receiving and reading text messages at various frequencies

	Study Arm
	Malaria-only	Malaria, pneumonia, and diarrhea
	N = 43	N = 43
Received messages	39.5	46.5
Topics of messages	N = 17	N = 20
Malaria	94.1	5.0
Malaria and pneumonia	0.0	5.0
Malaria, pneumonia, and diarrhea	5.9	90.0
Frequency of message receipt
Three times/day	5.9	5.0
Twice/day	76.5	65.0
Once/day	17.6	25.0
Once/week	0.0	5.0
Frequency of reading messages
Twice/day	70.6	65.0
Once/day	29.4	30.0
Once per week	0.0	5.0

HW = health worker. No HWs in control arm reported receiving messages. No significant differences in any indicators between intervention arms.

Health workers scored on average 6.34 of nine points on the clinical case scenarios at baseline, and there was no difference among study arms (P = 0.136). At the endline, HWs scored on average 6.40, with no difference among arms (P = 0.836), including when analysis was limited to the PP cohort of HWs (P = 0.450) (Supplemental Table 2).

DISCUSSION

The baseline data from this cluster-randomized trial of text message reminders to HWs in Malawi found relatively good performance for several important malaria case management indicators (testing patients with suspected uncomplicated malaria, giving the correct drug in an appropriate dose, and not treating RDT-negative patients) but very poor performance for other aspects (giving the first dose of antimalarials at facilities and management of patients with suspected severe malaria). Text message reminders did not improve diagnosis and treatment practices among HWs. Pneumonia and diarrhea case management indicators also did not improve as a result of text message reminders. Measuring respiratory rates among children with cough or difficulty breathing, a key IMCI step in the clinical diagnosis of pneumonia, remained extremely low in all study arms at endline, and other indicators (giving first-line antibiotics for children with clinically diagnosed pneumonia, giving the first dose of antibiotics at the facility, and giving ORS for children with diarrhea) did not improve.

Stocks of key commodities (RDTs, antimalarials, antibiotics, and ORS) were relatively good at both time points, and our treatment indicators for the three diseases included dispensing or prescribing the drug (if not in stock), so drug/commodity stockouts do not explain the lack of improvement in case management indicators. At endline, fewer HWs than anticipated in the intervention arms reported receiving the text messages, but PP analyses restricted to these HWs still showed little effect of the messages. Prescription of first-line antibiotics improved slightly, but not significantly, in all arms, possibly because endline data collection took place in the dry season, when cough and respiratory infections are more prevalent among children, and thus potentially more in the forefront of HWs’ minds.^34,35 Similarly, prescriptions of ORS for diarrhea were lower in all arms at endline in the dry season, when treatment of diarrhea may have been less salient to HWs.

It is unclear why some malaria treatment indicators, including prescribing the first-line antimalarial to patients with malaria, in the correct dose, improved slightly in all arms. Because our definition of first-line antimalarial prescription included prescribing to patients with suspected malaria at facilities unable to test, which is lower than prescribing antimalarials to patients with confirmed malaria,²² the improvement in RDT availability between baseline and endline might help account for this improvement. Similarly, the lower test positivity rate at endline (in the dry season) and consistently good management of RDT-negative patients might help account for the improvement over time in overall correct malaria case management across arms. No HWs in the control arm reported receiving messages, and contamination across study arms was unlikely in most facilities, which are primarily rural, although it is possible that contamination occurred among the eight urban facilities in Blantyre that are within several kilometers of one another.

Qualitative interviews conducted with HWs 2 months after endline as part of this study indicated that messages were generally favorably received by both drug dispensers and clinicians, who appreciated the clinical reminders. However, some HWs reported that structural barriers, such as high patient loads, as well as lack of dedicated supervision to model and reinforce the practices suggested in the text messages, made it difficult to act on some of the recommendations, especially measuring respiratory rates and giving the first dose of drugs at the facility. Complete qualitative results are reported elsewhere (Kaunda et al., under review). Similar structural barriers have been reported by HWs using mHealth interventions in other low-income settings, such as an IMCI-based software on tablets and smartphones in Tanzania,³⁶ even when results have shown improvements in case management.³⁷

Two of the lowest indicators at baseline that did not improve—measuring respiratory rate and dispensing the first dose of medications at the facility—are also two of the most time-consuming tasks. The Tanzania study that used an adapted IMCI algorithm on smartphones and tablets found that although respiratory rate measurements did improve significantly, HWs still only did these about half the time.³⁷ Respiratory rates have relatively poor sensitivity for predicting pneumonia,³⁸ and it is possible HWs did not think the extra time to do them was warranted. Directly observing the first dose of medications requires HW time at overcrowded HFs in Malawi (the median number of daily visits per OPD at baseline was 98); furthermore, HWs and patients in Malawi have previously been taught that AL should be given with food, milk, or a milk-containing drink, and some HWs might have been afraid to give AL if they thought a patient had not eaten. The qualitative study undertaken shortly following the endline data for this study explores reasons why the text message intervention was not successful in changing HW behavior (Kaunda et al., under review).

It is not entirely clear why this intervention did not lead to improvements in “simpler” case management practices emphasized by the text messages. Two rigorous trials involving drug sellers in Ghana and Tanzania found that text message reminders on diarrhea and malaria case management, respectively, improved provider knowledge, but did not improve their actual practices.^14,15 In our study, however, HW knowledge did not substantially improve as a result of the text messages.

Although text messages are very inexpensive—the messages cost on average US$0.035 each to send—messages alone might not be sufficient to improve HW practices in Malawi. mHealth interventions using tablets or smartphones with programmed clinical algorithms that more directly guide case management practices might be more effective and have shown promising results in similar settings.^37,39 Even more promising might be electronically programmed clinical algorithms that integrate additional point-of-care tests, which have shown improved child health outcomes than electronic algorithms alone.⁴⁰ Other alternative strategies to consider for improving HW performance include adding a small financial incentive to the text message intervention for participation, which resulted in good participation rates in an SMS-based drug stock reporting system in Tanzania,⁴¹ and more systematic supervision to reinforce case-management practices, with or without text messages. A study of weekly text message reminders to HWs in Cameroon to encourage reporting of adverse events following immunization with meningococcal vaccine found that messages improved reporting rates, but they were not as effective as weekly, in-person supervision.⁴²

Limitations.

Our study had several limitations. Study teams did not directly observe HW consultations but relied on patient exit interviews at baseline and endline. Although nearly all patients (more than 98% in our study) have passports where HWs document clinical findings, it is possible that study capture of certain assessment tasks that were not documented in the health passport or noticed by patients, such as measuring respiratory rate, was lower than it should be.^22,25 We also did not conduct dedicated formative research before intervention design to understand in more depth the barriers to good case management; instead, we relied on baseline data and discussions and feedback during the stakeholders’ workshop to shape the message content and delivery. Although our study area was limited to southern Malawi, HFs there are generally similar to those in other parts of Malawi, and previous studies have found no significant regional differences in malaria case management practices.³¹ The baseline and endline surveys occurred at different times of the year, in the high and low malaria transmission seasons, respectively, which might have influenced HW decisions. However, our survey timings were based on the previous successful study in Kenya,¹⁰ and our randomized study design and inclusion of a control group enabled us to see potential differential effects of the text messages. Despite a multitude of small-scale mHealth pilot initiatives, our study is one of the few rigorously designed trials to assess the effectiveness of text message reminders.

Conclusions.

This study found that simple text message reminders alone were not successful in improving diagnosis and treatment for malaria, pneumonia, or diarrhea among HWs in Malawi. Management of severe malaria, giving the first dose of antimalarials and antibiotics at the facility, measuring respiratory rates when indicated by IMCI, and treatment of diarrhea remained poor. Improving case management practices might require more resource-intensive or multifaceted interventions.

Note: Supplemental tables and figure appear at www.ajtmh.org.