Impact of a mHealth Intervention for Peer Health Workers on AIDS Care in Rural Uganda: A Mixed Methods Evaluation of a Cluster-Randomized Trial

Abstract

Mobile phone access in low and middle-income countries is rapidly expanding and offers an opportunity to leverage limited human resources for health. We conducted a mixed methods evaluation of a cluster-randomized trial exploratory substudy on the impact of a mHealth (mobile phone) support intervention used by community-based peer health workers (PHW) on AIDS care in rural Uganda. 29 PHWs at 10 clinics were randomized by clinic to receive the intervention or not. PHWs used phones to call and text higher level providers with patient-specific clinical information. 970 patients cared for by the PHWs were followed over a 26 month period. No significant differences were found in patients’ risk of virologic failure. Qualitative analyses found improvements in patient care and logistics and broad support for the mHealth intervention among patients, clinic staff, and PHWs. Key challenges identified included variable patient phone access, privacy concerns, and phone maintenance.

Introduction

mHealth (mobile technologies for health) has been advocated as an innovative tool for improving both access to and quality of health care in low and middle-income countries (LMIC) [1–3]. Mobile phone access in LMIC has risen dramatically, creating significant opportunities for creative and cost-effective implementation of mHealth interventions [4]. However, despite the growing interest in mHealth, there remains limited evidence on their acceptability and impact on health care outcomes [5–10].

Concurrent with the rise of mobile telecommunications in LMIC has been the growth in antiretroviral therapy (ART) [11]. While there have been successes with ART scale-up, a continuing barrier to ART delivery are human resource limitations [12]. One strategy to address this crisis is through task shifting, the rational redistribution of tasks among health workforce teams from higher trained providers to those who require less training [13]. Community health workers (CHWs) are a key cadre to whom tasks are shifted but their effectiveness can be limited by inadequate resources and supervision [13, 14]. Training people living with HIV (PLHIV) to care for peers as Peer Health Workers (PHWs) is one application of the CHW concept.

To evaluate the impact of PHWs on AIDS care, we previously conducted a cluster-randomised trial at a rural ART program in Uganda [15–17]. Details and evaluations of this main trial are reported elsewhere [18]. Embedded within this trial was an exploratory cluster-randomised substudy on the impact of a mHealth support intervention used by PHWs. Descriptive pilot and feasibility data on this intervention were previously presented [19], and this mixed methods evaluation reports the final substudy trial outcomes and process results [20–22].

Methods

The parent trial was pragmatic, meaning a general framework for the PHW intervention was developed, but was allowed to adapt to local needs [17]. Mixed methods research has been deemed critical to understanding these types of pragmatic trials with complex interventions and helping move study findings to programs and policy [16, 20]. We therefore chose to conduct a mixed methods evaluation of the substudy to best understand trial results and the underlying processes [20, 21].

Study Setting and Parent Trial

In 2004, the Rakai Health Sciences Program (RHSP) began providing ART through a President’s Emergency Plan for AIDS Relief (PEPFAR)-funded mobile clinic program. The program consisted of staff traveling from a central facility to each remotely-located clinic biweekly. The main PHW trial took place from 2006 to 2008 and randomized 10 clinic sites to the PHW intervention arm and 5 as controls. PHWs received a two day residential training on basic HIV pathogenesis, prevention, treatment, adherence counseling, performing pill counts, protecting patient confidentiality, and filling out a home visit form. PHWs provided clinical and adherence monitoring and psychosocial support to fellow patients at clinic and during periodic home visits. At home visits, PHWs recorded a review of symptoms, an adherence self-report, and pill count results using a standardized form. To assist with their duties, PHWs were each given a bicycle, t-shirts, basic supplies, and an initial monthly allowance of about 12.5 USD. PHW day-to-day supervision was largely performed by a single RHSP staff member working part-time. The PHW intervention was found to decrease virologic failures among patients on ART for 96 weeks or more, decrease loss to follow-up, and successfully task shifted, but had no effect on adherence measures or shorter-term virologic outcomes [18, 22].

Substudy Trial Description

The 10 sites originally randomized to the PHW intervention were further randomized 2:3 to PHWs receiving a mHealth support intervention or not [18]. In addition to their baseline training, PHWs randomized to the mHealth Arm (mHealth+ Arm, 4 clusters, 13 PHWs) were each given a mobile phone, a one day residential training, and an hour long field-based practicum training on the mHealth intervention. These PHWs were asked to send a text message reporting adherence and clinical data back to a centralized database immediately after or during each home visit. These texts were numeric codes produced by converting all quantitative data on the home visit form to simple integers arranged in a predetermined order separated by asterisks, e.g. pills given, taken, defaulted might convert to *282828*. PHWs in the mHealth+ Arm were also encouraged to call a RHSP mobile phone hotline or toll-free warmline (staffed only during clinic hours) with questions or concerns [23]. Clinic staff receiving PHW texts and calls could opt to provide care instructions to PHWs, send a higher level care provider to the patient, or arrange transport to health care facilities. The comparison group (mHealth− Arm, 6 clusters, 16 PHWs) consisted of PHWs who did not receive the mHealth intervention.

Quantitative Methods and Analysis

The primary outcome was patients’ cumulative risk of virologic failure (any failure during follow-up period equaling failure with viral loads conducted every 24 weeks). Secondary outcomes included patient adherence (pill counts and 3 day self-report), virologic failure at 24 and 48 weeks of ART (failure defined as >400 copies/ml), lost to follow-up, and mortality. Lost to follow-up was defined as lack of a pharmacy refill visit in over 90 days. Mortality was ascertained through verbal autopsies. Study power was calculated estimating 660 patients at a 4:6 intervention:control ratio with balanced numbers of cluster participants and no matching. Assuming a 5% drop out rate, we anticipated approximately 3,273 person-weeks of follow-up. Based on previous RHSP studies, we used a 24 week virologic failure risk of 28%, a coefficient of variation (k) of 0.11 and intra-class correlation coefficient (P) of 0.0024 (the intracluster correlation coefficient reported from the parent trial was eventually noted to be 0.0015) [18, 24, 25]. With a two-sided alpha = 0.05 and 1-beta = 0.80, the study was estimated to have power to detect a reduction in any virologic failure with a rate ratio of ≤0.70. Efficacy analyses were by intention to treat using cluster summaries and a generalized linear model appropriate for cluster-randomised trials [26]. This method allowed for estimation of risk ratios and was chosen due to the relative small number of clusters randomized and instability of more complex multi-level models. Given the very low intraclass correlation coefficient, alternative analyses assuming independence on individuals in clusters were also performed and found to be similar to our primary analyses results. A Likert scale survey (1 = agree strongly, 3 = neutral, 5 = disagree strongly) of clinical staff perceptions was also administered near trial end and analyzed descriptively. Analyses were done with STATA v10 (StataCorp, College Station, TX).

Qualitative Methods and Analysis

Qualitative evaluation of the substudy occurred in conjunction with a mixed methods evaluation of the parent trial [22]. In the mHealth+ Arm, we conducted 4 in-depth interviews with PHWs and 6 with patients. In the mHealth− Arm, we conducted 6 interviews with PHWs and 6 with patients. Ten RHSP clinic staff were also interviewed. We conducted 4 patient focus groups, 2 from each Arm, composed of 7–10 patients each. One PHW focus group from each Arm consisting of seven participants each was conducted. All participants were purposively selected. Written informed consent was obtained from all participants. Study investigators reviewed and edited semi-structured data collection guides after conducting pilot interviews. Interview questions were open-ended. Interviews and focus groups specific to the substudy focused on personal experiences with phones in general and, in the mHealth+ Arm, specifically on intervention processes. All interviews and focus groups were tape recorded and transcribed, conducted in Luganda with patients and PHWs and in English with staff. Translation and transcription was done in a single step by the interviewer. After transcription, data were entered into NVivo 8 for coding and analysis (QSR International, Victoria, Australia). Transcript review began while data collection was still underway and participant responses were used to shape future interview questions. We developed a codebook by categorizing responses from interviews into major thematic categories and performed line-by-line coding of transcripts.

Ethical Review

The trial was approved by institutional review boards at the Uganda Virus Research Institute’s Safety and Ethics Committee, the Uganda National Council for Science and Technology, and Johns Hopkins University.

Results

Quantitative Results

Figure 1 shows the flow of participants in the substudy. There were a total of 970 patients (446 in mHealth+ Arm, 524 in mHealth− Arm) followed during the study period. Median follow-up time for virologic outcomes was 103 weeks per participant (interquartile range, 97–111 weeks). Figure 2 shows a map of the study area and clinic sites. Table 1 shows enrollment characteristics. Sociodemographic characteristics and key predictors of clinical outcomes appeared well balanced between arms.

Fig. 1.

Trial profile

Fig. 2.

Map of cluster sites in Rakai, Uganda. Legend: Triangles Peer Health Workers with mHealth Support Clinics (mHealth+) Circles Peer Health Workers without mHealth Support Clinics (mHealth−), Thumbtack Central Clinic

Table 1.

Patient characteristics according to randomization arm

Characteristic	Subcharacteristic	PHW mHealth+ Arm	PHW mHealth− Arm
No clusters		4	6
No subjects total		446	524
No subjects per cluster, mean (range)		112 (74–163)	87 (47–132)
Female, n (%)		299 (67)	339 (65)
Age, median (range), years		34 (15–67)	36 (16–76)
CD4 cell count at ART initiation, median (IQR), cells/μl		156 (72–216)	167 (81–219)
Plasma HIV-1 RNA at ART initiation, geometric mean, copies/ml		44,833	44,083
Plasma HIV-1 RNA at ART initiation, mean (SD), log10 copies/ml		4.65 (0.99)	4.64 (0.87)
Baseline viral load > 100,000 copies/ml, n (%)		203 (46)	218 (42)
Baseline WHO stage, n (%)
	1	132 (30)	155 (30)
	2	165 (37)	184 (35)
	3	94 (21)	130 (25)
	4	54 (12)	55 (10)
Baseline ARV regimen, n (%)
	Combivir/Efavirenz	121 (27)	155 (30)
	Combivir/Nevirapine	162 (36)	191 (36)
	Stavudine/Lamivudine/Efavirenz	44 (9.9)	51 (9.7)
	Stavudine/Lamivudine/Nevirapine	115 (26)	124 (24)
	Other	4 (0.9)	3 (1)
Clinic distance to central clinic, mean (range), km		18.4 (7.7–32.5)	28.3 (12.9–40.5)
Subjects on ART prior to start of trial, n (%)		165 (37)	165 (31)
Subject pre-trial subject duration on ART, median (range), weeks		44.3 (1.0–89.1)	37.7 (1.1–89.4)
Pre-trial 24 week virologic failures, n/N (%)		45/93 (48.4)	40/98 (40.8)
Pre-trial 48 week virologic failures, n/N (%)		16/66 (24.2)	20/62 (32.3)

PHW Peer health worker, IQR Interquartile range, SD Standard deviation

Table 2 shows estimates of effect for all outcomes. All patients had at least one adherence measure recorded, and 698 (72%) had at least one viral load result performed with 320 (72%) and 378 (72%) from the mHealth+ and mHealth− Arms respectively. The primary outcome of any virologic failure was not significantly different between study arms (19.4% vs. 16.4%, mHealth+ vs. mHealth− Arm respectively), nor were any secondary outcomes. A Likert scale survey of 38 clinic staff found that most agreed strongly/agreed (89%) that “Mobile phones used by PHWs improved the overall care of the clinic patients” and agreed strongly/agreed (89%) that “All PHWs should be given mobile phones to use for patient care.”

Table 2.

Estimates of effect for patient virologic and non-virologic outcomes

Outcome	mHealth+ Arm		mHealth− Arm		mHealth+ vs mHealth−
	N	Outcome n (%)	N	Outcome n (%)	Estimate RR (95% CI)	P value
Any virologic failure	320	62 (19.4)	378	62 (16.4)	1.17 (0.84–1.64)	0.34
Virologic failure at 24 weeks	203	25 (12.3)	259	20 (7.7)	1.47 (0.83–2.61)	0.19
Virologic failure at 48 weeks	201	18 (9.0)	255	24 (9.4)	0.88 (0.47–1.65)	0.69
< 95% pill count adherence	401	2 (0.50)	473	10 (2.1)	1.01 (0.97–1.06)	0.59
< 100% pill count adherence	401	101 (25.2)	473	122 (25.8)	1.00 (0.88–1.14)	0.95
Self-report, any missed doses vs. never	411	68 (16.6)	487	90 (18.5)	0.89 (0.67–1.18)	0.42
Died	446	37 (8.3)	524	53 (10.1)	0.82 (0.55–1.22)	0.33
Lost to follow-up	446	11 (2.5)	524	10 (1.9)	1.29 (0.50–3.32)	0.60

PHW Peer health worker, CI Confidence interval, SD Standard deviation, RR Risk ratio

Qualitative Results

Several major themes emerged from qualitative analyses. An overarching theme was that health care communication was an interconnected web involving patients, PHWs, clinic staff, and family and friends (see Fig. 3) whereby two-way communication occurred via in-person conversations, text messaging, and voice calls. Conceptualizing this network, with its formal (PHW–Clinic Staff) and informal (Patient–Family) components helped illustrate pathways through which mobile phones expedited communication. As one patient illustratively said:

“I may have no money and I go to a friend. I might ask him to help me call the PHW because the PHW gave us their numbers. From that I will be able to explain the problems that I am going through. The PHW will call the health workers and they will come to attend to me.” (Patient, Focus Group).

Fig. 3.

Health care communication diagram

Impact of Voice Calls

Patients, PHWs, and staff repeatedly noted how voice calls on RHSP-provided mobile phones expedited patient care. As one PHW noted: “Now for me, a PHW with a phone, I can’t be like that PHW without a phone. His/her information cannot [travel] as fast as mine who has a phone.” (PHW, Focus Group). An anecdote from another PHW was illustrative:

“The phones have greatly helped us because when they give drugs [ART]… the patient kind of gets mad, funny dreams and starts developing nightmares. I have even found a patient tied up by ropes in the room. They called me at my home in the night at 2:00 am, they told me that the patient is tied…. The next day when it came to about 9:00 am when the health workers have reached the office I made a phone call. I told them that the patient at this number has run mad. The medical staff…called me and told me to first withdraw the patient from that type of drug and get him on Nevirapine.” (PHW, Focus Group).

PHWs and clinic staff frequently emphasized that phones improved PHW and clinic logistics. With a phone, requesting supplies required simply a “phone call to the head office informing health workers that you do not have such and such an item.” (PHW, Focus Group) PHWs with phones appeared to save significant amounts of travel time compared to those without phones as clinic sites were up to 40 km away from centralized staff: “I think that it has made it much easier for them because they can call in and communicate promptly. I know some PHWs would sometimes actually come up to Kalisizo [the central clinic] to discuss a situation with a patient. With mobile phones I think it is much easier.” (Clinic Staff, Interview).

Impact of Text Messaging

Some participants noted how the text messaging component of the intervention may have encouraged some patients to improve their adherence as patients realized the phones were being used to convey near real-time adherence information to clinic staff.

“…people have been motivated to take the drugs because they know that once you get to their home and you press the phone to send a message it means that you are reporting the patient that he has not taken the pills properly…. Patients are motivated now to take their pills.” (PHW, Interview)

However, reservations about text messaging were also raised. In contrast to voice calls which typically generated a rapid response, clinic staff had to first review PHW texts on a computer before responding. Staff did not appear able to always utilize or act promptly on these texts:

“We have been using the mobile phones for text messaging. The text messaging, we have real-time communication with the central office here, only that there is one challenge. We do not have enough manpower at all times to check on that, on the database, to take an immediate action.” (Clinic Staff, Interview)

Task shifting

The mHealth intervention seemed to facilitate task shifting. The use of mobile phones typically led to more Patient–PHW communication and less Patient-Staff. One staff member noted: “Instead of calling us, they first call the PHW. … If the PHW can solve it, they don’t bother to call us. If they cannot solve it, then the PHW calls us.” (Clinic Staff, Interview) One consequence of this task shifting was that clinic staff efforts may have shifted toward sites without the mHealth intervention:

“Where there were phones, we spend little time in those areas, and we shift more attention to areas where they don’t have phones. That can be a good improvement. Because where we have this communication we take little time where there are phones. We take a lot of time where there are no phones.” (Clinic Staff, Interview)

Impact on PHW Morale

An unexpected benefit of the intervention appeared to be improved PHW morale. PHWs frequently noted their personal satisfaction with the mHealth intervention and sense of empowerment. One PHW felt that “having a mobile phone has motivated us to visit the patients and do our work truthfully. You cannot send a message a patient had not taken his pills, yet you have not visited the patient.” (PHW, Interview). Another noted “the confidence that I developed after getting a mobile phone is rather too great. Previously, I did not have that confidence.” (PHW, Interview). Another PHW noted how his work had been enabled by the phones:

“Phones help us a lot. You may find a patient in a very critical state and you make a call to the health worker. Here the health worker might advise you to take the patient to a health unit. After getting to the health unit still you might be required to inform head office of the health unit you have sought treatment. Indeed mobile phones have made our work easy.” (PHW, Focus Group)

The mHealth intervention appeared to improve PHW capabilities and job satisfaction, as well as improve PHW-Staff working relationships: ‘It makes communication with RHSP staff faster. … Phones have cemented our relationship.’ (PHW, Focus Group)

Improved but Incomplete Phone Access

While the mHealth intervention was originally intended to primarily improve PHW–Staff communication, an important benefit was also improving PHW–Patient communication. However, patient access to phones varied. Most patients did not own phones themselves, though many leveraged phones in the communities. Patients borrowed phones from family and friends and also accessed phones through local kiosks: “You can go to the shop where there is a phone.” (Patient, Focus Group). However, call costs was a key factor limiting patient communication:

“There are some places you go to and they just ignore you. He/she can ask you whether you have the money so that you can call. You can just plead for his/her mercy by saying “Please help me and make that call. After selling my yellow banana, I will be able to pay you.” He/she can refuse because they want to first get paid.” (Patient, Interview)

These twin challenges in accessing phones—availability and cost—were overcome to a certain extent by the mHealth intervention. As one staff member noted:

“I was saying that you can find a whole village without a phone. So, giving these PHWs phones, is helping a lot…You know we are working with people who are very poor…. So someone, instead of someone going to a public payphone to pay for a call, they just wait for the clinic day. Because, you know what? No money. By giving these PHWs phones I think it helped them a lot because they just go and contact their PHW, and the PHW calls us.” (Clinic Staff, Interview)

Confidentially Concerns

While phones were accessible through friends, family, and shops, several patients raised privacy concerns when using others phones:

“Some people are not trustworthy in regard to maintaining confidentiality and when you are to make a call some information could be of secrecy. Suppose you are in a very poor health state. You are seriously ill. You could want to call the health workers when you go to a call box. You have to call in the presence of the public without privacy. You are not even sure whether the telephone attendant will not discuss your information with other people.” (Patient, Focus Group)

Patients who were fortunate enough to have a personal phone noted their privacy benefits: “If at all you want to ensure privacy then you have to use your personal phone, that is when people will not get to know what you are talking about.” (Patient, Focus Group) Despite these privacy concerns, most patients appeared to believe that it was worthwhile risk for the benefits of improved access to providers. As one patient noted: “I cannot fear using it. It is easy to call that person and save my life. I call the PHW who contacts the health workers and they come to help me.” (Patient, Focus Group).

Challenges with Phones

PHWs encountered some challenges with phone maintenance, primarily with keeping them charged. As one PHW noted: “We only encounter problems with charging…. Charging is the only challenge. Most of us come from areas without electricity. So we have to charge using [car] batteries.” (PHW, Focus Group) PHWs often took advantage of an informal network of local businesses which charged phones for a small cost. Another important challenge was that phones were occasionally targeted for theft, and a few PHWs reported stolen phones. RHSP staff were concerned that PHWs might not care for phones adequately if RHSP replaced these phones; therefore, PHWs were required to replace stolen phones themselves.

Discussion

This exploratory substudy trial of a mHealth support intervention used by PHWs did not demonstrate significant differences between study arms in virologic, adherence, mortality, or retention outcomes. Qualitative results demonstrated improved health communication and patient care, and found broad and deep support for the mHealth intervention. Challenges with the use of phones included variable patient phone access, privacy concerns, and phone maintenance.

Quantitative results from this study should be interpreted cautiously due to the small number of clusters, participants, and events for most study outcomes. The primary outcome of virologic failure occurred at a lower rate than anticipated and was not available for 28% of participants, resulting in this endpoint being further underpowered. Adherence outcomes were limited by the poor sensitivity of our adherence measures and programmatic challenges obtaining accurate adherence data [27].

Qualitative results found strong support from PHWs, patients, and clinic staff for the mHealth intervention. The clinic staff Likert scale survey results reinforced these qualitative findings. The intervention was felt to have improved the ability of PHWs to rapidly communicate patient findings from the field and for clinic staff to respond. While some felt text messaging improved PHW work quality and encouraged patient adherence, the main benefit of phones appeared to lie in simply making and receiving voice calls. After the trial was completed, the program decided to continue the mHealth support intervention, but eliminated the text messaging component because of unclear benefits and difficulty monitoring text messages.

While equipping PHWs with phones helped improve communication, the variable patient access to phones may have limited intervention benefits. A recent estimate of mobile phone penetration in Uganda was 39% and dominated by urban ownership [28]. Helping fill this gap in rural settings are “village phone” kiosks [29], which improved patient access but raised privacy concerns. However, costs remained a significant barrier to patient access and were also an important consideration for PHWs. While PHWs were able to call the clinic warmline toll-free and were provided a monthly call stipend, this stipend was modest. We previously reported early, rough costs of this intervention [19], and more rigorous cost analyses are planned.

Not surprisingly, in an environment with limited electricity, charging was a logistical challenge. However, this difficulty was mostly overcome by small businesses providing charging services, typically using adapted lead-acid battery systems, and through using long battery life phones made for LIMC. Phone theft was a mostly unavoidable risk but may have been minimized by program policies placing responsibility for replacement on PHWs which in turn led PHWs to handle their phones with great care. Reporting of mHealth implementation challenges and solutions is slowly growing, and their continued documentation should help inform future efforts [7].

There are several considerations for consolidating the quantitative and qualitative results. Some perceived discordances (e.g. high enthusiasm for the intervention in contrast to lack of significant quantitative benefits), may be due to the methods assessing different processes and outcomes rather than being truly discordant. For example, qualitative results showed improvements in access to care, while quantitative results focused on more downstream events such as virologic failure. Alternatively, the novel, “flashy” technology intervention may have been perceived more positively by participants despite not having quantifiable improvements [30]. Participants may have had personal motivations to convey a positive impression of the intervention. Study power issues could explain quantitative results as this study was exploratory and would have required a large detected effect size to reach significance. However, the lack of a direction of effect favoring the mHealth+ Arm for most outcomes is an indication that this intervention may not strongly affect these quantitative outcomes. Finally, patient care in the mHealth+ Arm may have become easier for clinic staff and resulted in a shift in their efforts toward the mHealth− Arm; this indirect contamination would have made it more difficult to ascertain quantitative intervention effects [22].

Additionally, the hotline and warmline numbers were available program-wide to all patients and PHWs. While initial mobile phone penetration was quite low among the PHWs (16% owned phones, 79% had previously used a phone), PHWs in the mHealth− Arm could still call clinic staff and occasionally did so with non-RHSP phones [23]. This process likely introduced some element of contamination in the comparison arm and reduced measurable differences. A final potential study limitation is limited generalisability as it was implemented in the setting of a mobile clinic care model affiliated with a long-stranding research program.

In summary, a mHealth support intervention used by PHWs was successfully deployed in a rural, LMIC setting. While no significant quantitative impact was found in this exploratory study, qualitative and staff survey results demonstrated substantial benefits and support for the intervention. Further studies of mHealth tools used by health workers which focus on health systems and patient-oriented outcomes are warranted.