Abstract
Background:
mHealth is a promising means of supporting adherence to treatment. The Start TB patients on ART and Retain on Treatment (START) study included real-time adherence support using short-text messaging service (SMS) text messaging and trained village health workers (VHWs). We describe the use and acceptability of mHealth by patients with HIV/tuberculosis and health care providers.
Methods:
Patients and treatment supporters received automated, coded medication and appointment reminders at their preferred time and frequency, using their own phones, and $3.70 in monthly airtime. Facility-based VHWs were trained to log patient information and text message preferences into a mobile application and were given a password-protected mobile phone and airtime to communicate with community-based VHWs. The use of mHealth tools was analyzed from process data over the study course. Acceptability was evaluated during monthly follow-up interviews with all participants and during qualitative interviews with a subset of 30 patients and 30 health care providers at intervention sites. Use and acceptability were contextualized by monthly adherence data.
Findings:
From April 2013 to August 2015, the automated SMS system successfully delivered 39,528 messages to 835 individuals, including 633 patients and 202 treatment supporters. Uptake of the SMS intervention was high, with 92.1% of 713 eligible patients choosing to receive SMS messages. Patient and provider interviews yielded insight into barriers and facilitators to mHealth utilization. The intervention improved the quality of health communication between patients, treatment supporters, and providers. HIV-related stigma and technical challenges were identified as potential barriers.
Conclusions:
The mHealth intervention for HIV/tuberculosis treatment support in Lesotho was found to be a low-tech, user-friendly intervention, which was acceptable to patients and health care providers.
Key Words: mobile health, mHealth, SMS, tuberculosis, HIV/AIDS, adherence
BACKGROUND
Mobile health (mHealth) is a promising means of improving adherence to medication and clinic appointments thereby potentially improving clinical outcomes.1 Mobile technology allows health care providers to communicate remotely with patients and more easily reach rural populations with poor or challenging transportation infrastructure.2 The number of mobile phone subscriptions in sub-Saharan Africa has been rising more rapidly than anywhere else in the world,3–5 allowing for innovative use of inexpensive mHealth technologies such as short-text messaging service (SMS). However, the results of recent studies using SMS reminders for antiretroviral therapy (ART) adherence from resource-limited settings have been mixed, with some studies showing an improvement6–9 and others showing no benefit.10,11 Recent meta-analyses using evidence from randomized controlled trials have demonstrated that weekly SMS is efficacious at improving self-reported ART adherence.12–14
The Start TB patients on ART and Retain on Treatment (START) study is an innovative mixed methods cluster-randomized trial that evaluated the effectiveness, cost-effectiveness, and acceptability of a combination intervention package (CIP) versus standard of care (SOC) to improve early ART initiation and retention during tuberculosis (TB) treatment, as well as TB treatment success, among patients with HIV/TB in Lesotho. Study methods have been described elsewhere,15 but in brief, 12 health facilities in the Berea district were randomized to receive CIP vs. SOC from April 2013 to August 2015. The CIP included a range of programmatic, structural, and psychosocial components, including real-time patient adherence support using mHealth technology and trained village health workers (VHWs). VHWs are community-based lay health workers who, subsequent to being trained under the Ministry of Health, provide essential health services at the community and household level, including health education, social welfare, and preventive health care services and the timely referral of individuals who are in need of facility-based medical care.
The aim of this article is to describe the use and acceptability of the mHealth component of the START study intervention at the 6 CIP health facilities, where the intervention was implemented.
METHODS
mHealth Intervention
The mHealth intervention comprised the design and delivery of a standardized, automated SMS system to provide real-time adherence support to patients on HIV and TB treatment. The study team developed a simple SMS reminder application using CommCare, an open source mobile platform developed by Dimagi (http://www.dimagi.com/), a Boston-based health informatics service provider that has supported many public health projects in developing countries. Tools available online were used to build an application to send appointment and medication SMS reminders to mobile phones based on a predefined algorithm. SMS messages were sent “one-way,” which meant that recipients could not respond.
In Lesotho, patients with HIV and TB routinely identify a treatment supporter, usually a family member, to assist with day-to-day treatment support. For this study, patients and treatment supporters were given a choice to have the SMS reminders sent to either one of them, or simultaneously to both. Two types of SMS reminders were sent at the recipients' preferred time of day: appointment reminders were sent 2 days and 1 day before patients' scheduled monthly clinic appointments and medication reminders were sent daily for the first 6 weeks of ART, and daily or weekly thereafter according to individual preferences. Patients and treatment supporters selected the time of day they wanted to receive appointment and medication reminders. Medication reminders used code words (“Did you eat your meal today?”) to protect patients' confidentiality. Patients were not issued study phones because mobile phone usage at study initiation was found to be relatively high (>85%); additionally, it was assumed that there was mobile phone reception throughout the study district. Patients with access to a working phone were provided with approximately $3.70 in monthly airtime to facilitate communication with their VHW or nurse in case of difficulties; incoming SMS messages did not incur a charge.
At each START CIP facility, 1–2 VHWs were selected to be facility-based VHWs and to coordinate the activities of VHWs providing community-based patient support. Facility-based VHWs were provided with password-protected mobile phones with the customized CommCare application and SMS text message capability, as well as monthly airtime to facilitate patient follow-up and support and to contact community-based VHWs. Nurses were also given monthly airtime to refer potential participants to research staff. Facility-based VHWs were trained to log patients' information and preferred message time and frequency in the mobile phone application. Data were transmitted to a secure cloud server in South Africa that received the parameters entered by the VHWs and sent outgoing messages in bulk to patients and treatment supporters (Fig. 1). CommCare provided access to patient data as well as data on outgoing messages on a secure Web site, which helped the study team troubleshoot emergent problems, such as network and registration issues. The study team used the CommCare reports module to monitor outgoing messages and to evaluate the implementation process. Facility-based VHWs provided detailed SMS instructions to patients and treatment supporters, including practice exercises, to ensure their comfort and familiarity with mobile technology.
Data Collection and Analysis
mHealth use and acceptability were assessed using mixed methods. mHealth use was quantitatively analyzed from study process data, drawn from a monthly Program Characteristics Survey to track CIP implementation, and an intervention receipt log for SMS messages and mobile airtime to document the “dosage”16 of mHealth that was received by study patients and treatment supporters at the CIP sites. mHealth acceptability was assessed using questions about intervention acceptability that were integrated into monthly follow-up interviews and qualitatively evaluated via in-depth interviews with a purposive, heterogeneous subset of patients and health care providers, 6–12 months post study initiation, at the CIP sites. The use and acceptability of mHealth tools were contextualized by adherence data, drawn from monthly follow-up interviews with 371 patients (measurement cohort) at CIP and SOC sites during the course of TB treatment (6–9 months).15
All monthly follow-up interviews were based on standardized questionnaires and administered face-to-face by trained research assistants. All qualitative interviews were based on a semi-structured guide, conducted face-to-face by separate trained staff not directly involved in patient care, and thematically analyzed using a grounded theory approach.17
Ethics and Consent Process
The study was reviewed and approved by the Columbia University Medical Center Institutional Review Board and the Lesotho National Health Research and Ethics Committee. Written informed consent was obtained from all participants completing study questionnaires and interviews. The study was registered at ClinicalTrials.gov (NCT01872390).
FINDINGS
Evaluation of mHealth Use
Over the study course of 29 months, the automated SMS system successfully delivered 39,528 messages to 835 individuals who consented to receive study SMS messages. Of 713 patients who were eligible for SMS reminders at CIP sites, 657 (92.1%) consented to receive the mHealth intervention, with no difference by gender. Most patients received SMS messages on their own (n = 455); 24 treatment supporters received SMS messages on behalf of their patients; and in 178 cases, patients and their treatment supporters received the same SMS message (Fig. 2). Ninety-six patients (15.2%) switched from daily to weekly medication reminders and did so on average 45.4 ± 39.1 days after enrollment; a small number of patients [n = 15 (15.6%)] subsequently switched back to daily medication reminders.
Evaluation of Intervention Acceptability
During monthly follow-up interviews, 171 measurement cohort participants in CIP sites were asked what had made it easier or helped them take their TB medications or ART; 41.9% stated the SMS messages were a facilitator to adherence in at least 1 monthly follow-up interview. No measurement cohort participant reported that the SMS messages posed a challenge to adherence in follow-up interviews.
Between February 2014 and January 2015, in-depth qualitative interviews were completed with 30 health care providers and 30 patients, approximately equally distributed across the CIP sites. Health care provider participants included 10 nurses and 20 VHWs with median 12 and 18 years of experience, respectively, providing services for patients and communities affected by HIV and/or TB. VHW participants included 7 facility-based VHWs and 13 community-based VHWs. Patient participants were on average 38.1 ± 9.7 years old, 43.3% were women, and 73.3% had a primary education or no formal education; these sociodemographic characteristics are representative of the populations served by participating health facilities (Table 1). Literacy, defined by the ability to read a full sentence, was found in 83.3% of patient participants. Most patient participants (83.3%) reported owning a mobile phone; 30.0% reported electricity in the house.
TABLE 1.
Patient Acceptability
Patient participants were appreciative of the mHealth intervention. They perceived the appointment and medication reminders were complementary cues to adherence, alongside visits from VHWs, and appreciated being able to self-select the time and frequency of messages. Most patient participants also understood the coded text messages were a form of private health communication.
“It [ie, SMS] helps me a lot because you may find that it is time when I get back home from work… and by then I am tired, and I will just hear ringing phone, then when I take the phone to read I find out that it is the message that reminds me.” (male participant, age 35 years)
“I think the one that remind me to take my medication being the first class [ie, excellent], I don't make any mistake with the medication I fear and I can't miss them at all.” (male participant age 29 years)
“They don't cause any problems because each and every one has his [own] phone and is private.” (female participant, age 26 years)
Patient participants were very appreciative of the airtime provided by the study. Phone calls, above and beyond SMS messages, empowered them to communicate with their health care providers and treatment supporters in a timely manner, without incurring a personal cost. Patients said they more frequently called their provider to report a side effect, seek advice, or inform their clinic about potential delays to a clinic appointment. They also felt more inclined to call and request assistance from their treatment supporters.
“We are given airtime that we use it to communicate with our supporters if we have a problem.” (female participant, age 31 years)
However, several participants reported routinely taking medications on their own with the aid of a phone alarm or were sufficiently reminded by their relatives, treatment supporter, or VHW. Although they appreciated receiving reminders for times when they were preoccupied or fatigued, they perceived the SMS service to be less necessary.
“My remembrance is that because I have a paper which I constantly mark in the morning and evening. It is the one which gives me understanding that in the morning when I have taken some, I mark… there was no way I would say I have forgotten to take them… I have a programme of how to go accordingly.” (male participant, age 28 years)
“You know these ones of TB I am still reminded by this person who is close to me, who is my wife, the one who comes and wakes me and also do things to help me. These ones of the afternoon which are the ARVs I am still able to, even [despite] the SMS enters to remind me.” (male participant, age 38 years)
A few patient participants did not use or own a phone. They had registered their treatment supporter to receive SMS reminders, who in turn, reminded them to take medications. As a result, they did not fully understand or appreciate the utility of the SMS service.
“This little lady [ie, my daughter] is the one who supports me. She is the one who would be telling me that, “Hey, it's time”… She has a phone… I don't have a phone myself… I have not received them [ie, SMS] personally.” (female participant, age 56 years)
Patients who were generally less familiar with mobile phones were at times confused by the coded messages. Nonetheless, they managed to link the SMS messages to medication intake.
“It [ie, SMS] says “have you eaten supper” it reminds me that I have to eat before I take medication.” (male participant, age 35 years)
Unstable access to electricity and temporary technical difficulties appeared to bar a few patient participants from receiving regular SMS reminders.
“This phone has issue because at home we don't have electricity it keeps giving us trouble as to where we charge and what to do like that.” (female participant, age 52 years)
“There was a time when they [ie, SMS] were not coming through, but when I arrived here, I came to report it here and they said, there was a bit of a challenge but it is all well.” (male participant, age 53 years)
Health Care Provider Acceptability
Health care provider participants reported receiving sufficient training and technical support to partake in phone-related study activities. They expressed support for the mHealth component of the intervention as it facilitated communication between patients, treatment supporters, and the various cadres of providers engaged in the CIP.
Facility-based VHWs appreciated the flexibility of being able to use study-issued phones for text messages as well as phone calls. Nurses and facility-based VHWs reported patients expressed feeling more cared for when they were followed up more frequently. Often, patients preferred to talk by phone rather than type a text message. Text messages and airtime were thereby understood to strengthen the patient–provider bond.
“Actually, the importance and goodness of airtime is communication, because, at times I left the patient not feeling well, so maybe when I'm at home I will call to ask how she is feeling and she answers. That is the goodness of airtime. Even when it comes to the patients, I find it very good because it encourages them, showing them that they are cared for, they are assisted with health. Even if they have nothing, they should have airtime.” (VHW)
Community-based VHWs appreciated being able to monitor patients over phone, as they faced substantial geographic barriers and unpredictable weather when following patients living in disparate, remote rural areas. Some VHWs would call patients nearly every day to remind them to take daily doses and to ensure that they made it to their next clinic visit with the appropriate tools in hand such as sputum samples. This helped to avoid delays at the facility, capture and resolve potential reasons for missing a visit, and prevent nonadherence. Routine communication helped them triage patients for emerging medical complications or adverse drug effects more efficiently and support patients through difficult periods in their treatment in a more meaningful way. VHWs were also more easily able to track and follow-up with patients at risk for nonadherence or treatment interruption.
“The use of SMS is very important… suppose it rains heavily and I am unable to attend him, I text him and say, ‘It is your time now. Have you remembered your food?’ He already knows. I will have taught him that when I say that, I mean it's time to take his pills. So it is very helpful this SMS thing, it helps us meet our patients.” (VHW)
VHWs were sensitive to patients' experiences with HIV-related stigma and supported the use of simple, coded text messages that protected patients' privacy and confidentiality. However, they worried that study calls could disrupt dynamics within households where patients had not disclosed their HIV status. In consultation with such patients, VHWs developed innovative, patient-acceptable strategies to protect against inadvertent breaches in confidentiality.
“It is very important, … the way it [ie, SMS] is written. If it says, have you taken medication… [if] you find that somone's phone is in the wrong hands, then they get to know the patient's issues too soon.” (VHW)
“The case of Mrs. [name], when she was with the in-laws, her case was her secret. I would call, make her a callback. She was familiar with my number and if she did not respond to it I would know that the clouds have covered [ie, there are people around]… But if I make her a callback and she does the same, I knew that it’s clear and so we can talk.” (VHW)
Akin to reports from patient participants, health care providers also experienced technical challenges with the SMS service; most were quickly resolved after discussions with study staff or affected patients.
“When I was initially taught it was a bit of a struggle to understand as quick. But I ultimately got it even though a phone at times can give you problems here and there, and you find that here it gives a delivery report and at the head office where it is supposed to be sent to they say it does not arrive but when we fix that problem again you find that it goes through smoothly. For others you find that the patient has given you a certain number, in a blink of an eye he has changed it without telling you that he doesn't use that number anymore.” (VHW)
Evaluation of Adherence
Data from monthly adherence assessment interviews among measurement cohort participants found that self-reported 30-day adherence to 100% of ART were 86.3% in the CIP versus 80.7% in SOC, and to TB medications was 89.1% in CIP versus 79.5% in SOC (Table 2); adherence did not vary by gender. Among patient participants in the qualitative evaluation (N = 30), the proportion reporting 100% adherence to ART and TB medications was higher, at 90.0% and 93.3%, respectively.
TABLE 2.
DISCUSSION
In this mixed-methods implementation science study evaluating the use and acceptability of a CIP to improve early ART initiation and retention, and TB treatment success among HIV/TB patients in Lesotho, the mHealth component of the intervention package was considered beneficial by both patients and health care providers. Intervention delivery process data highlighted very high uptake of the SMS intervention among both male and female patients. The perceived importance of SMS messages in supporting adherence also arose during patients' monthly follow-up interviews and in-depth qualitative interviews.
Qualitative interviews revealed many overlaps between the viewpoints of patient and health care provider participants. Drivers of mHealth use included: cues to adherence, for medication intake and clinic appointments; access to airtime, via the issuance of vouchers; ability to place outgoing calls as well as to send SMS messages; personal choice in selecting the time and frequency of incoming SMS messages; and the use of private SMS messages (Fig. 3). These factors were perceived to enhance the quality and frequency of communication between patients and providers and between VHWs and nurses, with positive impacts on patient–provider interactions, provider workload, and quality of care. The interviews also allowed us to identify 2 groups of patients among whom the mHealth component of the CIP was of less perceived value or less appreciated. This included patients with existing adherence supports who may have had less need for additional cues and patients who had not disclosed their HIV status to household members and were thus less comfortable reading text messages at home or in public places. Technical challenges and confusion related to general phone use may have also impeded optimal utilization of the SMS service.
The optimal frequency of providing SMS messages to patients has not been clear. In early 2012 when this study was designed, the only 2 SMS trials published were from Lester et al (2010),6 who tested a weekly SMS message and found it effective, and Pop-Eleches et al (2011),8 who compared weekly and daily SMS messages and found that weekly messages had a positive effect on adherence, whereas daily messages did not have an effect on adherence. In the START study, we chose to provide daily messages for the first 6 weeks of HIV/TB cotreatment and then to allow patients choose whether to continue daily messages or switch to weekly messages. It is interesting to note that only 15.2% of patients elected to switch from daily to weekly medication reminders and a percentage of those (15.6%) subsequently switched back to daily reminders.
Overall, delivery of the mHealth intervention was found to be inexpensive, as one server can provide SMS messages to thousands of patients in a large geographic area with very few human resources needed beyond the initial setup. The application was developed using an open source mobile platform. During 29 months of study implementation, the costs for troubleshooting the application before intervention launch and providing SMS services to 835 individuals totaled $3124; study-issued airtime vouchers totaled $22–33 per patient or treatment supporter, depending on the length of cotreatment. During dissemination of study results, the Lesotho Ministry of Health (MOH) indicated an interest in rolling out implementation of the SMS intervention in a programmatic setting; however, they did not commit to the provision of airtime.
A 2011 systematic review of ART adherence in 20 countries found the average rate of reporting ≥90% ART adherence is 62%.18 This level of adherence in HIV monoinfected individuals is significantly lower than what was seen in our study, where 86.3% and 89.1% of HIV/TB co-infected patients in the CIP arm reported adherence levels of 100% to HIV and TB medications, respectively; a trend of higher adherence was noted in the CIP arm versus the SOC arm. The study design precludes evaluation of the effectiveness of individual components of the CIP; however, process data and qualitative analyses have highlighted patient and health care provider perspectives on the utility and acceptability of the mHealth intervention.
There are several implications to be drawn from our findings. First, mHealth interventions are more likely to be successful if they allow for flexibility in SMS/call capability, frequency, and timing and are accompanied with training and technical support, routine patient instruction (eg, on phone use and coded messages), and consideration of regional infrastructure (eg, network, electricity). Although airtime provision may increase intervention uptake and is likely to enhance acceptability, it is more costly and its adoption would depend on local feasibility and resources. Second, the potential challenge of HIV stigma should be recognized and addressed in context-specific ways as HIV nondisclosure, an established proxy for stigma,19 may interfere with mHealth uptake. Third, although SMS technology is relatively inexpensive, in highly resource-constrained settings, it may be reserved for those patients who lack other sources of social and adherence support.
The study has several limitations. Unlike other studies that either provided a mobile phone8,20 or required access to one,6,9,21 phone ownership was not an inclusion criterion for the CIP. Patients without mobile phones may be different from those who have access to phones, thus limiting generalizability of our findings. However, given our implementation science framework and focus on sustainability, we did not find it feasible to issue study phones in this resource-limited setting. Another limitation is in the lack of precision in measuring dosage of the mHealth intervention. Similar to other mHealth studies,8 participants in the CIP were not asked to respond to SMS messages. One-way messages precluded us from being certain that messages were received and read by the individual patient. Adherence may have been overreported in monthly follow-up interviews due to social desirability bias. To protect patient confidentiality, we did not link patients' phone numbers and study IDs, and thus we could not compare adherence levels of patients who received daily versus weekly adherence reminders. We were also unable to interview treatment supporters and learn from their unique experiences as recipients of the SMS intervention.
The study has several strengths. First, this is a mixed-methods implementation science trial. As such, we carefully measured intervention implementation processes as well as dosage of the intervention received. Additionally, we integrated mixed methods,22,23 which allowed us to assess barriers and facilitators for medication adherence and to evaluate the acceptability and utilization of the SMS intervention. Second, the participating health facilities have adequate heterogeneity, covering both urban and rural locations, which strengthens the external validity of findings. Third, more than half of the study sample were men, a group that is often hard to reach and support, and thus our findings can inform adherence and retention strategies targeting this priority population to achieve epidemic control. Finally, having the support and engagement of the MOH in the design and implementation phases of this study has fostered MOH ownership at the national and district levels and will help to ensure sustainability of the mHealth intervention.
CONCLUSION
The mHealth intervention for HIV/TB treatment support in Lesotho was found to be a low-technology, user-friendly intervention, which was acceptable to patients and health care providers.
ACKNOWLEDGMENTS
The authors thank Wafaa El-Sadr, Blanche Pitt, Angela Campbell, Mary-Elizabeth Vachon, Mashale Shale, and Limakatso Lebelo for their contributions to study design and implementation. They are indebted to the study participants for their time and willingness to contribute to the understanding of service delivery for patients with HIV-infected TB. They thank staff at the study sites, village health workers in the surrounding communities, the Berea District Health Management Team, and the Lesotho Ministry of Health for their invaluable assistance in conducting this study.
Footnotes
Supported by the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) through United States Agency for International Development (USAID) under the terms of Award Number USAID-OAA-A-12-00022. The content is solely the responsibility of the authors and does not necessarily represent the official views of the U.S. Government.
The authors have no conflicts of interest to disclose.
REFERENCES
- 1.Anglada-Martinez H, Riu-Viladoms G, Martin-Conde M, et al. Does mHealth increase adherence to medication? Results of a systematic review. Int J Clin Pract. 2015;69:9–32. [DOI] [PubMed] [Google Scholar]
- 2.Lewis T, Synowiec C, Lagomarsino G, et al. E-health in low- and middle-income countries: findings from the Center for Health Market Innovations. Bull World Health Organ. 2012;90:332–340. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sub-Saharan Africa Ericsson Mobility Report Appendix. June 2014. Available at: http://www.ericsson.com/res/docs/2014/emr-june2014-regional-appendices-ssa.pdf. Accessed October 14, 2014. [Google Scholar]
- 4.Pew Research Center. Cell Phones in Africa: Communication Lifeline. Available at: http://www.pewglobal.org/2015/04/15/cell-phones-in-africa-communication-lifeline/. Accessed April 15, 2015. [Google Scholar]
- 5.Clouse K, Schwartz SR, Van Rie A, et al. High mobile phone ownership, but low Internet and email usage among pregnant, HIV-infected women attending antenatal care in Johannesburg. J Telemed Telecare. 2015;21:104–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lester RT, Ritvo P, Mills EJ, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet. 2010;376:1838–1845. [DOI] [PubMed] [Google Scholar]
- 7.Maduka O, Tobin-West CI. Adherence counseling and reminder text messages improve uptake of antiretroviral therapy in a tertiary hospital in Nigeria. Niger J Clin Pract. 2013;16:302–308. [DOI] [PubMed] [Google Scholar]
- 8.Pop-Eleches C, Thirumurthy H, Habyarimana JP, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS. 2011;25:825–834. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.da Costa TM, Barbosa BJ, Gomes e Costa DA, et al. Results of a randomized controlled trial to assess the effects of a mobile SMS-based intervention on treatment adherence in HIV/AIDS-infected Brazilian women and impressions and satisfaction with respect to incoming messages. Int J Med Inform. 2012;81:257–269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Mbuagbaw L, Thabane L, Ongolo-Zogo P, et al. The Cameroon Mobile Phone SMS (CAMPS) trial: a randomized trial of text messaging versus usual care for adherence to antiretroviral therapy. PLoS One. 2012;7:e46909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shet A, De Costa A, Kumarasamy N, et al. Effect of mobile telephone reminders on treatment outcome in HIV: evidence from a randomised controlled trial in India. BMJ. 2014;349:g5978. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Finitsis DJ, Pellowski JA, Johnson BT. Text message intervention designs to promote adherence to antiretroviral therapy (ART): a meta-analysis of randomized controlled trials. PLoS One. 2014;9:e88166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Mbuagbaw L, van der Kop ML, Lester RT, et al. Mobile phone text messages for improving adherence to antiretroviral therapy (ART): a protocol for an individual patient data meta-analysis of randomised trials. BMJ Open. 2013;3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mills EJ, Lester R, Thorlund K, et al. Interventions to promote adherence to antiretroviral therapy in Africa: a network meta-analysis. Lancet HIV. 2014;1:e104–e111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Howard AA, Hirsch-Moverman Y, Frederix K, et al. The START Study to evaluate the effectiveness of a combination intervention package to enhance antiretroviral therapy uptake and retention during TB treatment among TB/HIV patients in Lesotho: rationale and design of a mixed-methods, cluster-randomized trial. Glob Health Action. 2016;9:31543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–350. [DOI] [PubMed] [Google Scholar]
- 17.Corbin JM, Strauss A. Grounded theory research: procedures, canons, and evaluative criteria. Qual Sociol. 1990;13:3–21. [Google Scholar]
- 18.Ortego C, Huedo-Medina TB, Llorca J, et al. Adherence to highly active antiretroviral therapy (HAART): a meta-analysis. AIDS Behav. 2011;15:1381–1396. [DOI] [PubMed] [Google Scholar]
- 19.USAID. Can We Measure HIV/AIDS-Related Stigma and Discrimination? Current Knowledge About Quantifying Stigma in Developing Countries. Available at: https://www.icrw.org/files/publications/Can-We-Measure-HIV-Stigma-and-Discrimination.pdf. Accessed January 15, 2016. [Google Scholar]
- 20.Hardy H, Kumar V, Doros G, et al. Randomized controlled trial of a personalized cellular phone reminder system to enhance adherence to antiretroviral therapy. AIDS patient care and STDs. 2011;25:153–161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Haberer JE, Musiimenta A, Atukunda EC, et al. Short message service (SMS) reminders and real-time adherence monitoring improve antiretroviral therapy adherence in rural Uganda. AIDS. 2016;30:1295–1300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Creswell JW, Plano-Clark VL. Designing and Conducting Mixed Methods Research. Los Angeles, CA: Sage Publications; 2011. [Google Scholar]
- 23.Greene JC, Caracelli VJ, Graham WF. Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Pol Anal. 1989;11:255–274. [Google Scholar]