Patient-centered mobile tuberculosis treatment support tools (TB-TSTs) to improve treatment adherence: A pilot randomized controlled trial exploring feasibility, acceptability and refinement needs

Summary

Background

Digital adherence technologies hold promise to improve patient-centered tuberculosis (TB) monitoring, yet few studies have incorporated direct adherence monitoring or assessed patients’ experiences with these technologies. We explored acceptability, feasibility, and refinement needs of the TB Treatment Support Tools (TB-TSTs) intervention linking a mobile app, a urine drug metabolite test, and interactive communication with a treatment supporter.

Methods

This pilot study was a parallel-designed single-center randomized controlled trial with exit interviews. Newly diagnosed TB patients from a respiratory medicine hospital in the province of Buenos Aires, Argentina were randomized 1:1 using a treatment allocation button in the REDCap software preloaded with a random allocation sequence to usual care or usual care plus the TB-TSTs intervention and followed for 6-months. Due to the nature of the intervention, blinding to the group allocation could not be achieved for the recruiter or patients. The treatment outcome data extractor was blinded to the group allocation of the participants. Intervention participants used the app to report self-administering medication, potential side effects, submit photos of the urine test, and interact with a treatment supporter. Outcomes were feasibility, acceptability, and treatment outcomes.

Findings

Forty-two patients were enrolled and evenly assigned to each group. Intervention participants submitted 147·2±58 (mean, SD) medication self-administration and 144·5±55 side effect reports out of 180 and 47.5±38·4 photos of the urine test out of 77. Treatment success for usual care was 81% [17/21] and 95% [20/21] for the TB-TSTs intervention. Thirty-three themes were identified from the interviews within the main categories of motivation, what worked, issues experienced, and recommendations. Participants (n=12) rated it as ‘easy to use’ (4.57/5), ‘would highly recommend to others’ (4·43/5) and reported that access to the treatment support was a critical component. Recommendations included adding an alarm, appointment reminders, and off-line functionality.

Interpretation

Findings suggest that the TB-TSTs intervention was feasible and acceptable and further refinement and testing is warranted.

Funding

National Institute of Health K23NR017210. ClinicalTrials.gov Identifier: NCT03544476.

Research in context.

Evidence before this study

We searched PubMed and Web of Science from 1/1/2010 to 12/1/2021 for digital adherence technology of TB monitoring in low- and middle-income countries using key terms, “digital health” OR “mHealth” OR “Digital adherence technologies” AND “TB” OR “TB.” We referenced Subbaraman et al (2018) who described the landscape and research priorities of digital adherence technologies (DATs) for the management of TB. A sytematic review of the impact on digital technolgies on TB by Ngwatu et al (2018) found four trials assessing SMS; two observational study assessing video observed therapy; and one observational study and one trial assessing medication monitors. From these studies they concluded that the evidence of digital technologies to improve TB remained limited. In a scoping review of digital technologies by Lee et al (2020) 145 relevant studies were identified of which 107 targeted health care providers and only 20 studies targeted clients. Similarly, a descriptive review of mobile health apps to improve TB treatment by Keutzer et al (2020) idenified an increasing number of TB apps since prior TB app reviews, however, the majority continue to target health care providers or TB information and few targeted patients. DiStefano et al (2016) described direct metabolite testing as an accurate and ethical way to monitor treatment adherence, yet few studies have included them.

Added value of this study

Our study reports on a real-world assessment and patient feedback of digital adherence technologies – the TB Treatment Support Tools (TB-TSTs) – for active TB treatment in a high TB burden region of Argentina. We conducted a pilot randomized controlled trial to (1) gather preliminary data as to whether the TB-TSTs was feasible and acceptable, (2) determine if the intervention use shows promise in improving treatment outcomes (treatment success), and (3) determine the need for refinement for next iteration of the tools. While the number of available TB apps has increased, the majority do not provide direct patient support, patient engagement in self-management of their care, or direct adherence monitoring. In addition, few studies include participants in the refinement or understanding users experience and mainly focus on technology alone. Moreover, although alternative metrics that include biological tests of drug ingestion such as urine testing have been highlighted for their potential, few studies have assessed home-based direct metabolite testing. DATs hold promise to support patient-centered monitoring, yet few studies have incorporated direct adherence monitoring or assessed patients’ experiences with these technologies.

Implications of all the available evidence

Evidence of DATs to address patient and health system challenges is growing and the COVID-19 pandemic has hastened the need to transition to alternatives remote monitoring and services and may become permanent, post-pandemic solutions. Based on findings the TB-TSTs appears to be feasible and acceptable and shows promise to improve TB treatment outcomes. Findings from this study were used to refine the TB-TSTs which is currently being evaluated in an adequately powered randomized controlled trial with approximately 400 participants in four public hospitals in Argentina.

Alt-text: Unlabelled box

Introduction

Tuberculosis (TB) remains an urgent global health threat and a leading cause of death worldwide despite it being a treatable and preventable disease. Globally, over 10 million people have active disease and approximately 1·5 million die from TB each year.¹ A contributing factor to the spread of disease and death is treatment non-adherence which is multifactorial, complex, costly and a major obstacle to TB control. Non-adherence reduces cure rates, leads to more severe disease, prolongs infectiousness and economic hardship, and contributes to the emergence of drug resistant strains of TB.² Known TB treatment adherence barriers include long course of treatment (minimum of 6 months), medication side effects, stigma, income loss, poor clinical understanding of the disease and its treatment, lack of support during treatment, and healthcare systems barriers (e.g., stockouts of drugs and supplies, poor coordination of care).^3,4 Health care systems are burdened by the volume of patients, the HIV epidemic, lack of resources, and the lack of advanced monitoring options for tracking and returning patients to treatment.^5,6 Therefore, there is an urgent need for more economical, efficient, and patient-centered alternatives to ensure treatment adherence.⁷

As a target to end the TB epidemic, the Sustainable Development Goals and the WHO End TB Strategy recommend interventions that place patients and communities at the forefront of the response, heighten their involvement in their own care, improve communication with providers, and promote a collaborative care approach.⁸ Although interest in digital adherence technologies (DATs) to address patient and health system challenges is growing,^7,9,10 the COVID-19 pandemic has hastened the need to transition to such alternatives for TB services, along with other health services, and may become permanent, post-pandemic solutions.^11,12 Given the high access to mobile phones globally and rapidly increasing smartphones use, mobile health technologies hold promise to address some of the treatment adherence barriers. Mobile health applications (apps), for example, have increased sophistication capable of accommodating multiple tools (e.g., automated reminders, symptom tracking) to improve monitoring, communication, and individual tailoring.^7,13 To date, few TB related apps target patients and none support patient engagement in self-management of their care or direct adherence monitoring.^14,15 DATs hold promise to support patient-centered monitoring, yet few studies have incorporated direct adherence monitoring or assessed patients’ experiences with these technologies. To address the unmet need for supportive treatment monitoring strategies, we converted and expanded a previously developed texting intervention, TextTB, into a mobile optimized app with innovative direct adherence monitoring – the TB Treatment Support Tools (TB-TSTs).

The objectives of this pilot randomized controlled trial were to (1) gather preliminary data as to whether the TB-TSTs was feasible and acceptable, (2) determine if the intervention use shows promise in improving treatment outcomes (i.e., treatment success and cure), and (3) determine refinement needs for next iteration of the intervention.

Methods

We conducted a sequential mixed-methods study using a 2-arm parallel pilot randomized controlled trial (RCT). After the pilot's completion, participant feedback was gathered by exit questionnaire or in-person interviews, and the data captured within the application was analyzed for trends. Participants were recruited from Hospital Cetrangolo, a hospital specialized in respiratory medicine in the Province of Buenos Aires, Argentina.

Ethics approval was obtained from the Institutional Review Board at the University of Washington and the ethics committee of the research site. There were no deviations to methods or outcomes after trial commencement. All participants provided informed consent in person prior to participating in the research activities. The trial was registered in ClinicalTrials.gov (ClinicalTrials.gov Identifier: NCT03544476). The trial and interventions are described according with the CONSORT-EHEALTH guidelines.¹⁶

Eligibility criteria

Patients were eligible if they were 18 years of age or older, starting TB treatment for pulmonary TB, with no known TB drug resistance, owned or had regular access to a smartphone, and were able to operate the mobile phone to communicate or have someone else in the household able to assist. The case definition included TB confirmed by positive results on sputum smear test or the diagnosis of pulmonary TB based on radiological findings, clinical signs and symptoms but with negative results on sputum smear test. The diagnosis could be confirmed by other methods, such as nucleic acid amplification (polymerase chain reaction) or enzyme-linked immunosorbent assay. Exclusion criteria were if the patient was severely ill (i.e., requiring hospitalization), resided in the same household with another study participant, or had known drug resistance (differing treatment regimen and duration). HIV coinfection was not included in the exclusion criteria. A recruitment log was maintained to document screened patients and reasons for declining participation. Recruitment was performed between April 2019 and July 2019, and the follow-up continued until June 2020.

Sample size

The power calculation was based on the outcome of treatment success. To detect a 15% increase in treatment success with 80% power and an α of 0·05 (two-tailed) a total sample size of 348 was required in 2 arms. Based on recommended sample size calculations for pilot RCT,¹⁷ to determine if the intervention should be tested further, 9% of the sample size of an adequately powered trial is needed; for this pilot study, a minimum of 35 participants (17 per arm). To be cautious and account for an estimated 20% attrition, the recruitment target was 42 participants.

Randomization and masking

Participants were randomized to either usual care or usual care plus the TB-TSTs intervention at a ratio of 1:1 in blocks of 10. A random allocation sequence was generated using http://www.randomization.com/ and uploaded to REDCap software for treatment allocation. Due to the nature of the intervention, blinding to the group allocation could not be achieved for the recruiter or patients. Participants were aware that the intervention of interest was using the app and test strips. Clinicians were not made aware of the group allocation unless their patient informed them. The treatment outcome data extractor was blinded to the group allocation of the participants.

Interventions

In the Argentinean public healthcare system usual care for TB treatment is provided free of charge and includes medication, routine clinical care and laboratory tests. In general, patients receive a 1-month supply of medication and are asked to self-administer treatment and return monthly for follow-up appointments. Patients may return earlier if they are experiencing issues, but no supervision takes place between visits. Standard guidelines are followed in the treatment of drug susceptible TB. Currently, four-drug fixed-dose combination pills (isoniazid 75 mg, rifampin 150 mg, ethambutol 275 mg, and pyrazinamide 400 mg) are commonly provided with either three or all four of the medications in a combined pill that includes isoniazid. If using fixed-dose combined pills the daily pill burden can range from 3-4 tablets compared to an average of 10 tablets per day if provided as single-medication pills.¹⁸

TB-TSTs includes a patient and treatment supporter facing mobile app (version 1.1) and a direct drug metabolite test. Screenshots of the patient and provider facing apps used in the pilot study, a list of contributors to the code development, and links to codebase are archived in publicly available webpage (https://tb-treatment-support-tools.github.io/pilot-artifacts/). The TB-TSTs was iteratively developed by converting and expanding TextTB, a texting-based intervention to support patients with active TB, into a mobile app with additional features.19, 20, 21 The app allows patients to report self-administration of their TB medication, track potential medication side-effects, and upload a photo of the urine test to verify their adherence. Additional features include access to accurate information about TB, a calendar view of their treatment progress and the ability to communicate with a Treatment Supporter or anonymously with other patients in a group discussion forum.

The paper-based test strips are derived from a previously established isoniazid detection protocol developed by the Arkansas Department of Health.²² The presence of the Isoniazid metabolite in the urine is approximately 2 hours after digestion of the drug and remains detectable in the patient's urine for approximately 24 hours.²³ If the drug metabolite is present the test turns a blue-purple color within 20 minutes. With higher concentrations of the metabolite present in the sample, the reaction occurs within a few minutes. Sensitivity of this test has been found to be over 97% and specificity 98% in patient samples.²² Color variability due to the presence of other pyridine compounds within the urine such as nicotine has been previously documented but does not interfere with the interpretation of the result as it produces a different colorimetric dye.²² The test strips used in this study were produced in-house by the research team and reengineered for home use, user accessibility, and robustness. The changes made to the test included added wick to allow absorption of more urine, and modifications to the strip enclosure. Prior iterations of the Arkansas test reported successful detection of INH metabolite in patient urine at concentrations as low as 5ug/ml within 30 minutes of color development in paper-based formats. Our reengineered test shows faster development time with successful identification of samples as low as 1ug/ml within 20 minutes of color development in similar lab-based settings. There are no commercially available INH test strips and prior tests were noted as unaffordable for most developing countries (e.g., $6.40US per test).²⁴ The estimated cost for this test was less than $1.

Procedures

Participants randomized to the intervention arm received assistance downloading the app, as well as written and verbal instructions for app use and completing the at-home urine test. All participants were given a one-on-one demonstration of how to use the app to submit reports and complete the urine test. Participants were asked to report daily self-administration of their medication using the app and to complete the urine test during weekdays when a Treatment Supporter was available (3 days per week based on holiday or work schedule variations). Our goal was to evaluate the combination of a daily indirect and intermittent direct adherence monitoring and to assess the ideal frequency of urine testing throughout treatment. We determined that daily urine testing was likely not realistic or necessary for early identification of adherence challenges. The treatment supporter was a local nurse from the TB program with expertise in TB treatment protocols and the healthcare system. The Treatment Supporter used the treatment allocation button in the REDCap software for group assignment, trained participants to use the app, monitored in app submissions, and followed-up with participants as needed (e.g., if a participant reported an issue, missed reporting). There were no automatic prompts or reminders in the app. The treatment supporter was trained on the use of the patient and provider facing apps, on the interpretation and input of test results, and on research protocols by the primary investigator and the research team. After the urine test results were entered by the treatment supporter, the participant could view the results in their app account. A TB expert/pulmonologist from the TB program was available to the treatment supporter for guidance with technical or challenging participant issues. Participants were informed that interaction with the treatment supporter through the intervention was available within a clinic-based system and available only during office hours (Monday-Friday) and that emergencies must be directed through standard routes. All participants received standard instructions on TB and its treatment and available national TB program educational material and were followed for the 6-month treatment course.

Outcomes and data collection

The outcomes were feasibility, acceptability, and determining if the intervention showed promise in improving treatment outcomes to recommend further testing. Treatment outcomes were measured using standard definitions set by the WHO Standards of TB treatment.²⁵ Treatment success is defined as either completion of medication (without bacteriological confirmation) or cured (negative sputum smear at 6 months and at least once prior to 6 months). Other treatment outcomes are: died, defaulted (treatment interruption for ≥ 2 months), or transferred out (transferred to another reporting unit and treatment outcome is unknown). Medical records or national registry were reviewed to collect treatment outcomes, including sputum sample results if collected.

We used REDCap to administer the baseline survey which included standard demographics and the Global Health Patient-Reported Outcomes Measurement Information System (PROMIS) short form – a 10-item questionnaire to assess an individual's physical, mental, and social health and is available in the Spanish language.

To assess feasibility and acceptability (perceived usefulness and ease of use), we assessed app usage (e.g., reports and test strip photos submitted) and invited all intervention participants to complete an exit interview with a minimum target of 10 participants. A semi-structured interview guide incorporated open ended questions to understand participants experiences of what worked and didn't work, their motivations, recommendations, and two Likert Scale questions (see Supplementary material). Interviews were conducted by the treatment supporter. Participants were also sent the exit interview questions by phone survey as an alternative to provide feedback. The primary investigator (SI) conducted an interview with the treatment supporter and the hospital TB director. The treatment supporter also wrote field notes and a summary of his experiences using the intervention and interacting with patients.

Analysis

We used Fisher's exact test and two-sample test of proportions with Stata Statistical Software (version 17·0) to evaluate possible differences between groups in sociodemographic characteristics and treatment outcomes. Descriptive statistics were used to assess app usage data. For treatment outcomes, analyses were based on intention-to-treat.

Interviews were transcribed verbatim in Spanish (the language of data collection) and uploaded to Nvivo 10 for coding management. Thematic analysis was used to identify recurring patterns within the main categories of motivation, what worked, issues experienced, and recommendations. Content analysis was used to support a descriptive summary of themes.²⁶ The seven phases of thematic analysis that were used included: familiarizing with the data, generating initial codes, searching for themes, reviewing themes, defining and naming themes, and producing a report of the analysis. Interviews were double coded by research staff (HM, AV) and discrepancies were resolved through discussions with a third author (SI). Frequencies of themes were reported. Findings were synthesized into recommended app design changes for the human centered design and software engineering team. The treatment supporter field notes and summary of experiences during the study were coded separately and focused on reported experiences and perception of participant experiences.

Role of the funding source

The funders (The National Institute of Health (NIH), United States) had no role in study design, data collection, data analysis, interpretation, or writing of the report.

Results

Among 56 patients initiating TB treatment during the recruitment period, 42 were enrolled in the study. Those excluded had severe illness/hospitalized (5), were under 18 (2), did not have access to a mobile phone (1), did not have WiFi access at home (1), did not return/declined to participate (4), and had an incomplete registration (unclear if patient was informed about the study) (1). Mean age of participants was 36·5±16·6 years (range 18-79) with nearly equal sex distribution (Table 1). The majority of participants were single (25, 60%), not working (28, 67%), completed through secondary school (14, 33%), never or quit smoking (34, 81%), did not take medication on a daily basis (32, 76%), and consumed alcohol once a month or less (30, 71%). Intervention group participants were on average older and more were married or with a long-term partner.

Table 1.

Socio demographic parameters of study participants.

Study Characteristic	Control	Intervention	Total
N	21	21	42
Age (mean, sd, range)	31·5 (14·3) 18-68	41·4 (17·6) 19-79	36·5 (16·6) 18-79
Sex
Male	10 (48%)	10 (48%)	20 (48%)
Female	10 (48%)	11 (52%)	21 (50%)
Other	1 (4%)	0 (0%)	1 (4%)
Ethnicity
Hispanic or Latino	21 (50%)	21(50%)	42 (100%)
Race
White	17 (81%)	13 (62%)	30 (71%)
Indian American or native	0 (0%)	1 (5%)	1 (2%)
Not specified	4 (19%)	7 (33%)	11 (26%)
Marital status
Widow/widower	0 (0%)	1 (5%)	1 (2%)
Separated	4 (19%)	0 (0%)	4 (10%)
Long term partner	1 (5%)	4 (19%)	5 (12%)
Married	1 (5%)	6 (28%)	7 (17%)
Single	15 (71%)	10 (48%)	25 (59%)
Employment status
Working	5 (24%)	9 (43%)	14 (33%)
Not working	16 (76%)	12 (57%)	28 (67%)
Occupation
Temporary/Informal/Day work	5 (24%)	2 (10%)	7 (17%)
Other	12 (57%)	16 (76%)	28 (67%)
Not specified	4 (19%)	3 (14%)	7 (17%)
Student
Yes	6 (29%)	2 (10%)	8 (19%)
No	14 (67%)	18 (86%)	32 (76%)
On hold	1 (5%)	0 (0%)	1 (2%)
Not specified	0 (0%)	1 (5%)	1 (2)%
Education level
Primary school not completed	1 (5%)	0 (0%)	1 (2%)
Primary school completed	3 (14%)	5 (24%)	8 (19%)
Secondary school not completed	7 (33%)	5 (24%)	12 (29%)
Secondary school completed	8 (38%)	6 (29%)	14 (33%)
University not completed	1 (5%)	2 (10%)	3 (7%)
University completed	1 (5)%	2 (10%)	3 (7%)
Other	0 (0%)	1 (5%)	1 (2%)
Income per month
Pension	1 (5%)	1 (5%)	2 (5%)
Retired	2 (10%)	4 (19%)	6 (14%)
Child allowances	2 (10%)	0 (0%)	2 (5%)
Other	3 (14)%	2 (10%)	5 (12%)
Not specified	13 (62%)	14 (67%)	27 (64%)
Total monthly family income
< 20,000 pesos ($450 USD)*	20 (95%)	14 (67%)	34 (81%)
Between 20,000 and 25,000 pesos ($450-480 USD)*	0 (0%)	4 (19%)	4 (10%)
> 25,000 pesos (> $480USD)*	1 (5%)	3 (14%)	4 (10%)
Additional Health Problems reported
None	18 (86%)	14 (67%)	32 (76%)
Lung disease	1 (5%)	2 (10%)	3 (7%)
Kidney disease	0 (0%)	1 (5%)	1 (2%)
High blood pressure	0 (0%)	1 (5%)	1 (2%)
Diabetes	1 (5%)	0 (0%)	1 (2%)
Other	1 (5%)	3 (14%)	4 (10%)
Taking a daily medication
Yes	3 (14%)	7 (33%)	10 (24%)
No	18 (86%)	14 (67%)	32 (76%)
Current smoker
Current	4 (19%)	4 (19%)	8 (19%)
Prior smoker	7 (33%)	7 (33%)	14 (33%)
Never	10 (48%)	10 (48%)	20 (48%)
Alcohol consumption
None	11 (52%)	5 (24%)	16 (38%)
Once a month or less	6 (29%)	8 (38)%	14 (33%)
2-3 times per month	4 (19%)	7 (33%)	11 (26%)
2-3 times per week	0 (0%)	1 (5%)	1 (2%)
Number of glasses of alcohol (beer or wine) consumed during a common day
0-2	16 (76%)	17 (81%)	33 (79%)
3-4	5 (24%)	4 (19%)	9 (21%)
Global Health Outcome Questionnaire
Global Physical Health (mean, sd)	40·9 (8·3)	40·4 (7·4)	40·7 (4·7)
Global Mental Health (mean, sd)	45·0 (6·0)	46·6 (6·0)	45·8 (6·0)

^⁎conversion values are from the time of the study, not current value.

Participant phone use

Most participants had prepaid mobile phone plans (23, 54·5%), regular or access for a large part of the day to Internet (38, 94·5%), access to Internet solely through their mobile phone (27, 64·3%), personal access to a mobile phone/smartphone (33, 78·6%), and were very familiar with using WhatsApp (33, 88%) (Table 2). Fewer were familiar with other apps (17, 40·5) and all had Android phones.

Table 2.

Phone and internet access.

	Control	Intervention	Total
N	21	21	42
Type of Phone plana
Monthly payment	6 (29%)	12 (57%)	18 (43%)
Prepaid	14 (67%)	9 (43%)	23 (55%)
Other	0 (0%)	2 (10%)	2 (5%)
Not specified	1 (5%)	0 (0%)	1 (2%)
Regular internet access
All the time	6 (29%)	10 (48%)	16 (38%)
A large part of the day	14 (67%)	8 (38%)	22 (52%)
Sometimes	1 (5%)	3 (14%)	4 (10%)
Access to the internet
Phone and another source (e.g., computer)	4 (19%)	11 (52%)	15 (36%)
Phone only	17 (81%)	10 (48%)	27 (64%)
Phone access personal or shared
Personal phone	16 (76%)	17 (81%)	33 (79%)
Family phone	4 (19%)	4 (19%)	8 (19%)
Not specified	1 (5%)	0 (0%)	1 (2%)
Phone typea
Android	21 (100%)	21 (100%	42 (100%)
Other	1 (5%)	0 (0%)	1 (2%)
Phone company in Argentina used by participant
Movistar	5 (24%)	5 (24%)	10 (24%)
Claro	6 (29%)	6 (29%)	12 (29%)
Personal	9 (43%)	8 (38%)	17 (40%)
Other	0 (0%)	2 (10%)	2 (5%)
Not specified	1 (5%)	0 (0%)	1 (2%)
Familiarity with using WhatsApp
Very familiar	18 (85%)	19 (90%)	37 (88%)
Familiar	2 (10%)	0 (0%)	2 (5%)
A little familiar	0 (0%)	1 (5%)	1 (2%)
Not familiar	1 (5%)	1 (5%)	2 (5%))
Familiarity with using other phone applications
Very familiar	6 (29%)	11 (52%)	17 (40%)
Familiar	7 (33%)	5 (24%)	12 (29%)
A little familiar	5 (24%)	4 (19%)	9 (21%)
Not familiar	3 (14%)	1 (5%)	4 (10%)

^aIndividuals could have more than one type of phone plan and phone.

In-app reports of daily medication self-administration, potential side effects, and the urine metabolite test photo

Participants submitted each type of report separately, allowing them to choose whether to submit throughout the day or all at once. Of the total expected number of reports 85·6% of the medication self-administrations (3238 out of 3780 if each participant submitted daily), 81% of the medication side effects (which include an option to report ‘no side effects’) (3061 out of 3780 if each participant submitted daily), and 61·6% of the urine test photos (998 out of 1620 if each participant submitted 3 per week) were received.

Medication self-administration reports

The average number of days of medication self-administration reports per patient out of 180 (6-month treatment) was 147·2±34·4 (mean, SD). Eight participants reported through the app that they did not take their medication for a total of 9 days due to being hospitalized or medication on hold. There was some variation in medication reporting over the treatment course (Figure 1). The majority reported taking their medication regularly while a few either did not get started or reported intermittently. Reasons for varied reporting included: one after not reporting or responding to inquiries once contacted indicated that her father had passed away and she did not feel like reporting but was taking her medication, one reported a problem with logging in to the app, and three who shared a phone (with spouse, mother, and son) indicated that at times sharing the phone made it challenging to report. Three participants had their treatment extended and reported beyond the 6 months.

Figure 1.

Daily reporting per patient.

green = medication taken; gray = not reported or reported not taking.

Potential side effects reports

Of the side effect reports completed, 93·6% (2864/3061) selected ‘no side effects’ and 6·4% (197/3061) included one or more (range of 1-5) side effects. One participant reported a significantly higher number of potential side effects than all other participants (this individual reported side effects 154 days of their treatment). Removing the outlier participant, the most common side effects reported included upset stomach (17, 39·5%), other (14, 32·6) (e.g., headache, tiredness, dizziness, general discomfort), nausea (9, 20·9%), facial swelling (5, 11·6%), hives, and rash (4 each, 9·3%). Headache and dizziness were added later since they were being reported frequently in the other category. The majority of the side effects were reported in the first 7 weeks of treatment (Figure 2). The participant who reported experiencing a significantly higher number of side effects than other participants (0-5 per report) was evaluated by their healthcare team for potential drug side effects.

Figure 2.

Reports of at least one symptom over the course of treatment (excluding outlier).

Urine metabolite test photo submissions

The average number of urine test photos submitted per patient out of 77 was 47·5±38.4 (mean, SD) (range 1-152). Similar to medication self-administration reports, there were variations in submitting urine test photos with fewer submitting regularly and fewer submitting over time. One participant was identified to have drug resistance to isoniazid at approximately 2 months therefore stopped using the urine tests but asked to continue to use the app. The test strips accompanied 31% of all submitted reports. The treatment supporter classified 73·8% (n=737) as clearly indicating that medication was detected, 10·4% (n=104) were marked as negative or unclear, and 15·7% (n=157) were not coded. Some negative or unclear classifications were due to user error (e.g., submerging the test strip, taking photo too quickly). The uncoded tests were likely a result of a limitation of the provider interface at the time. Specifically, if multiple photos were submitted by a patient before being reviewed, only the last photo in the group could be coded. Modifications to the provider application have been made as a result of this finding to ensure each photo will be reviewed and classified.

Intervention participants feedback

Twelve (57%) participants in the intervention group completed an in-person exit interview or answered the exit interview questions by phone either written or as audio responses (7 in-person interviews and 8 surveys of which 3 also completed in-person interview and were only counted once). Interviews were 30-60 minutes each, with a total of 204 minutes of interview recordings. One interview was not recorded, and a summary was written up based on notes immediately after and reviewed with the participant. Three other participants had appointments scheduled but cancelled and were unable to reschedule.

We identified 33 themes within the main categories of motivation, what worked, issues experienced, and recommendations (Table 3). Each theme is further described below with exemplar quotes translated to English.

Table 3.

Main themes, definitions and count by participant (n=12).

Themes	Definition	N (%)
Motivation
Cure	No longer being TB positive·	5 (42)
Safety of others	Possibility of infecting others·	4 (33)
Loved one's involvement/support	Being held accountable and reminding from people in patient's life·	3 (25)
Being remotely monitored	Application creates sense of accountability·	3 (25)
Seeing the progress of treatment	Updated view of treatment as days were completed·	1 (8)
Helping others	Contributing to future patient's experience·	1 (8)
What worked
Easy to use app	Simple to navigate app and/or did not take a lot of time·	10 (83)
Helpful for tracking and reminding	Registering the medication was helpful in adherence and served as a reminder of responsibility to report·	8 (67)
Access to the treatment supporter	Having the treatment supporter available for issues/questions was useful and effective·	7 (58)
Routine	Establishing a schedule for medication was useful in adherence·	7 (58)
Calendar tracking	The calendar was useful for viewing progress and noting any missed medication days·	7 (58)
Side effect management	Supportive medication helped mitigate TB drug side effects·	5 (42)
Being supported from afar	Feeling like you had help from afar·	3 (20)
Education on disease and treatment	The TB information provided in the app was useful and relevant·	3 (25)
Report confirmation	Good confirmation of report submitted·	3 (25)
Messaging to pick up supplies	It was used when needing to pick up supplies, mainly test-strip and sometimes medication·	3 (25)
Test-strip	Confirming medication having been taken and initial teaching of how to perform·	2 (17)
Trying again later	If app did not appear to be functioning, a solution was trying again later·	2 (17)
Forum discussion	Having the open dialogue for anyone to participate helped with knowledge·	1 (8)
Taking preventative measures	Being on top of medication supply·	1 (8)
Issues
Test-strip	The main difficulties with test-strip was initially figuring out the ideal timing for color change and waiting approval· Treatment supporter helped instruct proper use·	5 (42)
Treatment supporter hours of availability	Test result interpretation delays were noted – expected quicker response times or additional treatment supporter hours of availability	4 (33)
Wi-Fi/Internet access	Inconsistent availability of either Wi-Fi or Internet affected access to app at times·	4 (4)
Inexperience with mobile phones	Lack of familiarity using phones·	3 (25)
Report did not go through	The reporting process was completed, and it did not register in app requiring additional attempts·	2 (17)
Forgotten password	If password was lost some difficulties occurred·	2 (17)
Errors in internationalization	Date not formatted in local standard; month and day switched·	2 (17)
Back button placement	Issue with the app's back-button location and home button of the phone·	1 (8)
Decreased motivation for test-strip	As time went on the test strip became less important·	1 (8)
Primary TB provider unaware of app intervention	The care providers of TB treatment were not aware of application existence·	1 (8)
Recommendations
App for others	Useful for others completing TB treatment·	8 (67)
Alarm	A built-in alarm would be helpful to tailor to the person's routine·	4 (33)
Discussion forum	Discussion forum is a useful tool·	2 (17)
Treatment supporter 24 hours	Increasing response time past day hours·	1 (8)
Event organization	An event organizer, such as appointments or in-person visits, with reminders could be helpful·	1 (8)
Test-strip instructions	Provide clear test strip instructions e·g·, duration to leave in urine and meaning of results·	1 (8)
Test-strip design	Add mark for where urine should reach on test	1 (8)
Function without internet	Use the app without Internet and upload once connected·	1 (8)

Motivation: Motivation to stay in treatment included to be cured, not infect others, support from loved ones, remote monitoring from the treatment supporter, seeing their treatment progress in the app, and to help others.

“for someone to remain attentive [treatment supporter] is an important help, more than anything to not forget [to take the medication]” [M, 67].

What worked: Participants reported that the app was easy to use and learn, helped keep them on track, served as a reminder, provided a routine, the calendar view helped to self-monitor, access to remote support from a treatment supporter was critical, access to accurate TB information was helpful, the test strips provided confirmation, and having access to medication that mitigated side effects were aspects that worked to keep them in treatment. All but two participants stated the application was simple to navigate and that it did not take a lot of time. The average ease of use was rated 4·57 out of 5 in the exit questionnaire. Having access to a treatment supporter was reported as a critical component of the intervention even if responses were not immediate. If an issue or question arose the treatment supporter was there to reach out to.

“with the possibility to make a consult... they [treatment supporter] would respond, not immediately but the next day and it was a positive experience” [M, 67].

Issues experienced: Technical issues included uncertainties in conducting or waiting for metabolite test result classification, inconsistent WiFi access, inexperience using a smartphone, uncertainty if reports went through, forgetting password, errors in internationalization (the process to design products to meet needs of users in many countries, for example, time display preferences), and issues with the back-button placement. It was also noted that there was less of a focus on reporting when worries were on other things. Other issues unrelated to the app or study described by participants included challenges of cost and access to TB medication and living far away from the healthcare facility.

“if you have that button [home button] with your phone, the one you have with...android...the [app's] back button takes you directly out of the app” [M, 32].

Recommendations: Recommendations included adding an alarm, appointment reminders, off-line function, keeping the discussion forum, increasing access to a treatment supporter, and improving test strip instructions (e.g., add clear mark for depth to dip in urine sample).

“I think there should be someone to support 24 hours because one does not know at what moment one could have a reaction or something” [F, 26].

Treatment supporter interview and reflections

What worked: The treatment supporter described a learning curve which was most difficult in the beginning due to new issues and learning the technology. He described that conducting the intervention became easier after finding solutions, recognizing individual patient routines and their patterns of reporting, and being able to successfully assist and build relationships with the participants. More support and often more time was needed during treatment initiation. After the first or second month less time was required for each patient. On average the intervention took approximately 30 minutes or less per day to review 20 patients and up to an hour if multiple participants had complex issues that required coordination or additional support. The treatment supporter, who was a nurse within the TB program with other responsibilities, indicated that he reviewed and responded to patients as needed quickly first thing in the morning and then again at the end of his shift. He noted that participants often reported consistently during the first 2 months and then their reporting changed due to life events or changes in schedules. To him, it was important to form a supportive relationship with the participants by listening and being available to the patients who he felt are often not listened to. He also described the need to accommodate to the patients schedule and be patient-centered which included coming up with a reporting plan that worked for the patient that could include reporting less frequently based on their needs rather than be overly concerned with regular documentation.

“Person-centered care implies recognizing the singularity and uniqueness of each individual and focusing on their needs as they arise. The intent is to accompany and support individuals’ self-determination of their own health/disease/care/care processes by respecting their decisions, preferences, and personal options. The idea is "to adapt the support tool to people” and not for people to "adapt to the application and its protocols." That is why it is essential to establish a good interpersonal connection from the beginning [with the patients] to bypass the usual "biomedical" or "asymmetric relationship of doctor-patient.” The goal is to enable a space for exchange, where all situations can be raised without people feeling judged or watched.”

Issues: The treatment supporter noted issues experienced using the test strip, technical issues, and unrelated side effects. Technical issues included a lack of Internet access, issues adding the app to their home display, and losing settings when their phone would restart. If there was an app issue, the treatment supporter would be notified by a number of participants while individual technical issues took more time to resolve. Participants also reached out regarding issues considered to be unrelated to their TB or TB treatment.

Recommendations: Recommendations included adding more options for technical support and screenshots that were captured and sent to the technical team to demonstrate issues.

Interface design recommendations based on findings

A summary of what worked well in the intervention according to participants, as well as areas to target future refinement are outlined in Table 4. Additions could include instructional videos or additional visuals to improve test strip performance, clearer submission feedback to reduce uncertainty of submission status, medication reminders, and a group announcement option for the treatment supporter interface. The initial discussion board was set up using a separate established system that required email login. We found that over half of the participants did not have an email therefore we transitioned to a within app discussion board which took time to develop and roll out. Limited use of this function was believed to be due to it being added at a later date, once participants were accustomed to using the app and taking treatment. The discussion forum was recommended to be maintained but changed to a similar style as WhatsApp because of the familiarity and common use.

Table 4.

Focus area and potential solutions to improve intervention based on findings.

Focus	Potential solution and refinement strategies
Test strip	•Develop additional resources on completing test (e.g., video instruction, visual diagram outlining steps, list of ‘tips’ to take high-quality pictures) •Request urine tests on random days once per week rather than more frequent submissions
Unclear report submission	•Provide clear feedback that report was submitted/completed (e.g. “Today's report has been submitted”) •Reduce photo loading and upload times
Participant facing features	•Add alarms and alerts (e.g., medication, appointment reminders) •Add a treatment timeline or milestone indicators (e.g., treatment duration, what to expect such as medication changes at month two of treatment) •Make the notes section more specific (e.g., things to remember, medication names, appointments, etc.) One participant used a separate notes app to keep track of these things.
Treatment supporter facing features	•Add an announcement function that the treatment supporter could use to alert all patients (e.g. will be gone for a few days) •Increase technical support
Forum not heavily utilized	•Change forum layout more familiar style (e.g., WhatsApp-style chats) to increase participation/ease of use •Initiate use at the start of treatment
Maintain and improve useful app features	•Access to a treatment supporter •Side-effect reporting •Calendar view/visualization of treatment tracking •Treatment progress indicator/graphic (e.g., days in treatment, remaining)

Treatment outcomes

The majority of participants successfully completed treatment (37, 88·1%), 2 were transferred out, 2 abandoned treatment, and one died (Table 5). The treatment success (cure or completion) was 81·0% [17/21] for usual care and 95·2% [20/21] for the TB-TSTs intervention (two-sample test of proportions was p=0·14 but the study was not powered for statistical significance). The 14.2% higher rate of treatment success may suggest clinical importance and need for further research using an adequately powered sample size to assess efficacy. Only intervention participants had follow-up sputum test to be classified as cured (n=7, 33%). Treatment outcome of those who transferred out are unknown. The two participants with the lowest reported adherence using the app (Figure 1 – participants 11 and 26) completed treatment.

Table 5.

Final treatment outcome by group.

	Control	Intervention	Total
N	21	21	42
Cure	0 (0%)	7 (33%)	7 (17%)
Completion (≥6 months)	17 (81%)	13 (62%)	30 (71%)
Transfer out	1 (5%)	1 (5%)	2 (5%)
Default/abandonment (≥2 months)	2 (10%)	0 (0%)	2 (5%)
Death	1 (5%)	0 (0%)	1 (2%)

Discussion

We evaluated the feasibility, acceptability, and explored if the TB-TSTs intervention showed promise in improving treatment outcomes. Overall participants had high use of the TB-TSTs intervention, rated it as easy to use, and would recommend it to others starting TB treatment. A critical feature of the intervention was remote access to a treatment supporter to help mitigate issues and provide support throughout treatment. Overall treatment success was higher than historic country averages in both groups and there were more who defaulted treatment in the usual care group. According to the 2020 WHO Argentina country report, the treatment success rate for new and relapsed TB cases was 47%, however, rates of lost to follow up or unknown outcome have historically been between 30 – 40% with treatment success rates of 77·2% among those with known outcomes. This study was the first to assess user-centered design issues of an app and a direct adherence test reengineered for home-use to support patients with active TB in this setting. Few studies include participants in the refinement process or prioritize understanding users experience and mainly focus on technology alone.¹² Moreover, although alternative metrics that include biological tests of drug ingestion, such as urine testing, have been highlighted for their potential, few studies have assessed home-based direct metabolite testing.

Although reporting patterns varied, and for some decreased over time, those in the intervention successfully completed treatment and most had high rates of intervention engagement suggesting acceptability and feasibility of these digital tools. The intervention combined direct and indirect adherence monitoring to identify adherence issues more accurately and quickly. A recognized strength of the intervention was in providing access to a treatment supporter. Once the treatment supporter received the real-time adherence information, he was able to send inquiries, offer support, and customize care based on patient needs. For example, the treatment supporter helped trouble shoot potential side effects. In a systematic review of factors affecting adherence in patients undergoing TB treatment, a lack of knowledge on side effects and experiencing side effects was associated with non-adherence and decreased follow-up for patients.²⁷ Having access to a treatment supporter has been well documented to improve adherence and outcomes. One metanalysis found that patient education and counselling increased the cure rate for TB with evidence of nurse support being more effective than solely physician appointments.²⁸ In patients undergoing TB treatment in Uganda, SMS reminder messaging were found to create a shared desire between patient and supporter in getting well and completing treatment.²⁹

Prior studies have highlighted the issue of technology fatigue³⁰ and that adherence behaviors developed while using an electronic monitor could potentially be maintained even after withdrawing the technology.³¹ For those not reporting regularly the treatment supporter had established relationships and knew various situations of patients that contributed to lower documentation of adherence (e.g., returned to work but indicated not having issues and taking medication regularly, shared a phone with husband who had a varying work schedule, drug resistance identified and not submitting test photos). Thus, the emphasis was on providing support and at times establishing a tailored reporting schedule that was more aligned with the patient's needs. Our findings are consistent with others describing DATs as helping to remind patients to take their medication and facilitating the provision of individualized care.¹²

Participants reported being motivated to adhere to treatment to cure themselves, prevent spread to others, and by loved ones reinforcing the importance of a cure or directly reminding participants. Similarly, higher adherence was seen for patients undergoing antiretroviral therapy due to their motivation to have good health and having social support from family or partners.³²

Using a direct measure of adherence, such as a urine metabolite test, has been identified as an ethical and more accurate way to monitor adherence compared to directly observed therapy (DOT).¹³ Despite some challenges learning how to use the urine test, the treatment supporter confirmed adherence and used the test results to interact with participants and assess progress. A urine metabolite test to detect isoniazid in HIV patients undergoing preventative TB treatment in Brazil was found to be effective in giving insight on adherence rates and that typically self-reported adherence was greater than metabolite test proven adherence.³³ Similarly, other emerging digital adherence technologies, such as 99DOTS,’ have been found to have suboptimal benefits for identifying nonadherent patients.³⁴ These findings support the need and effectiveness of urine metabolite tests in confirming adherence, particularly for remote self-administered treatment.

Despite the potential benefits of DATs, concerns of autonomy, privacy, confidentiality, trust in patient, and ancillary care obligations have been described in the literature.³⁵ In this study, participants did not raise concerns of privacy or confidentiality and described the intervention as helpful to stay on track or get questions answered rather than a sense of perceived distrust. However, as described by Campbell et al (2016), an additional strain from ancillary care obligations when an electronic adherence monitoring detects non-adherence was noted by the treatment supporter. There were participants who requested support not related to their treatment, such as for appointments for family members or for other health issues, that resulted in additional time being spent. In this setting, the standard of care is self-administration of treatment and, where possible, referral to health care centers closer to where the patient lives. In settings using directly observed therapy it could result in time savings. An option of having a treatment supporter be part time or who oversees patients across a number of systems was discussed as a possible alternative to the additional strain from ancillary care obligations for those within the clinic structure.

Issues such as difficulty accessing Wi-Fi or Internet were experienced during this study and have been reported in the literature.³⁶ Few participants in this study reported being inexperienced in using mobile devices and reported that most technical issues were able to be resolved by reaching out to the treatment supporter. Nonetheless, not knowing how to use the technology, such as mHealth apps, have been reported as a barrier.³⁷

The most common patient recommendations were adding an alarm and reminders function to the app. Not having a built-in reminder function was a limitation of the web-app. In a systematic review, higher treatment success rates were found when reminders and tracers were incorporated.³⁸ Similarly, findings from another systematic review of patient's perceptions of mHealth apps found users stressed the ability to tailor prompts to their need.³⁹ In our study the discussion forum was not highly used, yet participants recommended keeping the feature. In a qualitative study with adult patients undergoing DOT TB treatment in Lima, Peru, patients desired increased peer activities and found that forming valuable relationships with other patients was helpful.⁴⁰

In this study treatment success rates were higher than historic averages. The higher rate of treatment success in the control group compared to the WHO country report may reflect issues of small sample size and a cohort of participants with a higher likelihood to complete treatment at baseline although participants were enrolled on a rolling basis as they were diagnosed and started treatment. Only participants in the intervention group had follow-up sputum tests to be classified as cured. This finding is possibly due to increased TB treatment information provided in the app and by the treatment supporter asking if follow-up sputum tests were completed.

Our study had several limitations. First, we included only participants who had access to a mobile phone. It is likely when considering scaling that healthcare facilities would be unable to provide phones. However, the use and access to smartphones is widespread and increasing globally. Secondly, there were participants with inconsistent access to Internet or who had shared phones which reduced their ability to report regularly. For these cases, the treatment supporter was aware of the variability and made individualized arrangements (e.g., report less frequently). Lastly, the interviews were conducted at the end or a few months after the completion of their treatment. Therefore, some participants may have had difficulty recalling all of the challenges or issues and not all responded. In order to account for this potential limitation, we also analyzed all interactive messages to identify issues as they occurred (reported elsewhere).⁴¹ Additionally, participant recommendations were not shared with other participants to assess agreement.

In conclusion, the high use of the app to self-report adherence and submit photos of the drug metabolite test to confirm progress, along with participant feedback suggests that the TB-TSTs intervention was feasible and acceptable. The interactive communication with the treatment supporter to address needs and create a sense of partnership was considered an essential intervention component. The treatment supporter learned the patients’ routines, provided treatment and technical support, and gained experience to better support patients over time. We believe that these findings support the need to further refine the TB-TSTs based on participant feedback and evaluate the intervention in an adequately powered pragmatic clinical trial.

Contributors

SJI: Conceptualization, Formal analysis, Investigation, Funding acquisition, Visualization, Writing - original draft, Writing - review and editing; HM: Formal analysis, Data interpretation, Writing - review and editing; CC: Conceptualization, Supervision, Writing - review and editing, Resources; RS: Conceptualization, Supervision, Writing - review and editing; KG: Investigation, Accessed and verified the data, Formal analysis, Figures, Writing - review and editing; HT: Data curation, Project administration, Resources; AI: Supervision, Writing - review and editing, Resources; MS: Supervision, Writing - review and editing, Resources; BL: Conceptualization, Investigation, Methodology, Writing - review and editing; KP: Accessed and verified the data, Formal analysis, Figures, Tables, Writing - review and editing; FR: Supervision, Writing - review and editing; GD: Conceptualization, Supervision, Writing - review and editing, Resources; HT: Data curation, Project administration, Resources.

Data sharing statement

The data supporting the findings of this study, which does not contain any identifiable data, as well as the spreadsheet that contains the tables, figures, and the analysis are available in the supplementary materials.

Declaration of interests

Authors (SI, KG) participated in the iterative software. Authors (SI, BL, KG, DL, MR, JK) submitted a provisional patent of the test strip. The authors declare no financial conflict of interest. The corresponding author (SI) is a recipient of NIH research funding (K23NR017210) that provided funding for software development, data collection, and analysis but it did not have any role in writing of the manuscript or the decision to submit it for publication. No authors have been paid to write this article by a pharmaceutical company or other agency. The authors were not precluded from accessing data in the study, and they accept responsibility to submit for publication.