Patient-reported usability and satisfaction with electronic medication event reminder and monitor device for tuberculosis: a multicentre, randomised controlled trial

Summary

Background

The success of a tuberculosis digital adherence technology relies on patients’ satisfaction with and the usability of the technology. This study aimed to evaluate treatment satisfaction and usability of a digital medication event reminder and monitor (MERM) device for patients with tuberculosis to address the prespecified secondary endpoint of the SELFTB trial.

Methods

In this multicenter, randomised controlled trial, adults (≥18 years) with new or previously treated, bacteriologically-confirmed, drug-sensitive pulmonary tuberculosis who were eligible to start anti-tuberculosis therapy were recruited from 10 healthcare facilities in Ethiopia. With a computer-generated random number sequence, participants were randomly assigned 1:1 to receive a 15-day tuberculosis medication supply dispensed with an evriMED500® MERM device to self-administer and return every 15 days or the standard in-person DOT. Both arms were followed throughout the standard two-month intensive treatment phase. Treatment was based on the WHO-recommended two-month fixed-dose-combination of first-line anti-tuberculosis drug delivered as a single daily dose (2RHZE). Treatment Satisfaction Questionnaire for Medication version 4 (TSQM 1.4©) was used to measure and compare treatment satisfaction between arms. Adapted System Usability Scale (SUS) was used to assess the usability of the device, with emphasis on ease of use, challenges, benefits, motivation, popularity, and recommendation. The findings were correlated with adherence and clinical endpoints including sputum smear conversion and IsoScreen urine isoniazid test results. This trial is registered with ClinicalTrials.gov, NCT04216420.

Findings

Between June 2, 2020, and June 15, 2021, 337 patients were screened for eligibility, of whom 109 participants enrolled and completed the satisfaction [control (n = 57) and intervention (n = 52) arms] and usability [intervention arm (n = 52)] questionnaires. TSQM 1.4© geometric mean scores were: Effectiveness 73.25 [geometric standard deviation (GSD) 1.28], Side Effects 100, Convenience 63.31 (GSD 1.45), and Global Satisfaction 77.29 (GSD 1.25). TSQM score was significantly higher in the intervention vs the control: Effectiveness [85.78 vs 63.43, 95% CI 1.35 (1.26–1.45), p < 0.001], Convenience [85.41 vs 48.18, 95% CI 1.77 (1.63–1.93), p < 0.001], and Global Satisfaction [90.19 vs 67.11, 95% CI 1.34 (1.26–1.43), p < 0.001]. There were significant associations between Global Satisfaction and medication adherence (p = 0.017). Average SUS score was 97.45%, which was close to the best imaginable SUS value of 100%. Likelihood to Recommend (LTR) value was ≥9, on a scale of 0–10, for 90.4% of MERM users, yielding higher net promoters. There was no significant association between usability and medication adherence (p = 0.691).

Interpretation

Our findings suggested that treatment satisfaction scores were superior in the intervention vs control arms and across the domains of Effectiveness, Convenience, and Global Satisfaction. There was excellent usability of the MERM device and a significantly higher number of users likely to promote the device. High tuberculosis burden countries may transform patient-centered care through ongoing evaluation and scale-up of digital health innovations.

Funding

U.S. National Institute of Health (NIH) Fogarty International Center and National Institute of Allergy and Infectious Diseases (D43 TW009127) and the Emory Center for AIDS Research (P30 AI050409).

Research in context.

Evidence before this study

We did a systematic search of PubMed, Embase, ScienceDirect, African Journals Online, Cochrane Central Registry of Controlled Trials, ClinicalTrials.gov, and the WHO International Clinical Trials Registry Platform databases that assessed the effectiveness or efficacy of digital adherence technologies for patients with TB until January 14, 2020. Search terms used were ((Digital health [MeSH Terms]) OR (Digital [Title/Abstract] OR Mobile [Title/Abstract] OR Smartphone [Title/Abstract] OR “Cell phone” [Title/Abstract] OR Techno∗[Title/Abstract] OR “short message service” [Title/Abstract] OR SMS [Title/Abstract] OR Tele∗[Title/Abstract] OR Telemedicine [Title/Abstract] OR Telehealth [Title/Abstract] OR E-health [Title/Abstract] OR eHealth [Title/Abstract] OR Remote [Title/Abstract] OR Electro∗[Title/Abstract] OR Comput∗[Title/Abstract] OR cloud [Title/Abstract] OR Software [Title/Abstract] OR Application [Title/Abstract] OR Robotics [Title/Abstract] OR Blockchain [Title/Abstract] OR “Artificial intelligence” [Title/Abstract] OR genomics [Title/Abstract] OR “big data” [Title/Abstract] OR cybersecurity [Title/Abstract] OR wireless [Title/Abstract])) AND (Tuberculosis [Title/Abstract]). Mobile health, electronic event medication monitors, and video DOT were major technologies of interest. However, studies were limited, and for those available studies, reports at different resource-constrained countries have been a subject of research and controversy. Using similar databases, we conducted a systematic review, until February 01, 2021, to understand whether and how digital health technologies are absorbed in Africa, tracking Ethiopia as a key node. We found that digital health technologies hold much promise in strengthening healthcare systems in Ethiopia, while the use of such technologies was a relatively new phenomenon and randomized trials were critically limited to provide robust evidence of their potential.

Added value of this study

We did a multicentre, two-arm, effectiveness-implementation type 2 hybrid, randomised controlled trial among patients with tuberculosis in Ethiopia to evaluate treatment satisfaction and usability of MERM-observed self-administered therapy as compared to the standard in-person DOT. Treatment satisfaction scores were superior in the intervention vs control arms and across the domains of Effectiveness, Convenience, and Global Satisfaction, and there was excellent usability of the MERM device and a significantly higher number of users likely to promote the device.

Implications of all the available evidence

Our findings suggested that an electronic medication reminder and monitor device that records adherence, stores medication, emits audible and visual on-board alarms to remind patients to take their medications on time and refill, and enables providers to download the data and monitor adherence can facilitate care in a more patient-centric approach, all without affecting the inherent dignity and wellbeing of patients. This is especially important in the two-month intensive phase where patients’ early disease condition, daily travels, and economic status could interrupt treatment.

Introduction

One of the major challenges facing healthcare today is providing patient-centered care that is respectful of, and responsive to, individual patient preferences, needs, and values in health decision-making.¹ Healthcare systems are unable to ensure that care is organized and delivered in a transparent, safe, and responsible way for all facets of patient health. Maximizing patient engagement in treatment decisions and which care packages meet the needs and expectations of patients and their households remains a priority for research and quality improvement in healthcare.² This can be traced to the origins of “empowerment theory”³ - connecting individual-wellness with the larger political and social environments through which individuals and groups gain greater control over their lives, acquire rights, and reduce marginalization. These are critical milestones to improve clinical outcomes, build sustainable healthcare systems, and address global health strategies.

Exacerbated by socioeconomic factors, Tuberculosis (TB) continues to be the deadliest infectious disease, posing a high burden on global health. Patient-centered care is the critical first pillar of the World Health Organization's (WHO) End TB Strategy which anticipates reducing 90% incidence and 95% deaths due to TB by 2035.⁴ The End TB Strategy positions patients with TB and their households at the heart of healthcare service delivery. However, global progress to End TB has been slow, with huge disparities within and across countries while the world's poorest and most vulnerable people carry the heaviest burden of this disease.⁵ Lost to follow-up, treatment interruptions, disease transmission, and drug resistance remain substantial challenges in many high TB prevalence countries.6, 7, 8, 9, 10 The End TB Strategy calls for intensified research and innovation for the discovery, development, and rapid uptake of new tools, interventions, and strategies that would maximize TB program efficiency, although only a few studies have succeeded.¹¹ One of the major patient-centered platforms that are being studied for their potential to engage and assist behavioral change of patients with TB is the use of digital health technologies. Currently, digital technologies such as electronic device treatment monitors, video treatment monitors, and phone calls or short message service (SMS) treatment reminders are being studied in various contexts.

In support of this initiative, we designed and implemented an effectiveness-implementation type 2 hybrid, randomized controlled trial to assess whether a digital medication event reminder monitor (MERM)-observed self-administered therapy would be effective for patients with TB compared with the standard in-person DOT in a high-burden, low-income country (Ethiopia) context.¹² For the primary endpoints, medication adherence among participants assigned to MERM-observed self-administered therapy was non-inferior when compared with the standard in-person DOT.¹³ Similarly, the MERM-observed therapy was associated with higher health-related quality of life (HRQoL) and lower catastrophic costs compared to the standard DOT.¹⁴ The MERM device studied (evriMed500®, manufactured by Wisepill Technologies, South Africa) holds an electronic module and a medication container to record adherence to treatment, store medication, emit audible alerts, and illuminate three visual light diodes to remind patients to swallow and refill their medication, and enable healthcare providers to monitor adherence digitally.¹⁴ Some previous studies done elsewhere have demonstrated improved TB care with the use of electronic medication event monitors15, 16, 17, 18 but results were inconsistent and few followed a randomised trial design.

There has been a growing interest in using patient-reported outcomes (PROs) as major endpoints in clinical trials to evaluate the potential of new digital health interventions.^1,19 The success of a digital health intervention is dependent on the end-users satisfaction with and the usability of the technology and the overall benefits to their health and wellbeing. Accordingly, some studies have assessed the acceptability and satisfaction with the use of the MERM device for patients with TB and the findings were either promising²⁰ or variable between patients.²¹ PROs are relatively understudied for the MERM device and very few followed randomised controlled trials design, indicating the need for more trials to understand patient usability and satisfaction with such devices.

Thus, this randomised controlled trial aimed to evaluate the patient-reported usability and treatment satisfaction with MERM-observed self-administered TB therapy compared with the standard in-person DOT.

Methods

Study design and participants

This multicentre, attention-controlled, non-inferiority, effectiveness-implementation type 2 hybrid, randomised controlled trial was conducted in 10 healthcare facilities in Ethiopia, and the current report presents pre-specified secondary outcomes of the trial. The trial is registered with ClinicalTrials.gov, NCT04216420. Findings of the study on medication adherence and treatment outcomes¹³ and HRQoL and catastrophic costs,¹⁴ a full description of the study protocol,¹² (Supplemental file 1) a systematic review,²² and cross-sectional mixed-methods studies that assessed human resource²³ and infrastructure²⁴ capacity of study sites for digital health were published elsewhere.

Eligible potential patients were adults aged ≥18 years; new or previously treated, bacteriologically-confirmed drug-sensitive pulmonary TB; eligible to start the standard 6-month first-line anti-TB medication; receiving outpatient care; and willing and able to provide informed consent. Patients were ineligible if they had known drug-resistant TB or if they had a concurrent health condition that precluded informed consent or their ability to safely participate in the study procedures. The national algorithm for TB diagnosis defines a bacteriologically-confirmed PTB case as a person who has at least one positive result on AFB microscopy or whose molecular WHO-recommended rapid diagnostics test result such as Xpert MTB/RIF or Xpert Ultra assay indicates MTB detected.²⁵

The intervention tested in this trial was MERM (evriMED500®, manufactured by Wisepill Technologies, South Africa) observed self-administered therapy that recorded treatment adherence, stored medication, emitted audible and visual alerts to remind patients to self-administer their medications, and enabled healthcare providers to monitor adherence digitally. The device is described further in the study protocol.¹²

Randomisation and masking

With a computer-generated random number sequence, eligible participants were randomly assigned (1:1) to either the intervention arm where participants received a 15-day TB medication supply in the evriMED500® MERM device to self-administer and return every 15 days or the control arm where participants visited the healthcare facility each day throughout the two-month intensive phase to swallow their daily anti-TB dose with direct observation by TB healthcare providers per the standard in-person DOT. A permuted block randomisation method was used to randomly allocate participants and maintain a balance of the number of participants assigned to each arm. The study investigators who were responsible for assessing study outcomes and writing the report were blinded to group allocation until the manuscript was completed. Because of the trial logistics, participants and the other study staff were not blinded to group allocation. No stratification was needed for key variables. A statistician masked to group allocation performed the analyses.

Procedures

Baseline assessments included detailed demographic, socioeconomic, behavioral, social, and clinical characteristics using a researcher-administered questionnaire [Supplemental file 2]. Participants assigned to the intervention arm received clear instructions on the functions of and how to use the evriMED500 MERM device. They were also given a graphical leaflet outlining the procedures [Supplemental file 3]. The provider opened the container, removed the MERM Module from the designated area in the MERM container, inserted batteries to activate the MERM module, connected it to the computer via the USB cable, and configured the module with medication reminder and refill schedules in consultation with the participant and based on the Wispill evriMED® user manual. Once the MERM Module setup was completed, the provider disconnected the device from the computer and returned it to the designated slot of the MERM container. Then, the provider added the patient instruction label inside the MERM device, placed a 15-day supply of TB medication in the medication storage area of the MERM device, and gave the entire device to the participant for medication self-administration. The participants returned every 15 days, where the healthcare provider counted any remaining tablets in the pillbox and connected the MERM module with the computer. Along with the participant, the provider downloaded the data from the Wisepill® device onto the computer and reviewed the event reports over the previous 15 days. This included the dates and times that the user opened the device, to define how adherent the user was to the prescribed ingestion times. Any missed event, where no ingestion occurred over a particular prescribed ingestion period in the event report, was evaluated against any remaining tablets in the pillbox and discussed with the participant for confirmation. With these data, using the study's paper-based daily treatment adherence monitoring tool, the provider recorded the information about daily medication adherence and reasons for any missed doses. To avoid contamination, the TB clinicians were well informed of the risk of such contamination and hence they avoided any potential exchange of information between arms. Once enrolled and given the device, participants in the intervention arm visited the facility only four times, which contributed to minimizing contamination.

The evriMED500 dispenser has three indicator light-emitting diodes (LEDs) that are visible through the front of the container for the daily medication reminder (green LED), medication refill reminder, (yellow LED), and low-battery alerts (red LED). It also has a buzzer that is activated during the alarm sequences, and it emits a soft tone when the container is opened or closed. The trial's principal investigator and the trial coordinators received training on the application and use of the Wisepill evriMED® technology by the developer, Wisepill Technologies, South Africa. The Wisepill evriMED® application was set up on computers that have already been in use in TB or other similar clinics to understand the broad sustainability of the intervention.

Participants in the control arm were managed according to the standard DOT practice.²⁵ Both arms were followed throughout the intensive treatment phase of two months for drug-susceptible TB. Treatment was based on the WHO-recommended two-month fixed-dose-combination of first-line anti-TB drug delivered as a single daily dose, 2RHZE (rifampicin [R]/150 mg + isoniazid [H]/75 mg + pyrazinamide [Z/400 mg + ethambutol [E]/275 mg). Treatment follow-up was conducted by full-time clinicians in the TB clinic following a moderate on-site orientation. The participants underwent IsoScreenTM urine isoniazid test, GFC Diagnostics Ltd, Bicester, England,²⁶ which is a colorimetric assay with results interpreted as positive (treatment adherence) or negative (treatment non-adherence) based on an observed color following a mixture of the urine specimen with the dried reagent in the reaction chamber. At the end of the intensive phase, participants underwent microbiological testing to assess sputum smear conversion. Trained study staff collected several data for both arms, including patient-reported treatment satisfaction, patient-reported usability, treatment outcomes, adherence self-report, and side effect reports.

Estimates of patient-reported treatment satisfaction

Patient treatment satisfaction was assessed using the Treatment Satisfaction Questionnaire for Medication (TSQM v1.4©).²⁷ IQVIA©, Plymouth Meeting, Pennsylvania 19462, United States, owns the copyright to the TSQM and a license agreement was obtained for the use of the instrument. TSQM© is a PRO instrument designed to evaluate treatment satisfaction with a wide variety of medications. It comprises 14 questions subdivided into four domains: Effectiveness, Convenience, Side Effects, and Global Satisfaction. Each of the four domains have at least three questions: Effectiveness (Qs 1–3), Side Effects (Qs 4–8), Convenience (Qs 9–11), and Global Satisfaction (Qs 12–14). Of the 14 questions; 13 were designed as a 5- or 7-point Likert scale to assess the level of satisfaction or dissatisfaction a participant had with the medication last used in the clinical trial. The sole remaining question contains a binary (yes/no) score. Each domain was computed independently by adding the TSQM items from each domain and the composite score transformed into a value ranging from 0 to 100, with a higher score indicating greater satisfaction. The questionnaire was administered by independent study staff using a paper format at the end of the two-month intensive treatment phase.

Estimates of patient-reported usability of the MERM device

Participant experience using the MERM device during the two-month intensive phase was assessed using an adapted System Usability Scale (SUS),²⁸ where the specific usability questions were derived from the WHO practical guide monitoring and evaluating digital health interventions²⁹ and the WHO handbook for the use of digital technologies to support tuberculosis medication adherence.³⁰ As the 10 specific questions in the SUS were not detailed enough to meet the WHO's requirements for measuring usability of digital health technology, a MERM devise-specific and sufficient (18-item) tool was developed. In addition, as the questions in the SUS tool had not been formally translated to the local (Amharic) language to clearly define responses from 1 (strongly disagree) to 5 (strongly agree), using the Likert system would compromise the interpretation of our findings. For this reason, a dichotomous (yes/no) approach was followed that the WHO and other trials in similar settings also prefer for comparable studies. The questions focus on the MERM device's ease of use, challenges, benefits, perceptions of motivation, popularity, and recommendations. The study adapted and implemented the SUS procedures to score, calculate, and interpret usability more efficiently. Participants in the intervention arm responded to each item, and 50% of the items were negatively worded questions. The aim was to understand participants' experiences using the device during the two-month intervention.

Outcomes

This report contains the pre-planned patient-reported secondary outcomes of the trial including treatment satisfaction between the two arms, where the treatment satisfaction was measured and calculated for each participant by arm using the TSQM v1.4 from 0 to 100, with a higher score indicating greater satisfaction. An additional PRO was the usability of the MERM device, where the participant experience of using the MERM device during the two-month intervention was measured and calculated with a SUS-adapted, 18-item questionnaire transformed into a scale of 100, where 0 was the worst possible usability and 100 was the best possible usability. The report also included predictors of lower treatment satisfaction and usability of the MERM device.

Statistical analysis

To compare the TSQM scores between study arms, independent sample t-tests were done on log-transformed scores. Effects of the arms were estimated using a geometric mean (GM) with geometric standard deviation (GSD) and mean ratios (MR) with 95% confidence intervals (CI). A general linear regression was done on log-transformed TSQM scores to assess the relationship between TSQM scores and participants’ sociodemographic and clinical characteristics. The strategy for selecting the potential covariates was based on considerations made by the trial team, taking into account the importance of the potential confounders to the results of the main trial, reports from prior literature, and considering the clinical aspects of tuberculosis. During the selection procedure, care was taken to avoid potential mediator effects, and associations between predictors were also performed to assess for collinearity. After potential covariates were identified, a simple regression of one predictor at a time was fitted with the outcome. All predictors with a p-value of ≤0.20 were included for the adjusted multiple regression. All statistical tests were two-sided, and without accounting for multiple comparisons, the type I error rate was 0.05.

Effects were measured using an adjusted mean ratio (AMR) with 95% CI. A correlation coefficient was calculated to demonstrate a link between the SUS score and treatment adherence level. In all analyses, a 5% significance threshold was used to determine statistical significance. The p-values reported in this paper were not adjusted for multiple comparisons considering the exploratory nature of this study. Detailed procedures about the trial's other primary and secondary endpoints and their underlying assumptions and statistical procedures including the sample size calculation are published elsewhere.^13,14

The usability data were converted to and interpreted as percentages, ranging from 0 to 100, with the 50th percentile average score of 68.³¹ The SUS scores were described further using adjectives and their corresponding SUS score as Excellent, Good, Poor, and Awful. The SUS scores were computed by adding the responses for the 18 items. A positive response was given a score of 1 and a negative response was given a score of 0, which yielded a minimum of 0 and a maximum of 18 for the total responses. The scores were transformed into a scale from 0 (worst) to 100 (absolute best) for MERM system usability based on the SUS scoring guideline. A score >68 was considered above average system usability, and >80 was considered high usability. This was further categorized into Excellent (≥80.3), Good (68–80.2), Poor (51–67), and Awful (≤51)]. The likelihood to recommend (LTR)³¹ was measured using the research metric derived from the corresponding SUS scores as LTR = SUS/10. On a range from 0 to 10, this designated three classes of recommenders: promoters (LTR≥9), detractors (LTR≤6), and neutrals (6 < LTR<9). Net Promoter Score (NPR)³¹ was calculated using the proportion of promoters (score ≥9) minus the proportion of detractors (score ≤6). A correlation coefficient was estimated to assess the relationship between SUS and the trial's medication adherence. Responses to the open-ended question on usability were described narratively.

IBM SPSS version 25.0 (Chicago, IL, USA) was used for data analysis. The sample size was based on the primary trial endpoint, which was the individual-level percentage adherence. To achieve a statistical power of 80%, it was determined that a sample size of 57 in each arm for a total of 114 participants would be required for the primary endpoints.¹³

Ethical considerations

The protocol was approved by the Ethiopian Food and Drug Authority (ref. No. 02/25/30/138), the Ethiopian National Research Ethics Review Committee (ref. No MoSHE/RD/4.1/12/07/20), the Institutional Review Board of the College of Health Sciences, Addis Ababa University (ref No. 077/19/CDT), and the Ethical Clearance Committee of the Ababa Health Bureau (ref. No. A/A/H/B/7246/227). All participants provided written informed consent. No financial compensation was offered to participants. Importation of all study-related devices, reagents, and supplies was reviewed and certified by the authority responsible by law (the Ethiopian Food and Drug Authority).

Role of the funding source

The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. T.M., T.G. and O.F. had access to the dataset and had final responsibility for the decision to submit for publication.

Results

Patient characteristics

Participants were enrolled into the study between 02 June 2020 and 15 June 2021, with the last participant completing follow-up on 15 August 2021. A total of 337 patients from 10 healthcare facilities in Addis Ababa, Ethiopia, were screened for eligibility and, of these, 114 were selected, randomly assigned 1:1 into the trial with 57 (50%) to the intervention arm and 57 (50%) to the control arm (Fig. 1). The most frequent reasons for exclusion were having extrapulmonary TB, smear/Xpert-negative, and drug-resistant TB on the initial TB diagnostic test.

Fig. 1.

CONSORT trial diagram.

The mean (SD) age was 33.1 (11.1) and 72 (66.1%) were male. Eleven (10.1%) were re-treatment cases and had completed their previous treatment, and 15 (13,9%) had HIV co-infection, of whom 11 (73.3%) were receiving antiretrovirals. Laboratory diagnostic tools for pre-treatment confirmation of TB were the Xpert MTB/RIF assay [70 (61.4%) participants] and acid-fast bacillus smear microscopy [44 (38.6%) participants]. Among the participants diagnosed with pulmonary TB using smear microscopy, 16 (38.1%) were graded 3+ and 15 (35.7%) 2+. Mean (SD) monthly income was US $67.50 ($71.20; range, $0-$333.30), six (5.5%) were homeless, 68 (62.4%) lived in a house with a single bedroom, and 18 (17.4%) smoked cigarettes. Baseline characteristics and HIV status were balanced between the two groups (Table 1).

Table 1.

Characteristics of study participants.

Variables	Categories	Participants, No. (%)
		Total (N = 109)	Intervention (n = 52)	Control (n = 57)
Sex	Women	37 (33.9)	15 (28.8)	22 (38.6)
	Men	72 (66.1)	37 (71.2)	35 (61.4)
Marital status	Never married	50 (45.9)	23 (44.2)	27 (47.4)
	Married	50 (45.9)	28 (53.8)	22 (38.6)
	Widowed	2 (1.8)	0 (0)	2 (3.5)
	Divorced	7 (6.4)	1 (1.9)	6 (10.5)
Occupation status	No job	22 (20.2)	9 (17.3)	13 (22.8)
	Student	4 (3.7)	4 (7.7)	0 (0)
	Farmer	1 (0.9)	0 (0)	1 (1.8)
	Trader	10 (9.2)	5 (9.6)	5 (8.8)
	Housewife	9 (8.3)	2 (3.8)	7 (12.3)
	Government employee	10 (9.2)	5 (9.6)	5 (8.8)
	Daily laborer	42 (38.5)	19 (36.5)	23 (40.4)
	Other	11 (10.1)	8 (15.4)	3 (5.3)
Highest level of education	No formal education	9 (8.3)	3 (5.8)	6 (10.5)
	Primary	41 (37.6)	22 (42.3)	19 (33.3)
	Secondary	28 (25.7)	12 (23.1)	16 (28.1)
	Preparatory	11 (10.1)	5 (9.6)	6 (10.5)
	University diploma	9 (8.3)	4 (7.7)	5 (8.8)
	University diploma or above	11 (10.1)	6 (11.5)	5 (8.8)
Residential status	Lives alone	13 (11.9)	5 (9.6)	8 (14.0)
	Lives with family	83 (76.1)	42 (80.8)	41 (71.9)
	Lives with friends	5 (4.6)	1 (1.9)	4 (7.0)
	Homeless	6 (5.5)	3 (5.8)	3 (5.3)
	Other	2 (1.8)	1 (1.9)	1 (1.8)
No. of cohabitants	≤3	63 (57.8)	32 (61.5)	31 (54.4)
	4–6	37 (33.9)	16 (30.8)	21 (36.8)
	7–9	7 (6.4)	2 (3.8)	5 (8.8)
	≥10	2 (1.8)	2 (3.8)	0 (0)
No. of bedrooms	1	68 (62.4)	33 (63.5)	35 (61.4)
	2	30 (27.5)	14 (26.9)	16 (28.1)
	3	6 (5.5)	3 (5.8)	3 (5.3)
	≥4	5 (4.6)	2 (3.8)	3 (5.3)
Household head	Yes	61 (56.0)	31 (59.6)	30 (52.6)
	No	48 (44.0)	21 (40.4)	27 (47.4)
Residency status	Permanent	89 (81.7)	40 (76.9)	49 (85.9)
	Temporary	20 (18.3)	12 (23.1)	8 (14.0)
Smoking, No./day	Never	90 (82.6)	46 (88.5)	44 (77.2)
	1–5	18 (16.5)	5 (9.6)	13 (22.8)
	6–10	1 (0.9)	1 (1.9)	0 (0)
Khat (a stimulant)	Never	87 (79.8)	43 (82.7)	44 (77.2)
	1/week	10 (9.2)	3 (5.8)	7 (12.3)
	≥2/week	8 (7.3)	4 (7.7)	4 (7.0)
	1/month	4 (3.7)	2 (3.8)	2 (3.6)
Alcohol	Never	67 (61.5)	35 (67.3)	32 (56.1)
	>1/day	12 (11.0)	5 (9.6)	7 (12.3)
	2-5/day	19 (17.4)	7 (13.5)	12 (21.1)
	≥6/day	3 (2.8)	1 (1.9)	2 (3.5)
	Rarely	8 (7.3)	4 (7.7)	4 (7.0)
HIVa	Negative	93 (86.1)	44 (84.6)	49 (87.5)
	Positive	15 (13.9)	8 (15.4)	7 (12.5)
On antiretroviral (If HIV positive)	Yes	11 (73.3)	6 (75.0)	5 (71.4)
	No	4 (26.7)	2 (25.0)	2 (28.6)
TB treatment	New	98 (89.9)	46 (88.5)	52 (91.2)
	Relapse	11 (10.1)	6 (11.5)	5 (8.8)
Place of diagnosis	Study facility	75 (68.8)	33 (63.5)	42 (73.7)
	Health center	3 (2.8)	1 (1.9)	2 (3.5)
	Public hospital	10 (9.2)	8 (15.4)	2 (3.5)
	Private clinic/hospital	20 (18.3)	9 (17.3)	11 (19.3)
	Other	1 (0.9)	1 (1.9)	0 (0)
TB test methodology	Microscopy	41 (37.6)	15 (28.8)	26 (45.6)
	Xpert MTB/RIF	68 (62.4)	37 (71.2)	31 (54.4)
Microscopy result (If test with Microscopy)b	1-9 (Scanty)	1 (2.6)	0 (0)	1 (4.0)
	1+	10 (25.6)	5 (35.7)	5 (20.0)
	2+	12 (30.8)	7 (50.0)	5 (20.0)
	3+	16 (41.0)	2 (14.3)	14 (56.0)
Completed treatment (If ever treated for TB)	Yes	11 (100)	6 (100)	5 (100)
	No	0 (0)	0 (0)	0 (0)

TB: tuberculosis; ARV: antiretroviral; MTB/RIF: Mycobacterium tuberculosis/resistance to rifampicin.

^a1 missing value.

^b2 missing values.

Treatment satisfaction

Treatment satisfaction was the key patient-reported outcome of the trial. Of the 114 participants enrolled and randomised, 109 participants were included in the final complete-case ITT analysis for treatment satisfaction, 52 from the intervention arm and 57 from the control arms. This imbalance results from four participants who were transferred and one was lost to follow-up from the intervention arm, thus were unable to complete the questionnaire at the end of the two-month intensive phase. Imputations were not considered as the percentage of missing data was below 5%.

For the four TSQM domains, the GM score for the intervention and control arms were: Effectiveness 85.78 (GSD 1.19) vs 63.43 (GSD 1.23), Convenience 85.41 (GSD 1.17) vs 48.18 (GSD 1.33), Global Satisfaction 90.19 (GSD 1.13) vs 67.11 (GSD1.22), and Side Effects 100 for both arms (Table 2).

Table 2.

Description of three domains of TSQM items.

TSQM domains	Geometric Mean (GSD) (Overall)	Geometric Mean (GSD) (Intervention)	Geometric Mean (GSD) (Control)
Effectiveness	73.25 (1.28)	85.78 (1.19)	63.43 (1.23)
Satisfaction with treatment	4.59 (1.16)	5.27 (0.82)	3.96 (1.07)
Satisfaction with symptom relief	4.71 (1.10)	5.37 (0.79)	4.11 (0.99)
Satisfaction with time to start working	4.27 (1.32)	5.00 (1.19)	3.60 (1.05)
Convenience	63.31 (1.45)	85.41 (1.17)	48.18 (1.33)
Treatment easy to use	3.74 (1.73)	5.00 (1.28)	2.60 (1.21)
Easy planning of use	3.82 (1.77)	5.23 (1.17)	2.53 (1.10)
Intake convenience	4.57 (1.08)	5.31 (0.70)	3.89 (0.92)
Global satisfaction	77.29 (1.25)	90.19 (1.13)	67.11 (1.22)
Confidence in benefits	3.17 (0.75)	3.60 (0.53)	2.77 (0.71)
Balance between good and bad things	3.13 (0.73)	3.48 (0.64)	2.81 (0.67)
Overall satisfaction	4.78 (1.27)	5.63 (0.69)	4.00 (1.17)

GSD: geometric standard deviation.

TSQM between arms. The GM TSQM score was significantly higher in the intervention vs the control across the three domains: global satisfaction [90.19 vs 67.11, 95% CI 1.34 (1.26–1.43), p < 0.001], effectiveness [85.78 vs 63.43, 95% CI 1.35 (1.26–1.45), p < 0.001], and convenience [85.41 vs 48.18, 95% CI 1.77 (1.63–1.93), p < 0.001] (Table 3).

Table 3.

Comparison of TSQM domains between intervention and control arms.

TSQM Domains N = 109	GM	GSD	MR (95% CI)	P-value
Effectiveness
Intervention (n = 52)	85.78	1.19	1.35 (1.26–1.45)	0.001
Control (n = 57)	63.43	1.23
Convenience
Intervention (n = 52)	85.41	1.17	1.77 (1.63–1.93)	0.001
Control (n = 57)	48.18	1.33
Global satisfaction
Intervention (n = 52)	90.19	1.13	1.34 (1.26–1.43)	0.001
Control (n = 57)	67.11	1.22

GM: geometric mean; GSD: geometric standard deviation; MR: mean ratio; p-value with a cutoff of 0.05.

A forest plot of the TSQM score ratios with 95% CI bars by arms showed that the intervention arm is superior to the control arm across the three domains (Fig. 2).

Fig. 2.

Forest plot of TSQM domain by study arm. Comparison of treatment satisfaction between intervention and control arms using mean ratio scores with 95% CI across the three domains of Convenience, Effectiveness, and Global Satisfaction. The intervention arm was superior to the control arm across the three domains of treatment satisfaction.

For the linear regression fitted to determine the adjusted effect of participant characteristics on the three TSQM domains, the finding showed a statistically significant association between study arms and the TSQM's domain of Global Satisfaction, Effectiveness, and Convenience (p < 0.001). Participants in the intervention arm had significantly higher TSQM scores across all three domains compared to the scores in the control arm. There was an upward trend between treatment satisfaction and level of adherence, indicating a positive relationship between the two, though the trends were not monotonic (Fig. 3).

Fig. 3.

Trends between treatment satisfaction and treatment adherence. The relationship between the level of adherence to anti-TB medication and treatment satisfaction as it is explained by the outcome domains A) Effectiveness, B) Convenience, and C) Global Satisfaction, and a linear least squares fit with R². There was an upward trend between the level of adherence and treatment satisfaction across the three domains, indicating a positive relationship between the two, though the trends were not monotonic.

There was a significant association between medication adherence and the TSQM Global Satisfaction domain, reflecting the overall level of satisfaction or dissatisfaction with their anti-TB medication regimen (p = 0.017) (Table 4). Similarly, there was a statistically significant association between household family size and the TSQM Effectiveness domain, thus satisfaction due to the effectiveness of the anti-TB medication (p = 0.047) (Table 5).

Table 4.

Mean Ratio and adjusted Mean Ratio for global satisfaction according to participants' characteristics.

Variables	Categories (n) N = 108∗	MR (95%CI)	P value	AMR (95%CI)	P value
Arm	Control (56)	0.74 (0.69–0.79)	0.001	0.74 (0.69–0.79)	0.001
	Intervention (52)	1		1
Adherence Level	–	1.01 (1.00–1.01)	0.001	1.00 (1.00–1.01)	0.017
Sociodemographic characteristics
Sex	Women (37)	0.93 (0.85–1.01)	0.103	0.96 (0.89–1.02)	0.197
	Men (71)	1		1
Age	–	0.99 (0.99–1.00)	0.170	0.99 (0.99–1.00)	0.888
Marital Status	Married (50)	1.09 (1.00–1.18)	0.046	1.04 (0.98–1.11)	0.194
	Unmarried (58)	1		1
Occupation	No Job (35)	0.96 (0.88–1.05)	0.397
	Have Job (73)	1
Education	Below Prep. (77)	1.02 (0.93–1.12)	0.644
	Prep. & above (31)	1
No. of cohabitant	≤3 (62)	1.01 (0.93–1.10)	0.736
	≥4 (46)	1
No. of bedrooms	1 (67)	1.02 (0.93–1.11)	0.687
	≥2 (41)	1
Household head	No (47)	1.00 (0.92–1.09)	0.946
	Yes (61)	1
Residency	Permanent (88)	0.97 (0.87–1.08)	0.533
	Temporary (20)	1
Behavioral characteristics
Smoking, No/day	Never (90)	1.03 (0.92–1.16)	0.559
	Yes (18)	1
Khat	Never (87)	0.94 (0.85–1.04)	0.230
	Yes (21)	1
Alcohol	Never (67)	1.00 (0.98–1.07)	0.726
	Yes (41)	1
Disease conditions
HIV	Negative (93)	1.01 (0.89–1.14)	0.827
	Positive (15)	1
TB treatment	New (98)	0.90 (0.78–1.04)	0.167	0.94 (0.85–1.05)	0.296
	Relapse (10)	1		1
At least one -ve urine isoniazid	No (95)	1.11 (0.98–1.26)	0.103	0.98 (0.89–1.09)	0.736
	Yes (13)	1		1

MR: mean ratio; AMR: adjusted mean ratio; ∗: n = 108 as 1 missing for HIV; Prep: preparatory; the first p-value (4th column) is from the univariable analysis with a cutoff of 0.20; the second p-value (last column) is from the adjusted model (including all variables with a p-value of ≤0.20 in the univariable analysis) with a cutoff of 0.05.

Table 5.

Mean Ratio and adjusted Mean Ratio for effectiveness according to participant characteristics.

Variables	Categories (n) N = 108∗	MR (95%CI)	P value	AMR (95%CI)	P value
Arm	Control (56)	0.74 (0.69–0.79)	0.001	0.74 (0.69–0.79)	0.001
	Intervention (52)	1		1
Adherence level	–	1.01 (1.00–1.01)	0.001	1.00 (1.00–1.00)	0.186
Sociodemographic characteristics
Sex	Women (37)	0.90 (0.82–0.99)	0.038	0.93 (0.86–1.00)	0.054
	Men (71)	1		1
Age	–	0.99 (0.99–1.00)	0.077	0.99 (0.99–1.00)	0.385
Marital status	Married (50)	1.04 (0.95–1.15)	0.347
	Unmarried (58)	1
Occupation	No Job (35)	0.92 (0.84–1.02)	0.111	1.00 (0.91–1.09)	0.993
	Have Job (73)	1		1
Education	Below Prep (77)	1.04 (0.94–1.15)	0.476
	Prep & above (31)	1
No. of cohabitants	≤3 (62)	1.10 (1.00–1.21)	0.037	1.08 (1.00–1.17)	0.047
	≥4 (46)	1		1
No. of bedrooms	1 (67)	1.02 (0.93–1.12)	0.653
	≥2 (41)	1
Household head	No (47)	0.98 (0.89–1.08)	0.684
	Yes (61)	1
Residency	Permanent (88)	0.98 (0.87–1.10)	0.754
	Temporary (20)	1
Behavioral characteristics
Smoking, No/day	Never (90)	1.00 (0.89–1.14)	0.972
	Yes (18)	1
Khat	Never (87)	1.00 (0.89–1.13)	0.960
	Yes (21)	1
Alcohol	Never (67)	0.96 (0.88–1.06)	0.456
	Yes (41)	1
Disease conditions
HIV	Negative (93)	0.99 (0.86–1.13)	0.855
	Positive (15)	1
TB treatment	New (98)	0.87 (0.74–1.02)	0.081	0.93 (0.82–1.05)	0.231
	Relapse (10)	1		1
At least one negative urine isoniazid	No (95)	1.17 (1.02–1.34)	0.029	1.04 (0.94–1.17)	0.439
	Yes (13)	1		1

R: mean ratio; AMR: adjusted mean ratio; ∗: n = 108 as1 missing for HIV; Prep: preparatory; the first p-value (4th column) is from the univariable analysis witha cutoff of 0.20; the second p-value (last column) is from the adjusted model (including all variables witha p-value of ≤0.20 in the univariable analysis) witha cutoff of 0.05.

Usability of the MERM device

Usability was assessed as an additional secondary PRO of the trial. In the final complete-case ITT analysis for the usability evaluation, 52 participants in the intervention arm who also completed the two-month intensive phase completed the SUS usability questionnaire, with a substantial proportion of participants were in favor of the MERM device across the 18 items, and with 100% recommendation the MERM device to others (Table 6).

Table 6.

Description of participants’ responses to the 18-item questionnaire.

Items	Yes n (%)	No n (%)
Ease of use
Q.1. Device easy to use? (+)	52 (100)	0 (0)
Challenges
Q.2. Difficulty opening? (−)	1 (1.9)	51 (98.1)
Q.3. Needs extra demonstration? (−)	1 (1.9)	51 (98.1)
Q.4. Worried children opening? (−)	5 (9.6)	47 (90.4)
Q.5. Worried missing doses? (−)	0 (0)	52 (100)
Q.6. Worried losing the device? (−)	1 (1.9)	51 (98.1)
Q.7. Disturbed by flashing light? (−)	1 (1.9)	51 (98.1)
Benefits
Q.8. Maintained confidentiality? (+)	52 (100)	0 (0)
Q.9. Kept medication safe? (+)	52 (100)	0 (0)
Q.10. Kept medication organized? (+)	52 (100)	0 (0)
Motivation
Q.11. Motivated you? (+)	52 (100)	0 (0)
Q.12. Induce a sense of care? (+)	52 (100)	0 (0)
Popularity
Q.13. Liked the box? (+)	52 (100)	0 (0)
Q.14. Fee incapable using? (−)	2 (3.8)	50 (96.2)
Q.15. Fear consequences of missing a dose? (−)	3 (5.8)	49 (94.2)
Q.16. Family reacted positively? (+)	52 (100)	0 (0)
Q.17. Family curious? (−)	9 (17.3)	43 (82.7)
Recommendation
Q.18. Recommend the device to others? (+)	52 (100)	0 (0)

(+): Positively worded questions; (−): Negatively worded questions.

The score showed a 100% excellent usability of MERM, with an average SUS of 97.45 (4.03) close to absolute/imaginable use (SUS = 100). The LTR value was ≥9 for 90.4% of participants, yielding 90.4% net promoters, and the remaining 9.6% were neutral. With this, 90.4% of MERM device users are likely to promote the device. There was no correlation between the treatment level of medication adherence and usability scores (Table 7).

Table 7.

Assessment of MERM usability by intervention arm.

	Mean (SD); (min, max) N = 52
SUS (n = 52)	97.45 (4.03); (83.25, 100)
SUS categories	n (%)
Excellent (≥80.3)	52 (100)
Good (68–80.3)	0 (0)
Poor (51–67)	0 (0)
Awful (≤51)	0 (0)
LTR Categories	n (%)
Promoters (≥9)	47 (90.4)
Neutrals (6–9)	5 (9.6)
Detractors (≤6)	0 (0)
NPS (Promoters-Detractors)	90.4%
Correlation
SUS∗Adherence	−0.06 (p = 0.691)

Discussion

In this paper, patient-reported treatment satisfaction and usability of the MERM device for patients with TB were investigated as pre-planned secondary outcomes of a randomised controlled trial. 109 participants from the control and intervention arms completed the standard treatment satisfaction and usability questionnaires. The treatment score was significantly higher in the intervention arm compared to the control arm and across the domains of Global Satisfaction, Effectiveness, and Convenience. There was a significant association between the Global Satisfaction domain and medication adherence as well as the Effectiveness domain and the number of cohabitants. The average usability score was high and closer to the imaginable SUS value of 100%. Similarly, the LTR value was very high, yielding significantly higher net promoters to the MERM device. No association was observed between adherence and usability.

In this study, patients with TB who were assigned to receive a 15-day supply of TB medication in the evriMED500 MERM device to self-administer and return every 15 days had superior treatment satisfaction compared to those who visited the healthcare facilities each day to swallow their daily dose with DOT by health care providers. The findings also showed high usability of the device for participants. There were no noticeable differences in satisfaction or usability between different sociodemographic or behavioral characteristics. Our findings are supported to some extent by a previous non-experimental study in India that evaluated the MERM device and reported several features of the device that support acceptability among patients with MDR-TB but reported some barriers to patient use. Some of the barriers could be addressed by improving the design of the device, and some other barrier such as disease-related stigma would be more difficult to modify.³² Another operational research study in China that used secondary program data reported satisfactory uptake of the MERM device though there were specific groups less likely to use the device.³³

The study showed a positive association between treatment satisfaction and medication adherence. Treatment satisfaction is an important psychometric measurement of the degree to which a patient perceives that the treatment fulfills their health needs. Adherence, in the same sense, looked at the extent to which patients take their medications as prescribed for dosage and dosage intervals throughout the treatment period. These two domains may improve with favorable intervention strategies. Patient-reported treatment satisfaction and usability of such adherence technologies, coupled with improved adherence and treatment outcomes, could enable the global community to achieve a critical milestone on the path towards End TB.

The major goal of this trial was to facilitate and monitor adherence in a more patient-centric approach, all without affecting the dignity and economic wellbeing of patients.¹⁴ There is no single measurement strategy deemed a universal solution to improve adherence; however, DOT is the only option for the management of TB treatment in many low-income countries. There is a natural tendency to focus on patient-related domains as core determinants of adherence, while for other chronic diseases, treatment relies on patient self-management, giving them freedom and ownership of their own and their communities' health. Adherence is a multidimensional phenomenon determined by the interplay of five sets of factors (patient, therapy, condition, healthcare system, social/economic), of which patient-related factors are just one determinant.¹² The common belief that patients are solely responsible for taking their treatment is misleading and most often reflects a misunderstanding of how other factors affect people's behavior and capacity to adhere to their treatment. It also places the burden solely on the patient and stigmatizes and undermines the individual who is deemed delinquent or a defaulter when this breaks down.^6,8,10 Recognizing these barriers, efforts need to continue to provide patient-centered, ideal mechanisms that would improve patient outcomes while keeping patients' preferences, needs, and values.

This trial provided important information about treatment satisfaction and usability of a digital medication event reminder and monitor device for patients with TB in Ethiopia, one of the countries with the highest burden of TB but poorly represented in such clinical trials. However, the report has some limitations. Although standardized tools were used to measure satisfaction and usability, the tools may have missed capturing details about the outcomes of interest. Additional validation of the SUS tool should be conducted in the future in order to determine its theoretical soundness. The use of an individual participant randomised trial, instead of a pragmatic trial, may limit the findings to inform decision-making.

Patients with TB who were assigned to receive a 15-day TB medication supply in digital evriMED500 medication event reminder and monitor device to self-administer and return every 15 days had superior treatment satisfaction compared to those who visited the health care facilities each day to swallow their daily dose via directly-observed therapy administered by health care providers as the standard of care. The score showed excellent usability of the device and a significantly higher number of users likely to promote the device. More efforts are needed to continue to provide patient-centered mechanisms that would improve patient outcomes while maintaining patients' preferences, needs, and values in health decision-making. To achieve this, high TB burden countries may transform patient-centered care through ongoing evaluation and scale-up of digital health innovations.

Contributors

TM conceived the study and was the Principal Investigator. TM, VCM, YW, DPH, and AF designed the trial. TM, VCM, YW, DPH, OF, AF, TG, KB, AH, and BG made data curation. TM, TG, VCM, YW, and DPH made the analysis. TM, VCM, AF, and KB involved in funding acquisition. TM, VCM, YW, DPH, and AF interpreted the finding. TM, KB, AH, OF, and BG administered the trial. TM, TG, and OF verified the underlying data. TM drafted the initial version of the manuscript. VCM, AH, YW, and DPH critically reviewed the draft manuscript. All authors had full access to all the data in the study and accept responsibility for the decision to submit for publication. All authors read and approved the final manuscript.

Data sharing statement

The raw data and informed consent form will be shared by the corresponding author (tsegahunm@gmail.com, tsegahun.manyazewal@aau.edu.et), upon reasonable request, subject to approval by the Institutional Review Board of the College of Health Sciences, Addis Ababa University, Ethiopia.

Declaration of interests

V.C.M. has received investigator-initiated research grants (to the institution) and consultation fees (both unrelated to the current work) from Eli Lilly, Bayer, Gilead Sciences, and ViiV. All other authors report no potential conflicts.