Treatment adherence and clinical outcomes amongst in people with drug-susceptible tuberculosis using medication monitor and differentiated care approach compared with standard of care in South Africa: a cluster randomized trial

Summary

Background

Poor treatment adherence contributes to lower treatment completion and higher loss to follow-up among people with tuberculosis (PWTB). Medication monitors have shown some evidence of improved adherence.

Methods

We conducted a cluster randomised trial in 18 primary health clinics in South Africa between May 2019–February 2022. Persons (aged ≥ 2 years) with drug-sensitive tuberculosis (DS-TB) were enrolled. All participants were provided with monitors which were silent in the standard of care (SoC) arm. In the intevention arm, weekly adherence reports were reviewed and participants received intensified support as appropriate (text, phone call, home visit, motivational counselling). The primary outcome was adherence, which was calculated as days box was opened (proxy for drug taken)/total expected treatment days as a binary variable (<80% versus ≥80%). Analysis took into account clustered design. The trial was registered with the Pan African Trial Registry PACTR20190268115772.

Findings

We enrolled 2727 participants (38% women, median age 36 (IQR 27–45 years), of whom 2584 had available adherence data. The primary outcome (measured as ≥80% adherence) was higher in intervention versus SoC arm (81.0% versus 50.8%, adjusted risk ratio (ARR) 1.51 (1.36–1.66). Similarly, overall percentage adherence was higher in intervention versus SoC arm (88.5% versus 69.7%, adjusted risk difference 16.8% (13.3%–20.4%)).

Interpretation

People with DS-TB had improved treatment adherence in the intervention arm. We believe the effect on adherence is important and warrants continued use and evaluation of these technologies.

Funding

The study is funded by Bill & Melinda Gates Foundation, Uinted States, the Stop TB Partnership, Switzerland, and the South African Medical Research Council, South Africa.

Research in context.

Evidence before this study

We searched Medline and Embase in June 2018 for papers published from Jan 1, 2000, to May 1, 2019, with no language restrictions, using the terms (“digital pill box∗” OR “smart pill box∗”) AND “TB” OR “tuberculosis”. We found one systematic review of sixteen randomized control trials (RCTs) of digital adherence tools to improve adherence to tuberculosis (TB) with only two including medication monitors. One study on medication monitors showed reduced missed doses while the other showed no differences. Prior to our study, the only important evidence came from one large trial conducted in China, completed in 2012, included 4292 new pulmonary TB participants were enrolled across the 36 clusters. The adherence outcome of at least 20% of doses missed (as measured by box-opening), was lower in the in the medication monitor arms. The study was not powered for treatment outcomes, and it was found that the differentiated care aspect of the intervention was not well implemented. The only study for South Africa, was a study showed high acceptability of medication monitors has previously been described in people with drug-resistant TB. However, no study had been conducted among people with drug-sensitive TB (DS-TB) and no study had measured treatment outcomes.

Added value of this study

Our study was a large study done in routine clinics in three distinct settings in South Africa and is one of few studies that evaluated medication monitors using a cluster-randomised approach and that included follow up for recurrence. The use of medication monitors and differentiated care showed an improvement in adherence although there was no significant difference in unfavourable outcome in people with DS-TB, despite a trend to improvement in the intervention arm. There was an impact of adherence on unfavourable outcomes in some sub-cohorts e.g., women which indicates that this is potentially a particularly important intervention and should be evaluated further.

Implications of all the available evidence

We are not able to recommend medication monitors across the board for people with drug-sensitive TB at this point. Our study supports existing literature indicating improved adherence with medication monitors but failure to show a change in treatment outcomes. It may be that there needs to introduce additional interventions to improve treatment outcomes.

Introduction

Globally, tuberculosis (TB) treatment success rates among new and relapse cases have not improved over the last ten years.¹ Among the 7.5 million people with TB in 2023, almost 900,000 individuals were not successfully treated.¹ Poor treatment adherence contributes to lower treatment completion rates and to higher rates of loss to follow-up among people with TB (PWTB)^2,3 and has been associated with increased relapse of TB.⁴ In a patient-level pooled analysis of treatment shortening trials, poor medication adherence (≤90%) was associated with increased risk of poor treatment outcomes and recurrence.⁵

Digital adherence technologies (DAT) are increasingly used to improve medication adherence⁶ and treatment outcomes.⁷ Medication monitors in particular, if effective and accurate, may potentially reduce the need for clinic visits allowing for monitoring of adherence patterns from a central location.⁸ The newer adherence devices enable real-time monitoring of pill-taking, generating more detailed adherence histories which can be actioned more quickly, and allowing for a differentiated use of limited human resources for managing challenging PWTB. The World Health Organisation (WHO) updated treatment guidelines currently includes a conditional recommendation, with very low certainty of evidence, to offer medication monitors and tracers (such as mobile phone short message service (SMS)) for TB treatment adherence support.⁹

Previous evidence for effectiveness of digital monitors mainly come from two large trials conducted in China. The first trial, which was completed in 2012 and included 4292 new pulmonary TB participants across the 36 clusters, showed that poor adherence (measured as at least 20% of doses missed, as measured by box-opening), was lower in the in the medication monitor arms.⁸ In the more recent Chinese trial, the medication monitor intervention had no effect on unfavourable outcomes (adjusted risk ratio 1.01, 95% CI 0.73–1.40), but non-adherence was reduced by 57–64% in the intervention group compared with the control group,¹⁰ although there were concerns about a failure to change patient management following identification of treatment non-adherence at monthly reviews. Other evidence from Ethiopia, includes one individually randomised trial of 337 individuals which showed superior adherence (using different measures), but no change on smear conversion, among participants assigned to monitor-observed self-administered therapy when compared with the standard in-person facility-administered DOT.¹¹ More recently published smaller studies from Tibet (n = 278) and Peru (n = 106), indicate improved treatment outcomes with use of digital^12,13 with improved adherence in one (Tibet) and no change in adherence in the other (Peru). The Tibet study did include a more comprehensive package of interventions with electronic monitors, family treatment supporters and improved communication through a linked smartphone app. Thus to date, although some smaller programmes and with more intensive adhrence support have shown some benefit, large programme implementation of DAT had failed to show consistent effect on treatment outcomes.

South Africa, a country with a high estimated TB incidence rate (468/100,000 population), reported treatment success rates of 79% among new and relapse PWTB, 79% among HIV-positive PWTB and 62% among people on rifampicin resistance TB treatment,¹ falling short of targets of 90% treatment success of all forms of TB in the Global Plan to End TB (2023–2030).¹⁴ The acceptability of medication monitors has previously been described in people with drug-resistant TB in South Africa.¹⁵ However, no study had been conducted using adherence monitors among people with drug-sensitive TB (DS-TB). In addition, no study has evaluated treatment outcomes, including recurrence, in patients using the real-time medication monitors which allow for differentiated care in South Africa. We conducted a pragmatic trial with the aim to evaluate an adherence monitoring system with a differentiated response to patient care, among people with DS-TB in three provinces in South Africa.

Methods

The study was a parallel cluster-randomised trial conducted across 18 primary health care facilities in three provinces in South Africa (Ekurhuleni district in Gauteng, Klipfontein and Mitchell’s Plain districts in Western Cape, and eThekwini district in Kwa-Zulu Natal) and was described fully previously.¹⁶ Clusters were public health clinics with at least 200 TB registrations in 2017 and included six clinics per province. Adult HIV prevalence in the general population was 18.2% in Kwa-Zulu Natal, 12.5% in Gauteng and 8.9% in Western Cape province.¹⁷ Standard of Care (SoC) in South Africa was mostly self-administered treatment with directly observed treatment provided to people thought to have risk factors for poor adherence (e.g., homeless, substance abuse history etc). In the Western Cape province, it was mostly self-administered treatment except for the first two weeks of treatment that were directly observed.

The study enrolled adults aged ≥ 18 years and children aged 2–17 years with clinically or microbiologically diagnosed DS-TB, satisfying inclusion criteria of: having initiated TB treatment within the last 7 days at the time of enrolment; willing to use the medication monitor as directed; agreeing to be followed-up with text messaging; phone calls and home visits; living within the study catchment area; and willing to inform the study team of any change of address during the treatment as well as follow-up period. Participants (or caregivers in the case of children) were also required to have access to a mobile phone and no phones were given to participants. Participants were excluded if: they were diagnosed with drug-resistant TB; they were not fluent in the languages in which the informed consent was provided or study members were not able to communicate; those who do not have access to or were not able to use a mobile phone or to read SMS text messages; people unable to use the device after training, people who expect to leave the study area, people taking part in the other investigational product or device trials related to TB and/or lung diseases. The inclusion criteria for initiating TB treatment were later revised to within 14 days as patients that were initiated on TB treatment in hospitals were referred to continue with their treatment in primary health facilities having been on treatment for more than 7 days.

Randomisation and masking

Clusters were randomised 1:1 to intervention or SoC arm using restriction to ensure a difference in clinics per arm in each of province of no more than one. Randomisation allocation sequences were done by the study statistician using STATA v16. Following the randomisation but before participant enrolment had started, two clinics from the same province, one in each arm, were withdrawn due to ongoing TB studies. The two clinics were replaced by another two clinics in the same province and were randomly allocated to the intervention and control arm. Research staff were placed at each of the 18 clusters to enrol participants into the study. There was no masking of the intervention. Both providers and participants were aware of the randomisation.

Procedures

Participants enrolled from clinics in the intervention arm received a differentiated care adherence package. The package included standard patient education and provision of the Wisepill EvriMED medication monitor, wherein treatment blister packs were placed, and which were programmed to have daily visual or audio alerts for treatment intake and monthly reminders for treatment refills. If a PWTB missed one dose, they received an automated SMS stating, ‘Please remember to take your medication’. A second or third missed dose in a week, required research staff to initiate a phone call to the participant, and four or more missed doses in a week required a home visit. Research staff were provided with scripts to use for the phone calls and home visits. If four or more doses were repeatedly missed, motivational counselling was initiated. When participants went to the clinic for their routine monthly dispensing visit, the research staff showed the participant their own data and discussed their medication adherence history.

Participants enrolled from clinics in the SoC arm received counselling regarding their TB treatment and were given a return date for collection of repeat medication in 30 days. Participants were also provided with the medication monitor wherein treatment blisters packs were placed and educated on its use by the research staff. The medication monitor did not provide any visual or audio reminders for intakes or refills but recorded data on box opening in real-time just as in the intervention arm. In contrast to the intervention clinic participants, data collected on the system were not reviewed during routine monthly follow-up visits by research staff or participants. In both arms, the medication monitor transmitted a daily “heartbeat” signal to the system to indicate that it is working properly. If not working, health care workers did assist participants in both arms to ensure the monitor was functioning and able to send signals.

For study visits, participants in both arms were followed up by the research team passively each month during TB treatment. As per usual care, facility staff documented end of treatment outcomes without any interference by our research staff.

Following the end of treatment, participants in both arms who had cured or completed treatment were followed-up every three months in person (except during the COVID-19 lockdown period) for 12 months. During this follow-up period, participants were determined as having “died” if research staff were informed by their contacts as having died or “lost to follow-up” if they could not be found after multiple phone call or home visits attempts where possible to them or their contacts. At each follow-up visit, TB symptom screening was performed and, if symptomatic, a sputum sample for Mycobacteria Growth Indicator Tube (MGIT) culture and Xpert MTB/Rif (as per routine) was collected. All participants were requested to provide a single sputum specimen for MGIT culture testing at 6 months from enrolment and 12 months post TB treatment completion (18 months from enrolment), regardless of symptoms. Participants were reimbursed ZAR 50 (approximately US$3) for travel costs for each visit after the end of treatment as these were deemed research visits.

Outcomes

Outcomes were described in the protocol prior to start. The primary study endpoint was adherence to TB medication, which was measured as a binary outcome of percentage adherence ≥80% over the whole period, as measured by box opening. For those lost to follow-up during treatment, we assumed no drug intake (100% non-adherence) for the period from the date of last contact to the date of scheduled treatment completion. TB medication adherence was also calculated as a continuous variable, by arm, defined as the percentage adherence over the entire treatment period.

Secondary study endpoints included poor outcome at the end of treatment and unfavourable outcomes at 18 months post enrolment. Poor end of treatment outcome was defined as death, lost to follow up, treatment failure (including positive culture on the six-month sputum) and diagnosis of rifampicin resistant TB. Transfer outs and where an outcome was not documented (and no negative culture at end of treatment) were coded as not assessable. Unfavourable outcome at 18 months after enrolment included: on-treatment lost to follow up, death, treatment failure, diagnosis of multiple drug resistant TB (MDR-TB) and treatment recurrence. Treatment recurrence was defined as a participant having a positive TB culture result (either as part of the trial or in routine care) or restart of TB treatment at any time during the 12 months follow-up period post TB treatment completion. All analyses were conducted using the intent to treat population, defined as all participants enrolled on the study, excluding those with diagnosis changed to not TB or MDR TB, those incarcerated in follow-up and those who no longer wished to participate. Multiple imputation (25 imputations) was conducted for individuals who had cured or completed treatment, had not met the recurrence endpoint and who were not seen at 18 months (either due to lost to follow-up or death). Two sensitivity analyses were also done: 1) participants who had a treatment outcome of cured or completed treatment, but who had been on treatment for >10 months (280 days), defined as having as poor end of treatment outcome; and 2) participants who had a treatment outcome of success or completed treatment, but who had died between end of treatment and 18 months, defined as an unfavourable outcome.

Statistics

Sample size calculations were conducted accounting for the clustered design and assumed a harmonic mean of 145 participants/cluster, nine clusters per arm and a two-sided type I error of 5%. For the primary outcome (adherence) we assumed the percentage with adherence less than 80% in the SoC arm of 30% and coefficient of variation of 0.25, had 90% power to detect a 40% relative reduction in the endpoint in the intervention arm. For a successful outcome at the end of treatment, we assumed 80% successful outcome in the SoC arm and a coefficient of variation of 0.06, we had 90% power to detect an increase to 90%. For unfavourable outcome 18 months after enrolment, we assumed 13–20% in the SoC arm and a coefficient of variation of 0.25, we had at least 80% power to detect a 40% relative reduction in the intervention arm. Allowing for some clinics to enrol more than others, we did allow enrolment of up to 170 per clinic. The STATA “clustersampsi” command was used for the sample size calculations.

Analysis was conducted at the cluster-level due to the small number of clusters.¹⁸ For each cluster, the proportion of participants with <80% adherence was measured. Our main effect estimate is a risk ratio based on the natural logarithm-transformed risks, compared by study arm across clusters using a t-test. For all binary outcomes, we reported by study arm the overall risk, ignoring clustering, and the geometric means of cluster-level risks. We also conducted an adjusted analysis for the intervention effect, adjusting for imbalances of individual-level variables at baseline, using a two-stage approach. Using logistic regression, the expected outcome for each individual was calculated, adjusting for baseline imbalances and summed at the cluster-level. The log of cluster-level residual (expected number of outcomes with the observed number of outcomes) was compared by study arm using a t-test. Risk differences and associated 95% confidence intervals by study arm were reported, based on untransformed cluster-level risks. Prespecified subgroup analyses were conducted without control of the overall type I error rate. All analyses were conducted in STATA v16 using the clan command.¹⁹ No interim analysis was performed.

For fidelity of the intervention, we opted to measure the required phone calls (since home visits were not done during COVID time) and how many were attempted and the number of circumstances where the person was successfully contacted.

Ethics

Written informed consent was obtained from all adults. For children under 18 years, caregivers gave informed consent and in children aged 7–17 years old informed assent was also obtained. The trial obtained approval from Wits Human Research Committee (Ref 180,705), Johannesburg, South Africa; University of Cape Town Human Ethics Research Committee (Ref 452/2018), Cape Town, South Africa; and the London School of Hygiene & Tropical Medicine (Ref 16,107), London, United Kingdom. The study also obtained approval from the three district health departments: eThekwini, Ekurhuleni, and City of Cape Town. The trial was registered with the Pan African Trial Registry PACTR201902681157721, registered on 11 February 2019.

Role of funding source

Funders did not play a part in the design of the study or the decision to submit the manuscript for publication.

Results

From 17 May 2019 to 31 December 2020, we enrolled 2727 participants (38%, 983 women; median age 36 years, Interquartile range 27–45 years) from 18 facilities (10 community health centres and 8 primary health clinics): 19 (0.7%) participants were incorrect enrolments and 51 (1.9%) withdrew their participation in the study, leaving 2657 individuals (Fig. 1). The COVID pandemic and subsequent restrictions imposed interrupted enrolment and reduced home visits. Recruitment was paused from March–June 2020. In addition, home visits were replaced with phone calls between March and July and also in other lockdown periods following July 2020 depending on the levels of restrictions. Facilities closures days varied between 0 and 22 days from March 2020–February 2021. Follow up was done for at least 12 months and up to 18 months after enrolment for all participants, completing on the 28 February 2022.

Fig. 1.

Consort diagram for adherence population. n = number; MDR = multi-drug resistance; TB = tuberculosis; d = days.

For the primary outcome of adherence, we excluded a further 73 (2.7%) participants as they did not have adherence data, leaving 2584 (38% females; median age 36 years) for analysis (Fig. 1, Table 1). For the end of treatment outcome, 2538 participants (95.5%) were included (119 were not assessable: 2 missing treatment outcome and 117 transferred out), and for the unfavourable outcome at 18 months, before imputation, 2070 participants (77.9%) had their outcome known and 587 were not assessable (115 transferred out, 33 died and 438 lost to follow-up from the end of treatment). The CONSORT diagram and baseline characteristics for cohorts for the two secondary outcomes of end of treatment and unfavourable outcome at 18 months is available in the Supplementary Appendix (Supplementary Figs. S1 and S2 and Table S1).

Table 1.

Baseline characteristics of population included for adherence outcome (n = 2584).

Variable	Intervention		SoC		Total
Number of clusters	9		9		18
Type of clinic
Primary health clinic	4		4		8
Community health centre	5		5		10
Number of participants	1306		1278		2584
Province
Gauteng	460	35.2%	316	24.7%	776	30.0%
Kwa-Zulu Natal	378	28.9%	469	36.7%	847	32.0%
Western Cape	468	35.8%	493	38.6%	961	37.2%
Age, median (IQR)
Years	36	(27–45)	35	(27–45)	36	(27–45)
Agea
<18 years	107	8.2%	65	5.1%	172	6.7%
Sex
Female	481	36.8%	502	39.3%	983	38.0%
TB diagnosis
Bacteriologically positive	1035	79.4%	893	70%	1928	74.4%
Country of origin
South African	1273	97%	1217	95.3%	2559
Ethnic group
Black African	1165	89.2%	1218	95.3%	2383	92.2%
Education
≤Grade 7	264	20.2%	270	21.1%	534	20.7%
Grade 8–11	557	42.6%	545	42.6%	1102	42.6%
≥Grade 12	485	37.1%	463	36.2%	948	36.7%
Marital status
Single	924	70.8%	977	76.4%	1901	73.6%
Married/cohabitating	320	24.5%	241	18.9%	561	21.7%
Separated/divorced	30	2.3%	36	2.8%	66	2.6%
Widowed	32	2.5%	24	1.9%	56	2.2%
Hospitalised
Yes	109	8.3%	118	9.2%	227	8.8%
Previous TB
Yes	315	24.1%	297	23.2%	612	23.7%
Ever smoked
Yes	416	31.9%	421	32.9%	837	32.4%
Alcohol consumption
Never	788	60.4%	818	64.0%	1606	62.2%
Monthly or less	221	16.9%	177	13.8%	398	15.4%
Between 2 and 4 times/month	145	11.1%	188	14.7%	333	12.9%
Between 2 and 3 times/week	58	4.4%	56	4.4%	114	4.4%
≥4 times/week	93	7.1%	39	3.1%	132	5.1%
Missing	1				1
Recreational drug use
Yes	99	7.6%	79	6.2%	178	6.9%
HIV statusb
HIV positive -no ART	257	19.8%	264	20.7%	521	20.3%
HIV positive -on ART	320	24.7%	512	40.2%	832	32.4%
HIV negative	719	55.5%	497	39.0%	1216	47.3%
Missing	10		5		15

SoC = Standard of Care; IQR = interquartile range; HIV = human immunodeficiency virus; ART = antiretroviral treatment.

^a70/107 and 36/65 were aged < 13 years in the intervention and SOC arms, respectively.

^bOne cluster in the standard of care arm with different population and lower HIV prevalence resulted in imbalance by study arm for HIV status.

For the primary outcome analysis population, most of our participants were adults (93.3%; 2412), Black African (92.2%; 2383) and single marital status (73.6%, 1901) and bacteriologically positive either on MGIT culture, smear or Xpert (74.7%; 1928). Over half (52.7%, 1353/2569) of participants were HIV-positive as tested by routine services, of whom, 832 (61.5%) were on ART. Baseline variables were in the most part similar by study arm, except for a higher proportion with bacteriological diagnosed TB and HIV negative in the intervention versus SoC and lower proportion of HIV positive on anti-retroviral therapy (ART) and those of African descent, in the intervention versus SoC arm (Table 1).

The proportion of participants with ≥80% adherence was higher 1056/1306 (80.9%, GM 81.0%) in the intervention arm compared to 650/1278 (51.6%; GM 50.8%) in the SoC [Adjusted risk ratio of 1.51 (95% CI 1.36–1.66) (Table 2, Supplementary Fig. S3). For the continuous adherence measure, the overall mean proportion adherence was higher 88.5% in intervention clusters compared to 69.7% in SoC clusters, giving a risk difference of 16.8 (95% CI 13.3–20.4) (Table 2, Supplementary Fig. S3). Looking at proportion of adherence months, the overall mean proportion with <80% adherence was 17.5% in intervention clusters compared to 43.0% in SoC clusters giving a risk difference of 25.4 (95% CI 19.5–31.3). Subgroup analyses for the primary adherence outcome showed improvement in adherence in all subgroups in the intervention arm (Fig. 2). The coefficient of variation for the primary outcome was 0.21.

Table 2.

Comparison of study endpoints of adherence, end of treatment and overall poor outcome in the SoC versus intervention arm.

	SoC n/N (GM%)	Intervention n/N (GM%)	Risk ratio (95% CI)a	P value
Primary outcome: adherence ≥80%b	650/1278 (50.8%)	1056/1306 (81.0%)	1.50 (1.36–1.66)	<0.001
Secondary outcomes:
	SoC mean %	Intervention mean %	Mean difference (95% CI)a	P value
Overall % adherence	69.7%	88.5%	16.8% (13.3–20.4%)	<0.001
Secondary: percentage of months with <80% adherence	43.0%	17.5%	25.4% (19.5–31.3%)	<0.001
	SoC n/N (GM%)	Intervention n/N (GM%)	Risk Ratio (95% CI)a	P value
Poor end of treatment outcomec	172/1259 (13.4%)	176/1279 (11.0%)	0.82 (0.56–1.22)	0.31
Unfavourable outcome by 18 monthsd—complete case	216/974 (22.3%)	216/1096 (17.1%)	0.78 (0.53–1.16)	0.21
Unfavourable outcome by 18 monthsd—MI	241/1261e (19.1%)	234/1281e (15.5%)	0.83 (0.56–1.22)	0.32

SoC = Standard of Care; GM = geometric mean of percentage with outcome at the cluster-level; CI = confidence interval; MI = multiple imputation.

^aAdjusted for age group, sex, TB diagnosis, ethnic group, marital status, HIV/ART status and province, comparing intervention versus control.

^bRisk difference 26.1 (95% CI: 20.1%–32.1%).

^cTreatment failure, death, lost to follow-up or switched to an MDR regimen.

^dComposite outcomes of poor end of treatment outcome or recurrence/restarting TB treatment by 18 months.

^eDenominators excluded 66 and 49 participants who transferred out during treatment in the SoC and intervention arm, respectively. Numerator is the arithmetic mean of total number of unfavourable outcomes across the 25 imputations.

Fig. 2.

Forest plot for subgroup analysis for primary adherence outcome (unadjusted). SoC = Standard of Care; Int = Intervention; CI = confidence interval; HIV = human immunodeficiency virus; gr = grade; ART = antiretroviral; Bact = bacteriological; CI = confidence interval; GP = Gauteng province; KZN = Kwazulu Natal province; WC = Western Cape province. The box represents the risk ratio; and the horizontal line through the box is the 95% confidence interval.

The proportion with poor end of treatment outcomes was similar across the arms with 176/1279 (13.7%; GM 11.0%) in the intervention versus 172/1259 (13.7%; GM13.4%) in SoC arm, adjusted risk ratio 0.82 (95% CI 0.56–1.22) and adjusted risk difference −0.4% (95% CI -6.0% to +5.0). In the complete case analysis, for the composite unfavourable outcome at 18 months, proportions were lower in 216/1096 (17.1%) in intervention arm compared to 216/974 (22.3%) in the SoC arm however the effect was not significant with an adjusted risk ratio of 0.78 (0.53–1.16) (Table 2, Supplementary Fig. S3). Following multiple imputation for missing outcomes, the adjusted risk ratio was 0.83 (0.56–1.22).

Reasons for the poor end of treatment outcomes and composite unfavourable outcomes at 18 months are shown in Table 3. Approximately 20% of unfavourable outcomes were after the end of treatment. The most common outcomes were on treatment loss to follow up (n = 149) and deaths (n = 129) which make up 64% of all unfavourable outcomes. The proportions of, and reasons for poor end of treatment outcome per facility are shown in the Supplement (Supplementary Fig. S4a and b). Some clinics had high levels of on-treatment loss to follow-up which contributed to high proportion of poor outcomes during treatment and after end of treatment. A post-hoc sub-group analysis showed the intervention reduced the unfavourable outcome at 18 months among women and in the Western Cape cohort (Supplementary Fig. S5).

Table 3.

Components of the composite unfavourable outcome (poor end of treatment and after end of treatment outcomes) by arm.

	SoC	%a	%a	Intervention	%a	%a
Total unfavourable outcomes	216			216
Poor end of Treatment outcome	172	79.6%		176	81.5%
Treatment failure	30		13.9%	30		13.9%
Lost to follow up	81		37.5%	68		31.5%
Died	57		26.4%	72		33.3%
MDR diagnosis	4		2.3%	6		2.8%
After end of treatment	44	20.3%		40	18.5%
MDR TB diagnosis	2		0.5%	2		0.9%
Culture positive	22		10.9%	23		10.6%
Restarted treatment	17		7.9%	12		5.6%
Positive TB sputum identified	3		1.4%	3		1.4%

SoC = Standard of Care; MDR = multi-drug resistant TB; TB = tuberculosis.

^aPercentage among those with unfavourable outcome.

The sensitivity analysis that defined those who were on treatment for >10 months (280 days) as having an unfavourable outcome, showed that the proportions with poor end of treatment outcomes seemed to be higher in the SoC arm 202/1259 (13.7%; GM 15.6%) versus 197/1279 (15.4%; GM 11.9%) in intervention arm but there was no significant difference, with an adjusted risk ratio of 0.79 (0.54–1.18). The composite unfavourable outcome at 18 months, similarly it was higher in the SOC arm: 242/974 (24.8%; GM 24.8%) versus 234/1096 (21.6%; GM 18.1%) in intervention arm, but also with no effect, adjusted risk ratio of 0.76 (0.52–1.12) (Supplementary Table S2). Similarly, for the sensitivity analysis where those who had died between end of treatment and 18 months were defined as having an unfavourable outcome, the proportions with unfavourable outcome as 234/992 (23.6%; GM 23.6%) in SoC and 231/1111 (20.8%; GM 18.2%) in the intervention arm, with an adjusted risk ratio of 0.79 (055–1.14) (Supplementary Table S2). Correlation between adherence and unfavourable outcome was examined as in Supplementary Table S4. There does appear to be a relationship as those with lower adherence show a higher proportion with poor outcomes. We show some measures of fidelity with regards to phone calls in Supplementary Table S6, that show a high proportion (78–81%) of episodes where a phone call was required, were successfully contacted.

Discussion

Among people treated for DS-TB, use of DAT with differentiated care resulted in improved treatment adherence when using a proxy measure of “days the medication monitor was opened” in the intervention versus SoC arm, when measured as either the proportion of participants with poor adherence (<80% versus ≥80%), or the secondary outcomes of overall adherence and percentage months with <80% adherence. Although adherence was improved, there was no significant difference in end of treatment outcome or unfavourable outcome at 18 months among people treated for DS-TB in the intervention versus SoC arms. There was some inbalance regarding HIV status and ART by arm which could have accounted for better adherence (lower numbers with HIV) and poorer treatment outcomes (lower % with HIV on antiretrovirals) in the intervention arm, although adjustment for HIV/ART status was conducted. These results are very consistent to the cluster-randomised trial done in China¹⁰ where adherence was improved but did not lead to improved treatment outcomes or reduced recurrence, but differ from an individually randomised trial in Tibet, which used a treatment supporter and digital device (pillbox and video supported therapy for those with adherence challenges), and showed both improved adherence and treatment outcomes.¹² In addition, pragmatic cluster randomised trials conducted in four countries under the same protocol showed no improved treatment outcomes using digital adherence tools.²⁰

While these results may seem counterintuitive, there are studies in other fields which show improved adherence yet fail to show a difference in biological outcomes.^21,22 The delinking of adherence and outcome is likely due to either the inaccuracy of the adherence measurement or the insensitivity of the TB treatment outcome measures. As found in the qualitative work where people reported opening the box to deal with the nuisance of reminders, it may be that in the SoC arm, box openings may have been less frequent since there were no reminders and no consequence to not opening the box in this group.²³ The problem is that there is no gold standard for measuring TB adherence. Evidence from HIV studies can be considered to understand how measurements using box openings may be a good measure of adherence as viral load suppression is a much more reliable indicator of adherence to treatment. A recent Cochrane review,²² that compared electronic medication monitoring against viral load suppression in people living with HIV, found only 3 studies, with a total of 186 participants. Sensitivity of the adherence measured through box openings ranged from 60% to 88% and specificity ranged from 27% to 67%. In a recent study of 198 people living with HIV with MDR-TB, modeling identified a significant (P < 0.001), linear association between ART adherence and emergent HIV resistance, suggesting a strong association without a specific threshold.²⁴ Further studies to understand the relationship between medication openings and adherence through testing of metabolites may assist in understanding the data.

TB treatment outcomes can be insensitive^25,26 and may overestimate treatment success in a routine programmes, especially in the new TB era of molecular diagnostics (and a higher proportion of smear negative TB on treatment), where treatment success is mostly made of treatment completion rather than cure. Clinical trials use more robust outcomes²⁷ that include multiple cultures and post treatment followup and that may be why there was a more a more direct association between adherence and treatment outcomes identified in TB treatment trials. In our pragmatic study, we collected one sputum culture at month 6 and another at month 18 which may not have been sufficient to detect progression to treatment failure or relapse. In addition, due to operational difficulties, COVID-19 disruptions and participants being unable to produce sputum, we were only able to determine bacteriological outcomes in 30.8% at 6 months and 29.1% at 18 months, speaking to the difficulties with outcomes definitions. A study conducted among people with DR-TB showed that bedaquilline adherence through 6 months independently predicted end of MDR-TB treatment outcome,²² and it may be that more intensive bacteriological monitoring in routine DR-TB programmes may have had a different result.

As with HIV,²⁸ successful TB treatment outcomes may not require perfect treatment adherence for successful treatment outcomes, i.e., the treatment regimen may be more “forgiving” than originally assumed or the timing of non-adherence may play a role. Since the end of the DOTs era, there has been little data exploring TB treatment adherence, and potentially the lower adherence seen in our SoC arm may be sufficient for a TB cure. Historically there were regimens, particularly in the continuation phase, that were thrice weekly or at least not daily (5 days per week),²⁹ therefore the lack of improvement may reflect the forgiving nature of the regimen for most patients. The analyses of the adherence data of the treatment-shortening trials, as well as the correlation of adherence and outcomes, may however contradict this.⁵ Further analysis to understand whether patterns of adherence and timing of poor adherence are related to outcomes are planned.

We have considered whether the discordance between our adherence and treatment outcomes, could be that the study population (and therefore the control) may have been less “hard-to-treat” (as described by Imperial et al.) which refers to PWTB with smear positivity and cavitatory lung disease, who do not respond well to treatment.⁵ As our study had very few exclusion criteria and included all people with drug susceptible PWTB in outpatient clinics, we think this is unlikely. Compared to trial participants, we did have a lower proportion with smear positive disease (70% versus 90%) and although we didn’t have chest x-ray findings, we had a higher proportion of participants who were HIV positive in our study population (53% versus 16%), indicating that we may have had less cavitatory disease.

When considering whether this intervention should be implemented, consideration of issues of feasibility and acceptability are important. The published work from this study on qualitative measures such as feasibility and acceptability have shown that the medication monitors are very well received by people with TB and health care providers^23,30 and that may indicate that since there was no harm caused by the intervention, it may still be worth considering. Further supporting evidence for acceptability have also been published from studies including Ethiopia, Tanzania, and Philippines.³¹

A major strength of our study was that it was a large-scale implementation trial and one of the first trials in the world to measure TB adherence and TB treatment outcomes among patients treated for DS-TB using real-time adherence monitoring. Strengths include: equal representation of sexes; cluster randomised design that allowed comparison in a more routine setting; the documentation of adherence using the silent monitors in our control sites and the follow up of patients after treatment completion. The study involved three regions of South Africa, with varied TB disease burden, health system infrastructure and socio-economic factors that influence behaviour, so demonstrating flexibility and adaptability and genaralisability of the intervention. Our study was complemented with qualitative and economics research which allows a more holistic evaluation of DAT.²³ Our initial data indicate that the differentiated care intervention was implemented with high fidelity.

Weaknesses include the relatively small number of sites. We also relied on standard measurement of outcomes although we did attempt to include TB cultures, the sputums were often not collected at the six and eighteen month points, and loss to follow up of participants. Another weakness of the study was that our adherence data was not supported by the measurement of another objective adherence measure e.g., INH urine metabolites.²⁴ The pragmatic nature of the trial was a strength as it allowed for real-world implmentation but it also meant that problems such as COVID-19 clinic shut-downs and social unrest impacted our measurements of outcomes and increased loss to follow up, although this would have affected both arms. There were some facilities in high crime areas where clinic closures were common for safety reasons and which prevented home visits where these would have been required.

In conclusion, our trial supports further evaluation of digital adherence tools for adherence support for TB as we did demonstrate an improved adherence by using the medication monitor and differentiated care package in people with DS-TB in South Africa. Although there was an improved adherence, there was no significant difference in unfavourable outcomes in people with DS-TB. The reasons for the limited impact of improved adherence on clinical outcomes are likely complex, may be related to patterns of adherence and differential use of the medication monitor between the arms is possible. If the impact increased adherence on unfavourable outcomes in some sub-cohorts, for example among females is real, this is potentially a very important intervention. Also, there might be a need to further improve support by allowing for more patient-centred care with more involvement of patients/healthcare workers to co-develop interventions.

Contributors

KLF, SC, PN, CO, NM, KV and CMCM conceptualised the study and interpreted the study findings.

RM, NX, LM, PH, contributed to study implementation and collection of data.

KLF and LM verified the data.

KLF analysed the data.

SC an NM wrote the original draft and revised subsequent versions of the manuscript.

NM, CO, LJ, PN, KV, KLF, SC, IR, DC, RM, LM, PH, NX and CMCM reviewed and edited previous versions.

All the authors read and approved the final manuscript.

Data sharing statement

Study protocol, Statistical analysis plan and data will be available upon submission of a request to the Aurum Data Governance committee.

Declaration of interests

CO received honoraria from MSD in Dec 2022 and November 2023 for consultation at their Africa ART meetings. PN has received support from BMGF to attend the Union Conference on Lung Health and SA TB Conference. All other authors declare that they have no competing interests.