Effect of Directly Observed Therapy for Highly Active Antiretroviral Therapy on Virologic, Immunologic, and Adherence Outcomes: A Meta-Analysis and Systematic Review

Abstract

Introduction

Directly observed therapy of highly active antiretroviral therapy (DOT-HAART) is a feasible adherence intervention. Prospective DOT-HAART studies have shown mixed results, and optimal target groups have yet to be defined. We performed a meta-analysis and systematic review to assess the effect of DOT-HAART on adherence and virologic and immunologic response.

Methods

We performed a comprehensive search through August 2009 to identify peer-reviewed controlled studies that involved outpatient DOT-HAART among adults and reported at least 1 outcome assessed in this meta-analysis. Random-effects meta-analyses were performed; differences in effect on virologic suppression were examined using stratified meta-analyses and meta-regression on several study characteristics.

Results

Seventeen studies met inclusion criteria. Compared with control groups, DOT-HAART recipients were more likely to achieve an undetectable viral load (random effects risk ratio 1.24, 95% confidence interval (CI): 1.08 to 1.41), a greater increase in CD4 cell count (random effects weighted mean difference 43 cells/μL, 95% CI: 12 to 74 cells/μL), and HAART adherence of ≥95% (random effects risk ratio 1.17, 95% CI: 1.03 to 1.32). Results varied with respect to virologic response. DOT-HAART did not have a significant effect on virologic suppression when restricted to randomized controlled studies. Post-treatment effect was not observed in a limited number of studies.

Conclusions

DOT-HAART had a significant effect on virologic, immunologic, and adherence outcomes, although its efficacy was not supported when restricting analysis to randomized controlled trials. DOT-HAART shows greatest treatment effect when targeting individuals with greater risk of nonadherence and when delivering the intervention that maximizes participant convenience and provides enhanced adherence support. Further investigation is needed to assess the postintervention effect and cost-effectiveness of DOT-HAART.

INTRODUCTION

Highly active antiretroviral therapy (HAART) is standard of care for individuals infected with HIV.^1–4 Strict adherence to treatment is required to achieve optimal clinical responses.^5–10 Unfortunately, nonadherence is common among HIV-positive patients because of the life-long nature of HAART,^11,12 adverse events,^13–15 and numerous psychosocial and economic stressors.^11,16–21 Interventions to improve HAART adherence vary widely and often include education and counseling,^22–25 patient reminders,^26,27 behavioral therapy,^27,28 and social support.^28–30 Other interventions promote combined strategies.^31–33 Recent meta-analyses show that individuals receiving an adherence intervention are more likely to achieve 95% adherence than those receiving standard of care across a broad range of intervention designs.^34,35

Directly observed therapy (DOT) has been the cornerstone of a strategy endorsed by the World Health Organization to improve tuberculosis treatment adherence and outcomes worldwide.^36–38 As early as 1996, HIV providers considered the utility of DOT for HAART (DOT-HAART).³⁹ Critics of DOT-HAART have voiced concerns about the feasibility of applying DOT to life-long treatment, the acceptability of DOT given confidentiality concerns and HIV-related stigma, and the potential threat of generating excess drug resistance.⁴⁰ Conversely, proponents have endorsed DOT-HAART because of its ability to provide intensive support to otherwise hard-to-reach HIV-infected populations.⁴¹ Although this debate persists,^32,42 recent data demonstrate that DOT-HAART is feasible, acceptable, and does not seem to increase the risk of drug resistance among participants.^{30,32,43–50} As a result, DOT-HAART has gained increasing recognition as an important antiretroviral adherence strategy.⁵¹ Unlike many other HAART adherence interventions, DOT-HAART has been successfully “test-driven” in real-world settings and has been delivered to more than 12,000 individuals to date.^{28,44,51–69} However, efficacy data from controlled trials are mixed, and interventions vary widely in terms of the nature of DOT-HAART (eg, site and frequency of DOT, additional support provided, DOT worker background); target populations (eg, substance users, HAART-naive resource-poor settings); and assessment (eg, duration of follow-up, outcomes).⁵¹ The question of translating evidence into implementation⁷⁰ is not “can DOT-HAART be implemented,” but rather, “should it, how, and for whom”?

Recognizing the growing attention toward DOT-HAART and the need to synthesize findings across a diverse array of studies, Ford et al⁷¹ recently conducted a systematic review and meta-analysis of DOT-HAART randomized clinical trials (RCTs). Their analysis did not show an intervention effect on virologic suppression at study completion, although benefit was observed among individuals at high risk of nonadherence and among trials with DOT lasting less than 6 months. The moderate heterogeneity observed among studies and the identification of certain treatment characteristics which may confer greater treatment effect invite further exploration of the growing experience of DOT-HAART.

We sought to expand the current scientific knowledge of DOT-HAART by performing a meta-analysis and systematic review of controlled DOT-HAART trials that differs from that of Ford et al⁷¹ in several important respects. First, we included nonrandomized studies in recognition of the complexity and flexibility of many DOT-HAART interventions, which have adapted over time in response to community needs. Second, we differentiated between on-treatment and post-treatment effects, rather than pooling measures conducted at study completion. Finally, we investigated the modifying effect of several aspects of intervention design and target population that were not considered by Ford et al.⁷¹

METHODS

Search Strategy

We followed PRISMA⁷² and MOOSE⁷³ guidelines in this systematic review. We extensively searched the following databases to identify controlled studies that described the provision of HAART as directly observed: MEDLINE via PubMed, the Computer Retrieval of Information on Scientific Projects database, www.clinicaltrials.gov, www.controlled-trials.com, and Google Scholar from 1995 to August of 2009 and 2006 to 2009 proceedings from the Conference on Retroviruses and Opportunistic Infections, the International AIDS Society Conference on HIV Pathogenesis, Treatment and Prevention, and the National Institute of Mental Health and the International Association of Physicians in AIDS Care International Conference on HIV Treatment Adherence. We limited our searches to the post-HAART era, which began in 1995. We chose to search relevant conferences because of recent proceedings presenting preliminary and/or pre-publication data from DOT-HAART clinical trials that have influenced DOT-HAART discussions. We limited conference searches to recent years with the rationale that data published before these years should have resulted in a published article. We used the following queries: “HIV” OR “HAART” AND “directly observed” OR “DOT” OR “mDOT” OR “directly supervised” OR “directly observed therapy” OR “DOT-HAART,” OR “DAART”. We also used the “related articles” search tool in PubMed and examined the bibliographies of all reviewed sources, including several review papers.^{34,35,46,51,70,74} We did not restrict our searches to English. We compared sources to exclude duplicate references (ie, same outcomes reported on the same cohort). We contacted authors and experts for additional studies and data not available in publications and abstracts.

Study Selection

Studies were included if they (1) described a DOT-HAART intervention (ie, involved direct observation of at least some proportion of HAART), (2) were peer reviewed, (3) included a randomized or non-randomized comparison group, (4) took place in an outpatient setting, (5) exclusively enrolled adults, and (6) included at least 1 of the outcome measures of this analysis (viral load suppression, change in CD4 cell count, and HAART adherence).

Although RCTs are considered the gold-standard design for evaluating efficacy, they may be limited in assessing of the effectiveness of DOT-HAART in a diversity of settings and may not inform the adaptation of an intervention to suit local needs.⁷⁵ RCT enrollment may be biased, and better outcomes are often observed across all RCT arms compared with standard of care,⁷⁶ particularly among vulnerable populations and/or settings of poor service infrastructure.^76–80 Furthermore, although DOT-HAART may seem straightforward on first blush, the intervention can be both complex and heterogeneous when implemented in the real-world setting. DOT-HAART interventions deployed within RCTs may be inherently simpler and less flexible because of the need to standardize the intervention and monitor treatment fidelity. For many nonrandomized studies of DOT-HAART interventions, evaluation using a randomized study design may not have been possible or ethical. We therefore included nonrandomized reports to maximize the diversity of experiences represented in our systematic review.

Data Abstraction

Using standardized coding forms, 2 reviewers independently abstracted information from the articles and posters. Each study was coded for study design, intervention and control characteristics, sample size, retention, and outcome data. We had an inter-rater agreement of 96% on key variables. Discrepant abstractions were resolved through discussion, including arbitration of an additional reviewer.

Study Outcomes

Studies varied in their definitions of virologic, immunologic, and adherence endpoints. For instance, some reported virologic success as achieving either an undetectable viral load or at least a log₁₀ drop in viral load at the end of the study. For meta-analysis, we chose three endpoints: virologic suppression (proportion achieving an undetectable HIV load based on the assay used for the study); immunologic response (mean change in CD4 cell count from baseline); and adherence (proportion of individuals achieving ≥95% adherence to prescribed doses). Because adherence measures varied across studies; we used adherence outcomes as measured by study authors as long as data were available using the threshold of ≥95%. If data could not be gathered from published information, including endpoints that were not reported according to our meta-analysis endpoint definitions, we contacted the authors and invited them to provide additional information.

For studies that included multiple intervention or control arms, we analyzed the 1 intervention arm that represented the most frequent administration of DOT and the 1 control arm that was most comparable to the DOT group. For instance, the study by Idoko et al⁵⁹ involved 3 intervention groups: 1 receiving daily, another receiving twice-weekly, and the third receiving once-weekly DOT. We chose the daily DOT arm to use as the intervention group for the purposes of our meta-analysis. Among the 3 control groups in the study by Lucas et al,⁶⁰ we chose the control most comparable to the intervention group (intravenous drug users on methadone). Gross et al⁸¹ also conducted a 3-arm study; we compared the DOT arm with the control group that received the same once-daily regimen under self-administration. For Wohl et al,⁶⁹ who tested 2 interventions (DOT and intensive case management), the DOT group was compared with the control arm receiving standard of care.

Methodologic Assessment

We summarized the methodologic features of the studies based on the following variables: study design, comparability of control and intervention arms, study retention (including differential retention by trial arms), methods for handling missing data, and methods of measuring adherence. We did not exclude studies based on quality assessment. Given the limited validity of quality scores,^82,83 we did not create a quality score or weight studies differentially based on quality assessments. Rather, we performed stratification and meta-regression on key quality-associated study characteristics.

Analytic Approach

We employed risk ratios (RRs) to describe the associations between DOT-HAART and undetectable viral load and DOT-HAART and adherence. We assessed the effect of DOT-HAART on CD4 cell count by comparing the mean differences of CD4 cell count in each arm. Standard deviations of mean change in CD4 cell count and the number of observations for each arm were used to compute the standard error of the difference in CD4 change. For studies that did not provide the standard deviations or the mean change in CD4, we inferred them from other data available (see Supplemental Digital Content 1, http://links.lww.com/QAI/A43). Our analyses employed observations from 1 time point for each study; for studies with multiple time points of assessment, we used the last available on-intervention measurements to assess intervention effect among all pooled studies. Post-intervention measures occurring more than 1 month after the intervention were pooled separately to assess durability of intervention effect. Further, to examine the trajectory of postintervention virologic effect of DOT-HAART, we plotted the log-transformed RRs against the time since duration for studies that made multiple assessments of virologic success after intervention.^81,84–87

We assessed the heterogeneity of effect estimates using the Cochran Q test,⁸⁸ and we quantified the magnitude of between-study heterogeneity using the Higgins I² estimate.⁸⁹ We performed the Dersimonian-Laird random-effects (REs) meta-analysis⁹⁰ to aggregate the effects of DOT-HAART on undetectable viral load and ≥95% adherence across studies because the studies showed significant heterogeneity by the Cochran Q Test (P < 0.05). The Dersimonian-Laird RE meta-analysis was also performed to compute the weighted mean difference (WMD) in change of CD4 cell count as the studies showed significant heterogeneity with respect to immunologic outcome.

Because studies varied with respect to outcome ascertainment, intervention, and study population, we looked for possible effect modifiers through stratified meta-analyses and meta-regression, which tests the difference in effect between 2 groups. For both virologic and immunologic outcomes, we employed RE meta-analyses to summarize the data within each stratum. Meta-regression was performed by regressing the natural logarithm of the RR for virologic success of the studies by the study-specific values for the effect modifiers of interest, weighting the studies by the inverse of the sum of study-specific variance and between-study variance. A priori, we identified variables that we hypothesized were most likely to contribute to heterogeneity in 3 areas: (1) intervention design; (2) target population; and (3) study quality.

Effect modifiers pertaining to intervention design were DOT site (hospital or HIV clinic vs. methadone clinic vs. residence-based, ie, patient homes, mobile community van, prison, or hospice); and DOT intensity (enhanced DOT-HAART vs. not enhanced DOT-HAART). Guided by a systematic review on DOT for tuberculosis,⁹¹ we defined “enhanced DOT-HAART” as any intervention that included additional formal adherence support not offered to the control group (ie, material or financial incentives/enablers) or a behavioral intervention or ancillary services aimed at improving adherence. Because certain services were often provided as necessary and ethical consequences of DOT, we did not consider the following activities to constitute formal additional support: asking about side effects and adherence at DOT visits and reporting any problems to providers; prepackaging and delivering HAART via DOT visits; and referring patients to other social services unless additional staff (eg, case manager, social worker) was integrated into the DOT team. Differences in target population included percent HAART naive (≥50% vs. <50%); study setting (resource-poor vs. resource-rich setting, based on groupings of low and middle vs. high human development, respectively, from the United Nation’s Human Development Index⁹²); and substance use (≥50% substance users vs. <50% substance users). For studies that did not specify the proportion of substance users, we assumed <50%. Variables reflecting study characteristics were study design (RCT vs. nonrandomized study); control comparability (baseline virologic or immunologic differences between arms vs. no difference); and differential attrition of <8% vs. ≥8%.

We tested for publication bias by the Begg (rank correlation),⁹³ the Egger (weighted regression) tests⁹⁴ and also used a modified Macaskill test, which avoids the problem of correlation between the logarithm of the RR and its standard errors.⁹⁵ A funnel plot of standard error estimates vs. effect size estimates based on intervention effects on virologic suppression was created to visually assess for asymmetry as an indication of publication bias. We performed all statistical analyses using the “meta” and “rmeta” packages in R version 2.8.1.⁹⁶

RESULTS

Figure 1 outlines the selection process used to identify studies that met our inclusion criteria. Of the 2293 citations returned from our queries, all but 283 were excluded after abstract review because they were not about DOT-HAART or were duplicate citations. Fifty-four of the 283 citations retrieved for further review were excluded because they described either ongoing studies or studies that were as yet complete but not peer reviewed. Other citations were excluded because they described case series, qualitative or descriptive reports, controlled studies that did not measure any of the meta-analysis outcomes, or were duplicate references. We identified 21 citations that met our criteria for inclusion; however, outcome data for 3 were not available yet per communication with authors.^67,97,98 Two of the remaining 18 citations reported data from the same study.^53,85 Therefore, we included 17 studies in our systematic review and meta-analyses.

FIGURE 1.

Flow diagram of selection process for systematic review and meta-analysis. CRISP, Computer Retrieval of Information on Scientific Projects database; CROI, Conference on Retroviruses and Opportunistic Infections; IAS, International AIDS Society; NIMH/IAPAC, National Institute of Mental Health and the International Association of Physicians in AIDS Care International Conference on HIV Treatment Adherence.

Study and Sample Characteristics

Characteristics of the 17 studies are summarized in Table 1. Cumulatively, these studies involved a total of 3169 patients (range 49–500, mean cohort size 186), of whom 38% were female. Twelve studies (71%) were published in peer-reviewed journals at the time of analysis and 5 were presented at conferences. Six studies (35%) were conducted in resource-poor countries. Eight studies (47%) targeted HAART-naive participants; 7 (41%) restricted inclusion to substance users.

TABLE 1.

Characteristics of Controlled DOT-HAART Studies Meeting Inclusion Criteria (n = 17)

ID	Study	Setting	Study Size	Salient Inclusion and Exclusion Criteria	DOT Duration and Frequency	DOT Site	Description of DOT, Other Support), in Addition to Standard of Care	VL (Y/N), Sensitivity	CD4 (Y/ N), Definition	Adherence (Y/N), Method; Assessment Interval; Definition
A	Altice et al53 and Maru et al85	Area HIV clinics in New Haven, CT	141	HIV positive; used heroin and/or cocaine in past 6 months; twice daily (or once daily HAART	6 months, once daily on week days	Van travels with needle exchange to neighborhood sites	Enhanced community- based DOT of HAART and other medications by DOT specialist; beeper reminders & outreach; van with on- site clinician, drug treatment coordinator, case manager, outreach workers	(Y), <400 copies/mL	(Y), Mean change	(Y), Measured at baseline, 1, 3, & 6 months using 3-day ACTG self- reported adherence
B	Arnsten et al54	Methadone clinics in New York City, NY	65	Stable methadone dose 5–6 day/week pick-up; receiving HIV care from methadone provider or affiliated clinic	6 months, once daily 5–6 days/week	Methadone clinic	DOT-HAART at methadone clinic	(Y), <75 copies/mL	(N)	(Y), Pill count; defined as median number doses taken
C	Babudieri et al55	Prisons in Italy	84	Incarcerated HIV-positive IDU	Variable (mean 8.7 months); all doses	Prison	DOT-HAART by prison nurse	(Y), <200 copies/mL	(Y), CD4 <200 cells/mL	(N)
D	Bangsberg et al84	Community research site in San Francisco, CA	81	HIV viral load >400 copies/mL; nonadherence	3 months, once daily 5 days/week	Community research site	Enhanced DOT-HAART and other medications by study staff; deliver missed doses to patients. DOT followed by 3 months of adherence case management	(Y)	(N)	(Y), Measured by 7-day self- report at baseline then monthly
E	Gross et al81	HIV treatment centers at ACTG sites	243	HAART naïve; VL >2000 copies/mL; weight ≥40 kg	6 months, once daily 5 days/week	ACTU or other clinic site	DOT-HAART by health professional (nurse or pharmacist, not a relative, partner, or close friend)	(Y), <200 copies/mL	(Y), Median change	(N)
F	Horta et al58	Tertiary infectious diseases hospital in Porto, Portugal	422	HIV positive; active IDU	Variable duration; once daily all days	Hospital methadone clinic	Enhanced DOT of HAART & methadone; multidisciplinary support	(Y), <50 copies/mL	(Y), mean change	(Y), self-report and pharmacy refill
G	Idoko et al59	Tertiary hospital in Jos, Nigeria	175	HAART naive. Excluded: medical condition that interferes with study; <1 year prognosis; substance users	12 months, Three DOT groups: once daily all daysa; once daily twice weekly, once daily weekly	Patient home	Community-based DOT- HAART by patient- assigned treatment partner trained on record-keeping, side effects and follow-up	(Y), <400 copies/mL	(Y), median	(N)
H	Lanzafame et al101	Tertiary hospital and prisons in Legnago, Italy	49	Naive to protease inhibitor therapy	6 months, all doses	Prison	DOT-HAART by prison nurses	(Y), N/R	(Y), mean	(N)
I	Lucas et al60	Methadone clinics in Baltimore, MD	891	Regular HIV treatment provider; methadone for >30 days without plans to discontinue; baseline VL >500 copies/mL; no triple- class resistance	Goal at least 12 months, once daily on methadone days	Methadone clinic	DOT-HAART by nurse or medical assistant; prepackaged doses for non-DOT days	(Y), <400 copies/mL	(Y), Median change	(N)
J	Macalino et al61	HIV clinics in MA and RI	87	Active substance use; no genotypic resistance to a once-daily regimen	Up to 12 months; once daily for 3 months, then tapered	Location chosen by patient	Community-based DOT-HAART by trained outreach worker; packaged pills	(Y), <50 copies/mL	(Y), median change	(Y), self-report 30-day recall
K	Munoz et al100	HIV clinic in tertiary hospitals Lima, Peru	120	HAART naïve; residence & receiving HIV care in study area. Priority given to tuberculosis coinfected patients and women	12 months, all doses (all twice daily regimens) for 8 months then tapered	Patient home or alternative site per patient preference	Enhanced community-based DOT of HAART & other medications by lay health workers who also monitor side effects & provide social support; financial aid for tests, meds, transportation; nutritional support as needed	(Y), <400 copies/mL	(Y), Mean change	(Y), measured at baseline, then every 3 months using self-report 30-day recall
L	Nachega et al86	Secondary hospital in Cape Town, South Africa	274	Stavudine/lamivudine/efavirenz or nevirapine regimen	12 months, at least once daily	ART roll-out clinic at secondary hospital	DOT-HAART by trained treatment supporters	(Y), N/R	(Y), Mean change	(N)
M	Pearson et al65	HIV clinic in tertiary hospital in Beira, Mozambique	350	Reside near clinic; no severe mental illness or dementia	1.5 months, once daily on weekdays	HIV clinic in tertiary hospital	Enhanced DOT-HAART provided by HIV-positive peers trained based on Informational, Motivation and Behavioral Skills model; peers also provide social support and education, and encourage participation in support groups; transportation costs	(N)	(Y), Mean change	(Y), Self-report 7-day and 30-day recall
N	Sarna et al87	HIV clinics in Mombasa, Kenya	234	reside in Mombasa; HAART naïve	6 months, once daily twice weekly	Health center clinic	Enhanced DOT-HAART by clinic nurses who also provide adherence support; outreach if missed appointments; medication delivery to patient & travel costs provided if needed	(Y), < 400 copies/ml	(Y), median change	(Y), pill counts & self- reported 4-day recall measured every 8 weeks from weeks 1–48 then at week 72
O	Taiwo et al99	Tertiary hospital in Jos, Nigeria	500	HAART naive, Receiving care at PEPFAR Program at Jos University Hospital. Excluded: life expectancy <1 year despite HAART	6 months, at least once daily all days	Patient home	Community-based DOT- HAART by patient-selected treatment partner who lives in same house or in close proximity. Partner receives adherence counseling and helps patient complete medication log, monitor side effects and medication supply	(Y), <400 copies/mL	(Y), Change	(Y), Assessed by prescription refill data, # reminder calls from pharmacist, medication log, visual analog scale of doses in past month
P	Tinoco et al66	AIDS welfare homes & area reference hospitals in Cadiz, Spain	98	Stage C; IDU; enrolled in methadone program	9 months, all doses	AIDS welfare homes	Enhanced DOT-HAART delivered by nurse volunteers; housing & psychological support, access to addiction- treatment & rehabilitation programs	(Y), <50 copies/mL	(Y), Mean change	(N)
Q	Wohl et al69	HIV clinics in Los Angeles, CA	250	<1 prior failed regimen; ≥23 on Folstein Mini Mental Status Exam; live or work in study area; Excluded: advanced liver or kidney disease; DOT for tuberculosis	6 months, once daily on weekdays	Home or other site decided by patient and community worker	Enhanced community-based DOT-HAART by bilingual community worker who also delivers non-DOT doses and addresses adherence problems; financial incentives up to $205	(Y), <400 copies/mL	(Y), Median change	(Y), self-report with 24 hour, 3-day and 7-day recall

IDU, intravenous drug users; N/R, not reported; VL, viral load; SAT, self-administered therapy.

Intervention and Control Characteristics

As shown in Table 1, interventions varied widely. Five studies (29%) provided DOT for all doses, another 4 reported daily DOT that did not explicitly cover all doses (24%), 6 (35%) provided once-daily DOT observed 5–6 times a week, 1 delivered DOT “on methadone days,” and 1 provided twice weekly DOT. The duration of DOT ranged from 6 weeks to 29 months, with a median duration of 6 months. The site of DOT also varied: 5 interventions required travel to a hospital or HIV clinic to receive DOT, 3 took place in methadone clinics, and 9 were residence based (ie, patient homes, mobile community van, hospice, or prison). DOTwas performed by nurses or clinic staff in 9 interventions, whereas 8 used lay workers (including family members) to deliver DOT. Seven studies (47%) provided enhanced DOT, with additional support ranging from case management to outreach for nonadherent patients to financial or material enablers. Standard of care varied by treatment site but did not involve direct observation of medications. Ten studies (59%) provided baseline adherence education and counseling to both study groups.

Methodologic Assessment

Indicators of study quality were also examined (see Table, Supplemental Digital Content 2, http://links.lww.com/QAI/A44). Eleven studies (65%) were RCTs. Six studies reported baseline virologic or immunologic differences between the DOT-HAART and comparison groups, and 7 studies experienced attrition of ≥8% during the study. Follow-up varied: 9 studies followed patients for 3–9 months, whereas the remaining 8 were 12 months or longer in duration. All studies reported on-treatment or immediate postintervention measures. Only 6 studies reported postintervention data, ranging from 6 to 12 months after completion of DOT. Of the 10 studies measuring HAART adherence, 6 relied on self-report, 1 used pill counts, and 3 combined multiple assessments including pill count, pharmacy refill, and self-report. Recall periods for adherence assessment ranged from 4 to 30 days.

Summary Effects

Viral Load

Although 16 studies assessed virologic response, for meta-analytic assessment, we included 14 studies for which data were available per our outcome definition: the proportion of patients with undetectable viral load at the time of DOT-HAART completion. We used viral load detection limits utilized by study authors (50 copies/mL,^58,61,66 75 copies/mL,⁵⁴ 200 copies/mL,^55,81 400 copies/mL,^{53,59,60,69,84,85,87,99,100} and 1 study did not specify⁸⁶). Of data available, 67% (700 of 1049) of DOT-HAART participants and 53% (584 of 1110) of control participants achieved an undetectable viral load. Six studies^{55,58,60,84,99,100} showed significantly greater virologic suppression with DOT. As shown in Figure 2, DOT-HAART was associated with a 24% increase in virologic suppression (RE RR: 1.24, 95% confidence interval (CI): 1.08 to 1.41). However, the effect estimates were heterogeneous (P < 0.001, I² = 80.3%, 95% CI: 67.8% to 87.9%), with between-study variability explaining 80% of the total variance. Meta-analysis of DOT-HAART durability on the studies that ascertained outcomes at least 1 month subsequent to the cessation of intervention^81,84–87 revealed an RE RR of 0.95 (95% CI: 0.86 to 1.05), suggesting lack of postintervention effect. When RRs from studies with multiple outcome assessments were plotted against time since end of intervention (Fig. 3), the effect for virologic suppression decreased over time in the 1 study with a significant effect at the end of intervention.⁸⁴

FIGURE 2.

Forest plot of studies with results on the effect of DOT-HAART on virologic suppression. Values shown are among data available. The size of the squares is proportional to the inverse variance of log-transformed RRs. The arrow indicates that the 95% CI has been truncated to limits of the x axis scale. n, the number of patients achieving virologic suppression; N, the total number of patients in the study arm; RE, random effects.

FIGURE 3.

Durability of DOT-HAART. Postintervention change in effect of DOT-HAART on virologic suppression in studies that assessed durability of DOT-HAART.

CD4

Thirteen studies in the systematic review assessed CD4 cell count according to our definition of immunologic response: the mean change in CD4 cell count from baseline to the time of DOT-HAART completion.^{53,55,58–61,66,69,81,86,87,100,101} We obtained necessary data for computation of mean difference and its standard error from 9 studies and/or authors. For the remaining 4 studies,^53,59,87,101 we inferred the mean change and standard deviations from other data available (see Supplemental Digital Content 1, http://links.lww.com/QAI/A43). Four studies showed significantly greater increase in the DOT-HAART group.^53,58,60,66 As summarized in Figure 4, DOT-HAART was associated with greater increase in CD4 cell count (RE WMD 43 cells/μL, 95% CI: 12, 74 cells/μL) compared with standard of care. As with virologic suppression, the effects varied widely, as indicated by the significant Q test and a Higgin I² value greater than 50% (P <0.001, I² = 82.6%, 95% CI: 71.5% to 89.4%). Meta-analysis of DOT-HAART durability on change in CD4 included 5 studies for which postintervention data were available^{55,65,85–87} and did not show a significant effect (RE WMD: 40 cells/μL, 95% CI: −13 to 93 cells/μL).

FIGURE 4.

Forest plot of studies with results on the effect of DOT-HAART on mean change in CD4 cell count. Values shown are among data available. The size of the squares is proportional to the inverse variance of mean differences. The arrow indicates that the 95% CI has been truncated to limits of the x axis scale.

Adherence

Six of the studies included in the systematic review assessed adherence as taking at least 95% of prescribed doses at time of DOT-HAART completion.^{61,65,69,84,87,100} Of data available, we found that 88% (359 of 408) of those receiving the intervention compared with 75% (302 of 402) of patients in control groups achieved ≥95% adherence. As shown in Figure 5, the studies analyzed showed a positive intervention effect on adherence (RE RR: 1.17, 95% CI: 1.03 to 1.32). Again, the results varied from study to study (P = 0.01, I² = 62.7%, 95% CI: 9.3% to 84.6%), with between-study variability explaining 63% of the total variance in the effect.

FIGURE 5.

Forest plot of studies with results on the effect of DOT-HAART on ≥95% adherence to prescribed doses. Values shown are among data available. The size of the squares is proportional to the inverse variance of log-transformed RRs. The arrow indicates that the 95% CI has been truncated to limits of the x axis scale. n, the number of patients achieving ≥95% adherence; N, the total number of patients in the study arm.

Stratified Analysis and Meta-Regressions

To explore sources of heterogeneity in the effects of DOT-HAART on undetectable viral load, we stratified the studies and conducted meta-regressions by 8 variables, as shown in Table 2. Although meta-regression analyses were not statistically significant, several trends in treatment effects by stratified meta-analyses were notable. Treatment effect was greater among studies delivering DOT at patients’ residences compared with those delivering clinic-based DOT; in HAART-experienced individuals compared with HAART-naive individuals; in nonresource-poor settings compared with resource-poor settings; in substance-using populations compared with nonsubstance-using populations; and in those receiving enhanced DOT compared with those given nonenhanced DOT. Effect estimates were greater among nonrandomized observational studies compared with RCTs, although this meta-regression did not show evidence for a significant difference (P = 0.52). Associations with virologic suppression in RCTs did not meet statistical significance (RR = 1.18, 95%CI: 0.99, 1.42, P = 0.068). There were no differences in the effect of DOT-HAART by presence of baseline virologic or immunologic differences (P = 0.66) or by differential attrition (P = 0.96).

TABLE 2.

Stratification to Test for Effect Modification of Intervention Effect on Viral Load Suppression

Categories	Subcategories	Number of Studies	Studies	Summary RR*	95% CI of RR	P Heterogeneity for	I2,†	I2 95% CI	P for Meta-Regression
Intervention design
DOT Site	Hospital/HIV Clinic	3	D,E,L	1.10	(0.91 to 1.34)	0.05	67%	(0% to 90%)	REF
	Residence	8	A,C,G,J,K,O,P,Q	1.19	(1.09 to 1.30)	0.45	0%	(0% to 67%)	0.36
	Methadone Clinic	3	B,F,I	1.46	(0.88 to 2.41)	<0.0001	94%	(84% to 97%)	0.99
DOT intensity	Enhanced	6	A,D,F,K,P,Q	1.35	(1.03 to 1.67)	0.032	80%	(56% to 91%)	REF
	Non-enhanced	8	B,C,E,G,I,J,L,O	1.14	(1.02 to 1.28)	0.009	63%	(19% to 83%)	0.31
Target population
Proportion HAART naive	<50%	9	A,B,C,D,F,I,J,P,Q	1.33	(1.03 to 1.72)	<0.0001	82%	(67% to 90%)	REF
	≥50%	5	E,G,K,L,O	1.12	(1.04 to 1.21)	0.24	28%	(0% to 72%)	0.42
Study setting	Resource-poor setting	4	G,K,L,O	1.17	(1.07 to 1.28)	0.0008	0%	(0% to 85%)	REF
	Resource-rich setting	10	A,B,C,D,E,F,I,J,P,Q	1.28	(1.04 to 1.59)	0.022	87%	(78% to 92%)	0.58
Substance using proportion	<50%	6	E,G,K,L,O,Q	1.11	(1.04 to 1.19)	0.33	13%	(0% to 78%)	REF
	≥50%	8	A,B,C,D,F,I,J,P	1.39	(1.04 to 1.84)	0.024	82%	(66% to 91%)	0.58
Study quality
Study design	RCT	9	A,B,D,E,F,J,L,O,Q	1.18	(0.99 to 1.42)	<0.0001	86%	(75% to 92%)	REF
	non-RCT	5	C,G,I,K,P	1.32	(1.11 to 1.58)	0.08	52%	(0% to 82%)	0.52
Baseline immunologic or virologic differences	No difference	9	B,C,D,I,J.K.L,O,P	1.23	(1.05 to 1.44)	0.006	62%	(23% to 82%)	REF
	Difference	5	A,E,F,G,Q	1.23	(0.96 to 1.57)	<0.0001	92%	(84% to 96%)	0.66
Differential loss to follow-up‡	≥8% loss to follow-up	5	A,D,J,K,Q	1.18	(0.94 to 1.50)	0.13	43%	(0% to 79%)	REF
	<8% loss to follow-up	7	C,E,F,G,L,O,P	1.27	(1.06 to 1.52)	<0.0001	88%	(78% to 94%)	0.96

^*Random effects summary estimates within subcategories.

^†Proportion of total variance of summary estimate due to between studies variance.

^‡Among data available.

REF, referent.

Publication Bias

There was no evidence of publication bias, as assessed by the Begg (P = 0.21) and Egger tests (P = 0.36). The modified Macaskill test confirmed this finding (P = 0.90). The funnel plot did not manifest any noticeable asymmetry (see Figure, Supplemental Digital Content 3, http://links.lww.com/QAI/A45).

DISCUSSION

In this systematic review of controlled DOT-HAART studies, we observed an overall beneficial effect of DOT-HAART on virologic, immunologic, and adherence outcomes. DOT-HAART was found to improve HAART adherence, supporting the presumed mechanism of DOT-HAART effectiveness on clinical outcomes through improved antiretroviral adherence.^4,9,102,103 Qualitative data suggest that other mechanisms may also mediate DOT-HAARTeffectiveness, including positive effects on patients’ trust and communication with providers; increased patient motivation to engage in daily activities and become involved in the community; improved adherence to other aspects of medical care; and greater the utilization of other forms of social and adherence support.^{30,44,65,104–106}

We encountered large variation in methodologic quality, intervention design, and population characteristics and explored their influence on the observed virologic effects through stratification and meta-regression. When stratified by study design, the positive effect of DOT-HAART on virologic and immunologic outcomes among RCTs was attenuated and not statistically significant, whereas the association remained significant in nonrandomized studies. The meta-analysis by Ford et al¹⁰⁷ also found a lack of effect among RCTs (RR = 1.04 (95% CI: 0.91 to 1.20, P = 0.55), but this summary estimate was smaller than our findings. The potential reason for the difference may be that Ford et al¹⁰⁷ included effect estimates from the postintervention period, during which the efficacy of DOT-HAART may have waned, as indicated by our findings. Experts often rely on RCTs for causal inference as randomization prevents the imbalance of confounding factors between intervention and control groups. Recognizing that those who were selected for DOT may have differed from those who received standard of care in characteristics that would affect outcomes, we investigated the impacts of baseline virologic or immunologic differences and of differential attrition on effect heterogeneity. Meta-regressions and stratifications did not detect any significant difference in effect based on these study characteristics. Thus, we cannot attribute the difference in effect between RCTs and nonrandomized studies to these factors. Instead, these findings may reflect true differences in effect by population characteristics or intervention design that varied between RCTs and non-randomized studies. Nonrandomized DOT-HAART experiences may have allowed greater flexibility in intervention design and modification and may have enrolled vulnerable populations in whom the intervention effect could be greatest.

Beyond methodological quality, there were considerable variations in DOT-HAART interventions and populations targeted. Meta-regression analyses failed to identify a clear source of heterogeneity. Nonetheless, some of the trends in intervention effect upon stratification merit further discussion. Greater effect on virologic outcome was observed among substance-using and HAART-experienced cohorts. These findings support the intuitive hypothesis that individuals at greatest risk of treatment nonadherence (including HAART-experienced individuals^108,109 and substance users^{53,60,61,74,110}) benefit most from this intervention. Residence and methadone-based DOT-HAART interventions demonstrated greater treatment effect compared with clinic-based interventions, although the effect among methadone-based interventions was not statistically significant. Choosing a convenient site—such as a methadone clinic or the patient’s residence—could enhance the effect of the intervention. Interventions delivered in patient homes, community-based vans, prisons, and methadone clinics may impose minimal additional burden on patients’ routines. On the other hand, the time and expenses of daily travel to a site (eg, HIV clinic, hospital) that is not part of a patient’s daily routine may pose important barriers to DOT-HAART adherence, in particular in resource-poor settings where the relative cost of traveling to health facilities may be even greater.

Not all DOT is the same. Enhanced DOT-HAART, defined as an intervention that provides additional material or behavioral adherence support not offered to the control group, seemed to enhance treatment effect. Consensus guidelines for treatment of tuberculosis endorsing DOT have pointed out that studies of DOTwith enablers have shown the highest treatment completion rates.⁹¹ Our findings suggest that the same observation may be true for DOT-HAART and that the use of additional motivations for adherence may improve outcomes,^70,87,111 particularly among substance users.^{53,74,112–114} Ongoing RCTs such as that of Bangsberg et al⁸⁴ and MOTIV8 are examples of enhanced DOT-HAART interventions intended to maximize the potential impact of DOT by administering other forms of adherence support, such as case management or adherence counseling based on motivational interviewing and cognitive behavioral techniques.^51,84,104 Final data from such studies will provide important information on enhanced forms of DOT-HAART.

Although there were few studies that assessed post-intervention effect, we found that initial intervention effect may wane after completion of DOT support. Although these findings are consistent with a meta-analysis of a wide spectrum of HIV treatment adherence interventions,³⁴ exploring this time-limited effect may be even more important for DOT-HAART, if the mechanism of action is through improved adherence via direct supervision. If DOT-HAART is to have a sustained effect on postintervention outcomes, interventions must be designed to engender psychosocial and behavioral changes in patients through DOT encounters, such as those described by several groups studying DOT-HAART.^{53,57,65,87,100,104,115} For this reason, although we did not identify a significant difference in intervention effect of enhanced vs. nonenhanced DOT on immediate outcomes, we speculate that enhanced DOT-HAART could lead to more lasting durability of intervention effect. Efforts to sustain the benefits of DOT postintervention may also require closer attention to the transition from DOT to self-administration and to individualizing DOT through varied frequency, intensity, and duration of support.¹¹⁶ If DOT-HAART effect is not durable, another option would be long-term or even life-long DOT-HAART for certain individuals or populations.⁴¹ Creating and implementing durable HAART adherence interventions remains an enormous challenge.^34,85,87

Our review has several limitations. It was not feasible to blind abstractors to authors, institutions, or journals of the data reviewed; however, use of a standard extraction form, resolution of discordant abstractions, and involvement of third party minimized bias from lack of blinding. We could not overcome some of the heterogeneity across studies and differences in adherence measures, and we were unable to investigate the moderating effect of other potential variables, including unmeasured differences in DOT vs. control groups. As previously mentioned, the small number of studies limited the inferences that could be drawn from meta-regressions. To better understand findings across studies and to assemble data for the purposes of meta-analysis with greater ease in the future, we recommend that all forthcoming controlled studies on DOT-HAART report the 3 outcome measures as defined in this analysis.

Despite these limitations, our review of peer-reviewed controlled studies shows that DOT-HAART seems to be effective among selected patient populations, such as those with a history of prior HAART experience and/or substance use. Features of DOT-HAART which may increase treatment effect include nonclinic-based DOT and the provision of additional forms of adherence support. Because the impact of DOT-HAART on virologic response did not reach statistical significance when restricted to RCTs, the efficacy of DOT-HAART still remains in question. Areas for future research include assessment of long-term treatment effects and the refinement of DOT-HAART interventions to optimize the intensity, duration, and frequency according to patient need. Similar to the body of knowledge that has guided decisions on DOT for tuberculosis, efficacy trials should be complemented by outcomes data from large-scale DOT-HAART programs and cost-effectiveness analyses to inform public health decisions regarding whether and under what circumstances DOT-HAART should be employed.