Challenges in the Evaluation of Interventions to Improve Engagement Along the HIV Care Continuum in the United States: A Systematic Review

Abstract

In the United States (US), there are high levels of disengagement along the HIV care continuum. We sought to characterize the heterogeneity in research studies and interventions to improve care engagement among people living with diagnosed HIV infection. We performed a systematic literature search for interventions to improve HIV linkage to care, retention in care, reengagement in care and adherence to antiretroviral therapy (ART) in the US published from 2007-mid 2015. Study designs and outcomes were allowed to vary in included studies. We grouped interventions into categories, target populations, and whether results were significantly improved. We identified 152 studies, 7 (5%) linkage studies, 33 (22%) retention studies, 4 (3%) reengagement studies, and 117 (77%) adherence studies. ‘Linkage’ studies utilized 11 different outcome definitions, while ‘retention’ studies utilized 39, with very little consistency in effect measurements. The majority (59%) of studies reported significantly improved outcomes, but this proportion and corresponding effect sizes varied substantially across study categories. This review highlights a paucity of assessments of linkage and reengagement interventions; limited generalizability of results; and substantial heterogeneity in intervention types, outcome definitions, and effect measures. In order to make strides against the HIV epidemic in the US, care continuum research must be improved and benchmarked against an integrated, comprehensive framework.

Introduction

In 2015, the White House released an updated comprehensive National HIV/AIDS Strategy (NHAS) [1], and outlined specific measures to assess progress along the HIV care continuum. It is increasingly recognized that effective approaches to ending HIV in the United States (US) will require comprehensive strengthening of multiple components of the HIV care continuum [2]. While recent models have suggested that improved retention of HIV-positive persons in care is critical to reducing transmission [3, 4], large numbers of people living with HIV (PLHIV) in the US remain unaware of their infection (~13% of all PLHIV), unlinked to care, disengaged from care (~61% of all PLHIV), incompletely adherent to antiretroviral therapy (ART), and virologically unsuppressed (~70% of all PLHIV) [5, 6]. While the NHAS characterizes federal and local implementation strategies to address these gaps, it does not specify particular programs to implement in given areas with particular populations. To date, the majority of scientific effort has been dedicated to improving each stage of the HIV care continuum independently—each of which has been reviewed in recent years, including by the Centers for Disease Control and Prevention’s Prevention Research Synthesis (PRS) team [7–13]. The PRS project provides a vital resource for the identification of evidence-based interventions to improve steps of the HIV care continuum. However, these evidence-based interventions have variability in their magnitude of effect, sustainability of effects, costs (or lack of assessment of costs) and heterogeneity in study design and outcomes assessments.

The next step in designing a comprehensive approach to HIV prevention and engagement is to build upon prior work [13] by synthesizing the literature on interventions to strengthen the HIV continuum of care, in order to describe the heterogeneity of studied interventions’ approaches, costs, study designs, and study outcomes. Characterizing these heterogeneities will lay the foundation for decision-makers to develop a common framework for assessing HIV care continuum interventions, while helping researchers to better understand the evidence gaps we most urgently need to fill. For example, the extent to which currently studied interventions could achieve the NHAS goals is unknown, and without a common framework it will remain a challenge to evaluate.

As such, we performed a broad-based systematic review of all published interventions designed to strengthen the HIV care continuum after HIV diagnosis, with the aim of collecting evidence for prioritization of interventions, direction of future research, and evaluation of interventions to improve health outcomes and prevent HIV across the continuum of care.

Methods

We conducted a systematic review of English language literature to comprehensively characterize interventions designed to improve HIV care engagement in the US after HIV diagnosis. We conducted separate searches for the following care continuum steps: initial linkage to care (for those newly diagnosed with HIV), care retention (for those currently in care), care reengagement (for those previously in care), and medication adherence while on ART (for those in care and prescribed ART) (see Fig. 1 for a schematic of the steps of the HIV care continuum addressed). Our primary objectives were to characterize and describe the spectrum of HIV care continuum interventions for different target populations, and to characterize the heterogeneity of outcomes measured among such studies. Our secondary objectives were to describe the efficacy and whether costs are reported for identified interventions.

Fig. 1.

Schematic of HIV care continuum in the United States indicating steps of interest in review, and summary of outcomes in review. Each box’s area is proportionally sized to the United States population of people living with HIV in 2012 [5, 173]. Boxes representing very small proportions of the population (Newly Diagnosed and Newly in Care) are enlarged for purposes of display. Table displays the continuum steps of interest in review, number of studies included in review (N), NHAS indicator relevant to the continuum step, and a sampling of the outcomes identified in included studies in the review (for full list of outcomes identified, please see Supplemental Table)

Search Strategy and Study Selection

We systematically searched MEDLINE and EMBASE for citations published from 1 Jan 2007 to 17 June 2015. A list of keywords was created around the domains of interest (see Appendix 1 for complete search strategy). The reference lists of 64 reviews found in our search were evaluated to identify any manuscripts meeting inclusion criteria that were not found in our search strategy.

The overall target population for the review was people living with diagnosed HIV infection in the US. We included all study participants regardless of age, gender, and ethnicity. We included studies that had explicitly defined study populations exposed to an intervention, and a comparator population that did not receive an intervention. This review did not seek to evaluate any specific intervention, and interventions included were any biomedical, behavioral, health system, or policy strategy that sought to increase engagement in the four areas listed above. We excluded studies evaluating or comparing specific ART regimens or dosing frequencies, those without a comparator group, those without a defined intervention, and those without at least one quantifiable outcome (i.e., qualitative analyses). We included prospective and retrospective cohort studies, pre-post studies, and randomized clinical trials (RCTs); we excluded cross-sectional studies, mathematical modeling studies, and studies without empirically collected patient data. We included any study-defined outcomes and measures of intervention efficacy and quantified the variability in these outcome definitions.

Two reviewers independently evaluated titles, and then performed a review of abstracts to identify potentially relevant studies. We then conducted full-text review according to the established inclusion and exclusion criteria. Two reviewers abstracted the following data from all included studies: dates of study, location, intervention and comparator description, target population, eligibility criteria, study design, sample size, description of outcome measures, results, and any cost data. We utilized the Cochrane Collaboration’s tool to assess quality of included RCTs [14] and the Newcastle-Ottawa quality assessment scale for observational studies [15].

Data Analysis

Studies were classified based on the HIV care continuum step that the intervention sought to modify. We defined linkage studies as those targeting a population newly diagnosed with HIV and not yet in care, in which the outcome included time required to initiate care or percentage of the population establishing initial care within a defined period of time after diagnosis. Retention in care studies were those in which the target population was in care at the intervention initiation, with outcomes measuring any event that provided evidence of care engagement. By contrast, we defined reengagement studies to be those in which the target population was previously HIV-diagnosed (not newly diagnosed) but not currently engaged in HIV care. Adherence studies were defined to include populations currently in care and prescribed ART, with outcomes dependent on the degree of receipt and/or medication adherence to daily ART. If a publication included outcomes that fit our definitions for more than one care continuum step, it was included in multiple categories.

Anticipating significant heterogeneity of study outcomes within and across stages of the care continuum, we did not seek to perform meta-analyses. Rather, we approached summarization and characterization of data along the following domains. First, for each step in the care continuum we sought to qualitatively categorize interventions based on similarities of modality or approach utilized. We grouped education and behavioral/counseling interventions separately with systematic (non-individualized) interventions described as education, while behavioral/counseling interventions were those that appeared to be centered on a client-specific exchange. We additionally categorized the target populations included within intervention studies: general population, people who use drugs (PWUD) (any use), men who have sex with men (MSM), prison/jail, adolescents/youth, women, homeless population, other, and when possible racial/ethnic minorities. Next, we quantified the number of different study outcomes and effect measures reported across studies. Many studies reported multiple outcomes, and these were counted separately. Finally, we summarized intervention efficacy by characterizing the study outcomes as being significantly improved or not, as defined by each individual study (a study counted as having significant results if at least one outcome was statistically significantly improved in study defined statistical analysis). When possible we compiled effect sizes in tabular format and/or in descriptive analysis. For each step of the HIV care continuum, we additionally assessed whether study outcomes could be compared to metrics outlined in the updated NHAS progress indicators (i.e. % linking to care within one month, % retained in care among those diagnosed, % virally suppressed among those diagnosed). Finally, we also reported on whether costs and cost-effectiveness data were provided.

Results

Our search yielded 5786 articles, of which 152 were included in the final analysis (Fig. 1 and Supplemental Figure). Among included studies, 7 (5%) presented data on ‘linkage’, 33 (22%) on ‘retention‘, 4 (3%) on ‘reengagement’, and 117 (77%) on ‘adherence’, with some categorized to multiple care continuum steps (Supplemental Figure).

Linkage

Among 7 studies that targeted newly diagnosed HIV-positive persons (shown in Table 1), 4 (57%) were cohort studies while the remainder utilized a pre-post study design; there were no RCTs identified. None of the 7 ‘linkage’ studies assessed costs of their intervention. The target population in each of the studies (7/7, 100%) was the general population of PLHIV, without further targeting of risk groups.

Table 1.

Included linkage to care studies, systematic review of interventions to impact engagement along the HIV care continuum in the United States, 2007–2015

Study	Intervention type	Intervention description	Comparator	Target population	Significant results?	Outcomes	Intervention results	Comparator results	Cost data?
Cohort studies
Bocour et al. 2013 [40]	CM	Field Services Unit provided brief case management-like services as part of partner services	SOC	G	Y	1. Linkage to care within 3 months (CD4, VL as proxy for visit) 2. Established in care among those linked (2 lab tests >90 days apart)	79% 87%	66% 84%	N
Willis et al. 2013 [41]	CM	HIV medical case management for linkage	SOC	G	N	1. Linkage to care (based on CD4 and/or VL) at 3 months 2. Linkage to care (based on CD4 and/or VL) at 6 months	1. PEMS data source–80%; Surveillance data source–72% 2. PEMS data source–83 %; Surveillance data source–80%	1.PEMS–76%; Surveillance 80% 1.PEMS–81%; Surveillance 85%	N
Keller et al. 2011 [42]	HT	Oral-based point-of-service testing	Conventional tests	G	N	Linkage/engagement in care within 6 months (2 CV after post-test counseling visit)	52%	42%	N
Craw et al. 2008 [43]	CL	HIV medical care co-located at facility that participants received the ARTAS-II intervention	Non-co-located	G	Y	AOR for HIV medical care in past 6 months	AOR = 3.0 (95% Cl: 1.9 to 4.9)	Ref	N
Pre-post studies
Mugavero 2008 [44]	CM	Orientation visit within 5 days of initial visit to clinic	Pre-period	G	Y	% ‘No show’ at initial visit	19%	31%	N
Castel et al. 2014 [45]	P	City-wide initiative to implement routine opt-out testing and complete linkage with confirmatory positive test (2006–2009)	Pre-period (2005–2006)	G	Y	Linkage to care within 3 months of diagnosis (based on CD4 test or VL)	76%	51%	N
Onyeajam et al. 2013 P [46]	P	Routine, opt-out HIV testing (2008–2010) in clinical settings for all adults 13–64 years old (implemented in 2007 in South Carolina)	Pre-period (2004–2006)	G	Y	1. AOR to take >12 months to be in care (Intervention vs. Comparator) 2. AOR viral suppression at 12 months (Intervention vs. Comparator)	AOR = 0.42 (95% Cl 0.34–0.51) AOR = 8.4 (95% Cl 7.0–10.2)	Ref Ref	N

Bold rows represent studies with statistically significant improvements in linkage to care as defined by the study authors. When authors report multiple outcomes, these are shown independently in different rows

AOR adjusted odds ratio, CI confidence interval, CV clinic visit, Ref Reference, VL viral load, SOC standard of care

Intervention types include: CM case management, HT HIV testing modality, P policy, CL co-located services

Target populations: G general

PEMS Program Evaluation and Monitoring System. Study assessed outcomes from two data sources

Overall, we found no two studies that measured identical linkage to care outcomes (Supplemental Table), and no studies (0/7, 0%) reported data that would allow direct comparison against the current NHAS progress indicator for linkage to care targets (i.e. percent linked to care within one month of diagnosis date—see Fig. 1 for relevant NHAS indicators), though one study did directly assess viral suppression (NHAS indicator 6). Most studies (5/7, 71%) identified the proportion of patients that ‘established’ care after new diagnosis—measures of establishing care included documentation of a clinic visit, laboratory test, or ART initiation within a set period of time (varied between 3 months and 1 year). Effect sizes of interventions were largely reported as an absolute comparison of proportions (5/7, 71%), or relative measures such as hazard ratios or odds ratios (3/7, 43%). Alternatively, one study (14%) reported outcomes as a continuous measure of time until first visit.

Interventions were broadly categorized as involving case management (3/7, 43%, e.g. providing a case manager until an individual established HIV care), policy changes (2/7, 29%, e.g. routine opt-out HIV testing), change in HIV testing modality (1/7, 14%, e.g. rapid vs conventional tests), and co-location of care (1/7, 14%, e.g. HIV medical care co-located with ARTAS II site). Efficacy of these differing strategies was mixed. Two (2/3, 66%) case management interventions showed statistically significant improvements in linkage to care, two (2/2, 100%) policy interventions and the one co-location of services study also showed improvement. However, the overall effect size of these interventions was modest and ranged from an incremental 3% to 24% linked compared to standard of care.

Retention

Among 33 total studies that targeted patients already established in care, 31 (94%) studies evaluated 35 separate interventions addressing retention in care, while the remaining two (6%) addressed only the costs of such interventions. Of the 31 intervention studies, 13 (42%) were RCTs, while 10 (32%) were cohort studies, and 8 (26%) were pre-post study designs. The target populations studied included PWUD (13% of studies), adolescents/youth (10%), young black and Latino MSM (6%), prison or jail populations (6%), and other target populations (23%); 42% lacked a pre-specified target risk group.

Overall, these 31 studies utilized 39 different measures to evaluate the impact of retention in care interventions, with each study measuring from 1–5 (median = 2) outcomes (Supplemental Table). These study outcomes could be broadly grouped into four categories: change in clinic visits within given period of time, change in the number of laboratory tests (such as CD4 count) within a given period of time, change in ART prescriptions or ART usage, and change in viral load/viral suppression. Follow-up times at which the outcome assessments were made varied across studies. Methodology for reporting effect sizes was heterogeneous. Some studies reported dichotomized data (23/31, 74%) based on the number or percentage of the study participants that met the study definitions for being retained in care over the study period, while others reported continuous data (13/31, 42%, e.g. mean number or proportion of clinic visits kept). Eleven (35%) reported on the downstream care continuum target of viral suppression. The current NHAS progress indicator related to retention in care (NHAS indicator 5) seeks to engage (cross-sectionally) 90% of all diagnosed persons in care. Among the 31 intervention studies, 19 (61%) provided results in a format (i.e. proportion in care among intervention group) that would allow some comparison to this NHAS progress indicator, while only 3 (10%) used the precise outcome of retention (two care visits, 90 days apart, within the calendar year) listed by the NHAS.

Interventions studied by investigators were diverse (Table 2). The majority of interventions focused on implementing novel technology (7/31, 23%, example: text message appointment reminders, interactive clinical decision-support with alerts for poor patient outcomes), case management or outreach (8/31, 26%, example: enhanced personal contact, medical case management), or counseling/behavior modification strategies (8/31, 26%, example: motivational interviewing, peer mentoring).

Table 2.

Included retention in care intervention studies, systematic review of interventions to impact engagement along the HIV care continuum in the United States, 2007–2015

Study	Intervention type	Intervention description	Comparator	Target population	Significant results?	Outcomes	Intervention results	Comparator results	Cost data?a
Randomized controlled trials
Gardner et al. 2014 [16]	CMO	1. Enhanced personal contact (EC) (12 month)	SOC	G	Y	1. Visit constancy, % keeping 1 visit for 3 consecutive quarters 2. Visit Adherence, % kept appointments	56% 73%	46% 67%	Y [47]
	CMO, E	2. EC plus basic HIV education (EC+ skills)	SOC		Y	1. Visit constancy, % keeping 1 visit for 3 consecutive quarters 2. Visit Adherence, % kept appointments	56% 71%	46% 67%
Gwadz et al. 2015 [48]	BC	Motivational interviewing, support partner, navigation, support groups	SOC	YBL-MSM	N	% attended at least one primary care visit in past 6 mo	60%	48%	N
Konkle-Parker et al. 2014 [17]	E, BC	One-on-one educational sessions, motivational interviewing, social motivation, and adherence enhancing devices (6 month)	SOC	G	N	% with a visit in each 4-month block over 12 months: year prior to, and year of study	34%, 18%	46%, 20%	N
Konkle-Parker et al. 2012 [18]	E, BC	One-on-one sessions and phone calls including: HIV education, peer video about adherence, motivational interviewing, feedback, communication training and adherence enhancing devices (3 visits)	SOC	G	N	1. Mean number of missed medical visits 2. % Viral suppression	0.79 75%	1.11 78%	N
Lucas et al. 2010 [20]	IS	Clinic-based buprenorphine/naloxone (12 months)	Referred out for opioid tx	PWUD	Y	1. Number of visits 2. Months receiving ART	3.5 11 months	3.0 12 months	N
MacGowan et al. 2015 [19]	E	Education and skills building (4 sessions pre-release [3 months]; 2 sessions post-release [3 months])	SOC	PJ	N	% reporting healthcare at HIV clinic: baseline, 3-month follow-up	63%, 84%	44%, 63%	N
Naar-King et al. 2009 [49]	BC	Motivational interviewing by peer outreach workers (2 sessions)	Masters-level staff outreach	AY	N	Gap score (1 point per quarter with missed appointment) change pre to post intervention	−1.50 gaps	−1.06 gaps	N
Norton et al. 2014 [22]	T	Text message appointment reminders	SOC	G	N	Next appointment attendance	72%	81%	N
Proeschold-Bell et al. 2010 [23]	T, C	Health information exchange between ancillary care sites and medical care providers (2 years)	SOC	G	N	1. % suppressed viral load at baseline, 12 and 24 months 2. % prescribed ART at baseline, 12 and 24 mo	56%, 54%, 55% 72%, 79%, 81%	42%, 49%, 54% 72%, 76%, 79%	N
Purcell et al. 2007 [50]	BC	Intervention to develop participants as informal peer mentors (10 sessions)	Video discussion	PWUD	N	HIV care 2+ times in past 6 months: baseline, 6 & 12 mo	71%, 71%, 69%	69%, 72%, 64%	N
Robbins et al. 2012 [21]	T, C	Interactive clinical decision-support system that generates alerts to notify HIV outpatient providers of poor patient outcomes (1 year)	Static alerts	G	Y	Rate of 6-month suboptimal follow-up (events per 100 patient years)	20.6/100 patient years	30.1/100 patient years	N
Wohl et al. 2011 [51]	CMO	Bridging case management following release from prison (3 months prior and 6 months after release)	SOC	PJ	N	Linkage/Continuation of care (at least one medical appointment) by week 4, 12, 24 post release	65%, 88%, 91%	54%, 78%, 89%	N
Wolitski et al. 2010 [52]	H	Immediate HOPWA rental assistance	Customary housing services	O	N	% any medical care in the past 6 months at: baseline, 6, 12, and 18-month follow-up	85%, 70%, 77%, 78%	80%, 71%, 72%, 73%	N
Cohort studies
Altice et al. 2011 [53]	IS	Integration of HIV care with BUP/NX treatment for 3–4 quarters (Evaluated over 12 months)	BUP/NX for <3 quarters	PWUD	Y*	1. ART initiation among those not on ART at baseline 2. Viral suppression among those not on ART at baseline	β = 1.34 (95% Cl 1.18, 1.53) β = 1.25 (95% Cl 1.10, 1.42)	Ref Ref	N
Bogart et al. 2012 [54]	BC, CMO	Treatment Advocacy Program including counseling and motivational interviewing, patient–provider relationship building, referrals to necessary services	SOC	G	N	Odds of attending at least one medical visit and not reporting any missed visits (3 months)	AOR = 2.64 (95% Cl 0.67, 10.36)	Ref	N
Cabral et al. 2007 [55]	CMO	Outreach, advocacy and some support services (9+ contacts within 3 months)	No intervention contacts in first 3 months of program	G	Y	% with gap in care (over 12 months)	22%	39%	N
Cunningham et al. 2011 [56]	IS	Integrated HIV and opioid addiction treatment with buprenorphine (6 months)	Non-integrated opioid tx	PWUD	Y	1. Median clinic visits (6 mo) 2. Lab tests (≥ 2 CD4 counts) (6 months) 3. % VS (6 months)	8 53% 47%	2 8% 0%	N
Davila et al. 2013 [57]	IS	1. Centralized multidisciplinary youth clinic (C)	De-centralized	AY	Y	1. Over 12 months: “Adequate” visit constancy (3 quarters with at least one visit) 2. No gap in care >180 days between visits	57% 80%	31% 83%	N
	IS, BC, E	2. Centralized multidisciplinary youth clinic with youth-specific support groups and education activities (12 months) (C + ES)	Centralized		Y	1. Over 12 months: “Adequate” visit constancy (3 quarters with at least one visit) 2. No gap in care >180 days between visits	65% 96%	57% 80%
Hanna et al. 2013 [58]	P	1. ADAP state characteristics—state funding provided to the annual ADAP budget	No state funding	G	Y	ART initiation within 6 months of ART eligibility	58%	39%	N
	P	2. ADAP state characteristics—no use of ADAP waiting lists in the state	Waiting lists		N	ART initiation within 6 months of ART eligibility	55%	73%
Himelhoch et al. 2009 [59]	IS	6+ mental health visits	No mental health visits	O	Y	Discrete time survival analysis ART discontinuation (AOR comparing intervention to comparator)	6–11 visits/yr AOR 0.78 (95% Cl 0.61–1.00); 12+ visits/year 0.60 (95% Cl 0.45–0.80)	Ref	N
Schranz et al. 2015 [60]	C	Hospital-based HIV clinics	Community-based HIV clinics	G	N	1. Retention in care (2+ outpatient visits separated by 90+ days in year) 2. Retained and on ART 3. Retained on ART and suppressed	82% 72% 59%	85% 74% 59%	N
Terzian et al. 2015 [61]	H	NYC HOPWA provided rental assistance, housing placement assistance, and supportive permanent housing (receiving intervention during 2011)	No housing assistance	O	Y N	1. Retention in care (2 + lab tests >3 months apart over 12 months) 2. % viral suppression (at least once in 12 months)	94% 78%	82% 82%	N
Willis et al. 2013 [62]	CMO	HIV medical case management (MCM) (fiscal year 2010)	Non-MCM funded facility	G	Y	Retention in care (2+ lab tests 3+ months apart) % (AOR)	76% (AOR 4.13 [1.93–8.85])	60% (Ref)	N
Pre-post studies
Andersen et al. 2007 [62]	Tr	1. Transportation only [TO] (12 mo)	Pre-period	W	N	Mean number of medical visits for 6 months pre-(comparator), 6 months post-intervention, 12 months post-intervention	1.5, 1.1 mean visits	1.3 mean visits	N
	Tr, BC	2. Transportation Plus Personalized Counseling [TO Plus] (12 months of Transportation with initial 6 months of counseling)	Pre-period		Y	Mean number of medical visits for 6 months pre-(comparator), 6 months post-intervention, 12 months post-intervention	1.6, 1.0 mean visits	1.1 mean visits
Enriquez et al. 2008 [63]	C	Bilingual/bicultural team (12 months post)	12 months pre-period	O	Y	1. Mean clinic visits 2. % viral suppression (end of 12 months pre- and post-periods)	5.30 visits 53%	2.81 visits 21%	N
Henry et al. 2012 [64]	T	Automated telephone reminders 2 weeks prior to regularly scheduled HIV clinic appointments (6 months)	6 months pre-period	O	Y*	% missed appts overall (% missed among non-depressed)	24% (18%)	24% (23%)	N
Hightow-Weidman et al. 2011 [65]	T, CMO	Social marketing campaign, intensified outreach, tightly linked medical-social support network (3 years)	3 year pre-period	YBL-MSM	Y	% of visits attended (OR)	80% (OR 2.58[95% Cl 1.34–4.98])	67% (Ref)	N
Irvine et al. 2015 [66]	CMO	HIV Care Coordination Program included outreach, case management, multidisciplinary care team communication, patient navigation, adherence support and health promotion (1 year)	12 months pre-period	G	Y	1. % engaged in care (≥ 2 laboratory tests 90 days apart) 2. % viral suppression (on most recent test in second half of 12-month period)	91% 51%	74% 32%	N
Saifu et al. 2012 [67]	T	Telemedicine clinic encounters for HIV (2009–2011) and HCV patients (2010–2011) at “spoke” sites with healthcare providers at “hub” sites	In-person clinic visits (pre-period starting 2008)	O	Y	Appointment completion	76%	61%	N
Shade et al. 2015 [68]	T, C	6 Health Information Technology Interventions at 6 sites (6 months)	6 months pre-period	G	Y*	1. % engaged in care (≥ 1 visit or lab test over 6 months) 2. % ART prescription 3. % viral suppression across 6 intervention sites	83–96% 75–92% 11–68%	65–100% 73–89% 3–62%	N
Gardner et al. 2012 [69]	C	Clinic-wide printed reminder materials and verbal messages used by clinic staff (12 months)	12 months pre-period	G	Y	1. % keeping next 2 visits 2. Mean % of visits kept	53% 70%	49% 68%	Y [70]

Bold rows represent study outcomes with statistically significant improvements in retention in care as defined by the study authors. When authors report multiple interventions and/or outcomes, these are shown in different rows. When possible, the timing of outcome measurement is indicated; if not listed, indicates unclear or unreported timing. In the results column, we present only selected representative analyses and results from each study and some outcomes and subgroup analyses are not shown

ADAP AIDS Drug Assistance Program, AOR adjusted odds ratio, appt appointment, BUP/NX buprenorphine/naloxone, CI confidence interval, OR odds ratio, Ref Reference, SOC standard of care, tx treatment

Intervention types include: BC behavioral/counseling, CMO case management/outreach, C clinic-wide (such as a clinic-wide social marketing campaign, integration of care across specialties, or shared medical records), E education, H housing assistance, IS integration of services, P policy, T technology-based, Tr transportation

Target populations include: AY adolescents/youth, G general, MSM men who have sex with men, O other, PJ prison/jail, PWUD people who use drugs

Y in significant results column indicates that any significant result was presented and does not imply that all analyses were significant. Y* indicates that results were significant in subgroup analyses only, or within only some sites

^aTwo studies assessing the costs of parent efficacy studies were published separately

A low proportion of studies that evaluated behavior modification/counseling (1/7, 14%) found significant improvements in care retention. By contrast, integration of services (5/5, 100%), and to a lesser extent, case management (5/7, 71%), technology (5/7, 71%) and clinic-based interventions (4/6, 67%) all had higher proportions of potentially efficacious interventions according to the effect measures and outcomes chosen by the study investigators. The heterogeneity of these interventions, target groups, and effect measures precluded meta-analysis. Overall, the effect sizes of interventions with significant results (bold rows, Table 2) were modest, and few (3/31) assessed outcomes longitudinally over more than 1 year. Of the included studies and interventions that could be assessed against the NHAS progress indicator 5, only 4/19 (21%) reported achieving retention of 90% of study participants at 1 year (or the end of their study period).

Among the most rigorously designed studies (RCTs), only 3/13 (23%) reported significantly improved retention among the intervention groups. None (0/5, 0%) of the RCTs that centered on behavioral or counseling interventions demonstrated significant impact on care retention. The effect sizes and interventions among RCTs suggesting improvements in retention in care varied. Gardner et al. implemented an intervention for education and enhanced personal contact for a general HIV clinic population over 12 months and found that visit adherence (defined as proportion kept out of all scheduled primary care visits) increased from 67% to 73% [16]. By contrast, however, three other RCTs evaluating educationally focused interventions found no significant effect [17–19]. Lucas et al. found that clinic-based substance abuse treatment modestly improved the number of clinic visits over the twelve month study period from 3 to 3.5 when comparing control to intervention arms [20]. Robbins et al. studied a technology-based decision support intervention to alert providers to poor patient outcomes and found a 9.5 per 100 person year reduction in the rates of 6 month suboptimal follow-up [21]; two other technology-centered interventions, however, showed no significant change in retention in care [22, 23].

Reengagement

Few studies (4) assessed reengagement in care (see Table 3), and none of these studies addressed costs. One of these was a cohort study, while the other three employed a pre-post study design. All four studies addressed general populations of PLHIV who were out of care. Intervention types were variable, and included provider notifications, policy (routine opt-out HIV screening), case management, collaboration between clinics and health departments, and navigation-like interventions. Reengagement studies reported on 8 different outcomes; three of the four studies reported viral load or viral suppression as one of their outcomes. Three of these four studies found significant improvements in engagement in care following the intervention, though these significant impacts included a range of effect sizes (such as a 5% increase in re-linkage [24], 11.2% increase in viral suppression [25], and 3.9–5.4% reduction in no care in the past 6 months [26]).

Table 3.

Included reengagement studies, systematic review of interventions to impact engagement along the HIV care continuum in the United States, 2007–2015

Studies	Intervention type	Intervention description	Comparator	Target population	Significant results?	Outcomes	Intervention results	Comparator results	Cost data?
Magnus et al. 2012 [71]	PN	Automatic notification to providers when out of care PLHIV registers at care facility, decision support to staff to engage patients in HIV care	Previously experienced delayed entry into care	G	N	1. AOR VL > 10,000 copies/ml 2. AOR ART Prescription	1.72 (95% CI 0.95–3.14) 0.86 (95% CI 0.38–1.93	Ref Ref	N
Flash et al. 2015 [25]	P, CM	Routine opt-out HIV screening and nonmedical case management by service linkage workers—evaluated among those previously tested positive	Pre-period	G	Y	1. Engaged in care (HIV primary care visit in 6 months) 2. Viral suppression	59%, 34%	41%, 23%	N
Bove et al. 2015 [24]	C	HIV clinic & local health department collaboration to identify disengaged patients and relink with a linkage specialist	Historical cohort	G	Y	Relinked in one year	15%	10%	N
Bradford et al. 2007 [26]	N	Navigation-like interventions	Pre-period	G	Y	1. No care in the past 6 months for 6 months pre- (comparator), 6 months post-intervention, 12 months post-intervention 2. Viral Suppression at baseline (comparator), 6 months, 12 months	5%, 8% 54%, 53%	12% 35%	N

Bold rows represent studies with statistically significant improvements in reengagement in care as defined by the study authors. When authors report multiple interventions and/or outcomes, these are shown independently in different rows

AOR adjusted odds ratio. CI confidence interval. Ref reference

Intervention types include: C collaboration between clinics and health departments, CM case management, N navigation, P policy, PN provider notifications

Target populations include: G general

Adherence

Among 117 total studies evaluating interventions addressing medication adherence among patients in care, 111 adherence studies evaluated efficacy alone, 4 addressed costs alone, and 2 addressed both efficacy and costs. Of the 113 efficacy studies, 65 (57%) were RCTs, while 20 (18%) were cohort studies, and 28 (25%) were pre-post study designs. Nearly half of studies (53/113, 47%) targeted a general population of PLHIV taking ART, while 23 (20%) targeted PWUD, 9 (8%) targeted adolescents/youth and 8 (7%) targeted women. Only 2 (2%) studies exclusively recruited MSM, and an additional 2 (2%) exclusively recruited racial minorities. Interventions ranged from adjunctive treatment for drug use, to active reminder systems (using technology) to clinic-wide interventions that were not individually targeted (Table 4).

Table 4.

Included adherence studies by intervention type, systematic review of interventions to impact engagement along the HIV care continuum in the United States, 2007–2015

Intervention type	Count	Intervention examples	Study design	Target populationab	% Reporting viral suppression	Significant results? (%)	% with cost data	Studies
Drug use treatment	5	Integrated BUP/NX tx for opioid dependence	RCT: 1 Cohort: 3 Pre-post: 1	PWUD-PJ: 2 PWUD-W: 1 Other: 1 PWUD: 1	60	60	0	[53, 72–75]
Financial incentives	6	Contingency management	RCT: 4 Pre-post: 2	Other: 1 PWUD: 5	67	83	17 [76]	[76–81]
Structural	5	Ecosystem intervention, harm reduction housing program	RCT: 3 Cohort: 1 Pre-post: 1	General: 1 Other: 2 Prison/Jail: 1 PWUD: 1	20	80	0	[82–86]
Pharmacy-based	10	HIV-specialized pharmacies, pharmacist-managed drug-optimization, pharmacist adherence counseling	Cohort: 6 Pre-post: 4	General: 9 AY: 1	50	70	0	[31, 32, 87–94]
Active reminder devices	7	Phone adherence support reminders, SMS reminders, two-way SMS messaging, Wisepill device with immediate phone adherence counseling on missed dose	RCT: 5 Pre-post: 2	General: 3 Other: 1 AY: 2 MSM: 1	14	71	14 [95]	[96–102]
Passive reminder devices	5	Pillboxes	RCT: 3 Cohort: 1 Pre-post: 1	General: 3 PWUD: 1 Homeless: 1	80	60	0	[17, 18, 103–105]
Treatment supporter	7	Visits from community health workers, visits from advanced practice nurse to community, adherence support partner, peer support	RCT: 7	General: 5 Other: 1 MSM: 1	28	57	14 [106]	[102, 107–112]
DAART	15	Directly administered ART in health care vans, methadone clinics, during hospitalization	RCT: 8 Cohort: 1 Pre-post: 6	General: 3 PWUD: 8 Prison/Jail: 1 AY: 2 BPWUD: 1	93	60	0	[28, 33, 34, 104, 113–123]
Clinic-based	5	Clinic-wide social marketing campaign, multi-disciplinary care team, shared medical record, adherence data shared with provider	RCT: 1 Cohort: 2 Pre-post: 2	General: 5	20	60	0	[124–128]
Behavioral/counseling	37	Motivational interviewing, cognitive behavioral therapy, peer mentoring, group and individual counseling, delivered via phone and in person	RCT: 29 Cohort: 3 Pre-post: 5	General: 19 Other: 5 AY: 2 Women: 5 Black/Latino: 1	43	51	5 [27, 129]	[17, 18, 27, 30, 48, 50, 54, 56, 78, 101, 130-156]
Education	13	PDA video, multimedia education with quizzes, interactive computer-based tutorial, pill swallowing demonstration, audio music program, group and one-on-one educational sessions, printouts	RCT: 9 Pre-post: 4	General: 10 AY: 2 Women: 1	46	31	8 [157]	[17, 18, 29, 111, 158-166]
Depression treatment	4	SSRIs (including direct administration), collaborative care, therapy groups	RCT: 3 Cohort: 1	Other: 3 Homeless: 1	50	25	0	[73, 167–169]
Case management	2	Transitional care coordination when leaving prison, medical case management	Cohort: 1 Pre-post: 1	General: 1 Prison/Jail: 1	50	0	0	[41, 170]
Other	2	Unannounced pill counts, journal writing	RCT: 1 Cohort: 1	General: 1 Women: 1	0	0	0	[171, 172]

Viral suppression is as defined by the authors and thus varies between studies (cutoffs ranged from 20-400 copies/ml as the limit of detection). The “Significant results” column indicates studies that found any significant results in primary or secondary analyses. Studies were allowed to belong to multiple “Intervention types” if the implemented intervention contained more than one component

BUP/NX buprenorphine/naloxone, PDA personal digital assistant, RCT randomized clinical trial, SSRI selective serotonin reuptake inhibitor, tx treatment, AY = adolescents/youth, MSM = men who have sex with men, PWUD = people who use drugs, BPWUD Black people who use drugs, PWUD-PJ people who use drugs among prison/jail populations, PWUD-W women who use drugs. Other people with mental health diagnoses, veterans, those who abuse alcohol, and high-risk insurance pool members

Outcomes of interest ranged from adherence measures to biological outcomes, with 50% reporting on viral suppression as an outcome. Adherence was measured in numerous ways, including self-report [using several measures, including the visual analog scale (VAS) and the AIDS clinical trial group (ACTG) questionnaire], electronic pill bottles [such as Medication Event Monitoring System (MEMS) caps], pill counting, medication refills, and presence of antiretrovirals in specimens (such as hair or plasma). Some of these measures included a time component (proportion of doses taken on time), and measures used a variety of recall periods (3 days, 7 days, 3 weeks, etc.).

Efficacy of adherence interventions was extremely varied across and within intervention types (Table 4). While the greatest number of studies focused on counseling (37, 33%), half (51%) of these reported improved adherence and the remainder indicated no improvement or worsened adherence. NHAS progress indicator 6 calls for achieving 80% viral suppression among those diagnosed. In these cohorts of individuals that were diagnosed (and in care), 8 studies (15% of the 52 studies for which it was possible to assess) reported intervention viral suppression results that met this target. Assessing virologic outcomes was limited by methodologic issues—for example, cutoffs for viral suppression varied from <20 to <400 copies/ml (limiting comparability), some patients met criteria for viral suppression at some time points but not longitudinally, and some studies reported viral loads but did not formally assess viral suppression. Among the 8 studies that achieved at least 80% virologic suppression [27–34], the types of interventions varied [3 directly administered ART (DAART), 2 behavioral/counseling, 2 pharmacy-based and 1 education], as did the target population being addressed (6 general, 1 women, 1 PWUD), and the type of study design (4 RCTs, 1 cohort, 3 pre-post). Nonetheless, only four of these studies [27, 29, 32, 34] found a significant improvement in viral suppression in the intervention group versus the comparator group.

Study Quality

The quality of included studies varied substantially. Among the 74 included RCTs, most (63/74) reported random sequence generation though some (10/74) did not clearly describe their randomization method. Few studies clearly described concealment of treatment allocation (18/74). Very few RCTs (9/74) were completely blinded/-masked (largely due to the nature of interventions assessed), though few (21/74) used outcome assessors blinded/masked to treatment assignment either. A high proportion of RCTs (31/74) were deemed to be at high risk of bias due to attrition from the studies. Among 73 observational studies, most (64/73) were assessed to be somewhat or truly representative of the population of interest. Most (49/73) collected outcomes by linking to medical records, with just over a quarter using self-reported outcomes (20/73). Observational studies also had high levels of attrition, with only 32/73 having over 90% follow-up. Among 31 cohort studies, most (26/31) accounted for confounding in their analyses.

Discussion

Successful strengthening of the care continuum will require combination of, and prioritization between, different interventions—tasks that are difficult without common metrics to evaluate the effect of interventions on outcomes. Our review highlights the tremendous degree of heterogeneity across existing studies to improve the HIV continuum of care. We found a high degree of heterogeneity when defining outcomes of interest, as well as in measuring effect sizes of interventions. Moreover, only 3/38 linkage and retention studies offered care engagement results in a format that could be compared to external metrics of success set forth in the newest NHAS. These elements of study heterogeneity threaten our ability to effectively select the combinations of interventions likely to have the greatest effect. Additionally, models suggest that as little as 3.3% of HIV transmissions in the US occur as a result of individuals on ART but not virally suppressed (the population targeted by adherence interventions) whereas the population diagnosed but not retained in care contribute as much as 61.3% of infections [35]. We reveal a disconnect between the areas of the HIV continuum of care where greatest impact could be achieved (retention and reengagement) and the areas where intervention evidence is strongest (adherence, representing 77% of studies identified). We identify a paucity of evidence to guide interventions targeting persons at high risk (e.g., MSM, transgender individuals), a gap which has been identified previously [36, 37]. Further, we identify few studies that present cost data, despite the widespread understanding that funders and health departments need these data in an era of largely flat budgets. These findings suggest a need for a consensus process to develop more clear guidance to the HIV research community about which types of interventions are in greatest need of study, which populations should be targeted, and which outcomes should be measured in order to develop a more effective and coordinated HIV response in the US.

Overall, we found few consistent themes to suggest efficacy of specific interventions, or broader intervention types. Among studies of individuals in care, we found that a majority of studies that used integrated services (such as co-located substance abuse treatment with HIV treatment), case management, technology (such as provider alerts and automated messages to patients) and clinic-based interventions (such as clinic-wide messaging and bilingual care teams) had significantly improved results in retention. Financial incentives, active reminder devices, structural and pharmacy-based interventions had the highest proportion of significantly improved results among the included adherence studies. The CDC’s Prevention Research Synthesis compendium provides an excellent resource for policymakers looking to select an evidence-based intervention in their population of interest [13]. We note that given the heterogeneity in study populations and locally specific intervention details, that it is difficult to draw any conclusions about the generalizability of these intervention types across settings.

While the NHAS has outlined specific targets, which are laudable in outlining objective goals for providers, program managers, and researchers to assess progress along the HIV care continuum, we found that these metrics are mismatched to recent research efforts in important ways. For example, NHAS indicators for care retention and viral suppression (indicators 5 and 6) have denominators of all individuals who are diagnosed with HIV, while in general, retention and adherence studies have denominators of people who are in care. In order for researchers or program managers to easily compare their results of a retention or adherence intervention to these measures, they would have to extrapolate to the proportion diagnosed instead of directly comparing their data.

Beyond the challenges to comparing intervention effects with NHAS indicators, we found no consensus in how to define or measure any stage of engagement along the HIV care continuum. First, we found 11 different outcomes used in the literature to assess initial linkage to care, and 39 different outcomes among retention in care studies. This heterogeneity makes it very difficult to compare interventions or intervention types, and precludes our ability to draw any solid conclusions regarding intervention efficacy. While work has been done to consolidate definitions [38], this is not resulting in consolidation among researchers. Second, achieving 80% viral suppression among those diagnosed is part of the NHAS (indicator 6), but only 1/7 (14%) studies of linkage interventions, 11/31 (35%) studies of retention interventions, and only 56/113 (50%) studies of adherence interventions evaluated improvements in achieving the ultimate goal of viral suppression. By measuring ‘upstream’ effects, it is not possible to assess the net benefit of a care continuum intervention, which may be compromised by gaps in downstream care engagement (i.e., interventions showing incremental increases in initial linkage to care may have reduced net benefits due to poor downstream longitudinal retention in care). Third, compounding the challenge in evaluating the literature is the lack of standardization in how intervention efficacy is assessed. Beyond variability in defining outcomes of interest, researchers assessed intervention effects in a varied manner; as such, we were unable to provide pooled estimates of incremental benefit for any specific interventions. Finally, engagement in HIV care is dynamic over time and continuous retention and viral suppression is needed to reduce transmission and improve health [39]. As such, cross-sectional or short-term assessments of care engagement may misrepresent longitudinal efficacy. In our review, we found that very few studies assessed outcomes beyond a few months.

Throughout our analysis, we have highlighted the relative distribution of RCTs to observational studies, recognizing that observational study designs may have greater risk of some biases. However, it is important to note that causal inference methods have made tremendous strides to allow observational studies to better approximate randomized designs through novel analytic methods and designs which take advantage of “natural experiments.” Such novel methods of analysis or study designs would be a benefit to the care continuum research community, and appear to be under-utilized. For instance, propensity score matching, a method to make groups exposed and unexposed to an intervention as comparable as possible in observational studies, was used by only 4 cohort studies out of 31 included in our review. Other methods that take advantage of natural experiments or changes over time, such as instrumental variables, regression discontinuity, and interrupted time series could also help in providing valid causal inference regarding the effect of studied interventions. Given some of the resource and practical challenges of incorporating randomized trial designs into implementation research on care continuum interventions, investigators may wish to consider incorporation of alternative novel study designs and analytic methods in the future.

Given our findings, we recommend several steps be taken in order to improve the evidence base for strengthening HIV care engagement in the US. First, researchers, health departments and funders must come to consensus on definitions for linkage, retention and reengagement in care, and ART adherence for the purposes of scientific investigation. Second, we would propose that irrespective of the step in the care continuum being targeted, that a central outcome of interest (in addition to the proximal impact of an intervention) is the incremental number or percentage of individuals achieving viral suppression (over a defined period of time, such as a year). Since directly studying viral suppression may not be feasible in a study of ‘upstream’ interventions (such as linkage interventions), we propose that a tool be developed to aid researchers to translate upstream interventions to viral suppression outcomes. Such a tool or calculator would incorporate locally specific care continuum data to allow translation of incremental impact from upstream interventions into an incremental change in viral suppression; while such an approach would require several assumptions and have limitations, it would allow a common framework to measure care continuum interventions. By utilizing a consensus downstream effect measure, one begins to be able to compare absolute levels of benefit of interventions at varying stages in the care continuum. Third, studies of novel care continuum interventions need carefully constructed study designs. We found that over 1/3 of studies identified for full text review in this study (see Appendix for PRISMA diagram) were excluded because of a lack of a defined comparator group (i.e. one armed studies) or lack of quantifiable outcomes. Studies should clearly define the intervention, population under study, and distinguish the intervention group from a comparator group unexposed to the intervention, while clearly defining the comparator (for instance, “standard of care” needs to be explicitly defined). While qualitative information about interventions is vital to understanding aspects of acceptability, usability, and scalability, quantifiable outcomes and standard measures of effect size should be established. Fourth, the vast majority of interventions (95%) addressed only a single step of the care continuum, which fails to capture that care is lifelong and involves ongoing engagement and movement between continuum “steps.” Therefore, we recommend that where possible, integration of different steps of the care continuum be assessed. In conjunction with this suggestion, we acknowledge that within a given step of the care cascade, multiple intervention modalities could be implemented; thus far, a minority of studies implemented multi-modal interventions. It is unclear whether combination interventions will result in independent effects, and as such testing combination interventions should be prioritized. Moreover, while many of the absolute effect sizes for interventions were small, these effects could offer meaningful impact when scaled-up at a population level; future studies may consider incorporation of modeling approaches that allows one to extrapolate the potential impact of HIV care continuum strengthening at a population level. Fifth, we recommend that observational designs increasingly use appropriate causal inference methods and study designs where possible to allow greater confidence in the results of these studies. Finally, as the care continuum is dynamic over time, we propose that reporting of longitudinal measures of care continuum outcomes be prioritized. As an example, reporting the incremental change in the rate of loss from care per unit time (as a metric for evaluating interventions to improve retention in care) would allow a more comprehensive understanding of intervention efficacy; currently most reported outcomes in the studies identified in this review were limited by their cross-sectional nature and short time period for follow up.

Our review has several important limitations. First, in order to offer a comprehensive picture of all research studying interventions to strengthen the HIV continuum of care, we present studies of highly varying quality (for instance RCTs vs pre-post design, and with substantial variability within each of those design types) without an attempt to exclude low-quality studies. Other reviews (with different purposes) restrict themselves exclusively to interventions with a high-quality evidence base and the CDC Prevention Research Synthesis Team has compiled this data to promote best practices, despite limited data [7, 9, 13]. Second, since our purpose was to characterize the breadth of HIV care continuum interventions, we also were not able to evaluate any specific set of interventions (e.g. those related to adherence) in substantial depth. Other systematic reviews [7–10] have aimed to fill this gap. Third, due to the substantial variability in outcomes, study design, and target populations, we were unable to conduct a meta-analysis. The heterogeneity that we characterize, however, can help motivate HIV researchers to unify their methodology to enable future research to be more easily summarized. Additionally, we had no limit for the year of implementation of the study (only for year of publication) so some findings may be less relevant for the current care context.

In conclusion, this systematic review of over 150 interventions to strengthen the continuum of HIV care in the US highlights a small number of effective interventions but more importantly reveals tremendous heterogeneity in methodology and outcome assessment. Furthermore, there is a paucity of data on linkage and reengagement, less evidence on retention than adherence, and few studies targeting populations experiencing the highest incidence of HIV in the US (MSM, particularly black and Latino MSM). If we are to develop a coordinated strategy to achieve the ambitious NHAS targets, increased attention must be paid to filling the biggest gaps in the current continuum of care. Researchers must also report outcomes in a standardized fashion—with focus on the ultimate outcome of viral suppression—that will enable combination and prioritization. The current piecemeal approach to HIV continuum research must be improved; by looking at the HIV continuum of care from an integrated, top-down perspective, a more comprehensive strategy can be created to end the HIV epidemic in the US.