Missed HIV care visits have independent prognostic value for clinical events beyond core indicators of retention in care. As this information is readily available and immediately actionable, missed HIV care visits should be incorporated into clinical, programmatic, and policy initiatives.
Keywords: HIV, AIDS, antiretroviral therapy, engagement in care, continuum of care
Abstract
Background. The continuum of care is at the forefront of the domestic human immunodeficiency virus (HIV) agenda, with the Institute of Medicine (IOM) and Department of Health and Human Services (DHHS) recently releasing clinical core indicators. Core indicators for retention in care are calculated based on attended HIV care clinic visits. Beyond these retention core indicators, we evaluated the additional prognostic value of missed clinic visits for all-cause mortality.
Methods. We conducted a multisite cohort study of 3672 antiretroviral-naive patients initiating antiretroviral therapy (ART) during 2000–2010. Retention in care was measured by the IOM and DHHS core indicators (2 attended visits at defined intervals per 12-month period), and also as a count of missed primary HIV care visits (no show) during a 24-month measurement period following ART initiation. All-cause mortality was ascertained by query of the Social Security Death Index and/or National Death Index, with adjusted survival analyses starting at 24 months after ART initiation.
Results. Among participants, 64% and 59% met the IOM and DHHS retention core indicators, respectively, at 24 months. Subsequently, 332 patients died during 16 102 person-years of follow-up. Failure to achieve the IOM and DHHS indicators through 24 months following ART initiation increased mortality (hazard ratio [HR] = 2.23; 95% confidence interval [CI], 1.79–2.80 and HR = 2.36; 95% CI, 1.89–2.96, respectively). Among patients classified as retained by the IOM or DHHS clinical core indicators, >2 missed visits further increased mortality risk (HR = 3.61; 95% CI, 2.35–5.55 and HR = 3.62; 95% CI, 2.30–5.68, respectively).
Conclusions. Beyond HIV retention core indicators, missed clinic visits were independently associated with all-cause mortality. Caution is warranted in relying solely upon retention in care core indicators for policy, clinical, and programmatic purposes.
(See the Editorial Commentary by Armstrong and del Rio on pages 1480–2.)
In recent years, considerable attention has focused on the importance of engagement in human immunodeficiency virus (HIV) medical care in contributing to individual and public health outcomes. The HIV care continuum (“treatment cascade”) has become the sentinel image depicting the domestic HIV epidemic across a sequence of steps including acquisition of HIV infection, HIV diagnosis, linkage to medical care, retention in medical care, antiretroviral therapy (ART) receipt, and plasma viral suppression (<200 copies/mL) [1–4]. Of the estimated 1.2 million people living with HIV infection in the United States, only 25% have achieved plasma viral suppression, with dramatic drop-offs in linkage and retention in medical care representing the most prominent barriers to achieving this vital surrogate marker of effective treatment. Research has clearly shown that achieving and sustaining plasma viral suppression is associated with a decreased frequency of clinical events, including mortality, and with dramatic reductions in HIV transmission [5–7]. However, more than half of persons diagnosed with HIV infection in the United States are not engaged in ongoing medical care [8], making retention in care the greatest barrier to fully achieving the individual and population health benefits afforded by viral suppression [9, 10]. Accordingly, the US National HIV/AIDS Strategy and HIV Care Continuum Initiative, recently released by executive order, place considerable focus on HIV care engagement as a critical component to achieving the overarching goals of reducing new HIV infections, improving health outcomes for people living with HIV, and reducing HIV-related health disparities [11, 12]. In response to these initiatives, the Institute of Medicine (IOM) and the US Department of Health and Human Services (DHHS) have put forth clinical core indicators, including measures for retention in HIV care, which now serve as national benchmarks, with reporting on these indicators required by agencies receiving federal funding for the provision of HIV services [13, 14].
Although hundreds of trials have comparatively evaluated ART regimens over the past 2 decades, there is a paucity of rigorous scientific research that has been conducted on the early steps of the HIV care continuum [15]. In particular, studies on engagement in care including initial linkage, subsequent retention, and reengagement in medical care among those who drop out are limited, but rapidly emerging in the literature. As a nascent field, a number of approaches have been developed to quantify and measure retention in care, with no clear gold standard established [16, 17]. In broad terms, retention measures include both those based solely on attended clinic visits and others that account for missed (no-show) clinic visits. Recent research indicates that these 2 approaches to quantification (attended vs missed) may tap into different aspects of HIV care retention [16]. To date, most studies have utilized single retention measures in isolation, and have not evaluated the added value of using multiple measures concomitantly or sequentially. We evaluated the association of missed clinic visits for all-cause mortality when used in conjunction with the IOM and DHHS clinical indicators of retention in care, both of which are calculated based solely upon attended visits. We hypothesized that beyond the retention in care classification according to these core indicators (retained vs not retained), missed clinic visits would have independent and substantial associations with all-cause mortality.
METHODS
Design Overview
We conducted an analysis of systematically captured data from a multisite HIV clinical cohort collaboration, the Centers for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS).
Setting and Participants
CNICS is a nationally distributed HIV clinical cohort that has been described in detail previously [18]. In brief, the CNICS cohort includes >28 000 HIV-infected adults (contributing >125 000 person-years of follow-up, on average 4.5 years per patient) who have received HIV care at 1 of 8 CFAR sites, dating back to 1995. Every 3 months, sites transmit comprehensive and well-defined data elements captured from point-of-care electronic health record systems using standardized terminology and format. Systematic and rigorous processes for data verification and quality assurance are in place to generate a centralized high-quality clinical database. The participating cohorts and this study were approved by local institutional review boards.
For this study, we included antiretroviral-naive, HIV-infected patients starting ART at 1 of 5 participating CNICS sites contributing comprehensive clinic visit data. All patients starting ART between January 2000 and July 2010 who were alive 24 months following ART initiation were included. Because retention in care was calculated for the 24 months following ART start, in accordance with the 24-month measurement period for the DHHS retention core indicator [14], patients who died prior to this date (n = 105) were excluded as they did not have a complete observation measurement period. No other exclusion criteria were used, and because retention was the primary independent variable under study, participants lost to care within 24 months after ART initiation were not excluded or censored; rather, this information was implicitly captured by the retention measures under study.
Exposures and Outcomes
Retention in HIV medical care during a 24-month measurement period following ART initiation was the principal exposure of interest. Retention was calculated using 3 measures including the IOM core indicator (based on the Health and Resources Services Administration HIV/AIDS Bureau measure), the DHHS core indicator, and a count of missed primary HIV care clinic visits that were not canceled in advance by patient or provider (no-show visits). All retention measures were calculated based on scheduled appointments with the primary HIV care provider only, with subspecialty and urgent care visits excluded. The IOM retention indicator is defined as 2 attended visits separated by ≥90 days during a 12-month measurement period [13]. For study purposes, achieving this indicator for each of 2 consecutive 12-month periods following the ART start date was used to define IOM retention at 24 months. The DHHS core indicator is defined as at least 1 attended visit in each 6-month period during a 24-month measurement period, with ≥60 days between visits in adjacent 6-month periods [14]. Missed visits over the 24 months following ART start were categorized as 0, 1–2, or >2 missed visits. For each patient, a 24-month observation measurement period was determined individually based on the ART start date. Attended visits on the ART initiation date were not counted in the calculation of the IOM and DHHS core indicators, which included scheduled visits subsequent to this date.
All-cause mortality, the principal outcome of interest, was ascertained via query of the Social Security Death Index and/or National Death Index. Because we used these national databases, assessment of vital status as an outcome was not contingent on participants remaining in care and under observation in the clinical cohorts contributing to CNICS.
Statistical Analysis
Descriptive statistics including means, medians, frequencies, and proportions were calculated and visual plots assessed to evaluate the distribution of all study variables. Separate Cox proportional hazards models assessed the relationships between the 3 measures of retention at 24 months following ART start (excluding patients who died within 24 months) and all-cause mortality, with the origin for the time scale being 24 months after ART initiation. Next, separate Cox proportional hazards models assessed the independent association of missed clinic visits with all-cause mortality among patients grouped by retention classification (retained vs not retained) at 24 months according to the IOM and DHHS core indicators. Adjusted models control for age at ART start, race, sex, baseline plasma HIV RNA and CD4 count (date nearest ART start date within a window of −180 to 14 days), and are stratified by site. We did not adjust for time-updated CD4 count and plasma HIV RNA, as these biomarkers are on the causal pathway between our primary exposure (retention in care) and outcome (all-cause mortality). For all models, participants were censored on the date of death or administratively in July 2012. All analyses were conducted using SAS software, version 9.3.
RESULTS
Among 3672 study participants, the mean age was 38 years and the majority were white (53%) and male (80%), with patients starting ART with a baseline mean CD4 count and plasma HIV RNA of 220 cells/µL and 4.9 log10 copies/mL, respectively (Table 1). Participants were followed for a median of 6.0 years (interquartile range, 3.8–8.7 years) from ART initiation. At 24 months following ART initiation, 64% and 59% of patients met the IOM and DHHS retention core indicators, respectively, with an average of 2.1 missed (no-show) visits accrued. Subsequently, 332 patients (9.0%) died during 16 102 person-years of follow-up (20.6 deaths per 1000 person-years). Mortality rates were lower among patients classified as retained by the IOM indicator and DHHS indicator and with zero missed visits (16.0, 15.3, and 11.3 deaths per 1000 person-years, respectively) compared with those classified as not retained or experiencing missed clinic visits. In separate multivariable Cox proportional hazards models, failure to achieve the IOM indicator (hazard ratio [HR] = 2.23; 95% confidence interval [CI], 1.79–2.80; 29.5 deaths per 1000 person-years), failure to achieve the DHHS indicator (HR = 2.36 [95% CI, 1.89–2.96], 29.0 deaths per 1000 person-years), and missed clinic visits at 24 months (1–2 no shows: HR = 1.98 [95% CI, 1.45–2.72], 20.4 deaths per 1000 person-years; >2 no shows: HR = 3.20 [95% CI, 2.33–4.41], 30.9 deaths per 1000 person-years) were all associated with increased subsequent mortality (Table 2). Across all 3 models, older age, black race, and lower baseline CD4 count were consistently associated with increased mortality. Notably, the distribution of all 3 retention measures were fairly consistent in analyses stratified by year of ART initiation, with no clear temporal trends observed. Similarly, the relationship between each retention measure and all-cause mortality remained relatively stable over time during the study period (data not shown).
Table 1.
Characteristics of 3672 Antiretroviral-Naive HIV-1–Infected Patients Initiating Combination Antiretroviral Therapy at 5 CNICS Sites, 2000–2010
| Characteristic | No. (%) or Mean ± SD |
|---|---|
| Age, y | 38.2 ± 10.1 |
| Race | |
| White | 1950 (53) |
| Black | 1377 (38) |
| Other/unknown | 345 (9) |
| Sex | |
| Male | 2952 (80) |
| Female | 720 (20) |
| CNICS site | |
| Case Western Reserve University | 405 (11) |
| University of Alabama at Birmingham | 798 (22) |
| University of California, San Diego | 876 (24) |
| University of North Carolina at Chapel Hill | 723 (20) |
| University of Washington | 870 (24) |
| Baselinea CD4 count, cells/µL | 220 ± 183 |
| <50 | 815 (22) |
| 50–199 | 948 (26) |
| 200–349 | 1092 (30) |
| 350–500 | 487 (13) |
| >500 | 244 (7) |
| Missing/unknown | 86 (2) |
| Baselinea viral load, log10 copies/mL | 4.9 ± 0.7 |
| <10 000 | 453 (12) |
| 10 000–100 000 | 1521 (41) |
| >100 000 | 1577 (43) |
| Missing/unknown | 121 (3) |
| IOM retention core indicatorb at 24 mo | |
| Retained | 2358 (64) |
| Not retained | 1314 (36) |
| DHHS retention core indicatorc at 24 mo | |
| Retained | 2166 (59) |
| Not retained | 1506 (41) |
| Cumulative missed (no-show) visits at 24 mo | 2.1 ± 2.6 |
| 0 | 1175 (32) |
| 1–2 | 1414 (39) |
| >2 | 1083 (29) |
Abbreviations: CNICS, Centers for AIDS Research Network of Integrated Clinical Systems; DHHS, Department of Health and Human Services; HIV-1, human immunodeficiency virus type 1; IOM, Institute of Medicine; SD, standard deviation.
a Baseline defined as value nearest antiretroviral therapy (ART) start date within a window of −180 to 14 days.
b IOM retention core indicator based on the Health and Resources Services Administration HIV/AIDS Bureau measure, defined as 2 attended visits per 12-month period with >90 days between visits. Patients achieving this measure in each of the 12-month periods following ART initiation were classified as retained.
c DHHS retention core indicator defined as at least 1 attended visit in each 6-month period during a 24-month measurement period, with ≥60 days between visits in adjacent 6-month periods.
Table 2.
Separate Multivariable Cox Proportional Hazards Models Evaluating the Associations of Retention in Care Over the 24 Months Following Antiretroviral Therapy (ART) Initiation According to 3 Retention Indicators With Subsequent Mortality Among 3672 HIV-1–Infected Patients Initiating Combination ART at 5 CNICS Sites, 2000–2010
| Characteristic | IOM Model, HR (95% CI) | DHHS Model, HR (95% CI) | Missed Visits Model, HR (95% CI) |
|---|---|---|---|
| IOM retention core indicatora at 24 mo | … | … | |
| Retained | Referent | ||
| Not retained | 2.23 (1.79–2.80) | ||
| DHHS retention core indicatorb at 24 mo | … | … | |
| Retained | Referent | ||
| Not retained | 2.36 (1.89–2.96) | ||
| Cumulative missed (no-show) visits at 24 mo | … | … | |
| 0 | Referent | ||
| 1–2 | 1.98 (1.45–2.72) | ||
| >2 | 3.20 (2.33–4.41) | ||
| Age (per 10 y) | 1.51 (1.36–1.68) | 1.53 (1.37–1.70) | 1.53 (1.37–1.70) |
| Race | |||
| White | Referent | Referent | Referent |
| Black | 1.72 (1.34–2.20) | 1.70 (1.32–2.18) | 1.48 (1.15–1.91) |
| Other/unknown | 0.71 (.42–1.21) | 0.72 (.42–1.22) | 0.68 (.40–1.15) |
| Sex | |||
| Male | Referent | Referent | Referent |
| Female | 0.93 (.71–1.22) | 0.94 (.72–1.23) | 0.90 (.69–1.17) |
| Baselinec CD4 count, cells/µL | |||
| <50 | 2.61 (1.35–5.04) | 2.59 (1.34–5.01) | 2.37 (1.23–4.58) |
| 50–199 | 1.93 (1.00–3.73) | 1.88 (.97–3.63) | 1.80 (.93–3.49) |
| 200–349 | 1.18 (.60–2.31) | 1.16 (.59–2.26) | 1.17 (.60–2.28) |
| 350–500 | 1.00 (.47–2.13) | 0.97 (.46–2.06) | 1.00 (.47–2.12) |
| >500 | Referent | Referent | Referent |
| Missing/unknown | 1.06 (.39–2.90) | 1.02 (.37–2.80) | 1.43 (.53–3.86) |
| Baselinec viral load, log10 copies/mL | |||
| <10 000 | Referent | Referent | Referent |
| 10 000–100 000 | 1.30 (.83–2.02) | 1.34 (.86–2.08) | 1.27 (.81–1.98) |
| >100 000 | 1.32 (.85–2.07) | 1.37 (.88–2.15) | 1.28 (.82–2.01) |
| Missing/unknown | 1.77 (.88–3.56) | 1.85 (.92–3.73) | 1.81 (.91–3.59) |
Multivariable models stratified by study site.
Abbreviations: CI, confidence interval; CNICS, Centers for AIDS Research Network of Integrated Clinical Systems; DHHS, Department of Health and Human Services; HIV-1, human immunodeficiency virus type 1; HR, hazard ratio; IOM, Institute of Medicine.
a IOM retention core indicator based on the Health and Resources Services Administration HIV/AIDS Bureau measure, defined as 2 attended visits per 12-month period with >90 days between visits. Patients achieving this measure in each of the 12-month periods following antiretroviral therapy (ART) initiation were classified as retained.
b DHHS retention core indicator defined as at least 1 attended visit in each 6-month period during a 24-month measurement period, with ≥60 days between visits in adjacent 6-month periods.
c Baseline defined as value nearest ART start date within a window of −180 to 14 days.
Among patients classified as retained at 24 months by the IOM (n = 2358) and DHHS (n = 2166) retention core indicators, missed visits were common, with roughly two-thirds of persons having at least 1 no-show visit, and one-quarter of patients missing >2 visits over this interval (Table 3). Separate multivariable Cox proportional hazards models restricted to patients classified as retained by the IOM and DHHS retention core indicators demonstrated increased mortality risk among patients accruing more missed clinic visits over the 24 months following ART initiation (IOM: 1–2 no shows, HR = 1.78 [95% CI, 1.17–2.70], 15.3 deaths per 1000 person-years; >2 no shows: HR = 3.61 [95% CI, 2.35–5.55], 24.9 deaths per 1000 person-years; and DHHS: 1–2 no shows, HR = 1.71 [95% CI, 1.10–2.65], 14.4 deaths per 1000 person-years; >2 no shows: HR = 3.62 [95% CI, 2.30–5.68], 23.8 deaths per 1000 person-years; Tables 3 and 4, Figure 1).
Table 3.
Frequency of Missed Clinic Visits and Mortality Rates (Deaths per 1000 Person-Years of Follow-Up) Among Patients Classified as Retained and Not Retained at 24 Months Following Antiretroviral Therapy Initiation According to Institute of Medicine and Department of Health and Human Services Core Indicators at 5 CNICS Sites, 2000–2010
| Characteristic | Retained at 24 Months by IOM Core Indicatora (n = 2358) | Not Retained at 24 Months by IOM Core Indicatora (n = 1314) | Retained at 24 Months by DHHS Core Indicatorb (n = 2166) | Not Retained at 24 Months by DHHS Core Indicatorb (n = 1506) |
|---|---|---|---|---|
| Missed (no-show) visits at 24 mo | ||||
| 0 | 861 (37%); 9.9 | 314 (24%); 15.4 | 827 (38%); 9.8 | 348 (23%); 15.0 |
| 1–2 | 848 (36%); 15.3 | 566 (43%); 28.5 | 766 (35%); 14.4 | 648 (43%); 28.2 |
| >2 | 649 (28%); 24.9 | 434 (33%); 40.7 | 573 (26%); 23.8 | 510 (34%); 39.8 |
Data are presented as No. (%); deaths per 1000 person-years of follow-up.
Abbreviations: CNICS, Centers for AIDS Research Network of Integrated Clinical Systems; DHHS, Department of Health and Human Services; IOM, Institute of Medicine.
a IOM retention core indicator based upon the Health and Resources Services Administration HIV/AIDS Bureau measure, defined as 2 attended visits per 12-month period with >90 days between visits. Patients achieving this measure in each of the 12-month periods following antiretroviral therapy initiation were classified as retained.
b DHHS retention core indicator defined as at least 1 attended visit in each 6-month period during a 24-month measurement period, with ≥60 days between visits in adjacent 6-month periods.
Table 4.
Separate Cox Proportional Hazards Models Evaluating the Association of Missed Clinic Visits With Long-Term Mortality Among Patients Classified as Retained at 24 Months Following Antiretroviral Therapy Initiation According to Institute of Medicine and Department of Health and Human Services Core Indicators at 5 CNICS Sites, 2000–2010
| Characteristic | Retained at 24 Months by IOM Core Indicatora (n = 2358), HR (95% CI) | Retained at 24 Months by DHHS Core Indicatorb (n = 2166), HR (95% CI) |
|---|---|---|
| Missed (no-show) visits at 24 mo | ||
| 0 | Referent | Referent |
| 1–2 | 1.78 (1.17–2.70) | 1.71 (1.10–2.65) |
| >2 | 3.61 (2.35–5.55) | 3.62 (2.30–5.68) |
| Age (per 10 y) | 1.66 (1.42–1.94) | 1.63 (1.38–1.92) |
| Race | ||
| White | Referent | Referent |
| Black | 1.14 (.80–1.61) | 1.15 (.80–1.66) |
| Other/unknown | 0.65 (.31–1.36) | 0.65 (.29–1.43) |
| Sex | ||
| Male | Referent | Referent |
| Female | 0.69 (.46–1.03) | 0.75 (.50–1.14) |
| Baselinec CD4 count, cells/µL | ||
| <50 | 1.80 (.76–4.24) | 1.49 (.63–3.53) |
| 50–199 | 1.38 (.58–3.24) | 1.23 (.52–2.91) |
| 200–349 | 0.93 (.38–2.23) | 0.78 (.32–1.91) |
| 350–500 | 0.92 (.34–2.47) | 0.92 (.34–2.48) |
| >500 | Referent | Referent |
| Missing/unknown | 3.31 (.95–11.57) | 2.44 (.59–10.13) |
| Baselinec viral load, log10 copies/mL | ||
| <10 000 | Referent | Referent |
| 10 000–100 000 | 1.00 (.58–1.75) | 0.96 (.53–1.73) |
| >100 000 | 0.78 (.44–1.37) | 0.71 (.39–1.29) |
| Missing/unknown | 1.62 (.69–3.77) | 1.42 (.57–3.52) |
Multivariable models stratified by study site.
Abbreviations: CI, confidence interval; CNICS, Centers for AIDS Research Network of Integrated Clinical Systems; DHHS, Department of Health and Human Services; HR, hazard ratio; IOM, Institute of Medicine.
a IOM retention core indicator based upon the Health and Resources Services Administration HIV/AIDS Bureau measure, defined as 2 attended visits per 12-month period with >90 days between visits. Patients achieving this measure in each of the 12-month periods following antiretroviral therapy (ART) initiation were classified as retained.
b DHHS retention core indicator defined as at least 1 attended visit in each 6-month period during a 24-month measurement period, with ≥60 days between visits in adjacent 6-month periods.
c Baseline defined as value nearest ART start date within a window of −180 to 14 days.
Figure 1.
Kaplan–Meier survival curves for all-cause mortality among patients classified as retained and not retained at 24 months following antiretroviral therapy initiation according to the Institute of Medicine (IOM) (A and B) and Department of Health and Human Services (DHHS) (C and D) core indicators stratified by missed (no-show) clinic visits. A, Retained at 24 months according to IOM core indicator (n = 2358). B, Not retained at 24 months according to IOM core indicator (n = 1314). C, Retained at 24 months according to DHHS core indicator (n = 2166). D, Not retained at 24 months according to DHHS core indicator (n = 1506). Abbreviation: ART, antiretroviral therapy.
Missed visits were more common among patients classified as not retained at 24 months by the IOM (n = 1314) and DHHS (n = 1506) retention core indicators compared with those classified as retained, although roughly a quarter of “not retained” patients had zero no-show visits (Table 3). Separate multivariable Cox proportional hazards models restricted to patients classified as not retained by the core indicators demonstrated increased mortality risk among patients accruing more missed clinic visits over the 24 months following ART initiation (IOM: 1–2 no shows, HR = 1.63 [95% CI, .98–2.72], 28.5 deaths per 1000 person-years; >2 no shows: HR = 2.11 [95% CI, 1.26–3.51], 40.7 deaths per 1000 person-years; and DHHS: 1–2 no shows, HR = 1.76 [95% CI, 1.08–2.85], 28.2 deaths per 1000 person-years; >2 no shows: HR = 2.32 [95% CI, 1.43–3.77] 39.8 deaths per 1000 person-years; Tables 3 and 5, Figure 1). Moreover, whereas increased mortality rates were observed overall among patients classified as not retained by core indicators, and clear dose-response relationships were observed with increasing missed visits within retention categories, interesting relationships were observed when comparing mortality rates across retention categories (Table 3). For example, patients classified as retained by either core indicator who accrued 1–2 missed visits during the 24 months following ART initiation had mortality rates comparable to those classified as not retained and who had zero missed visits (Table 3).
Table 5.
Separate Cox Proportional Hazards Models Evaluating the Association of Missed Clinic Visits With Long-Term Mortality Among Patients Classified as Not Retained at 24 Months Following Antiretroviral Therapy Initiation According to Institute of Medicine and Department of Health and Human Services Core Indicators at 5 CNICS Sites, 2000–2010
| Characteristic | Not Retained at 24 Months by IOM Core Indicatora (n = 1314), HR (95% CI) | Not Retained at 24 Months by DHHS Core Indicatorb (n = 1506), HR (95% CI) |
|---|---|---|
| Missed (no-show) visits at 24 mo | ||
| 0 | Referent | Referent |
| 1–2 | 1.63 (.98–2.72) | 1.76 (1.08–2.85) |
| >2 | 2.11 (1.26–3.51) | 2.32 (1.43–3.77) |
| Age (per 10 y) | 1.56 (1.34–1.83) | 1.60 (1.38–1.86) |
| Race | ||
| White | Referent | Referent |
| Black | 1.89 (1.29–2.76) | 1.80 (1.27–2.57) |
| Other/unknown | 0.72 (.33–1.57) | 0.73 (.35–1.52) |
| Sex | ||
| Male | Referent | Referent |
| Female | 1.16 (.80–1.69) | 1.10 (.77–1.58) |
| Baselinec CD4 count, cells/µL | ||
| <50 | 3.58 (1.27–10.08) | 3.86 (1.38–10.79) |
| 50–199 | 2.65 (.94–7.46) | 2.63 (.94–7.37) |
| 200–349 | 1.53 (.53–4.36) | 1.62 (.57–4.59) |
| 350–500 | 1.13 (.35–3.68) | 1.02 (.31–3.33) |
| >500 | Referent | Referent |
| Missing/unknown | 0.89 (.19–4.24) | 1.29 (.30–5.45) |
| Baselinec viral load, log10 copies/mL | ||
| <10 000 | Referent | Referent |
| 10 000–100 000 | 1.84 (.87–3.91) | 1.76 (.89–3.47) |
| >100 000 | 2.26 (1.06–4.82) | 2.24 (1.13–4.41) |
| Missing/unknown | 2.32 (.76–7.06) | 2.29 (.81–6.44) |
Multivariable models stratified by study site.
Abbreviations: CI, confidence interval; CNICS, Centers for AIDS Research Network of Integrated Clinical Systems; DHHS, Department of Health and Human Services; HR, hazard ratio; IOM, Institute of Medicine.
a IOM retention core indicator based upon the Health and Resources Services Administration HIV/AIDS Bureau measure, defined as 2 attended visits per 12 month period with >90 days between visits. Patients achieving this measure in each of the 12-month periods following antiretroviral therapy (ART) initiation were classified as retained.
b DHHS retention core indicator defined as at least 1 attended visit in each 6-month period during a 24-month measurement period, with ≥60 days between visits in adjacent 6-month periods.
c Baseline defined as value nearest ART start date within a window of −180 to 14 days.
DISCUSSION
These data are among the first to provide empirical validation of the IOM and DHHS core indicators of retention in care with definitive clinical outcomes. When measured over the 24 months following ART initiation, failure to achieve these retention core indicators was strongly associated with subsequent all-cause mortality. However, study findings indicate that assessment of missed (no-show) clinic visits, in conjunction with these core indicators, provides additional, independent prognostic value. Among patients grouped by retention in care classification (retained vs not retained) by the IOM and DHHS retention core indicators, missed visits were exceedingly common and were associated with a substantially elevated mortality risk. Accordingly, caution is warranted in relying solely on retention core indicators for HIV policy, clinical, and programmatic purposes. Although these measures have clear value, considerable additional prognostic information is provided, for both patients classified as retained and not retained, by further evaluating missed clinic visits, a readily available and immediately actionable clinical marker.
The HIV care continuum and retention in care are at the forefront of the domestic HIV policy, public health, and clinical agenda, with enhanced emphasis garnered by the recently released HIV Care Continuum Initiative, which magnifies the focus of the US National HIV/AIDS Strategy on this pivotal area [11, 12]. In response to these initiatives from the federal government, the IOM and DHHS have released clinical core indicators [13, 14], including measures for retention in care, which are being widely implemented with required reporting on these measures for agencies receiving federal funding for the provision of HIV services. Implementation and adoption of these core indicators are important to assess progress toward local and national goals, and to standardize assessment and comparison across settings, but there is a potential shortcoming in using these indicators alone to define HIV care retention. Our findings suggest that the additional inclusion of missed clinic visits in HIV policy, clinical, and public health planning is prudent to optimize classification, risk stratification, and resource allocation to those in greatest need. Agencies with access to missed clinic visits should be encouraged to take advantage of these additional data, as our findings demonstrate their value.
In recent years a number of approaches to measuring retention in care have emerged, each with strengths and limitations, and with no clear gold standard established [16]. Broadly speaking, retention measures include those based solely on attended clinic visits (eg, the IOM and DHHS core indicators) and others that account for missed clinic visits. Prior research has shown that both types of measures predict mortality among patients newly entering HIV care or initiating ART [19–21]. A novel contribution of this study is the concomitant use of a measure from each broad category, rather than using them in isolation, as has typically been the approach to date. This observation supports recent research suggesting that measures based on attended and missed visits may be tapping into different aspects of retention [16], and that there is complementary value in using measures in combination.
Recent guidelines have recommended systematic monitoring of linkage and retention in HIV care for all persons living with HIV infection [15]. It has been noted that a number of data systems are available to monitor HIV care engagement including public health surveillance, administrative claims, and clinic-based utilization databases. As for retention measures, each monitoring system has distinct advantages and limitations, and integration of systems has been shown to enhance correct classification of HIV care engagement [15]. In recent years there has been a dramatic shift in paradigm, with the use of CD4 counts and plasma HIV RNA laboratory tests reported to public health surveillance being used as a proxy for care visits to monitor HIV care engagement and to inform interventions for persons identified as out of care [22]. Notably, laboratory surveillance data can be used to calculate retention measures only, as they are a proxy for attended visits, and missed clinic visits are not reported to public health agencies including the Centers for Disease Control and Prevention. Importantly, surveillance allows for improved classification of retention status of persons who have transferred from one clinic to another, which may not be captured at the clinic level. However, information about missed clinic visits are uniquely available through administrative, billing, or clinical data systems at the clinic level. Accordingly, our findings are germane to HIV clinic directors, providers, and staff. The added value of missed clinic visits for identifying patients at increased mortality risk can help guide allocation of limited resources to those who may derive the greatest benefits. For example, cost- and time-intensive peer mentor, patient navigation, and intensive case management programs are among the few evidence-based approaches to enhancing HIV care engagement [15, 23]. Such programs could be targeted to those with missed clinic visits—even among patients considered retained according to the IOM and DHHS core indicators, as resources allow. Moreover, integration of surveillance and clinic-based data systems to comprehensively capture retention in a given geographical area affords the opportunity to improve classification of HIV care engagement. Such integrated approaches could capitalize upon the strengths and overcome the limitations of each data system and allow for retention in care programs that leverage the unique information provided by measures based on attended and missed clinic visits, captured by public health departments and clinics, respectively.
Our study has limitations. Findings may not generalize to other settings, although we note the geographic diversity of study sites within the United States. As an observational study, we can identify associations but cannot attribute causality. We measure retention over a relatively short observation period of 24 months. Although longer-term retention over decades of treatment is the current paradigm of HIV management, discrete measurement over shorter time periods as evaluated here is highly actionable in terms of risk stratification for programmatic purposes. Additional studies are ongoing within CNICS to evaluate the impact of retention over longer measurement periods on health outcomes. There is potential for misclassification of study variables, but this is believed to be minimal based on the CNICS data quality systems and use of national vital status databases.
In conclusion, our study contributes novel findings germane to the HIV care continuum, with implications for the policy, clinical, and population health communities. The additional assessment of missed clinic visits in conjunction with the IOM and DHHS HIV retention in care core indicators meaningfully enhanced prognostic value for all-cause mortality among patients initiating ART. Accordingly, caution is warranted in relying solely on core indicators to define retention in care and to inform local, state, and national programmatic planning. Missed clinic visits are an important indicator with independent value that can be used along with core indicators to guide allocation of limited resources in an effort to optimize individual- and population-level health outcomes.
Notes
Acknowledgments. We extend our sincere appreciation to the patients, providers, and clinic staff at participating sites, and to the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) investigators and research and informatics staff at each site. We also thank the CNICS Administrative, Data Management, and Epidemiology/Biostatistics cores.
Financial support. This study was supported by the National Institutes of Health (grant numbers R24AI067039, P30AI027767, R01AI100654, and R01AI103661).
Potential conflicts of interest. All authors: No reported conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1.Gardner EM, McLees MP, Steiner JF, Del Rio C, Burman WJ. The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clin Infect Dis. 2011;52:793–800. doi: 10.1093/cid/ciq243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Centers for Disease Control and Prevention. Vital signs: HIV prevention through care and treatment—United States. MMWR Morb Mortal Wkly Rep. 2011;60:1618. [PubMed] [Google Scholar]
- 3.Centers for Disease Control and Prevention. HIV in the United States: the stages of care. Available at: http://www.cdc.gov/nchhstp/newsroom/docs/2012/Stages-of-CareFactSheet-508.pdf. Accessed 3 February 2013.
- 4.Greenberg AE, Hader SL, Masur H, Young AT, Skillicorn J, Dieffenbach CW. Fighting HIV/AIDS in Washington, D.C. Health Aff (Millwood) 2009;28:1677–87. doi: 10.1377/hlthaff.28.6.1677. [DOI] [PubMed] [Google Scholar]
- 5.Cohen MS, Chen YQ, McCauley M, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365:493–505. doi: 10.1056/NEJMoa1105243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Thompson MA, Aberg JA, Hoy JF, et al. Antiretroviral treatment of adult HIV infection: 2012 recommendations of the International Antiviral Society–USA panel. JAMA. 2012;308:387–402. doi: 10.1001/jama.2012.7961. [DOI] [PubMed] [Google Scholar]
- 7.Das M, Chu PL, Santos GM, et al. Decreases in community viral load are accompanied by reductions in new HIV infections in San Francisco. PLoS One. 2010;5:e11068. doi: 10.1371/journal.pone.0011068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hall HI, Gray KM, Tang T, Li J, Shouse L, Mermin J. Retention in care of adults and adolescents living with HIV in 13 U.S. areas. J Acquir Immune Defic Syndr. 2012;60:77–82. doi: 10.1097/QAI.0b013e318249fe90. [DOI] [PubMed] [Google Scholar]
- 9.Mugavero MJ, Amico KR, Horn T, Thompson MA. The state of engagement in HIV care in the United States: from cascade to continuum to control. Clin Infect Dis. 2013;57:1164–71. doi: 10.1093/cid/cit420. [DOI] [PubMed] [Google Scholar]
- 10.Horstmann E, Brown J, Islam F, Buck J, Agins BD. Retaining HIV-infected patients in care: Where are we? Where do we go from here? Clin Infect Dis. 2010;50:752–61. doi: 10.1086/649933. [DOI] [PubMed] [Google Scholar]
- 11.White House Office of National AIDS Policy. Washington, DC: The White House; 2010. National HIV/AIDS Strategy for the United States. Available at: http://www.whitehouse.gov/sites/default/files/uploads/NHAS.pdf . Accessed 7 August 2014. [Google Scholar]
- 12.The White House. Executive Order—HIV Care Continuum Initiative. Available at: http://www.whitehouse.gov/the-press-office/2013/07/15/executive-order-hiv-care-continuum-initiative. Accessed 7 August 2014.
- 13.Institute of Medicine. Monitoring HIV care in the United States: indicators and data systems. Available at: http://www.iom.edu/Reports/2012/Monitoring-HIV-Care-in-the-United-States.aspx. Accessed 7 August 2014. [PubMed]
- 14.US Department of Health and Human Services . HIV core indicators. Available at: http://aids.gov/pdf/hhs-common-hiv-indicators.pdf. Accessed 7 August 2014. [DOI] [PubMed]
- 15.Thompson MA, Mugavero MJ, Amico KR, et al. Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an International Association of Physicians in AIDS Care Panel. Ann Intern Med. 2012;156:817–33. doi: 10.7326/0003-4819-156-11-201206050-00419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Mugavero MJ, Westfall AO, Zinski A, et al. Measuring retention in HIV care: the elusive gold standard. J Acquir Immune Defic Syndr. 2012;61:574–80. doi: 10.1097/QAI.0b013e318273762f. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Yehia BR, Fleishman JA, Metlay JP, et al. Comparing different measures of retention in outpatient HIV care. AIDS. 2012;26:1131–9. doi: 10.1097/QAD.0b013e3283528afa. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kitahata MM, Rodriguez B, Haubrich R, et al. Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. Int J Epidemiol. 2008;37:948–55. doi: 10.1093/ije/dym231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Giordano TP, Gifford AL, White AC, Jr, et al. Retention in care: a challenge to survival with HIV infection. Clin Infect Dis. 2007;44:1493–9. doi: 10.1086/516778. [DOI] [PubMed] [Google Scholar]
- 20.Horberg MA, Hurley LB, Silverberg MJ, Klein DB, Quesenberry CP, Mugavero MJ. Missed office visits and risk of mortality among HIV-infected subjects in a large healthcare system in the United States. AIDS Patient Care STDS. 2013;27:442–9. doi: 10.1089/apc.2013.0073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Mugavero MJ, Lin HY, Willig JH, et al. Missed visits and mortality among patients establishing initial outpatient HIV treatment. Clin Infect Dis. 2009;48:248–56. doi: 10.1086/595705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sweeney P, Gardner LI, Buchacz K, et al. Shifting the paradigm: using HIV surveillance data as a foundation for improving HIV care and preventing HIV infection. Milbank Q. 2013;91:558–603. doi: 10.1111/milq.12018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Higa DH, Marks G, Crepaz N, Liau A, Lyles CM. Interventions to improve retention in HIV primary care: a systematic review of U.S. studies. Curr HIV/AIDS Rep. 2012;9:313–25. doi: 10.1007/s11904-012-0136-6. [DOI] [PMC free article] [PubMed] [Google Scholar]