Data from 6 human immunodeficiency virus clinics showed improvements in clinic attendance after versus before a clinic-wide intervention (7% on short-term measure; 3% on longer-term measure). Several subgroups showed larger improvements: younger patients, new or reengaging patients, and patients with elevated viral loads.
Abstract
Background. Retention in care for human immunodeficiency virus (HIV)–infected patients is a National HIV/AIDS Strategy priority. We hypothesized that retention could be improved with coordinated messages to encourage patients' clinic attendance. We report here the results of the first phase of the Centers for Disease Control and Prevention/Health Resources and Services Administration Retention in Care project.
Methods. Six HIV-specialty clinics participated in a cross-sectionally sampled pretest-posttest evaluation of brochures, posters, and messages that conveyed the importance of regular clinic attendance. 10 018 patients in 2008–2009 (preintervention period) and 11 039 patients in 2009–2010 (intervention period) were followed up for clinic attendance. Outcome variables were the percentage of patients who kept 2 consecutive primary care visits and the mean proportion of all primary care visits kept. Stratification variables were: new, reengaging, and active patients, HIV RNA viral load, CD4 cell count, age, sex, race or ethnicity, risk group, number of scheduled visits, and clinic site. Data were analyzed by multivariable log-binomial and linear models using generalized estimation equation methods.
Results. Clinic attendance for primary care was significantly higher in the intervention versus preintervention year. Overall relative improvement was 7.0% for keeping 2 consecutive visits and 3.0% for the mean proportion of all visits kept (P < .0001). Larger relative improvement for both outcomes was observed for new or reengaging patients, young patients and patients with elevated viral loads. Improved attendance among the new or reengaging patients was consistent across the 6 clinics, and less consistent across clinics for active patients.
Conclusion. Targeted messages on staying in care, which were delivered at minimal effort and cost, improved clinic attendance, especially for new or reengaging patients, young patients, and those with elevated viral loads.
After patients with a diagnosis of human immunodeficiency virus (HIV) infection enter treatment, they must remain in care to realize the full benefits of effective antiretroviral therapy [1, 2]. Improving retention of HIV-infected patients in care is a priority from both clinical and public health perspectives [3–16] and is a major objective of the National HIV/AIDS Strategy [17]. Yet, few studies have evaluated better methods of conveying to patients the importance of attending all HIV care appointments. One major research gap has been a lack of studies examining interventions that encourage attendance at all HIV primary care visits [18]. An evaluation of a coordinated multisite clinic-wide strategy, with the primary goal of improving retention in primary care using messages specifically tailored to that purpose, has not been published.
To address this gap, the Centers for Disease Control and Prevention (CDC) and the Health Resources and Services Administration (HRSA) jointly funded a 2-phase, 5-year project to implement interventions to retain HIV-infected patients in care [19, 20]. Phase I included a pretest and posttest evaluation of a clinic-wide (Stay Connected) intervention that used posters, brochures, and brief messages delivered to patients by their primary care providers. A second phase consisted of a randomized, controlled trial of individual level approaches to retain patients in care. Here we report the findings from phase I.
METHODS
Intervention Description
Six HIV clinics participated in the intervention. Two types of messages were developed: (1) print reminder materials, including brochures and examination and waiting room posters; and (2) brief verbal messages to be used by all clinic staff. The brochures, distributed to patients attending the clinics, contained brief information about the importance of staying in care, a message encouraging retention, and contact information for the clinic. Posters were placed in all examination rooms and most waiting rooms of the clinics. Posters in the examination rooms communicated research findings [1] showing better patient clinical status with regular care. The messages delivered by the brochures and providers were reviewed by community advisory boards representing patient populations from 3 sites. Example messages include the following:
We have good evidence that people with HIV who come to their appointments do better than those who don't. When you miss your appointments, we can't work together to keep you healthy.
Thank you for doing such a good job of keeping your appointments. It makes it easier for all of us to work together to keep you healthy.
Clinic staff at each site received a standardized training on strategies for reminding patients about the importance of attending all clinic visits.
Intervention Analysis Design
We implemented a clinic-wide intervention that involved structural changes in the clinics (giving brochures to patients, posters in the waiting room and examination rooms, and providers giving messages to all patients). There was no concurrent control group. Thus, we evaluated whether the structural changes had an impact on patients' attendance for primary care by comparing rates of clinic attendance after exposure to the structural changes with rates of clinic attendance before those changes were initiated. Patients were included in the 2 cross-sectional samples according to predetermined criteria (see Figure 1). The intervention 12-month period ran from 1 May 2009, to 30 April 2010, with slight adjustments based on when each clinic began implementing the intervention. The preintervention year ran from 1 May 2008 to 30 April 2009. Because exposure to the clinic environment was necessary before a subsequent response (attendance) could be attributed to the intervention, we assessed attendance after an initial clinic visit during the first 6 months of each year, thus standardizing the method across the 2 12-month periods. Exposure was defined as the first attended primary care clinic visit within the first 6 months of the preintervention or intervention year (described hereafter as the “anchor visit”). If a patient had only an anchor visit, with no subsequent scheduled appointments, he or she was exposed to the intervention but contributed no outcome data and was excluded (see Figure 1). Each patient with an anchor visit was followed up for both missed and attended primary care visits for the remainder of that year. Each scheduled appointment had 3 possible visit outcomes: arrived (“kept”), canceled, or no show. Arrived and no-show outcomes were retained; canceled visits were omitted.
Figure 1.
Flow chart for inclusion in data analysis of the pretest-posttest Retention in Care Study. Of 21 057 patients included from both years, 15 870 (75.4%) were in both years, 3104 (14.7%) were in the intervention year only, and 2083 (9.9%) were in the preintervention year only. Abbreviation: HIV, human immunodeficiency virus.
Historical Data Used to Distinguish Active From New or Reengaging Patients
To distinguish among new, reengaging, and active patients, historical information on patients' clinic attendance (ie, predating the 2 study years) was collected as outlined in Figure 2. To standardize the definition of past clinic attendance, we placed a date limit on the historical information. Each clinic supplied records of scheduled HIV primary care appointments dating back to 1 January 2004. Active patients were those with ≥1 kept primary care visit in the 12 months immediately preceding the preintervention year or ≥1 kept visit in the 12 months immediately preceding the intervention year. New patients were defined as those without an arrived visit at the clinic from 1 January 2004, until their first arrived visit during the initial 6 months of one of the observational years. New patients are therefore relatively new to the clinic but not necessarily newly diagnosed with HIV infection or new to care (eg, some patients are transferring from another facility). Reengaging patients were patients who had not been seen in the 12-month period immediately preceding the year in which their records were included in the analysis but who had been seen in clinic at least once since 1 January 2004.
Figure 2.
Flow chart for historical data defining new, reengaging and active patients. Abbreviation: HIV, human immunodeficiency virus.
Dependent Variables
Dependent variables were chosen to assess short-term effects (“next 2 visits”) and longer-term effects (proportion of all scheduled visits kept). The “next 2 visits” outcome was binary: keeping 2 consecutive visits was defined as success, and keeping 1 or 0 visits was defined as failure. Patients with only a single scheduled postanchor visit were excluded, but were analyzed separately in a sensitivity analysis reported below. The proportion of all scheduled primary care visits that were kept was a continuous measure ranging from 0 to 1.
Stratification Variables
Because of small numbers of reengaging patients, the analyses in Tables 2–5 combine new and reengaging patients. CD4 cell count and HIV RNA (viral load) levels were matched to the date of the anchor clinic visits by ±99 days. The undetectable level was set at ≤400 copies/mL, the level of suppression that all sites could report. Variables in Table 1 were assessed as of the baseline (anchor) visit for each year. For race/ethnicity status, Hispanics were classified as “Hispanic” regardless of race. The number of scheduled visits referred to scheduled visits after the anchor visit.
Table 2.
Adjusted Percentage of Patients Keeping Next 2 Primary Care Visits During Preintervention and Intervention Periods, Retention in Care Study, 2008–2010
| Patients Keeping Next 2 Visits, % (No.) |
||||
|---|---|---|---|---|
| Variable | Preintervention Year (2008–2009) | Intervention Year (2009–2010) | % Relative Improvementa | P |
| Overall (no adjustment) | 52.7 (8535) | 58.2 (9227) | 10.4 | <.0001 |
| Overall (adjusted) | 49.3 (8535) | 52.7 (9227) | 7.0 | <.0001 |
| Patient type | ||||
| New or reengaging | 45.2 (1147) | 57.9 (1210) | 28.2 | <.0001 |
| Active | 48.1 (7388) | 50.6 (8017) | 5.3 | <.0001 |
| Viral loadb | ||||
| Undetectablec | 57.2 (5537) | 60.4 (6287) | 5.6 | <.0001 |
| Detectable | 44.4 (2998) | 51.5 (2940) | 16.0 | <.0001 |
| CD4 cell count,b cells/mm3 | ||||
| <350 | 49.8 (3443) | 55.0 (3616) | 10.3 | <.0001 |
| ≥350 | 53.3 (5012) | 56.4 (5376) | 5.7 | <.0001 |
| Scheduled visits for care, No. | ||||
| 2–3d | 52.6 (3270) | 55.1 (4098) | 4.7 | .003 |
| 4–6 | 49.4 (3589) | 53.4 (3600) | 8.2 | <.0001 |
| ≥7 | 45.4 (1676) | 50.8 (1529) | 12.0 | .001 |
| Sex | ||||
| Male | 52.6 (5491) | 56.3 (5909) | 7.0 | <.0001 |
| Female | 50.1 (3012) | 54.7 (3304) | 9.1 | <.0001 |
| Age group, years | ||||
| 16–29 | 40.5 (470) | 48.6 (577) | 19.9 | .006 |
| 30–39 | 45.9 (1485) | 51.4 (1534) | 11.9 | .0003 |
| 40–49 | 50.9 (3201) | 54.6 (3299) | 7.2 | .0006 |
| 50–85 | 57.0 (3379) | 60.5 (3817) | 6.1 | .0002 |
| Race/ethnicity | ||||
| Black | 50.1 (5542) | 54.6 (6061) | 9.0 | <.0001 |
| White | 53.3 (1307) | 56.5 (1327) | 6.0 | .018 |
| Other race | 48.8 (108) | 61.2 (121) | 25.2 | .022 |
| Hispanic | 50.7 (1578) | 53.5 (1718) | 5.5 | .040 |
| HIV risk group | ||||
| MSM | 55.6 (2273) | 57.0 (2432) | 2.5 | .224 |
| MSM plus IDUs | 54.1 (198) | 55.3 (193) | 2.2 | .780 |
| Othere | 48.2 (659) | 55.7 (755) | 15.5 | .004 |
| Heterosexuals | 51.7 (4228) | 56.6 (4636) | 9.5 | <.0001 |
| IDUs | 42.9 (1177) | 48.0 (1211) | 11.8 | .004 |
| Insurance | ||||
| Private | 54.5 (1315) | 57.9 (1354) | 6.3 | .010 |
| Medicare | 51.3 (1916) | 54.2 (1968) | 5.5 | .013 |
| Medicaid | 43.3 (2891) | 47.8 (3091) | 10.6 | <.0001 |
| Other/RW/nonef | 46.4 (2413) | 50.1 (2814) | 8.0 | .002 |
Model adjusted for age, viral load, number of scheduled appointments, insurance, and clinic site. Missing data on age, viral load, and insurance are excluded from the table. Patients with a single postanchor scheduled appointment were also excluded.
Abbreviations: HIV, human immunodeficiency virus; IDUs, injection drug users; MSM, men who have sex with men; RW, Ryan White coverage.
a Change in retention measure from preintervention period to intervention period, expressed as a percentage of the measure for the preintervention period.
b Based on clinical records no more than ±99 days from anchor visit in preintervention and intervention periods.
c Undetectable viral loads were defined as HIV RNA levels ≤400 copies/mL.
d Patients with only a single postanchor scheduled visit were not included in this model.
e “Other” category includes other risk factors, unknown or undetermined risk factors, and no risk factors identified.
f For insurance, “other” includes university and local charity programs.
Table 3.
Adjusted Mean Proportion of All Primary Care Visits Kept Among Patients During Preintervention and Intervention Periods, Retention in Care Study, 2008–2010
| Visits Kept, Mean Proportion (No.) |
||||
|---|---|---|---|---|
| Variable | Preintervention Year (2008–2009) | Intervention Year (2009–2010) | Relative Improvement, %a | P |
| Overall (no adjustment) | 0.700 (9407) | 0.724 (10 344) | 3.4 | <.0001 |
| Overall (adjusted) | 0.679 (9407) | 0.699 (10 344) | 3.0 | <.0001 |
| Patient type | ||||
| New or reengaging | 0.649 (1310) | 0.699 (1371) | 7.6 | <.0001 |
| Active | 0.678 (8097) | 0.694 (8973) | 2.4 | <.0001 |
| Viral loadb | ||||
| Undetectablec | 0.723 (6142) | 0.738 (7131) | 2.0 | .0004 |
| Detectable | 0.622 (3265) | 0.656 (3213) | 5.5 | <.0001 |
| CD4 cell count,b cells/mm3 | ||||
| <350 | 0.663 (3719) | 0.697 (3922) | 5.1 | <.0001 |
| ≥350 | 0.688 (5558) | 0.702 (6115) | 1.9 | <.0020 |
| Scheduled visits for care, No. | ||||
| 1–3 | 0.647 (4142) | 0.676 (5215) | 4.5 | <.0001 |
| 4–6 | 0.705 (3589) | 0.720 (3600) | 2.1 | .003 |
| ≥7 | 0.668 (1676) | 0.678 (1529) | 1.5 | .131 |
| Sex | ||||
| Male | 0.677 (6124) | 0.697 (6708) | 3.0 | <.0001 |
| Female | 0.680 (3249) | 0.702 (3598) | 3.3 | .0001 |
| Age group, years | ||||
| 16–29 | 0.604 (526) | 0.662 (638) | 9.6 | .0002 |
| 30–39 | 0.666 (1667) | 0.684 (1749) | 2.7 | .060 |
| 40–49 | 0.688 (3554) | 0.708 (3739) | 2.8 | .0010 |
| 50–85 | 0.742 (3660) | 0.761 (4218) | 2.5 | .0003 |
| Race/ethnicity | ||||
| Black | 0.668 (5985) | 0.689 (6641) | 3.3 | <.0001 |
| White | 0.693 (1593) | 0.712 (1697) | 2.7 | .022 |
| Other race | 0.715 (123) | 0.757 (142) | 5.9 | .184 |
| Hispanic | 0.686 (1706) | 0.705 (1864) | 2.7 | .033 |
| HIV risk group | ||||
| MSM | 0.698 (2629) | 0.712 (2888) | 2.1 | .03 |
| MSM plus IDUs | 0.640 (225) | 0.645 (226) | 0.9 | .790 |
| Otherd | 0.638 (710) | 0.690 (819) | 8.1 | <.0001 |
| Heterosexuals | 0.689 (4597) | 0.706 (5120) | 2.4 | .0010 |
| IDUs | 0.615 (1246) | 0.645 (1291) | 4.9 | .0020 |
| Insurance | ||||
| Private | 0.709 (1589) | 0.722 (1709) | 1.8 | .11 |
| Medicare | 0.682 (2087) | 0.702 (2186) | 3.0 | .004 |
| Medicaid | 0.638 (3047) | 0.656 (3275) | 2.9 | .002 |
| Other/RW/nonee | 0.656 (2684) | 0.683 (3174) | 4.2 | .0002 |
Model adjusted for age, viral load, number of scheduled appointments, insurance, and clinic site. Missing data on age, viral load, and insurance are excluded from the table.
Abbreviations: HIV, human immunodeficiency virus; IDUs, injection drug users; MSM, men who have sex with men; RW, Ryan White coverage.
a Change in retention measure from preintervention period to intervention period, expressed as a percentage of the measure for the preintervention period.
b Based on clinical records no more than ±99 days from anchor visit in preintervention and intervention periods.
c Undetectable viral loads were defined as HIV RNA levels ≤400 copies/mL.
d “Other” category includes other factors, unknown and undetermined factors, no risk factors identified, and missing data.
e For insurance, “other” includes university or local charity programs.
Table 4.
Adjusted Relative Improvement in Percentage of Patients Keeping Next 2 Visits After Anchor Visit, by Patient Status and Clinic Site
| Clinic | Patients Keeping Next 2 Visits After Anchor Visit, % (No.) |
Relative Improvement, %a | GEE-Based P | |
|---|---|---|---|---|
| Preintervention Year (2008–2009) | Intervention Year (2009–2010) | |||
| New or reengaging patients | ||||
| A | 65.1 (77) | 85.1 (87) | 30.7 | .005 |
| B | 41.6 (106) | 52.1 (78) | 25.4 | .177 |
| C | 37.1 (241) | 48.7 (219) | 31.4 | .006 |
| D | 41.0 (222) | 53.8 (202) | 31.2 | .013 |
| E | 57.5 (137) | 60.6 (177) | 5.3 | .530 |
| F | 37.1 (364) | 48.3 (447) | 30.3 | .0005 |
| Active patients | ||||
| A | 64.3 (822) | 71.1 (892) | 10.6 | .0003 |
| B | 45.4 (807) | 48.3 (876) | 6.3 | .131 |
| C | 40.5 (1448) | 41.4 (1598) | 2.2 | .486 |
| D | 48.5 (1284) | 45.1 (1478) | −7.0 | .015 |
| E | 57.2 (788) | 54.8 (797) | −4.0 | .170 |
| F | 41.2 (2239) | 49.1 (2376) | 19.4 | <.0001 |
Model adjusted for age, viral load, number of scheduled appointments and insurance. Missing data on age, viral load, and insurance are excluded from the table.
Abbreviation: GEE, generalized estimating equation.
a Change in retention measure from preintervention period to intervention period, expressed as a percentage of the measure for the preintervention period.
Table 5.
Adjusted Relative Improvement in Mean Proportion of Scheduled Visits Kept After Anchor Visit, by Patient Status and Clinic Site
| Clinic | Visits Kept After Anchor Visit, Mean Proportion (No.) |
Relative Improvement, %a | GEE- based P | |
|---|---|---|---|---|
| Preintervention Year (2008–2009) | Intervention Year (2009–2010) | |||
| New or reengaging patients | ||||
| A | 0.798 (84) | 0.890 (95) | 11.5 | .008 |
| B | 0.644 (112) | 0.621 (80) | −3.6 | .603 |
| C | 0.595 (275) | 0.654 (255) | 9.9 | .026 |
| D | 0.610 (266) | 0.597 (232) | −2.2 | .640 |
| E | 0.714 (166) | 0.760 (208) | 6.4 | .214 |
| F | 0.579 (407) | 0.658 (501) | 13.7 | .0002 |
| Active patients | ||||
| A | 0.773 (867) | 0.785 (931) | 1.7 | .173 |
| B | 0.642 (823) | 0.650 (895) | 1.2 | .391 |
| C | 0.614 (1558) | 0.623 (1732) | 1.5 | .255 |
| D | 0.652 (1404) | 0.640 (1574) | −1.7 | .250 |
| E | 0.738 (1066) | 0.748 (1220) | 1.2 | .355 |
| F | 0.637 (2379) | 0.681 (2621) | 6.9 | <.0001 |
Model adjusted for age, viral load, number of scheduled appointments, and insurance. Missing data on age, viral load, and insurance are excluded from the table.
Abbreviation: GEE, generalized estimating equation.
a Change in retention measure from preintervention period to intervention period, expressed as a percentage of the measure for the preintervention period.
Table 1.
Baseline Characteristics of Patient Population by Year of Study, Retention in Care Study, 2008–2010
| Patients, No. (%) |
|||
|---|---|---|---|
| Variable | Preintervention Year (2008–2009) | Intervention Year (2009–2010) | P From Heterogeneity (df) |
| Patient type | .28 (2) | ||
| New | 1129 (11.27) | 1184 (10.73) | |
| Reengaging | 320 (3.19) | 329 (2.98) | |
| Active | 8569 (85.54) | 9526 (86.29) | |
| Viral loada | <.0001 (1) | ||
| Undetectableb | 6160 (65.11) | 7149 (68.93) | |
| Detectable | 3301 (34.89) | 3222 (31.07) | |
| CD4 cell count,a cells/mm3 | .1058 (1) | ||
| <350 | 3819 (40.27) | 3978 (39.14) | |
| ≥350 | 5665 (59.73) | 6186 (60.86) | |
| Scheduled visits for care, No. | <.0001 (2) | ||
| 1–3 | 4518 (45.10) | 5622 (50.93) | |
| 4–6 | 3761 (37.54) | 3818 (34.59) | |
| ≥7 | 1739 (17.36) | 1599 (14.49) | |
| Sex | .77 (1) | ||
| Male | 6520 (65.34) | 7166 (65.15) | |
| Female | 3459 (34.66) | 3834 (34.85) | |
| Age group, years | .0035 (3) | ||
| 16–29 | 561 (5.6) | 683 (6.19) | |
| 30–39 | 1768 (17.65) | 1861 (16.86) | |
| 40–49 | 3790 (37.84) | 3992 (36.16) | |
| 50–85 | 3898 (38.91) | 4503 (40.79) | |
| Race/ethnicity | .608 (3) | ||
| Black | 6370 (63.59) | 7095 (64.27) | |
| White | 1676 (16.73) | 1776 (16.09) | |
| Other race | 132 (1.32) | 151 (1.37) | |
| Hispanic | 1840 (18.37) | 2017 (18.27) | |
| HIV risk group | .259 (4) | ||
| MSM | 2771 (27.66) | 3042 (27.56) | |
| MSM plus IDUs | 243 (2.43) | 244 (2.21) | |
| Otherc | 771 (7.70) | 891 (8.07) | |
| Heterosexuals | 4893 (48.84) | 5475 (49.60) | |
| IDUs | 1340 (13.38) | 1387 (12.56) | |
| Clinic site | .593 (5) | ||
| A | 1034 (10.32) | 1102 (9.98) | |
| B | 960 (9.58) | 1017 (9.21) | |
| C | 1938 (19.35) | 2130 (19.20) | |
| D | 1949 (19.45) | 2108 (19.10) | |
| E | 1253 (12.51) | 1448 (13.12) | |
| F | 2884 (28.79) | 3234 (29.30) | |
| Insurance | .0086 (3) | ||
| Private | 1696 (17.05) | 1862 (16.92) | |
| Medicare | 2178 (21.89) | 2269 (20.62) | |
| Medicaid | 3206 (32.23) | 3479 (31.62) | |
| Other/RW/noned | 2868 (28.83) | 3393 (30.84) | |
Abbreviations: HIV, human immunodeficiency virus; IDUs, injection drug users; MSM, men who have sex with men; RW, Ryan White coverage
a Based on clinical records no more than ±99 days from anchor visit in preintervention and intervention periods.
b Undetectable viral loads were defined as HIV RNA levels ≤400 copies/mL.
c “Other” category includes other risk factors, unknown or undetermined risk factors, no risk factors identified, and missing data.
d For insurance, “other” includes university and local charity programs.
Implementation Assessment
Study coordinators at each clinic conducted a survey of 50 patients during 1 week every 3 months beginning January 2010. Selected patients were asked these questions after their medical examinations about receipt of intervention components: “At today's visit or any previous clinic visit, did you receive the Stay Connected brochure about the importance of keeping all of your clinic appointments?” and “At your clinic visit today, did your healthcare provider (doctor, nurse practitioner, physician assistant) talk to you about the importance of keeping all of your appointments at this clinic?”
Statistical Methods
A patient could contribute outcome data in either or both observational periods (see Figure 1). Generalized estimating equations methods with an unstructured correlation matrix were used to control for repeated measures per patient. Overall and subgroup-specific mean proportions attended for each year were provided by the models' least-squares means, providing adjustment for confounders and clustering effects due to clinical site. The continuous dependent variable was modeled using a linear regression model; the binary (next 2 visits) dependent variable was analyzed with a log-binomial model. The measure of effect of the intervention was the percentage of relative improvement—the difference between the adjusted mean for the intervention year and the mean for the preintervention year, divided by the preintervention year mean and expressed as a percentage. These measures of effect were calculated for the overall samples and for the subgroups, including clinic site, for both dependent variables. SAS software, version 9.2 (Proc GENMOD), was used for all model estimates. The study was approved by the institutional review board at each participating site and at the CDC. As a low-risk, clinic-wide intervention, a waiver of the requirement to obtain individual informed consent was granted by each review board.
RESULTS
Table 1 presents data describing the patient population for both years of the study. In the intervention year, the following significant changes relative to the preintervention year were noted: fewer patients had ≥7 scheduled visits and more patients had 1, 2, or 3 scheduled visits (P < .0001); more patients had an undetectable viral load (P < .0001); regarding health insurance, more patients had other, none, or Ryan White coverage, and fewer had Medicaid, Medicare, and private insurance (P = .009); and fewer patients were in the 30–39- and 40–49-year age groups (P = .004). These 4 variables plus clinic site were adjusted for in all multivariable models.
During the first quarterly patient survey period (1 January 2010 through 30 April 2010 of the intervention year), 87.6% of surveyed patients (255 of 291) reported that a physician, nurse practitioner, or physician's assistant talked to them about the importance of keeping all of their appointments. Moreover, 57.6% of patients (170 of 295) reported receiving the Stay Connected brochure either at that day's visit or at a previous clinic visit.
Covariate Adjusted Results
Table 2 presents the relative improvement in keeping the next 2 visits for the overall population and by patient characteristics. The overall adjusted percentage of improvement for keeping 2 consecutive visits was 7.0% (P < .0001. Several subgroups had a particularly large relative improvement: new or reengaging patients (28.2%; P < .0001) versus active patients (5.3%; P < .0001), as well as patients with a detectable viral load (16.0%; P < .0001) versus those with an undetectable load (5.6%; P < .0001). Subgroup differences were also found based on sex (greater improvement in female patients), age (patients aged 16–29 years had significantly more improvement than other age groups), race (patients of “other” race had the greatest improvement), and risk group (greater improvement in heterosexuals and injection drug users than in men who have sex with men). Table 3 presents the relative improvement in the mean proportion of all visits kept, overall and by patient characteristics. The overall adjusted relative improvement was 3.0% (P < .0001. For this outcome, groups with particularly high rates of improvement included new or reengaging patients, patients with detectable viral loads, and patients aged 16–29 years.
There was one major difference in relative improvement between the 2 outcome measures. For the percentage of patients who attended the next 2 visits (Table 2), the improvement was highly significant regardless of the number of scheduled visits. In contrast, for the proportion of all visits attended (Table 3), as the number of scheduled visits increased, the percentage of improvement during the intervention year decreased, and the improvement was nonsignificant at ≥7 scheduled visits.Tables 4 and 5 present the percentage of relative improvement results by site, stratified by new or reengaging versus active patients. For the “next 2 visits” measure, there was consistency in the percentage of improvement across the sites for new or reengaging patients and a lack of consistency for active patients. For the “mean proportion of all visits” measure (Table 5), there was somewhat less consistency across sites for new or reengaging patients (3 of 6 sites with significant improvement and 1 site with nearly significant improvement), and no evidence of consistency for active patients.
Patients with only a single scheduled visit after their anchor visit (n = 2211) could not be included in the “next 2 visits” analyses. We examined these 2211 patients compared with the 18 846 nonexcluded patients and found that they were significantly more likely to have an undetectable viral load, significantly more likely to be 30–49 years old and less likely to be 50–85 years old, significantly more likely to be privately insured, significantly more likely to be men who have sex with men, white, and significantly more likely to have a CD4 cell count ≥350 cells/mm3. For these patients, their adjusted percentage of improvement for their “next 1 visit” was 7.3% (P = .003), similar to the 7.0% level of relative improvement observed overall in Table 2.
DISCUSSION
We conducted a clinic-wide intervention to improve retention in care in 6 HIV clinics in the United States. The intervention produced improvement in attendance for all visits after the anchor date, as well as the next 2 visits after the anchor date. The relative improvement was larger (7.0%) for the next 2 visits after an anchor visit than the percentage of improvement in the proportion of all scheduled visits kept (3.0%). This implies an effect that is easier to detect in visits occurring soon after the intervention was initiated. In subgroups, there are many effects worth noting, most importantly that new /reengaging patients showed a result (28% for next 2 visits and 8% for mean proportion visits kept) that was 4–5 times the effect observed in active patients (6% for next 2 visits and 2% for proportion of visits kept).
One explanation for the pronounced difference between new or reengaging and active patients is that providers may have given the Stay Connected messages more consistently to new or reengaging patients because these patients were new or less familiar to the clinic staff. Providers may have felt less inclined to give messages to patients with a long history of visits to the clinic. Even if the new or reengaging patients did not get the intervention more frequently, they may have been more receptive to the intervention than experienced patients. In fact, receptiveness to the intervention may have been a common characteristic of subgroups of patients (those with high viral load, young patients, and new or reengaging patients) who showed greater percentage improvement in appointment keeping. The Stay Connected messages may have seemed more compelling to new or reengaging and younger patients than to experienced and older patients because the messages were new. Also, clinicians may have emphasized Stay Connected messages more strongly for patients who were not virally suppressed.
The 2 outcome measures had very different trends across the number of scheduled appointments. For patients with 2 or 3 scheduled visits, the 2 outcomes were similar. However, in patients with >3 scheduled visits, there was a slight increase in the percentage of relative improvement for keeping the next 2 visits, but a waning of the intervention effect for the mean proportion of all visits kept. It is important to note that the “next 2 visits” measure assesses an effect based entirely on 2 scheduled visits immediately after the anchor visit, regardless of the number of scheduled visits the patient had for the year, whereas the mean proportion of all visits measure includes all of a patient's scheduled visits in the denominator of that measure. This finding suggests that the intervention measurably affected behavior (regardless of the total number of scheduled appointments) for at least 2 visits after the anchor date, and that its effect faded over time, as measured by the mean proportion of kept appointments among patients who had >3 scheduled primary care appointments.
Another critically important factor was viral load at the beginning of each year. We controlled for viral load as a main effect in all analyses (Tables 2–5) because the proportion of patients with an undetectable viral load was significantly higher in 2009–2010 than in 2008–2009 and because viral load was significantly associated with both measures of attendance. This main effects model, however, also tended to obscure the strong modifier effect of viral load. Compared with those with undetectable viral loads, patients with detectable viral loads were found to have an approximately 3-fold higher percentage improvement in retention, overall and for all subgroups by number of scheduled visits. The consistent effect on attendance for patients with higher viral loads at the start of follow-up suggests that the impact of the messages was felt by the group for whom the messages were intended.
Multisite interventions that provide systematic messages to patients about remaining in care as a design element are uncommon, so there was little to compare to our intervention. However, there has been evidence of success for HIV interventions employing clinician messages to address sex and drug-using behaviors [21–23]. Intervention designs often employ more elements than clinician messages to effect change. Meta-analyses of HIV interventions on risk behaviors found that interventions that promoted personal and communication skills had a larger impact on subsequent risk behavior [24, 25]. A review of studies implementing the chronic disease care model for primary care found that interventions including clear divisions of labor, electronic information systems, patient reminder systems, and individualized patient goal setting were more effective than the standard of care [26]. Even greater improvement of retention in primary care than we observed with our simple clinic-wide messages strategy is very likely for an intervention design that includes individualized attention to help with personal skills and more personalized appointment reminders. These elements are included in the ongoing phase 2 component of the CDC/HRSA Retention in Care Study.
The pretest-posttest study design is not as strong as a trial with control groups; however, the consistency of our outcomes, when stratified by the array of additional variables, increases our confidence that our intervention had an impact on visit attendance, especially for new or reengaging patients. The January 2010 patient survey start prevented us from observing trends in the receipt of messages. We did not systematically gather data on clinic practice changes. Changes of this sort are difficult to measure and control even in group-randomized trials. Our data emphasize the consistency across clinics: for every clinic (Table 4), the percentage of relative improvement in keeping the next 2 appointments was less for active than for new or reengaging patients, and for every clinic except one (Table 5), the percentage of relative improvement in the mean proportion of scheduled visits kept was again less for active than for new or reengaging patients. One significant contrary result was observed: active patients at one site had 7% lower retention in the intervention year than in the preintervention year. This may have been due to structural and institutional changes occurring at the site during the intervention year, which presented barriers to keeping appointments. Those intervention year changes included restricted use of tokens for free transportation, disruption of the clinic's appointment reminder services, policy changes that required more frequent reestablishment of eligibility for clinic services, and suspension of established copay waiver systems.
This clinic-wide intervention study resulted in relative overall improvements in clinic attendance that ranged from 7% for 2 consecutive appointments kept, to 3% based on the proportion of all appointments kept. Although as a percentage the impact of the intervention was relatively small across all patients, the effect was highly significant and, if generalizable to all other HIV care sites, it provides a relatively low-cost and low-effort clinic-wide process that could improve clinic visit adherence for thousands of patients. There was a particularly large impact for new or reengaging patients, younger patients, and patients with a detectable viral load. These groups of patients particularly could benefit from improved clinic adherence, and alone would justify the minimal expenditure of effort and cost to obtain these gains.
Notes
Acknowledgments. The Retention in Care Study group consists of the following persons: Boston Medical Center: Mari-Lynn Drainoni (principal investigator [PI]), Cintia Ferreira, Lisa Koppelman, Roosevelt Lewis, Maya McDoom, Michal Naisteter, Karina Osella, Glory Ruiz, Meg Sullivan (PI); SUNY Downstate Medical Center: Sophia Gibbs-Cohen, Elana Desrivieres, Mayange Frederick, Kevin Gravesande, Susan Holman, Harry Johnson, Tonya Taylor, Tracey Wilson (PI); Centers for Disease Control and Prevention (CDC) and Health Resources and Services Administration (HRSA): Laura Cheever, HRSA, Faye Malitz, HRSA, Robert Mills, HRSA, Jason Craw, CDC/ICF International, Lytt Gardner, CDC, Sonali Girde, CDC/ICF International, Gary Marks, CDC; University of Alabama at Birmingham: Scott Batey, Stephanie Gaskin, Michael Mugavero (PI), Jill Murphree, Jim Raper, Michael Saag (PI), Suneetha Thogaripally, James Willig, Anne Zinski; Baylor College of Medicine, Houston, TX: Monisha Arya, David Bartholomew, Tawanna Biggs, Hina Budhwani, Jessica Davila, Thomas P. Giordano (PI), Nancy Miertschin, Shapelle Payne, William Slaughter; Johns Hopkins: Mollie Jenckes, Jeanne Keruly (PI), Angie McCray, Mary McGann, Richard Moore (PI), Melissa Otterbein, Liming Zhou; University of Miami: Carolyn Garzon, Jesline Jean-Simon, Kathy Mercogliano, Lisa Metsch (PI), Allan Rodriguez (PI), Gilbert Saint-Jean, Marvin Shika; Mountain Plains AIDS Education and Training Center: Lucy Bradley-Springer, Marla Corwin.
Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the US Centers for Disease Control and Prevention or the Health Resources and Services Administration.
Financial support. This work was supported by the CDC and the HRSA (CDC contracts 200-2007-23685 to Baylor College of Medicine, 200-2007-23690 to Boston Medical Center, 200-2007-23689 to Johns Hopkins University School of Medicine, 200-2007-23687 to Research Foundation of the State University of New York, SUNY Downstate Medical Center, 200-2007-23684 to University of Alabama at Birmingham, and 200-2007-23692 to University of Miami Miller School of Medicine).
Potential conflicts of interest. All authors: No reported conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1.Giordano TP, Gifford AL, White AC, Jr, et al. Retention in care: a challenge to survival with HIV infection. Clin Infect Dis. 2007;44:1493–9. doi: 10.1086/516778. [DOI] [PubMed] [Google Scholar]
- 2.Mugavero M, Lin H, Willig J, et al. Missed visits and mortality among patients establishing initial outpatient HIV treatment. Clin Infect Dis. 2009;48:248–56. doi: 10.1086/595705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Samet J, Freedberg K, Savetsky J, Sullivan L, Padmanabhan L, Stein M. Discontinuation from HIV medical care: squandering treatment opportunities. J Health Care Poor Underserved. 2003;14:244–55. doi: 10.1353/hpu.2010.0798. [DOI] [PubMed] [Google Scholar]
- 4.Tobias C, Cunningham WE, Cunningham CO, Pounds MB. Making the connection: the importance of engagement and retention in HIV medical care. AIDS Patient Care STDS. 2007;21(Suppl 1):S3–8. doi: 10.1089/apc.2007.9992. [DOI] [PubMed] [Google Scholar]
- 5.Gardner EM, McLees MP, Steiner JF, del Rio C, Burman WJ. The spectrum of engagement in HIV care and its relevance to test-and treat strategies for the prevention of HIV infection. Clin Infect Dis. 2011;52:793–800. doi: 10.1093/cid/ciq243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mugavero M, Norton W, Saag M. Health care system and policy factors influencing engagement in HIV medical care: piecing together the fragments of a fractured health care delivery system. Clin Infect Dis. 2011;52(Suppl. 1):S238–46. doi: 10.1093/cid/ciq048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ulett K, Willig J, Hui-Yi L, et al. The therapeutic implications of timely linkage and early retention in HIV care. AIDS Patient Care STDs. 2009;23:41–9. doi: 10.1089/apc.2008.0132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Mugavero M. Improving engagement in HIV care: what can we do? Top HIV Med. 2008;16:156–61. [PubMed] [Google Scholar]
- 9.Greenberg A, Hader S, Masur H, Young T, Skillicorn J, Dieffenbach C. Fighting HIV/AIDS in Washington D.C. Health Aff. 2009;28:1677–87. doi: 10.1377/hlthaff.28.6.1677. [DOI] [PubMed] [Google Scholar]
- 10.Magnus M, Jones K, Phillips G, et al. Characteristics associated with retention among African-American and Latino adolescent HIV-positive men: results from the outreach, care and prevention to engage HIV-seropositive young MSM of color special project of national significance initiative. JAIDS. 2010;53:529–36. doi: 10.1097/QAI.0b013e3181b56404. [DOI] [PubMed] [Google Scholar]
- 11.Giordano T, Visnegarwala F, White C, et al. Patients referred to an urban HIV clinic frequently fail to establish care: factors predicting failure. AIDS Care. 2005;17:773–83. doi: 10.1080/09540120412331336652. [DOI] [PubMed] [Google Scholar]
- 12.Cunningham C, Sohler N, Wong M, et al. Utilization of health care services in hard-to-reach marginalized HIV-infected individuals. AIDS Patient Care STDs. 2007;21:177–86. doi: 10.1089/apc.2006.103. [DOI] [PubMed] [Google Scholar]
- 13.Traeger L, O'Cleirigh C, Skeer M, Mayer K, Safren S. Risk factors for missed HIV primary care visits among men who have sex with men. J Behav Med. 2011 doi: 10.1007/s10865-011-9383-z. doi:10.1007/s10865-011-9383-z. Accessed 24 July 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Giordano T. Retention in HIV care: what the clinician needs to know. Top Antivir Med. 2011;19:12–6. [PMC free article] [PubMed] [Google Scholar]
- 15.Dieffenbach C, Fauci A. Universal voluntary testing and treatment for prevention of HIV transmission. JAMA. 2009;301:2380–2. doi: 10.1001/jama.2009.828. [DOI] [PubMed] [Google Scholar]
- 16.Holtgrave D. On the epidemiologic and economic importance of the National AIDS Strategy for the United States. JAIDS. 2010;55:139–42. doi: 10.1097/QAI.0b013e3181f4107a. [DOI] [PubMed] [Google Scholar]
- 17.White House Office of National AIDS Policy. National HIV/AIDS strategy. Available at: http://www.whitehouse.gov/administration/eop/onap/nhas. Accessed 22 June 2012.
- 18.Horstmann E, Brown J, Islam F, Buck J, Agins BD. Retaining HIV-infected patients in care: where are we? Where do we go from here? Clin Infect Dis. 2010;50:752–61. doi: 10.1086/649933. [DOI] [PubMed] [Google Scholar]
- 19.Craw J, Gardner L, Malitz F, et al. Intervention trials to retain HIV+ patients in primary medical care. Presented at: 2009 National HIV Prevention Conference; August 2009, Atlanta, GA. Poster 209T; abstract 116. [Google Scholar]
- 20.Gardner L, Marks G, Craw J, et al. Brief clinic-wide intervention promotes retention in care among HIV patients. Presented at: 2011 National HIV Prevention Conference; 15 August 2011, Atlanta, GA. Abstract 2018. [Google Scholar]
- 21.Richardson J, Milam J, McCutchan A, et al. Effect of brief safer-sex counseling by medical providers of HIV-1 seropositive patients: a multi-clinic assessment. AIDS. 2004;18:1–8. doi: 10.1097/00002030-200405210-00011. [DOI] [PubMed] [Google Scholar]
- 22.Fisher J, Fisher W, Cornman D, Amico R, Bryan A, Friedland G. Clinician-delivered intervention during routine clinical care reduces unprotected sexual behavior among HIV-infected patients. JAIDS. 2006;41:44–52. doi: 10.1097/01.qai.0000192000.15777.5c. [DOI] [PubMed] [Google Scholar]
- 23.Gardner L, Marks G, O'Daniels C, et al. Implementation and evaluation of a clinic-based behavioral intervention: positive steps for patients with HIV. AIDS Patient Care STDs. 2008;22:627–35. doi: 10.1089/apc.2007.0210. [DOI] [PubMed] [Google Scholar]
- 24.Johnson WD, Diaz RM, Flanders WD, et al. Behavioral interventions to reduce risk for sexual transmission of HIV among men who have sex with men. Cochrane Database Syst Rev. 2008;16:CD001230. doi: 10.1002/14651858.CD001230.pub2. [DOI] [PubMed] [Google Scholar]
- 25.Darbes L, Crepaz N, Lyles C, Kennedy G, Rutherford G. The efficacy of behavioral interventions in reducing HIV risk behaviors and incident sexually transmitted diseases in heterosexual African Americans. AIDS. 2008;22:1177–94. doi: 10.1097/QAD.0b013e3282ff624e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Bodenheimer T, Wagner E, Grumbach K. Improving primary care for patients with chronic illness. JAMA. 2002;288:1909–14. doi: 10.1001/jama.288.15.1909. [DOI] [PubMed] [Google Scholar]