Abstract
Background
Maintaining retention in preexposure prophylaxis (PrEP) care among diverse patient populations will be needed to support PrEP’s efficacy. We characterized patterns of PrEP care retention in a US municipal primary care health network and examined whether missed visits, a metric of care retention that is easy to evaluate in clinic, are associated with subsequent discontinuation.
Methods
We included individuals on PrEP from July 2012 until August 2017 in the San Francisco Primary Care Clinics, a 15-clinic municipal health network. We categorized PrEP usage patterns as follows: early discontinuation (<90 days), later discontinuation (after ≥90 days), and continuing use at the end of follow-up. We first examined early discontinuation using adjusted Poisson regression. In patients who continued PrEP for ≥90 days, we examined factors associated with late discontinuation.
Results
Of the 364 individuals who started PrEP, 16% discontinued PrEP before 90 days, 46% discontinued later, and 38% were retained in care over a median 12 months of observation. Transgender women were more likely to discontinue PrEP early (adjusted risk ratio; 2.16; 95% confidence interval, 1.36–3.49), and younger users were more likely to discontinue late (0.82 per 10-year increase in age; .70–.96), as were persons who use illicit drugs (1.59; 1.02–2.47). Missed visits during use of PrEP were associated with future discontinuation (adjusted risk ratio, 1.52; 95% confidence interval, 1.14–2.03). Later year of current PrEP use was associated with both early and late discontinuation.
Conclusion
Diverse populations may require differentiated care to continue PrEP. Missed visits should trigger tailored interventions to maximize the impact of PrEP.
Keywords: preexposure prophylaxis, PrEP persistence, retention in care, missed visits, primary care
Preexposure prophylaxis (PrEP) users in primary care continued PrEP for a median of 8 months. Transgender women experienced higher risk of early discontinuation; younger users experienced late discontinuation. Missed visits during PrEP are associated with discontinuation and should trigger outreach.
Preexposure prophylaxis (PrEP) is a highly efficacious intervention to prevent human immunodeficiency virus (HIV), and demonstration projects and clinical cohorts in specialty clinics have demonstrated high retention [1–5]. However, PrEP users in specialized settings may be more motivated and less diverse than the at-risk general population [6]. To meet the estimated 5 million visits annually required to meet PrEP demand nationally [7], provision of PrEP in primary care will be critical to the next phase of roll-out [1, 6, 8]. Furthermore, PrEP may be a gateway to other services that primary care can offer, including vaccines, cancer screening, screening for sexually transmitted infections, and detection of clinically silent chronic diseases, such as diabetes [6, 8].
Diverse populations receiving PrEP within primary care settings may face challenges to remain engaged in care, such as stigma, competing priorities, and the need for quarterly in-person visits for laboratory monitoring and PrEP refills [6, 9]. For instance, in a observational cohort where PrEP was provided free to young, African American men who have sex with men (MSM) in Atlanta, a high cumulative HIV incidence was demonstrated (6.2 % in 1 year), with 13 of 14 individuals who seroconverted having previously stopped PrEP or contemplated but not started PrEP [10]. Conversely, predisposing factors (eg, positive attitudes toward PrEP or concern for HIV risk), enabling factors (eg, ease at attending follow-up or contacting providers), and reinforcing factors (eg, outreach from clinic staff) may support PrEP retention [11–14]. For PrEP to help bend the curve of the HIV epidemic, primary care will need to keep individuals engaged with PrEP [6].
Care retention can be measured through various methods, with the proportion attending follow-up visits or receiving medications favored as the metric of retention in prior PrEP studies [1, 4, 15]. One method of measuring retention in care used in the HIV treatment literature, and recently examined in HIV prevention, is visit constancy, defined as visit attendance during regularly spaced intervals [16]. A recent study examining retention within health clinics for lesbian, gay, bisexual, and transgender persons in Chicago found that PrEP users, even those who remained engaged with PrEP, had difficulty attending quarterly visits, with only 15% achieving perfect visit constancy over a year (ie, attending visits in all 4 quarters) [9]. Although visit constancy better captures the density and appropriate spacing of care encounters, it is computationally intensive, and complementary methods are needed to direct real-time reengagement efforts [17]. Scheduled visits for which a patient is a no-show (ie, missed visits) have been examined extensively in the HIV care literature as a proxy for future HIV care engagement and can be evaluated easily in a clinic setting [17]. Researchers have found that missed visits are associated with both higher HIV viral loads and higher mortality rates and can be used effectively to guide reengagement interventions [16–18]. However, the relevance of missed visits for PrEP outcomes has not yet been explored to our knowledge.
We first sought to characterize patterns of PrEP persistence, including early discontinuation, late discontinuation, and persistence with PrEP in a large municipal-based network of primary care clinics. We then evaluated reasons for discontinuing PrEP care, using in-depth review of the medical record. We characterized factors associated with the various patterns in PrEP engagement to determine whether missed primary care visits—a metric of retention that is easy to evaluate in the clinic—are a useful triage tool for directing reengagement interventions. We further explored whether, once individuals lapse on PrEP, they may access other primary care services, which could be additional loci for PrEP reengagement.
METHODS
Study Population
Data for individuals prescribed PrEP in the San Francisco Primary Care Clinics (SFPCC), a 15-clinic integrated, municipal health system, were analyzed from July 2012 until August 2017, excluding those who started <6 months before observation end. Individuals who stopped PrEP <90 days before observation end were not included to avoid including those who might eventually restart PrEP. Demographics, visit data, missed visits while receiving PrEP (defined as scheduled visits for which the patient was a no-show without cancelling in advance), and PrEP prescriptions were obtained via download from the electronic medical record and chart abstraction.
Mental health diagnoses were collected via International Classification of Diseases, Tenth Revision, codes documented in the medical record. Schizophrenia and bipolar disorder were grouped, given their strong association with poor care utilization outcomes and because they are both considered severe mental illnesses, and patients in this group were compared with patients with depression and anxiety and those with no mental health diagnosis [19, 20]. Substance use, excluding cannabis, and housing status, were both evaluated based on review of the medical record, so these data could be captured only if documented by staff. PrEP indication was classified according to the following descending hierarchy: serodifferent couples, MSM, persons who inject drugs (PWID), transgender women who have sex with men (TGWSM), or at-risk heterosexual not in serodifferent couples. All data were collected anonymously, the study was determined to be exempt by the University of California, San Francisco, Institutional Review Board, and study procedures were in accordance with the Declaration of Helsinki.
Data Collection
PrEP start/stop dates were collected as documented in the electronic medical record. For the fewer than one-fifth of records in which a stop date was not available, the last date of an available prescription was used. Other missing data triggered repeated chart abstraction, with no missing discontinuation outcomes. Those who did not report illicit drug use or housing instability were classified as “none,” and those who declined to report their race/ethnicity were classified as “other.” The reason for PrEP discontinuation in the SFPCC was categorized based on in-depth review of the medical record. In-person visits were coded as PrEP-related versus not related, based on whether PrEP was discussed during the visit, and telephone visits in which PrEP was discussed were also recorded.
In 2015, a panel management/patient navigation intervention was initiated in 4 clinics in response to HIV seroconversions at higher-volume PrEP clinics; services included creating PrEP patient registries, routinizing follow-up/laboratory reminders, and making patient navigators available by text message on patient or provider request [21]. In 2 clinics, a pharmacist was available for scheduling follow-up visits. The panel management/navigation variable was coded at the clinic level as a time-dependent covariate, based on when the program started at each clinic [21].
Study Outcomes and Data Analysis
To better understand PrEP discontinuation as well as primary care utilization, follow-up for each participant was divided into 90-day periods, beginning at the initiation of PrEP use. The indicator for early PrEP discontinuation was <90 days of PrEP use in the first period, with no subsequent resumption of use. Similarly, repeated indicators of late PrEP discontinuation for each full period of PrEP use were defined by no PrEP use in the next full 90-day period. Final periods of PrEP use ending <90 days before the end of follow-up on 1 August 2017 were excluded. Early and late discontinuation were examined separately to understand factors associated with each type of retention pattern (Table 1). The proportion of individuals attending ≥1 in-person visit in each quarter while receiving PrEP (ie, 3-month visit constancy) [9] and the proportion with missed clinic visits while receiving PrEP were calculated [16, 17].
Table 1.
Characteristics of Users of Preexposure Prophylaxis in the San Francisco Primary Care Clinics by Retention Outcome
| Characteristic | PrEP Users, No. (%)a | P Valueb | |||
|---|---|---|---|---|---|
| Overall | Discontinued Early (<90 d) | Discontinued Late (≥90 d) | Continued PrEP | ||
| Total | 364 (100) | 58 (16) | 168 (46) | 113 (38) | |
| Age | |||||
| <25 y | 43 (12) | 15 (35) | 15 (35) | 13 (30) | .001 |
| 25–39 y | 187 (51) | 21 (11) | 101 (53 | 65 (35) | |
| 40–64 y | 129 (35) | 21 (16) | 51 (40) | 57 (44) | |
| ≥65 y | 5 (1) | 1 (20) | 1 (20) | 3 (60) | |
| Birth sex | |||||
| Female | 56 (15) | 13 (23) | 20 (36) | 23 (41) | .14 |
| Male | 308 (85) | 45 (15) | 148 (48) | 115 (37) | |
| Race/ethnicity | |||||
| African American | 45 (12) | 11 (24) | 19 (42) | 15 (33) | .51 |
| Asian | 29 (8) | 5 (17) | 12 (41) | 12 (41) | |
| Latino | 95 (26) | 14 (15) | 42 (44) | 39 (41) | |
| White | 136 (37) | 18 (13) | 62 (46) | 56 (41) | |
| Mixed/other | 59 (16) | 10 (17) | 33 (56) | 16 (27) | |
| Insurance | |||||
| Uninsured | 50 (14) | 10 (20) | 19 (38) | 21 (42) | .28 |
| Private | 28 (8) | 3 (11) | 18 (64) | 7 (25) | |
| Public | 286 (79) | 45 (16) | 131 (46) | 110 (38) | |
| Housing instabilityc | |||||
| Yes | 46 (13) | 8 (17) | 23 (50) | 15 (33) | .74 |
| None | 318 (87) | 50 (16) | 145 (46) | 123 (39) | |
| Reported illicit drug usec | |||||
| Yes | 57 (16) | 10 (18) | 28 (49) | 19 (33) | .72 |
| None | 307 (84) | 48 (16) | 140 (46) | 119 (39) | |
| Mental health diagnosisc | |||||
| Anxiety/depression | 113 (31) | 18 (16) | 47 (42) | 48 (42) | .71 |
| Bipolar disorder/schizophrenia | 50 (9) | 10 (20) | 26 (52) | 14 (28) | |
| None | 192 (59) | 30 (16) | 91 (47) | 71 (37) | |
| PrEP patients per provider | |||||
| 1–4 | 161 (47) | 29 (18) | 66 (41) | 71 (42) | .08 |
| ≥5 | 203 (53) | 29 (14) | 102 (50) | 67 (35) | |
| PrEP indication | |||||
| At-risk heterosexual | 17 (5) | 4 (24) | 6 (35) | 7 (41) | .25 |
| MSM | 240 (66) | 32 (13) | 112 (47) | 96 (40) | |
| PWID | 3 (1) | 1 (33) | 1 (33) | 1 (33) | |
| Serodifferent couple | 56 (16) | 8 (14) | 26 (46) | 22 (39) | |
| TGWSM | 45 (12) | 13 (29) | 20 (44) | 12 (27) | |
| Panel management/navigationd | |||||
| Yes | 83 (23) | 19 (23) | 29 (35) | 35 (42) | .04 |
| None | 281 (77) | 39 (14) | 139 (49) | 103 (37) |
Abbreviations MSM, men who have sex with men; PrEP, preexposure prophylaxis; PWID, persons who inject drugs; TGWSM: transgender women who have sex with men.
aParticipants were classified as discontinued early if they stopped within 90 days of use, discontinued late if they stopped after initially using PrEP for ≥90 days, and continued PrEP if they were still using PrEP after 12 months median observation.
bFisher exact test.
cAs documented in the medical record. Illicit drugs excluded cannabis, and mental health diagnoses were classified according to International Classification of Diseases, Tenth Revision, code.
dProgram included registry of all PrEP users, access to a patient navigator by text, and visit reminders; this was analyzed as a clinic-level variable during intervals in which it was active for the entire interval.
To estimate overall—as opposed to direct—covariate effects on each measure of PrEP discontinuation [22], we used directed acyclic graphs to select models including all measured confounders of the relationship of each covariate with PrEP discontinuation, based on the prior PrEP retention literature [1, 3, 9, 23], while excluding any mediators. Confounders included in the model for each covariate are listed in the footnotes of Table 2. Covariates were categorized as follows: age per 10-year increase, sex at birth, race/ethnicity, insurance status (public, private, or uninsured), PrEP indication (serodifferent couple, MSM, TGWSM, PWID, or at-risk heterosexual), mental health diagnosis (schizophrenia/bipolar disorder and anxiety/depression vs none), housing instability, substance use reported in the medical record (binary), higher-volume PrEP provider (≥5 vs <5 patients per provider), PrEP initiation year, and a time-dependent indicator for PrEP use periods beginning after the introduction of a panel management/patient navigation intervention at selected clinics. In the models for late discontinuation, a time-dependent indicator for any missed visit during the 90-day PrEP use interval, defined as a no-show visit not canceled in advance, was also examined [16].
Table 2.
Factors Associated With Early and Late Discontinuation of Preexposure Prophylaxis in the San Francisco Primary Care Clinics
| Factor | Early Discontinuationa | Late Discontinuationa | ||
|---|---|---|---|---|
| RR (95% CI) | aRR (95% CI) | RR (95% CI) | aRR (95% CI) | |
| Age per ten-year increaseb | 0.88 (.69–1.11) | 0.91 (.77–1.08) | 0.81 (.70–.92) | 0.82 (.70–.96) |
| Female birth sexb | 1.59 (.92–2.75) | 1.62 (.95–2.75) | 1.01 (.64–1.58) | 1.00 (.77–1.29) |
| Race/ethnicityb | ||||
| White | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) |
| Black | 1.85 (.94–3.61) | 1.70 (.74–3.93) | 1.95 (1.20–3.17) | 1.76 (.94–3.28) |
| Latino | 1.11 (.58–2.13) | 1.01 (.53–1.93) | 1.19 (.82–1.74) | 1.14 (.78–1.65) |
| Asian/other | 1.29 (.68–2.42) | 1.14 (.71–1.83) | 1.26 (.90–1.77) | 1.09 (.84–1.43) |
| Insuranceb | ||||
| Private | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) |
| Uninsured | 1.87 (.56–6.24) | 2.13 (.65–6.96) | 0.77 (.41–1.46) | 0.83 (.45–1.51) |
| Public | 1.47 (.49–4.43) | 1.61 (.57–4.54) | 0.70 (.46–1.07) | 0.73 (.40–1.33) |
| Housing instabilityc | 1.11 (.56–2.18) | 1.04 (.46–2.36) | 1.26 (.83–1.91) | 1.20 (.67–2.15) |
| Reported illicit drug usec | 1.12 (.60–2.09) | 1.01 (.53–1.93) | 1.53 (1.08–2.17) | 1.59 (1.02–2.47) |
| Mental health diagnosisc | ||||
| None | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) |
| Anxiety/depression | 1.01 (.60–1.70) | 0.96 (.61–1.51) | 0.69 (.50–.95) | 0.71 (.50–1.02) |
| Bipolar/schizophrenia | 1.05 (.48–2.33) | 0.94 (.42–2.11) | 1.27 (.87–1.85) | 1.17 (.76–1.82) |
| PrEP patients per provider (≥5 vs <5)d | 0.83 (.52–1.33) | 0.66 (.36–1.20) | 1.54 (1.15–2.05) | 1.36 (1.11–1.66) |
| Current PrEP use year b | ||||
| 2012-2014 | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) |
| (time-dependent) 2015 | 2.05 (.59–7.10) | 1.91 (.73–5.01) | 1.97 (1.34–2.89) | 1.92 (1.42–2.60) |
| 2016–2017 | 5.38 (1.73–16.78) | 4.80 (2.05–11.23) | 2.86 (1.95–4.20) | 2.79 (1.98–3.92) |
| PrEP indicationb | ||||
| MSM | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) | 1.00 (Reference) |
| TGWSM | 2.17 (1.24–3.80) | 2.16 (1.36–3.49) | 1.62 (1.00–2.62) | 1.84 (.98–3.46) |
| Othere | 1.28 (.71–2.32) | 1.15 (.60–2.17) | 0.96 (.67–1.40) | 1.00 (.71–1.42) |
| Panel management/patient navigationf | 1.65(1.01–2.70) | 1.05 (.47–2.34) | 1.68 (1.25–2.25) | 1.13 (.81–1.58) |
| Missed a visit (ie, no-show) while using PrEPg | … | … | 1.58 (1.18–2.11) | 1.52 (1.14–2.03) |
Abbreviations aRR, adjusted risk ratio; CI, confidence interval; MSM, men who have sex with men; PrEP, preexposure prophylaxis; RR, risk ratio (unadjusted); TGWSM, transgender women who have sex with men.
aEach adjusted model depends on the specific variables identified as confounders from direct acyclic graph analysis, noted below in footnotes. Each analysis used Poisson regression with robust standard errors accounting for clinic-level clustering. The early discontinuation model examines individuals who discontinue PrEP within <90 days compared with those who continue PrEP. The late discontinuation model examines discontinuation only among those who continued PrEP for ≥90 days.
bControlling for the 3 main demographic variables (age, sex, and race/ethnicity).
cControlling for the 3 main demographic variables, mental health diagnosis (classified per International Classification of Diseases, Tenth Revision, code), illicit drug use (excluding cannabis), and housing status (illicit drug use and housing status as documented in the medical record).
dControlling for the 3 main demographic variables, year, and PrEP indication.
eOther PrEP indications include serodifferent couple, at-risk heterosexual, and persons with injection drug use.
fControlling for the 3 main demographic variables, higher-volume provider (≥5 PrEP patients per provider), year, PrEP indication. Program included registry of all PrEP users, access to a patient navigator by text, and visit reminders; this was analyzed as a clinic-level, time-dependent covariate.
gControlling for the 3 main demographic variables, insurance status, higher-volume provider (≥5 PrEP patients per provider). Missed visits were examined as a time-dependent covariate over 90-day periods of PrEP use in the late discontinuation model.
Covariate effects on our binary indicators of discontinuation during or after each 90-day period were estimated using log-link Poisson models [24]. Robust standard errors with clustering by clinic were used to account for the correlation of repeated outcomes by both clinic and participant [25].
To characterize primary care utilization during PrEP gaps—that is, opportunities for PrEP reengagement—factors associated with the number of primary care visits in the 90 days after discontinuation were also examined using Poisson models with robust standard errors, again with clustering on clinic. As a sensitivity analysis, individuals who were excluded from the primary analysis because of stopping PrEP <90 days before the end of observation were included in each of the models.
RESULTS
Of the 364 individuals who started PrEP between 1 July 2012 and 31 January 2017 in the SFPCC, 58 (16%) discontinued PrEP before 90 days, 168 (46%) discontinued later, and 138 (38%) were persistent users over a median of 12 months of observation through 31 July 2017 (473 person-years of observation total; Figure 1). Of the individuals who continued PrEP, 25 (18%) had a ≥90-day gaps in use. Participants continued PrEP for a median of 8.2 months. Overall, 16% (n = 56) were female sex at birth and the median age was 35 years. They were racially/ethnically diverse: 12% (n = 45) were African American, 8% (n = 29) Asian, 26% (n = 95) Latino, 16% (n = 59) mixed/other, and 37% (n = 136) white; the indication for starting PrEP was serodifferent couple in 16% (n = 56), MSM in 66% (n = 240), TGWSM in 12% (n = 45) , PWID in 1% (n = 3), and at-risk heterosexual in 5% (n = 17). A substantial proportion of the sample (41%; n = 149) had a mental health diagnosis, 16% (n = 57) had substance use documented, and 13% (n = 46) reported housing instability during the observation interval. Almost a quarter (23%; n = 83) were seen at a clinic providing panel management and patient navigation support for their entire period of PrEP use (Table 1).
Figure 1.
Preexposure prophylaxis (PrEP) retention outcomes over a median 12 months of observation (July 2012 to August 2017) in the San Francisco Primary Care Clinics. In the first grouping (left) the number of individuals who started PrEP are graphed, overall and stratified by age (≥40, 30–39, or <30 years). The next grouping (middle) demonstrates the number of PrEP users who continued PrEP for ≥90 days, followed by those who continued PrEP at the end of the review (right). The percentage decrease is shown above the number of individuals in each stratum.
Reasons for Discontinuing PrEP
The documented reasons for PrEP discontinuation were as follows: cost or insurance issues for 13%, difficulty attending visits or completing laboratory tests for 44%, self-perceived decreased HIV risk for 11%, adverse effects for 4%, HIV seroconversion for 1% (3 individuals, including 1 self-attempting to use PrEP intermittently and 2 after self-assessment of being at low risk of HIV infection), other for 12% (concerns about taking a daily medication or future toxicity); for 15%. a reason was not documented.
Factors Associated With PrEP Discontinuation
In adjusted analysis of 364 first 90-day periods of PrEP use, the only factors associated with early discontinuation in the SFPCC were more recent year (P < .001 for trend) and initiating PrEP as a TGWSM versus MSM (adjusted risk ratio [aRR], 2.16; 95% confidence interval [CI], 1.36–3.49).
In examining the next part of the PrEP retention cascade, late discontinuation among individuals fully engaged with PrEP for ≥90 days, a total of 4587 person-months were available for analysis, with a median of 12 person-months per participant (interquartile range, 6–21; range, 3–51). In this analysis, the factors associated with late PrEP discontinuation in the SFPCC were quite different (Table 2). In adjusted analysis, more recent year of PrEP use was associated with a higher risk of late discontinuation (P < .001 for trend), as was reported illicit substance use versus not (aRR, 1.59; 95% CI, 1.02–2.47), being a higher-volume PrEP provider (defined as having ≥5 vs <5 patients per provider) (1.36; 1.11–1.66), and younger age (0.82 per 10-year increase in age; .70–.96) (Table 2 and Figure 1). Missing a visit while receiving PrEP, analyzed as a time-dependent covariate, was associated with future PrEP discontinuation (aRR, 1.52; 95% CI, 1.14–2.03).
Care Utilization During PrEP
Of the individuals who persisted a year with PrEP, only 32% of individuals attended an in-person visit during every quarterly interval while receiving PrEP, with an additional 31% attending a visit in 3 of 4 quarters; the median number of visits per year was 3 (interquartile range, 1–5). In 8% of PrEP use intervals, patients received a PrEP-related telephone visit only. Overall, 55% of individuals had a missed visit while receiving PrEP.
Factors Associated With Accessing Primary Care During a Gap
In adjusted analysis, older age was the only factor associated with accessing primary care in the 90 days after stopping PrEP; that is, younger individuals had less primary care contact (aRR, 1.27 per 10-year increase in age; 95% CI, 1.13–1.42).
Sensitivity Analysis
In the sensitivity analysis, including individuals who discontinued PrEP within 90 days of observation end resulted in African American versus white race/ethnicity being associated with late discontinuation (aRR, 1.97; 95% CI, 1.06–3.08), with other results not meaningfully affected.
DISCUSSION
Overall, in a diverse population of PrEP users in a municipal primary care setting, only 38% continued PrEP over a median of 1 year (12 months of observation), with a median PrEP duration of about 8 months. The rate of PrEP retention in this San Francisco-based primary care clinic network was much lower than that reported among PrEP users in specialty clinics or demonstration projects [1, 3] and similar to that reported in a network of clinics for lesbian, gay, bisexual, and transgender persons [9]. Health systems will need to focus on PrEP retention in primary care settings to extend the impact of PrEP in diverse populations [6]. For instance, cost or insurance issues experienced by many could have potentially been addressed with assistance of a PrEP navigator [26]. Finally, missed visits while using PrEP are particularly strongly associated with future discontinuation and should be treated as a red flag to target patients for additional reengagement interventions.
By examining patterns of PrEP retention, we were able to identify key populations specifically at risk of falling out of the PrEP engagement continuum. Given challenges with early discontinuations in transgender women, an intensive, tailored time-limited intervention, such as PrEP case management or a mobile health intervention [27], could be considered proximate to PrEP initiation. Youth, on the other hand, were more likely to discontinue PrEP late but also less likely to contact the primary care system during gaps. Mobile health interventions permitting 2-way communication are highly acceptable and may be particularly effective at improving PrEP adherence in youth, requiring additional study [27, 28]. Persons who use drugs were also more likely to discontinue late. For PrEP users with opioid use disorders, integrated substance use and PrEP care should be considered [29], whereas for those who use stimulants, additional interventions are needed, such as cues, behavioral interventions, or cognitive remediation strategies [3, 30, 31].
Individuals who were using PrEP in recent years were more likely to permanently discontinue PrEP in our system, suggesting that later PrEP users may need additional support to remain on PrEP and/or assistance to manage insurance “churn”/care transition. Analyses that have found increasing overall prevalence of PrEP use in the population could overestimate the population-level protection from PrEP, particularly if PrEP discontinuation is increasing over time [32]. The association between more experienced providers and higher risk of PrEP interruption may be related to higher-volume providers offering PrEP to patients who are less self-motivated to be on PrEP. Alternatively, higher-volume providers may have difficulty tracking more PrEP patients.
Although we have previously found that panel management and patient navigation is associated with improved laboratory monitoring adherence and more rapid PrEP initiation, we did not find evidence that the program was associated with a lower PrEP discontinuation risk [21, 26]. Similar programs may need to address PrEP persistence with proactive outreach throughout the course of PrEP. Future studies should examine whether using missed visits to guide potential reengagement interventions, such as targeted outreach with enhanced contact [27], behavioral interventions [33], or incentives [34], can prospectively reduce PrEP discontinuations, as has been seen in HIV treatment [18].
Routine care delivery in primary care focuses on scheduled in-person visits [7], which may limit opportunities to reach individuals who have few PrEP-related visits after discontinuing. In addition, few individuals, even while using PrEP, were able to attend quarterly in-person visits as recommended by the Centers for Disease Control and Prevention [9]. Alternate PrEP service delivery strategies, such as drop-in or pharmacy-led delivery, will need to be considered [5, 7, 35]. An integrated PrEP and mental health case management model should be considered, particularly for patients with mental health diagnoses [36]. Close coordination of PrEP and transgender health visits is probably preferable for transgender individuals [37]. Although urgent visits rarely include a discussion of preventive health, they may represent important opportunities for PrEP reengagement, particularly for PWID or those with unstable housing [38]. Finally, telephone or video chat follow-up is a promising strategy for carefully selected individuals who have difficulty attending in-person visits [7, 34].
Limitations of the current study include our inability to analyze data not documented by providers, our inability to determine whether individuals who permanently discontinued PrEP in our system restarted PrEP in another (eg, private clinic) setting, limited generalizability to patients not in a safety-net primary care setting, and a challenge inherent to all studies of PrEP retention, the difficulty of determining whether those who experience PrEP gaps remain at risk of HIV infection. We attempted to address this challenge by performing in-depth review of the medical record to identify documented reasons for discontinuation. In addition, recent data from Montreal has suggested that the HIV incidence is quite high in individuals who discontinue PrEP and are lost to follow-up [39], and 2 individuals in our sample who discontinued after assessing themselves as low risk ultimately seroconverted. Furthermore, it is unlikely that the higher discontinuation among younger individuals is related only to decreasing HIV risk [40], although it is possible that they experience higher levels of insurance churn.
The finding that the PrEP discontinuation risk was higher in later years may be affected by individuals who discontinued PrEP later, having less time to restart PrEP, particularly in the 2016–2017 period. Overall, the number of individuals who stopped and restarted PrEP was low (7% of the sample), and this phenomenon would have been unlikely to have affected higher discontinuation risk in 2015, >1.5 years before the end of review. PrEP discontinuation should be examined by year in future studies. Finally, we cannot exclude the possibility that residual confounding, despite attention to confounder selection and model specification, has occurred in this observational study.
PrEP retention is an important concern, and a one-size-fits-all model of PrEP care delivery may not work for many individuals. Attending quarterly in-person visits may be a barrier for some PrEP users, and innovative PrEP service redesign may be needed. Our analysis suggests that differentiated care delivery may be the way forward. Some populations may require intensive case management/navigation up front but may stabilize over time. Other PrEP users may need ≥1 of the following interventions: PrEP care integrated with mental health/substance use services, proactive outreach deployed through mobile health, and/or more flexible follow-up strategies, such as phone visits or drop-in availability. Missed visits in particular should trigger additional outreach, an assessment of care needs, and an individualized action plan that can support individuals to remain engaged in PrEP if we hope to maximize PrEP’s impact in diverse populations.
Acknowledgment
We acknowledge Hali Hammer and Catherine James for programmatic support of the PrEP registry, and Anne Hirozawa and Patricia von Felten for assistance with data collection and management.
Financial support: This work was supported by the National Institute of Mental Health, National Institutes of Health (grant R01MH109320).
Potential conflicts of interest. A. Y. L. and S. P. B. have led studies in which Gilead Sciences has donated the study drug. All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1. Marcus JL, Hurley LB, Hare CB, et al. . Preexposure prophylaxis for HIV prevention in a large integrated health care system: adherence, renal safety, and discontinuation. J Acquir Immune Defic Syndr 2016; 73:540–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Volk JE, Marcus JL, Phengrasamy T, et al. . No new HIV infections with increasing use of HIV preexposure prophylaxis in a clinical practice setting. Clin Infect Dis 2015; 61:1601–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Hojilla JC, Vlahov D, Crouch PC, et al. . HIV pre-exposure prophylaxis (PrEP) uptake and retention among men who have sex with men in a community-based sexual health clinic. AIDS Behav 2018; 22:1096–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Liu AY, Cohen SE, Vittinghoff E, et al. . Preexposure prophylaxis for HIV infection integrated with municipal- and community-based sexual health services. JAMA Intern Med 2016; 176:75–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Hoagland B, Moreira RI, De Boni RB, et al. . High pre-exposure prophylaxis uptake and early adherence among men who have sex with men and transgender women at risk for HIV infection: the PrEP Brasil demonstration project. J Int AIDS Soc 2017; 20:1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Calabrese SK, Krakower DS, Mayer KH. Integrating HIV preexposure prophylaxis (PrEP) into routine preventive health care to avoid exacerbating disparities. Am J Public Health 2017; 107:1883–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Siegler AJ, Mayer KH, Liu AY, et al. . Developing and assessing the feasibility of a home-based preexposure prophylaxis monitoring and support program. Clin Infect Dis 2019; 67:501–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Marcus JL, Levine K, Grasso C, et al. . HIV preexposure prophylaxis as a gateway to primary care. Am J Public Health 2018; e1–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Rusie LK, Orengo C, Burrell D, et al. . PrEP initiation and retention in care over five years, 2012–2017: are quarterly visits too much? Clin Infect Dis 2018; 67:283–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Serota DP, Rosenberg ES, Lockard AM, et al. . Beyond the biomedical: preexposure prophylaxis failures in a cohort of young black men who have sex with men in Atlanta, Georgia. Clin Infect Dis 2018; 67:965–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Green LW. Toward cost-benefit evaluations of health education: some concepts, methods, and examples. Health Education Monograms 1974; 2:34–64. [Google Scholar]
- 12. Semitala FC, Camlin CS, Wallenta J, et al. . Understanding uptake of an intervention to accelerate antiretroviral therapy initiation in Uganda via qualitative inquiry. J Int AIDS Soc 2017; 20(4). doi:10.1002/jia2.25033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Stekler JD, McMahan V, Ballinger L, et al. . HIV pre-exposure prophylaxis prescribing through telehealth. J Acquir Immune Defic Syndr 2018; 77:e40–2. [DOI] [PubMed] [Google Scholar]
- 14. Rolle CP, Rosenberg ES, Siegler AJ, et al. . Challenges in translating PrEP interest into uptake in an observational study of young black MSM. J Acquir Immune Defic Syndr 2017; 76:250–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Chan PA, Mena L, Patel R, et al. . Retention in care outcomes for HIV pre-exposure prophylaxis implementation programmes among men who have sex with men in three US cities. J Int AIDS Soc 2016; 19:20903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Mugavero MJ, Westfall AO, Cole SR, et al. ; Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) Beyond core indicators of retention in HIV care: missed clinic visits are independently associated with all-cause mortality. Clin Infect Dis 2014; 59:1471–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Zinski A, Westfall AO, Gardner LI, et al. . The contribution of missed clinic visits to disparities in HIV viral load outcomes. Am J Public Health 2015; 105:2068–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Gardner LI, Giordano TP, Marks G, et al. ; Retention in Care Study Group Enhanced personal contact with HIV patients improves retention in primary care: a randomized trial in 6 US HIV clinics. Clin Infect Dis 2014; 59:725–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Ebuenyi I, Taylor C, O’Flynn D, et al. . The Impact of co-morbid severe mental illness and HIV upon mental and physical health and social outcomes: a systematic review. AIDS Care 2018; 30:1586–94. [DOI] [PubMed] [Google Scholar]
- 20. Mangurian C, Cournos F, Schillinger D, et al. . Low rates of HIV testing among adults with severe mental illness receiving care in community mental health settings. Psychiatr Serv 2017; 68:443–8. [DOI] [PubMed] [Google Scholar]
- 21. Spinelli MA, Scott HM, Vittinghoff E, et al. . Provider adherence to pre-exposure prophylaxis monitoring guidelines in a large primary care network. Open Forum Infect Dis 2018; 5:ofy099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am J Epidemiol 2013; 177:292–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Lankowski AJ, Bien-Gund CH, Patel VV, Felsen UR, Silvera R, Blackstock OJ. PrEP in the real world: predictors of 6-month retention in a diverse urban cohort. AIDS Behav 2018. doi: 10.1007/s10461-018-2296-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Zou G. A modified Poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004; 159:702–6. [DOI] [PubMed] [Google Scholar]
- 25. Pepper JV. Robust inferences from random clustered samples: an application using data from the panel study of income dynamics. Econ Lett 2002; 75:341–5. [Google Scholar]
- 26. Spinelli MA, Scott HM, Vittinghoff E, Liu AY, Morehead-Gee A, Gonzalez R, Gandhi M, Buchbinder SP. Brief Report: A Panel Management and Patient Navigation Intervention Is Associated With Earlier PrEP Initiation in a Safety-Net Primary Care Health System. J Acquir Immune Defic Syndr2018; 79(3):347–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Fuchs JD, Stojanovski K, Vittinghoff E, et al. . A mobile health strategy to support adherence to antiretroviral preexposure prophylaxis. AIDS Patient Care STDS 2018; 32:104–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Moore DJ, Jain S, Dubé MP, et al. . Randomized controlled trial of daily text messages to support adherence to preexposure prophylaxis in individuals at risk for human immunodeficiency virus: the TAPIR study. Clin Infect Dis 2018; 66:1566–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Choopanya K, Martin M, Suntharasamai P, et al. ; Bangkok Tenofovir Study Group Antiretroviral prophylaxis for HIV infection in injecting drug users in Bangkok, Thailand (the Bangkok Tenofovir Study): a randomised, double-blind, placebo-controlled phase 3 trial. Lancet 2013; 381:2083–90. [DOI] [PubMed] [Google Scholar]
- 30. Hojilla JC, Vlahov D, Glidden DV, et al. . Skating on thin ice: stimulant use and sub-optimal adherence to HIV pre-exposure prophylaxis. J Int AIDS Soc 2018; 21:e25103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Shrestha R, Altice FL, Karki P, Copenhaver MM. Integrated bio-behavioral approach to improve adherence to pre-exposure prophylaxis and reduce HIV risk in people who use drugs: a pilot feasibility study. AIDS Behav 2018; 22:2640–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Sullivan PS, Giler RM, Mouhanna F, et al. . Trends in the use of oral emtricitabine/tenofovir disoproxil fumarate for pre-exposure prophylaxis against HIV infection, United States, 2012–2017. Ann Epidemiol 2018; 14:e0210096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Landovitz RJ, Beymer M, Kofron R, et al. . Plasma tenofovir levels to support adherence to TDF/FTC preexposure prophylaxis for HIV prevention in MSM in Los Angeles, California. J Acquir Immune Defic Syndr 2017; 76: 501–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Marcus JL, Buisker T, Horvath T, et al. . Helping our patients take HIV pre-exposure prophylaxis (PrEP): a systematic review of adherence interventions. HIV Med 2014; 15:385–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Okoro O, Hillman L. HIV pre-exposure prophylaxis: Exploring the potential for expanding the role of pharmacists in public health. J Am Pharm Assoc (2003) 2018; 58:412–20.e3. [DOI] [PubMed] [Google Scholar]
- 36. Arnold T, Brinkley-Rubinstein L, Chan PA, et al. . Social, structural, behavioral and clinical factors influencing retention in pre-exposure prophylaxis (PrEP) care in Mississippi. PLoS One 2017; 12:e0172354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Restar AJ, Kuhns L, Reisner SL, et al. . Acceptability of antiretroviral pre-exposure prophylaxis from a cohort of sexually experienced young transgender women in two U.S. cities. AIDS Behav 2018; 22:3649–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Ayangbayi T, Okunade A, Karakus M, Nianogo T. Characteristics of hospital emergency room visits for mental and substance use disorders. Psychiatr Serv 2017; 68:408–10. [DOI] [PubMed] [Google Scholar]
- 39. Greenwald Z, Beuachemin M, Benomar K, et al. . High seroconversion rates following PrEP discontinuance in a Montreal clinic [#1038 ]. In: Conference on Retroviruses and Opportunistic Infections on March 4–7, 2018; Boston, Massachusetts. [Google Scholar]
- 40. Pines HA, Gorbach PM, Weiss RE, et al. . Sexual risk trajectories among MSM in the United States: implications for pre-exposure prophylaxis delivery. J Acquir Immune Defic Syndr 2014; 65:579–86. [DOI] [PMC free article] [PubMed] [Google Scholar]