Self-reported use of HIV pre-exposure prophylaxis is highly accurate among Sexual Health Clinic patients in New York City

Abstract

In NYC, 91% of sexually transmitted infection clinic patients reported pre-exposure prophylaxis (PrEP) use that matched detection of PrEP in their serum. Self-report had 80% sensitivity and 96% specificity (kappa=0.79) compared to measured PrEP. Our findings suggest that self-report may be a valid indicator of PrEP uptake.

Summary

Compared to an objective biomarker of drug level measurements, self-report provides an accurate indicator for assessing pre-exposure prophylaxis use among New York City (NYC) sexually transmitted infection (STI) clinic patients.

Introduction

Scaling up access to HIV pre-exposure prophlaxis (PrEP) is one of the pillars of the U.S. Ending the HIV Epidemic initiative ^1,2. Integrating PrEP into broader routine sexual health services can improve equitable access and PrEP uptake ³. Expanding PrEP access is important in settings such as publicly funded sexually transmitted infection (STI) clinics that serve racial/ethnic and sexual minority groups who often do not have access to regular health care ⁴ but have disproportionately high rates of new HIV diagnoses ^5-8, and where there are key opportunities for offering PrEP at the time of STI diagnosis. Administrative data used to track PrEP uptake and coverage (e.g., pharmacy claims) often cannot capture many characteristics of individuals who are provided HIV prevention services (e.g., their STI status) ⁹, whereas ascertaining PrEP use among a well-characterized sentinel population of patients seeking sexual health services can provide useful information on the need for, and effectiveness of, PrEP scale-up efforts ¹⁰.

In most clinical settings, PrEP use is commonly captured through patient self-report, which can be subject to over-reporting due to social desirability bias or under-reporting related to stigma around HIV risk prevention ^11-13. Agreement between self-report and a biomarker for PrEP has been shown in demonstration projects and clinical trial settings; accuracy of self-report (e.g., over-reporting of PrEP use) may be affected when study retention is incentivized^14,15. From a programmatic perspective, having an accurate ascertainment of patients’ PrEP status, regardless of regimen adherence, can inform the provision of PrEP-related services and also potentially minimize the misallocation of resources for counseling and providing PrEP medication for patients who are already on PrEP. Our study aimed to assess agreement between self-report and an objective measure of PrEP use in a sample of patients diagnosed with STIs in NYC sexual health clinics.

Methods

Study setting and sample

Our sample was drawn from patients attending eight clinics operated by the NYC Department of Health and Mental Hygiene (DOHMH). Remnant serum specimens from blood collected for routine syphilis testing were available for a systematic sample of 744 cismale and cisfemale patients who received clinical evaluations, were negative for HIV (by recent/day of nucleic acid amplification test or self-report), and were diagnosed with Chlamydia trachomatis (CT), Neisseria gonorrhoeae (GC), or early syphilis (primary, secondary, or early latent syphilis) on the day of the clinic visit (January-June 2019). Serum levels of tenofovir were performed on the remnant sera in order to provide an objective marker of recent PrEP use for these participants; we then compared this objective measure to their self-reported PrEP use.

For patients who do not report current PrEP use, the clinics provide PrEP services that include assessment of PrEP eligibility and interest in starting PrEP, a one-month regimen of PrEP, and linkage to longer-term PrEP care with community PrEP providers.

Measures

Objective PrEP measure: Serum samples from study patients were sent to the University of California San Francisco (UCSF) Hair Analytical Laboratory for analysis, where tenofovir (TFV) and emtricitabine (FTC) levels were analyzed by liquid chromatography-tandem mass spectrometry (LC/MS/MS) using validated methods ¹⁶; detectable serum levels (cut-off values for TFV and FTC detection: 0.150 ng/mL and 1.50 ng/mL, respectively) indicated drug exposure within the previous 48-72 hours. PrEP results were returned to NYC DOHMH as “detected” or “not detected”.

Self-reported PrEP measure: Clinic patients are routinely asked about PrEP use at each visit that includes a clinical evaluation that ascertains sexual history and behaviors. Current PrEP use was defined as patient reporting being on PrEP on the day of the visit that yielded a remnant serum sample. For patients without documented PrEP use status on the day of serum collection who had a clinic visit within the prior three weeks, we used PrEP status from the previous visit.

Other measures of interest: Patient characteristics included socio-demographic and sexual behaviors captured in clinic electronic medical records. Income or other individual-level data on socioeconomic status were not available; therefore, we examined neighborhood-level poverty as a proxy for socioeconomic status. Neighborhood-level poverty, using standard cut-points, is defined as the percent of the population in a given zip code tabulation area whose household income is below the federal poverty level (FPL) ¹⁷, and was assigned based on patient address.

Analysis

We compared proportions of patients who self-reported PrEP use with those who were on PrEP according to LC/MS/MS assay results (“measured” use), stratified by socio-demographics factors. We assessed the agreement between self-reported and measured PrEP use with Cohen’s kappa statistic test ¹⁸. We also assessed the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of self-report compared to the objective measure.

Patients with concordant measures were those for whom there was agreement between self-reported and measured PrEP use (i.e., self-reported PrEP use and TFV/FTC measured in serum, or self-reported no PrEP use and TFV/FTC not detectable in serum). Patients with discordant measures were those for whom there was disagreement between the two (i.e., self-reported PrEP use and no measured TFV/FTC in serum, or self-reported no PrEP use and measured TFV/FTC in serum). Given small numbers in discordant subgroups, we combined the two concordant subgroups and, separately, the two discordant subgroups and evaluated associations between patient characteristics and discordance using Chi-square tests of independence (significance level: p<0.05). To assess the implications of under-reporting of PrEP use in the clinics, we summarized clinic-based PrEP counseling and navigation outcomes.

Analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC). This study was approved by the NYC DOHMH Institutional Review Board.

Results

Our sample included 738 patients who had both a self-reported and serum PrEP assay result. Among those, there were 7 patients for whom self-reported current PrEP status was obtained from a previous visit (3 users and 4 non-users). To avoid misclassification of PEP as PrEP use, two patients who reported being on PEP on the day of clinic visit and had detectable TFV in their sera were excluded from analysis. Of the remaining 736 patients, 381 (51.8%) were ages 25—35 years; 640 (87.0%) were male; 188 (25.5%) were non-Hispanic Black and 235 (31.9%) were Hispanic; and 235 (39.1%) lived in medium-poverty neighborhoods (where 10%-<20% of households had incomes below FPL). A total of 211 of the 734 patients (28.7%) self-reported current PrEP use, and 239 of the 734 patient (32.5%) had detectable TFV/FTC in sera (Table 1).

Table 1.

Characteristics of study sample, New York City Sexual Health Clinics, January-June 2019.

	All patients						Males						Females
Characteristic	All patients		Self- reported PrEP use		Detected PrEP in serum		All males		Self- reported PrEP use		Detected PrEP in serum		All females		Self- reported PrEP use		Detected PrEP in serum
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
Total	736	100	211	28.7	239	32.5	640	87.0	211	100.0	237	99.2	96	13.0	0	0.0	2	0.8
Age Group (years)
15 – 24	202	27.5	22	10.4	33	13.8	153	23.9	22	10.4	32	13.5	49	51.0	0	0.0	1	50.0
25 – 34	381	51.8	130	61.6	146	61.1	349	54.5	130	61.6	145	61.2	32	33.3	0	0.0	1	50.0
35 – 44	103	14.0	43	20.4	44	18.4	92	14.4	43	20.4	44	18.6	11	11.5	0	0.0	0	0.0
45 +	50	6.8	16	7.6	16	6.7	46	7.2	16	7.6	16	6.8	4	4.2	0	0.0	0	0.0
Race/ethnicity
White	239	32.5	98	46.5	109	45.6	153	23.9	22	10.4	32	13.5	7	7.3	0	0.0	0	0.0
Black	188	25.5	31	14.7	38	15.9	349	54.5	130	61.6	145	61.2	55	57.3	0	0.0	1	50.0
Hispanic	235	31.9	56	26.5	64	26.8	92	14.4	43	20.4	44	18.6	29	30.2	0	0.0	1	50.0
Other/unknown	74	10.1	26	12.3	28	11.7	46	7.2	16	7.6	16	6.8	5	5.2	0	0.0	0	0.0
Neighborhood Poverty Level *
Low poverty	55	7.5	16	7.6	17	7.1	47	7.3	16	7.6	17	7.2	8	8.3	0	0.0	0	0.0
Medium poverty	288	39.1	89	42.2	103	43.1	266	41.6	89	42.2	102	43.0	22	22.9	0	0.0	1	50.0
High poverty	267	36.3	70	33.2	77	32.3	224	35.0	70	33.2	76	32.1	43	44.8	0	0.0	1	50.0
Very high poverty	72	9.8	17	8.1	21	8.8	54	8.4	17	8.1	21	8.9	18	18.8	0	0.0	0	0.0

^*Low poverty: < 10% below federal poverty level (FPL); Medium poverty: 10% to < 20% below FPL; High poverty: 20% to < 30% below FPL; Very high poverty: ≥ 30% below FPL

54 patients were missing neighborhood poverty level information; 5 females and 49 males.

Agreement between self-reported and measured PrEP use was 91.0% (192/211) among PrEP users and 91.1% (478/525) among non-users (kappa = 0.79, 95% CI: 0.74 – 0.84). The sensitivity, specificity, PPV, and NPV of self-report were 80.3% (192/239), 96.2% (478/497), 91.0% (192/211), and 91.1% (478/525), respectively.

There was demonstrated discordance for 66 (9.0%) patients: 47 with measured PrEP did not self- report PrEP use and 19 without measured PrEP self-reported PrEP use. Patients with demonstrated discordance were more likely to be male (10.0% versus 2.1% among females), live in medium-poverty neighborhoods, report 6-10 sex partners in the previous three months (15.7% versus 6.1%-9.6% among patients with < 6 partners, and 6.5% among those with > 10 partners), have a partner living with HIV infection (12.7% versus 7.4% among patients without partner(s) living with HIV), and have taken PEP in the past year (20.0% versus 8.5% among patients without a past-year history of PEP use) (Table 2).

Table 2.

Concordance and discordance between self-reported and measured HIV pre-exposure prophylaxis use - New York City Sexual Health Clinic study patients, January-June 2019.

Characteristic	Concordant				Discordant				Both Concordant groups		Both Discordant Groups		P- value‡
	Self-report + Biomarker +		Self-report − Biomarker −		Self-report + Biomarker −		Self-report − Biomarker +
	No.	Row %	No.	Row %	No.	Row %	No.	Row %	No.	Row %	No.	Row %
Age (years)													0.68
15 – 24	20	10.4	167	34.9	2	10.5	13	27.7	187	92.6	15	7.4
25 – 34	121	63.0	226	47.3	9	47.4	25	53.2	347	91.1	34	8.9
35 – 44	38	19.8	54	11.3	5	26.3	6	12.8	92	89.3	11	10.7
45 +	13	6.8	31	6.5	3	15.8	3	6.4	44	88.0	6	12.0
Gender													0.01
Female	0	-	94	19.7	0	-	2	4.3	94	97.9	2	2.1
Male	192	100	384	80.3	19	100	45	95.7	576	90.0	64	10.0
Race/ethnicity													0.09
White	94	49.0	126	26.4	4	21.1	15	31.9	220	92.1	19	8.0
Black	28	14.6	147	30.8	3	15.8	10	21.3	175	93.1	13	6.9
Hispanic	45	23.4	160	33.5	11	57.9	19	40.4	205	87.2	30	12.8
Other/unknown	25	13.0	45	9.4	1	5.3	3	8.4	70	94.6	4	5.4
Neighborhood poverty level													0.03
< 10% below FPL	15	7.8	37	7.7	1	5.3	2	4.3	52	94.6	3	5.4
10% to < 20% below FPL	78	40.6	174	36.4	11	57.9	25	53.2	252	87.5	36	12.5
20% to < 30% below FPL	66	34.4	186	38.9	4	21.1	11	23.4	252	94.4	15	5.6
≥ 30% below FPL	16	8.3	50	10.5	1	5.3	5	10.6	66	91.7	6	8.3
Limited English proficiency													0.68
Yes	2	1.0	13	2.7	0	-	2	4.3	15	88.2	2	11.8
No	190	99.0	465	97.3	19	100	45	95.7	655	91.1	64	8.9
No. of sex partners (past 3 months)
0 – 2	25	13.0	221	46.2	4	21.1	12	25.5	246	93.9	16	6.1	0.03
3 – 5	67	34.9	179	37.5	8	42.1	18	38.3	246	90.4	26	9.6
6 – 10	45	23.4	41	8.6	6	31.6	10	21.3	86	84.3	16	15.7
> 10	52	27.1	35	7.3	1	5.3	5	10.6	87	93.6	6	6.5
Condom use (past 3 months)													0.43
Always	5	2.6	79	16.5	3	15.8	3	6.4	84	93.2	6	6.7
Inconsistent/never	176	91.7	376	78.7	15	79.0	41	87.2	552	90.8	56	9.2
HIV+ partner* (past 3 months)													0.02
Yes	96	50.0	96	20.1	10	52.6	18	38.3	192	87.3	38	12.7
No	96	50.0	382	79.9	9	47.4	29	61.7	478	92.6	28	7.4
Prior PEP (past 6 months)													0.03
Yes	9	4.7	15	3.1	0	-	6	12.8	24	80.0	6	20.0
No	183	95.3	463	96.9	19	100	41	87.2	646	91.5	60	8.5
Total	192	26.2	478	64.8	19	2.6	47	6.5	670	91.0	66	9.0

FPL: federal poverty level

^‡A chi square test of independence showing differences between combined concordant and combined discordant groups

^*Sex or needle-sharing partner in past 3 months

Among the 47 patients with measured PrEP who did not self-report PrEP use, 18 (38.3%) accepted PrEP navigation services; 17 received PrEP-related education and adherence counseling, 6 accepted a referral to a facility for ongoing PrEP care, and one was given a one-month supply of PrEP medication in the clinic at the time of visit.

Discussion

We observed high levels of agreement between self-reported and measured PrEP use among a sentinel population diagnosed with STIs and at high risk for acquiring HIV, demonstrating the validity of self-report for ascertaining PrEP use among patients in sexual health clinics in NYC. In our study, self-report had high sensitivity, identifying 80% of patients on PrEP, and even higher specificity, resulting in few missed opportunities to offer PrEP to persons who would benefit from it. These findings align with prior work that examined concordance between self-report and an objective PrEP marker in a clinical setting ¹⁹.

Roughly 9% of patients in our sample inaccurately reported their true PrEP use. The high concordance between our PrEP use measures may reflect established trust between patients and providers in the communities that these clinics serve ²⁰ and normalized perceptions of PrEP as a HIV prevention strategy within STI clinic settings ²¹. Some patients may be less likely to misreport PrEP use to staff in such settings, compared with their regular providers/PrEP providers with whom there may be social desirability bias. While the proportion of patients demonstrating discordance was small, discordance was associated with male gender, a greater number of sex partners, partners living with HIV, and prior PEP use, suggesting the need to emphasize accurate reporting of PrEP use for certain groups of individuals ²². Accurate self-report would be beneficial in terms of clinical operations and resources, as more than one-third of patients on PrEP received time-intensive clinic PrEP navigation services.

Our findings may not be generalizable to other clinic settings or in places outside of NYC. Given that the primary objective of this study was to measure PrEP uptake, we used low cut-offs for TFV/FTC serum levels to determine PrEP use, which did not allow for inference about the consistent or daily use of PrEP regimens. Furthermore, while clinic staff collect information on PrEP use, there is no additional available information on prescription filling or frequency of PrEP use that could qualify results on discordance. For example, some patients reporting PrEP use may have suspended it temporarily yet still considered themselves PrEP users, while others may have had lower-level adherence resulting in PrEP being undetectable in the plasma via the assay. We could not rule out on-demand use of PrEP as reason for discordant findings; for example, patients may use PrEP as an event-based regimen yet consider themselves PrEP users ²³. We also could not exclude lab error as reasons for discordance. Finally, due to small numbers, our examination of patient characteristics associated with discordance combined two discordant subgroups, which could have masked heterogeneity in the correlates of discordance.

In summary, the use of self-report is an excellent indicator of PrEP uptake in our setting. Objective measures of PrEP use are often used in demonstration projects but are expensive for routine clinical use. Other jurisdictions with high HIV burden may benefit from conducting a similar study to examine the agreement between self-report and an objective measure of PrEP. As PrEP uptake is increasing in many cities ^24-26, such evidence can inform the continued reliance on self-report to monitor the effectiveness of efforts to both target and increase PrEP uptake.