Exploring Effects of Race and Differential Item Functioning on PrEP Stigma Experiences: Implications for Latent Stigma Measurement in Racially and Sexually Diverse Populations

Abstract

PrEP stigma measurement remains a challenge to the validity of studies and interventions addressing HIV prevention. It may lead to inaccurate assessment of the relationship between PrEP stigma and health outcomes such as PrEP persistence and care retention in groups experiencing HIV-related inequities. The present research explored the psychometric properties of a novel IV pre-exposure prophylaxis (PrEP) stigma scale in a cohort of racially diverse men who have sex with men (MSM). Using item response theory, analyses explored presence of differential item functioning (DIF) among Black and White respondents. Participants completed baseline surveys measuring psychosocial factors, sociodemographic factors, and PrEP stigma items. The primary analysis used a machine learning approach to assess (a) the presence of DIF; and (b) compare latent stigma between Black and White respondents, after correcting for any DIF. The model identified four out of 13 scale items as having a high probability of DIF for Black respondents, which is relatively good given that the original PrEP stigma scale was neither designed nor tested for validation comparing Black and White respondents. The DIF-adjusted latent PrEP stigma measure reveals statistically and substantially significantly higher levels of stigma for Black compared to White respondents (Diff.: 1.05 +/− 0.19). While most items performed well, findings demonstrate the importance of assessing measurement error in populations where stigma is rampant and being studied or intervened upon (and in this case, where multilevel and intersectional stigma may be present).

Introduction

Broadly defined, stigma is societal judgment and bias experienced by individuals who are minoritized on the basis of factors such as their race and/or sexual orientation [1–7]. Stigmatization can include individual-level experiences of internalized sexual stigma, interpersonal-level social stigma in one’s daily life due to having sex with men or being seen accessing preventive care, and/or healthcare stigmatization of Black patients by healthcare providers [1–7]. All forms of stigma, including the intersectional experience of them, are associated with poorer care satisfaction and inequitable access to preventive services, including uptake of HIV pre-exposure prophylaxis (PrEP) [1–3, 6–9]. African Americans continue to comprise half of all new HIV diagnoses in the US but only 12% of the US population [1, 10, 11]. Rates of HIV diagnoses are higher in African Americans communities as compared to all other racial groups, and this remains true within higher incidence groups such as men who have sex with men (MSM) [2, 12–14]. The ‘Ending the HIV Epidemic’ Plan set forth by the Department of Health and Human Services includes several goals in HIV preventive care for populations at heightened likelihood of HIV diagnosis, such as receipt of regular HIV testing and uptake of and persistence on PrEP [15]. Achieving any of these goals requires greater progress to be made in addressing stigma and better understanding of the needs of African American MSM who persistently have the highest rates of new HIV infections [2, 16–18].

Although African American MSM are more likely than other racial groups to experience socioeconomic disadvantage, this does not account for all barriers to positive HIV health outcomes they experience. As mentioned, stigma is a well-documented additional barrier to uptake of HIV preventive healthcare among MSM of color [1–4, 7–9, 19]. HIV stigma literature often describes the avoidance of healthcare by individuals living with HIV [20–23]. By comparison, the literature specific to PrEP stigma and HIV preventive care avoidance is relatively less robust. In other words, much of the literature on HIV stigma is focused on the experiences of individuals already diagnosed and living with HIV, as compared to individuals navigating stigmatizing experiences due to seeking HIV preventive care and/or taking PrEP specifically. Moreover, to our knowledge, nearly all studies describing PrEP stigma experienced by people who are HIV-negative as distinct from global HIV stigma (which can include those living with HIV) were published in roughly the last five years, although PrEP has been FDA-approved and widely available for over a decade and the HIV stigma literature extends far earlier than this timeframe [3, 6, 14, 24–26].

Like HIV stigma, PrEP stigma often includes shame experienced in relation to social interactions (e.g., being known as a PrEP user in social circles [14, 27–30] or seen accessing HIV preventive care [24–32]). Recent literature on PrEP stigma has identified that both injectable PrEP and HIV prevention care provided outside of traditional healthcare settings (such as virtually or in community-based settings) has the potential to meaningfully reduce experiences of PrEP stigma for MSM of color [11, 14, 24–32]. PrEP stigma reduction intervention literature also suggests that the ‘purview paradox’ remains a barrier to care engagement, i.e., many individuals who would benefit from PrEP seek care from providers who are not trained in HIV, and the providers that are trained in this area typically focus on clients already diagnosed with HIV [23]. Facilitators of PrEP uptake identified include PrEP-positive discourses which focus on promoting community awareness of psychosocial and physiological benefits of PrEP use including protecting one’s community [23, 25, 28–30].

Given persistent need for interventions focused on communities experience these disparities, gold standards on PrEP stigma measurement are still needed. Few studies have specifically explored internal and external validity of PrEP stigma measures among MSM of color populations [14, 24, 27–29, 31, 32]. Past quantitative measurements of PrEP stigma have typically included only individual-level belief statements, with each statement then assessed for association with PrEP-related outcomes such as willingness to take PrEP and without consideration of many of the multidimensional factors described above (e.g., social interaction concerns, structural barriers to care, intersectional experiences of stigma due to factors like racial minority identity and sexual minority identity [14]).

Challenges with multidimensional stigma measurement are not specific to HIV prevention research or MSM populations, yet both are direct threats to both internal and external validity of study design and outcomes. Instrumentation (measurement) error of multifaceted constructs like PrEP stigma are one of the major threats to internal validity [27, 32]. Further, presence of measurement error can cause both Type I and Type II errors when assessing associations between PrEP stigma and outcomes related to PrEP outcomes, thereby limiting the external validity of both an individual measure and the broader study results [27–29, 31, 32].

In the context of PrEP stigma research, most existing measurement tools including recently published PrEP stigma scales were not developed specifically with African American populations in mind. This is partly because African Americans are less likely to engage in research than other populations, due to factors including but not limited to less engagement in clinical settings where research recruitment typically occurs, and structural racism which results in care avoidance overall [18, 19]. Consequently, measures validated within dominant groups are likely to result in misspecification of the effects of PrEP stigma and its relationship on meaningful clinical outcomes like long-term PrEP persistence for all non-dominant groups, and in this case, for MSM of color [27–30, 32–34]. Robust assessment of stigma measures is vital to ensure the potential to mitigate effects of PrEP stigma on populations experiencing the greatest HIV burden and least benefits of PrEP, i.e., African American MSM [18–28, 31, 32].

The purpose of the present research was to explore the role of differential item functioning in our validated HIV pre-exposure prophylaxis stigma scale in a racially diverse sample of MSM to contribute further psychometric data on its utility as a PrEP stigma measure for populations in need of PrEP-specific research and interventions. We assess: (a) the presence of measurement error; and (b) differences in latent PrEP stigma between African American (heretoafter referred to as Black) and White respondents, after correcting for any DIF. While existing literature suggests that MSM of color experience greater levels of stigma and barriers to care [27–30, 32–34], previous studies have not been tested using our modeling approaches and with the HPSS. Therefore, we hypothesize non-zero effect differences between Black and White respondents, rather than assuming directionality of these differences.

Methods

Parent Study

The parent study is an ongoing randomized controlled trial to evaluate the feasibility, acceptability, and efficacy of a home-based monitoring system for MSM to initiate and maintain PrEP adherence. The intervention being tested seeks to make PrEP uptake and maintenance more manageable by reducing the total number of in-person healthcare visits required, which is a major deterrent among PrEP patients who are MSM of color and often healthcare-avoidant [18, 19]. The present analysis uses baseline data from study participants; initial study enrollment is complete, but an additional site for data collection is being onboarded so follow-up and study. closeout is ongoing.

Procedures

Participants self-completed screener questions via online platform or by phone if they. called to speak to study staff. Eligibility criteria include: (1) male sex assigned at birth, (2) age 18–49, (3) residence in the metro area of a study site, (4) self-reported HIV negative status, (5) ownership of a smartphone, (6) greater than 3 months since any previous PrEP use, (7) no simultaneous enrollment in another HIV prevention trial, (8) willingness to use study-provided PrEP navigation services to obtain PrEP, and (9) willingness to use a home test kit.

Primary recruitment methods were online recruitment strategies, venue-based recruitment, and flyers and visual advertisements. Links to the screener survey were advertised through banners or electronic messages on social media platforms such as Facebook, Grindr, or Instagram. After the onset of the COVID-19 global pandemic, inclusion criteria were modified to allow for individuals who were previously prescribed PrEP but not adherent to enroll (even if this exceeded 3 months of total which was the original inclusion criterion). After recruitment, a three-stage process was used to screen participants for eligibility. For Stage 1, a screener was used to determine study visit eligibility. Those eligible for a visit based on the screener results were shown the full study consent to review and sign which was Stage 2. Consented participants were also called (or met with via Zoom) to confirm consent and for further eligibility determination – participants were also given the chance to ask any questions or share concerns prior to confirming their willingness to participate and providing written consent.

After eligibility was confirmed via screener in Stage 1 and by study staff in Stage 2, participants completed baseline assessments via mobile app hosted on surveygizmo. com. Electronic methods for survey collection were used for privacy, and participants were invited to decline to answer any questions in the survey. The baseline survey assessed domains including demographics, healthcare, HIV/STI knowledge and testing history, PrEP knowledge and interest, sexual history including overall and detailed partner-level data, substance use history including injection drug use, and scientifically validated scales for depression, anxiety, resilience, self-efficacy, experience of racism, and stigma. Additionally, the survey included measures which participants were notified would be shared with providers once they were randomized into the study (e.g., sexual risk, illicit substance use risk, self-reported PrEP adherence, and any PrEP-related side effects from previous use and discontinuation). App metadata suggests that participants took between 25 and 40 min to complete the baseline assessment. Participants received $25 for their initial study visit and $25 for completing the baseline survey assessments.

After completion of the baseline survey, Stage 3 screening consisted of verification that HIV 4th generation and creatinine clearance tests indicate eligibility, and a PrEP prescription has been filled. Using the mobile app, participants were asked to use their smartphone to take a photo of their prescription label that identifies their name, date of prescription, and the medication name within two weeks of week of receiving the prescription. Once all three steps of eligibility were completed, participants were randomized to either standard in-person PrEP care or a home-based care intervention using test kits and virtual healthcare visits.

Home-based monitoring was conducted through the mobile app and a site coordinator at each study site coordinated visits for participants in both study arms. The study sought to enroll a sample diverse along the lines of age and race/ethnicity. Study sites are in five urban areas of the United States, with inclusion of multiple cities in the South due to their overrepresentation in new HIV cases (Atlanta GA, Boston MA, Jackson MS, and St. Louis MO; Cleveland OH was onboarded as a new site after the completion of our analyses). This study sample allows for exploration of stigma in a socio-demographically and racially diverse population. The parent grant and the present research were fully approved by the Emory University Institutional Review Board, with an IRB reliance agreement in place with FHI 360.

Measures

Besides the HPSS items, the following variables were measured: (a) psychosocial factors: generalized anxiety, depression, (b) PrEP intentions: preferences for home versus clinic-based care and intentions for planned length of time on PrEP. Socio-demographic characteristics of respondents include age, sex at birth, sexual orientation, region, educational attainment, race, Hispanic ethnicity, access to a regular healthcare provider, and relationship status.

Psychosocial Factors

The main scale used in our analyses, the HPSS is a robust statistical measure with items corresponding to three theoretically-informed domains of stigma (internal, anticipated, experienced). Unlike many previous stigma scales, the HPSS was assessed using both semantic differential and Likert formats which demonstrated (a) face validity; (b) high internal consistency; and (c) had unidimensionality despite covering multiple domains of PrEP stigma [14]. Within each domain, at least one item corresponds to key constructs of stigma: personal shame, judgment of people on PrEP, and perceived social support for taking PrEP [14, 27–30]. The constructs were identified a priori via (a) literature review; and (b) hand search of measures from both the HIV Prevention Trials and Adolescent Trials Networks given the focus on younger MSM [14]. As shown in Table 2, the full HPSS is a thirteen-item scale, with questions such as “I would have sex with someone who is taking PrEP,” “People taking PrEP receive praise for being responsible,” and “I would feel proud to take PrEP every day [14].” Items were presented on a 5-point Likert scale ranging from “Strongly disagree” to “Strongly agree.” Positively rated items 4, 6, 7, 12, and 13 were reverse-scored in order to orient all items in the direction of higher levels of stigma. Previously, the HPSS was validated with a study population of 279 MSM using both a Likert scale and sematic differential format, including randomized order of the scales that participants completed. Both scales demonstrated high internal consistency (Cronbach’s a Semantic Differential: 0.88; a Likert 0.82). However, the Likert scale was selected for better construct validity and is the version included in the present analyses; additional details on the HPSS are published elsewhere [14].

Table 2.

HIV PrEP Stigma Scale Mean Responses Noting Differences Comparing Black and White PrEP@Home Enrollees (Cisgender Men who have Sex with Men Ages 18–49 from 3 Regions in the United States)

Individual items	Black (N = 130) Mean (Standard Deviation)	White (N = 215) Mean (Standard Deviation)	Difference of meansa (standard error) [95% confidence interval]
I would feel ashamed to take PrEP pills in front of others.	1.51 (0.9)	1.84 (1.0)	−0.33 (0.11) [−0.54, −0.16]
Someone taking PrEP should keep their pills hidden.	1.76 (1.1)	1.97 (1.0)	−0.22 (0.11) [−0.44, 0.01]
People experience negative judgment because they take PrEP.	2.28 (1.1)	2.66 (1.1)	−0.37 (0.13) [−0.62, −0.13]
I would have sex with someone who is taking PrEP.b	1.47 (0.8)	1.34 (0.6)	0.12 (0.08) [−0.03, 0.28]
Someone taking PrEP would be seen by others as slutty	1.84 (1.0)	2.20 (1.1)	−0.37 (0.12) [−0.06, −0.14]
People taking PrEP receive praise for being responsible.b	2.23 (1.0)	2.10 (0.9)	0.13 (0.10) [−0.08, 0.33]
My friends would be supportive of me taking PrEP. b	1.43 (0.7)	1.56 (0.8)	−0.13 (0.08) [−0.29, 0.04]
Someone taking PrEP would be treated unfairly by their doctors.	1.72 (0.9)	2.12 (0.9)	−0.40 (0.10) [−0.60, −0.21]
People experience problems when they tell their sex partner(s) they are taking PrEP.	2.05 (1.1)	2.04 (0.9)	0.01 (0.11) [−0.20, 0.22]
I would feel proud to take PrEP every day.b	1.58 (0.8)	1.88 (0.8)	−0.30 (0.09) [−0.48, −0.12]
People taking PrEP experience verbal harassment.	2.04 (1.1)	2.11 (0.8)	−0.07 (0.11) [−0.28, 0.14]
People on PrEP are taking care of their health.b	1.55 (0.8)	1.40 (0.6)	0.14 (0.07) [−0.01, 0.29]
My family would be supportive of me taking PrEP.b	1.85 (1.0)	2.54 (1.2)	−0.69 [−0.94, −0.44]

^aMean differences reported are calculated as the difference in mean responses between Black and White respondents. Standard errors are denoted in parentheses, and confidence intervals are noted in brackets

^bReverse-scored items (where lower scores indicate more stigma on the HIV PrEP Stigma Scale completed at baseline by all survey respondents)

Generalized anxiety was assessed with the Generalized Anxiety Disorder Screener (GAD), which has previously been validated with many populations in different clinical settings [35, 36]. The seven-item scale includes questions such as “feeling afraid as if something awful might happen over the last two weeks” and items are on a 4-point Likert scale ranging from “Not at all” to “Nearly every day.” Items were summed to create a scale where higher numbers indicate greater presence of generalized anxiety (cutoff of 10 or more indicates at least moderate symptoms) [35, 36]. Depression was assessed with the Patient Health Questionnaire (PHQ), a well-validated measure of depressive symptoms which has been tested in racially-diverse populations [37, 38]. Items include “feeling down, depressed or hopeless, over the last two weeks?” and items were on a 4-point Likert scale ranging from “Not at all” to “Nearly every day.” Items were summed to create a scale where higher numbers indicate greater presence of depressive symptoms [37, 38].

PrEP care preference was assessed with a question which asked “If a PrEP home care service had a yearly in-person visit with a doctor, self-administered HIV/STI testing from home, and video visits with a doctor as needed every 3 months, would you?” and responses were either 0 = no preference; 1 = prefer all 4 medical visits with an in-clinic provider, and 2 = prefer home-based care for 3 out of 4 visits every year. Planned time on PrEP was assessed via “How long do you think you will stay on PrEP?” with the option to enter a numeric number each for months and years.

Sociodemographic Factors

Sex was measured via, “What sex were you assigned at birth, on your original birth certificate?” where 1 = Male, 2 = Female. Gender was assessed via “What is your current gender identity?, with response options of 1 = Male, 2 = Female 3 = Trans woman, 4 = Trans man, 5 = Genderqueer, 6 = Gender nonconforming, 7 = Other, and 8 = Decline to answer. The decision to refer to sex in the present study as gender interchangeably is because over 90% of the sample were cisgender, identified as men, and also reported having sex with men. The small sample size (less than 10% of sample) of individuals who did not report being cisgender men who have sex with men were excluded from analyses. Next, age was assessed as a continuous variable, with “How old are you,” where participants reported their age in years. Sexual orientation was assessed via “What is your current sexual identity?” Responses were categorized where “1 = gay, 2 = bisexual, and 3 = queer or other identity. Education was assessed via “What is the highest level of education that you completed?” Race was assessed with “What race are you?, with answer choices of White, Black, Asian, American Indian, Multiracial, and Other” Due to small sample sizes of all other races (less than 13% of sample) and theoretical significance of assessing Black and White racial differences, only Black and White respondents were included in our analyses. Hispanic ethnicity was binary, where 0 = no, and 1 = yes. Participants were asked what best describes their employment status, and access to regular healthcare was assessed via “Do you have a primary health care provider that you see regularly?” where 0 = no, and 1 = yes. Region was categorized based on the state in which participants resided, and were categorized as 0 = Northeast, 1 = Midwest, and 3 = South.

Data Analyses

Descriptive Analyses

Univariate frequencies were generated for all variables on Black and White respondents from the baseline survey sample (N = 360). As shown in Table 1, bivariate analyses were conducted between race and all other variables. As previously noted, respondents who identified as any other race, and as other than MSM were excluded from adjusted analyses due to very small sample sizes. Psychosocial variables, i.e., generalized anxiety and depression were utilized as variables to assess any potential baseline differences between Black and White respondents in bivariate analyses, which may confound their experiences of latent PrEP stigma. These two variables were chosen for theoretically justified reasons, given that anxiety and depression are well-established correlates of all forms of stigma including among African American MSM populations [20–25, 39].

Table 1.

Demographic Characteristics for PrEP@ Home Participants Screened at Baseline with Bivariate Differences Denoted by Race

Characteristic	Black N (%) or Mean (SD) (N = 136)	White N (%) or Mean (SD) (N = 224)
Sexual orientation**
Gay	107 (79.9%)	196 (88.7%)
Bisexual	22 (16.4%)	20 (9.0%)
Queer or Other Identity	5 (3.7%)	5 (2.3%)
Hispanic ethnicity ***
Yes	7 (5.2%)	10 (4.5%)
No	129 (94.9%)	214 (95.5%)
Education**
High school diploma equivalent or less	30 (22.1%)	21 (9.4%)
Some college/Associates/Technical School	56 (41.2%)	73 (32.7%)
College degree or above	50 (36.8%)	129 (57.9%)
Employment ***
Part time employee	13 (9.9%)	9 (4.1%)
Full time employee	77 (58.3%)	145 (65.3%)
Part-time or full-time student	18 (13.6%)	59 (26.6%)
Unemployed	24 (18.2%)	9 (4.1%)
Region***
Northeast	6 (4.4%)	76 (34.0%)
South	101 (74.3%)	84 (37.7%)
Midwest	29 (21.3%)	63 (28.3%)
Regular healthcare provider***
Yes	43 (31.2%)	127 (57.5%)
No	92 (68.2%)	94 (42.5%)
Relationship status*
Single	81 (60.5%)	118 (53.6%)
Casually dating	32 (23.9%)	34 (15.5%)
Main partner	18 (13.4%)	43 (19.6%)
Civil union/domestic partnership/legally married	3 (2.2%)	25 (11.4%)
PrEP care preference
No preferences	19 (6.6%)	38 (17.0%
Prefer home-based PrEP care	88 (64.7%)	143 (63.8%)
Prefer medical visits in-person with provider	20 (14.7%)	24 (10.7%)
Generalized Anxiety Disorder symptoms (Range: 0–21) ǂ	6.8 (7.47)	6.3 (6.05)
Depressive symptoms (Range: 0–27)	5.8 (6.7)	4.8 (5.01)
Intended time on PrEP years (Range: 0–100) ǂ	15.2 (23.8)	14.7 (23.9)
Age (Range: 18–50)*	29.6 (6.14)	30.2 (8.46)

Respondents who identified as any other races than Black or White and as any gender other than cisgender were excluded from adjusted analyses due to small sample sizes Chi-square significant at:

^ǂp < .10

^*p < .05

^**p < .01

^***p < .001

Item Response Theory Analyses Adjusting for Differential Item Functioning and Model Selection

Item response theory (IRT) is a well-known means by which to statistically assess the relationships between a latent factor (PrEP stigma) and categorical items intended to measure it (the HPSS) [37, 40–43]. This paper employs a graded response model, an IRT formulation suitable for observed categorical indicators [44–47]. This IRT model can be seen as a factor analysis model for categorical variables [47]. In fact, we will use the language of factor models when discussing our model. Conceptually, a factor model is a system of regression equations linking the probability of responding positively to a set of HPSS items to an unobserved covariate – the latent variable representing stigma. The parameters of this measurement system are the coefficients on the latent variable (referred to as loadings and discrimination in the factor analysis and IRT literature, respectively) and the itemcategory intercepts (referred to as thresholds or discrimination parameters). Item intercepts take on special importance when assessing the problem of differential item functioning (DIF).

Assessment of the measurement relationship between a latent factor and its indicators is complicated because different groups (e.g., as defined by race) have different survey response propensities irrespective of their true values on the latent variable. In terms of the model discussed above, this amounts to a race-specific shift in item intercepts holding the latent variable constant. (A more technical discussion is provided in Appendix A).

DIF is illustrated in Fig. 1. Panel A shows a hypothetical example of the probability of a respondent with an average (i.e., 0) level latent PrEP stigma agreeing to an item.

Fig. 1.

Hypothetical illustration of differential item functioning (DIF) within an item response theory (IRT) model in all participants (Figure 1a) and comparing by Black and White race (Figure 1b). Panel (a) plots an item response curve. For any given item, the probability of endorsing PrEP stigma approaches 0 for those with very low levels of latent PrEP stigma, and the probability approaches 1 for those with very high levels. The gray lines indicate the probability of endorsing an item for respondents at the mean of the latent PrEP stigma distribution (where θ =0). Panel (b) illustrates that under the presence of DIF, the probability is lower for Black respondents indicating their lower likelihood of endorsing PrEP stigma items. Each curve shows a single item’s adjusted and unadjusted category response curve, i.e., the probability of endorsing ‘strongly agree’ with PrEP stigma scale items with and without DIF correction. PrEP@Home enrollees were cisgender men who have sex with men ages 18–49 from sites in 3 regions of the United States

Panel B plots item response curves to illustrate the presence of DIF using values approximated by the actual data set; this was accomplished in place of writing the full model equation. A white respondent with latent PrEP value 0 still agrees with a probability of 0.5. In contrast, a Black respondent with the same latent PrEP value only agrees with a probability of about 0.25. This shift of the response curve is due to the differential functioning (a race-specific upward shift of its item parameters) of this particular measurement item and our extended model captures this by introducing a DIF parameter δ. We include such a parameter for each of the 13 items and employ Stochastic Search Variable Selection methodology to select the items where DIF is present (${\delta }_j$ is non-zero) [48]. This approach is a Bayesian variable selection method that uses discrete indicator variables to select which item-specific DIF parameters should be included or excluded from the model [49]. Appendix A presents the model in detail.

Results

Descriptive Characteristics

Table 1 reports descriptive characteristics of PrEP@Home participants at baseline, and differences in sociodemographic and psychosocial factors by race (Black, White) without any corrections for DIF (N = 360). Skewness was evident in age, whereby nearly 80% of participants in the sample were under the age of 35 years; this distribution did not differ by race. White participants more frequently identified as homosexual or same gender-loving than Black respondents. White participants also more frequently endorsed having full-time employment more than either Black respondents, and more frequently reported access to a regular healthcare provider than Black participants (57.5% vs. 31.2%). Black participants endorsed marginally statistically significantly higher intended length of time on PrEP (15.2 vs. 14.7 years respectively). Black participants were also far more likely than White participants to reside in the South (74.3% vs. 37.7%, respectively; p <.001).

With regard to baseline differences in psychosocial characteristics, Table 1 also reports anxiety and depression scores comparing Black and White respondents. No statistically significant differences which would account for potential differences in latent PrEP stigma experiences. Black respondents did report slightly higher mean presence of depressive symptoms compared to White respondents (6.8 vs. 6.3 respectively) and generalized anxiety (6.8 to 6.3, respectively).

HPSS Bivariate Associations by Race

Table 2 reports HPSS item means and standard deviations as well as differences in means and their corresponding standard errors for Black and White respondents. The total scale mean score, obtained by summing item means, was slightly lower among Black participants than White participants (23.2 vs. 25.7, respectively; 95% Confidence Interval (CI): [− 3.94, − 1.06]. White participants endorsed greater stigma in: feeling ashamed of taking PrEP in front of others (1.84 vs. 1.51, respectively; difference 95% CI: [−0.54, −0.16]). White respondents also endorsed greater stigma with experiencing negative judgment due to taking PrEP (2.66 vs. 2.28, respectively; difference 95% CI [−0.62, −0.13]); being seen as slutty for taking PrEP (2.20 vs. 1.84; [−0.06, −0.14]; and being treated unfairly by doctors for taking PrEP (2.12 vs. 1.72; [−0.60, −0.21]). With reverse-scored items where lower scores indicate more stigma, Black participants endorsed greater PrEP stigma than White participants with: being proud to take PrEP every day (1.58 vs. 1.88; [−0.48, −0.12]); and having family support them taking PrEP (1.85 vs. 2.54; 95% CI [−0.94, −0.44]). White participants’ responses also endorse more experiences with stigma related to the item assessing if people who take PrEP are not seen as taking care of their health.

HPSS Parameter Estimates without and with DIF Correction

Table 3 shows parameter estimates of the DIF-IRT model. Difficulty parameter estimates show that the previously validated HPSS items capture latent PrEP stigma experiences reliably. All HPSS items were able to discriminate between respondents with different levels of latent PrEP stigma: the majority of discrimination parameter estimates are greater than one. Some parameter estimates, such as with ‘feeling ashamed’ or ‘experiencing harassment, were closer to two which indicates rather strong item discrimination. The last two columns in Table 3 present the estimated DIF parameters for each of the HPSS items. Column $\mathrm{P}\mathrm{r}\left({\omega }_j\right)$ lists the model-implied probability that HPSS item $j$ exhibits nonzero DIF (i.e., no difference in item functioning between Black and White respondents). The next column (${\delta }_j$) shows the size of the estimated DIF parameter; items with DIF probability greater than 0.5 are marked in boldface and are those most likely to exhibit DIF by race in items related to latent PrEP stigma. The results are encouraging for stigma measurement in a racially diverse population. Out of 13 scale items, four items exhibited presence of DIF for Black respondents: (1) ‘having sex with someone who is taking PrEP;’ (2) ‘people taking PrEP would receive praise for being responsible; (3) ‘people experience problems when they tell their sex partners they are taking PrEP;’ and (4) ‘people on PrEP are taking care of their health.’

Table 3.

Graded Response Model with Black Respondents Only (PrEP@Home Enrollees from Sites in 3 Regions of the United States were Cisgender Men who have Sex with Men Ages 18–49)

	Item-category intercepts			Loadings.	SSVS-DIFa
	${\mathrm{\tau }}_1$	${\mathrm{\tau }}_2$	${\mathrm{\tau }}_3$	${\beta }_j$	$\mathrm{Pr}({\omega }_j)$	${\delta }_j$
Feel ashamed	− 0.05 [ −0.44, 0.32]	1.91 [1.47, 2.41]	3.41 [2.78, 4.06]	2.06 [1.60, 2.61]	0.11	0.06 [ −0.08, 0.55]
Hiding pills	−0.45 [−0.73, −0.15]	0.89 [0.56, 1.18]	2.53 [2.08, 2.99]	1.06 [0.75, 1.38]	0.16	−0.07 [ −0.55, 0.08]
Negatively judged	−1.99 [ −2.39, −1.61]	−0.3 [ −0.64, 0.01]	1.25 [0.88, 1.61]	1.71 [1.33, 2.07]	0.04	0 [ −0.08, 0.07]
Would have sex with PrEP user*	0.98 [0.67, 1.30]	3.27 [2.75, 3.83]	4.02 [3.33, 4.75]	0.82 [0.52, 1.15]	1.00	−1.26 [−1.77,−0.69]
Seen as ‘slutty’ taking PrEP	−1.21 [ −1.56, −0.82]	0.89 [0.53, 1.27]	2.51 [2.05, 3.03]	1.9 [1.49, 2.35]	0.05	0.01 [ −0.08, 0.08]
Praised for taking PrEP*	−1.23 [ −1.56, −0.89]	1.1 [0.75, 1.41]	2.81 [2.35, 3.27]	1.06 [0.79, 1.36]	1.00	−1.29 [−1.72, −0.81]
Friends support	0.25 [ −0.11, 0.58]	2.66 [2.18, 3.15]	4.43 [3.60, 5.25]	1.57 [1.18, 2.00]	0.10	−0.04 [ −0.35, 0.11]
Treated unfairly	−1.12 [ −1.45, −0.81]	0.85 [0.51, 1.15]	3.1 [2.53, 3.62]	1.2 [0.88, 1.49]	0.05	0.02 [ −0.07, 0.08]
Partner problems*	−0.97 [ −1.30, −0.64]	1.1 [0.75, 1.44]	3.18 [2.61, 3.70]	1.23 [0.95, 1.56]	0.97	−0.86 [−1.47, −0.35]
Feel proud	−0.63 [ −0.97, −0.34]	1.54 [1.18, 1.95]	3.88 [3.18, 4.60]	1.41 [1.08, 1.79]	0.04	0.01 [ −0.07, 0.07]
Harassment	−1.5 [ −1.90, −1.08]	0.94 [0.58, 1.34]	3.26 [2.67, 3.82]	1.77 [1.38, 2.17]	0.27	−0.15 [ −0.75, 0.08]
Taking care*	0.63 [0.33, 0.95]	3.21 [2.70, 3.74]	4.36 [3.54, 5.15]	0.93 [0.60, 1.23]	1.00	−1.36 [−1.88, −0.85]
Family support	−1.32 [ −1.68, −0.98]	0.15 [ −0.17, 0.46]	1.64 [1.26, 2.01]	1.23 [0.93, 1.54]	0.21	0.1 [ −0.09, 0.69]

Graded response is an item response theoretical model that deals with ordered polytomous categories. Thresholds report the probability of a particular response category by the latent trait (e.g., latent PrEP stigma). Entries are estimates (posterior means) and 95% credible intervals are added in brackets

Bold values indicate the 4 items on the HIV PrEP Stigma Scale that showed presence of differentialitem functioning (DIF) in our item response theory (IRT) model

Items with an * denote presence of DIF in IRT models

^aStochastic search variable selection for Differential Item Functioning (SSVS-DIF) is a Bayesian variable selection method, the goal of which is to estimate a model to include only those predictors supported by data. This is achieved by adding a hierarchical prior to the model, where the relevance of a variable is assessed in each step of the sampling algorithm. This method is based on 10,000 Markov chain Monte Carlo samples. Column Pr(ω) lists DIF parameter inclusion probabilities estimated using stochastic search variable selection. Columnδlists DIF parameter estimates

There is evidence of DIF by race on these four items of the HPSS because (a) their associated DIF probability is 1; and (b) the DIF magnitude estimates are large and statistically significantly different from zero. Figure 2 shows the impact of DIF among Black respondents for these four items graphically, by plotting the predicted probability of a Black respondent choosing the “strongly agree” category as a function of latent PrEP stigma. Some of the HPSS items, specifically users being ‘negatively judged because they take PrEP’ were at the lower end of the distribution of latent PrEP stigma, while items such as ‘having sex with someone who is taking PrEP’ and ‘family would be supportive of me taking PrEP’ and ‘friends would be supportive of me taking PrEP’ were all on the higher end of the latent PrEP stigma distribution.

Fig. 2.

Item category response curves for each latent PrEP stigma item with presence of diferential item functioning (DIF) and measurement bias and corrected for DIF among Black respondents at baseline. PrEP@Home Enrollees from Sites in 3 Regions of the United States Were Cisgender Men Who Have Sex with Men Ages 18–49. Each curve shows a single item’s adjusted and unadjusted category response curve, i.e., the probability of endorsing ‘strongly agree’ with PrEP stigma scale items with and without DIF correction

Figure 3 shows the distribution of latent PrEP stigma after correcting for DIF in the four items noted above (in Table 3; Fig. 2). The plot reveals substantial heterogeneity among respondents, with latent PrEP stigma spanning more than four standard deviations. There are clear differences between individuals experiencing lower and higher amounts of PrEP stigma. The upper end of the stigma distribution is observed to be predominantly populated by Black respondents. The stigma distribution for Black respondents is shifted to the right, with mean latent stigma for Black respondents being about one standard deviation higher; the exact difference is estimated as $\alpha =1.05(\pm 0.19)$; cf. (Table 3). In Appendix B we confirm that this finding also holds when accounting for sample differences in region, respondents’ education and sexual orientation.

Fig. 3.

Distribution of experiences of latent PrEP stigma after being corrected differential item functioning (DIF), and comparing Black and White respondents (PrEP@Home Enrollees from Sites in 3 Regions of the United States Were Cisgender Men Who Have Sex with Men Ages 18–49). Kernel density plots using a Gaussian kernel with bandwidth at 0.3

Hypothesis Testing (Identifying High Levels of Latent PrEP Stigma)

We illustrate the consequences of ignoring DIF for the classification of responses by race. We compute (latent) stigma measures based on (i) our preferred DIF-corrected IRT model, (ii) a standard IRT model, and (iii) a simple item summed score. We then classify respondents as high (low) stigma if they have scores above the 75th percentile (or below the 25th) of the respective stigma measure. Table 4 shows the cross-tabulation of (i) with (ii) and (iii). Comparing the DIF-corrected with the standard IRT classification, we find that 84 (67 + 17) percent of respondents are consistently classified as having either low or high stigma, while 16% of respondents are inconsistently classified (e.g., classified as low stigma despite the DIF-corrected measure indicating high) stigma. If we are willing to assume that the DIF-corrected model accurately represents the social process that generated the data, employing a standard IRT model misclassifies 16% (95% CI: [15–23%]) of respondents. This misclassification is even more pronounced when examining a simple summed score measure. Here, only 68% are consistently classified, leading to a misclassification rate of 27% (CI: [21–33%]). Therefore, our hypothesis was supported – i.e., there was presence of significant non-zero differences in the effects of latent PrEP stigma between Black and White MSM in our study sample (and presence of DIF).

Table 4.

Proportion of Black Respondents Classified as Having Low Vs High Latent PrEP Stigma With and Without Correcting for Measurement Bias (PrEP@Home Enrollees from Sites in 3 Regions of the United States were Cisgender Men who have Sex with Men Ages 18–49)

		Standard IRTa		Sum score
		Low	High	Low	High
DIF-corrected	Low	0.67	0.08	0.56	0.16
IRT analyses	High	0.08	0.17	0.11	0.12
Proportion differentially classified		0.16 [CI: 0.11, 0.23]		0.27 [CI: 0.21, 0.33]

^aItem response theory analyses (without correction for differential item functioning [DIF])

Discussion

The purpose of the present research was to explore the impact of differential item functioning on aa newly developed scale to capture PrEP stigma among men who have sex with men in the PrEP@Home study. In descriptive analyses, we noted differences in dimensions of PrEP stigma between Black and White MSM study participants which were substantial. While 9 of the 13 scale items did not have this issue, the differences in the DIF-correct values are impactful given the need for tailored interventions and research focused on Black MSM experiencing HIV inequities. After accounting for DIF, race differences were present in experiences of levels of latent PrEP stigma. However, contrary to prior research, we found greater levels among White MSM. Further, we assessed known correlates of latent PrEP stigma in descriptive analyses, i.e., depression and anxiety, and found no statistically significant differences between Black and White MSM which would explain these differences in latent PrEP stigma experiences. We conclude, therefore, that these are true differences not subject to confounding and that our hypothesis of non-zero racial differences was supported.

Given the recent emphasis on stigma measurement and reduction in this population, this difference is highly impactful since PrEP stigma is usually conceptualized as a driver of HIV prevention outcomes, rather than as an outcome itself. Golub notes that PrEP stigma directly impacts the pace of PrEP uptake and its dissemination among highest priority populations including Black men who have sex with men [28]. She also notes that in both clinical trials and demonstration projects, PrEP stigma has emerged as a critical factor in users’ experiences, including their adherence and persistence behavior, and PrEP users have explicitly linked being seen taking pills by friends and family as a deterrent to staying on the medication [28]. PrEP users being seen as sexually irresponsible, promiscuous, and immoral manifest at multiple levels and may discourage PrEP interest and uptake, disrupt PrEP adherence, and motivate PrEP discontinuation. Even looking elsewhere along the HIV care continuum, Rao and colleagues found substantial evidence that Black respondents living with HIV had a higher probability of indicating greater stigmatization on items that described situations in which others discriminated against them compared to White respondents [39].

Our hypotheses of race differences in endorsement of stigma was supported, though previous literature would have suggested greater experiences of latent PrEP stigma among Black MSM (while our hypothesis was presence of non-zero differences). White MSM reported greater endorsement of PrEP stigma in comparison to Black MSM for several dimensions of PrEP stigma, including taking PrEP in front of others, and being treated unfairly by doctors for taking PrEP. It is possible that White MSM may experience greater than expected experiences of PrEP stigma because they are more likely than Black MSM to know about PrEP, have discussed PrEP with a provider, and/or have already taken PrEP. Existing literature shows White MSM are much more likely than nearly all other groups to receive PrEP counseling, obtain a PrEP prescriptions, initiate PrEP initiation, and persist on PrEP persistence at 12 months [11–13, 50].

Receipt of more consistent and frequent PrEP care may translate to greater endorsement of stigma for this population. It is possible that White MSM may experiences stigma related to HIV prevention given the over-representation of people of color in new HIV cases which may create the perception that HIV isn’t something they ‘should’ have to worry about. White MSM reported greater endorsement of PrEP stigma in comparison to Black MSM for several dimensions of PrEP stigma, including taking PrEP in front of others and being treated unfairly by doctors for taking PrEP. It is possible that White MSM may experience greater than expected experiences of PrEP stigma because they are more likely than Black MSM to know about PrEP, have discussed PrEP with a provider, and/or have already taken PrEP. Existing literature shows White MSM are much more likely than nearly all other groups to receive PrEP counseling, obtain a PrEP prescription, initiate PrEP, and persist on PrEP at 12 months [11–13, 50]. Receipt of more consistent and frequent PrEP care may translate to greater endorsement of stigma for this population; nonetheless, Black MSM may be experiencing greater global stigma across multiple domains which was not captured in the present analyses and is necessary for future research. We also note that Black respondents were far more statistically likely to reside in the South where (a) the greatest number of HIV cases are diagnosed; and (b) greater disparities in access to care have been documented since the beginning of the HIV pandemic. Future analyses will need to better tease apart the relationship between geography, race, access to care and other social determinants as predictive of longitudinal PrEP outcomes.

Our findings underscore the need for renewed examination of measurement bias and DIF by race when using previously validated, “gold standard” measures. The importance of rigorous assessment of measurement bias cannot be overstated or removed from stigma-related HIV prevention. Systematic underestimation of stigma, in our case PrEP stigma, among individuals who are minoritized by race, due to measurement error, severely inhibits the potential for future research and interventions to make progress in ending the HIV epidemic inequities these communities experience [51, 52].

Limitations

This study is not without limitations. First, these data are cross-sectional baseline data since the PrEP@Home study is still ongoing. As a result, there may be inherent differences between the sample included in this research and the full sample that may be recruited at completion of the study; characteristics such as volunteer bias and willingness to initiate PrEP may differ, along with views and needs related to healthcare in light of the COVID-19 global pandemic. Next, the present analyses explored a limited number of sociodemographic and psychosocial variables to understand the role of latent PrEP stigma. More extensive analyses might uncover different associations by accounting for other relevant factors such as norms related to HIV risk, measurement of intersectional stigma, and/or further exploration of exposure to PrEP information in participants’ social networks. Next, differential item functioning was present, though not common, in the present sample. Considerations of DIF and presence of inherent racial differences as tested in our hypotheses may be different in the context of longitudinal analyses; therefore, our cross-sectional findings must be interpreted with caution. Lastly, the study sample was generally educated, had access to healthcare, was cisgender, self-identified as gay and were mostly located in large cities. Findings would likely differ in and cannot be generalized to rural communities, nonbinary adults, individuals of other races, and/or individuals with limited access to healthcare. We also note that disparities in care would potentially be greater in communities with additional marginalized identities. However, as noted in our methods and table footnotes, the present sample did not have enough respondents who identified as (a) multiracial or as a race other than white or black; and/or (b) as gender minorities to assess such differences. Future analyses beyond the scope of this research would need to further impact racial differences by region and access to community resources beyond socio-demographic variables included in the present analyses.

Conclusions

Our findings suggest that the HPSS is a psychometrically valid scale for exploring a latent PrEP stigma factor among a racially diverse sample of MSM despite some minimal presence of DIF. Out of 13 scale items, only four show evidence of DIF for Black respondents. We present a model that identifies the presence of DIF for specific items while simultaneously correcting for it when estimating latent stigma. Assessing and correcting for DIF is important to evaluate in future studies with larger samples, given the ubiquity of PrEP stigma measurement in HIV prevention research among communities experiencing the HIV inequities and its known impact on key HIV prevention outcomes like care retention and care satisfaction. Future analyses will also assess the causal effects of race on latent PrEP stigma and health outcomes, which was beyond the scope of this work. Future interventions informed by our work should not only utilize this measure but expect differences in the experiences of MSM of color regarding need for stigma reduction interventions given that present analyses accounted for race and PrEP stigma, though experiences of intersectional stigma and marginalization that MSM of color may experiences differently than their White counterparts. The importance of assessing DIF and stigma measurement error cannot be overstated when working in racially and sexually diverse populations that are the most likely to be stigmatized, experience health inequities, and least likely to have validated measures that can be utilized in addressing their needs.

Appendix A

A standard IRT model for categorical responses is defined as: [46, 53, 54]

$$Pr\left(y_{ij}\ge k\mid {\mathrm{\theta }}_i\right)=\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left({\mathrm{\eta }}_{ijk}\right)}{\mathrm{e}\mathrm{x}\mathrm{p}\left({\mathrm{\eta }}_{ijk}\right)},{\mathrm{\theta }}_i~N(0,1)$$

where $y_{ij}$ denotes the response of individual $i$ to item $j$ and $k$ represents the respective category (“agree”, “strongly agree”). The latent variable ${\mathrm{\theta }}_{\mathrm{I}}$ captures a respondent’s level of PrEP stigma. It is normalized to have zero mean and unit variance. The probability of a respondent choosing a certain category is a function of the linear predictor ${\mathrm{\eta }}_{ijk}$ given by

$${\mathrm{\eta }}_{ijk}={\mathrm{\tau }}_{jk}-{\mathrm{\beta }}_j{\mathrm{\theta }}_i$$

where ${\mathrm{\beta }}_j$ is the discrimination of item $j$ and ${\mathrm{\tau }}_{jk}$ is the kth threshold for item $j$. To account for potential DIF, the linear predictor term is extended following Tutz and Schauberger to include a DIF-causing group level variable (race) $z_i$ in the following form: [44, 55, 56]

$${\mathrm{\eta }}_{ijk}=\left({\mathrm{\tau }}_{jk}+\mathrm{\alpha }{z}_i+{\delta }_j{z}_i\right)-{\mathrm{\beta }}_j{\mathrm{\theta }}_i$$

Racial differences enter the linear predictor in two ways [56]. First, as an item-specific shift of response probabilities, captured by the set of ${\mathrm{\delta }}_j$ coefficients. The larger ${\delta }_j$, the larger the difference in the probability of responding positively to an HPSS item for respondents of a different race but with identical levels of PrEP stigma. The ${\delta }_j$ coefficients allow detection of the presence of DIF, but they can also be used to produce estimates of $\mathrm{\theta }$ that are “purged” of DIF-related measurement error. Second, race enters as constant response differences between racial groups. This difference, captured by $\alpha$, simply represents the fact that some groups experience higher levels of stigma (and is unrelated to measurement problems to DIF).

An IRT model with all ${\delta }_j$ coefficients included is not identified. Existing strategies often include items one-by-one, by subsets, or rely on a priori judgments of items suspected of DIF. A more principled approach is to use machine learning techniques to separate items exhibiting DIF from those who do not; one such approach is using the LASSO to select DIF items [48] This requires inspection of LASSO paths to determine the presence of DIF for a given item. This paper uses the stochastic search variable selection methodology [48, 49]. to identify DIF items by replacing the term ${\delta }_j{z}_i$ with ${\omega }_j{\delta }_j{z}_i$ where ${\omega }_j$ is a binary selection indicator equal to one if DIF is present (${\delta }_j$ is nonzero) and zero otherwise. Estimating the model using Markov Chain Monte Carlo sampling stochastically searches for relevant subset of non-zero DIF parameters and provides an estimate of the probability of the presence of DIF, $P\left({\omega }_j=1\right)$, for each HPSS item together with estimates of all other IRT parameters.¹

Appendix B

In this appendix we explore if our DIF estimates change substantially when adjusting for key covariates in our sample including education, sexual orientation, and region.

We extend the item response linear predictor to include a vector of covariates, $x_i$ :

$${\mathrm{\eta }}_{ijk}=\left({\mathrm{\tau }}_{jk}+\mathrm{\alpha }{z}_i+{\delta }_j{z}_i+{\gamma }^{\prime }{x}_i\right)-{\mathrm{\beta }}_j{\mathrm{\theta }}_i$$

Panel a of Figure 4 plots the resulting DIF estimates (for those items where the SVSS DIF probability is greater than 0.6) comparing unadjusted and covariate adjusted estimates. We find that both estimates are rather close. Adjusting for covariates shortens the confidence interval for the “partner problems” item. Given the similarity of DIF estimates, it is not surprising that panel (b), which compares kernel density estimates of latent PrEP stigma, reveals no substantive differences. Thus, no revisions to the conclusion of substantial differential item functioning for Black respondents need to be made.

Fig. 4.

Comparison of DIF parameter estimates (4a) and distribution of latent PrEP stigma (4b) when adjusting for additional covariates (sexual orientation, region, education) in the item response function. PrEP@Home baseline enrollees were from sites in 3 regions of the United States who were cisgender men who have sex with men ages 18–49. Sexual orientation was captured via two indicator variables for bisexual, queer or other (with gay as baseline); region are South and Northwest indicators (Midwest as baseline), education is an indicator for a college or further advanced degree. Priors for covariates are standard normal