Abstract
Antiretroviral pre-exposure prophylaxis (PrEP) is highly effective in preventing HIV. Despite its promise, PrEP use is low, especially among young Black men who have sex with men (YBMSM). The prevalence of HIV in Mississippi (MS) is among the highest in the United States, with the bulk of new infections occurring amongst YBMSM living in Jackson, MS. We recruited 20 PrEP-eligible YBMSM and 10 clinic staff from MS health clinics between October 2021 and April 2022. Data were collected remotely using in-depth interviews and a brief survey, which lasted approximately 45–60 min. Interview content included PrEP knowledge/experiences, HIV risk perception, and PrEP use barriers and facilitators. Qualitative data were coded then organized using NVivo. Using thematic analysis methodology, data were assessed for current barriers to PrEP use. An array of barriers were identified by participants. Barriers included structural factors (cost of PrEP, lack of discreet clinics, time commitment, competing interests); social factors (unaware of HIV risk, stigma and homophobia, fear that partners would find out about PrEP use, not knowing anyone on PrEP); behavioral factors (sexual risk factors, denial, less priority for prevention vs treatment); and clinical factors (misunderstood side effects, fear PrEP won’t work). Significant barriers to PrEP use among YBMSM stem from structural, social, behavioral, and clinical factors. These results will inform intervention efforts tailored to mitigate barriers and improve PrEP uptake among YBMSM in the southern United States.
Keywords: HIV prevention, Pre-exposure prophylaxis (PrEP), Black men who have sex with men (BMSM), Sexual orientation
Introduction
In 2020, the South had more diagnoses of HIV infection than any other region with the majority of diagnoses among men who have sex with men (MSM) (Centers for Disease Control and Prevention [CDC], 2022). A large percentage of those diagnoses were among young Black MSM (YBMSM) (CDC, 2022). The “Ending the HIV Epidemic: A Plan for America” initiative to end HIV prioritized six Southern states, including Mississippi (MS) (Fauci et al., 2019; Rawlings, 2021). Historically, Jackson, MS has had the highest rate of HIV infections among urban MSM (Rosenberg et al., 2016), with the majority among YBMSM (Chan et al., 2017; State of Mississippi Department of Health, 2015). Pre-exposure prophylaxis (PrEP) is a highly effective medication used to prevent HIV (Grant et al., 2010; Marcus et al., 2016). Although PrEP was approved over ten years ago by the United States (U.S.) Food and Drug Administration, uptake has been suboptimal among populations who would benefit. For example, 42% of new HIV diagnoses in 2020 were among Black people; yet in 2021, only 14% of PrEP users were Black (AIDSVu, 2022). In the South, Black individuals accounted for 52% of incident HIV diagnoses, but only 21% of PrEP users (AIDSVu, 2022). Mississippi is among the states with the greatest unmet need for PrEP among Black people (AIDSVu, 2022), including YBMSM in Jackson, MS (Bush et al., 2016; CDC, 2018). Given the high rates of HIV among YBMSM in Jackson, prevention efforts such as PrEP are essential to ending the HIV epidemic.
Barriers to PrEP use among MSM have been widely studied and include stigma, access to PrEP care, attitudes and beliefs about PrEP, internalized homonegativity, insufficient knowledge of PrEP, patient-provider relationship, and perceived side effects (Edeza et al., 2021; Hannaford et al., 2018; Matacotta et al., 2020). Systematic reviews specific to BMSM found that perception of HIV risk, PrEP and HIV stigma, cost, anticipated side effects, and medical mistrust were the top barriers to PrEP use (Ezennia et al., 2019; Russ et al., 2021). In 2016, a qualitative study in Jackson, MS evaluated barriers to PrEP persistence among a sample of YBMSM (Arnold et al., 2017). This was one of the first studies to evaluate barriers to PrEP use among YBMSM in MS. Results indicated the barriers to PrEP use among this population included structural factors (cost and access to financial assistance and medical appointments), social factors (stigma and relationship status), behavioral factors (sexual risk behaviors), and clinical factors (perceived and actual side effects).
The goal of this study was to assess changes in the barriers to PrEP use among YBMSM living in Mississippi since the study completed in 2016 (Arnold et al., 2017). We completed in-depth interviews with YBMSM and clinic staff to assess barriers to PrEP uptake, adherence, and persistence. Although the initial study did not recruit clinical staff, clinic staff were recruited for the current study to add breadth to the newly collected data and assess for additional barriers. The HIV epidemic among YBMSM continues to be a public health crisis, and effective prevention efforts are urgently needed. Knowing if, and how, these barriers to PrEP use have changed will help with both designing novel interventions and selecting implementation strategies to address these barriers among this population.
Method
Participants
This study consisted of in-depth interviews and brief surveys with 20 PrEP-eligible YBMSM and 10 clinic staff. To be eligible to participate, YBMSM had to be: (1) English speaking, (2) 18–34 years old, (3) assigned male at birth, (4) African American/Black, (5) not enrolled in another PrEP related study, (6) not taken PrEP in past three months, and (7) PrEP-eligible according to CDC guidelines. To be eligible to participate, clinic staff had to be: (1) English speaking and (2) employed by clinics serving YBMSM in Mississippi. Clinic staff included research assistants, nurses, PrEP prescribers, and PrEP navigators.
Measures and Procedure
Study interviews (approximately 45–60 min) were completed between October 2021 and April 2022. Participants were recruited through word-of-mouth with YBMSM being informed about the study from clinic staff during clinic visits and clinic staff learning about the study from research staff. Those interested were screened for eligibility. Participants completed informed consent via the HIPAA-compliant, electronic signature software, DocuSign. Upon enrollment, research staff emailed participants a brief survey using the HIPAA compliant, online data collection and management platform, REDCap (Harris et al., 2009). The survey took approximately ten minutes to complete and gathered information pertaining to demographics and PrEP concerns. All participants received a $30 gift card.
All interviews occurred remotely via Zoom. Interview questions were open-ended and informed by existing research on PrEP barriers and barrier domains. Interview content included: PrEP knowledge/experiences, HIV risk perception, and PrEP use barriers and facilitators. Participants were probed to discuss various barriers to PrEP initiation and persistence. Throughout data collection, interview guides were edited and adapted to incorporate novel and/or unanticipated topics. Interviews were completed until data saturation occurred and no new content related to PrEP barriers or facilitators surfaced. Please refer to sample interview questions below.
Patient Participant Specific Questions
“What made you decide to take PrEP?”
“What makes it easier or harder for you to take PrEP?”
“How at risk for HIV do you think you are?” “Why?”
“What events could happen, or have happened, that made you feel at risk for HIV?”
“Why do you think some men do not take PrEP even though they know it could be useful?
“How do partners, family, friends, or doctors influence your decision to take PrEP or stay on PrEP?”
“What resources do or did you need to get PrEP?” (payment assistance, transportation, PrEP navigator)
“What is the number 1 reason you would not get on PrEP?”
Clinic Staff Participant Specific Questions
“What kind of agency do you work for?”
“Can you describe your job description and in what capacity you work with BMSM?”
“Do you think Black men in Mississippi worry about HIV?”
“Why do you think some Black men do not take PrEP even though they know it could be useful?
“What do you think makes it harder for men in MS to take PrEP?”
“How do partners, family, friends, or doctors influence the men you work with to take PrEP or stay on PrEP?”
“What resources do the men you work with need to get PrEP?” (payment assistance, transportation, PrEP navigator).
“What is the number 1 reason the men you work with do not get on PrEP?”
Data Analysis
All interviews were audio-recorded then transcribed by an external, HIPAA-certified transcription company. Transcripts were reviewed for accuracy. A coding scheme was created a priori based on the interview guide and existing research on PrEP barriers. Members of the research team used the scheme to code data within the transcripts noting which code(s) were present for each area of the text. To ensure consistency, 50% of the transcripts were coded by two people, independently. Discrepancies in coding decisions were discussed and resolved. When a piece of text represented a new, important idea, the researchers added it as an inductive code into the coding scheme. Once all transcripts were coded, the data were organized using NVivo software (QSR International Pty Ltd., 2018) and then deductively analyzed using reflexive thematic analysis: familiarization with data, generating codes, constructing themes, and reviewing, defining, and naming themes (Braun et al., 2019).
Analyses of survey data occurred using SPSS software, Version 28.0 (IBM SPSS Statistics for Windows, 2021). Descriptive statistics including count data and frequencies were calculated for demographic and behavioral variables.
Results
Sample Characteristics
Table 1 provides demographics for all participants (N = 30) including YBMSM patients (n = 20) and clinic staff (n = 10). Among YBMSM (n = 20), 80% had at least some post-secondary education (with 25% being active students), 90% were employed (60% full-time and 30% part-time), 50% reported an annual income less than $30,000, and 60% had health insurance. The majority of patient participants were not presently in a relationship (85%), identified as Baptist (65%), and lived on their own (65%). Most (70%) had experience taking PrEP.
Table 1.
Patient and staff demographic variables
Patient demographics (N = 20) | Staff demographics (N = 10) | ||
---|---|---|---|
Education | # of participants | Race | # of participants |
High school diploma | 4 (20%) | Black | 7 (70%) |
Some college | 11 (55%) | White | 2 (20%) |
College degree | 5 (25%) | Mixed race | 1 (10%) |
Currently a student | Education | ||
Yes | 5 (25%) | Some college | 4 (40%) |
Employment status | Bachelors degree | 2 (20%) | |
Full-time | 12 (60%) | Graduate degree | 4 (40%) |
Part-time | 6 (30%) | Sexual Orientation | |
Unemployed | 2 (10%) | Heterosexual | 6 (60%) |
Religious affiliation | Bisexual | 2 (20%) | |
Baptist | 13 (65%) | Homosexual | 2 (20%) |
Other | 7 (35%) | Years working providing care to MSM | |
Annual income | Less than a year | 4 (40%) | |
Less than $12,000 | 2 (10%) | 1–2 years | 1 (10%) |
$12,000 to $29,999 | 8 (40%) | 3–5 years | 2 (20%) |
$30,000 to $59,999 | 7 (35%) | 6–9 years | 1 (10%) |
$60,000 to $99,999 | 3 (15%) | 10 years or more | 2 (20%) |
Relationship status | Hours per day of interaction with YBMSM | ||
Single | 17 (85%) | 1–2 h | 1 (10%) |
In a Relationship | 3 (15%) | 3–4 h | 3 (30%) |
Living situation | 5–8 h | 6 (60%) | |
Your own house or apartment | 13 (65%) | Years of PrEP knowledge | |
At a family member’s house or apartment | 6 (30%) | 0–2 years | 2 (20%) |
Other | 1 (5%) | 3–5 years | 6 (60%) |
Health insurance status | 6–10 years | 2 (20%) | |
No | 8 (40%) | Years working in a PrEP setting | |
Yes | 12 (60%) | Less than a year | 4 (40%) |
Taken PrEP | 1–2 years | 1 (10%) | |
Yes | 14 (70%) | 3–5 years | 2 (20%) |
No | 6 (30%) | 6–10 years | 2 (20%) |
Among clinic staff participants (n = 10), 70% identified as Black, all had at least some post-secondary education (60% holding a college degree), and 40% were LGBTQ + . Clinic staff experience working directly with YBMSM varied from less than a year to over ten years.
Participants were asked about their concerns regarding PrEP and indicated either “yes-it is a concern” or “no-it is not a concern” for the following: side effects, cost, taking a medication daily, interactions with drugs/alcohol, prescription interactions, fear people will think they are living with HIV. Side effects, cost, and taking a daily medication were all selected as concerns by over half the clinic staff and patients. Staff rated fear of people thinking the patient is living with HIV more frequently (7/10 selecting yes) than patients (2/20 selecting yes).
Themes Related to Barriers
The resulting thematic categories related to PrEP barriers included (1) structural factors, (2) social factors, (3) behavioral factors, (4) clinical factors. There were no significant differences in themes among patient or staff participants. Table 2 provides an overview of the themes related to barriers to PrEP initiation and persistence compared to the themes reported in 2016 (Arnold et al., 2017). Table 3 illustrates themes with representative quotes.
Table 2.
Barriers to PrEP initiation and Persistence
Broad factors | Current study themes | Themes reported in 2016 (Arnold et al., 2017) |
---|---|---|
Structural factors | Cost (being told PrEP was free and then having to pay a fee) Lack of discrete clinics Time commitment (for medical appointments and picking up medication) Competing interests / Not wanting to take a daily medication |
Access to payment assistance programs Copayments and deductibles for medications and related services |
Social factors | Not knowing anyone personally taking PrEP (Needing to “mentally prepare”) Unaware of HIV risk HIV stigma and homophobia (assumption of promiscuity, not wanting to be labeled gay, fear people think they are living with HIV) Fear partners will find out they take PrEP |
HIV stigma and homophobia Relationship status changes |
Behavioral factors | Sexual risk behaviors (only one sex partner) Denial (“it won’t happen to me”) Less priority for prevention vs. treatment |
Changes in sexual risk behaviors |
Clinical factors | Misunderstood side effects (Fear of dependency) Fear PrEP won’t work |
Perceived and actual medication side effects |
Table 3.
Barriers to PrEP initiation and persistence
Broad factors | Themes | Participant quotes |
---|---|---|
Structural factors | Cost (being told PrEP was free and then having to pay a fee) | “They told me PrEP would be free, and so I said, “Okay.” Once I got to the pharmacy, I had to pay for it. It was, like, $50. Having to pay for it was an issue for me.” – [23-year-old, currently not taking PrEP] |
Lack of discreet clinics | “The stigma cause we have an HIV clinic in the same location.”—[Nurse] “I think telehealth would be the best, because you don't have to worry about people who sees them comin' to a certain clinic, and, it could be more discrete if it's offered.” – [Nurse Practitioner] |
|
Time commitment (for medical appointments and picking up medication) | “Honestly, just, like, time. Time to go to the pharmacy to pick it up, time to go to the checkup, time to you know, anything. I just don't have time to do it, so I kinda just said, "Well, let me just stop having so much sex and stop having sex without condoms.”—[26-year-old, currently not taking PrEP] |
|
Competing Interests / Not Wanting to Take a Daily Medication | “I think it’s mostly because it’s something you have to take daily. And most men don’t want that responsibility.”—[34-year-old, currently taking PrEP] “Some people feel like their everyday lives are too busy to have to remember to take a pill.” – [PrEP Navigator] |
|
Social factors | Not Knowing Anyone Taking PrEP (Needing to “mentally prepare”) | “I didn’t really know anyone on PrEP. I had friends who were HIV positive. Even though I know some of the same medications are used, I still think it’s probably different.”—[24-year-old, currently not taking PrEP] “They're thinking about PrEP, and they'll get back with me at a later date.”- [PrEP Navigator] |
Unaware of HIV Risk | “From the people that I have been workin' with, especially those newly diagnosed, I don't think they were really worried about it prior because they are shocked. I don't think they realized the risk and how rampant the cases are in this area.” – [Licensed Master Social Worker] “A lot of it is the ones with the “I don’t need it because I’m in a relationship and so I don’t have to be on it.”—[Nurse] |
|
HIV Stigma and Homophobia (assumption of promiscuity, not wanting to be labeled gay, fear people think they are living with HIV) | “I think it's just not wanting to be labeled as gay or as someone who has sex with men.”—[26-year-old, currently not taking PrEP] “I think they relate PrEP to HIV. And, sometimes, no matter how much you tell’em that it actually prevents HIV, they’re like, ‘Ah, no. People will think I got it.’”- [Nurse] “They think that you're sexual promiscuous. I think that's where the message needs to be. It only takes one time to get HIV.”—[Licensed Master Social Worker] |
|
Fear Partners Will Find Out They Take PrEP | “They don’t want their partners to know that they’re takin’ PrEP. They don’t want their partners to believe they out there cheatin’ and that’s why they taking PrEP every day.”— [Nurse] “Because I’m married, and I don’t want my husband to think that I’m out there doin’ something.” – [Nurse] |
|
Behavioral factors | Sexual Risk Behaviors (only one sex partner) | “I've been in a relationship with my fiancé for nine years now. I just didn't feel the need for it to be for me because I know what my partner is doing, and he knows what I'm doing.”— [26-year-old, currently not taking PrEP] “I’m seeing only one person, and we are monogamous as well as that person goes and gets regular checkups and things of that nature. I’m not really too concerned about it.—[24-year-old, currently not taking PrEP] |
Denial (“it won’t happen to me”) | “I think some people, especially in their thirties, just think that ‘If I haven’t gotten it yet, I’m probably not gonna get it’ or ‘That can’t happen to me.’”—[34-year-old, currently not taking PrEP] “Most people, I guess, aren't so conscious of how bad HIV is. When it comes to STDs or HIV, people feel like, "Oh, it's not gonna happen to me, so I don't feel like I need to take that." And it just really all comes back to being self-conscious about your health.”—[26-year-old, currently not taking PrEP] “I think a lot of them are still in the mindset of, "It won't happen to me. "Even though they don't always use condoms and they can come back with Chlamydia or Gonorrhea, or Syphilis. I think it's more so the denial.”—[Nurse Practitioner] |
|
Less Priority for Prevention VS. Treatment | “Even though it can prevent something, they don't see the benefit of it because it's not actually treating somethin'.”—[Nurse Practitioner] “One of my best friends, we talked about it, and I was like, "You know you got to do what you got to do." Sometimes, we do get wrapped up and caught up in the social life and just havin' a good time, and we're not thinkin'. One time can change your life.”—[32-year-old, currently taking PrEP] “Comin' in when they're technically not sick. That's a challenge.”—[Nurse Practitioner] |
|
Clinical factors | Misunderstood Side Effects (Fear of dependency) | “When I took PrEP, it kinda freaked me out when I started breaking out in hives, so I stopped takin' it for a while. And I just recently got on Descovy.”—[24-year-old, currently taking PrEP] “I know one guy who told me, ‘No. I’m not trying to be poisoned.’” – [Nurse] “I just don't wanna be dependent on medication if that's a better way of saying it. Minor headaches and things of that nature, I try to wait them out.”—[32-year-old, currently taking PrEP] |
Fear PrEP Won’t/Work | “I'm paralyzed with the fear. "I don't know if this really works," What if this is just somethin' else? I don't know. What if it really doesn't work? I'm a person where I am gonna try to form my own opinion of it. But stuff like that definitely weighs on people's head – [29-year-old, currently not taking PrEP] “A lot of them I have talked with believe you could end up catching HIV if you take PrEP”.—[Nurse] |
Structural Factors
Although there are many payment assistance programs available for PrEP, participants reported cost as a barrier. Several participants reported being told PrEP was free only to have a co-pay they couldn’t afford. One patient stated,“They told me PrEP would be free, and so I said, “Okay.” Once I got to the pharmacy, I had to pay for it. It was, like, $50. Having to pay for it was an issue for me” [23-year-old, currently not taking PrEP]. Many reported a lack of discreet clinics, stating that most clinics that offer PrEP are known for providing HIV preventative care or serving primarily MSM. One nurse stated, “I think telehealth would be the best, because you don't have to worry about people who sees them comin' to a certain clinic, and, it could be more discrete if it's offered” [Nurse Practitioner]. Participants described not having enough time to attend medical appointments, complete labs, and pick up their PrEP medication. Similarly, participants noted competing interests such as work or school, and taking a daily medication for prevention was low on their priority list. A patient expressed this by saying, “Honestly, just, like, time. Time to go to the pharmacy to pick it up, time to go to the checkup, time to you know, anything. I just don't have time to do it, so I kinda just said, "Well, let me just stop having so much sex and stop having sex without condoms” [26-year-old, currently not taking PrEP].
Social Factors
Many participants stated that they didn’t know anyone taking PrEP and needed time to “mentally prepare.” One patient stated, I didn’t really know anyone on PrEP. I had friends who were HIV positive. Even though I know some of the same medications are used, I still think it’s probably different” [24-year-old, currently not taking PrEP]. Some reported being unaware of their risk for HIV. A few participants reported that YBMSM don’t take PrEP, because they do not want to be labeled gay or assumed to be HIV positive. Furthermore, participants noted that taking PrEP may cause conflict with partners. Participants indicated their partner would think that they have other partners, and PrEP would negatively impact relationship trust. One nurse explained, “They don’t want their partners to know that they’re takin’ PrEP. They don’t want their partners to believe they out there cheatin’ and that’s why they taking PrEP every day” [Nurse].
Behavioral Factors
Some participants reported only having one sexual partner; therefore, they didn’t think PrEP would benefit them. One patient in a relationship said, “I’m seeing only one person, and we are monogamous as well as that person goes and gets regular checkups and things of that nature. I’m not really too concerned about it” [24-year-old, currently not taking PrEP]. A few participants indicated that they or their social networks have thoughts of denial that “it won’t happen to me” regarding HIV. A nurse highlighted this with the comment, “I think a lot of them are still in the mindset of, “It won't happen to me. “Even though they don't always use condoms and they can come back with Chlamydia or Gonorrhea, or Syphilis. I think it's more so the denial” [Nurse Practitioner]. Many reported being unaccustomed to taking preventive medication, only using medications when sick. This is expressed by one nurse saying, “Comin' in when they're technically not sick. That's a challenge” [Nurse Practitioner].
Clinical Factors
Participants commonly misunderstood PrEP side effects and were unaware of the variety of current PrEP medications. Some participants expressed a fear of becoming reliant on PrEP medication, like dependency on illicit drugs. One patient specifically expressed, “I just don't wanna be dependent on medication if that's a better way of saying it. Minor headaches and things of that nature, I try to wait them out” [32-year-old, currently taking PrEP]. Others voiced concerns regarding PrEP’s efficacy, fearing they would contract HIV anyway. One nurse stated that based on their clinical experience, “A lot of them I have talked with believe you could end up catching HIV if you take PrEP” [Nurse].
Themes Related to Facilitators
The resulting thematic categories related to PrEP facilitators included (1) stigma reduction, (2) patient provider alliance, and (3) structural factors. Table 4 provides an overview of the themes related to facilitators to PrEP initiation and persistence. Table 5 illustrates themes with representative quotes.
Table 4.
Facilitators to PrEP initiation and persistence
Broad factors | Themes |
---|---|
Stigma reduction | Normalize/Rebrand PrEP to be for everyone Provide peer driven PrEP education Provide PrEP education to younger people (target schools) Have some MSM /members of the community as clinic staff |
Patient/provider alliance | Don’t act surprised by anything patients say Inform patients of the different medications for PrEP Be available to patients by phone for questions |
Structural factors | Offer same day PrEP appointments Offer the option to ship PrEP to their home Inform patients of payment assistance programs Offer variety of forms of PrEP |
Table 5.
Facilitators to PrEP initiation and persistence
Broad factors | Themes | Participant quotes |
---|---|---|
Stigma reduction | Normalize/Rebrand PrEP to be for everyone | “PrEP is for everyone not just MSM. But, whenever it's advertised on TV or anywhere else, I see MSM associated with PrEP, so if I talk to anyone who identifies as heterosexual, they think that PrEP is for homosexual activity, and I'm like that's not necessarily the case.”—[PrEP Navigator] “You know, PrEP is not a negative or a positive person's medicine or a gay man's medicine, PrEP is for everybody. Those are some of the stigmas.”—[PrEP Navigator] |
Provide Peer Driven PrEP education | “The guy that helped me to make the decision to get PrEP, that’s one of the things he was really harping on. It’s protecting yourself from HIV if I was to come in contact with someone that did have HIV. Not just protecting me, but protecting the person I’m in a relationship with.”—[34-year-old, currently taking PrEP] “I think it should be a peer, someone they can relate to. Sometimes children seem to learn better from a peer. I believe even in adulthood, it takes a peer to really push someone or encourage them to do something. Like even in my case, it took a peer to get me to get PrEP. Someone who’s experienced it. Someone who knows the ins and out of it.”—[34-year-old, currently taking PrEP] “Friends can be a big influence, especially in a community of MSM. If you have friends that is taking it, they can provide positive feedback. And if you have friends who are not, then, it’s the opposite.”—[Nurse] |
|
Provide PrEP Education to Younger People | “I feel like if we can build some type of proper organization or some type of speaker who could come down here to talk to these certain schools where you have young black men and women are being sexually active. Like, what are the risks? And how could this affect you long term?—[26-year-old, currently not taking PrEP] “Once they get older, you get set in your ways and it’s hard to break that barrier, but educatin’ them while they’re young and just lettin’ them know about the medicine. And also, educatin’ them about other STDs as well. Educate them while they’re young before they get too old and set in their ways.”—[Nurse] “I would even say, high school if you can, targetin' college freshmen just because we see so many young people comin' in shock that didn't know that they could possibly get HIV. I think as early as possible.”—[Nurse Practitioner] |
|
Have MSM / Members of the Community as Clinic Staff | “I do believe that they should be men who have sex with men and also on PrEP since that’s what we’re advocating for.”—[23-year-old, currently taking PrEP] “I think it makes it easier to where having people to actually introduce it to them if they don’t know about it and someone who may look like them or who may be MSM. I believe you don’t really see too many MSM nurses or physicians who are willing to express that, but I think it’ll make them comfortable.”—[Nurse] “Like the patient I had this morning, was askin' me if I knew of any providers who were LGBT in that community or were LGBT friendly. I think that would make a difference as to who's givin' you this information. Someone who has walked a mile in their shoes.”—[Nurse Practitioner] |
|
Patient/provider alliance | Don’t Act Surprised by Anything Patients Say | “I think being open with these patients and not acting shocked or surprised at anything that they say really does play a part in making them feel comfortable enough to tell you the truth.”—[Nurse Practitioner] |
Inform Patients of the Different Medications for PrEP | “We have Descovy which is another option. Just let them know about that. To have an honest dialogue. Not just to mention the medicine but tell them what it does and what it doesn't.”—[Licensed Master Social Worker] | |
Be Available for Questions | “I guess, being available. We use our cell phones a lot to get ahold of people, and just letting them know that we're here. We have lots of services for people, so they know that they can call us, come in, and just talk, and just be available. If we can't answer, we guide them to what they need.”—[Patient Care Coordinator] | |
Structural factors | Offer Same Day PrEP Appointments | “Definitely the rapid PrEP; the same-day PrEP. They’re served at the medical college, so able to get everything they need right then and there.”—[Nurse] |
Offer the Option to Ship PrEP | “I mean, just last week I had a client tell me, "I just don't know why I'm still taking this?" Because we had an issue with his pharmacy and getting the medicine delivered to him. And, he just absolutely did not want to pick that medicine up, so I had to work some magic and get it shipped to him.”—[PrEP Navigator] | |
Inform Patients of Payment Assistance Programs | “For some, it may be not taking PrEP, being able to pay for it. Because PrEP is pretty expensive.”—[34-year-old, currently taking PrEP] “I think people are worried about the price because I was kinda shocked to find out how much it woulda costed if I didn’t have insurance and assistance plan.”—[23-year-old, currently taking PrEP] |
|
Offer Variety of Forms of PrEP | “The best way for me to take PrEP is to give me an injection. I would prefer it, as a person that takes PrEP. I would prefer an injection over pills.”—[PrEP Navigator] “Probably, the longer-term one. If the implant is gonna last three years, I would say that route. That's probably gonna be the most preferred because of the time and minimal pain associated with gettin' it placed versus comin' ever so often to get a shot.”—[Nurse Practitioner] “I'm gonna stick with the old-fashioned everyday pill because they don't want to forget and don't want to take that chance.”—[Clinical Research Coordinator] |
Stigma Reduction
Participants reported that addressing PrEP stigma will increase uptake among YBMSM. Many participants noted the lack of variety in PrEP advertisements and emphasized the need to target not just gay men but also individuals who identify as heterosexual. A PrEP navigator stated, “PrEP is for everyone not just MSM. But, whenever it's advertised on TV or anywhere else, I see MSM associated with PrEP, so if I talk to anyone who identifies as heterosexual, they think that PrEP is for homosexual activity, and I'm like that's not necessarily the case” [PrEP Navigator]. Additionally, offering PrEP education delivered by peers may help reduce stigma. One patient attributed his motivation to take PrEP to learning about how PrEP would protect both him and his partners saying, “The guy that helped me to make the decision to get PrEP, that’s one of the things he was really harping on. It’s protecting yourself from HIV if I was to come in contact with someone that did have HIV. Not just protecting me, but protecting the person I’m in a relationship with” [34-year-old, currently taking PrEP]. Some felt that teaching prevention techniques at a younger age, such as in high school, could help reduce stigma. A nurse emphasized this, “Once they get older, you get set in your ways and it’s hard to break that barrier, but educatin’ them while they’re young and just lettin’ them know about the medicine. And also, educatin’ them about other STDs as well. Educate them while they’re young before they get too old and set in their ways” [Nurse]. Lastly, hiring members of the community, such as MSM, to work in clinics may alleviate stigma. One patient expressed, “I do believe that they should be men who have sex with men and also on PrEP since that’s what we’re advocating for” [23-year-old, currently taking PrEP].
Patient/Provider Alliance
Patients emphasized the importance of the patient and provider relationship. Many noted that providers should create safe environments that make YBMSM comfortable enough to share their experiences. One nurse shared, “I think being open with these patients and not acting shocked or surprised at anything that they say really does play a part in making them feel comfortable enough to tell you the truth” [Nurse Practitioner]. Additionally, many patients appreciated being offered different options of PrEP, such as Truvada and Descovy (Grant et al., 2010; Ogbuagu et al., 2021). One participant shared that he had side effects and was unaware of other options, so discontinued PrEP. Participants also mentioned that PrEP use may increase if providers are available by phone to answer questions. A patient care coordinator specifically said, “I guess, being available. We use our cell phones a lot to get ahold of people, and just letting them know that we're here. We have lots of services for people, so they know that they can call us, come in, and just talk, and just be available. If we can't answer, we guide them to what they need” [Patient Care Coordinator].
Structural Factors
Participants reported several facilitators to address the structural barriers to PrEP use. Many reported that offering same day PrEP appointments and prescription pick-up would eliminate several barriers. One nurse stated, “Definitely the rapid PrEP; the same-day PrEP. They’re served at the medical college, so able to get everything they need right then and there” [Nurse]. Some felt that offering PrEP via mail delivery would increase use. Additionally, informing patients of all PrEP payment assistance programs up-front would help address financial barriers. A patient highlighted this,“I think people are worried about the price because I was kinda shocked to find out how much it woulda costed if I didn’t have insurance and assistance plan” [23-year-old, currently taking PrEP]. Lastly, providing YBMSM options for PrEP (e.g., injection, once a month PrEP, or daily PrEP) them may help enhance PrEP persistence.
Discussion
A recent literature review evaluating barriers to HIV prevention in MS found that public health policy, stigma, cost, and distrust of the healthcare system continue to be significant barriers (Hrostowski & Pelts, 2018). Our study provides updated information on the barriers to PrEP, previously reported in 2016 among YBMSM living in MS (Arnold et al., 2017). Additionally, we offer insight from clinic staff, as well as PrEP facilitators. These data provide insight on the current barriers to PrEP use among YBMSM amidst several intervention and exploratory efforts to overcome these barriers (Chase et al., 2023; Rouffiac et al., 2020; Whiteley et al., 2019, 2021). Barriers continued to be attributed to structural, social, behavioral, and clinical factors. Most barriers cited in 2016 persist in 2022 (Arnold et al., 2017). Our results provided a more detailed picture of the existing barriers and the changes over time.
Stigma and homophobia constitute major barriers to PrEP use, especially in the South (Adeagbo et al., 2021). Reif et al. (2018), explained that misinformation about HIV and its prevention contributed to HIV stigma in the South, which could be associated with negative perceptions of PrEP There were several social barriers to PrEP use that persisted over time. For example, YBMSM continue to fear stigma related to PrEP (i.e., assumptions of sexual promiscuity or HIV positive status). Many of our identified PrEP facilitators focused on de-stigmatization methods. These results mirrored those of a recent study in Memphis, Tennessee, that found facilitators to PrEP use included using trusted peers, relatable healthcare providers, and social media to disseminate information (Pichon et al., 2022). Our participants highlighted the need for PrEP to be rebranded and normalized for everyone. PrEP marketing targeting MSM is a barrier and increases stigma (Elopre et al., 2021; Pichon et al., 2022). Future research should develop interventions and implementation strategies that specifically address stigma related to PrEP and enhanced marketing.
YBMSM continue to report fear of repercussions if their partner discovered they were taking PrEP. Although previous research suggests that most HIV transmission occurs in primary partnerships for MSM (Sullivan et al., 2009), many YBMSM continue to perceive their HIV risk as low due to being in a monogamous relationship. HIV risk denial also decreased their willingness to obtain a PrEP prescription. Our results indicated that many YBMSM prioritize treatment over prevention, and preventive medicine is a newer concept. YBMSM in this study and others have expressed the importance of offering PrEP and sexual health education at a younger age (Elopre et al., 2021). Thus, it may be beneficial to target high schools, college campuses, and community centers who serve youth. Researchers should focus on creating education that fosters support and acceptance of YBMSM in Southern Black communities to change social norms, increase risk awareness, and increase willingness to seek prevention services.
Although PrEP has been available since 2012, many reported not knowing anyone taking it and needing to think about PrEP for an extended time prior to taking it. This hesitation is a notable hurdle. Understanding peer influence on PrEP use is crucial to overcome these barriers. Quinn et al. (2020) showed that an increase in PrEP use among Black gay and bisexual men was associated with peer and social networks. Peer-driven education is an effective tool to facilitate PrEP uptake and adherence (He et al., 2020) and should be incorporated into prevention efforts. Additionally, having providers who are also YBMSM may increase comfort, decrease stigma, and enhance rapport with patients. Avenues to enhance the therapeutic alliance for YBMSM include hiring community members, encouraging patients to be open without judgement, being available for questions, and offering different forms of PrEP.
Although many different forms of PrEP have evolved over the past ten years, participants continue to report anticipated side effects as a primary hesitation. Participants in this study identified misunderstood side effects, including fear of becoming dependent on PrEP and PrEP not working. There are three PrEP medications currently approved for YBMSM: emtricitabine/tenofovir disoproxil fumarate (Truvada®; F/TDF) approved 2012, emtricitabine/tenofovir alafenamide (Descovy®; F/TAF) approved 2019 (Grant et al., 2010; Ogbuagu et al., 2021), and cabotegravir extended-release injectable suspension (Apretude) approved 2021 (Administration, 2021). Recent literature examining the patient-focused selection of the appropriate PrEP medications, based on biological, behavioral, and health characteristics of an individual have demonstrated the need for offering options (Fields & Tung, 2021). A variety of forms of PrEP (e.g., oral, injection, implant) should be provided to patients (Greene et al., 2017). Allowing patients to select the form of PrEP may decrease stigma and increase comfort and adherence.
Cost continues to be a barrier for YBMSM pursuing PrEP. In 2016, YBMSM reported that access to payment assistance programs was a barrier (Arnold et al., 2017). While this was noted less in these updated interviews, cost of copays continues to be a barrier. This highlights the need for further dissemination of PrEP payment assistance programs. Aside from cost, structural barriers identified were lack of discreet clinics, time commitment, and competing interests. Offering same day PrEP appointments and PrEP mail delivery were listed as possible facilitators.
Limitations
There were some study limitations. The study focused solely on YBMSM in Mississippi. These sample restrictions were intentional, as the intention of the study was to provide an update on prior work that had focused on this population. This focus may limit generalizability to other subpopulations in varied geographic areas; however, similar findings have emerged from other regions and locations (Adeagbo et al., 2021; Pichon et al., 2022; Reif et al., 2018). Only those willing to partake in a PrEP-related interview participated, with many reporting previous PrEP use. Thus, our results may underestimate or overlook PrEP barriers for those unable to engage in PrEP. Lastly, interviews occurred over Zoom, which may have affected participants’ comfort and openness in discussing health-related topics.
Conclusions
Although PrEP was approved over ten years ago, barriers to uptake persist among YBMSM living in MS. There is an urgent need to address the structural, social, behavioral, and clinical barriers to PrEP use in this population. This study highlights not only the barriers to PrEP uptake, but also offers facilitators to enhance PrEP use. Results will inform intervention efforts tailored to mitigate barriers and improve PrEP uptake among YBMSM in the South.
Funding
This work was supported by the National Institutes of Mental Health: K23MH124539-01A1 and T32MH078788. Author time was also supported by the Providence/Boston Center for AIDS Research P30AI042853. Dr. Elwy is partially supported by Institutional Development Award Number U54GM115677 from the National Institute of General Medical Sciences of the National Institutes of Health, which funds Advance Clinical and Translational Research (Advance-CTR). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
Conflict of interest Dr. Rogers receives research funding from Gilead Sciences #IN-US-276-5463. Authors have disclosed all conflicts of interest.
Ethical Approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the University of Mississippi Institutional Review Board reviewed and approved this project, and its conduct was consistent with applicable federal law (Approval Number FWA00003630).
Informed Consent All participants completed written informed consent prior to participating.
Data Availability
Data are available upon reasonable request.
References
- Adeagbo O, Harrison S, Qiao S, & Li X (2021). Pre-exposure prophylaxis (PrEP) uptake among black men who have sex with men (BMSM) in the Southern U.S. International Journal of Environmental Research and Public Health, 18(18), 9715. [DOI] [PMC free article] [PubMed] [Google Scholar]
- AIDSVu. (2022). Local data: Mississippi https://aidsvu.org/local-data/united-states/south/mississippi/
- Arnold T, Brinkley-Rubinstein L, Chan PA, Perez-Brumer A, Bologna ES, Beauchamps L, Johnson K, Mena L, & Nunn A (2017). Social, structural, behavioral and clinical factors influencing retention in Pre-Exposure Prophylaxis (PrEP) care in Mississippi. PLoS ONE, 12(2), e0172354. 10.1371/journal.pone.0172354 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Braun V, Clarke V, Hayfield N, & Terry G (2019). Thematic analysis. In Liamputtong P (Ed.), Handbook of research methods in health social sciences (pp. 843–860). Springer. 10.1007/978-981-10-5251-4_103 [DOI] [Google Scholar]
- Bush S, Magnuson D, Rawlings KM, Hawkins T, McCallister S, & Mera Giler R (2016). ASM/ICAAC: Racial characteristics of FTC/TDF for pre-exposure prophylaxis (PrEP) users in the US. In Interscience Conference on Antimicrobial Agents and Chemotherapy. Boston, MA. http://www.natap.org/2016/HIV/062216_02.htm [Google Scholar]
- Center for Disease Control and Prevention. (2018). HIV prevention pill not reaching most Americans who could benefit – especially people of color https://www.cdc.gov/nchhstp/newsroom/2018/croi-2018-prep-press-release.html [Google Scholar]
- Centers for Disease Control and Prevention. (2022). HIV in the United States by region: HIV diagnoses. https://www.cdc.gov/hiv/statistics/overview/diagnoses.html [Google Scholar]
- Chan GA, Johnson KL, Mosca NG, Dobbs TE, Dombrowski JC, Bennett AB, Buskin SE, & Golden MR (2017). Emerging regional and racial disparities in the lifetime risk of human immunodeficiency virus infection among men who have sex with men: A comparative life table analysis in King County, WA and Mississippi. Sexually Transmitted Diseases, 44(4), 227–232. 10.1097/olq.0000000000000589 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chase E, Mena L, Johnson KL, Prather M, & Khosropour CM (2023). Patterns of pre-exposure prophylaxis (PrEP) use in a population accessing PrEP in Jackson, Mississippi. AIDS and Behavior, 27, 1082–1090. 10.1007/s10461-022-03845-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Edeza A, Karina Santamaria E, Valente PK, Gomez A, Ogunbajo A, & Biello K (2021). Experienced barriers to adherence to pre-exposure prophylaxis for HIV prevention among MSM: A systematic review and meta-ethnography of qualitative studies. AIDS Care, 33(6), 697–705. 10.1080/09540121.2020.1778628 [DOI] [PubMed] [Google Scholar]
- Elopre L, Ott C, Lambert CC, Amico KR, Sullivan PS, Marrazzo J, Mugavero MJ, & Turan JM (2021). Missed prevention opportunities: Why young, black MSM with recent HIV diagnosis did not access HIV pre-exposure prophylaxis services. AIDS and Behavior, 25(5), 1464–1473. 10.1007/s10461-020-02985-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ezennia O, Geter A, & Smith DK (2019). The PrEP care continuum and black men who have sex with men: A scoping review of published data on awareness, uptake, adherence, and retention in PrEP care. AIDS and Behavior, 23(10), 2654–2673. 10.1007/s10461-019-02641-2 [DOI] [PubMed] [Google Scholar]
- Fauci AS, Redfield RR, Sigounas G, Weahkee MD, & Giroir BP (2019). Ending the HIV epidemic: A plan for the United States. Journal of the American Medical Association, 321(9), 844–845. 10.1001/jama.2019.1343 [DOI] [PubMed] [Google Scholar]
- Fields SD, & Tung E (2021). Patient-focused selection of PrEP medication for individuals at risk of HIV: A narrative review. Infectious Diseases and Therapy, 10(1), 165–186. 10.1007/s40121-020-00384-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grant RM, Lama JR, Anderson PL, McMahan V, Liu AY, Vargas L, Goicochea P, Casapia M, Guanira-Carranza JV, Ramirez-Cardich ME, Montoya-Herrera O, Fernandez T, Veloso VG, Buchbinder SP, Chariyalertsak S, Schechter M, Bekker LG, Mayer KH, Kallas EG, & Glidden DV (2010). Preexposure chemoprophylaxis for HIV prevention in men who have sex with men. New England Journal of Medicine, 363(27), 2587–2599. 10.1056/NEJMoa1011205 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greene GJ, Swann G, Fought AJ, Carballo-Diéguez A, Hope TJ, Kiser PF, Mustanski B, & D’Aquila RT (2017). Preferences for long-acting pre-exposure prophylaxis (PrEP), daily oral PrEP, or condoms for HIV prevention among U.S. men who have sex with men. AIDS and Behavior, 21(5), 1336–1349. 10.1007/s10461-016-1565-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hannaford A, Lipshie-Williams M, Starrels JL, Arnsten JH, Rizzuto J, Cohen P, Jacobs D, & Patel VV (2018). The use of online posts to identify barriers to and facilitators of HIV pre-exposure prophylaxis (PrEP) among men who have sex with men: A comparison to a systematic review of the peer-reviewed literature. AIDS and Behavior, 22(4), 1080–1095. 10.1007/s10461-017-2011-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- He J, Wang Y, Du Z, Liao J, He N, & Hao Y (2020). Peer education for HIV prevention among high-risk groups: A systematic review and meta-analysis. BMC Infectious Diseases, 20(1), 338. 10.1186/s12879-020-05003-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hrostowski S, & Pelts M (2018). The promises and challenges of preexposure prophylaxis (PrEP) in Mississippi. Journal of HIV & Retro Virus. 10.21767/2471-9676.100038 [DOI] [Google Scholar]
- IBM. (2021). IBM SPSS Statistics (Version 28) [Software]. IBM Corporation. [Google Scholar]
- Marcus JL, Volk JE, Pinder J, Liu AY, Bacon O, Hare CB, & Cohen SE (2016). Successful implementation of HIV preexposure prophylaxis: Lessons learned from three clinical settings. Current HIV/AIDS Reports, 13(2), 116–124. 10.1007/s11904-016-0308-x [DOI] [PubMed] [Google Scholar]
- Matacotta JJ, Rosales-Perez FJ, & Carrillo CM (2020). HIV preexposure prophylaxis and treatment as prevention—beliefs and access barriers in men who have sex with men (MSM) and transgender women: A systematic review. Journal of Patient-Centered Research and Reviews, 7(3), 265–274. 10.17294/2330-0698.1737 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ogbuagu O, Ruane PJ, Podzamczer D, Salazar LC, Henry K, Asmuth DM, Wohl D, Gilson R, Shao Y, Ebrahimi R, Cox S, Kintu A, Carter C, Das M, Baeten JM, Brainard DM, Whitlock G, Brunetta JM, Kronborg G, & Spinner CD (2021). Long-term safety and efficacy of emtricitabine and tenofovir alafenamide vs emtricitabine and tenofovir disoproxil fumarate for HIV-1 pre-exposure prophylaxis: Week 96 results from a randomised, double-blind, placebo-controlled, phase 3 trial. The Lancet HIV, 8(7), e397–e407. 10.1016/s2352-3018(21)00071-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pichon LC, Teti M, McGoy S, Murry VM, & Juarez PD (2022). Engaging Black men who have sex with men (MSM) in the South in identifying strategies to increase PrEP uptake. BMC Health Services Research, 22(1), 1491. 10.1186/s12913-022-08914-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- QSR International Pty Ltd. (2020). NVivo (Version 12). https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home?_ga=2.147882994.43489428.1663089518-1458777242.1663089518
- Quinn KG, Christenson E, Spector A, Amirkhanian Y, & Kelly JA (2020). The influence of peers on PrEP perceptions and use among young black gay, bisexual, and other men who have sex with men: A qualitative examination. Archives of Sexual Behavior, 49(6), 2129–2143. 10.1007/s10508-019-01593-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rawlings MK, & Parham-Hopson D (2021). HIV in the South. In Ojikutu B & Stone V (Eds.), HIV in US communities of color (pp. 135–146). Springer. [Google Scholar]
- Reif S, Wilson E, & McAllaster C (2018). Perceptions and impact of HIV stigma among high risk populations in the US Deep South. Journal of HIV and AIDS, 4(2). 10.16966/2380-5536.154 [DOI] [Google Scholar]
- Rosenberg ES, Grey JA, Sanchez TH, & Sullivan PS (2016). Rates of prevalent HIV infection, prevalent diagnoses, and new diagnoses among men who have sex with men in US states, metropolitan statistical areas, and counties, 2012–2013. JMIR Public Health and Surveillance, 2(1), e22. 10.2196/publichealth.5684 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rouffiac A-E, Whiteley L, Brown L, Mena L, Craker L, Healy M, & Haubrick K (2020). A mobile intervention to improve uptake of pre-exposure prophylaxis for southern black men who have sex with men: Protocol for intervention development and pilot randomized controlled trial. JMIR Research Protocols, 9(2), e15781–e15781. 10.2196/15781 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Russ S, Zhang C, & Liu Y (2021). Pre-exposure prophylaxis care continuum, barriers, and facilitators among black men who have sex with men in the United States: A systematic review and meta-analysis. AIDS and Behavior, 25(7), 2278–2288. 10.1007/s10461-020-03156-x [DOI] [PubMed] [Google Scholar]
- State of Mississippi Department of Health. (2015). 2015 STD/HIV epidemiologic profile. https://msdh.ms.gov/msdhsite/_static/resources/7543.pdf [Google Scholar]
- Sullivan PS, Salazar L, Buchbinder S, & Sanchez TH (2009). Estimating the proportion of HIV transmissions from main sex partners among men who have sex with men in five US cities. AIDS, 23(9), 1153–1162. 10.1097/QAD.0b013e32832baa34 [DOI] [PubMed] [Google Scholar]
- United States Food and Drug Administration. (2021). FDA approves first injectable treatment for HIV pre-exposure prevention https://www.fda.gov/news-events/press-announcements/fda-approves-first-injectable-treatment-hiv-pre-exposure-prevention [Google Scholar]
- Whiteley L, Rouffiac A-E, Brown L, Mena L, Craker L, Healy M, & Haubrick K (2019). A mobile intervention to improve uptake of pre-exposure prophylaxis (PrEP) for Southern Black MSM: A protocol for intervention development and randomized controlled trial (Preprint). doi: 10.2196/preprints.15781 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Whiteley L, Craker L, Sun S, Tarantino N, Hershkowitz D, Moskowitz J, Arnold T, Haubrick K, Olsen E, Mena L, & Brown LK (2021). Factors associated with PrEP adherence among MSM living in Jackson, Mississippi. Journal of HIV/AIDS & Social Services, 20(3), 246–261. 10.1080/15381501.2021.1956666 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data are available upon reasonable request.