Abstract
Behavioral health conditions are disproportionately experienced by people living with Human immunodeficiency virus (HIV), including young Black gay, bisexual, and other men who have sex with men (GBMSM). Left unaddressed, these symptoms can adversely impact HIV care outcomes. Improving the integration of behavioral health and HIV care services has been proposed as a strategy to address this challenge. To conduct a pre-implementation study exploring barriers and facilitators to improving HIV and behavioral health care integration at two HIV clinics in Atlanta, Georgia. We conducted a mixed-methods study guided by the Consolidated Framework for Implementation Research (CFIR). Sixty (60) HIV care providers, behavioral health care providers, and social service providers participated in cross-sectional surveys, and a subset of survey participants (15) also participated in a qualitative in-depth interview to explore CFIR constructs in greater depth. We focused on Intervention Characteristics, Outer Setting, and Inner Setting as the most relevant CFIR domains. Within each of these domains, we identified both facilitators and barriers to improving HIV and behavioral care integration in the two clinics. Participants agreed that enhancing integration would provide a relative advantage over current practice, would address young Black GBMSM and other patient needs, and would be compatible with the organizational mission. However, they also expressed concerns about complexity, resource availability, and priority relative to other clinic initiatives. Participants were enthusiastic about improving care integration but also invoked practical challenges to translating this idea into practice. Future research should test specific implementation strategies and their potential effectiveness for improving the integration of behavioral health and HIV care, as a strategy for improving well-being among young Black GBMSM and other people living with HIV.
Keywords: implementation science, behavioral health, care integration, HIV, Black/African-American, sexual minority health
Multidisciplinary care providers provided feedback about the potential benefits and challenges of enhancing HIV and behavioral health services in their clinics.
Implications.
Practice: Integration of behavioral and Human Immunodeficiency Virus (HIV) care is a potentially promising strategy for improving clinical outcomes; however, more work is needed to determine optimal strategies for implementation.
Policy: Policymakers should address structural factors that limit clinicians’ ability to address behavioral health needs when providing HIV care.
Research: Future research should examine specific implementation strategies related to the successful integration of behavioral health and HIV care.
Introduction
Human immunodeficiency virus (HIV) disproportionately impacts Black gay, bisexual, and other men who have sex with men (GBMSM). In 2019, approximately 26% of new HIV infections in the USA occurred among Black GBMSM, despite this group comprising less than 1% of the US population [1]. One of the most effective strategies for decreasing HIV incidence is ensuring that those who are currently living with HIV are engaged in care and adherent to antiretroviral therapy (ART). Unfortunately, Black GBMSM, particularly young Black GBMSM ages 18–29, have lower rates of engagement in HIV care and, as a result, are less likely to achieve the goal of HIV viral suppression compared to individuals from other racial/ethnic groups [2–4]
These gaps in care engagement are attributable to multilevel barriers, many of which fall under the umbrella of behavioral health conditions (e.g. depression, anxiety, and substance use). We use “behavioral health care” as a term to include care for “any behavioral problems bearing on health, including mental health, substance use, stress-linked physical symptoms, patient activation and health behaviors” [5]. Several studies have described the significant burden of behavioral health challenges among young Black GBMSM, who experience high rates of anxiety and depression, attributable in part to continuous experiences of structural and interpersonal racism and discrimination [6–9]. Other studies of behavioral health among young Black GBMSM living with HIV have also demonstrated a high prevalence of psychosocial stress, including rates of depressive symptoms amongst young Black GBMSM living with HIV ranging from 38% to as high as 58.6% [2, 10, 11].
Despite this high psychological stress burden, Black individuals, including Black GBMSM, have been shown to underutilize behavioral health services in comparison to individuals of other races, due to both sociocultural (e.g. community norms against behavioral health care help-seeking) and structural/logistical (e.g. lack of access to or knowledge of behavioral health care resources) barriers [12, 13]. Integration of behavioral health care within HIV service settings has been proposed as a structural intervention to increase rates of behavioral health care utilization [14–18]. Behavioral health and HIV care integration is also important because of its potential for reducing stigma associated with these services [19] which is particularly critical for multiple minoritized populations such as young Black GBMSM. To date, the structure, extent, and effectiveness of integration models have varied greatly. Co-location of services is one type of care integration that has been shown to facilitate access and improve behavioral health outcomes [14, 15]. Increasing access to Cognitive Behavioral Therapy and other counseling modalities has been shown to improve rates of viral suppression and HIV medication adherence [14, 17]. However, young Black GBMSM may not benefit as much from current integration strategies, which by necessity are developed for a more general clinic population and thus are typically not designed specifically with young Black GBMSM’s cultural and developmental needs in mind. Thus, further research is needed to identify how to best structure and implement care integration models to meet the unique needs of young Black GBMSM living with HIV.
We undertook a pre-implementation study with the aim of identifying barriers and facilitators likely to influence efforts to improve the integration of behavioral health services in HIV care for this population. The study sought to collect input from a sample of multidisciplinary service providers regarding behavioral health services implementation in outpatient HIV care settings. While the sample surveyed for this study included providers who serve persons with HIV across the age spectrum, there was particular interest in understanding implications for young Black GBMSM given unique challenges with respect to HIV care adherence and retention. Our implementation planning was guided by the Consolidated Framework for Implementation Research (CFIR), which outlines a menu of constructs from multiple existing theories, resulting in a comprehensive typology of factors that may influence implementation outcomes [20]. The original CFIR constructs are organized into five major domains: Intervention characteristics (referring to attributes of an intervention that might influence implementation), Outer setting (factors external to an organization that could impact implementation), Inner setting (features of an organization’s culture or processes that impact implementation), Characteristics of individuals (attitudes and beliefs of organizational staff and leaders that could influence implementation), and the Process of implementation (e.g. planning, engaging and executing).
Methods
Setting
The study was conducted between November 2019 and January 2020 at two HIV clinics in Atlanta, Georgia, representing different types of implementation environments. Clinic A serves approximately 6000 patients living with HIV (including an estimated 600–700 young Black GBMSM) each year and is part of a public safety-net health care system. Within Clinic A, young Black GBMSM ages 18–24 years are served within a separate pediatrics department that has its own psychologist, psychiatrist, and social services team embedded on the same floor as the HIV medical care team. Patients over 25 years receive behavioral health services within the same building but in a different area from where adult-oriented HIV medical care services are located. The adult behavioral health team is comprised of psychiatrists, psychologists, and licensed counselors. Clinic B serves approximately 2000 patients living with HIV (including an estimated 100–200 young Black GBMSM) each year and is located within a private academic health care system. In Clinic B, behavioral health providers, including licensed counselors and a psychiatrist, work in the same physical clinic space as the HIV medical care providers.
Study design and procedures
Utilizing a convergent parallel mixed-methods study design, we administered surveys and in-depth interviews with service providers at the two clinics. To recruit survey participants, we partnered with leadership at the two clinic sites to obtain a comprehensive list of staff, including HIV care providers (physicians, advanced practice providers, and nurses), behavioral health care providers (psychiatrists, psychologists, and counselors), and social service providers (social workers and case managers). We offered participation in a cross-sectional survey via e-mail, from which providers could link directly to a self-administered online survey via REDCap, a secure online platform [21]. The e-mail was sent to N = 114 individual e-mail addresses, with responses received from N = 60 (53% participation rate). Participants received a $25 electronic gift card upon completion of this survey. During the survey, participants were asked if they would be interested in participating in an in-depth interview. A subset of survey participants from the two clinic sites (N = 15) were selected based on their clinical roles to participate in a qualitative in-depth interview to explore CFIR constructs in greater depth. We chose participants to interview based on clinical role and clinic location, to ensure that diverse perspectives were represented. Interviews were conducted by a study team member in a private, mutually convenient location, and lasted an average of 30–60 min. Interview participants received another $25 for their time. All interviews were digitally recorded and transcribed verbatim by a professional service.
Study team
Our investigative and analytic teams were comprised of researchers including HIV clinicians (in both clinics A and B), behavioral health clinicians (in clinic A), individuals with administrative leadership roles (in clinics A and B), and Masters of Public Health (MPH)-level staff and graduate research assistants. To minimize the potential for conflict of interest and to maximize participant comfort, interviews, and surveys were not directly administered by any team member with a clinical or administrative role in either clinic. Research team members with clinical and/or administrative roles never worked directly with identifiable data, and potential participants were reassured of this at multiple times throughout the process.
Measurement
Table 1 depicts our CFIR-informed mixed-methods assessment strategy. We selected domains and constructs from the CFIR “menu” based on the highest relevance to our proposed intervention, setting, and study questions; which in turn were determined based on our a priori knowledge of the two clinical settings. To assess the CFIR constructs of Relative Advantage, Complexity, Compatibility/Alignment, and Relative Priority, we selected items from the Readiness for Integrated Care Questionnaire (RICQ) [22], a tool originally developed for use in primary care clinics in underserved areas and designed specifically to measure organizational readiness for integration of behavioral health and primary care services. RICQ items listed responses on a 7-item Likert scale: (i) Strongly Disagree, (ii) Disagree, (iii) Slightly Disagree, (iv) Neither Disagree nor Agree, (v) Slightly Agree, (vi) Agree, or (vii) Strongly Agree. The reliability of this measure as a whole was adequate in our sample (Cronbach’s α = 0.86). We also separately assessed reliability for subscales that we created from the RICQ items to represent these four CFIR constructs and found acceptable reliability for three of these: (α = 0.88 for Relative Advantage, α = 0.90 for Compatibility and Alignment, and α = 0.84 for Relative Priority). The three RICQ items about Complexity had low reliability (Cronbach’s α = 0.59) when assessed as a subscale; hence we did not report composite descriptive statistics for this construct. To assess the Networks and Communications, we utilized items from a measure adapted by Beacham et al. (2012) that provide indices of perceived access to, and usefulness of behavioral health consultative services from the perspective of HIV primary care clinicians. To assess behavioral health clinicians’ perspectives on this dynamic, we created parallel survey items as well. These items were scored on a 6-point Likert scale: (i) Strongly Disagree, (ii) Disagree, (iii) Slightly Disagree, (iv) Slightly Agree, (v) Agree, or (vi) Strongly Agree. The reliability of these measures was acceptable in our sample (Cronbach’s α = 0.89 for HIV primary care providers and α = 0.70 for the behavioral health providers for the total sample).
Table 1.
CFIR-guided pre-implementation assessment strategies
| CFIR domain (Definition) |
Construct (Assessment strategy) |
Exemplar questions |
|---|---|---|
|
Intervention Characteristics
(Attributes of an intervention that might influence implementation) |
Relative Advantage
(Quantitative and Qualitative) |
Survey items: Integrated care is better than other processes we are currently using in our clinic to meet the needs of our patients; Integrating behavioral health and HIV services (more than they currently are) will be beneficial for young Black GBMSM |
| Qualitative: What would be some advantages to further integrating behavioral health and HIV services in your clinic? | ||
|
Complexity
(Quantitative and Qualitative) |
Survey items: In our clinic, integrated care is simple and easy to implement; The complexity of integrated care will make it difficult to put this project into place | |
| Qualitative: What would be some challenges in achieving this level of integration? | ||
|
Outer Setting
(Influences on implementation that are external to the organization) |
Patient Needs and Resources
(Qualitative) |
Qualitative: Tell me about the needs for behavioral health services for your patients, especially those who are young Black GBMSM |
|
External policies and incentives
(Qualitative) |
Qualitative: What external factors, beyond the clinic and its management, might influence your clinic to adopt integrated MH/HIV services in your clinic? | |
|
Inner Setting
(Characteristics of implementing organizations) |
Compatibility/Alignment
(Quantitative) |
Survey items: Integrated care fits well with the culture and values of our patients; Integrated care fits well with other initiatives in our clinic |
|
Networks and Communications
(Quantitative and Qualitative) |
Survey items: I am satisfied with the current referral process from HIV/primary care to behavioral health; My patients have easy access to a behavioral medicine expert | |
| Qualitative: Can you talk about your thoughts on a collaborative care model in which there was increased, consistent communication between the HIV care team and MH care team | ||
|
Tension for Change
(Qualitative) |
Qualitative: In general, what facilitates the adoption of new practices or initiatives at your clinic? | |
|
Relative Priority
(Quantitative and Qualitative) |
Survey items: Integrated care is a top priority in our clinic; We are aware of how important integrated care is for our clinic right now | |
| Qualitative: To what extent might the further integration of HIV and behavioral health services take a backseat to other high-priority initiatives going on now? | ||
|
Available Resources
(Qualitative) |
Qualitative: How would you describe the overall current ability of your clinic to provide integrated HIV-behavioral health services? |
For the qualitative portions of the study, we developed a semi-structured interview guide comprised of open-ended questions and probes corresponding to selected domains and constructs of the CFIR: Relative Advantage, Complexity, Patient Needs and Resources, External Policies and Incentives, Networks and Communications, Tension for Change, and Available Resources.
Data analysis
Quantitative data analysis used SPSS v26 statistical software. We first examined data for non-normality and evidence of skewness or kurtosis. We conducted descriptive analyses to report frequencies and measures of central tendency and variability for RICQ survey responses. Qualitative analysis utilized a primarily deductive, code-based approach based on the initial steps of thematic analysis [23]. We utilized MAXQDA qualitative software to facilitate a team coding approach. First, a codebook was developed using both deductive codes corresponding to our CFIR constructs of interest, as well as some inductive codes emerging from the data. Preliminary codes were applied to the transcripts to highlight key themes and guide comparisons. The codebook was modified through an iterative process, which included discussions of transcripts, recurring themes, codes, and definitions. Two study team members independently coded each transcript and then reconvened to discuss and resolve discrepancies. After consensus was established, the remaining transcripts were coded independently. Research staff wrote thick descriptions (detailed analytic memos) of each theme, to explain depth, breadth, context, nuance, and relationships [24]. Thematic saturation occurred when no new patterns or themes emerged in the data.
Results
Demographic characteristics along with clinical settings and professional roles of our survey participants are shown in Table 2. Our sample averaged 45 years of age, was majority female (73%), and was mixed in terms of race/ethnicity. Most respondents were direct HIV care providers; however, social service providers, behavioral health care providers, and other staff also participated. Racial, gender, and job role distributions were similar between the larger survey study participants and participants selected for the qualitative sub-study. Our findings are organized by CFIR domains and constructs below. For the most part, identified facilitators and barriers were similar between the two clinics; where there were differences, the specific site (Clinic A or Clinic B) was explicitly named.
Table 2.
Demographics of the sample
| Survey participants (N = 60) | Interview participants (N = 15) | |
|---|---|---|
| Mean (SD) | Mean (SD) | |
| Age | 44.9 years (10.5) | 45.4 years (10.4) |
| N (%) | N (%) | |
|---|---|---|
| Gender | ||
| Male | 16 (26.7%) | 3 (33.3%) |
| Female | 44 (73.3%) | 12 (66.7%) |
| Race/Ethnicity | ||
| Asian | 8 (13.3%) | 4 (26.7%) |
| Black/African-American | 21 (35%) | 6 (40.0%) |
| White/Caucasian | 26 (43.3%) | 5 (33.3%) |
| Hispanic/Latino | 3 (5%) | 0 (0.0%) |
| Other | 2 (3.3%) | 0 (0.0%) |
| Country of birth | ||
| USA | 47 (78.3%) | 14 (93.3%) |
| Other | 13 (21.7%) | 1 (6.7%) |
| Sexual orientation | ||
| Gay/Homosexual/SGL | 6 (10.0%) | 1 (6.7%) |
| Straight/Heterosexual | 53 (88.3%) | 14 (93.3%) |
| Queer | 1 (1.7%) | 0 (0.0%) |
| Highest level of education | ||
| Bachelor’s degree | 5 (8.3%) | 0 (0.0%) |
| Graduate degree | 54 (90%) | 15 (100%) |
| Other | 1 (1.7%) | 0 (0.0%) |
| Work setting | ||
| Clinic B | 11 (18.3%) | 6 (40.0%) |
| Clinic A | 49 (81.7%) | 9 (60.0%) |
| Pediatric/Adolescent clinic | 12 (20%) | 4 (44.4%) |
| Adult clinics | 37 (61.7%) | 5 (55.6%) |
| Provider role | ||
| Direct HIV care provider (MD, PA-C, NP) | 37 (61.7%) | 5 (33.3%) |
| Direct behavioral health care provider | 8 (13.3%) | 4 (26.7%) |
| Social service provider | 12 (20%) | 5 (33.3%) |
| Clinical support staff (nursing, MA, pharmacy) | 3 (5%) | |
| Administration | 4 (6.7%) | 2 (13.3%) |
| Other | 2 (3.3%) | 1 (6.7%) |
Intervention characteristics
Relative advantage
Within the Intervention Characteristics domain, survey responses indicated that most providers felt there was a relative advantage to increasing integration of HIV and behavioral health services, with 63%–70% of participants either agreeing or strongly agreeing with statements that enhanced integration would represent an improvement over current or other potential processes. Table 3 depicts the distributions of answers to these items, along with the mean scores (from 1 to 7) for each item. The overall mean score was 5.7 (SD 1.0).
Table 3.
Readiness for Integrated Care Questionnaire—selected items (CFIR constructs: relative advantage, complexity, compatibility/alignment, priority, innovation specific knowledge; N = 60)
| CFIR construct | Strongly Disagree (1) |
Disagree (2) |
Slightly Disagree (3) |
Neither Disagree nor Agree (4) |
Slightly Agree (5) |
Agree (6) |
Strongly Agree (7) |
Mean (SD) |
|---|---|---|---|---|---|---|---|---|
| Relative advantage | ||||||||
| Integrated care is better than other processes we are currently using in our clinic to meet the needs of our patients | 0 (0%) | 1 (2%) | 1 (2%) | 7 (12%) | 9 (15%) | 24 (40%) | 18 (30%) | 5.8 (1.1) |
| Integrated care is better than other processes we have considered using in our clinic | 0 (0%) | 1 (2%) | 0 (0%) | 12 (20%) | 9 (15%) | 26 (43%) | 12 (20%) | 5.6 (1.1) |
| Integrated care represents an advance over other methods that are already available for our clinic | 0 (0%) | 1 (2%) | 1 (2%) | 9 (15%) | 8 (13%) | 27 (45%) | 14 (23%) | 5.7 (1.1) |
| Complexity | ||||||||
| In our clinic, integrated care is simple and easy to implement | 2 (3%) | 12 (20%) | 8 (13%) | 10 (17%) | 13 (22%) | 12 (20%) | 3 (5%) | 4.1 (1.7) |
| There are so many components to integrated care that is hard to understand all of the piecesa | 5 (8%) | 11 (18%) | 9 (15%) | 11 (18%) | 16 (27%) | 8 (13%) | 0 (0%) | 3.2 (1.6) |
| The complexity of integrated care will make it difficult to put this project into placea | 8 (13%) | 12 (20%) | 14 (23%) | 13 (22%) | 7 (12%) | 6 (10%) | 0 (0%) | 4.2 (1.5) |
| Compatibility/alignment | ||||||||
| Integrated care fits well with other initiatives in our clinic | 0 (0%) | 0 (0%) | 2 (3%) | 3 (5%) | 9 (15%) | 27 (45%) | 19 (32%) | 6.0 (1.0) |
| Integrated care will help us meet the needs of our patients | 0 (0%) | 0 (0%) | 0 (0%) | 3 (5%) | 11 (18%) | 24 (40%) | 22 (37%) | 6.1 (0.9) |
| Integrated care is timely given the current needs of our patients | 0 (0%) | 0 (0%) | 1 (2%) | 3 (5%) | 9 (15%) | 25 (42%) | 22 (37%) | 6.1 (1.0) |
| Integrated care fits well with the culture and values of our patients | 0 (0%) | 1 (2%) | 1 (2%) | 1 (2%) | 8 (13%) | 29 (48%) | 20 (33%) | 6.0 (1.0) |
| Priority | ||||||||
| Integrated care is a top priority in our clinic | 1 (2%) | 13 (22%) | 9 (15%) | 10 (17%) | 6 (10%) | 14 (23%) | 7 (12%) | 4.2 (1.8) |
| Our clinic emphasizes that integrated care is very important to improve primary care quality and access to behavioral health services in our community | 0 (0%) | 12 (20%) | 6 (10%) | 11 (18%) | 13 (22%) | 13 (22%) | 5 (8%) | 4.4 (1.6) |
| We are aware of how important integrated care is for our clinic right now | 0 (0%) | 2 (3%) | 7 (12%) | 10 (17%) | 15 (25%) | 17 (28%) | 9 (15%) | 5.1 (1.4) |
aThese two items are reverse coded so that higher score indicates higher readiness/more favorability towards the intervention.
In qualitative interviews, providers elaborated upon several advantages of integrating behavioral health and HIV services for their young Black GBMSM patients. Integration could make behavioral health care more accessible to patients by eliminating “messy” referral processes. Providers might gain a more holistic view of their patients’ care if behavioral health and HIV services were further integrated, as this might improve HIV primary care providers’ understanding of their patients’ behavioral health conditions. Providers also stated that there would be better compliance with medications and appointments if behavioral health and HIV care were more integrated. For example, if appointments could be scheduled on the same day, then there would be less of a chance of the patient failing to show up for each respect appointment. Some participants expressed that employing a team-based approach or having behavioral health providers consistently co-located with HIV care providers would increase communication, which would also lead to better patient outcomes. Finally, another relative benefit to enhancing integration could be that patients would view behavioral health as a part of their healthcare plan, potentially becoming more proactive about taking care of their behavioral health and thus having better health outcomes overall.
I would definitely [think] having mental health on each floor that we provided [HIV care] services, having somebody that is quickly accessible, like an on-call person who is actually on-call, and I think having regular meetings and case conferences about patients, one thing that I think would be really helpful with that, too, is if we are going to have case conferencing about patients and have some type of action plan afterwards, that somebody is following up about the action plan…I think we just need to do a better job integrating it.
-Social Service Provider
I think [integrating behavioral health and HIV care] that’s vital. I really do. Um, not just even in terms of, um, identifying the needs and experience of the patients, but I think when we can hear from each other and see what the other providers’ needs and experiences are, it sort of creates this way of sort of developing passion, and a willingness to work with each other. And in that way, it’s sort of modeling a family for the patient. A functional family.
-Behavioral Health Provider
Some participants also identified potential disadvantages to enhanced integration. One perceived disadvantage was the difficulty of sharing an office space, particularly when that space is constrained, as participants described their clinics. Space constraints were particularly emphasized in Clinic B. Potential difficulties discussed in this vein included noise levels in a shared workspace, as well as the potential for unintentional disclosures of sensitive information within the workspace. Additionally, some participants stated that there could be a greater chance for disagreement when there were multiple care providers involved. Finally, another disadvantage discussed by several participants was that some patients could be concerned about confidentiality, and not necessarily want all their care providers to know all of the same sensitive information.
I think sometimes patients have some concern, like if I tell this person, this provider, are they going to go tell this other provider? Even though both providers are responsible for my care, right? If I tell my psychologist something, are they going to go and tell my doctor or my [medical provider] the things that we just talked about?
-HIV care provider
Complexity
Survey responses to the three items regarding the potential complexity of implementing increased behavioral health integration averaged close to 4 (neither agree nor disagree) when participants were asked to rate the ease or difficulty of implementation (Table 3). Interview participants voiced concerns about logistical challenges including time constraints, physical space limitations (particularly at Clinic B), and workflow disruptions relating to current processes for scheduling appointments and rooming patients.
I think it will take a little bit of time. I say that because we’re a system, like any system, change takes some time. I think the biggest change would be physically how do you implement this space-wise? How are patients scheduled for appointments in terms of time blocks? There would be, that’s going to be a shift, because if you’re seeing three providers, somebody has to be mindful that when they scheduled you, so you’re talking about affecting a lot of different systems. That change will be slow I would think. Then if you’re seeing three providers, and we’re coming to you in the room, how many exam rooms do we have? … there’s some structural, systemic issues that type of care would affect, but I think it would have to be implemented slowly. I would do a progressive model of having a target point population or a day when you’re going to do it. I would phase that in rather than have some kind of…all-encompassing change.
-Behavioral Health Provider
Participants differed in terms of how complex they truly thought this change would be, with some being quite optimistic about the feasibility of this process.
I don’t think that actually would be that bad. It probably, I think that if there was a team that was motivated to do it, and to make those plans and test it out, it really wouldn’t take probably more than a few months to operationalize the whole thing.
-HIV Care Provider
Outer setting
Patient needs and resources
There was a consensus among all of the providers interviewed, that there is a high level of unmet behavioral health need among their young Black GBMSM patients. Providers acknowledged that at baseline, young Black GBMSM are often subjected to numerous structural inequalities. Experiences such as violence, low socioeconomic status, multiple overlapping stigmas, and general lack of social support were commonly cited factors that negatively impact their behavioral health.
I think all the obstacles that come along with being Black and lower income, and often undereducated, which I see a lot with my patients, um, I see a lot of depression. I think a lot of my patients wouldn’t necessarily call it depression but I would. A significant lack of social support, whether it’s from family or if you just think about like, their communities or our society as a whole. I think a lot of our patients really lack support and guidance that would be helpful for them.
-HIV Care Provider
Providers also described behavioral health needs related to the trauma associated with HIV diagnosis itself, particularly for young Black GBMSM, and endorsed the need for increased engagement with behavioral health services by their patients.
I think [mental health] is so integral… especially when it’s affecting someone young … it’s a struggle to cope with and deal with and there’s a lot of challenges and a lot of my patients have had significant hardships and struggles, even before being diagnosed with HIV. And I think it, you know, the disease itself can add to that burden. And I don’t know, I just think it’s hard to separate out sometimes you can’t put things in different categories and people, you know, it should be an integrated services with mental health. And yeah, again, it’s hard to separate sometimes the issues or just say you need to go to doctor A, or doctor B, or doctor C, when all of these problems sometimes overlap or compound each other.
-HIV Care Provider
External policies and incentives
Grant funding was described as an essential facilitator to the implementation of clinic interventions, with some providers going so far as to describe it as the main determinant of implementation priorities. One provider noted their frustration that without funding, even initiatives that were supported by staff and clients would not be enacted.
It’s money. But that’s what’s [frustrating], and I’ll say that you can sit there and the providers can tell you this is what we need, this is what’s lacking, this is what’s going to help us, and it falls on deaf ears until the grant part. And that’s why you lose providers here on a regular basis.
-HIV Care Provider
Providers, particularly in Clinic A, spoke about the challenges of working within an under-resourced public hospital system with patients who faced multiple challenges.
I mean, I think the issues that we have that create the most obstacles for patients are systemic, right? So some of them are more, you know, systemic from a federal funding kind of standpoint, like Ryan White, [AIDS drug assistance program], all these things, also some of it is, you know, systemic issues just within the [hospital name] system. Like I feel like in a lot of ways [hospital] does a lot for what you would call indigent populations, but we’re talking about patients who are really vulnerable, and groups of people who are already really vulnerable. But then they have to jump through so many hoops to get the care that they need...We make patients jump through so many hoops and make providers jump through so many hoops in order to care for these patients that I just can’t take it anymore…Then when you have all these systemic issues and then you have patients who either have really complicated social histories, a really complicated medical issue, and then you mix that in with the systemic issue, I think it’s like a ticking time bomb. I think it really, um, is a catalyst for burnout.
-HIV care provider
Inner setting
Mission alignment
Providers agreed strongly with statements about whether the proposed increase in behavioral health-HIV integration was compatible with the mission and core values of the two clinical organizations (Table 3). The mean score on the four RICQ items focused on Compatibility and Alignment was 6.0 (SD 0.8) on a scale of 1–7.
Networks and communications
Survey responses were somewhat neutral (averaging between slightly disagree and slightly agree) regarding satisfaction with current levels of communication between HIV care and behavioral health providers. The mean satisfaction score (on a scale of 1–6) for primary care providers was 3.4 (SD 1.2); for behavioral health providers, the mean score was 4.6 (SD 0.6). Table 4 depicts these responses separated by provider type (HIV and behavioral health; social service providers; or other non-clinical personnel did not receive these questions). The qualitative interviews similarly demonstrated varying degrees of communication, but most participants endorsed some room for improvement in interdepartmental communication processes.
Table 4.
Primary Care Provider Questionnaire/Modified Behavioral Health Care Provider Questionnaire—selected items (CFIR constructs: network and communications)
| Strongly Disagree (1) |
Disagree (2) |
Slightly Disagree (3) |
Slightly Agree (4) |
Agree (5) | Strongly Agree (6) |
Mean (SD) |
|
|---|---|---|---|---|---|---|---|
| It is difficult to reach mental health colleagues for an informal consultation (HIV providers; N = 38) | 2 (5%) | 8 (21%) | 8 (21%) | 8 (21%) | 6 (16%) | 6 (16%) | 3.7 (1.5) |
| I have been satisfied with the amount of contact I have with mental health providers regarding the care of my patients (HIV providers; N = 38) | 5 (13%) | 9 (24%) | 7 (18%) | 9 (24%) | 7 (18%) | 1 (3%) | 3.2 (1.4) |
| I have easy access to a behavioral medicine expert (i.e. chronic pain management, sleep hygiene, medication adherence issues) (HIV providers; N = 38) | 3 (5%) | 15 (39%) | 6 (16%) | 5 (13%) | 8 (21%) | 1 (3%) | 3.1 (1.4) |
| My patients have easy access to a behavioral medicine expert (i.e. chronic pain management, sleep hygiene, medication adherence issues) (HIV providers; N = 38) | 5 (8%) | 12 (32%) | 6 (16%) | 6 (16%) | 7 (18%) | 2 (5%) | 3.1 (1.5) |
| I am satisfied with the current referral process for mental health (HIV providers; N = 38) | 5 (13%) | 7 (18%) | 5 (13%) | 10 (17%) | 9 (24%) | 2 (5%) | 3.5 (1.5) |
| I know how to easily contact a mental health professional for a patient consultation (HIV providers; N = 38) | 1 (3%) | 6 (16%) | 2 (5%) | 13 (22%) | 8 (21%) | 8 (21%) | 4.2 (1.4) |
| HIV/primary care providers in this clinic can easily contact a mental health professional for a patient consultation (BH providers; N = 7) | 0 (0%) | 0 (0%) | 0 (0%) | 1 (14%) | 5 (71%) | 1 (14%) | 5.0 (0.6) |
| It is difficult to reach HIV/primary care colleagues with my recommendations for care (BH providers; N = 7) | 1 (14%) | 4 (57%) | 1 (14%) | 1 (14%) | 0 (0%) | 0 (0%) | 2.3 (1.0) |
| I have been satisfied with the amount of contact I have with HIV/primary care providers regarding the care of my patients (BH providers; N = 7) | 0 (0%) | 0 (0%) | 1 (14%) | 2 (28%) | 3 (43%) | 1 (14%) | 4.6 (1.0) |
| HIV/primary care providers frequently call on us as behavioral medicine specialists to support patients’ lifestyle changes (i.e. smoking, weight loss) (BH providers; N = 7) | 0 (0%) | 0 (0%) | 1 (14%) | 2 (28%) | 2 (28%) | 2 (28%) | 4.7 (1.1) |
| HIV/primary care providers in this clinic can easily access a behavioral medicine expert (i.e. for chronic pain management, sleep hygiene, medication adherence issues) (BH providers; N = 7) | 0 (0%) | 0 (0%) | 1 (14%) | 1 (14%) | 4 (57%) | 1 (14%) | 4.7 (1.0) |
| I am satisfied with the current referral process from HIV/primary care to mental health (BH providers; N = 7) | 0 (0%) | 0 (0%) | 2 (28%) | 2 (28%) | 2 (28%) | 1 (14%) | 4.3 (1.1) |
BH, behavioral health.
aShaded rows represent responses of HIV care providers; un-shaded rows represent responses of behavioral health providers.
A lot of stuff feels very siloed, and I don’t think that we do a good enough job with communication because people don’t read their emails, or people don’t check their voicemails and we do a lot of word of mouth things, but then we don’t follow that up with written documentation so that we have a way to reference things... So I feel like communication is really terrible…
-Social Service Provider
In contrast, the pediatrics section (Clinic A) was described as a model for effective communication.
Well, it’s called rounds...Every Thursday we have rounds from three to four. This is like a treatment team, it’s an interdisciplinary team. It’s made up [of]the social worker, the nurse, the doctor, and the psychologist. We staff challenging patients or we just inform each other with what’s going on with particular patients, or we come bringing a difficult or challenging case to the team for resolution.
-(Pediatric) Social Service Provider
A few respondents described challenges with accessing other providers, including not being able to reach people on the phone, not having enough behavioral health providers available, and providers not reading or responding to electronic health record (EHR) messages. Interestingly, the EHR was discussed by some as a facilitator, and by others as a barrier to effective communication between providers.
I can see what [the behavioral health providers are] doing with the patients in [name of EHR] and if my patients are telling me something, I can see that they’ve maybe addressed something that they told mental health as well.
-HIV Care Provider
On [the EHR], we have a direct link and it lights up when there’s a message. But some people don’t look at that. And like we were talking about, I think that some people check their email. Some people don’t check their email. Some people check their [EHR] messaging system, some people don’t. [we should] at least adopt a standard line of communication is like, this is how we communicate. When it’s urgent, I call you, please pick up or I’ll page you. You know, and if it’s not urgent, I’ll direct message you on [the EHR] or something like that.
-HIV Care Provider
This need for a more universal protocol for communication between departments was discussed by several participants.
Tension for change
Several participants spoke generally about inertia and resistance to change within the clinic environments.
Again, I think people are just in general resistant to change. And yeah, I think see it as a burden often, to their clinic flow, even though it might be a better system. I think getting over that hump, in a way, of like, welcoming something different and trying it out, knowing that it might slow you down at first.
-HIV Care Provider
Relative priority
Participants overall described behavioral health care as a high priority for their young Black GBMSM patients, with some going as far as saying it should be the highest priority due to the multiple structural barriers and psychosocial stressors faced by this group. Participants also talked about frequent behavioral health crises occurring at the clinics.
I don’t think that we will be able to provide patients optimal HIV and primary care without providing mental health services in the clinic. It’s too high of a need, um, and it’s too connected to all the other conditions that we’re trying to manage.
-HIV Care Provider
Most participants similarly expressed that their colleagues in the clinics would agree on the importance of behavioral health. However, when asked on the survey (Table 3) about Relative Priority compared to other initiatives in the clinics, the mean score was 4.6 (SD 1.4) on a 7-point scale, indicating that participants were neutral with regard to whether behavioral health integration would truly be seen as a top priority in their clinics once other initiatives were taken into consideration.
Available resources
In our interviews, providers highlighted inadequate staffing as an obstacle to optimizing behavioral health care delivery for young Black GBMSM. Participants from Clinic B described having patients with a full spectrum of behavioral health needs but no psychiatrist on-site to treat them; this staffing change occurred during our study.
So we definitely have a large mental health need in the clinic…to add to that need, we used to have a psychiatrist in the clinic, which was terrific, a couple of days a week, who is now no longer in the clinic. She left her role a few weeks ago and so that’s put a big hole in the clinic. We have counselors who are terrific, but they don’t have prescribing capacity. So what’s happened right now is that our HIV providers are actually prescribing meds in the interim.
-HIV Care Provider
Some HIV primary care providers expressed interest in further training to support their ability to prescribe psychotropic medications themselves, as a strategy for working with suboptimal behavioral health specialist support.
With respect to our population of interest, one participant specifically brought up the issue of inadequate representation of Black and/or GBMSM identities among the mental health providers at the clinic.
I think there is a problem, some demographic issues as well. We don’t have a lot of African-American providers. We don’t have a lot of African-American Black MSM providers. I think there is a, somewhat, like I’m not particularly represented at that clinic with mental health or my providers. So in a way, some staffing with maybe a little more relatable population in both those arenas could help, because there are definitely, that’s definitely been voiced in the community.
- HIV care provider
Inadequate staffing pertaining to other clinical roles (e.g. nurses) was discussed as well. Some described how demands to fill in ancillary support roles impeded providers’ ability to focus on providing clinical care, including their ability to address behavioral health. Finally, several providers acknowledged the potential logistical challenges of incorporating additional behavioral health duties into the time constraints of a typical primary care visit.
I think that it’s not a task that I can undertake. I don’t think I can change the way that I provide care…I think this is more on the administrative side that it needs to be integrated into the administrative workflow and the nursing workflow to be able to facilitate. There’s just not enough time for me to spend facilitating is what it comes down to.
-HIV Care Provider
Discussion
We found that providers in two Atlanta HIV clinics were, in concept, highly enthusiastic about improving the integration of behavioral health and HIV care services as a strategy for helping young Black GBMSM. However, when the potential mechanics and details of enhancing integration were explored, both qualitative and quantitative participants described concerns about complexity, staffing, and the ability to prioritize such an initiative when discussing potential implementation within their clinic settings. In particular, survey participants rated complexity somewhat unfavorably, despite very high agreement with statements about mission alignment and compatibility with their existing clinic cultures. These somewhat contradictory findings speak to the need for dedicated resource teams and workflow support to achieve enhanced integration that functions well within the existing clinical infrastructure.
Our findings also highlight provider-identified needs of Black GBMSM patients living with HIV due to structural inequalities, which were discussed primarily in the context of describing the outer setting around the clinics. Participants generally agreed that young Black GBMSM patients had a high burden of unmet behavioral health needs, with experiences of violence, stigma, and general lack of support negatively impacting health outcomes. Further, providers agreed that addressing behavioral health conditions should be a high priority among their young Black GBMSM patients and that young Black GBMSM patients could benefit from more streamlined referral processes. Although qualitative participants expressed these sentiments, survey results were more equivocal with respect to Relative Priority, indicating that while mental health is important, other competing concerns could overshadow mental health-related initiatives.
Figure 1 depicts the CFIR constructs as facilitators, barriers, or mixed facilitators/barriers in the contexts of the clinics we studied. Both facilitators and barriers can directly inform efforts to implement enhanced integration strategies in the clinic. For example, an ideal approach would build on identified facilitators by emphasizing the importance of HIV-behavioral health integration for young Black GBMSM patients, and the alignment with the overall goals of the organizations in question. At the same time, proposed implementation strategies would need to also attend to identified barriers; especially complexity (a concern identified through the surveys), and tension for change and available resources (discussed by interview participants). It should be noted that; although Fig. 1 depicts these barriers and facilitators as discrete entities, there is significant overlap between them and between CFIR constructs more generally. The conclusions and recommendations that follow are thus presented with an understanding that multi-pronged solutions will be needed to address this complex problem.
Figure 1.
Implementation barriers and facilitators for enhanced HIV-behavioral health care integration
The Expert Recommendations for Implementing Change (ERIC) tool is an online, searchable database that can be used to help map CFIR barriers onto effective implementation strategies [25, 26]. A search of the ERIC database, based on the barriers that we identified in this study, suggests that effective strategies to address the complexity of the intervention might include ongoing training with providers, the addition of workflow support, and conducting small cyclical tests of change during implementation. Addressing tension for change is similarly feasible via ongoing engagements with clinic staff, which might involve conducting local consensus discussions and needs assessments, while also identifying clinic champions/provisions of resource teams to promote the intervention.
On the other hand, addressing the availability of resources may have the greatest influence on the model that could be implemented at each clinic, but may also be the most daunting barrier as it requires structural change and likely changes in funding. Participants identified funding and staffing as essential resources that, if lacking, would severely inhibit the implementation of enhanced integration. Given that staff and providers have already described being overwhelmed with their current responsibilities, it could be challenging to incorporate additional responsibilities without hiring additional dedicated staff. Accordingly, these clinics may have difficulty implementing strategies such as additional care coordination or incorporation of counseling and prescribing in their HIV care visits without hiring additional staff and reorganization of workflows. These findings echo a recent narrative review of systematic reviews and meta-analyses, wherein the authors highlighted inherent challenges in single-site integration interventions, along with a need for systems-level integration interventions [27].
Several intervention and implementation studies have been previously conducted to try to address these and related gaps in care integration. Dombrowski et al. [18] adapted the collaborative care model—an evidence-based intervention in which HIV and behavioral health care providers work closely together using shared care plans—for implementation in a unique “low-barrier” HIV clinic that specifically targets individuals who are unable to maintain HIV care engagement. The authors have reported on their implementation process, in which they found that significant adaptations were needed for this unique clinical setting characterized by high patient need and complexity. Given our participant-identified barriers to both HIV and behavioral health care that are particularly prevalent among (though not unique to) young Black GBMSM (e.g. low socioeconomic status, experiences of violence and trauma, and stigma), it is likely that implementation of care integration strategies in our setting will similarly require flexibility and culturally tailored adaptation. Additionally, collaborative care models are appealing to many, but not all young Black GBMSM in our setting. We previously reported on interviews and surveys conducted with patients in these two clinics, who had differing preferences with regards to the amount of communication and coordination between their HIV and behavioral health providers [28].
Other studies have focused on systematic, universal, self-administered screenings to address the behavioral health needs of patients living with HIV, which was both acceptable to patients and useful for providers in developing treatment plans [29–31]. Universal screening allows providers to identify patients experiencing sub-clinical behavioral health symptomatology facilitates behavioral health referrals and has also been proposed as an anti-stigma intervention, since screeners are being delivered to all patients regardless of background or perceived symptoms. One CFIR-guided study examined the implementation of a substance use and mental health screener administered by case managers as part of a routine annual Ryan White re-enrollment visit and similarly found high levels of acceptability by patients and providers [32]. In our context, universal screening tools would ideally be adapted to include concerns specific to young Black GBMSM, such as mental health symptoms related to the stress of contending with intersectional stigma and discrimination at interpersonal and systemic levels.
Our study adds to the literature by applying the CFIR framework to identify modifiable barriers and facilitators of behavioral health-HIV care integration, with a specific emphasis on explicitly investigating selected CFIR constructs while also addressing the needs of young Black GBMSM in light of the multilevel challenges faced by this population. While many of our findings were generalizable across subpopulations of people living with HIV, some were particularly relevant for young Black GBMSM. Although most of the existing literature does not have such a specific demographic focus; one notable exception is the recent “Black MSM Initiative,” a Ryan White Special Projects of National Significance evaluation of the implementation of behavioral health models to improve HIV health outcomes for Black GBMSM [33, 34]. That evaluation did not invoke a specific framework such as the CFIR, however, there is a significant resonance between our findings and those studies. For example, our findings about clients’ desires for representation and about the need for flexibility in light of structural barriers, were echoed in their recent reports [35, 36]. Of note, the most common strategies across sites in this demonstration project were intensive case management and peer navigation, ideally provided by individuals with shared demographic characteristics and lived experience (i.e. Black GBMSM). While these strategies would likely be useful in our setting as well, our findings suggest that a combination of this type of tailored approach, together with larger systemic changes, would be needed for the successful integration of care.
Limitations
While we believe it is a strength to have conducted this qualitative study in two different clinics/ implementation environments, we acknowledge that each exists within a system of resources that may not be available in other settings. Additionally, we did not disaggregate our data summarization between the two clinics; the small overall sample size and particularly smaller sample size from Clinic B precluded this level of analysis. Our results could have been biased by differential refusal rates; we were not able to compare the characteristics of participants with non-participants due to a lack of information about the latter, and it is possible that those who agreed to participate in this study have different perceptions about benefits and limitations on care integration than those who did not.
Conclusions
Integrated HIV and behavioral health services can improve interdisciplinary collaboration in addressing the global needs of young Black GBMSM living with HIV. Our results identified systemic factors that impact the climate for enacting changes to improve outcomes within this key population, and specifically point to the need for multifaceted, bundled implementation strategies to optimize the chances of successful behavioral health and HIV care integration. Future research should develop and test these bundled implementation strategies to address complexity, priority, tension for change, and available resources, in order to improve care integration and ultimately improve physical and behavioral health outcomes for young Black GBMSM living with HIV.
Acknowledgements
We are grateful to the patients, providers, and staff members of Emory University Hospital-Midtown and the Grady Ponce Center, for their participation and open feedback. We also acknowledge Exceptional TBS Solutions and Ms. Audrey Dixon for the transcription of the qualitative data.
Contributor Information
Sophia A Hussen, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA; Division of Infectious Diseases, Department of Medicine, Emory University School of Medicine, Atlanta, USA.
Rosalind N Byrd, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA.
Kamini Doraivelu, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA.
Shamia J Moore, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA.
Daniel M Camp, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA.
Drew Wood-Palmer, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA.
Ameeta S Kalokhe, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA; Division of Infectious Diseases, Department of Medicine, Emory University School of Medicine, Atlanta, USA.
Eugene W Farber, Department of Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, USA.
Mohammed K Ali, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, USA; Department of Family and Preventive Medicine, Emory University School of Medicine, Atlanta, USA.
Funding Sources
This study was funded by the National Institutes of Health through a supplement to the Emory Center for AIDS Research (P30AI050409). The funder had no role in the conceptualization of the research, study design/conduct, collection, management interpretation of data, writing/review of the manuscript, and/or the decision to submit the article for publication.
Conflict of interest statement. All authors declare no conflicts of interest.
Human Rights
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was reviewed and approved by the Emory University IRB as well as the Grady Health System Research Oversight Committee.
Informed Consent
Informed consent was obtained from all participants included in the study.
Welfare of Animals
This article does not contain any studies with animals performed by any of the authors.
Transparency Statements
Study registration: This study was not formally registered. Analytic plan pre-registration: The analysis plan was not formally pre-registered. Analytic code availability: Analytic code used to conduct the analyses presented in this study is not available in a public archive. They may be available by emailing the corresponding author. Materials availability: Materials used to conduct the study are not publicly available.
Data Availability
De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.
References
- 1. Centers for Disease Control and Prevention. HIV Surveillance Report, 2019. 2021. https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-report-2018-updated-vol-32.pdf. (1 May 2023, date last accessed). [Google Scholar]
- 2. Hightow-Weidman L, LeGrand S, Choi SK. et al. Exploring the HIV continuum of care among young Black MSM. PLoS One 2017;12:e0179688. 10.1371/journal.pone.0179688 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Traeger L, O’Cleirigh C, Skeer MR. et al. Risk factors for missed HIV primary care visits among men who have sex with men. J Behav Med 2012;35:548–56. 10.1007/s10865-011-9383-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Centers for Disease Control and Prevention. HIV and African American Gay and Bisexual Men. 2022. https://www.hiv.gov/blog/new-factsheets-hiv-s-impact-african-american-community (1 May 2023, date last accessed). [Google Scholar]
- 5. Peek CJ. Integrated behavioral health and primary care: a common language. In: Talen MR, Burke Valeras A (eds.), Integrated Behavioral Health in Primary Care: Evaluating the Evidence, Identifying the Essentials. New York: Springer Science + Business Media, 2013. [Google Scholar]
- 6. Liu Y, Brown L, Przybyla S. et al. Characterizing racial differences of mental health burdens, psychosocial determinants, and impacts on HIV prevention outcomes among young men who have sex with men: a community-based study in two U.S. Cities. J Racial Ethn Health Disparities 2022;9:1114–24. 10.1007/s40615-021-01052-1 [DOI] [PubMed] [Google Scholar]
- 7. Quinn KG, Walsh JL, DiFranceisco W. et al. The inherent violence of Anti-Black racism and its effects on HIV care for Black sexually minoritized men. J Urban Health 2024;101:23–30. 10.1007/s11524-023-00823-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Vincent W, Peterson JL, Huebner DM. et al. Resilience and depression in young Black men who have sex with men: a social-ecological model. Stigma Health 2020;5:364–74. 10.1037/sah0000206 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Hussen SA, Doraivelu K, Camp DM. et al. Burden and correlates of mental health symptoms among young Black gay, bisexual, and other men who have sex with men engaged in HIV care in Atlanta. AIDS Behav 2022;26:2844–54. 10.1007/s10461-022-03629-1 [DOI] [PubMed] [Google Scholar]
- 10. Hussen SA, Easley KA, Smith JC. et al. Social capital, depressive symptoms, and HIV viral suppression among young Black, gay, bisexual and other men who have sex with men living with HIV. AIDS Behav 2018;22:3024–32. 10.1007/s10461-018-2105-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Magnus M, Jones K, PhillipsG, 2nd. et al. ; YMSM of color Special Projects of National Significance Initiative Study Group. Characteristics associated with retention among African American and Latino adolescent HIV-positive men: results from the outreach, care, and prevention to engage HIV-seropositive young MSM of color special project of national significance initiative. J Acquir Immune Defic Syndr (1999) 2010;53:529–36. 10.1097/QAI.0b013e3181b56404 [DOI] [PubMed] [Google Scholar]
- 12. Salomon EA, Mimiaga MJ, Husnik MJ. et al. Depressive symptoms, utilization of mental health care, substance use and sexual risk among young men who have sex with men in EXPLORE: implications for age-specific interventions. AIDS Behav 2009;13:811–21. 10.1007/s10461-008-9439-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Whiteley LB, Brown LK, Swenson R. et al. Disparities in mental health care among HIV-infected youth. J Int Assoc Provid AIDS Care 2014;13:29–34. 10.1177/2325957413488172 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Aggarwal R, Pham M, Dillingham R. et al. Expanded HIV clinic-based mental health care services: association with viral suppression. Open Forum Infect Dis 2019;6:ofz146. 10.1093/ofid/ofz146 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Chuah FLH, Haldane VE, Cervero-Liceras F. et al. Interventions and approaches to integrating HIV and mental health services: a systematic review. Health Policy Plan 2017;32:iv27–47. 10.1093/heapol/czw169 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Pence BW, O’Donnell JK, Gaynes BN.. Falling through the cracks: the gaps between depression prevalence, diagnosis, treatment, and response in HIV care. AIDS 2012;26:656–8. 10.1097/QAD.0b013e3283519aae [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Safren SA, O’Cleirigh C, Tan JY. et al. A randomized controlled trial of cognitive behavioral therapy for adherence and depression (CBT-AD) in HIV-infected individuals. Health Psychol 2009;28:1–10. 10.1037/a0012715 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Dombrowski JC, Halliday S, Tsui JI. et al. Adaptation of the collaborative care model to integrate behavioral health care into a low-barrier HIV clinic. Implement Res Pract 2023;4:26334895231167105. 10.1177/26334895231167105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Rodriguez-Hart C, Mackson G, Belanger D. et al. HIV and intersectional stigma reduction among organizations providing HIV services in New York city: a mixed-methods implementation science project. AIDS Behav 2022;26:1431–47. 10.1007/s10461-021-03498-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Damschroder LJ, Aron DC, Keith RE. et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Harris PA, Taylor R, Thielke R. et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Scott VC, Kenworthy T, Godly-Reynolds E. et al. The Readiness for Integrated Care Questionnaire (RICQ): an instrument to assess readiness to integrate behavioral health and primary care. Am J Orthopsychiatry 2017;87:520–30. 10.1037/ort0000270 [DOI] [PubMed] [Google Scholar]
- 23. Braun V, Clarke V.. Using thematic analysis in psychology. Qual Res Psychol 2008;3:77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]
- 24. Hennink M, Hutter I, Bailey A.. Qualitative Research Methods. London: SAGE Publications, 2011. [Google Scholar]
- 25. Powell BJ, Waltz TJ, Chinman MJ. et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci 2015;10:21. 10.1186/s13012-015-0209-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Waltz TJ, Powell BJ, Fernandez ME. et al. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci 2019;14:42. 10.1186/s13012-019-0892-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. McKinnon K, Lentz C, Boccher-Lattimore D. et al. Interventions for integrating behavioral health into HIV settings for US adults: a narrative review of systematic reviews and meta-analyses, 2010–2020. AIDS Behav 2024;28:2492–9. 10.1007/s10461-024-04324-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Camp DM, Moore SJ, Wood-Palmer D. et al. Preferences of young Black gay, bisexual, and other men who have sex with men regarding integration of HIV and mental health care services. AIDS Patient Care STDS 2022;36:356–63. 10.1089/apc.2022.0065 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Satre DD, Anderson AN, Leibowitz AS. et al. Implementing electronic substance use disorder and depression and anxiety screening and behavioral interventions in primary care clinics serving people with HIV: protocol for the Promoting Access to Care Engagement (PACE) trial. Contemp Clin Trials 2019;84:105833. 10.1016/j.cct.2019.105833 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Lea AN, Altschuler A, Leibowitz AS. et al. Patient and provider perspectives on self-administered electronic substance use and mental health screening in HIV primary care. Addict Sci Clin Pract 2022;17:10. 10.1186/s13722-022-00293-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Zimmer D, Staab EM, Ridgway JP. et al. Population-level portal-based anxiety and depression screening perspectives in HIV care clinicians: qualitative study using the consolidated framework for implementation research. JMIR Form Res 2024;8:e48935. 10.2196/48935 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Lopez CM, Goodrum NM, Brown TP. et al. The REACH (Resiliency, Engagement, and Accessibility for Comorbid HIV/PTSD/SUD populations) Protocol: using a universal screener to improve mental health and enhance HIV care outcomes. J Behav Health Serv Res 2023;50:452–67. 10.1007/s11414-023-09850-7 [DOI] [PubMed] [Google Scholar]
- 33. Hodge SE, Johnson-Turbes A, Flemming SSC. et al. ; Black MSM Initiative Study Group. Implementation of evidence-informed behavioral health models to improve HIV health outcomes for Black men who have sex with men (Black MSM Initiative): protocol for program evaluation. JMIR Res Protoc 2022;11:e36025. 10.2196/36025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Tinsley MJ, Cajina A.. Implementation of evidence-informed behavioral health models to improve HIV health outcomes for Black men who have sex with men: a Ryan White HIV/AIDS Program Part F special projects of national significance. AIDS Patient Care STDS 2022;36:S1–2. 10.1089/apc.2022.29010.intro [DOI] [PubMed] [Google Scholar]
- 35. NORC at the University of Chicago. Evidence -informed behavioral health models to improve HIV health outcomes for Black men who have sex with men initiative monograph. 2022. [DOI] [PMC free article] [PubMed]
- 36. Hodge SE, Passero M, Robinson D. et al. Prioritizing flexibility, human connection, and representation: findings from the qualitative process evaluation of the special projects of national significance Black men who have sex with men initiative. AIDS Patient Care STDS 2022;36:S21–7. 10.1089/apc.2022.0094 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.

