Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Jul 1.
Published in final edited form as: J Acquir Immune Defic Syndr. 2022 Jul 1;90(Suppl 1):S197–S205. doi: 10.1097/QAI.0000000000002980

Development of an implementation facilitation strategy to link mental health screening and eHealth intervention for clients in Ryan White-funded clinics in Chicago

Tammy Stump 1, Kristen Ethier 2, Lisa Hirschhorn 1, Andrea Dakin 3, Nora Bouacha 3, Angela Freeman 1, Jacqueline Bannon 1, Walter Gómez 4, Judith Moskowitz 1, Alida Bouris 2,5
PMCID: PMC9204837  NIHMSID: NIHMS1791074  PMID: 35703772

Abstract

Background:

To reduce the impact of depression on PLWH, we are implementing a clinic-based behavioral health screener and referral to ORCHID (Optimizing Resilience and Coping with HIV through Internet Delivery), an evidenced-based intervention. We used the Consolidated Framework for Implementation Research (CFIR) to identify contextual barriers and facilitators in advance of implementation.

Setting:

16 Chicagoland area Ryan White Medical Case Management (RWMCM) sites.

Methods:

We conducted a sequential mixed-methods study with medical case managers and supervisors. Participants completed an online survey assessing CFIR domains, scored on a 1 (strongly disagree) to 5 (strongly agree) scale. Survey results informed a purposive sampling frame and interview protocol. Interviews were analyzed by rapid qualitative analysis.

Results:

On average, survey respondents (n=58) slightly agreed with positive views of team culture, learning climate, and implementation readiness (Ms=3.80–3.87). Potential barriers included intervention complexity (M=3.47), needed human resources (Ms=2.71–3.33), and only slight agreement with relative advantage over existing screening/referral systems (Ms=3.09–3.71). Qualitative results (n=15) identified low advantage for clinics with robust behavioral health systems but strong advantage in clinics without these services. Respondents identified system-wide training and monitoring strategies to facilitate implementation.

Conclusion:

RWMCM sites are a generally favorable context for implementation of the interventions. As illustrated in an implementation research logic model, barriers will be addressed through deploying strategies proposed to impact clinic-and individual-level outcomes, including electronic prompts (reduce complexity), training on ORCHID as a complement to other behavioral health services (increase relative advantage), and feedback during implementation (strengthen rewards/incentives).

Keywords: HIV, Positive Affect, Depression, Screening, Implementation

Introduction

Depression is highly prevalent among people living with HIV (PLWH),1, 2 and is associated not only with a lower quality of life but also with HIV progression, lower retention in care, and a lower likelihood of viral suppression.36 ORCHID (Optimizing Resilience and Coping with HIV through Internet Delivery) is an evidence-based intervention that lowers symptoms of depression by teaching skills that help individuals experience positive emotions, even during stressful times.7 Although prior research found ORCHID effective at reducing depression symptoms and decreasing viral load,79 it has not been evaluated in a clinic setting. In order to leverage ORCHID to effectively address depression among PLWH and help realize the goals of the National Ending the HIV Epidemic Plan,10 we are conducting research to guide, and later evaluate, its implementation in the context of a Hybrid Type 2 stepped wedge cluster-randomized trial. In partnership with AIDS Foundation Chicago (AFC), we will roll-out ORCHID in Ryan White Medical Case Management (RWMCM)11 sites in the Chicagoland area in tandem with their launch of a behavior health screener (BHS; i.e., mental health and substance use), through which clients with elevated depression symptoms will be referred to ORCHID.

The BHS developed by AFC includes standardized assessments on: depression (PHQ-912), anxiety (GAD-713), post-traumatic stress symptoms (PCL-C14), alcohol use (AUDIT15), and substance use (DAST-1016). The BHS was designed to be administered by medical case managers (MCMs), i.e., trained staff who provide patient-centered services to eliminate barriers and promote engagement with HIV-related medical care.1719 In early 2019, AFC launched a pilot to better understand the behavioral health needs of clients receiving RWMCM services. Six subcontracted agencies piloted the BHS; of 777 eligible clients, 374 (48.1%) were offered the screener. Of the 282 who completed the PHQ-9, 33.3% (n=94) had scores ≥5, indicating clinically significant depressive symptomatology. Qualitative data collected during the pilot indicated that staff were concerned that available mental health services were not sufficient (e.g., were too costly, had long wait times) to meet client needs identified through the screener.19

By virtue of its online, self-guided format, ORCHID addresses several barriers to receiving mental health services. ORCHID is an evidence-based intervention that consists of eight empirically-supported skills that increase positive affect (i.e., positively-valenced feelings, e.g., calm, satisfied, excited, thrilled), even in the face of stress:2026 (1) noting daily positive events; (2) savoring positive events; (3) gratitude; (4) mindfulness; (5) positive reappraisal; (6) focusing on personal strengths; (7) setting and working toward attainable goals; and (8) self-compassion.27 Among PLWH, positive affect has been linked to a number of benefits including lower depression levels,28 slower disease progression,29, 30 higher likelihood of viral suppression,31 and lower risk of mortality.27 The intervention has demonstrated feasibility, acceptability, and efficacy in pilot tests and RCTs among multiple clinical samples, including people with depression and PLWH.79, 27 Specifically, RCTs conducted among PLWH have found that participants in the in-person version of ORCHID reported higher levels of past day positive affect, were less likely to use antidepressants, and were more likely to be virally suppressed.8, 9 A pilot of the online self-guided version of ORCHID showed increased positive emotions and decreased depressive mood in PLWH.7

In the present trial, MCMs will refer all clients with PHQ-9 scores ≥5 to ORCHID through a brief interest form and/or distribution recruitment materials (e.g., online flyer). In the pre-implementation work reported here, we conducted a needs assessment to identify facilitators and barriers that could be used to guide implementation strategies to support the implementation of the BHS+ORCHID in the RWMCM system.

Methods

We conducted a sequential mixed-methods study with Medical Case Managers (MCMs) and supervisors in the RWMCM system in Chicago. Surveys and interviews were guided by the Consolidated Framework for Implementation Research (CFIR),32 a meta-theoretical framework focused on five domains important for implementing a new practice/intervention. We synthesized these data in an Implementation Research Logic Model (IRLM), a semi-structured tool designed to increase rigor and reproducibility by mapping the relationships between implementation determinants, strategies, mechanisms of influence, and outcomes.33

Participants

Survey and interview participants were MCMs and MCM supervisors at 16 RWMCM sites in the Chicagoland area. Eligibility criteria were (1) current employment as a MCM or Supervisor, (2) age 18 or older, and (3) ability to read or write in English.

Survey

Recruitment and Data Collection

All current MCMs and supervisors from the 16 RWMCM sites with co-located clinical services were emailed an invitation to participate in a survey about implementing the BHS+ORCHID. After completing an online consent form, participants were linked to a REDCap survey.34 All participants received a $20 e-gift card as compensation.

Measures

Survey items were selected and adapted from existing research35 (e.g., the Practice Adaptive Reserve Scale,36 ARTAS implementation evaluation37) and supplemented with several items written specifically for this project. CFIR constructs included in the survey were selected by the interdisciplinary study team (which included AFC staff and social/health psychology, social work, nursing, implementation science and public health researchers) based on findings from the BHS pilot and prior experiences working with MCMs and PLWH. Unless noted, all items were scored from 1=strongly disagree to 5=strongly agree.3537 See Tables 1 and 2 for specific items and internal consistency of included scales.

Table 1.

Outer and inner setting characteristics of 16 Ryan White HIV clinics as reported in a survey of medical case management staff

Domain Endorsement

Outer Setting – i.e, factors exogenous to the implementation setting
Behavioral Health Needs: How many of your clients do you think struggle with: [1=None to 5=All] M (SD)
 Depression 3.69 (.82)
 Anxiety 3.66 (.89)
 Substance Use 3.12 (.82)

 None 2.00 (.85)

Daily Technology Access (estimate of % clients with each resource)
 Smartphone (n=54) 62.59%
 Tablet (n=53) 22.08%
 Computer (n=53) 29.62%
 None (n=52) 17.69%
Reporting Requirements (n=15, Supervisors only): Is your clinic required to report any of the following to an outside organization (e.g., CDPH, IDPH, HRSA)? [% Yes]
 Doing Behavioral Health Screener 60.00%
 Results of Behavioral Health Screener 40.00%
 Referrals to Behavioral Health Services 66.70%
 Outcomes of Quality Improvement Projects 66.70%

Incentives Present (n=15, Supervisors only)3
 Public Recognition 26.70%
 Payment 6.70%
 Positive End-of-Year Score 46.70%

Inner Setting - i.e., factors within the setting of intervention implementation
Available Resources: would be problematic because we do not have effective referrals or other supportive care resources to care for any more PLWH with behavioral health needs.(Rev) 3.16 (1.21)
Responsiveness to Client Needs:36 7 items; α=.81 e.g., This team does a good job assessing client needs and expectations. 3.92 (.60)
Overall Culture:36 9 items; α=.89; e.g., People at all levels openly talk about what is and isn’t working. 3.80 (.60)

Culture Stress:36 4 items; α=.85; e.g., I am under too many pressures to do my job effectively. (Rev) 3.06 (.85)

Learning Climate:36 4 items; α=.85; e.g., We regularly take time to consider ways to improve how we do things. 36 3.87 (.68)
Implementation Readiness (leaders):36 4 items; α=.92 e.g., Leadership strongly supports change efforts. 36 3.83 (.79)

Compatibility1: BHS is compatible and consistent with the behavioral health needs of my clients.37 3.97 (.46)

Compatibility2: ORCHID is compatible and consistent with the behavioral health needs of my clients.37 3.69 (.73)
Compatibility3: ORCHID is compatible and consistent with the behavioral health needs of my clients with depression.37 3.71 (.70)

Notes. n=58 unless otherwise noted. For incentives and reporting requirements, percentages reports are the percentage of supervisors who replied Yes. All other variables rated on 1–5 scales; higher values indicate a more positive environment for implementation. Items without a citation were written by study authors.

Abbreviations. Rev=Reverse-scored; higher scores indicate a more positive environment for implementation, (e.g., less stressful work culture)

Table 2.

Intervention Characteristics and Characteristics of Individuals as reported in a survey of medical case management staff at 16 Chicagoland Ryan White HIV clinics.

Domain M (SD)

Intervention Characteristics
BHS
 Adaptability: could be administered in a way that is easily adapted to fit the needs of the team.37 3.84 (.67)
 Complexity1: would be easy to understand and use after training.37 4.00 (.62)
 Complexity2: is too complex.37 (Rev) 3.47 (.88)
 Cost1: would take too much time and effort.37 (Rev) 3.07 (1.06)
 Cost2: is something I would only be interested in adopting in the future if funding was provided for it.37 (Rev) 2.71 (.99)
 Relative Advantage1: would be successful in linking clients living with HIV and behavioral health needs to services.37 3.71 (.84)
 Relative Advantage2: is unnecessary because my clinic already screens for behavioral health issues. (Rev) 3.17 (.96)
ORCHID
 Adaptability: would be difficult to adapt to meet the needs of different PLHV populations with behavioral health needs.37 (Rev) 3.41 (.80)
 Cost: would take too much time and effort.37 3.33 (.85)
 Relative Advantage: would be more effective than methods we are currently using to address the behavioral health needs of clients living with HIV.37 3.09 (.78)
Characteristics of Individuals (Knowledge/Beliefs about Intervention Benefits)
BHS
 wouldn’t really work for my clients, even though it was shown to be effective in other agencies.37 (Rev) 3.64 (.87)
 would be difficult to administer because it is uncomfortable to ask questions about substance use or mental health.37 (Rev) 3.91 (.88)
 would have a visible and substantial impact on the health status of my clients.37 3.71 (.70)
 is something that would benefit clients living with HIV.37 4.12 (.60)
 is problematic because asking about suicidality or depression could trigger these symptoms.37 (Rev) 4.05 (.83)
ORCHID
 wouldn’t really work for my clients, even though it was shown to be effective in research trials.37 (Rev) 3.55 (.77)
 would be less effective at addressing behavioral health needs and improving viral suppression among my clients than it was in the original research studies.37 (Rev) 3.56 (.73)

 would have a visible and substantial impact on the health status of my clients.37 3.62 (.64)
 Is something that would benefit clients living with HIV.37 3.97 (.53)

Notes. Items were rated on 1–5 scales with higher values indicating_a more positive environment for implementation. Items without a citation were written by study authors. All knowledge/belief items were scored such that higher values indicated more favorable views of the intervention.

Abbreviations. BHS=Behavioral Health Screener; Rev=Reverse-scored item.

Quantitative Analysis

Subscales for each construct were created and descriptive statistics (mean, standard deviation, range) were computed, both overall and for each clinic. All items and subscales were scored such that higher values indicated a more positive environment for implementation.

Key Informant Interviews

Recruitment and Data Collection

After analyzing survey data, MCMs and supervisors, i.e., key informants, were purposively sampled to ensure roughly equal representation of: (1) clinics with low and high means on CFIR constructs, and (2) length of employment in case management (under and over 5 years). Staff selected for inclusion (n=24) were emailed an invitation, and, if interested, were linked to an online consent form and scheduled for an interview. All interviews were conducted and recorded via a video conferencing service (i.e., Zoom). Interviews lasted about 75 minutes (Range: 36–100 minutes; the shortest interview was with a seasoned supervisor who gave succinct answers) and each participant received a $50 e-gift card as compensation.

The team developed separate MCM and Supervisor interview guides that explored: (1) implementing the BHS, (2) implementing the referral to ORCHID, and (3) the overall implementation process. Questions were drawn from all five CFIR domains32 and selected to elicit details on key survey findings, e.g., perspectives on the relative advantage of the BHS+ORCHID. Before each section, interviewers provided descriptive information about the BHS and ORCHID. The guides were further refined after pilot interviews with two AFC staff who were former MCMs and Supervisors (See Supplementary Table 1 for sample questions). After each interview, interviewers completed a field note to capture relevant themes and follow-up areas.38

Qualitative Analysis

After interviews were professionally transcribed, five coders analyzed the data using a four-step Rapid Qualitative Analysis (RQA) approach.39 First, we developed a structured template mapping the relationships between major analytic domains, interview questions, and CFIR constructs. After piloting the template with a single interview to refine usability and inter-rater reliability, each coder independently analyzed 3–4 transcripts. The team then met to discuss key findings and to develop an analytic matrix of deductively derived codes, at which point coders independently synthesized key barriers, facilitators, and quotes, along with relevant CFIR constructs and potential implementation strategies. This approach was supplemented with open coding40 to capture themes not included in the matrix. The analysts met a final time to evaluate overall agreement in coding, with coding differences resolved via discussion.

All procedures were reviewed by the Institutional Review Board at Northwestern University, which deemed the study as not human subjects research.

Synthesis and Application using Implementation Research Logic Model

Survey and interview responses were converted to CFIR ratings using the following schema: Ms 1–1.499=−2 (strong barrier); Ms 1.5–2.499=−1 (moderate barrier); Ms 2.5–3.499=0 (neutral); Ms 3.5–4.499=+1 (moderate facilitator); Ms 4.5–5=+2 (strong facilitator); “mixed” (+/−) if > 2 sites had an opposite-valenced responses to the modal response. When key informants addressed constructs not captured in the surveys, the team devised a rating through discussion. In an iterative discussion-based process, our interdisciplinary team used these findings to select relevant implementation strategies. To support broader efforts to systematize implementation strategies, we also mapped our strategies on to those identified in the Expert Recommendations for Implementing Change (ERIC).41 We then developed an initial IRLM that included CFIR ratings of relevant contextual determinants, implementation strategies, and potential mechanisms of influence important for understanding the planned implementation and effectiveness outcomes (See Figure 1).

Figure 1.

Figure 1.

Implementation Research Logic Model for BHS+ORCHID

Results

Participants

In total, n=58 MCMs and supervisors from all 16 clinics completed the survey (70% response rate), and, of 24 invited, n=9 MCMs and n=6 Supervisors (n=15; 62.5% response rate) from 11 clinics completed an interview. Mean years in case management was 4.7 for MCMs, and 7.4 for supervisors. About half (43%) of respondents reported education beyond a bachelor’s degree and 11% of respondents reported participation in the BHS pilot.

Outer Setting

In surveys (see Table 1), depression was identified as the most common behavioral health concern. Most staff perceived that clients had daily internet access, with the most common device being a smartphone. Most clinics had some incentives and reporting requirements related to behavioral health screening, but less than half were required to report the results of these screenings.

Elaborating on survey findings, MCMs described serving clients with PTSD, psychosis, homelessness, and high levels of resource instability, in addition to depression and anxiety. All informants were frustrated by the lack of behavioral health services in the community, and access to care was especially limited for non-English speaking clients (Row 1, Table 3/Supplementary Table 2). MCMs feared that stigma could negatively influence clients’ willingness to provide truthful answers to the BHS or that clients would be frustrated by responding to BHS questions when their primary need was resource counseling. Staff also were concerned that high mental health acuity could be a barrier to using ORCHID—a concern heightened by the pandemic. In contrast to survey findings, MCMs perceived that client access to technology could be a significant barrier to using ORCHID. In addition, some staff indicated that their clients might not be motivated to use ORCHID, especially if their primary presenting problem was housing or food insecurity (Row 2, Table 3/Supplementary Table 2).

Table 3:

Illustrative Quotes from In-depth Interviews with Medical Case Managers (MCM: n=9) and Supervisors (SUP: n=6), Chicagoland Ryan White Medical Case Management System

Theme Select Quotations
High BH Needs and Low Availability of Services MCM: “…you do encounter waitlists…you’ve no idea how much longer it’ll be between now and them being able to receive the treatment they deserve.”
MCM: “…it’s harder for our Latino population because there’s not a lot of people out there that speak Spanish…”
SUP: “We’ll be referring people to our case managers and right now…every single therapist that we work with is full…”
Client Barriers to Accessing ORCHID MCM: “…a lot of my clients, they have a phone, they don’t have a phone, they have service, they don’t have service. Some…are currently jobless…or houseless, because of the pandemic…or even other reasons outside of their control…”
MCM: “Some people might take it as an offense. It can be really offensive. ‘Why do you offer this to me? I’ve never asked you. Did I tell you that I need the service? Do I look like I’m crazy?’”
MCM: “The barriers with language, will it be in Spanish and English and Russian and Aramaic..? How will that be solved?”
Team Culture MCM: “We’ve gotten hit with so many new things within this past year alone that we can just handle it no problem. We try to be as resourceful as we can be. That’s a really big value to us…being resourceful, empathetic…”
MCM: “We’re a really tight team…if we find a problem, we’ll talk about it, and then we always reach out to each other, we reach out to our supervisor. I don’t think we’ve seen anything that we’re like ‘Oh no, we can’t do that’…”
Variability in BH screening across clinics MCM: “We don’t have a set screening right now, so it’s really just in conversation.”
MCM: “I offer those resources to them in house here at my clinic. We have a mental health specialist. She and I have been working together. Oftentimes, with client permission, we may do our assessments together initially…”
Relative Advantage of BHS MCM: “…it seems like it’ll give me a wider scope to talk about the different types of assessments,…even though I’m not a mental health specialist, this might help me really refine my referrals.”
MCM: “I think it would just formalize the process. Right now, it’s very informal.”
SUP: “From an agency standpoint, leadership may say, “Well, why do we need to do that if we already have a screener…?”
Flexible and adaptable Implementation MCM: “They’re going to react differently…Some people don’t care and some…might get kind of insulted…if people have taken it three times, maybe we can go to once a year.”
Enthusiasm for ORCHID MCM: “Well, one, it’s self-empowering, so it’s… they’ll make time for themselves as opposed to having somebody else make time for them… I think that it’s very, like I said, empowering people to see what resources they have within themselves to address whatever issues and concerns they have, and needs that they have. I think in that regard, it would definitely be beneficial that they would have that on their own scheduling.”
MCM Role Compatibility MCM: “We have doctors who are there for years and they know patients for years so of course, they know more about them. they know better than us.”
MCM: “A lot of times we hear a lot of positive feedback from our clients saying, ‘Oh wow, even my therapist is not doing this job this good.’”
Perceptions of “ideal” clients MCM: “I wouldn’t refer somebody to [ORCHID] that didn’t really have the mental health need. If somebody is in need of these more intense services… with these skills, I feel like the interest would be there.”
Implementation expectations and boundaries MCM: “[A] lot of my clients are Latino and they do have technology issues, but if I see that it’s something that’s going to help them…I could always help them.”
SUP: “I will set up time to meet with each of my staff individually to review it after they…practice it. I would actually mandate that they set a specific time, like an hour where they’re actually going to sit down and review it
Training Needs MCM: “First, [training on] how to initiate these conversations…with our clients, and how to encourage our clients…how to make it simple for them.”
MCM: “I think more of the background on how [ORCHID] was developed and the fact that it is based in real science, and that there are a lot of other online tools that work for people. This is just going to be another one that, hopefully, probably, will also work.”
Recommended strategies MCM: “Hopefully, there’s, like, a support service or something. You know, like, we have problems with AFC, we call AFC. We have a problem with something, we could email you.”
SUP: “Training for the case managers on start to finish, from screener to all the things involved with ORCHID. And then, release good marketing materials, so the case managers have language around why it would benefit their clients.”
SUP: “… for my team, I think they find value in when we run our monthly reports and we look at the dashboards and people see--I color code them with green for current and yellow for due and orange for overdue…I think they love to see all the green pop up on their screens.”

Abbreviations. BH=Behavioral Health; BHS=Behavioral Health Screener; MCM=Medical Case Manager; SUP=Supervisor

Inner Setting

Survey responses (see Table 1) indicated that teams were perceived as being responsive to client needs, with a supportive culture, learning climate, and leadership that exhibited implementation readiness (e.g., supports change efforts). However, respondents also expressed moderate agreement that a stressful culture existed within teams, and low rates of effective referrals and supportive care resources to serve clients.

Consistent with survey findings, almost every key informant described having a strong team culture characterized by high levels of communication, a commitment to supporting client well-being, and a willingness to try new things (Row 3, Table 2/Supplementary Table 2). However, there was significant variability in how clinics screened for behavioral health, pointing to variable compatibility and tension for change across the system (Row 4, Table 2/Supplementary Table 2). For example, some staff described having robust screening systems, close communication and collaboration with behavioral health staff, and in-house behavioral health services. In these clinics, staff were concerned about how the BHS+ORCHID would affect their current workflow. In contrast, others described being in clinics with less systematic screening approaches and less communication with behavioral health teams. Staff in these clinics reported less difficulty with integrating the BHS into their workflow (Row 4, Table 2/Supplementary Table 2). Additional facilitators in the inner setting included access to private space, having engaged clinic leaders, and incentives in the form of public recognition at both clinic and system-wide meetings--resources not currently present in all clinics.

Intervention Characteristics

As shown in Table 2, both the BHS and ORCHID were viewed as adaptable. However, they were also viewed as being somewhat complex and costly, with only slight agreement that the BHS+ORCHID had a relative advantage compared to existing systems for screening and referrals for mental health treatment. Qualitative data revealed that staff in clinics with weaker screening systems believed that the BHS would help them to better understand, refer, and support their clients, but those in clinics with robust systems were uncertain about what the BHS would add (Row 5, Table 3/Supplementary Table 2). Staff emphasized the need for a flexible implementation process that would allow them to deliver client-centered services and offered suggestions such as screening after the initial intake visit, less frequent screening, or collaborating with on-site behavioral health teams (Row 7, Table 3/Supplementary Table 2). Across interviews, the referral to ORCHID was viewed as similar to making other referrals, suggesting low complexity and good compatibility. Overall, there was high enthusiasm for incorporating ORCHID as a referral option, with many staff noting that an online intervention could help to address the lack of community-based behavioral health services (Row 6, Table 3/Supplemental Table 2).

Characteristics of Individuals

Survey data indicated that MCMs and Supervisors agreed that both the BHS and ORCHID would be effective (see Table 2). Across interviews, most MCMs recognized that they were in a unique position to implement a BHS, as they had close relationships with clients. However, some MCMs perceived that it was outside their role to conduct a BHS and believed that medical/behavioral health clinicians were better qualified, suggesting mixed levels of self-efficacy (Row 8, Table 3/Supplementary Table 2). Some MCMs also had a mental image of an “ideal” client for ORCHID, e.g., someone who did not have high levels of mental health distress. MCMs also were disappointed to learn that ORCHID was only available in English (Row 9, Table 3/Supplementary Table 2). Whereas some MCMs were concerned about guiding clients through the uptake and completion of ORCHID, others were eager to help clients access and complete ORCHID, even offering to sit with their clients and translate the material into Spanish (Row 10, Table 3/Supplementary Table 2).

Process

Key informants had several suggestions for ensuring a successful implementation process (survey items did not assess this domain). All identified interactive trainings with opportunities to ask questions and practice with one another as critical for preparing them to screen and respond appropriately, especially for addressing potential stigma and/or reluctance to engage with mental health services. Importantly, all informants noted that AFC had a strong history of providing engaging training and effective technical support. Both MCMs and Supervisors thought it was important to share high-quality marketing materials across platforms, e.g., pamphlets/brochures distributed at meetings, email, and social media (Row 11, Table 3/ Supplementary Table 2). Given variability in screening and referral systems across the RWMCM system, staff also recommended engaging both clinic and behavioral health leaders in the implementation process. Reflecting and evaluating during the implementation process was identified as vital: supervisors recommended folding this into regular team meetings, with data support from AFC, and both groups recommended using a dashboard system to provide direct feedback on BHS reach and fidelity. During implementation, informants also wanted to learn from their colleagues and to share feedback with AFC leaders and the research team (Row 12, Table 3/Supplementary Table 2).

Discussion

Implementation research is critical for supporting the adoption, scale-up, and sustainment of evidence-based interventions in clinical care settings serving PLWH.42, 43 Through quantitative and qualitative data, we identified a number of important determinants that will guide the implementation of the BHS+ORCHID. MCMs and supervisors perceived a strong need for additional client services, and the BHS was viewed as a valid tool to screen for clients’ behavioral health needs (i.e., high compatibility). Yet MCMs also described potential barriers in the inner and outer settings, including redundancy with some clinic’s existing workflows (i.e., low relative advantage and compatibility), limited time and resources to conduct the screening and any needed follow-up (i.e., high human resource costs), and a need to address mental health stigma and clients’ immediate resource needs. Whereas ORCHID was viewed as a way to address the lack of immediately accessible referral options, some informants were concerned about access for clients with technological or linguistic barriers. Nevertheless, staff highlighted strong team cultures as well as training and leadership from AFC on prior initiatives as key predictors of implementation success; they also recommended several strategies to support implementation.

The identified barriers and facilitators are consistent with prior research on the implementation of behavioral health screenings and mHealth interventions in diverse care settings, including primary care clinics,44 federally qualified health centers,45 and clinics serving people with chronic health conditions. Within the CFIR, implementation readiness is a multidimensional construct that refers to the “tangible and immediate” indicators of an organization’s commitment to the decision to implement a new intervention.32 Both survey and interview data pointed to a strong prior history of AFC leadership engagement, support, training and technical assistance to implement new programs and initiatives within the RWMCM, strategies correlated with implementation readiness in prior research.32, 46, 47

In order to address relevant barriers and to strengthen implementation readiness and other facilitators, we selected an initial set of strategies to increase MCM awareness, knowledge, and self-efficacy and to support identified implementation outcomes (see IRLM in Figure 1). During the pre-implementation phase, we will release educational materials in multiple modalities and conduct a series of dynamic trainings. Trainings will include content on behavioral health among PLWH and information on screening and referral procedures. Given identified stigma concerns, trainings also will provide practice opportunities to increase MCM self-efficacy to address client reticence to discuss mental health. We also will identify clinic and system-wide champions who can share their positive experiences during meetings and trainings. Finally, trainings and materials will highlight ORCHID as a complement that may support clients as they wait for other services. Trainings will be provided before and during implementation. During implementation, we will refine and implement quality monitoring tools and systems. These strategies will enable the team to audit and provide feedback on the BHS+ORCHID to each clinic. Following staff recommendations, feedback will be given via a dashboard system that also provides electronic prompts. Technical assistance will be provided, with AFC supporting the BHS and the research team supporting ORCHID. As implementation proceeds in each clinic, the research team will conduct additional mixed-methods research to identify actual barriers and facilitators and to evaluate and adapt strategies as needed.48 Figure 1 and Supplementary Table 3 provide a complete list of planned implementation strategies, mapped to ERIC strategies.

Strengths and Limitations

Study strengths include the use of the CFIR to guide data collection and analysis,32,49 use of validated CFIR measures,35, 36 strong sampling methods, and the sequential mixed-methods study design. However, the study is not without limitations. The scales used to measure CFIR constructs assessed extent of agreement that specific potential barriers and facilitators (e.g., implementation climate) were present3537 but not whether these factors were perceived as being barriers, facilitators, or as not relevant (neutral) to implementing BHS+ORCHID. A different measurement strategy may have yielded different insights into perceived barriers or facilitators. Future mixed-methods research during the trial will provide additional data on how and whether these determinants operate as actual barriers or facilitators. Although MCMs and supervisors were recruited from across the RWMCM system, not all invitees opted to participate and few provided a reason for declining. It is possible that staff with the most burdensome schedules and/or least interest chose not to participate. Nevertheless, we obtained survey responses from all eligible clinic sites and reached coding saturation after 15 interviews. Lastly, data collection took place between November 2020 and March 2021–a time period when the COVID-19 pandemic likely resulted in changes to practice with long-term implications of these plans being unknown.

We additionally acknowledge that our implementation plans will not ameliorate all barriers for PLWH who could benefit from ORCHID, e.g., structural inequalities, dearth of community-based behavioral health services, technology access, and access for non-English speaking clients. Although we partially address technology access by optimizing ORCHID for mobile device delivery and by suggesting community resources through which clients can regularly access the internet, additional research is needed to adapt ORCHID for non-English speaking persons. Spanish-speaking versions of positive affect interventions have been developed and translation is a high priority for future iterations of ORCHID.50 We also do not explicitly intervene at the outer setting level, as would be ideal, due to limitations in project scope and resources. Future research should incorporate health equity frameworks in implementation science research on the outer setting, which may help to improve intervention reach.51, 52 We will track variation in outcomes by demographic categories, reasons for unsuccessful screenings of interested study participants, and reasons eligible participants cannot access ORCHID. These data will help to better understand outer setting barriers and may suggest the selection of future implementation strategies to address these barriers.

Conclusions

RWMCM sites are a generally favorable context for implementation of interventions. Identified barriers will be addressed through deploying implementation strategies proposed to impact clinic-and individual-level outcomes, including electronic prompts (reduce complexity), training on ORCHID as a complement to other behavioral health services (increase relative advantage), and feedback during implementation (strengthen rewards/incentives).

Supplementary Material

Supplementary Table 1
Supplementary Table 2
Supplementary Table 3

Conflicts of Interest and Source of Funding:

This research was funded by the National Institutes of Mental Health (NIMH) R01MH124632-01.

Research reported in this publication was also supported by the National Institute On Minority Health And Health Disparities of the National Institutes of Health under Award Number T37MD014248. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnotes

The authors report no conflicts of interest.

References

  • 1.Do AN, Rosenberg ES, Sullivan PS, et al. Excess burden of depression among HIV-infected persons receiving medical care in the United States: data from the medical monitoring project and the behavioral risk factor surveillance system. PloS one. 2014;9(3):e92842. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Glynn TR, Safren SA, Carrico AW, et al. High levels of syndemics and their association with adherence, viral non-suppression, and biobehavioral transmission risk in Miami, a US City with an HIV/AIDS epidemic. AIDS and Behavior. 2019;23(11):2956–2965. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Bhatia R, Hartman C, Kallen MA, Graham J, Giordano TP. Persons newly diagnosed with HIV infection are at high risk for depression and poor linkage to care: results from the Steps Study. AIDS and Behavior. 2011;15(6):1161–1170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Ickovics JR, Hamburger ME, Vlahov D, et al. Mortality, CD4 cell count decline, and depressive symptoms among HIV-seropositive women: longitudinal analysis from the HIV Epidemiology Research Study. Jama. 2001;285(11):1466–1474. [DOI] [PubMed] [Google Scholar]
  • 5.Leserman J, Pence BW, Whetten K, et al. Relation of lifetime trauma and depressive symptoms to mortality in HIV. American Journal of Psychiatry. 2007;164(11):1707–1713. [DOI] [PubMed] [Google Scholar]
  • 6.Mayne TJ, Vittinghoff E, Chesney MA, Barrett DC, Coates TJ. Depressive affect and survival among gay and bisexual men infected with HIV. Archives of internal medicine. 1996;156(19):2233–2238. [PubMed] [Google Scholar]
  • 7.Bassett S, Cohn M, Cotten P, Kwok I, Moskowitz J. Feasibility and acceptability of an online positive affect intervention for those living with comorbid HIV depression. AIDS and Behavior. 2019;23(3):753–764. [DOI] [PubMed] [Google Scholar]
  • 8.Moskowitz JT, Carrico AW, Duncan LG, et al. Randomized controlled trial of a positive affect intervention for people newly diagnosed with HIV. Journal of consulting and clinical psychology. 2017;85(5):409. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Carrico AW, Neilands TB, Dilworth SE, et al. Randomized controlled trial of a positive affect intervention to reduce HIV viral load among sexual minority men who use methamphetamine. Journal of the International AIDS Society. 2019;22(12):e25436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Fauci AS, Redfield RR, Sigounas G, Weahkee MD, Giroir BP. Ending the HIV epidemic: a plan for the United States. Jama. 2019;321(9):844–845. [DOI] [PubMed] [Google Scholar]
  • 11.(HRSA) HRaSA. About the Ryan White Program. Accessed January 22, 2020, https://hab.hrsa.gov/about-ryan-white-hivaids-program/about-ryan-white-hivaids-program
  • 12.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Journal of general internal medicine. 2001;16(9):606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kroenke K, Spitzer RL, Williams JB, Monahan PO, Löwe B. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Annals of internal medicine. 2007;146(5):317–325. [DOI] [PubMed] [Google Scholar]
  • 14.Prins A, Bovin MJ, Smolenski DJ, et al. The primary care PTSD screen for DSM-5 (PC-PTSD-5): development and evaluation within a veteran primary care sample. Journal of general internal medicine. 2016;31(10):1206–1211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA, Project ACQI. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Archives of internal medicine. 1998;158(16):1789–1795. [DOI] [PubMed] [Google Scholar]
  • 16.Cocco KM, Carey KB. Psychometric properties of the Drug Abuse Screening Test in psychiatric outpatients. Psychological Assessment. 1998;10(4):408. [Google Scholar]
  • 17.Ryan White HIV/AIDS Treatment Modernization Act: U.S. Department of Health and Human Services (2006).
  • 18.López JD, Shacham E, Brown T. The impact of the Ryan White HIV/AIDS medical case management program on HIV clinical outcomes: a longitudinal study. AIDS and Behavior. 2018;22(9):3091–3099. [DOI] [PubMed] [Google Scholar]
  • 19.Beebe J BN, Dakin A. Piloting a Client Behavioral Health Screener with HIV Case Managers: Challenges, Successes and Next Steps. 2021: [Google Scholar]
  • 20.Folkman S Positive psychological states and coping with severe stress. Social Science and Medicine. 1997;45:1207–1221. [DOI] [PubMed] [Google Scholar]
  • 21.Folkman S, Moskowitz JT. Positive affect and the other side of coping. American Psychologist. 2000;55(6):647–654. doi: 10.1037//0003-066x.55.6.647 [DOI] [PubMed] [Google Scholar]
  • 22.Fredrickson BL. What good are positive emotions? Review of General Psychology. 1998;2:300–319. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Fredrickson BL, Cohn MA, Coffey KA, Pek J, Finkel SM. Open hearts build lives: Positive emotions, induced through meditation, build consequential personal resources. Journal of Personality and Social Psychology. 2008;95:1045–1062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Tice DM, Baumeister RF, Shmueli D, Muraven M. Restoring the self: Positive affect helps improve self-regulation following ego depletion. Journal of Experimental Social Psychology. 2007;43:379–384. [Google Scholar]
  • 25.Wichers MC, Myin-Germeys I, Jacobs N, et al. Evidence that moment-to-moment variation in positive emotions buffer genetic risk for depression: a momentary assessment twin study. Research Support, Non-U.S. Gov’t Twin Study. Acta Psychiatr Scand. Jun 2007;115(6):451–7. doi: 10.1111/j.1600-0447.2006.00924.x [DOI] [PubMed] [Google Scholar]
  • 26.Zautra AJ, Johnson LM, Davis MC. Positive affect as a source of resilience for women in chronic pain. Research Support, Non-U.S. Gov’t. J Consult Clin Psych. Apr 2005;73(2):212–20. doi: 10.1037/0022-006X.73.2.212 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Moskowitz JT. Positive affect predicts lower risk of AIDS mortality. Psychosomatic medicine. 2003;65(4):620–626. [DOI] [PubMed] [Google Scholar]
  • 28.Li J, Mo PK, Wu AM, Lau JT. Roles of self-stigma, social support, and positive and negative affects as determinants of depressive symptoms among HIV infected men who have sex with men in China. AIDS and Behavior. 2017;21(1):261–273. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Ironson G, Balbin E, Stuetzle R, et al. Dispositional optimism and the mechanisms by which it predicts slower disease progression in HIV: proactive behavior, avoidant coping, and depression. International journal of behavioral medicine. 2005;12(2):86–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Ironson GH. Do positive psychosocial factors predict disease progression in HIV-1? A review of the evidence. Psychosomatic medicine. 2008;70(5):546. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Wilson TE, Weedon J, Cohen MH, et al. Positive affect and its association with viral control among women with HIV infection. Health Psychology. 2017;36(1):91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science. 2009;4(1):1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Smith JD, Li DH, Rafferty MR. The implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. Implementation Science. 2020;15(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of biomedical informatics. 2009;42(2):377–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Kegler MC, Liang S, Weiner BJ, et al. Measuring constructs of the consolidated framework for implementation research in the context of increasing colorectal cancer screening in federally qualified health center. Health services research. 2018;53(6):4178–4203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Fernandez ME, Walker TJ, Weiner BJ, et al. Developing measures to assess constructs from the inner setting domain of the consolidated framework for implementation research. Implementation Science. 2018;13(1):1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Norton WE. An exploratory study to examine intentions to adopt an evidence-based HIV linkage-to-care intervention among state health department AIDS directors in the United States. Implementation Science. 2012;7(1):1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Phillippi J, Lauderdale J. A guide to field notes for qualitative research: Context and conversation. Qualitative health research. 2018;28(3):381–388. [DOI] [PubMed] [Google Scholar]
  • 39.Gale RC, Wu J, Erhardt T, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science. 2019;14(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Strauss A, Corbin JM. Grounded theory in practice. 1997.
  • 41.Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 2015;10(1):1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Odeny TA, Padian N, Doherty MC, et al. Definitions of implementation science in HIV/AIDS. The Lancet HIV. 2015;2(5):e178–e180. [DOI] [PubMed] [Google Scholar]
  • 43.Davey DLJ, de Villiers L, Evens E. Importance of rigorous implementation science studies to scale-up evidence-based interventions to end the HIV epidemic in the United States. AIDS. 2021;35(2):335–336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Montena AL, Possemato K, Kuhn E, et al. Barriers and Facilitators to Peer-Supported Implementation of Mental Health Mobile Applications with Veterans in Primary Care. Journal of Technology in Behavioral Science. 2021:1–12. [Google Scholar]
  • 45.Brady KJ, Durham MP, Francoeur A, et al. Barriers and facilitators to integrating behavioral health services and pediatric primary care. Clinical Practice in Pediatric Psychology. 2020. Advance online publication. [Google Scholar]
  • 46.Quittner AL, Abbott J, Hussain S, et al. Integration of mental health screening and treatment into cystic fibrosis clinics: Evaluation of initial implementation in 84 programs across the United States. Pediatric Pulmonology. 2020;55(11):2995–3004. [DOI] [PubMed] [Google Scholar]
  • 47.Stanhope V, Ross A, Choy-Brown M, Jessell L. A mixed methods study of organizational readiness for change and leadership during a training initiative within community mental health clinics. Administration and Policy in Mental Health and Mental Health Services Research. 2019;46(5):678–687. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implementation Science. 2021;16(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Shangani S, Bhaskar N, Richmond N, Operario D, van den Berg JJ. A systematic review of early adoption of implementation science for HIV prevention or treatment in the United States. AIDS. Feb 2 2021;35(2):177–191. doi: 10.1097/QAD.0000000000002713 [DOI] [PubMed] [Google Scholar]
  • 50.Hernandez R, Cohn M, Hernandez A, et al. A web-based positive psychological intervention to improve blood pressure control in spanish-speaking hispanic/latino adults with uncontrolled hypertension: Protocol and design for the¡ Alégrate! randomized controlled trial. JMIR research protocols. 2020;9(8):e17721. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Snell-Rood C, Jaramillo ET, Hamilton AB, Raskin SE, Nicosia FM, Willging C. Advancing health equity through a theoretically critical implementation science. Translational Behavioral Medicine. 2021;11(8):1617–1625. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res. Mar 12 2020;20(1):190. doi: 10.1186/s12913-020-4975-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Table 1
Supplementary Table 2
Supplementary Table 3

RESOURCES