LGBQ-affirmative cognitive-behavioral therapy for young gay and bisexual men’s mental and sexual health: A three-arm randomized controlled trial

Abstract

Objective.

ESTEEM (Effective Skills to Empower Effective Men) represents the first intervention to address the psychological pathways through which minority stress undermines young sexual minority men’s (SMM’s) mental and sexual health using transdiagnostic cognitive- behavioral therapy. This study compared the efficacy of ESTEEM against two existing interventions.

Method.

Participants were young HIV-negative SMM (N=254; ages 18-35; 67.2% racial/ethnic minority) experiencing a depression, anxiety, and/or stress-/trauma-related disorder and past-90-day HIV-transmission-risk behavior. After completing HIV testing and counseling, participants were randomized to receive 10-session ESTEEM (n=100); 10-session community-based LGBQ-affirmative counseling (n=102); or only HIV testing and counseling (n=52).

Results.

For the primary outcome of any HIV-transmission-risk behavior at 8-months, ESTEEM was not significantly associated with greater reduction compared to HIV testing and counseling (RR=0.89, p=0.52). Supportive analyses of the frequency of HIV-transmission-risk behavior at 8-months showed a non-significant difference between ESTEEM compared to HIV testing and counseling (RR=0.69) and LGBQ-affirmative counseling (RR=0.62). For secondary outcomes (e.g., depression, anxiety, substance use, suicidality, number of mental health diagnoses) at 8-months, ESTEEM had a larger effect size than the two comparison conditions, but these comparisons did not reach statistical significance when adjusting for the false discovery rate. Observed effect sizes for condition comparisons were smaller than the effect sizes used to power the study. In exploratory analyses, ESTEEM showed promise for reducing comorbidity.

Conclusions.

Because the control conditions were associated with stronger effects than anticipated, and given the heterogeneous nature of transdiagnostic outcomes, the study possessed insufficient power to detect the consistently small-to-moderate benefit of ESTEEM compared to the two control conditions.

Gay, bisexual, and other sexual minority men (SMM) in the U.S. are at substantially elevated risk for depression, anxiety, substance use, suicidality, and HIV acquisition compared to the general U.S. population (Centers for Disease Control and Prevention, 2018; Cochran et al., 2017; Marshal et al., 2008). These co-occurring disparities start early in development (Bostwick et al., 2014; Irish et al., 2019; Mustanski et al., 2007) and have a shared source in the unique developmental and societal challenges facing this population (Meyer, 1995). These challenges, known as minority stress, include early and ongoing experiences of family and peer rejection, formal exclusion from community institutions such as some religious organizations, lack of opportunities to connect with similar others, and exposure to systemic invalidation of sexual minority identities and persons (Hatzenbuehler & Pachankis, 2016). Although minority stress theory (Brooks, 1981; Meyer, 2003) and its clinical extensions (Hatzenbuehler, 2009; Pachankis, 2015) suggest that interventions that support young SMM in coping with minority stress can improve their co-occurring health risks, such interventions have only recently been examined (O’Shaughnessy & Speir, 2018).

In fact, only two minority stress interventions have shown efficacy for reducing SMM’s co-occurring health risks. One of these interventions, delivered in India, involves supportive counseling focused on self-acceptance and self-esteem through group peer support and modularized counseling sessions focused on specific minority stress experiences (e.g., coming out). Compared to standard HIV testing and counseling, this intervention was associated with stronger reductions in HIV-transmission-risk behavior, depression, self-esteem, and self-acceptance (Safren, Thomas, et al., 2021; Safren et al., 2014). The other intervention, called ESTEEM (Effective Skills to Empower Effective Men), systematically adapted cognitive-behavioral therapy (CBT) to address young SMM’s cognitive, affective, and behavioral reactions to minority stress that have been shown to elevate risk for numerous adverse psychosocial health outcomes (Hatzenbuehler, 2009; Meyer, 2003). These reactions include, for example, chronic, anxious expectations of rejection (Pachankis et al., 2008), identity concealment (Pachankis, Mahon, et al., 2020), internalized stigma (Newcomb & Mustanski, 2010), social isolation (Hatzenbuehler et al., 2012), and difficulties identifying and expressing emotions (Hatzenbuehler et al., 2009). In an initial trial, ESTEEM was associated with reductions in depression, alcohol use, and condomless anal sex with casual partners up to three months post-treatment compared to a waitlist control (Pachankis et al., 2015a).

A CBT-based intervention such as ESTEEM provides a particularly suitable platform upon which to specifically address minority stress because it (1) locates the source of young SMM’s health risks in the environment rather than within the person, (2) facilitates the reworking of self-defeating internalized beliefs, such as those stemming from stigma, (3) builds a self-empowering behavioral repertoire to use in challenging situations, and (4) ultimately promotes coping self-efficacy (Balsam et al., 2019; Pachankis, 2018). Further, the CBT platform upon which ESTEEM is based is transdiagnostic (Barlow et al., 2017) and addresses the shared minority stress source of the co-occurring mental and sexual health risks that disproportionately and commonly affect young SMM (Bränström & Pachankis, 2018; Mustanski et al., 2007). Transdiagnostic treatments circumvent the practical challenges of single-outcome treatments, reduce the need for provider training in multiple intervention protocols, and facilitate efficient dissemination and implementation. As a protocol focused on reducing maladaptive minority stress reactions, ESTEEM also responds to the growing ethical and professional mandate to address minority stress, as relevant, in psychosocial treatments for sexual minority individuals. For instance, the American Psychological Association’s (2021) practice guidelines for sexual minority clients encourage mental health professionals to draw upon minority stress research and “recognize the influence that proximal minority stressors have on the mental, physical, and psychosocial health of sexual minority persons” (p. 18). The ESTEEM protocol provides concrete guidance for how to implement such guidelines.

Knowing the relative utility of a transdiagnostic, minority-stress-focused, CBT-based approach like ESTEEM also requires comparisons to control conditions, including currently available services for SMM’s mental and sexual health. Perhaps the most widely available existing mental health treatment for sexual minority individuals is LGBQ-affirmative supportive counseling (Fassinger, 2000; Garnets et al., 1991; Pachankis & Goldfried, 2004; Ritter & Terndrup, 2002), LGBQ-affirmative counseling grew out of the LGBTQ rights movement as an antidote to pathologizing narratives of the time and, as then, is still frequently delivered in LGBTQ community centers (Pachankis et al., 2021). This counseling is typically guided by general principles to help sexual minority individuals develop insight into the ways in which stress can impact their mental health; address stigma-related emotions, such as shame and guilt, and cognitive styles, such as low self-worth; promote resilience and pride as an antidote to stigma; and provide resources and advocacy against societal injustice. Unlike a theoretically informed treatment, however, LGBQ-affirmative counseling is not codified in a set of specific techniques other than a general directive to provide affirmative support. Additionally, the efficacy of LGBQ-affirmative supportive counseling has never been examined in a clinical trial.

Another support commonly available to sexual minority men is HIV testing and counseling, which typically consists of a single brief session involving review of sexual health risks, biological testing for HIV and other STIs, and referral to HIV/STI treatment as needed. This type of counseling does not typically address co-occurring mental (e.g., depression) or behavioral (e.g., substance use) health risks. Knowing the relative efficacy of single-session HIV testing and counseling compared to a multi-session transdiagnostic treatment that also includes HIV testing and counseling represents an important public health question. Specifically, this comparison could help inform whether to provide a more comprehensive treatment approach for young SMM experiencing HIV-transmission-risk behavior in the context of other psychosocial health concerns, such as depression or substance use problems (Safren et al., 2010).

The current trial was conducted to establish the utility of systematically addressing minority stress using a CBT-based transdiagnostic treatment approach to improve mental and sexual health among young SMM experiencing these comorbidities. We specifically compared ESTEEM to two of the most commonly available supports – LGBQ-affirmative counseling in community-based clinics and HIV/STI testing and counseling – among young, HIV-negative SMM experiencing HIV-transmission-risk behavior in the context of co-occurring diagnosed mental health problems. As a primary analysis, we examined whether ESTEEM was associated with greater reduction in HIV-transmission-risk behavior at follow-up as compared to HIV testing and counseling and, if so, as additionally compared to LGBQ-affirmative counseling. Our operationalization of HIV-transmission-risk behavior captured contemporary biomedical approaches to prevention that consider HIV pre-exposure prophylaxis (PrEP; Grant et al., 2010) and HIV treatment-as-prevention (Hayes et al., 2019) in addition to condom use with HIV-negative or status-unknown partners. In secondary analyses, we examined whether ESTEEM was associated with greater reductions in depression, anxiety, psychological distress, substance use, suicidality, and number of interviewer-based DSM-5 (American Psychiatric Association, 2013) diagnoses at follow-up as compared to HIV testing and counseling and LGBQ-affirmative counseling. Exploratory analyses examined the relative reduction across conditions in the number of comorbid syndemic conditions present, defined as depression, anxiety, substance use problems, and HIV-transmission-risk behavior (Pantalone et al., 2020).

Method

Recruitment

This study was reviewed and approved by the institutional review boards of Yale University and the University of Miami. From August 2016 to May 2019, we recruited young SMM in New York City and Miami through in-person venues (e.g., bars/clubs, LGBTQ community centers and events, clinic waiting rooms, college campuses), online (i.e., geo-targeted advertising on SMM’s social and sexual networking platforms and party listservs), and snowball sampling. Recruitment advertisements mentioned that the study was intended for young gay and bisexual men who were experiencing difficulties with mood and stress.

Participants

Figure 1 provides an overview of the study structure and participant screening and flow through each study stage. Participants met the following eligibility criteria: (1) aged 18-35 years given that this is the age range at which mental health problems, HIV-transmission-risk behavior, minority stress, and their co-occurrence peak (Bränström, 2017; Layland et al., 2020; Rice et al., 2019); (2) self-identification as a gay, bisexual or queer man; (3) HIV-negative status confirmed through in-office testing; (4) diagnosis of any DSM-5 depressive, anxiety, or trauma-/stress-related disorder (American Psychiatric Association, 2013); (5) risk of HIV acquisition through sexual activity (≥1 act of past-90-day-condomless anal sex involving a flesh penis with a partner with an unknown HIV status or an HIV-positive status, unless with an HIV-positive primary or main partner with known undetectable viral load [assessed through participant’s report of primary partner’s viral load] or an HIV-negative primary or main partner who is known to be adherent to PrEP [i.e., HIV pre-exposure prophylaxis most commonly prescribed in the form of a once-daily medication with strong efficacy for preventing HIV transmission; defined as participant’s report of primary partner taking PrEP on ≥4 days/week]); (6) not themselves currently adherent to PrEP (defined as taking PrEP on ≥4 days/week); (7) NYC or Miami residential stability and planned availability for 12 months; (8) English-language proficiency; and (9) provision of informed consent.

Figure 1. Flow of participants through study phases.

Note. ^a Participants who were excluded from analysis did not provide data on the primary outcome.

Participants were excluded for: (1) current active suicidality or homicidality (but not passive suicidality); (2) evidence of active untreated mania, psychosis, or gross cognitive impairment; (3) current enrollment in another intervention study; (4) currently receiving ≥1 mental health treatment sessions/month or ≥8 or more CBT sessions within the past year; and (5) HIV-positive status (confirmed through in-office testing).

Procedures

Screening

Potential participants completed an eligibility survey online. All individuals who were preliminarily eligible online were then asked to complete a more detailed telephone screen to confirm eligibility especially for time-variant criteria (e.g., HIV-transmission risk, depression/anxiety symptoms) and then scheduled for an initial in-office baseline assessment. Potential participants who called the research office directly completed the telephone screen as a first step. During the first in-office visit, participants completed consent procedures and were screened for exclusion criteria: active psychosis, mania, suicidality, or homicidality (First et al., 1997); and gross cognitive impairment (Folstein et al., 1975). Participants then completed the interviewer-administered Mini International Neuropsychiatric Interview (MINI; Sheehan et al., 2015; Sheehan et al., 1998) to confirm mental health diagnostic inclusion. Participants completed in-office HIV testing administered by study staff to confirm HIV-negative status at baseline; participants self-collected samples at the study site to test for chlamydia and gonorrhea (urethral, rectal, pharyngeal) at this visit given that these sexually transmitted infections are highly prevalent among young SMM (Chow et al., 2019; Tsuboi et al., 2021; Werner et al., 2018). At a second baseline visit, participants completed an interviewer-administered assessment of sexual behavior, including condomless anal sex, to confirm risk for HIV transmission through sexual activity.

Randomization

After completing HIV testing and counseling, eligible participants were randomized to receive ESTEEM, LGBQ-affirmative counseling, or only HIV testing and counseling with no further intervention. We used a 2:2:1 randomization scheme such that for every two participants randomized to ESTEEM, two participants were randomized to LGBQ-affirmative counseling and one participant was randomized to HIV testing and counseling only. This allocation was based on estimates that ESTEEM compared to a single-session of HIV-testing and counseling (received by all conditions) would be associated with stronger effects than ESTEEM compared to a time-matched LGBQ-affirmative counseling. A computer-generated program completed the randomization stratified by site (i.e., New York City or Miami). After randomizing 114 participants, we discovered a programming error in the randomization and participants were not being randomized 2:2:1. The randomization was corrected to ensure the final distribution was 2:2:1 and the sample size was increased to 254 to accommodate.

Participants randomized to receive ESTEEM were scheduled for their first session as soon as possible based on therapist and participant availability. Participants randomized to receive LGBQ-affirmative counseling were scheduled for a first session with a therapist at one of two community-partner organizations in New York City or Miami. Participants randomized to HIV testing and counseling were scheduled for their 4-month follow-up assessment.

Interventions

The complete study protocol is publicly available (Pachankis et al., 2019).

ESTEEM.

ESTEEM is a 10-session CBT intervention based on the Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders (Barlow et al., 2010), adapted through deep community consultation and formative qualitative research to enhance young SMM’s coping with minority stress (Pachankis, 2014). As described in detail elsewhere (Burton et al., 2019; Pachankis et al., 2019; Pachankis & Safren, 2019), the sessions are guided by minority stress focused principles (e.g., “mood and anxiety disorders are common responses to minority stress,” “encourage authentic and assertive communication across contexts”) and techniques (e.g., “increasing awareness of minority stress reactions,” “connecting minority stress to emotion-driven behaviors”) to build minority stress coping en route to reducing the comorbid mental and sexual health outcomes affecting young SMM.

Study Therapists, Supervision, and Intervention Fidelity.

The intervention was delivered by clinical and counseling psychologists, advanced clinical psychology doctoral students, and a Masters-level clinician. Study therapists were diverse in race/ethnicity, gender identity, and sexual orientation. Three clinical psychologists supervised ESTEEM delivery through weekly group and individual supervision. All sessions were video- or audio-recorded; clinical supervisors reviewed 139 (16.7%) sessions using fidelity checklists created for the study. Checklists were module-specific and contained between four and seven questions assessing session fidelity from 0 (not covered at all) to 2 (covered thoroughly). For example, Session 6 items included “Help the participant identify his own emotion avoidance strategies” and “Help the participant identify the relationship between avoidance and minority stress.” Therapists were highly adherent to protocol, with mean session ratings of 1.89 out of 2.00 across the 139 reviewed sessions (at least 10 videos per session were reviewed). Non-fidelity to protocol was discussed during clinical supervision, including during review of recorded sessions, with the goal of improving protocol fidelity.

LGBQ-affirmative Counseling.

Participants received up to 10 sessions of LGBQ-affirmative counseling completed within 4 months at one of two local community clinics (in New York City or Miami). To determine general session content in this non-protocoled therapy, two trained researchers coded early (sessions 2 or 3) and middle (sessions 5 or 6) sessions, as available, for all participants who received LGBQ-affirmative counseling (n = 114 coded sessions) using the Comparative Psychotherapy Process Scale (CPPS; Hilsenroth et al., 2005). Each session was coded by both coders, yielding an intraclass correlation coefficient (ICC) of .63, indicative of moderate reliability. Relatively low presence of CBT content (example item: “practice behaviors between sessions”) and psychodynamic-interpersonal content (example item: “focus on the patient-therapist relationship”) were indicated by a mean score of 1.98 (SD = 0.60) and 2.03 (SD = 0.31), respectively, with a score of 2 (out of a theoretical range 0-6) indicating that the respective theoretical orientation was “somewhat characteristic” of that session. Although community therapists worked at a separate location than the ESTEEM therapists and were unaware of the ESTEEM content, we also coded the above-mentioned sessions for possible overlap with the ESTEEM content using a 12-item contamination scale created for this study (example item: “discuss impact of minority stress on emotions”). Minimal overlap with ESTEEM content was found, with a mean score (0.51, SD = 0.58; theoretical range 0-4), which lies between anchors 0 (did not mention at all) and 1 (passing comment).

HIV Testing and Counseling.

HIV testing and counseling was conducted following protocols used in previous RCTs (Koblin et al., 2004) and clinical practice (AIDS Institute, 2017). In a 30-45-minute session, a trained counselor at the study research offices reviewed the purpose, benefits, and risks of HIV/STI testing; provided information about transmission risk and prevention; assessed the participant’s readiness for testing; conducted the testing and reviewed HIV test results; worked with the participant to develop a personalized risk reduction plan; and provided screening for mental health problems and referrals for sexual health care as relevant. Participants were contacted within one week to provide test results for chlamydia and gonorrhea. All participants, regardless of condition, received HIV testing and counseling.

Study Assessments

Participants completed assessments at baseline and 4-, 8-, and 12-month follow-up. At each timepoint, participants completed self-report surveys at home and interviewer-based assessments of mental health, HIV-transmission-risk behavior, and other sensitive topics (e.g., suicidality) at the research office. Participants also repeated HIV/STI testing at 12-month follow-up. Primary and secondary outcomes were selected a priori as documented in the protocol registration record on clinicaltrials.gov (NCT02929069). For all interviewer-rated assessments, trained interviewers were masked to participant condition; despite masking protocol, 13.7% of participants disclosed their study condition during a follow-up interview.

Primary Outcome Measure: HIV-transmission-risk Behavior

Timeline Follow-back (TLFB; Carey et al., 2001; Sobell & Sobell, 1992).

TFLB is an interviewer-administered assessment of past 90-day event-level behavior. Originally developed for alcohol use, the TLFB was adapted for the present study to record sexual behavior as well as alcohol and drug use in the context of sex. Participants were asked to identify personally meaningful past-90-day events (e.g., holidays, parties, breakups) as “anchor points” to facilitate recollection of events. Interviewers then worked backwards with participants to identify days on which the participant engaged in any sexual behavior. HIV-transmission-risk behavior was determined as a function of partner type (i.e., main partner, new casual partner, repeat casual partner), sexual act (e.g., anal receptive intercourse), condom use (i.e., yes, no), and participant’s and primary partners’ PrEP status (i.e., if participant reported that they or their primary partner were adherent to PrEP, defined as taking PrEP on ≥4 days/week) or viral load (i.e., detectable, undetectable; reported by the participant). For this study, HIV-transmission-risk behavior was operationalized as past-90-day condomless anal sex in the absence of either self-reported adherent PrEP use (for both participant and HIV-negative primary partner) or known undetectable viral load (for HIV-positive primary partner).

All interviewers were trained to reliability using mock participant videos until they were able to correctly code all applicable events. All interviews were recorded, and 20% of the sample was randomly reviewed for quality assurance and to provide feedback to interviewers to maintain reliability. The TLFB is a well-established measure for evaluating sexual behavior and substance use among gay and bisexual men (Leluţiu-Weinberger et al., 2018; Pachankis et al., 2015b; Pachankis et al., 2019).

Secondary Mental Health Outcome Measures

Depression

Hamilton Depression Rating Scale (HAM-D; Hamilton, 1960).

The HAM-D is a 17-item interviewer-administered semi-structured assessment of past-week depressive symptom severity (e.g., depressed mood, suicidal ideation, insomnia, loss of appetite). Most item ratings range from 0 to 4, with anchors specific to each item. Prior to conducting HAM-D interviews, interviewers were trained to reliability with mock participant videos. ICCs were calculated against the ratings of a clinical psychologist supervisor, whose ratings served as the criterion. Interviewers then attended supervision weekly (for the first 6 months) and monthly (thereafter) to maintain reliability and prevent drift. A meta-analysis of 409 studies over 48 years showed that the HAM-D has strong internal consistency, inter-rater, and test-retest reliability (Trajković et al., 2011). The HAM-D has also been utilized in other clinical trials with sexual minority men (e.g., Mimiaga et al., 2018; Safren et al, 2021). Cronbach’s alphas in the current study were 0.75, 0.81, 0.79, 0.84 at baseline, 4-month, 8-month, and 12-month assessment, respectively.

Overall Depression Severity & Impairment Scale (ODSIS; Bentley et al., 2014).

ODSIS is a 5-item self-report scale that asks participants to rate the severity and impairment associated with past-week symptoms of depression (e.g., “In the past week, when you have felt depressed, how intense or severe was your depression?) from 0 (little or none: Depression was absent or barely noticeable.) to 4 (extreme: Depression was overwhelming.). Cronbach’s α was 0.93, 0.94, 0.95, 0.95 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, consistent with prior research with sexual minority men (e.g., Lopez-Matos et al., 2021).

Anxiety

Beck Anxiety Inventory (BAI; Beck et al., 1988).

The BAI is a 21-item self-report scale assessing how much one has been bothered by past-month anxiety symptoms (e.g., “terrified or afraid”) on a 4-point scale from 0 (not at all) to 3 (severely – it bothered me a lot). Cronbach’s α was 0.91, 0.93, 0.93, 0.93 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, consistent with previous research with sexual minority men (e.g., Blain et al., 2012; Mimiaga et al. 2011).

Overall Anxiety Severity & Impairment Scale (OASIS; Norman et al., 2006).

The OASIS is a 5-item self-report scale that asks participants to rate the severity and impairment associated with past-week anxiety symptoms (e.g., “In the past week, when you have felt anxious, how intense or severe was your anxiety?”) from 0 (little or none: Anxiety was absent or barely noticeable.) to 4 (extreme: Anxiety was overwhelming. It was impossible to relax at all. Physical symptoms were unbearable). Cronbach’s α was 0.86, 0.89, 0.89, 0.92 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, consistent with previous research with sexual minority men (e.g., López-Matos et al., 2021).

Social Interaction Anxiety Scale (SIAS; Mattick & Clarke, 1998).

The SIAS is a 19-item self-report scale that assesses the extent to which social anxiety symptoms are characteristic of the respondent (e.g., “I tense up if I meet an acquaintance in the street”) on a 5-point scale from 0 (Not at all) to 4 (Extremely). Previous research with young sexual minority men has found high reliability and temporal stability of the SIAS (e.g., Pachankis et al, 2018). Cronbach’s α was 0.94, 0.93, 0.94, 0.94 at baseline, 4-month, 8-month, and 12-month follow-up, respectively.

Psychological Distress

Brief Symptom Inventory (BSI; Derogatis & Melisaratos, 1983; Meijer et al., 2011).

The Global Severity Index of the 18-item BSI provides a mean score across depression, anxiety, and somatization subscales, and assesses psychological distress (e.g., “feeling nervousness or shakiness inside”) on a 5-point scale from 0 (not at all) to 4 (extremely). Cronbach’s α was 0.89, 0.93, 0.91, 0.93 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, similar to prior research with sexual minority men (e.g., Gamarel et al., 2012.; Lelutiu-Weinberger et al., 2013).

Suicidality

Suicidal Ideation Attributes Scale (SIDAS; Van Spijker et al., 2014).

SIDAS is a 5-item self-report scale that assesses past-month frequency and controllability of suicidal thoughts, how close one has come to making an attempt, and distress and impairment associated with thoughts of suicide (e.g., “In the past month, how often have you had thoughts about suicide?”). Responses range from 0 (never or not at all) to 10 (always or extremely). Cronbach’s α was 0.88, 0.83, 0.93, 0.90 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, consistent with previous research with young sexual minority individuals (e.g., Kaniuka et al., 2020; Pachankis et al, 2020). For this outcome, we used a cutoff of ≥0 as an indicator of suicide risk.

Substance Use

Alcohol Use Disorders Identification Test (AUDIT; Saunders et al., 1993).

The AUDIT assesses past-3-month alcohol and related problems across 10 items (e.g., “How often did you have a drink containing alcohol?”) with varying numeric response options. Cronbach’s α was 0.87, 0.85, 0.86, 0.86 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, consistent with prior research with sexual minority men (e.g., Hamilton & Mahalik, 2009).

Short Inventory of Problems – Alcohol and Drugs (SIP-AD; Allensworth-Davies et al., 2012; Blanchard et al., 2003).

SIP-AD is a 15-item scale in which the sum of “yes” responses indicates participants’ past-3-month consequences of alcohol and drug use (e.g., “I have failed to do what is expected of me because of my drinking/drug use”). KR-20 were 0.90, 0.91, 0.90, 0.90 at baseline, 4-month, 8-month, and 12-month follow-up, respectively, consistent with previous research with young sexual minority individuals (e.g., Pachankis et al, 2020).

Transdiagnostic Outcomes

Mini International Neuropsychiatric Interview (MINI 7.0; Sheehan et al., 2015; Sheehan et al., 1998).

Participants were assessed for the following DSM-5 diagnoses (American Psychiatric Association, 2013) at baseline using the respective interviewer-based MINI modules: depression, dysthymia, generalized anxiety disorder, panic disorder, agoraphobia, social anxiety disorder, obsessive-compulsive disorder, posttraumatic stress disorder, alcohol use disorder, and substance use disorder. At 4-, 8-, and 12-month follow-up, participants were assessed again for the presence of the diagnoses for which they met criteria at baseline. Prior to conducting MINI interviews, interviewers were trained to reliability with mock participant video ratings until they correctly reached the diagnoses assigned by the clinical psychologist supervisor who initially rated the mock participants. Participants’ diagnoses at 4-, 8-, and 12-months were compared to their diagnoses at baseline, indicating whether the number of diagnoses was reduced or whether all diagnoses were eliminated entirely. The MINI has previously been used in studies with sexual minority men (e.g., Blashill et al., 2017; Boroughs et al., 2015).

Exploratory Comorbidity Outcome Measure.

Consistent with the transdiagnostic nature of ESTEEM, we conducted an exploratory (non-preregistered) analysis that examined the reduction in a count of co-comorbid outcomes. The comorbid outcomes examined represent those psychosocial syndemic factors that frequently co-occur with young SMM’s HIV-transmission-risk behavior (Mustanski et al., 2007) and that are commonly examined as outcomes alongside HIV-transmission-risk behavior in existing psychosocial interventions for sexual minority men (Pantalone et al., 2020). Measures forming this comorbidity index were selected from those that were preregistered for the main analyses. The outcome was specifically operationalized as a sum of whether a participant exceeded cutoff scores on: (1) the Hamilton Depression Rating Scale (HAM-D; ≥17; Zimmerman et al., 2013), (2) Beck Anxiety Inventory (BAI; ≥16; Beck & Steer, 1990), (3) Short Inventory of Problems – Alcohol and Drugs (SIP-AD; ≥7; based on baseline distribution), and (4) reporting any HIV-transmission-risk behavior.

Analysis Plan

Any HIV-transmission-risk behavior at 8-month follow-up served as the primary preregistered outcome. A primary supportive analysis used a count of the number of HIV-transmission-risk behaviors. Secondary preregistered outcomes at 8- and 12-month follow-ups included depression (HAM-D, ODSIS), anxiety (BAI, OASIS, BSI, SIAS), psychological distress (BSI), substance use (AUDIT, SIP-AD), suicidality (SIDAS), and MINI-derived mental health diagnoses, examined as both no longer having at least one baseline DSM-5 diagnosis still be present at 8- and 12-month follow-up and elimination of all baseline MINI-derived mental health diagnoses). However, in mixed models, which utilize all data across all timepoints, if there was no treatment-by-time interaction for secondary outcomes at p = 0.10 (set a priori in the preregistered statistical analysis plan), the overall treatment effect was averaged across all time points. An exploratory outcome (not preregistered) included a count of comorbid mental (i.e., depression, anxiety), behavioral (i.e., substance use), and sexual (i.e., HIV-transmission-risk behavior) conditions using the established cutoffs for each condition noted above.

Sample size estimates were informed by the waitlist-controlled trial of ESTEEM (Pachankis et al., 2015a), in which ESTEEM yielded a 60% reduction in condomless anal sex with casual partners at follow-up compared to waitlist control. Based on previous studies of HIV testing and counseling (Kamb et al., 1998) and the fact that LGBQ-affirmative counseling does not specifically focus on sexual health, we expected that these arms would yield lower reductions compared to ESTEEM, but a slightly larger reduction in LGBQ-affirmative counseling compared to HIV testing and counseling only (Pachankis et al., 2019). For secondary mental and behavioral health outcomes, our sample size provided 80% power to detect an effect size of 0.55 and 0.70 for ESTEEM versus LGBQ-affirmative counseling and ESTEEM vs. HIV testing and counseling only, respectively, at a Type I error rate of 0.01 (conservative due to multiple testing).

Statistical Analysis

The study protocol and statistical analysis plan were preregistered at clinicaltrials.gov (NCT02929069). Analyses used an intent-to-treat principle. Descriptive statistics (means and standard deviations [SD]; frequencies and proportions) were calculated overall and by treatment group. The primary analysis used a generalized linear model with a log-binomial link, using a contrast for the primary time point of 8-months and secondary time point of 12-months, and exploratory time point of 4-months, to test the comparison of the primary outcome (i.e., any HIV-transmission-risk behavior), adjusting for location (i.e., New York City or Miami) and baseline number of HIV-transmission-risk behavior acts. A similar generalized linear model was utilized for the supportive outcome of count of HIV-transmission-risk behavior acts. Due to overdispersion (larger variance than mean), we used a negative binomial model, which accounts for overdispersion. We tested for zero-inflation by comparing the observed and expected number of zeros and checked goodness of fit to the negative binomial model. Risk ratios (RR) and their 95% confidence intervals (CI) are presented.

In order to control for the overall Type I error for the primary outcome due to multiple comparisons, we used a gatekeeping strategy (Dmitrienko & Tamhane, 2010), which involves testing condition comparisons by order of importance. Specifically, a statistical test is performed for the first comparison and is only performed for the second comparison if the first was statistically significant. We conducted the comparison of ESTEEM versus HIV testing and counseling at p = 0.05; only if this test was statistically significant would we test ESTEEM vs LGBQ-affirmative counseling at p = 0.05. We chose this order of gatekeeping given our estimation that ESTEEM compared to a single-session of HIV-testing and counseling (received by all conditions) would be associated with stronger effects than ESTEEM compared to a time-matched LGBQ-affirmative counseling.

Analyses of the secondary mental health outcomes (HAM-D, ODSIS, BAI, OASIS, BSI, SIAS, BSI, AUDIT, SIP-AD) used linear mixed models assuming missing at random (MAR) to assess change from baseline. Contrast statements were used to test comparisons at 4-, 8- and 12-months, adjusting for study location. Suicidality (SIDAS) and MINI-derived mental health diagnoses (having at least one baseline DSM-5 diagnosis no longer be present at follow-up; elimination of all baseline MINI-derived mental health diagnoses) were assessed with generalized linear model with a log-binomial link, using a contrast for the time point of 4-, 8-and 12-months, adjusting for study location. There was no treatment-by-time interaction at p = 0.10, so we estimated average treatment effects for secondary outcomes using marginal least square means with 95% confidence limits.

To control for false discovery due to multiple comparisons of the secondary outcome, we used the Benjamini and Hochberg (1995) method. In this method, the raw p-values are ordered, and a new cutoff for statistical significance is constructed as: $\left(rank/numberofcomparisons\right)\ast significancelevel$. A given p-value achieves statistical significance if the raw p-value is less than the new adjusted cut-off. Further, every comparison of lesser rank is also considered statistically significant. The significance level was set to p = 0.05.

An additional exploratory comorbidity count outcome was analyzed using a generalized Poisson model given under-dispersion (Consul & Famoye, 1992). Contrast statements were used to test comparisons at 4-, 8- and 12- months, adjusting for study location. Additionally, we estimated average treatment effect using marginal least square means with 95% confidence limits.

The same models described above were utilized for exploratory outcomes of estimated probability or estimated least square means, for binary and count outcomes respectively, for each arm, for each time point, using contrast statements. No p-values are presented for these exploratory measures.

Because randomization was stratified by study location, a dummy variable denoting study location (i.e., New York City versus Miami) was included as a covariate in all models (Kernan et al., 1999).

The above analyses were conducted using SAS/Stat 9.4 (SAS Cary, NC).

Results

Table 1 describes sample characteristics. Participants’ mean age was 26.55 years (SD = 4.17 years). About one-third of participants (32.8%) identified as non-Hispanic/Latino White and 42.5% identified as Hispanic/Latino; overall 67.2% identified as a racial or ethnic minority. Participants were predominantly cisgender (98.1%). A majority identified as gay (73.2%); 20% identified as bisexual and 13% as queer. Most participants reported having obtained a college degree (91%). Slightly over half of participants (55.73%) reported earning less than $29,000 in the past year. Most participants were not in a relationship at baseline (70.1%).

Table 1.

Baseline Demographic and Diagnostic Characteristics

	Total		ESTEEM		HIV Testing and Counseling		LGBQ-affirmative Counseling
	(N = 253)		(n = 100)		(n = 52)		(n = 101)
Variable	N	%	N	%	N	%	N	%
Age, years (Mean SD)	26.55	4.17	26.31	4.47	26.88	3.43	26.6	4.23
Race
American Indian /Alaskan Native	1	0.4	1	1	0	0	0	0
Asian	10	3.95	5	5	2	3.8	3	2.97
Black/African American	43	17	20	20	2	3.8	21	20.79
Multiracial	38	15.02	12	12	12	23.1	14	13.86
Native Hawaiian/Pacific Islander	2	0.79	0	0	1	1.9	1	0.99
White	141	55.73	53	53	32	61.5	56	55.45
Other	18	7.11	9	9	3	5.8	6	5.94
Hispanic/Latino
No	145	57.31	57	57	28	53.8	60	59.41
Yes	108	42.69	43	43	24	46.2	41	40.59
Sex assigned at birth
Male	250	98.81	98	98	52	100	100	99.01
Female	3	1.19	2	2	0	0	1	0.99
Gender Identity a
Man	250	98.81	100	100	51	98.1	99	98.02
Woman	1	0.4	1	1	0	0	0	0
Transgender man	2	0.79	2	2	0	0	0	0
Genderqueer	6	2.37	2	2	2	3.8	2	1.98
Gender non-conforming/-binary	3	1.19	2	2	0	0	1	0.99
Two-spirit	3	1.19	3	3	0	0	0	0
Other	2	0.79	1	1	0	0	1	0.99
Sexual Orientation
Bisexual	53	20.95	25	25	9	17.3	19	18.81
Gay	186	73.52	69	69	39	75	78	77.23
Queer	13	5.14	6	6	4	7.7	3	2.97
Uncertain	1	0.4	0	0	0	0	1	0.99
Education
Less than college	22	8.7	6	6	6	11.5	10	9.9
College degree	231	91.3	94	94	46	88.5	91	90.1
Employment Status
Full-time	125	49.41	47	47	32	61.5	46	45.54
Part-time	51	20.16	20	20	9	17.3	22	21.78
Permanently or temporarily disabled but working off the books	1	0.4	0	0	0	0	1	0.99
Student (employed)	29	11.46	13	13	1	1.9	15	14.85
Student (unemployed)	23	9.09	11	11	4	7.7	8	7.92
Unemployed	24	9.49	9	9	6	11.5	9	8.91
Personal Income (past year)
Up to $29,000	141	55.73	61	61	27	51.92	53	52.48
$30,000 - $75,000 or more	112	44.27	39	39	25	48.07	48	47.52
Relationship Status
Casually Dating	28	11.07	12	12	6	11.5	10	9.9
Single	178	70.36	66	66	35	67.3	77	76.24
In a relationship	47	18.58	22	22	11	21.2	14	13.86
Diagnoses b
Depression	195	77.08	79	79	37	71.2	79	78.22
Dysthymia	82	32.41	33	33	9	17.3	40	39.6
Panic Disorder	53	20.95	16	16	12	23.1	25	24. 75
Agoraphobia	60	23.72	24	24	11	21.2	25	24.75
Social Anxiety Disorder	106	41.9	37	37	20	38.5	49	48.51
Obsessive Compulsive Disorder	50	19.76	23	23	7	13.5	20	19.8
Post-Traumatic Stress Disorder	37	14.62	18	18	5	9.6	14	13.86
Generalized Anxiety Disorder	124	49.01	53	53	20	38.5	51	50.5
Alcohol-Use Disorder	62	24.51	27	27	13	25	22	21.78
Substance-Use Disorder c	136	53.75	55	55	28	53.8	53	52.48

Abbreviations. ESTEEM = Effective Skills to Empower Effective Men (test intervention); SD = standard deviation

^aGender identity variable was “check all that apply;” percent out of total sample size denoted for each identity; for inclusion, eligible participants had to report at least one gender identity as “man”

^bDiagnoses obtained using clinician-administered modules for listed disorders using the Mini-International Neuropsychiatric Interview for DSM-5 and ICD-10 (Sheehan et al, 1998).

^cSubstance-use disorder was assessed for the following substance categories: stimulants, cocaine, opiates, hallucinogens, dissociatives, inhalants, cannabis, tranquilizers and miscellaneous.

Therapy Completion and Follow-up Retention

A higher percentage of participants in the ESTEEM condition (74.0%) completed all 10 therapy sessions than in the LGBQ-affirmative counseling condition (41.2%). A higher percentage of participants in the ESTEEM condition (82%) were retained at 8-month follow-up than in the LGBQ-affirmative counseling (75.5%) and HIV testing and counseling (69.2%) conditions.

HIV-transmission-risk Behavior Outcomes

ESTEEM was not associated with greater reduction of any HIV-transmission-risk behavior than HIV testing and counseling at 8 months (RR=0.88; 95% Confidence Interval [CI]: 0.52, 1.25) (Table 2). When HIV-transmission-risk behavior was evaluated as a count of risk acts, ESTEEM was also not associated with greater reduction than HIV testing and counseling at 8 months (RR=0.69, 95% CI: 0.33, 1.43). Because the a priori gatekeeping threshold was not met, ESTEEM was not compared to LGBQ-affirmative counseling for these primary outcomes.

Table 2.

Between-Condition Comparisons for Primary and Secondary Mental, Behavioral, and Sexual Health Outcomes

Primary and supportive risk behavior outcome
			Risk ratio				Estimated probability
Domain	Measure	Time	ESTEEM vs HIV Testing and Counseling (95% CI)	p a	ESTEEM vs LGBQ-affirmative Counseling (95% CI)	Time	ESTEEM (95% CI)	HIV Testing and Counseling (95% CI)	LGBQ-affirmative Counseling (95% CI)
HIV transmission risk behavior	Any HIV risk behavior b	4-months	0.91 (0.64, 1.19)	0.53	0.83 (0.63, 1.02)	4-months	0.65 (0.50, 0.79)	0.71 (0.53, 0.89)	0.78 (0.66, 0.90)
		8-months	0.88 (0.52, 1.25)	0.52	0.77 (0.52, 1.02)	8-months	0.52 (0.36, 0.68)	0.59 (0.38, 0.80)	0.68 (0.53, 0.82)
		12-months	0.87 (0.48, 1.27)	0.52	0.75 (0.48, 1.01)	12-months	0.48 (0.32, 0.64)	0.55 (0.34, 0.76)	0.64 (0.49, 0.79)
						Estimated least square means
	HIV risk behavior count c	4-months	0.67 (0.33, 1.34)	0.26	0.56 (0.31, 1.01)	4-months	1.50 (0.98, 2.29)	2.24 (1.26, 3.95)	2.68 (1.26, 4.09)
		8-months	0.69 (0.33, 1.43)	0.31	0.62 (0.35, 1.11)	8-months	1.41 (0.92, 2.17)	2.05 (1.11, 3.77)	2.27 (1.49, 3.46)
		12-months	0.65 (0.32, 1.32)	0.23	0.57 (0.32, 1.02)	12-months	1.29 (0.83, 2.01)	1.98 (1.11, 3.54)	2.27 (1.49, 3.45)

Secondary mental and behavioral health outcomes
			Difference in least square means					Estimated least square mean change from baseline
Domain	Measure (instrument range)	Time	ESTEEM vs HIV Testing and Counseling (95% CI)	p d	ESTEEM vs LGBQ-affirmative Counseling (95% CI)	p d	Time	ESTEEM (95% CI)	HIV Testing and Counseling (95% CI)	LGBQ-affirmative Counseling (95% CI)
Depression	HAM-D e (0 - 32)	Average f	−1.6 (−4.3, 1.09)	0.24	−2.1 (−4.3, 0.17)	0.07	Average f	−6.1 (−7.7, −4.5)	−4.5 (−6.7, −2.3)	−4.0 (−5.6, −2.4)
	ODSIS e (0 - 20)	Average f	−1.0 (−2.6, 0.61)	0.22	−0.70 (−2.0, 0.63)	0.30	Average f	−3.4 (−4.4, −2.5)	−2.4 (−3.8, −1.1)	−2.7 (−3.7, −1.8)
Anxiety	BAI e (0 - 63)	Average f	−0.66 (−4.4, 3.07)	0.73	−0.16 (−3.2, 2.91)	0.92	Average f	−6.1 (−8.3, −3.9)	−5.4 (−8.5, −2.4)	−5.9 (−8.1, −3.8)
	OASIS e (0 - 20)	Average f	−1.3 (−2.7, 0.02)	0.05	−0.61 (−1.7, 0.51)	0.29	Average f	−3.1 (−3.9, −2.3)	−1.8 (−2.9, −0.67)	−2.5 (−3.3, −1.7)
	SIAS e (1 - 75)	Average f	−0.18 (−3.9, 3.52)	0.92	−0.53 (−3.6, 2.51)	0.73	Average f	−5.8 (−8.0, −3.7)	−5.6 (−8.7, −2.6)	−5.3 (−7.5, −3.1)
Depression and anxiety	BSI GSI e (0 - 3)	Average f	−0.19 (−0.41, 0.04)	0.10	−0.10 (−0.28, 0.09)	0.30	Average f	−0.57 (−0.70, −0.44)	−0.38 (−0.56, −0.19)	−0.47 (−0.60, −0.34)
Alcohol use	AUDIT e (0 - 30)	Average f	−1.5 (−3.2, 0.08)	0.06	−0.43 (−1.8, 0.90)	0.53	Average f	−1.9 (−2.8, −0.92)	−0.33 (−1.7, 1.01)	−1.4 (−2.4, −0.47)
Substance use	SIP-AD e (0 - 15)	Average f	−1.7 (−2.8, −0.55)	0.004	−0.96 (−1.9, −0.03)	0.04	Average f	−1.7 (−2.4, −1.1)	−0.06 (−0.99, 0.86)	−0.77 (−1.4, −0.11)
			Risk Ratio					Estimated probability
Suicidality	SIDAS b (0 - 1)	Average f	0.69 (0.02, 1.35)	0.36	0.97 (0.15, 1.78)	0.93	Average f	0.08 (0.03, 0.14)	0.12 (0.02, 0.22)	0.09 (0.02, 0.15)
Interviewer-rated mental and behavioral health outcomes
			Risk ratio					Estimated probability
Domain	Measure	Time	ESTEEM vs HIV Testing and Counseling (95% CI)	p d	ESTEEM vs LGBQ-affirmative Counseling (95% CI)	p d	Time	ESTEEM (95% CI)	HIV Testing and Counseling (95% CI)	LGBQ-affirmative Counseling (95% CI)
Mental health	Baseline MINI diagnoses reducedb	Average f	1.01 (0.94, 1.08)	0.75	1.03 (0.96, 1.09)	0.43	Average f	0.95 (0.91, 1.00)	0.94 (0.87, 1.00) g	0.93 (0.87, 0.99)
	Baseline MINI diagnoses eliminated b	Average f	0.99 (0.60, 1.38)	0.96	1.44 (0.84, 2.03)	0.15	Average f	0.58 (0.44, 0.71)	0.58 (0.39, 0.77)	0.40 (0.26, 0.54)
Comorbidity outcome
			Risk ratio					Estimated least squared mean
Domain	Measure (range)	Time	ESTEEM vs HIV Testing and Counseling (95% CI)	p d	ESTEEM vs LGBQ-affirmative Counseling (95% CI)	p d	Time	ESTEEM (95% CI)	HIV Testing and Counseling (95% CI)	LGBQ-affirmative Counseling (95% CI)
Number of comorbidities	Comorbidity counth (0 - 4)	Average f	0.84 (0.69, 1.02)	0.08	0.80 (0.68, 0.94)	0.01	Average f	1.21 (1.07, 1.37)	1.45 (1.23, 1.70)	1.52 (1.36, 1.70)
		4-months	0.86 (0.63, 1.18)	0.36	0.80 (0.62, 1.04)	0.09	4-months	1.37 (1.13, 1.65)	1.58 (1.23, 2.02)	1.70 (1.42, 2.04)
		8-months	0.91 (0.63, 1.30)	0.60	0.77 (0.59, 1.02)	0.07	8-months	1.14 (0.93, 1.40)	1.26 (0.93, 1.70)	1.48 (1.22, 1.78)
		12-months	0.75 (0.54, 1.03)	0.08	0.82 (0.62, 1.08)	0.16	12-months	1.14 (0.93, 1.40)	1.53 (1.19, 1.96)	1.39 (1.15, 1.68)

Note: CI = confidence interval; ESTEEM = Effective Skills to Empower Effective Men (test intervention); HAM-D = Hamilton Depression Rating Scale, ODSIS = Overall Depression Severity and Impairment Scale, BAI = Beck Anxiety Index, OASIS = Overall Anxiety Severity and Impairment Scale, SIAS = Social Interaction Anxiety Scale, BSI GSI = Brief Symptom Inventory; Global Severity Index, AUDIT = Alcohol Use Disorders Identification Test, SIP-AD = Short Inventory of Problems – Alcohol and Drugs, SIDAS = Suicidal Ideation Attributes Scale, MINI = Mini International Neuropsychiatric Interview

^aESTEEM vs LGBQ-affirmative Counseling 8-/12-month difference only evaluated if ESTEEM vs HIV Testing and Counseling p < 0.05/0.01; no p-values significant

^bLogistic model adjusted for study location, group, time, interaction of group and time

^cNegative binomial model adjusted for study location, group, time, interaction of group and time, baseline condomless sex acts

^dNo p-values significant by false discovery rate (FDR); see Table 4 for details

^eGeneralized linear model, adjusted for study location, group, time, interaction of group and time

^fPoint estimate calculated by averaging all data across 4, 8, 12 months, using marginal least square means with 95% confidence limits

^gTruncated to 1 due to normal approximation

^hComorbidity count of diagnosis of depression (HAM-D), anxiety (BAI), substance use (SIP-AD), HIV Transmission Risk Behavior; Generalized Poisson model adjusted for study location, group, time, interaction of group and time

Participants in the ESTEEM condition reduced their probability of engaging in any HIV-transmission-risk behavior at 8-month follow-up by approximately half. Participants in the ESTEEM, HIV testing and counseling, and LGBQ-affirmative counseling conditions had a probability of engaging in any HIV-transmission-risk behavior of 52% (95% CI 36%, 68%), 59% (95% CI: 38%, 80%) and 68% (95% CI: 53%, 82%), respectively.

Mental and Behavioral Health Outcomes

Comparisons of conditions for each time point are illustrated in Figure 2. There was no treatment-by-time interaction for any outcome at p = 0.10, so we estimated average treatment effects for secondary outcomes using marginal least square means with 95% confidence limits. When comparing conditions in terms of differences in these outcomes, only the p-value for SIP-AD for ESTEEM vs. HIV testing and counseling reached statistical significance (Table 2). On average, those in ESTEEM scored 1.7 (95% CI 0.55, 2.8) lower than those in HIV testing and counseling on SIP-AD. Table 4 provides the cutoffs used for evaluating statistical significance using the Benjamini and Hochberg (1995) false discovery correction.

Figure 2.

Average mental and behavioral health outcomes by treatment condition and follow-up time point. HAM-D = Hamilton Depression Rating Scale; BAI = Beck Anxiety Inventory; AUDIT = Alcohol Use Disorders Identification Test; SIP-AD = Short Inventory of Problems – Alcohol and Drugs

Table 4.

Raw p-values Used as Cutoffs for Evaluating Statistical Significance Using the Benjamini-Hochberg (1988) Method

			ESTEEM vs HIV Testing and Counseling		ESTEEM vs LGBQ-affirmative Counseling
Rank	Cutoff, a = 0.01	Cutoff, a = 0.05	Variable	Raw p	Variable	Raw p
1	0.001	0.005	SIP-AD	0.004	SIP-AD	0.04
2	0.002	0.009	OASIS	0.05	HAM-D	0.07
3	0.003	0.014	AUDIT	0.06	MINI, eliminate	0.15
4	0.004	0.018	BSI-GSI	0.10	OASIS	0.29
5	0.005	0.023	ODSIS	0.22	BSI-GSI	0.30
6	0.005	0.027	HAM-D	0.24	ODSIS	0.30
7	0.006	0.032	SIDAS	0.36	MINI, reduce	0.43
8	0.007	0.036	BAI	0.73	AUDIT	0.53
9	0.008	0.041	MINI, reduce	0.75	SIAS	0.73
10	0.009	0.045	SIAS	0.92	BAI	0.92
11	0.010	0.050	MINI, eliminate	0.96	SIDAS	0.93

For the secondary mental (i.e., symptoms of depression, anxiety, suicidality, and psychological distress) and behavioral (i.e., substance use) health outcomes, the direction and magnitude of effect sizes consistently favored ESTEEM when compared to HIV testing and counseling and LGBQ-affirmative counseling at 8-month and 12-month follow-ups (Table 3).

Table 3.

Effect Sizes for Outcomes When Comparing ESTEEM to HIV Testing and Counseling and ESTEEM to LGBQ-affirmative Counseling

		Effect Size (95% CI) a
	Time	ESTEEM vs HIV Testing and Counseling	ESTEEM vs LGBQ-affirmative Counseling
HAM-D	4-months	−0.18 (−0.28, −0.03)	−0.22 (−0.35, −0.10)
	8-months	−0.15 (−0.31, −0.06)	−0.45 (−0.58, −0.32)
	12-months	−0.39 (−0.52, −0.27)	−0.27 (−0.39, −0.14)
ODSIS	4-months	−0.41 (−0.54, −0.28)	−0.23 (−0.39, −0.10)
	8-months	−0.15 (−0.27, −0.02)	−0.18 (−0.30, −0.05)
	12-months	−0.13 (−0.003, −0.25)	−0.07 (−0.19, 0.06)
BAI	4-months	0.003 (−0.12, 0.13)	0.11 (−0.01, 0.24)
	8-months	−0.22 (−0.34, −0.09)	−0.14 (−0.26, −0.01)
	12-months	0.04 (−0.08, 0.17)	−0.02 (−0.14, 0.11)
OASIS	4-months	−0.54 (−0.65, −0.41)	−0.23 (−0.36, −0.11)
	8-months	−0.40 (−0.53, −0.27)	−0.21 (−0.33, −0.08)
	12-months	−0.15 (−0.27, −0.02)	−0.06 (−0.18, 0.06)
SIAS	4-months	−0.05 (−0.17, 0.08)	−0.01 (−0.13, 0.12)
	8-months	−0.10 (−0.23, 0.02)	−0.15 (−0.27, −0.02)
	12-months	0.12 (−0.01, 0.24)	0.06 (−0.07, 0.18)
BSI	4-months	−0.42 (−0.55, −0.29)	−0.10 (−0.22, 0.03)
	8-months	−0.30 (−0.42, −0.17)	−0.25 (−0.37, −0.12)
	12-months	−0.16 (−0.28, −0.03)	−0.12 (−0.24, 0.01)
AUDIT	4-months	−0.14 (−0.26, −0.01)	−0.01 (−0.13, 0.11)
	8-months	−0.36 (−0.48, −0.23)	−0.16 (−0.28, −0.03)
	12-months	−0.17 (−0.48, −0.05)	−0.02 (−0.14, 0.10)
SIP-AD	4-months	−0.31 (−0.42, −0.19)	−0.12 (−0.25, 0.001)
	8-months	−0.45 (−0.58, −0.32)	−0.33 (−0.45, −0.20)
	12-months	−0.39 (−0.52, −0.26)	−0.21 (−0.34, −0.09)

note: ESTEEM = Effective Skills to Empower Effective Men, HAM-D = Hamilton Depression Rating Scale, ODSIS = Overall Depression Severity and Impairment Scale, BAI = Beck Anxiety Index, OASIS = Overall Anxiety Severity and Impairment Scale, SIAS = Social Interaction Anxiety Scale, BSI = Brief Symptom Inventory, AUDIT = Alcohol Use Disorders Identification Test, SIP-AD = Short Inventory of Problems – Alcohol and Drugs

^aeffect size is Cohen’s D with 95% Confidence Interval

Transdiagnostic Outcomes

Averaged across all time points, participants in the ESTEEM condition had a 58% probability (95% CI: 44%, 71%) of eliminating all baseline MINI-derived mental health diagnoses, compared with a 58% probability (95% CI: 39%, 77%) for participants in the HIV testing and counseling condition and a 40% probability (95% CI: 26%, 54%) for participants in the LGBQ-affirmative counseling condition. ESTEEM was also not associated with significantly greater reduction in the elimination of MINI diagnoses than HIV testing and counseling (RR=0.99; 95% CI: 0.60, 1.38) or LGBQ-affirmative counseling (RR=1.44; 95% CI: 0.84, 2.03) (Table 2).

Further, averaged across all time points, participants in the ESTEEM condition had a 95% probability (95% CI: 91%, 100%) of experiencing a reduction in their number of baseline MINI-derived mental health diagnoses, compared with a 94% probability (95% CI: 87%, 100%) for participants in the HIV testing and counseling condition and a 93% probability (95% CI: 87%, 99%) for participants in the LGBQ-affirmative counseling condition. ESTEEM was not associated with significantly greater reduction in number of baseline MINI-derived mental health diagnoses than HIV testing and counseling (RR=1.01; 95% CI: 0.94, 1.08) or LGBQ-affirmative counseling (RR=1.03; 95% CI: 0.96, 1.09).

When comparing conditions in terms of reduction in number of comorbidities from baseline to follow-up, we did not find a significantly greater reduction for ESTEEM compared to HIV testing and counseling (RR=0.84, 95% CI: 0.69, 1.02). However, ESTEEM was significantly stronger than LGBQ-affirmative counseling in reducing the number of comorbidities (RR=0.80, 95% CI: 0.68, 0.94; Table 2, Figure 3). At baseline, participants in the ESTEEM condition had an average of 2.47 (out of 4) comorbidities (95% CI: 2.28, 2.66) and those in HIV testing and counseling and LGBQ-affirmative counseling had an average of 2.43 (95% CI: 2.15, 2.71) and 2.43 (95% CI: 2.25, 2.61), respectively. At 8-months, participants in the ESTEEM condition had an average 1.14 of 4 comorbidities (95% CI: 0.93, 1.40), whereas participants in the HIV testing and counseling condition had 1.26 (95% CI: 0.93, 1.70) and participants in the LGBQ-affirmative counseling condition had 1.48 (95% CI: 1.22, 1.78) (Table 2; Figure 3).

Figure 3.

Average comorbidity count by treatment condition and follow-up time point. Comorbidity count (range: 0-4) includes exceeding cutoff scores on (1) the Hamilton Depression Rating Scale (HAM-D; ≥17; Zimmerman et al., 2013), (2) Beck Anxiety Inventory (BAI; ≥16; Beck & Steer, 1993), (3) Short Inventory of Problems – Alcohol and Drugs (SIP-AD; ≥7), and (4) reporting any HIV-transmission-risk behavior

Discussion

Although ESTEEM was not associated with a significantly greater reduction than the two comparison conditions in the primary outcome (i.e., any HIV-transmission-risk behavior at 8-month follow-up), it was associated with a consistent pattern of relatively stronger, if not statistically significant (in the context of the false discovery correction), reductions in several mental and behavioral health outcomes compared to the two existing treatments, with effect sizes in the small-to-moderate range. ESTEEM exhibited a greater ability than LGBQ-affirmative counseling, but not HIV testing and counseling, to reduce the highly prevalent comorbidity of psychosocial health conditions affecting sexual minority men (Bränström & Pachankis, 2018; Mustanski et al., 2007). Further, neither HIV testing and counseling nor LGBQ-affirmative counseling was associated with a superior benefit compared to ESTEEM on any outcome.

The lack of statistically significant differences across treatment conditions is potentially explained by several considerations. The fact that nearly all comparisons were in the expected direction despite not reaching statistical significance suggests that the trial did not possess sufficient power to detect the effect sizes found here. In fact, most of the effects were small-to-moderate, whereas the trial was powered to detect somewhat larger effects, based on the waitlist-controlled pilot study (Pachankis et al., 2015a) and informed by previous studies of HIV testing and counseling (e.g., Kamb et al., 1998) and the fact that LGBQ-affirmative counseling does not specifically focus on sexual health. In the present trial, however, we obtained a much smaller difference between conditions than expected, with ESTEEM being associated with a 48% reduction in HIV-transmission-risk behavior, whereas HIV testing and counseling was associated with a 41% reduction. Based on the observed effect sizes from this trial, a subsequent trial would need to enroll 2,578 participants (with a 20% dropout rate) – 10 times as many as enrolled in the present study. The effect sizes found in the present study are similar to, or in several cases greater than, those reported in a recent meta-analysis of 16 randomized controlled trials that address sexual minority men’s co-occurring mental and sexual health challenges (Pantalone et al., 2020). In these trials—which were not typically focused on addressing minority stress and not typically based in CBT approaches—effect sizes were typically small for both mental health (d = 0.23) and sexual-risk behavior (d = 0.17; Pantalone et al., 2020), whereas effect sizes for several of the present study’s outcomes (e.g., depression, anxiety, substance use) at the 8-month follow-up assessment ranged from d = 0.31 to 0.58.

Several factors might have affected the “voltage drop” from the pilot study to the current trial (Beets et al., 2020). For instance, the nature of the HIV-transmission-risk outcome changed from the pilot study to the current trial. Specifically, the primary HIV-transmission-risk outcome in the pilot trial was condomless anal sex with casual partners, whereas this outcome was expanded in the current trial to capture the additional nuance introduced by advances in biomedical HIV prevention since that time. For instance, PrEP was not widely available during the pilot study and therefore not included in that study’s operationalization of the HIV-transmission-risk outcome, whereas adherent PrEP utilization served as an exclusion criterion for the current trial and rendered participants to be without HIV-transmission-risk behavior during follow-ups. These changes could have reduced our power to find an effect given that routine PrEP use provides more global protection against HIV-transmission-risk than more event-level condom usage. In terms of mental health outcomes, the pilot study relied on self-reported questionnaires of mental health outcomes, whereas the current trial supplemented these questionnaires with interviewer-based assessment of mental health, which are potentially less susceptible to bias.

This trial also employed two intensive control conditions and all three conditions provided, at a minimum, screening for mental health problems at intake, a brief clinical assessment in the case of current emotional distress, and referrals to more in-depth care as needed. In this way, even the brief HIV testing and counseling condition had parallels to an approach known as screening, brief intervention, and referral to care (SBIRT), which shows robust effects for certain behavioral health outcomes (Babor et al., 2007). Further, although the content of LGBQ-affirmative counseling was not standardized across participants, and although coding of LGBQ-affirmative counseling sessions revealed minimal CBT content and ESTEEM content, the general principles of LGBQ-affirmative counseling (American Psychological Association, 2021) share overlap with the principles informing ESTEEM (Pachankis, 2014). This overlap in principles, if not exact content, raises the possibility that similarity in these two treatments could explain the modest effect sizes when comparing them. Overall, findings highlight the promise of the two control interventions for reducing psychosocial health risks and call for future research that considers how these interventions can be best enhanced to do so. The fact that HIV testing and counseling was associated with reductions in several mental health outcomes, despite its single-session format, highlights the potential promise of this intervention for addressing outcomes beyond its intended focus in an efficient, cost-effective manner.

Another potential explanation for the lack of more consistently significant condition comparisons concerns the preregistered application of the false discovery rate, which guards against Type 1 error when examining numerous secondary outcomes. At the same time, this stringent correction potentially obscures the fact that transdiagnostic treatments such as ESTEEM are geared toward addressing a heterogeneous set of outcomes depending on the presenting concerns of each individual. Consequently, such treatments do not select for elevations on any one specific primary mental health problem, with a corresponding primary mental health outcome variable (e.g., in the case of a depression trial, where all participants have major depressive episode upon entry). Therefore, rather than examining whether ESTEEM led to superior reductions across one primary outcome such as HIV-transmission-risk behavior or across numerous particular outcomes in the entire sample, perhaps a more suitable research question for transdiagnostic treatments is the number of any outcomes that was reduced. In fact, in our exploratory (non-preregistered) comorbidity analysis, we found that ESTEEM was more efficacious than LGBQ-affirmative counseling (although not more efficacious than HIV testing and counseling) at reducing the number of presenting concerns per individual regardless of whether those presenting concerns were mental (e.g., depression), behavioral (e.g., substance use), and/or sexual (e.g., HIV-transmission-risk behavior). Future research on the efficacy of this transdiagnostic treatment might also operationalize efficacy in terms of responder status for outcomes clinically present at baseline, consistent with prior research on other transdiagnostic treatments (e.g., Ellard et al., 2010). Finally, all conditions provided LGBQ-affirmative support, which might be particularly likely to be missing in the lives of comorbidly affected young SMM and might represent an essential common factor in psychotherapeutic practice with this population regardless of the specific modality in which it is delivered (Boroughs et al., 2015; Hope & Chappell, 2015; O’Shaughnessy & Speir, 2018). Relatedly, recruitment materials emphasized the LGBQ-affirmative nature of the study focused on improving young SMM’s health, which might have induced expectancies of support and improvement for all participants.

Results of this study must be interpreted in light of several limitations. First, the trial occurred in two large US cities, known for relatively accepting attitudes toward sexual minority individuals and visible LGBQ communities. Although the sample was diverse in terms of race/ethnicity and socioeconomic status, knowing whether results might generalize to other US and global locales and populations, perhaps with less LGBQ acceptance, awaits future research. The adaptation of ESTEEM to other modalities (e.g., online) may be particularly useful in reaching LGBQ populations in rural spaces and in environments where mental health care providers are either unavailable or unlikely to provide LGBQ-affirmative mental health care. Second, LGBQ-affirmative counseling was selected as a control group because it ethically provides a safe environment for young SMM to discuss the often sensitive and personal contexts of their current concerns. However, whether ESTEEM might yield stronger effects than a non-LGBQ-focused treatment, such as general CBT (i.e., not focused on minority stress), remains unknown, consistent with the general state of cultural adaptation research in which most studies of adapted treatments have not used non-adapted versions of the same treatment (e.g., CBT) as controls (Hatzenbuehler & Pachankis, 2021). Third, the ESTEEM treatment was delivered in clinical research settings under close supervision by the treatment developers. Future research is needed to determine the extent to which ESTEEM can be implemented with fidelity in community settings without these internal controls. Relatedly, ESTEEM was delivered in university-affiliated research offices whereas LGBQ-affirmative therapy was delivered in community settings, perhaps introducing a threat to internal validity. Fourth, future research is needed to understand the mechanisms through which efficacious treatments for young SMM, like ESTEEM, might operate, whether through general psychosocial risks (e.g., emotion dysregulation, social isolation), minority stress risks (e.g., internalized homonegativity, rejection sensitivity), or both. Finally, future research is needed to understand whether participant factors (e.g., baseline symptom severity, minority stress processes; Keefe et al., 2021) or contextual factors (i.e., structural stigma; Hatzenbuehler, 2016) might moderate the efficacy of any of the interventions examined here.

In conclusion, ESTEEM represents a promising intervention for addressing the mental and sexual health challenges that often occur and disproportionately affect young SMM. Although results suggest that HIV testing and counseling and LGBQ-affirmative counseling also represent promising approaches for these outcomes, ESTEEM was associated with small-to-moderate effect sizes (though not statistically significant) for reducing both individual outcomes and their co-occurrence. The transdiagnostic basis of ESTEEM lends itself to efficient dissemination for addressing the co-occurring concerns affecting this population. ESTEEM also responds to the growing ethical and professional mandate to address minority stress, as relevant, in psychosocial treatments for sexual minority individuals (American Psychological Association, 2021). Although the ultimate source of the co-occurring health risks facing young SMM lies in stigmatizing societal institutions and attitudes (Hatzenbuehler, 2016), until these inequities are eradicated, mental health providers face the imperative to support this population in coping with minority stress. ESTEEM, along with other LGBQ-affirmative treatments, offer clinical approaches capable of empowering the effective coping of this population.

Public Health Significance Statement.

As long as young sexual minority men (SMM) face minority stress, identity-affirming and effective psychological interventions are needed to address the impact that this stress has on their mental and sexual health. This study found evidence that an LGBQ-affirmative, minority-stress-focused, transdiagnostic CBT intervention called ESTEEM was associated with reductions across numerous mental and sexual health outcomes among young SMM experiencing co-occurring mental and sexual health concerns, though not significantly greater than the two existing interventions to which ESTEEM was compared.