Skip to main content
BMJ Open logoLink to BMJ Open
. 2022 Mar 8;12(3):e054792. doi: 10.1136/bmjopen-2021-054792

Adolescent stress experiences over time study (ASETS) protocol: design and methods of a prospective longitudinal study of sexual minority adolescents in the USA

Sheree M Schrager 1,, Mary Rose Mamey 2, Harmony Rhoades 2, Jeremy T Goldbach 3
PMCID: PMC8915334  PMID: 35264352

Abstract

Introduction

Sexual minority adolescents (SMA) report higher rates of anxiety, self-harm, depression and suicide than heterosexual peers. These disparities appear to persist into adulthood and may worsen for certain subgroups, yet the mechanisms that drive these concerns remain poorly understood. Minority stress theory, the predominant model for understanding these disparities, posits that poorer outcomes are due to the stress of living in a violently homophobic and discriminatory culture. Although numerous studies report associations between minority stress and behavioural health in adolescence, no study has comprehensively examined how minority stress may change throughout the course of adolescence, nor how stress trajectories may predict health outcomes during this critical developmental period.

Methods and analysis

Between 15 May 2018 and 1 April 2019, we recruited a US national sample of diverse SMA (n=2558) age 14–17 through social media and respondent-driven sampling strategies. A subset of participants (n=1076) enrolled in the longitudinal component and will be followed each 6 months until 1 July 2022. Primary outcomes include symptoms of depression, anxiety and post-traumatic stress disorder; suicidality and self-harm and substance use. The key predictor is minority stress, operationalised as the Sexual Minority Adolescent Stress Inventory. We will use parallel cohort-sequential latent growth curve models to test study hypotheses within a developmental framework.

Ethics and dissemination

All participants provided assent to participate, and longitudinal participants provided informed consent at the first follow-up survey after reaching age 18. All study procedures were reviewed and approved by the University of Southern California Social–Behavioral Institutional Review Board, including a waiver of parental permission given the potential for harm due to unintentional ‘outing’ to a parent during the consent process. The final anonymous data set will be available on request, and research findings will be disseminated through academic channels and products tailored for the lay community.

Keywords: mental health, public health, child & adolescent psychiatry


Strengths and limitations of this study.

  • This study leverages a newly developed, valid and psychometrically sound measure of minority stress in a large, diverse national sample of adolescents.

  • The longitudinal cohort design permits the first examination of change in minority stress experiences over time among adolescents.

  • The cohort sequential modelling approach also supports the first examination of how minority stress influences health across adolescence.

  • All outcome measures are self-reported and may be subject to recall and responses biases; no confirmatory behavioural data will be collected.

  • Generalisability of study findings may be constrained by study eligibility criteria, strict data quality procedures and recruitment methods.

Introduction

Sexual minority (eg, lesbian, gay, bisexual and pansexual) adolescents (SMA) experience significant behavioural health disparities compared with their heterosexual peers. In particular, SMA experience higher rates of internalising psychopathology including depression, anxiety and self-harm1–7 and externalising behaviours such as substance use8–11 and suicide attempt and completion.12–14 Longitudinal studies suggest that these disparities persist into young adulthood and may even worsen. For example, data from a national study of adolescents (Add Health) showed that average longitudinal trajectories for substance use among SMA are disparate from heterosexual youth beginning in early adolescence and increase as youth transition into young adulthood.15 When examining individual trajectories of suicidality, all sexual minority groups (lesbian, gay, bisexual and mostly heterosexual) reported higher rates of suicidality across all four waves than their heterosexual peers, from mid-adolescence to early adulthood,16 a phenomenon also found in a recent systematic review by Gilbey et al.17

There are also behavioural health disparities among SMA by demographic subgroup. For example, sexual minority girls are more likely to report both considering and attempting suicide than sexual minority boys,18 19 and bisexual youth show larger substance use disparities than other sexual minority groups.15 SMA living in rural areas also experience different behavioural health outcomes than their urban counterparts due to confidentiality concerns, values and limited access to cities with more extensive peer networks20 and a more comprehensive social support system.21 As there are likely to be subgroup differences among racial and ethnic lines as well, scholars have called for attention to racial and ethnic diversity in sexual minority research generally.22–24 Even in large meta-analytic studies that include adolescents and/or young adults, the lack of racial and ethnic diversity in sampling is noted as a significant limitation that often precludes subgroup comparisons.19 25 Recent studies of Black26 and Latinx27 adolescents do suggest that intersectional differences may exist, and understanding their experience is increasingly relevant: although population estimates specific to adolescents are lacking, national survey data suggest that racial and ethnic minority youth are more likely than white segments of the US population to identify as SMA.28

The primary theoretical framework for understanding the disparities found among sexual minorities is the minority stress theory (MST),29–31 which has been endorsed by the Centers for Disease Control and Prevention,32 the National Academy of Medicine33 and Healthy People 2030.34 MST suggests that discrimination, violence and victimisation due to a pervasive homophobic culture are the primary sources of stress and most probable driving mechanisms of mental health disparities among sexual minorities, including SMA.30 35–38 Numerous cross-sectional studies have attributed poor behavioural health outcomes among adolescents to minority stressors, such as negative disclosure experiences with family and peers,1 36 37 39 becoming homeless on disclosure,40 in-school victimisation (bullying) by students and faculty members41–43 and experiences of violence.19 44–46 However, no study has ever comprehensively examined the relationship between minority stress and health outcomes longitudinally among adolescents. Despite recognition that stigmatising experiences can disrupt adolescent development and contribute to negative outcomes,47 the gap between theoretically predicted relationships and empirical evidence to support them is largely due to four key concerns:

  1. Studies of minority stress during adolescence have been fraught with poor psychometric measurement.20 48 A review of psychometric measurements assessing discrimination against sexual minorities found that across 162 articles, nearly all had suboptimal psychometric properties.49 Few studies have used empirically validated measures, and most measures had been developed using small investigator-led samples or adapted from measures with adults in other minority populations.25 Previously available general stress measures, even those validated for use with adolescents, do not allow us to differentiate between common developmental stressors and those associated with minority stress.

  2. There is an absence of studies examining minority stress and behavioural health in adolescents over time. Only six studies (with four unique samples) have examined the relationship between minority stress and subsequent behavioural health outcomes, and each has several major limitations: (a) lack of a well-constructed comprehensive measure of minority stress for adolescents; (b) reliance on small regional samples and (c) lack of repeated-measures analyses and trajectory modelling to assess patterns of change in minority stress during this critical developmental time period.31 50–52 Although the field has generally assumed minority stress is the most probable cause of persisting behavioural health concerns among SMA, no study has examined this directly. As a related concern, no studies have provided evidence that SMA can be effectively retained over time outside of general population studies. Although not a primary outcome of the current study, establishing the feasibility of population retention is a critical step for future prospective research.

  3. Although some subgroup differences in behavioural health outcomes have been documented among adolescents, their determinants are not well understood. As previously described, differential outcomes are noted in sexual minority samples by race, ethnicity, gender and geography, and authors have called for increased attention to subgroup analyses in future research.16 19 25 53 These experiences have been explored in primarily young adult and adult samples.54–56 Although some evidence is emerging to support the assumption is that minority stress also drives these disparities,43 57–61 few studies have systematically explored subgroup differences in minority stress over time, particularly among youth. A large study that is well powered to examine differences among multiple demographic groups—that is, able to model more than simple binary comparisons—remains needed.

  4. The presence or absence of protective factors may add to the confusion. Some studies suggest avoidance strategies62 or emotionally focused cognitive restructuring63 may be helpful; others have recommended finding accepting friends, having supportive parents or family members, identifying supportive adults at school and relying on SMA community resources (eg, gay-straight alliances, SMA community centres) as methods for coping with minority stress.21 However, not all subgroups of SMA may have these opportunities. For example, youth who live in rural areas may have less access to affirming resources20 and be more likely to live in areas with less protective school policies.64 Thus, the relationships between minority stress, demographics, protective factors and outcomes remain poorly understood.

The current study is the first to address these four major gaps in the extant literature. We can now measure minority stress in adolescents with a psychometrically sound instrument, the Sexual Minority Adolescent Stress Inventory (SMASI), which was developed and validated by the research team in prior work funded by the National Institutes of Health (NIH).65–67 Using this measure, we will conduct a systematic investigation of minority stressors and behavioural health over time in a large, diverse national sample. With repeated measures of minority stress and a modelling approach (cohort sequential latent growth curve modelling (LGCM)—see Data Analysis) that considers change across age rather than time, we can answer questions not previously addressed, such as whether minority stress increases over time as young teenagers develop throughout adolescence; when do minority stressors peak; whether there are demographic differences in the frequency, severity and pattern of minority stressors and whether changes in minority stressors over time predict corresponding changes in health outcomes over time. Furthermore, we can test whether trajectories of minority stress are inversely associated with protective factors over time and if they too differ by demographic subgroup.

Thus, the present study will serve as one of the first longitudinal studies conducted with this vulnerable population. We address critical methodological design factors necessary to conduct high-quality longitudinal research with SMA, including: (a) a safe and effective recruitment approach, with built-in mechanisms to protect SMA from being ‘outed’ via study participation, which could increase risk of victimisation (eg, kicked out of home); (b) repeated measures over time of important psychosocial predictors and outcomes; (c) recruitment of participants at ages 14–17, because they are a particularly neglected subpopulation in SMA studies68 and (d) respondent-driven sampling methods to recruit youth who may have not disclosed their sexual orientation to others, including their parents, resulting in a lack of both scientific and clinical understanding about them. On completion, the study will provide critical information needed to inform the nature and timing of intervention efforts for this high-need, underserved and difficult-to-reach population of youth.

Methods and analysis

Sample selection

Population definition

Recent studies suggest that upwards of 15% of youth do not identify as exclusively heterosexual.69 70 Throughout this protocol, we use the term SMA to refer to adolescent individuals who endorse same-sex attraction or identity. Attraction includes romantic or sexual feelings, whereas identity describes how youth label themselves (eg, lesbian, gay and bisexual).71 These are consistent with constructs commonly used to operationalise sexual orientation.72 73 We recognise adolescents generally as youth aged 13–20, a common international convention.74 However, we restricted recruitment in this study to youth aged 14–17, as we have in our preliminary work, given literature suggesting youth commonly begin to define their sexual identity during these years.75 We also required participants to be cisgender, that is, to express a gender identity congruent with their sex assigned at birth, at the time of recruitment. At the time this study was open to enrolment, the SMASI had only been validated with cisgender adolescents. As the primary purpose of this study was the longitudinal validation of the SMASI instrument, we chose to mirror the inclusion criteria on which the SMASI was initially developed and validated in order to reduce error variance when assessing minority stress attributable to sexual identity, given the present inability to differentiate between sexual and gender minority stress among adolescents who are both sexual and gender minorities. Although transgender and non-binary youth were excluded from enrolling in the baseline sample, maintaining a cisgender identity over time was not a requirement for continuation in the longitudinal portion of the study; indeed, we expect changes in gender identity over time, and will assess those in later waves (see Measures).76 A separate NIH-funded study (R21HD082813-01A1) is now underway to examine gender minority stress among transgender and non-binary youth and parse out sexual versus gender minority stress experiences.

Study eligibility

Youth were eligible to participate in the study if they were at least 14 and no more than 17 years old; were cisgender male or female (ie, reported a current gender identity consistent with their sex assigned at birth); resided in the USA, as determined by ZIP code; identified as not 100% heterosexual using Add Health guidelines (ie, identified as mostly heterosexual, bisexual, gay, lesbian or unsure)77 and were willing and able to provide assent to participate.

Stratification variables

To ensure geographic diversity, ZIP code was recoded into two additional variables: region and urbanicity. Region (West, Southwest, Midwest, Northeast and Southeast; table 1) was based on the state associated with the participant’s reported ZIP code.

Table 1.

Assignment of US states to regions

US region US states
West Alaska Idaho Utah
California Montana Washington
Colorado Nevada Wyoming
Hawaii Oregon
Southwest Arizona Oklahoma Texas
New Mexico
Midwest Illinois Michigan North Dakota
Indiana Minnesota Ohio
Iowa Missouri South Dakota
Kansas Nebraska Wisconsin
Northeast Connecticut Massachusetts Pennsylvania
Delaware New Hampshire Rhode Island
Maine New Jersey Vermont
Maryland New York
Southeast Alabama Kentucky South Carolina
Arkansas Louisiana Tennessee
Florida Mississippi Virginia
Georgia North Carolina West Virginia

Urbanicity (rural or urban) was determined based on the rural urban commuting area (RUCA)78 codes. Specifically, ‘urban’ was defined as a ZIP code corresponding to RUCA codes of 1.0, 1.1, 2.0, 2.1, 4.1, 5.1, 7.1, 8.1 and 10.1. ‘Rural’ was defined as all other valid RUCA codes. When a ZIP code was associated with a RUCA 3.1 score, that score was used; for ZIPs that were not assigned a RUCA 3.1 score due to changes in the classification system between RUCA versions 2 and 3, the RUCA 2.0 score was used.

Participant recruitment

Targeted advertising

Initial participants were recruited through advertising on Facebook/Instagram (which now share a single advertising platform) and YouTube. Advertisements varied slightly by platform, but all included language asking youth to ‘Share Your Voice’ and described basic details of the research study and incentives that participants could earn. Advertising was stratified by gender, geographic region and urbanicity. This resulted in 20 target cohorts, as each of the five regions encompassed four unique groups: rural males, rural females, urban males and urban females. We used two different sets of advertising images: one featuring females (for the female groups) and one featuring males (for the male groups). To reach each of these groups, general specifications included age (14–17 years), gender (women or men) and location. Facebook/Instagram allows bulk uploading of up to 2500 ZIP codes per ad. A total of 44 targeted ads were required to reach all possible combinations of region, urbanicity and gender, as some combinations included nearly 10 000 eligible ZIP codes. Facebook/Instagram also allows advertisers to target audiences based on interests. Using keywords enumerated by members of the research team, we identified specific interest terms by gender. Male-targeted interests included Gay-friendly; Gay, Lesbian, Bisexual, Transgender, Straight Alliance; Homosexuality; LGBT community; LGBT symbols and Pansexuality. Female-targeted interests included all of the male-targeted interest keywords plus Lesbian Connection and Lesbian Romance.

Simultaneously, the research team identified YouTube channels for review using keywords including LGBTQ [lesbian, gay, bisexual, transgender, and queer or questioning], gay, coming out, transition and trans. Channels were reviewed for visibility, reach and engagement of each channel, operationalised as the number of subscribers per channel and number of video views for each channel’s three most-viewed videos. We initially identified 47 possible YouTube channels that had high visibility and engagement among sexual and gender minority adolescents, using a combination of keyword searches (eg, LGBTQ, gay, coming out and transition) and subscriber and video view counts; after reviewing this list, we advertised to 23 channels that were verified YouTube accounts, able to accept advertisements and agreed up on by the study team as being relevant to SMAs. Using the Google advertising system, we placed advertisements for the adolescent stress experiences over time study directly on the pages of those channels.

Respondent-driven sampling

Respondent-driven sampling (RDS) is a type of chain-referral sampling that allows for identified members of a hidden group, called ‘seeds’, to recruit other group members from their personal networks.79 Participants who completed their survey and were initially deemed eligible for retention were asked if they might be interested in referring friends who they thought might be eligible to participate. Participants who confirmed their interest in referring friends to the study were provided with an email that contained three unique survey links as well as two different language prompts to encourage peers to participate. In return for successfully recruiting an eligible participant who completed the survey, the recruiter participant (seed) was paid $10 per referral for up to three eligible participants.

Baseline study procedures

Initial eligibility screening

Advertisement clicks and referral links all directed youth to a screening page in Qualtrics that asked a series of demographic questions to determine their eligibility based on age in years, gender, ZIP code and sexual attraction. Ineligible participants were thanked for their interest in the study and then rerouted to a separate Qualtrics survey where they could optionally provide contact information (email and/or phone number) to be included in outreach for future studies. Eligible participants were shown the Institutional Review Board (IRB)-approved study assent text and asked to confirm assent in order to proceed with the main survey, implemented in Qualtrics (see Measures).

Postsurvey data collection

After completing the survey, the participant was rerouted to a separate Qualtrics survey for payment in order to keep their personally identifiable information separate from their main study data. This payment survey asked the participant for their private email address at which to receive an electronic gift card. Participants were also asked if they knew other sexual minority youth, and if so, whether they would consider referring any of those youth into the study, to aid RDS recruitment. Finally, participants were asked whether they would be interested in participating in the longitudinal study and given fields to provide up to five different contact methods if so. Contact options included email, phone numbers for call/text and usernames for Facebook, Twitter, Snapchat and any other social media accounts that allow for personal messaging. Participants were able to rank their provided methods of contact in order of preference. This step in the process was critical to establishing retention for the longitudinal study.

Final eligibility determination

Each business day, a study team member downloaded any new surveys from Qualtrics. Variables were created to represent region, urbanicity, response declinations (total number of ‘Decline to answer’ responses across the entire survey); survey duration and attention validation (number of attention-control questions the respondent answered correctly).80 Participants who failed to complete the entirety of the survey–that is, they exited the survey prior to completing and being routed to the payment survey–were excluded and could not be paid due to lack of contact information. Participants determined to have engaged in any type of fraudulent activity were also immediately excluded from both study eligibility and pay. ‘Fraudulent activity’ included providing information or response patterns, either within the main survey data or on the payment and contact information survey, that confirmed duplicate response by a previous participant, or screening out of a first survey attempt (ie, determined to be ineligible) and immediately reaccessing the survey with false responses in an attempt to access the full survey.81–84 Fraudulent participants were identified by duplicate IP address, duplicate email and/or contact information, similar patterned responses throughout survey (including open-ended responses with identical or unique wording) and/or survey time stamps. Fraudulent participation was not compensated even if sufficient contact information was provided.

Participants who completed the survey but provided very low quality data, defined as either an unrealistically short survey completion time (≤10 min), a low attention-control score (≤1 out of 4 correct responses) or very high (≥35) ‘Decline to answer’ response count, were compensated for their participation but were excluded from the baseline data set and not invited to participate in the longitudinal study or refer peers via RDS. Participants who had survey duration times of 10–15 min, attention-control scores of 2 and moderately high (≥25) responses of ‘Decline to answer’ were compensated for participation and further evaluated for inclusion on a case-by-case basis. By applying all of the above-described procedures prior to longitudinal recruitment, we ensured that only participants who provided valid and trustworthy data would be enrolled in the longitudinal study.

Incentive compensation

All baseline participants who were eligible for compensation, whether or not their data were retained for analysis, were sent a $15 Amazon gift card to the private email address they provided in the payment survey. Participants whose data were retained for analyses were assigned a unique four-digit participant identifier at this time.

All participants were recruited into the study and completed their baseline surveys (n=2558) between 15 May 2018 and 1 April 2019. Figure 1 illustrates the number of individuals retained and excluded at each step of the baseline recruitment and data collection process.

Figure 1.

Figure 1

CONSORT flow diagram for enrolment into baseline study phase (final n=2558). CONSORT, Consolidated Standards of Reporting Trials

Longitudinal study procedures

Longitudinal enrolment

Approximately 1 week from the date of a participant’s baseline survey completion, participants who expressed interest in longitudinal participation were entered into a master tracking log file. This artificial delay helped to ensure that we could detect participants willing to engage in fraudulent behaviour, including participants who were trying to take the baseline survey multiple times in an attempt to receive multiple payments, prior to inviting them to be part of the longitudinal study. Participants who reached this longitudinal recruitment stage were contacted by a research assistant in real-time via the participant’s preferred contact method, using a study-specific username or account shared by the research team. Participants were first reminded that they recently completed an online survey.

In an effort to protect their privacy and ensure that we were speaking with the right individual, we asked them to please tell us what that survey was about. Participants who provided the correct information (eg, ‘LGBT youth’) were asked if they were interested in learning more about the longitudinal study. Participants who expressed interest were given information about the study outlining their involvement, including a written information sheet.

Those who agreed to participate were then asked to confirm or update their contact information, and the research assistant verified the participant could receive emails from the study team that did not end up in their spam/junk folders. Participants were reminded that the study team’s next contact with them would be through an automated monthly check-in survey every month (see Monthly Check-In Surveys) and that they would receive an email in approximately 6 months for their next full survey. Finally, they were provided with all methods of contact to reach the study team and were encouraged to reach out in the event they had questions, concerns, or comments.

RDS referrals

Longitudinally enrolled participants were given the option to refer peers into the baseline survey for an additional incentive, that is, RDS referrals. Participants who stated they may know others who might be interested were provided three custom Qualtrics referral links, which contained an embedded RDS code that both identified the new participant as an RDS referral and allowed the study team to link the new survey to the referring participant for a referral payment. All referred participants went through the same validation, eligibility and payment process as those who entered the study through direct outreach methods. Additionally, the participant who referred them was provided with a $10 Amazon gift card as a referral incentive. Referrers were not paid referral incentives for distribution of survey links to youth who were ineligible for participation or those whose surveys were excluded from retention due to low data quality. Participants who attempted to refer themselves were easily identified by the quality assurance protocol previously described; in the case of self-referrals, the participant was immediately excluded from both the baseline and longitudinal study due to their demonstrated willingness to defraud the study team.

Monthly check-in surveys

Because the study was conducted entirely online, having up-to-date contact information for all participants was of critical importance. Additionally, with 6 months in between full surveys, it was important to have more regular contact with participants in order to maintain rapport and interest in the study. Thus, a brief check-in survey, consisting of one item asking whether any of the participant’s contact information had changed within the last 30 days, was automatically emailed to each longitudinal participant near the first day of every month. If a participant indicated that their contact information had changed, they were then prompted to provide any new or updated contact information. If a participant failed to respond to the automated check-in survey by the 15th of each month, a research assistant would manually reach out to them once through each of the participant’s preferred contact methods. Each check-in survey was accompanied by a raffle where all respondents to the check-in survey within the calendar month were entered into a random drawing to receive a $100 Amazon gift card, regardless of whether their contact information had changed.

Longitudinal follow-up surveys

A unique link to each Qualtrics follow-up survey was created by the study team for each participant. This link, provided to the participant once they became eligible to complete the survey, contained embedded information about the date on which they completed their previous survey along with their assigned unique participant identifier. This allowed information about prior participation dates to be prepopulated in survey items requesting retrospective information in an effort to aid in recall.

At the start of every week, all participants whose follow-up survey date fell within that week (ie, a multiple of 6 months after their baseline survey date) were sent an automated survey link in an email from the Qualtrics platform. Automated messages were sent on Monday, Wednesday and Sunday of the first week of eligibility. Participants who failed to complete their follow-up survey by the third automated attempt would begin to receive manual messages from research assistants containing their unique embedded survey link. Manual attempts to contact the participant were first sent via the participant’s most recently indicated preferred contact method. After 3 months, their survey window would close and participants would not be able to complete that wave of data collection to ensure any two consecutive waves of data collection would reflect a minimum time difference of 3 months apart. Follow-up data collection began on 15 August 2018, and will conclude on 1 July 2022 when the 3-month window of the 36month survey closes.

Longitudinal eligibility and payment determination

When a participant completed a follow-up survey, the participant was rerouted to the separate Qualtrics survey page for payment using the same procedures as at baseline. Each business day, a study team member would access the main survey through Qualtrics and download the previous day’s surveys. Variables were again created in each follow-up data set including survey duration, attention control score and count of ‘decline to answer’ responses. Participants who failed to complete the entire survey, or who had very low quality data (as defined above), were encouraged to retake the survey with an explanation provided as to why they were asked to retake the survey. This outreach process continued for five total manual attempts alternating every other day. This process was replicated at all subsequent survey waves. The longitudinal enrolment and retention diagram is shown in figure 2; to date, six participants have withdrawn from continued follow-up, bringing the longitudinal sample to n=1070.

Figure 2.

Figure 2

CONSORT flow diagram for enrolment and retention in longitudinal study phase (current n=1070). CONSORT, Consolidated Standards of Reporting Trials.

Measures

Beginning with baseline data collection, the following measures were collected:

Outcomes

We focus analysis on understanding behavioural health, consistent with terminology from the Substance Abuse and Mental Health Services Administration,49 to describe an integrated approach focused on the promotion of emotional health and prevention of mental illness, alcohol and other drug use and associated outcomes (eg, suicide). Behavioural health outcomes included: depressive symptoms, measured with the CESD-4;85 anxiety, assessed with the Generalised Anxiety Disorder 7-item measure (GAD-7);86 symptoms of Post-Traumatic Stress Disorder (PTSD),87 using the Abbreviated PTSD Checklist-Civilian (PCL-C) and five questions from the Youth Risk Behaviour Survey (YRBS) to assess suicidality and self-injury.88 All responses were recoded to binary indicators of suicidal ideation, plan, attempt, attempt resulting in injury and self-injurious behaviour. Finally, YRBS items were also used to assess lifetime and past-30-day substance use,88 including binary indicators of whether a participant had used alcohol, tobacco, marijuana, prescription pain relievers, prescription tranquillisers and prescription stimulants.

Key predictor

Minority stress

The focal measure in this study is the SMASI65–67 (whole-scale omega (ω)=0.97), which relies on 54 main items to measure 10 domains of minority stress: social marginalisation (eight items, ω=0.93), family rejection (11 items, ω=0.94), internalised homonegativity (seven items, ω=0.94), identity management (three items, ω=0.77), homonegative climate (four items, ω=0.88), intersectionality (three items, ω=0.82), negative disclosure experiences (five items, ω=0.77), religion (five items, ω=0.93), negative expectancies (three items, ω=0.81) and homonegative communication (five items, ω=0.77). An optional subscale assessed stress experiences at work among youth with any work history (10 items, ω=0.96). At baseline, youth were asked if they have had each experience ‘ever’ and ‘in the past 30 days’; the stem language changes to ‘since (they) last took the survey’ and ‘in the past 30 days’ at subsequent time points.

Demographic covariates

Variables used for eligibility screening included age (in years). Sex assigned at birth (male or female) was used in concert with gender identity (response options included male; female; trans male/trans man; trans female/trans woman; genderqueer; gender non-conforming; non-binary; gender identity not listed here) to determine cisgender status. For eligibility screening that depended on survey programming logic, sexual minority identity was determined with the recommended item from Add Health guidelines:77 ‘If you had to pick one of the following options, please choose the description that best fits how you think about yourself’, with all response options other than ‘100% heterosexual (straight)’ qualifying for eligibility. ZIP code was assessed to verify US residence and subsequently recoded for stratification into urbanicity and region as previously described.

Additional demographic variables included open-ended sexual identity, assessed by asking ‘What would you say is your sexual orientation or identity?’ and a text box for an open-ended responses that could be recoded for later analyses. Race and ethnicity was a forced-choice item with response options of Native American/American Indian/Alaska Native; Asian/Pacific Islander; Black or African American; White/Caucasian; Latino/Hispanic; Multiracial (with a text box to specify) and Race/ethnicity not listed here (with a text box to specify). Gender expression was captured with one item asking, ‘A person’s appearance, style, dress, or the way they walk or talk may affect how people describe them. How do you think other people would describe you?’ Response options were on a 7-point Likert-type scale from ‘Very feminine’ to ‘Very masculine.’ School enrolment was captured with a binary indicator of whether participants are currently enrolled in school, and educational attainment measured the highest grade already completed (less than seventh grade; seventh grade; eighth grade; ninth grade; 10th grade; 11th grade; high school graduate or GED; some college; trade school certification or Associate’s (AA) degree; Bachelor’s (BA/BS) degree or higher). Additionally, participants were asked whether they were eligible for free or reduced-price lunch at the school they most recently attended as a proxy for socioeconomic status (response options: ‘Yes’, ‘No’, ‘I don’t know’). Work status was assessed with one item asking whether participants are currently working, with response options of ‘Yes, full-time’; ‘Yes, part-time’; ‘No, but I have previously had a job’ or ‘No and I have not previously had a job’. In follow-up surveys, participants were asked whether they had worked since the last time they took the survey (‘Yes’ or ‘No’). Living situation was assessed by asking participants with whom do they currently live. Response options included two parents; mother only; father only; grandparents or other relatives; foster parents; group home; alone or with roommates in own apartment or home; boyfriend/girlfriend/romantic partner/spouse; with friends or couch-surfing; homeless or on the street and other (with a text box to specify). Participants were asked whether they had become involved in the foster care system (‘Yes’, ‘No’ or ‘Unsure’). Participants were asked about their primary language spoken (a) at home and (b) with their friends; response options included English; Spanish or another language (with text box to specify). Participants were asked to indicate their personal and family religion from a comprehensive list.89 Although not used in the eligibility process, sexual attraction to men, women and gender non-conforming people was also assessed with Likert-type response options ranging from ‘Not at all’ to ‘Extremely’

Theoretical covariates and protective factors

Participants completed the Perceived Stress Scale (PSS, a measure of general stress),90 91 to ensure that we can control for other common adolescent stress experiences unrelated to minority stress. Social support92 from friends, family and a significant other was measured with the Multidimensional Scale of Perceived Social Support. Disclosure questions were asked to understand categories of individuals who may know the participant is LGBTQ (baseline) or to whom, if anyone, the participant has disclosed their sexual orientation to for the first time since the last survey (follow-up surveys). Options included mother; father; siblings; other adult relatives; peers/supervisor(s) at work; members of the participant’s religious community; teacher(s); peers at school, childhood friends; heterosexual friends; LGBTQ friends and girlfriend/boyfriend/partner. Response options were ‘Yes’ (selected if they had disclosed to one or more of the people in the stated category), ‘No’ (selected if they had not disclosed to anyone in that category) or ‘Not Applicable’ (if they did not have the listed person in their life).

A series of binary items assessed the presence of supportive resources such as gay–straight alliances (GSA) and mentorship, as follows: three items captured whether the person currently had regular access to a GSA; whether their current (or most recently attended) school has a group or club specific to the LGBTQ+community and whether they participated in meetings or activities sponsored by an LGBTQ+ club at their school (or most recently attended school). Participants were also asked whether there is an adult 25 years or older, currently in their life, who they consider to be a mentor. Those who responded ‘yes’ were asked whether this person is part of their immediate family; if they responded yes, they received an additional question asking, ‘Other than an immediate family member (or the person who raised you), is there an adult 25 years or older who you consider to be your mentor?’ These items were recoded into a binary item reflecting the presence or absence of a non-family mentor age 25 or older.

Finally, adolescent coping strategies were captured with the Coping Strategies Inventory-Short Form,93 which includes 16 items on four subscales (problem-focused engagement; problem-focused disengagement; emotion-focused engagement and emotion-focused disengagement). An additional 20 items assessed LGBT-specific coping strategies. The first 10 questions asked about potentially recurring events, such as ‘I spent time with the LGBTQ community’ and ‘I tried to gain new knowledge about the LGBTQ community’. Response options for these statements included ‘Never’, ‘Sometimes’, ‘Often’ and ‘Regularly’. The second 10 questions asked participants for a binary response indicating if they had ever had the positive experience, such as ‘I went to an LGBTQ pride event’ and ‘I’ve been in a romantic relationship’.

With the exception of sex assigned at birth and race/ethnicity, which were only captured at baseline, all of the above measures were also collected at each follow-up time point. This includes explicitly inquiring about sexual identity and gender identity at each wave, given the fluidity of these identities during adolescence.

Additional measures

Although not the primary focus of the study, additional measures were added at later waves of data collection to probe emerging findings in the literature on SMA behavioural health. These included a more thorough investigation into the experiences of homeless and precariously housed youth; intersectionality of SGM status with cultural identity; experiences with body dysmorphia; sexual behaviour, intimate partner violence and non-consensual distribution of explicit images (‘revenge porn’); healthcare access and utilisation; specific forms of marijuana or nicotine used in the past 30 days and lifetime and past 30-day use of cocaine and methamphetamine.

Data analysis plan

We propose using cohort sequential LGCM94 to describe how minority stress and its associated behavioural health outcomes change over time among SMA. Compared with traditional longitudinal analysis methods such as comparing pretest and post-test scores or change by data collection wave, the cohort sequential LGCM approach examines individual change over age and is more appropriate for modelling developmental processes, coinciding with theoretical paradigms that are often person-centred longitudinal pathways.95 96

Several preliminary steps and decisions will be made prior to longitudinal model estimation and will vary depending on the type of analysis (eg, ordinary least squares vs logistic regression). Multicollinearity and influential cases will be assessed.97 Distributional properties of all continuous and categorical variables will be evaluated, and we will apply appropriate transformation or robust estimation procedures to correct for non-normally distributed variables (eg, specifying the weighted least squares mean- and variance-adjusted (WLSMV) estimator for binary indicators).98 99 Attrition analyses will be conducted to understand missingness. Missing data will be handled in all growth models using full information maximum likelihood estimators in Mplus assuming that data are missing completely at random or missing at random.100 Multiple imputation101 methods will also be used when appropriate. Depending on the analysis and the hypothesis being tested, demographic and some substantive variables will be included as covariates to increase the specificity of the effects; for example, geographic region, race/ethnicity or general stress. Prior to estimating full unconditional LGCMs, we will identify the best-fitting functional forms of trajectories (ie, linear, quadratic, cubic and/or piecewise) for each variable.102 Structural equation models will be evaluated using commonly accepted fit indices (eg, χ2, comparative fit index (CFI), incremental fit index (IFI), Tucker-Lewis index (TLI), and Root-Mean-Square Error of Approximation (RMSEA)) and modification indices (eg, LaGrange multipliers).103 Standard guidelines for small (0.2), medium (0.5) and large (0.8) effect sizes104 will be adopted. Confirmatory factor analysis will be used to assess measurement invariance of the SMASI and all outcome measures over time.

General approach to hypothesis testing

LGCMs will be estimated for the SMASI total score, the SMASI subscales and each of the behavioural health outcomes. The use of a cohort sequential LGCM will allow for the modelling of change in each outcome trajectory as adolescents age during the course of our study by plotting latent means across age to understand developmental trends among participants. These are useful in examining within-person change across time and between-person variability.102 Furthermore, LGCMs provide group-level statistics, including the average amount of change over time (ie, slope), the average starting point (ie, intercept) and the relationship between the two.105 106 One important advantage of LGCM is the implementation and comparison of appropriate functions to best fit the trend of the data. It is likely that several patterns of growth during the course of adolescence may emerge—that is, two or more separate slopes may be modelled within the same trajectory to demonstrate divergence in trends.105 107 These separate but related pieces incorporate the piecewise function, which allows for several linear slopes to be modelled within the same construct (eg, minority stress) and can provide information about differences in construct level (ie, intercept) and growth velocity (ie, slope) at various points throughout adolescence.

Due to the hypothesised relationships between minority stress and the behavioural health outcomes (depressive symptoms, anxiety, PTSD symptoms, suicidality and substance use), we propose to analyse these trajectories simultaneously via the use of parallel cohort sequential LGCM models.107 By modelling two growth processes (eg, minority stress and depressive symptoms) at once, we can evaluate the relationship of slopes and intercepts both within and across measures to understand their interrelated effects over time. This approach will allow us to test the following working hypotheses (WH):

WH1. There will be differences in minority stress across adolescent development

A cohort sequential LGCM108 for the total SMASI score and each of its 11 subscales will be estimated to describe individual and group-level trajectories of minority stress among all SMA during the course of the study period. This will establish the best-fitting LGCM and allow the selection and implementation of the most appropriate piecewise function. We will test piecewise models with one (ie, linear across development) to four growth trajectories to determine which function best represents the data. This will also allow us to understand how and when changes may occur throughout adolescence. The best-fitting model for each of the outcomes will be evaluated according to fit statistics (eg, Akaike’s information criterion (AIC), Bayesian information criterion (BIC), RMSEA and CFI/TLI) and by examining differences in χ2 statistics of nested models.

Once the functional form is chosen, the intercepts and slopes can be evaluated for each LGCM. Each model will capture the hypothesised differences in growth by estimating intercept means (defined as the starting point of the growth period) and slope means (change over time) for each of the growth processes and the correlations among and between them. We hypothesise that the intercepts of all growth functions will show statistically significant residual variance, indicating that adolescents vary significantly in their minority stress levels at the initial point of each growth process. We further hypothesise that the slopes of all of the growth trajectories will show significant residual variance, indicating that youth experience varied rates of increase or decrease of minority stress over time. Such findings would demonstrate that trajectories of minority stress across adolescence differ among individual youth.

WH2.1. Trajectories of minority stress and behavioural health outcomes will be associated over time (ie, considered parallel processes)

We will estimate cohort sequential LGCMs for each of the behavioural health outcomes to measure growth over time. Separate models describing depressive symptoms, anxiety, PTSD symptoms, suicidal ideation and attempt, self-injury and four substance use outcomes (alcohol, tobacco, marijuana and prescription drugs) will be estimated. A similar process to the analysis for WH1 will be applied to building these models. Subsequently, the LGCMs of the SMASI total score with each behavioural health outcome will be combined in a parallel LGCM to evaluate the relationship between the two variables over time. Regression coefficients reflecting influence of minority stress on each outcome (ie, regression of health outcome slopes and intercepts onto SMASI slopes and intercepts) will be estimated. Significant coefficients corresponding to the regression paths from the SMASI to health outcomes would provide strong evidence that minority stress affects behavioural health outcomes over time.

WH2.2. Reporting higher levels of minority stress in early adolescence will be associated with poorer behavioural health outcomes in later adolescence

Within the parallel LGCM framework, we will regress the intercepts and slopes of all behavioural health outcomes onto the intercept(s) of the SMASI to determine whether and how levels of minority stress predict later health outcomes during adolescence. Specifically, we hypothesise that: (a) the intercept of the SMASI latent variable (or in the case of a multiple trajectory piecewise model, the intercept of the first trajectory) will be positively and significantly associated with the intercept(s) of the behavioural health outcome, indicating that higher levels of minority stress in early adolescence result in worse health at each unique phase of development; (b) the first intercept of the SMASI will be significantly, positively associated with all outcome slopes, such that high levels of minority stress in early adolescence will result in a steeper increase in behavioural health problems in all growth periods and (c) the slope(s) of the SMASI will differentially predict the rate of change in later health outcome growth periods, such that a steeper increase or decrease in minority stress throughout adolescence will predict corresponding increases or decreases in behavioural health.

WH3.1. There will be significant differences in outcome trajectories by demographic subgroup

Building on the previous analyses, we will use four demographic stratification variables (race and ethnicity, gender, sexual identity and urbanicity) to explore whether there are subgroup differences (eg, male vs female; gay vs lesbian vs bisexual/pansexual vs queer) in trajectories of minority stress and behavioural health outcomes across adolescence. For example, prior literature suggests that girls are more likely to experience suicidality in adolescence than boys18 19 and bisexual youth are more likely to engage in substance use than other sexual minority groups of the same age.15 We hypothesise that we will see significant group differences in our data that confirm these findings. In a series of analyses using the multiple group function in Mplus, with up to four identity groups modelled within a single analysis, we will evaluate the structural invariance of each of our final parallel LGCMs across the subgroups comprising each of our stratification variables. The intercept and slope coefficients for each growth process will first be estimated freely across groups; the loadings will then be constrained to be equal across groups. If there is no decrement in fit (ie, CFI Δ<0.01 or non-significant χ2 difference test), we will conclude the model has structural invariance and thus there are no differences in either the minority stress or the behavioural health outcome process across demographic subgroups. If significant decrements in fit emerge (eg, when constraining across gender in the suicidality models), we will systematically free parameters to determine which intercept(s) or slope(s) differ by group and in which direction. Because no longitudinal study of this nature has been conducted, there is no evidence to support a priori hypotheses about minority stress differences by subgroup. Commensurate with the extant literature, however, we expect to find subgroup differences for each of our behavioural health outcomes. Therefore, we hypothesise that the parallel LGCMs will not demonstrate structural invariance.

WH3.2: Trajectories of minority stress will be inversely associated with protective factors over time and will differ by demographic subgroup

Using the approach described under WH2.1, we will first estimate the LGCM for protective factors (either simultaneously, ie, with a latent ‘protective factors’ variable or as separate manifest measures depending on results of preliminary analyses). Next, we will model the trajectories of the protective factor(s) and the SMASI total score simultaneously to estimate a parallel LGCM; minority stress growth parameters will be regressed on protective factors. Finally, as in WH3.1, we will examine the protective factor/minority stress parallel process model for differences by demographic subgroup using the multiple group function in Mplus and examining constrained and unconstrained models. We hypothesise that protective factors will show an overall inverse trajectory to minority stress; that is, greater intercepts and slopes of protective factors will be associated with lower intercepts and slopes of minority stress, and vice-versa. We further hypothesise that the parallel LGCMs of protective factors and minority stress will not demonstrate structural invariance—that is, there will be subgroup differences in the growth processes, owing to hypothesised sociodemographic differences in protective factors (eg, greater accessibility of social support systems in urban compared with rural environments).

Sample size calculation

For LGCM analyses, statistical power depends on sample size, df (the number of known minus free parameters), variable distributions, amount of missing data, measure reliabilities and strength of the relationships among variables. Based on the code provided by Preacher and Coffman,109 we used the hypothesis-testing framework for RMSEA as a vehicle to estimate the power for LGCM in our study. For the first, simplest models to be implemented (ie, one intercept, one linear slope), we expect 34 df; this value will decrease with each additional trajectory estimated in the piecewise models (eg, a 4-slope trajectory will have 7 df). With α=0.05, null hypothesis RMSEA of 0.05 and alternative hypothesis RMSEA of 0.08, df ranging from 7 to 56 and nominal statistical power of 0.80, a longitudinal sample size of up to 1075 may be needed to achieve adequate power for all analyses depending on the exact size of the model. For WH3.1 and WH3.2, which examine differences in minority stress and outcome trajectories by subgroups, statistical power depends on both group size and total sample size, as we cannot assume identical fit of the initial model in all groups.110 Using the trajectory forms developed for the previous hypotheses, we can support simultaneous trajectory comparisons with at least 190 participants per group; with a planned sample of n=1075, we would potentially be adequately powered to examine up to five demographic subgroups simultaneously. However, given the low likelihood of perfectly even recruitment across all demographic strata, it may be more feasible to limit models to four groups to ensure adequate group size without overly condensing across meaningful categories. With these approaches in mind, the range of statistical power for all proposed models provided by a sample of this size is between 0.80 and 0.99, depending on the closeness of the null and alternative hypotheses.

Patient and public involvement

Youth advisors were first involved in 2013 during an initial qualitative study funded by the Zumberge Foundation. That study provided the original basis for closed-ended items that eventually evolved into the SMASI measure. The current study design is a direct result of interviews conducted with SMA between 2013 and 2015, a small study of minority stress conducted between 2014 and 2016 and a set of focus groups conducted in 2016–2017 to understand stress and health patterns among the population. Youth were not directly involved in the choice of outcome measures; however, youth at several LGBTQ+drop in centres were involved in the development of study protocols (eg, advertisements used) and helped provide guidance on recruitment and retention methods. Some study participants were also directly involved in recruitment via their choice to refer other youth through RDS procedures. We are in the process of forming a youth advisory board that will assist with choosing the methods and developing plans for dissemination of study results to participants and linked communities.

Ethics and dissemination

A comprehensive informed assent document was provided to eligible youth immediately on screening into the study, and indeed, assent to participate was required in order to begin survey data collection. All study participants were willing and able to provide assent at the baseline survey. Because SMA constitute a vulnerable group whose parents may not be aware of their sexual minority status, we were granted a waiver of parental consent. At the beginning of each follow-up survey, participants who had reached 18 years of age since completing the previous survey were consented using adult protocols for informed consent. All study procedures for both baseline and longitudinal follow-up activities were reviewed and approved by the Social–Behavioral IRB at the University of Southern California. Because the study is purely observational with no researcher-controlled intervention, there is no external data safety monitoring board for the study. However, a member of the research team reviews study data immediately on downloading the new data files each business day, and any open-ended statements or data that could potentially suggest participant safety concerns are immediately brought to the attention of the study investigators, who are considered mandated reporters in the State of California. Statements are reviewed and assessed for information concerning abuse or neglect of a child; abuse or neglect of an elder or threat that the participant will harm themselves or someone else. An IRB-approved standard operating procedure is in place in the event of a positive disclosure; however, to date, no participant has disclosed any imminent safety concerns, and no other adverse events have been reported. The protocol included providing referrals to support resources for all participants, and following up with specific additional resources for those who screened at risk for suicidality (eg, both general and LGBTQ-specific crisis services).

To enhance protection for study data, we obtained an NIH Certificate of Confidentiality. The final data set will include self-reported demographic and behavioural health data, as described above, from surveys completed by the research participants. All identifying data will be destroyed at the end of the study after analysis. The final anonymous data set will be made available to other qualified members of the scientific community on request per policies of the NIH and the University of Southern California IRB. We are committed to participating in the sharing and building of research knowledge, and will adhere to the NIH Policy on Sharing of Unique Research Resources including the Guidelines for Recipients of NIH Grants and Contracts. Requests for research resources that are generated as part of this project (eg, qualitative outcomes, the stress measurement instrument) will be distributed in a timely manner.

Finally, the purpose of the current research is to examine pathways that may predict differing behavioural health outcomes in SMAs. To that end, the overarching purpose is to share our developed resources with the community. As the research team completes analyses and arrives at empirical results, we have contracted with a creative graphics firm to develop infographics that cleanly summarise research findings with terminology suitable for the lay public. In addition to presenting our work in peer-reviewed manuscripts and scientific meetings, we are pursuing opportunities to share our findings with the broader community, including hosting the infographics and other study materials and derivatives on the website of the University of Southern California Center for LGBTQ+ Health Equity.111

Supplementary Material

Reviewer comments
Author's manuscript

Footnotes

Contributors: SMS drafted the initial manuscript and critically revised all sections. MRM drafted the study procedures and measures, developed the analytic plan and critically revised the manuscript draft. HR developed the study procedures and critically revised the manuscript draft. JTG developed the introduction, drafted the patient and public involvement section and critically revised the manuscript draft. All authors reviewed and approved the final manuscript.

Funding: This work was supported by the National Institute on Minority Health and Health Disparities grant number R01MD012252-01. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the National Institutes of Health.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review: Not commissioned; externally peer reviewed.

Ethics statements

Patient consent for publication

Not required.

References

  • 1.Haas AP, Eliason M, Mays VM, et al. Suicide and suicide risk in lesbian, gay, bisexual, and transgender populations: review and recommendations. J Homosex 2011;58:10–51. 10.1080/00918369.2011.534038 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Hendricks ML, Testa RJ. A conceptual framework for clinical work with transgender and gender nonconforming clients: an adaptation of the minority stress model. Prof Psychol 2012;43:460–7. 10.1037/a0029597 [DOI] [Google Scholar]
  • 3.Anhalt K, Morris TL. Developmental and adjustment issues of gay, lesbian, and bisexual adolescents: a review of the empirical literature. Clin Child Fam Psychol Rev 1998;1:215–30. 10.1023/a:1022660101392 [DOI] [PubMed] [Google Scholar]
  • 4.Stettler NM, Katz LF, stress M. Minority stress, emotion regulation, and the parenting of sexual-minority youth. J GLBT Fam Stud 2017;13:380–400. 10.1080/1550428X.2016.1268551 [DOI] [Google Scholar]
  • 5.Lucassen MFG, Stasiak K, Samra R, et al. Sexual minority youth and depressive symptoms or depressive disorder: a systematic review and meta-analysis of population-based studies. Aust N Z J Psychiatry 2017;51:774–87. 10.1177/0004867417713664 [DOI] [PubMed] [Google Scholar]
  • 6.Jones A, Robinson E, Oginni O, et al. Anxiety disorders, gender nonconformity, bullying and self-esteem in sexual minority adolescents: prospective birth cohort study. J Child Psychol Psychiatry 2017;58:1201–9. 10.1111/jcpp.12757 [DOI] [PubMed] [Google Scholar]
  • 7.Taliaferro LA, Muehlenkamp JJ. Nonsuicidal self-injury and suicidality among sexual minority youth: risk factors and protective connectedness factors. Acad Pediatr 2017;17:715–22. 10.1016/j.acap.2016.11.002 [DOI] [PubMed] [Google Scholar]
  • 8.Marshal MP, Friedman MS, Stall R, et al. Sexual orientation and adolescent substance use: a meta-analysis and methodological review. Addiction 2008;103:546–56. 10.1111/j.1360-0443.2008.02149.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Moon MW, Fornili K, O'Briant AL. Risk comparison among youth who report sex with same-sex versus both-sex partners. Youth Soc 2007;38:267–84. 10.1177/0044118X06287689 [DOI] [Google Scholar]
  • 10.Goldbach JT, Mereish EH, Burgess C. Sexual orientation disparities in the use of emerging drugs. Subst Use Misuse 2017;52:265–71. 10.1080/10826084.2016.1223691 [DOI] [PubMed] [Google Scholar]
  • 11.Watson RJ, Goodenow C, Porta C, et al. Substance use among sexual minorities: has it actually gotten better? Subst Use Misuse 2018;53:1221–8. 10.1080/10826084.2017.1400563 [DOI] [PubMed] [Google Scholar]
  • 12.Coker TR, Austin SB, Schuster MA. The health and health care of lesbian, gay, and bisexual adolescents. Annu Rev Public Health 2010;31:457–77. 10.1146/annurev.publhealth.012809.103636 [DOI] [PubMed] [Google Scholar]
  • 13.Saewyc EM. Contested conclusions: claims that can (and cannot) be made from the current research on gay, lesbian, and bisexual teen suicide attempts. J LGBT Health Res 2007;3:79–87. 10.1300/j463v03n01_09 [DOI] [PubMed] [Google Scholar]
  • 14.Smith BC, Armelie AP, Boarts JM, et al. Ptsd, depression, and substance use in relation to suicidality risk among traumatized minority Lesbian, gay, and bisexual youth. Arch Suicide Res 2016;20:80–93. 10.1080/13811118.2015.1004484 [DOI] [PubMed] [Google Scholar]
  • 15.Marshal MP, Friedman MS, Stall R, et al. Individual trajectories of substance use in lesbian, gay and bisexual youth and heterosexual youth. Addiction 2009;104:974–81. 10.1111/j.1360-0443.2009.02531.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Marshal MP, Burton CM, Chisolm DJ, et al. Cross-Sectional evidence for a stress-negative affect pathway to substance use among sexual minority girls. Clin Transl Sci 2013;6:321–2. 10.1111/cts.12052 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Gilbey D, Mahfouda S, Ohan J, et al. Trajectories of mental health difficulties in young people who are attracted to the same gender: a systematic review. Adolesc Res Rev 2020;5:281–93. 10.1007/s40894-019-00128-8 [DOI] [Google Scholar]
  • 18.Bridge JA, Goldstein TR, Brent DA. Adolescent suicide and suicidal behavior. J Child Psychol Psychiatry 2006;47:372–94. 10.1111/j.1469-7610.2006.01615.x [DOI] [PubMed] [Google Scholar]
  • 19.Friedman MS, Marshal MP, Guadamuz TE, et al. A meta-analysis of disparities in childhood sexual abuse, parental physical abuse, and peer victimization among sexual minority and sexual nonminority individuals. Am J Public Health 2011;101:1481–94. 10.2105/AJPH.2009.190009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Cohn TJ, Leake VS. Affective distress among adolescents who endorse same-sex sexual attraction: urban versus rural differences and the role of protective factors. J Gay Lesbian Ment Health 2012;16:291–305. 10.1080/19359705.2012.690931 [DOI] [Google Scholar]
  • 21.Goldbach JT, Gibbs JJ. Strategies employed by sexual minority adolescents to cope with minority stress. Psychol Sex Orientat Gend Divers 2015;2:297. 10.1037/sgd0000124 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Cochran BN, Stewart AJ, Ginzler JA, et al. Challenges faced by homeless sexual minorities: comparison of gay, lesbian, bisexual, and transgender homeless adolescents with their heterosexual counterparts. Am J Public Health 2002;92:773–7. 10.2105/ajph.92.5.773 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Moradi B, Mohr JJ, Worthington RL, et al. Counseling psychology research on sexual (orientation) minority issues: conceptual and methodological challenges and opportunities. J Couns Psychol 2009;56:5–22. 10.1037/a0014572 [DOI] [Google Scholar]
  • 24.Szymanski DM, Kashubeck-West S, Meyer J. Internalized heterosexism: a historical and theoretical overview. The Counseling Psychologist 2008;36:510–24. 10.1177/0011000007309488 [DOI] [Google Scholar]
  • 25.Goldbach JT, Tanner-Smith EE, Bagwell M, et al. Minority stress and substance use in sexual minority adolescents: a meta-analysis. Prev Sci 2014;15:350–63. 10.1007/s11121-013-0393-7 [DOI] [PubMed] [Google Scholar]
  • 26.Mereish EH, Sheskier M, Hawthorne DJ, et al. Sexual orientation disparities in mental health and substance use among black American young people in the USA: effects of cyber and bias-based victimisation. Cult Health Sex 2019;21:985–98. 10.1080/13691058.2018.1532113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Anhalt K, Toomey RB, Shramko M. Latinx sexual minority youth adjustment in the context of discrimination and internalized homonegativity: the Moderating role of cultural orientation processes. Journal of Latinx Psychology 202;8:41–57. 10.1037/lat0000134 [DOI] [Google Scholar]
  • 28.Gates GJ, Newport F. Special report: 3.4% of US adults identify as LGBT. Washington, DC: Gallup, 2012. [Google Scholar]
  • 29.Hatzenbuehler ML, Nolen-Hoeksema S, Dovidio J. How does stigma "get under the skin"?: the mediating role of emotion regulation. Psychol Sci 2009;20:1282–9. 10.1111/j.1467-9280.2009.02441.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Meyer IH, Prejudice MIH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull 2003;129:674. 10.1037/0033-2909.129.5.674 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Rosario M, Schrimshaw EW, Hunter J, et al. Gay-related stress and emotional distress among gay, lesbian, and bisexual youths: a longitudinal examination. J Consult Clin Psychol 2002;70:967. 10.1037//0022-006x.70.4.967 [DOI] [PubMed] [Google Scholar]
  • 32.CCf D, Ca P. Lesbian gay bisexual and transgender health: youth, 2011. Available: http://www.cdc.gov/lgbthealth/youth.htm
  • 33.NAo M. Annual Report 2015, 2015. Available: https://nam.edu/wp-content/uploads/2016/06/NAM-Annual-Report-2015.pdf
  • 34.2020 HP. healthy people 2020, 2011. Available: https://www.healthypeople.gov/
  • 35.Kelleher C. Minority stress and health: implications for lesbian, gay, bisexual, transgender, and Questioning (LGBTQ) young people. Couns Psychol Q 2009;22:373–9. 10.1080/09515070903334995 [DOI] [Google Scholar]
  • 36.Russell JA. Core affect and the psychological construction of emotion. Psychol Rev 2003;110:145. 10.1037/0033-295x.110.1.145 [DOI] [PubMed] [Google Scholar]
  • 37.Savin-Williams RC. Mom, dad. I'm gay. How families negotiate coming out. Washington, DC, US: American Psychological Association, 2001. [Google Scholar]
  • 38.Myers W, Turanovic JJ, Lloyd KM, et al. The victimization of LGBTQ students at school: a meta-analysis. J Sch Violence 2020;19:421–32. 10.1080/15388220.2020.1725530 [DOI] [Google Scholar]
  • 39.Remafedi G, French S, Story M, et al. The relationship between suicide risk and sexual orientation: results of a population-based study. Am J Public Health 1998;88:57–60. 10.2105/ajph.88.1.57 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Rice E, Barman-Adhikari A. Internet and social media use as a resource among homeless youth. J Comput Mediat Commun 2014;19:232–47. 10.1111/jcc4.12038 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Russell ST, Ryan C, Toomey RB, et al. Lesbian, gay, bisexual, and transgender adolescent school victimization: implications for young adult health and adjustment. J Sch Health 2011;81:223–30. 10.1111/j.1746-1561.2011.00583.x [DOI] [PubMed] [Google Scholar]
  • 42.Toomey RB, Ryan C, Diaz RM, et al. Gender-nonconforming lesbian, gay, bisexual, and transgender youth: school victimization and young adult psychosocial adjustment. Dev Psychol 2010;46:1580. 10.1037/a0020705 [DOI] [PubMed] [Google Scholar]
  • 43.Pollitt AM, Mallory AB, Fish JN. Homophobic bullying and sexual minority youth alcohol use: do sex and race/ethnicity matter? LGBT Health 2018;5:412–20. 10.1089/lgbt.2018.0031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.D'Augelli AR, Grossman AH. Disclosure of sexual orientation, victimization, and mental health among lesbian, gay, and bisexual older adults. J Interpers Violence 2001;16:1008–27. 10.1177/088626001016010003 [DOI] [Google Scholar]
  • 45.Kosciw JG, Greytak EA, Bartkiewicz MJ. The 2011 National School Climate Survey: The Experiences of Lesbian, Gay, Bisexual and Transgender Youth in Our Nation’s Schools. ERIC, 2012. [Google Scholar]
  • 46.Bouris A, Everett BG, Heath RD, et al. Effects of victimization and violence on suicidal ideation and behaviors among sexual minority and heterosexual adolescents. LGBT Health 2016;3:153–61. 10.1089/lgbt.2015.0037 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Clatts MC, Goldsamt L, Yi H, et al. Homelessness and drug abuse among young men who have sex with men in New York City: a preliminary epidemiological trajectory. J Adolesc 2005;28:201–14. 10.1016/j.adolescence.2005.02.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Marshal MP, Goldbach JT, McCauley HL. Gay-related stress and suicide risk: Articulating three mediated pathways that increase risk for suicidality among sexual minority youth. In: Advancing the science of suicidal behavior: understanding and intervention. Nova Science Publishers, Inc, 2014: 253–68. [Google Scholar]
  • 49.Morrison TG, Bishop CJ, Morrison MA, et al. A psychometric review of measures assessing discrimination against sexual minorities. J Homosex 2016;63:1086–126. 10.1080/00918369.2015.1117903 [DOI] [PubMed] [Google Scholar]
  • 50.Birkett M, Newcomb ME, Mustanski B. Does it get better? A longitudinal analysis of psychological distress and victimization in lesbian, gay, bisexual, transgender, and Questioning youth. Journal of Adolescent Health 2015;56:280–5. 10.1016/j.jadohealth.2014.10.275 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Mustanski B, Liu RT. A longitudinal study of predictors of suicide attempts among lesbian, gay, bisexual, and transgender youth. Arch Sex Behav 2013;42:437–48. 10.1007/s10508-012-0013-9 [DOI] [PubMed] [Google Scholar]
  • 52.Burton CM, Marshal MP, Chisolm DJ, et al. Sexual minority-related victimization as a mediator of mental health disparities in sexual minority youth: a longitudinal analysis. J Youth Adolesc 2013;42:394–402. 10.1007/s10964-012-9901-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Kaplan D. Structural equation modeling: foundations and extensions. SAGE publications, 2008. [Google Scholar]
  • 54.Díaz RM, Ayala G, Bein E, et al. The impact of homophobia, poverty, and racism on the mental health of gay and bisexual Latino men: findings from 3 us cities. Am J Public Health 2001;91:927. 10.2105/ajph.91.6.927 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Mereish EH, Bradford JB. Intersecting identities and substance use problems: sexual orientation, gender, race, and lifetime substance use problems. J Stud Alcohol Drugs 2014;75:179–88. 10.15288/jsad.2014.75.179 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Szymanski DM, Sung MR. Minority stress and psychological distress among Asian American sexual minority persons 1Ψ7. Couns Psychol 2010;38:848–72. 10.1177/0011000010366167 [DOI] [Google Scholar]
  • 57.Toomey RB, Huynh VW, Jones SK, et al. Sexual minority youth of color: a content analysis and critical review of the literature. J Gay Lesbian Ment Health 2017;21:3–31. 10.1080/19359705.2016.1217499 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Layland EK, Exten C, Mallory AB, et al. Suicide attempt rates and associations with discrimination are greatest in early adulthood for sexual minority adults across diverse racial and ethnic groups. LGBT Health 2020;7:439–47. 10.1089/lgbt.2020.0142 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Mallory AB, Russell ST. Intersections of racial discrimination and LGB victimization for mental health: a prospective study of sexual minority youth of color. J Youth Adolesc 2021;50:1353–68. 10.1007/s10964-021-01443-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Bostwick WB, Meyer I, Aranda F, et al. Mental health and suicidality among racially/ethnically diverse sexual minority youths. Am J Public Health 2014;104:1129–36. 10.2105/AJPH.2013.301749 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Thoma BC, Huebner DM. Health consequences of racist and antigay discrimination for multiple minority adolescents. Cultur Divers Ethnic Minor Psychol 2013;19:404–13. 10.1037/a0031739 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Grossman AH, Haney AP, Edwards P, et al. Lesbian, gay, bisexual and transgender youth talk about experiencing and coping with school violence: a qualitative study. J LGBT Youth 2009;6:24–46. 10.1080/19361650802379748 [DOI] [Google Scholar]
  • 63.Scourfield J, Roen K, McDermott L. Lesbian, gay, bisexual and transgender young people's experiences of distress: resilience, ambivalence and self-destructive behaviour. Health Soc Care Community 2008;16:329–36. 10.1111/j.1365-2524.2008.00769.x [DOI] [PubMed] [Google Scholar]
  • 64.Hatzenbuehler ML, Keyes KM. Inclusive anti-bullying policies and reduced risk of suicide attempts in lesbian and gay youth. J Adolesc Health 2013;53:S21–6. 10.1016/j.jadohealth.2012.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Goldbach JT, Schrager SM, Mamey MR. Criterion and divergent validity of the sexual minority adolescent stress inventory. Front Psychol 2017;8:8. 10.3389/fpsyg.2017.02057 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Schrager SM, Goldbach JT, Mamey MR. Development of the sexual minority adolescent stress inventory. Front Psychol 2018;9:319. 10.3389/fpsyg.2018.00319 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Schrager SM, Goldbach JT. Minority stress measure development: theoretical concerns and suggested resolutions. Berlin, Germany: Logos Verlag, 2017. [Google Scholar]
  • 68.Elze DE. Research with sexual minority youths: where do we go from here? Journal of Gay & Lesbian Social Services 2005;18:73–99. 10.1300/J041v18n02_05 [DOI] [Google Scholar]
  • 69.Jones JM. Special report: LGBT Identification Rises to 5.6% in Latest US Estimate. Washington, DC: Gallup, 2021. [Google Scholar]
  • 70.Raifman J, Charlton BM, Arrington-Sanders R, et al. Sexual orientation and suicide attempt disparities among US adolescents: 2009-2017. Pediatrics 2020;145. doi: 10.1542/peds.2019-1658. [Epub ahead of print: 10 02 2020]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Russell ST, Clarke TJ, Clary J. Are teens "post-gay"? Contemporary adolescents' sexual identity labels. J Youth Adolesc 2009;38:884–90. 10.1007/s10964-008-9388-2 [DOI] [PubMed] [Google Scholar]
  • 72.Gates GJ. How many people are lesbian, gay, bisexual and transgender? 2011. Available: https://williamsinstitute.law.ucla.edu/publications/how-many-people-lgbt/
  • 73.Saewyc EM, Bauer GR, Skay CL, et al. Measuring sexual orientation in adolescent health surveys: evaluation of eight school-based surveys. J Adolesc Health 2004;35:345.e1–345.e15. 10.1016/j.jadohealth.2004.06.002 [DOI] [PubMed] [Google Scholar]
  • 74.World Health Organization . Young people’s health-a challenge for society: Report of a WHO Study Group on Young People and "Health for All by the Year 2000"[meeting held in Geneva from 4 to 8 June 1984], 1986. Available: https://apps.who.int/iris/handle/10665/41720 [PubMed]
  • 75.Mustanski B, Kuper L, Greene GJ. Development of sexual orientation and identity. In: Apa Handbook of sexuality and psychology, 2014. [Google Scholar]
  • 76.Chew D, Tollit MA, Poulakis Z, et al. Youths with a non-binary gender identity: a review of their sociodemographic and clinical profile. Lancet Child Adolesc Health 2020;4:322–30. 10.1016/S2352-4642(19)30403-1 [DOI] [PubMed] [Google Scholar]
  • 77.Harris KM CTH, Whitsel E, Hussey J. The National longitudinal study of adolescent to adult health: research design, 2009. Available: http://www.cpc.unc.edu/projects/addhealth/design
  • 78.Cromartie J. Rural-Urban Commuting area codes, 2020. Available: https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes.aspx
  • 79.Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl 1997;44:174–99. 10.2307/3096941 [DOI] [Google Scholar]
  • 80.Aust F, Diedenhofen B, Ullrich S, et al. Seriousness checks are useful to improve data validity in online research. Behav Res Methods 2013;45:527–35. 10.3758/s13428-012-0265-2 [DOI] [PubMed] [Google Scholar]
  • 81.Robinson-Cimpian JP. Inaccurate estimation of disparities due to mischievous responders: several suggestions to assess conclusions. Educational Researcher 2014;43:171–85. [Google Scholar]
  • 82.Bauermeister JA, Pingel E, Zimmerman M. Data quality in HIV/AIDS web-based surveys: handling invalid and suspicious data. Field Methods 2012;24:272–91. 10.1177/1525822X12443097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Teitcher JEF, Bockting WO, Bauermeister JA, et al. Detecting, Preventing, and Responding to “Fraudsters” in Internet Research: Ethics and Tradeoffs. J. Law. Med. Ethics 2015;43:116–33. 10.1111/jlme.12200 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Grey JA, Konstan J, Iantaffi A, et al. An updated protocol to detect invalid entries in an online survey of men who have sex with men (MSM): how do valid and invalid submissions compare? AIDS Behav 2015;19:1928–37. 10.1007/s10461-015-1033-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Melchior LA, Huba GJ, Brown VB, et al. A short depression index for women. Educ Psychol Meas 1993;53:1117–25. 10.1177/0013164493053004024 [DOI] [Google Scholar]
  • 86.Spitzer RL, Kroenke K, Williams JBW, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166:1092–7. 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
  • 87.Lang AJ, Wilkins K, Roy-Byrne PP, et al. Abbreviated PTSD checklist (PCL) as a guide to clinical response. Gen Hosp Psychiatry 2012;34:332–8. 10.1016/j.genhosppsych.2012.02.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.(CDC) CfDCaP . Youth risk behavior survey; 2010, 2010. Available: http://www.cdc.gov/yrbs
  • 89.Gibbs JJ, Goldbach JT. Religious identity dissonance: understanding how sexual minority adolescents manage antihomosexual religious messages. J Homosex 2021;68:1–25. 10.1080/00918369.2020.1733354 [DOI] [PubMed] [Google Scholar]
  • 90.Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983;24:385–96. 10.2307/2136404 [DOI] [PubMed] [Google Scholar]
  • 91.Cohen S. Perceived stress in a probability sample of the United States. In: The social psychology of health. Sage Publications, 1988: 31–67. [Google Scholar]
  • 92.Zimet GD, Dahlem NW, Zimet SG, et al. The multidimensional scale of perceived social support. J Pers Assess 1988;52:30–41. 10.1207/s15327752jpa5201_2 [DOI] [Google Scholar]
  • 93.Addison CC, Campbell-Jenkins BW, Sarpong DF, et al. Psychometric evaluation of a coping strategies inventory short-form (CSI-SF) in the Jackson heart study cohort. Int J Environ Res Public Health 2007;4:289–95. 10.3390/ijerph200704040004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94.Bollen KA, Curran PJ. Latent curve models: a structural equation perspective. John Wiley & Sons, 2006. [Google Scholar]
  • 95.Curran PJ, Hussong AM. The use of latent trajectory models in psychopathology research. J Abnorm Psychol 2003;112:526. 10.1037/0021-843X.112.4.526 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Curran P, Willoughby M. Reconciling theoretical and statistical models of developmental processes. Development and Psychopathology 2003;15:581–612. [DOI] [PubMed] [Google Scholar]
  • 97.Fox J. Regression diagnostics: an introduction. SAGE publications, incorporated, 2019. [Google Scholar]
  • 98.Mamey MR, Burns GL, Barbosa-Leiker C, et al. Parallel growth modeling to better understand smoking with stimulant use outcomes during an integrated treatment trial. Exp Clin Psychopharmacol 2022;30:51–8. 10.1037/pha0000439 [DOI] [PubMed] [Google Scholar]
  • 99.Muthén B. Second-Generation structural equation modeling with a combination of categorical and continuous latent variables: new opportunities for latent class–latent growth modeling. In: New methods for the analysis of change, 2001: 291–322. [Google Scholar]
  • 100.Little RJ, Rubin DB. Statistical analysis with missing data. John Wiley & Sons, 2019. [Google Scholar]
  • 101.Schafer JL. Analysis of incomplete multivariate data: Chapman and Hall/CRC, 1997. [Google Scholar]
  • 102.Borowsky IW, Ireland M, Resnick MD. Adolescent suicide attempts: risks and protectors. Pediatrics 2001;107:485–93. 10.1542/peds.107.3.485 [DOI] [PubMed] [Google Scholar]
  • 103.L-T H, Bentler PM. Evaluating model fit, 1995. [Google Scholar]
  • 104.Cohen J. Statistical power analysis for the behavioral sciences. 2nd edn. NJ, USA: Á/L. Erbaum Press, Hillsdale, 1988. [Google Scholar]
  • 105.Hancock GR, Lawrence FR. Using latent growth models to evaluate longitudinal change. In: Structural equation modeling: a second course. 2, 2006: 309–41. [Google Scholar]
  • 106.Preacher KJ, Hayes AF. Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behav Res Methods 2008;40:879–91. 10.3758/BRM.40.3.879 [DOI] [PubMed] [Google Scholar]
  • 107.Mamey MR, Barbosa-Leiker C, McPherson S, et al. An application of analyzing the trajectories of two disorders: a parallel piecewise growth model of substance use and attention-deficit/hyperactivity disorder. Exp Clin Psychopharmacol 2015;23:422–7. 10.1037/pha0000047 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 108.Meredith W, Tisak J. Latent curve analysis. Psychometrika 1990;55:107–22. 10.1007/BF02294746 [DOI] [Google Scholar]
  • 109.Preacher KJ, Coffman DL. Computing power and minimum sample size for RMSEA, 2006. [Google Scholar]
  • 110.Preacher KJ, Cai L, MacCallum RC. Alternatives to traditional model comparison strategies for covariance structure models. In: Little TD, Bovaird JA, Card NA, eds. Modeling contextual effects in longitudinal studies. Mahwah, NJ: Lawrence Erlbaum Associates, 2007: 33–62. [Google Scholar]
  • 111.California UoS . Center for LGBTQ+ health equity. Available: https://dworakpeck.usc.edu/research/centers-affiliations/center-for-lgbtq-health-equity-clhe

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES