Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jan 1.
Published in final edited form as: J Consult Clin Psychol. 2016 Jun 16;85(1):72–76. doi: 10.1037/ccp0000123

Omitted Data in Randomized Controlled Trials for Anxiety and Depression: A Systematic Review of the Inclusion of Sexual Orientation and Gender Identity

Nicholas C Heck 1, Lucas A Mirabito 2, Kelly LeMaire 3, Nicholas A Livingston 4, Annesa Flentje 5
PMCID: PMC5161712  NIHMSID: NIHMS782720  PMID: 27845517

Abstract

Objective

The current study examined the frequency with which randomized controlled trials (RCTs) of behavioral and psychological interventions for anxiety and depression include data pertaining to participant sexual orientation and non-binary gender identities.

Method

Using systematic review methodology, the databases PubMed and PsycINFO were searched to identify RCTs published in 2004, 2009, and 2014. Random selections of 400 articles per database per year (2400 articles in total) were considered for inclusion in the review. Articles meeting inclusion criteria were read and coded by the research team to identify whether the trial reported data pertaining to participant sexual orientation and non-binary gender identities. Additional trial characteristics were also identified and indexed in our database (e.g., sample size, funding source etc.).

Results

Of the 232 articles meeting inclusion criteria, only one reported participants’ sexual orientation and zero articles included non-binary gender identities. A total of 52,769 participants were represented in the trials, 93 of which were conducted in the U.S. and 43 acknowledged the National Institutes of Health as a source of funding.

Conclusions

Despite known mental health disparities on the basis of sexual orientation and non-binary gender identification, researchers evaluating interventions for anxiety and depression are not reporting on these important demographic characteristics. Reporting practices must change in order to ensure that our interventions generalize to lesbian, gay, bisexual, and transgender persons.

Keywords: systematic review, LGBT, anxiety, depression, randomized controlled trial

Introduction

Population-based and meta-analytic research indicate that lesbian, gay, and bisexual people evidence elevated rates of mood and anxiety disorders relative to heterosexual persons (Bostwick, Boyd, Hughes, & McCabe, 2010; King et al., 2008; Marshal et al., 2011). Although rigorous epidemiological research comparing mental health outcomes for transgender and cisgender (e.g., people whose sex assigned at birth is congruent with their internal sense of gender identification) people does not exist, there is cross sectional research suggesting that transgender individuals are also likely at risk for experiencing depression and anxiety (Hendricks & Testa, 2012; Herbst et al., 2007). Fortunately, a number of randomized controlled trials (RCTs) indicate that there are multiple, effective psychological and behavioral interventions that can be used to treat anxiety and depressive disorders (Stewart & Chambless, 2009; Weston & Morrison, 2001). However, the effectiveness of these interventions for lesbian, gay, bisexual and transgender (LGBT) people experiencing anxiety and depression has yet to be determined.

There are reasons to believe that sexual orientation and non-binary gender identification (e.g., transgender, gender queer, etc.) may moderate treatment effects. First, there are unique stress processes that give rise to health disparities among LGBT people (Meyer, 2003) and existing interventions may not adequately address these stressors. Second, there are likely to be differences in clinical presentations between LGBT and heterosexual/cisgender people at intake (Cochran & Cauce, 2006; Flentje, Heck, & Sorensen, 2014; 2015) that could impact the likelihood of treatment success. Third, just as the acceptability of psychotherapy may differ according to race (Cooper et al., 2003), LGBT people, especially those that are older and/or transgender, may be guarded about revealing their sexual minority statuses due to the profession’s history of pathologizing non-heterosexual identities and continued pathologization of transgender identities (Heck, Flentje, & Cochran, 2013). Thus to test whether sexual orientation and non-binary gender identification moderate treatment effects, psychotherapy researchers need to be routinely asking about and reporting on these demographic characteristics. Unfortunately, recent findings in the substance abuse literature suggest that these demographic characteristics are reported in less than 5% of peer-reviewed publications (Flentje, Bacca, & Cochran, 2015). Whether this is true in the psychotherapy literature remains unanswered.

Objective and Hypothesis

The present study sought to determine the degree to which studies reporting the results of RCTs for psychological and behavioral interventions targeting anxiety and depression report demographic data describing participant sexual orientation and non-binary gender identification (e.g., transgender, gender queer, etc.). With the growing awareness about the importance of research to address LGBT health (Institute of Medicine [IOM], 2011; United States Department of Health and Human Services, 2010), unique stress processes faced by LGBT people (Meyer, 2003) and the importance of providing affirmative LGBT psychotherapy services (Bieschke, Perez, & DeBord, 2009), we anticipated that there would be an increase over time in the reporting of LGBT identities as part of participant demographic characteristics in studies reporting the results of RCTs.

Methods

To identify articles for the current review, we searched the databases PsycINFO and PubMed using the search criteria “depression” OR “anxiety” AND “randomized control trial” OR “RCT”. In PsycINFO we narrowed the search by age group (i.e., “Adulthood (18 yrs & older)”), population (i.e., “NOT animal”), and methodology (i.e., NOT clinical case study, qualitative study, retrospective study, interview, experimental replication, literature review, meta analysis, systematic review, nonclinical case study, scientific simulation, twin study, brain imaging). Options to narrow the search in PubMed were more restricted but we narrowed to include only journal articles, with human participants. PubMed expands searches automatically to include Medical Subject Headings (MeSH), which expanded the search to include terms like “depressive disorder” and “randomised controlled trial”.

A separate search was conducted for each database for the years 2004, 2009, and 2014 to limit the number of articles returned, and to see if the reporting of sexual orientation and non-binary gender identities has increased over time. The search results for each database were then exported to a portable document file or text file with the results being individually numbered, in separate files for each of the databases and search years. We randomly selected 400 articles per database per year (2400 articles in total) and coded each article to determine whether it met our inclusion criteria. We screened our data to identify and eliminate potential article duplications and randomly selected new articles by alternating additional draws from our pools of PsycINFO and PubMed search returns.

To be included the article had to report the results of an RCT for a psychological or behavioral intervention where anxiety and/or depression were outcome variables. Studies also had to be in English and contain adult samples (ages 18 and older). For the RCT criterion, a study had to indicate that participants were randomly assigned to intervention or control groups. We did not differentiate between studies that used waitlist or active control conditions, thus comparative efficacy studies were included. Studies that used cross-over designs were also included, as were multi-site trials. Studies that evaluated systemic interventions (i.e., interventions with multiple components such as home check-ins, outpatient office visits, and medication management; enhanced vs. usual care, etc.) were included as long as the systemic intervention contained a psychological or behavioral intervention. Studies that only included baseline characteristics and protocol descriptions were excluded. For the psychological or behavioral intervention criterion, we included studies that necessitate active engagement on the part of the participant to produce change in behavioral or psychological processes. Some examples of included interventions were: cognitive therapy, interpersonal psychotherapy, exercise regimens, and psychoeducational programs. Some examples of excluded interventions were: acupuncture, medication trials without a psychological or behavioral comparison, reflexology, bright light therapy, and therapeutic touch.

Advanced undergraduate research assistants conducted the initial review of articles to identify articles meeting our inclusion criteria. They were provided with detailed instructions about how to review each article, along with examples of research design elements that would meet or not meet a given criterion. They then sorted the articles into categories of ‘does not meet’ ‘uncertain’ and ‘definitely meets’. This process was supervised by two doctoral-level psychologists and two graduate-level researchers who were responsible for answering questions, and making decisions about the ‘uncertain’ articles. After the initial coding process, a doctoral-level psychologist and a graduate-level researcher reviewed every article included to ensure that they met the inclusion criteria. Additionally, a doctoral-level psychologist randomly checked 10 excluded articles per database per year to verify that articles were being correctly excluded.

Studies that met the inclusion criteria were then reviewed and coded by the team of undergraduate research assistants. Included articles were coded by one research assistant; the coding was independently validated by a second research assistant. Again, a doctoral-level psychologist then checked the coding for a subset of randomly selected articles (20 per database per year). The primary outcomes of interest were the study sample size, whether or not participant sexual orientation and non-binary gender identities were reported, and whether or not the study was HIV-related. Secondary outcomes included location of the trial (e.g., United States vs. International) and funding source, if acknowledged. We followed the same procedures as Flentje and colleagues (2015) for determining whether the study reported sexual orientation and non-binary gender identities. Consistent with Flentje et al. (2015), articles were considered to report sexual orientation if the study reported same-sex sexual behavior, identity, or same-sex partner status. Articles that reported opposite-sex relationship or marriage status without acknowledging the potential for same-sex relationships were not coded as reporting sexual orientation. Undergraduate research assistants were instructed to carefully review methods and results sections as well as any tables reporting participant characteristics. Articles were coded as having non-binary gender response categories if they included gender categories beyond male and female, or men and women.

Results

Of the 2400 studies reviewed, 232 met inclusion criteria for our study (see Table 1 for the number of studies that met criteria for each year and database). Notably, only one study, published in 2009, reported participant sexual orientation; zero studies reported non-binary gender identities. A total of 52,769 participants were represented in the RCTs that met the inclusion criteria (M N per RCT = 230.44; SD = 381.17; n range 11–3205). Ninety-three RCTs were conducted in the U.S. and 43 acknowledged the National Institutes of Health (NIH) as a source of funding.

Table 1.

Returns from Searches, Articles Included, and Articles Reporting LGBT Participants

Year Database Articles Returned Articles Included n Reporting Sexual Orientation n Reporting Non-binary Gender
2004 PubMed 1339 61 0 0
PsycINFO 3705 17 0 0
2009 PubMed 2475 61 1 0
PsycINFO 6596 21 0 0
2014 PubMed 1078 62 0 0
PsycINFO 6245 10 0 0
Open in a new tab

The one study that measured sexual orientation compared the effects of a cognitive-behavioral, self-help program for depression among persons living with HIV to a writing intervention (Kraaij et al., 2009). Within this study, the authors reported that of the 44 participants who completed the trial “most respondents reported to be homosexual” (Kraaij et al., 2009; pp. 201) without reporting the exact number of respondents or providing additional information about how sexual orientation was measured. Non-binary gender identities were not mentioned in any of the studies.

Discussion

In 1994 the Office of Management and Budget, in conjunction with the Executive Office of President Clinton, proposed a review and possible revision of race and ethnicity measurement standards for the federal government. This action eventually led to revised standards for how race and ethnicity are reported, which were published in 1997 and adopted by the NIH in 2001. Thus, the collection of data on race and ethnicity was henceforth mandated for federally-funded clinical research. The results of this systematic review, which found that LGBT identities are not routinely reported in RCTs evaluating psychological and behavioral interventions for anxiety and depression, suggest that similar action may be warranted for the collection and reporting of sexual orientation and non-binary gender identities. The IOM (2011) has taken a stance indicating that sexual orientation and gender identity are important variables to measure in order to better understand the health of LGBT people, and we encourage psychotherapy researchers to adopt these recommendations.

The results of the present study paint a grim picture for how frequently researchers are reporting participant demographic characteristics of sexual orientation and non-binary gender identities within RCTs of psychological and behavioral interventions. Recent research suggests that roughly 3.5% of Americans identify as lesbian, gay, or bisexual and 0.3% identify as transgender (Gates, 2011). Given the total number of participants represented in this randomly selected pool of RCTs, upwards of 1,838 LGB and 158 transgender persons participated in the trials but were not identified on the basis of their sexual orientation or gender identities. Notably, this is a conservative estimate that does not account for the higher rates of mental health service utilization reported among LGBT people (Cochran, Sullivan & Mays, 2003).

Consistently querying and reporting demographic information for sexual orientation and non-binary gender identities within the context of RCTs has the potential to improve health outcomes for LGBT people. Notably, if such data were available, the pooling of data across multiple RCTs for the same or similar conditions could allow for meaningful secondary analyses to determine whether these demographic characteristics moderate treatment-related outcomes, and if so, identify under what conditions this is and is not the case. Ultimately this may lead the field to identify treatments that are most likely to be effective for LGBT people and treatments that may require modification to address the unique needs of this population. As a profession, the omission of this data poses significant challenges for determining whether our existing interventions are effective for LGBT people.

Moving forward, one concern that psychotherapy researchers may have involves how to measure sexual orientation and non-binary gender identities in their studies. Recommendations for the measurement of sexual orientation involve the administration of an item that captures how a person self-identifies with respect to their sexual orientation (e.g., lesbian, heterosexual, bisexual, etc.; see Wadsworth, Morgan, Hayes-Skelton, Romer, & Suyemoto, 2016). Researchers should also consider querying sexual behavior and sexual attraction, as there are important differences in mental health outcomes based on identity, behavior and attraction (see Bostwick et al., 2010). Current recommendations for querying gender identity utilize a two-step method, wherein assigned sex at birth and current gender identity are assessed in two separate questions (Center of Excellence for Transgender Health, 2016; Reisner et al., 2014). Notably, studies examining the acceptability of querying sexual orientation and gender identity in medical settings have found that these questions are administered with high levels of acceptability (Cahill et al., 2014).

In closing, this study was limited to RCTs in depression and anxiety, and thus cannot be generalized to other topic areas. We also cannot eliminate the possibility that researchers for the associated studies queried sexual orientation and non-binary gender identity, but then chose not to report it in final published reports. Our study found that among RCTs for depression and anxiety, the reporting of sexual orientation and non-binary gender identities is a rare phenomenon. This raises the question of how to evaluate whether current depression and anxiety treatments are effective for LGBT populations. Researchers are strongly encouraged to begin querying and reporting on these important demographic variables in an effort to better meet the needs of an already underserved population.

Public Health Significance.

The results of this study indicate that researchers are not reporting data pertaining to participant sexual orientation and non-binary gender identities within the context of RCTs evaluating interventions for anxiety and depression.

Not reporting data pertaining to participant sexual orientation and non-binary gender identities within RCTs poses significant challenges in determining whether current interventions for anxiety and depression are effective for lesbian, gay, bisexual, and transgender people.

Reseachers are advised to assess and report on participant sexual orientation and to expand the way that gender is conceptualized within the context of RCTs evaluating interventions for anxiety and depression.

Acknowledgments

The authors wish to thank Alex Czajkowski, Alexis Arias, Alexis Worden, Ben Pleuhs, Crystal Audi, Jillian Nichols, Nicolas Saldivar and Taylor Lee for their dutiful assistance in carrying out this review. Dr. Flentje's work on the development of this manuscript was supported in part by the National Institute on Drug Abuse under award number K23DA039800. The National Institute on Drug Abuse had no role in the design, execution, or reporting of the study.

Contributor Information

Nicholas C. Heck, Marquette University

Lucas A. Mirabito, Marquette University

Kelly LeMaire, Marquette University.

Nicholas A. Livingston, University of Montana

Annesa Flentje, University of California, San Francisco.

References

  1. Bieschke KJ, Perez RM, DeBord K. Handbook of counseling and psychotherapy with lesbian, gay, bisexual, and transgender clients. 2. Washington, DC: American Psychological Association; 2007. [Google Scholar]
  2. Bostwick WB, Boyd CJ, Hughes TL, McCabe SE. Dimensions of sexual orientation and the prevalence of mood anxiety disorders in the United States. American Journal of Public Health. 2010;100:468–475. doi: 10.2105/AJPH.2008.152942. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cahill S, Singal R, Grasso C, King D, Mayer K, Baker K, Makadon H. Do ask, Do tell: High levels of acceptability by patients of routine collection of sexual orientation and gender identity data in four diverse American community health centers. PLoS One. 2014;9:e107104. doi: 10.1371/journal.pone.0107104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Center of Excellence for Transgender Health. Recommendations for Inclusive Data Collection of Trans People in HIV Prevention, Care & Services. University of California; San Francisco: 2016. http://transhealth.ucsf.edu/tcoe?page=lib-data-collection. [Google Scholar]
  5. Cochran B, Cauce A. Characteristics of lesbian, gay, bisexual, and transgender individuals entering substance abuse treatment. Journal of Substance Abuse Treatment. 2006;30:135–146. doi: 10.1016/j.jsat.2005.11.009. [DOI] [PubMed] [Google Scholar]
  6. Cochran SD, Sullivan JG, Mays VM. Prevalence of mental disorders, psychological distress, and mental health services use among lesbian, gay, and bisexual adults in the United States. Journal of Consulting and Clinical Psychology. 2003;71:53. doi: 10.1037/0022-06X.71.1.53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cooper L, Gonzales J, Gallo J, Rost K, Meredith L, … Ford D. The acceptability of treatment for depression among African-American, Hispanic, and White primary care patients. Medical Care. 2003;4:479–489. doi: 10.1097/01.MLR.0000053228.58042.E4. [DOI] [PubMed] [Google Scholar]
  8. Flentje A, Bacca CL, Cochran BN. Missing data in substance abuse research? Researchers’ reporting practices of sexual orientation and gender identity. Drug and Alcohol Dependence. 2015;147:280–284. doi: 10.1016/j.drugalcdep.2014.11.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Flentje A, Heck N, Sorensen J. Characteristics of transgender individuals entering substance abuse treatment. Addictive Behaviors. 2014;39:969–975. doi: 10.1016/j.addbeh.2014.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Flentje A, Heck N, Sorensen J. Substance use among lesbian, gay, and bisexual clients entering substance abuse treatment: Comparisons to heterosexual clients. Journal of Consulting and Clinical Psychology. 2015;83:325–334. doi: 10.1037/a0038724. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Gates GJ. How many people are lesbian, gay, bisexual, and transgender? The Williams Institute on Sexual Orientation and Gender Identity Law and Public Policy at UCLA School of Law. 2011 Retrieved from http://williamsinstitute.law.ucla.edu/wp-content/uploads/Gates-How-Many-People-LGBT-Apr-2011.pdf.
  12. Heck N, Flentje A, Cochran B. Intake interviewing with lesbian, gay, bisexual, and transgender clients: Starting from a place of affirmation. Journal of Contemporary Psychotherapy. 2013;43:23–32. doi: 10.1007/s10879-012-9220-x. [DOI] [Google Scholar]
  13. Hendricks ML, Testa RJ. A Conceptual Framework for Clinical Work With Transgender and Gender Nonconforming Clients: An Adaptation of the Minority Stress Model. Professional Psychology Research and Practice. 2012;43:460–467. doi: 10.1037/a0029597. [DOI] [Google Scholar]
  14. Herbst JH, Kay LS, Passin WF, Lyles CM, Crepaz N, Marin BV. A systematic review and meta-analysis of behavioral interventions to reduce HIV risk behaviors of Hispanics in the United States and Puerto Rico. AIDS and Behavior. 2007;11:25–47. doi: 10.1007/s10461-006-9151-1. [DOI] [PubMed] [Google Scholar]
  15. Institute of Medicine. The Health of Lesbian, Gay, Bisexual, and Transgender People: Building a Foundation for Better Understanding. National Academies Press; Washington, DC: 2011. [PubMed] [Google Scholar]
  16. King M, Semlyen J, Tai SS, Killaspy H, Osborn D, Popelyuk D, Nazareth I. A systematic review of mental disorder, suicide, and deliberate self harm in lesbian, gay and bisexual people. BMC Psychiatry. 2008;8:1–17. doi: 10.1186/1471-244X-8-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Kraaij V, van Emmerik A, Garnefski N, Schroevers MJ, Lo-Fo-Wong D, van Empelen P, Dusseldorp E, Witlox R, Maes S. Effects of a cognitive behavioral self-help program and a computerized structured writing intervention on depressed mood for HIV-infected people: a pilot randomized control trial. Patient Education and Counseling. 2009;80:200–204. doi: 10.1016/j.pec.2009.08.014. [DOI] [PubMed] [Google Scholar]
  18. Marshal MP, Dietz LJ, Friedman MS, Stall R, Smith HA, McGinley J, Brent DA. Suicidality and depression disparities between sexual minority and heterosexual youth: A metanalytic review. Journal of Adolescent Health. 2011;49:115–123. doi: 10.1016/j.jadohealth.2011.02.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. McCabe SE, Hughes TL, Bostwick WB, West BT, Boyd CJ. Sexual orientation, substance use behaviors and substance dependence in the United States. Addiction. 2009;104:1333–1345. doi: 10.1111/j.1360-0443.2009.02596.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Meyer I. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: Conceptual issues and research evidence. Psychological Bulletin. 2003;129:674–694. doi: 10.1037/0033-2909.129.5.67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Reisner S, Conron K, Tardiff L, Jarvi S, Gordon A, Austin S. Monitoring the health of transgender and other gender minority populations: Validity of natal sex and gender identity survey items in a US national cohort of young adults. BMC Public Health. 2014;14:1224. doi: 10.1186/1471-2458-14-1224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Stewart R, Chambless D. Cognitive-behavioral therapy for adult anxiety disorders in clinical practice: A meta-analysis of effectivenss studies. Journal of Consulting and Clinical Psychology. 2009;77:595–606. doi: 10.1037/a0016032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. United States Department of Health and Human Services. Lesbian, Gay, Bisexual, and Transgender Health. 2010 http://www.healthypeople.gov/2020/topicsobjectives2020/objectiveslist.aspx?topicId=25 (retrieved on Sept 30, 2014)
  24. Wadsworth L, Morgan L, Hayes-Skelton S, Romer L, Suyemoto K. Ways to boost your research rigor through increasing your cultural competence. The Behavior Therapist. 2015;39:76–92. [Google Scholar]
  25. Westen D, Morrison K. A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: an empirical examination of the status of empirically supported therapies. Journal of Consulting and Clinical Psychology. 2001;69:875–899. doi: 10.1037/0022-006X.69.6.875. [DOI] [PubMed] [Google Scholar]

RESOURCES