Comparing substance use and mental health outcomes among sexual minority and heterosexual women in probability and non-probability samples

Abstract

Objective

To examine similarities and differences in demographics and key substance use and mental health outcomes in a probability sample of heterosexual women and two samples of sexual minority women (SMW), one recruited using probability and the other using non-probability methods.

Methods

Using data from four waves of the National Alcohol Survey (NAS; n = 315 SMW; 10,523 heterosexual women) and Wave 3 of the Chicago Health and Life Experiences of Women (CHLEW; n = 688 SMW) study, we examined hazardous drinking, drug use, tobacco use, depression, and help-seeking for alcohol or other drug problems.

Results

Compared to SMW in the probability sample, SMW in the non-probability sample were older, more likely to be college educated, and more likely to be in a partnered relationship. Compared to heterosexuals, SMW in both the probability and non-probability samples had greater odds of past-year hazardous drinking, marijuana use, and other drug use. We found similar results for lifetime help-seeking for alcohol or drug problems, past week depression, and co-occurring hazardous drinking and depression. In comparisons with heterosexual women, the magnitude of difference for drug use was greater for the SMW non-probability sample; for tobacco use, the difference was greater for the SMW probability sample.

Conclusion

Given the difficulties recruiting probability samples of SMW, researchers will continue to use non-probability samples in the foreseeable future. Thus, understanding how findings may differ between probability and non-probability samples is critically important in advancing research on sexual-orientation-related health disparities.

1. Introduction

Early research on alcohol-related problems among sexual minority women (SMW) relied on convenience samples, such as bar patrons, which raised concerns about generalizability (Hughes, 2011). Over the past several decades, the inclusion of sexual orientation questions in surveys using probability samples, and the use of more rigorous non-probability sampling methods, has contributed substantially to understanding sexual-orientation-related health disparities (Institute of Medicine, 2011). Nevertheless, different sampling methodologies have distinct strengths and limitations. Probability samples allow for greater generalizability and may support comparisons between heterosexual and sexual minority samples, but are limited by small samples of sexual minority people (Meyer and Wilson, 2009). In addition, studies that include large probability samples rarely include sexual-minority-specific measures (e.g., internalized stigma, connection to the sexual minority community, or the extent to which and to whom participants have disclosed their sexual orientation), which limits investigation of mechanisms underlying sexual-orientation-related health disparities. By contrast, although limited in generalizability, non-probability studies are often designed to obtain large samples of sexual minorities that enable examination of within-group differences (e.g., comparisons across sexual identity or race/ethnicity) and include sexual-minority-specific measures (Institute of Medicine, 2011; Meyer and Wilson, 2009).

Survey research, using either probability samples or non-probability samples, has generally found higher rates of alcohol consumption, hazardous drinking, tobacco use, and other drug use among SMW compared to heterosexual women (Blosnich et al., 2013; Drabble et al., 2005; Hughes et al., 2016; Jackson et al., 2016; Kerridge et al., 2017; Lee et al., 2009; McCabe et al., 2009; Operario et al., 2015; Trocki et al., 2009). For example, probability studies of alcohol and drug use in the United States that include sexual orientation measures, such as the National Alcohol Survey (NAS) and the National Epidemiological Survey on Alcohol and Related Conditions (NESARC), have found higher odds of reporting heavy drinking, alcohol-related consequences and dependence symptoms, other drug use, and substance use treatment among SMW than among heterosexual women (Allen and Mowbray, 2016; Drabble et al., 2005; Gattis et al., 2012; Kerridge et al., 2017; McCabe et al., 2009). Similarly, the Chicago Health and Life Experiences of Women Study (CHLEW), which includes a large mid-western non-probability sample, has documented higher levels of hazardous drinking among SMW compared to a national sample of heterosexual women (Hughes et al., 2010b; Wilsnack et al., 2008). Moreover, survey research with both probability and non-probability samples has generally found greater risk of psychological distress, including symptoms of depression, among SMW compared to heterosexual women (Bostwick et al., 2010; Hughes et al., 2010a; King et al., 2008; Mereish et al., 2015). Sexual minority women who meet criteria for alcohol abuse and/or dependence also appear more likely to have co-occurring psychiatric disorders (e.g., mood disorders) and drug use disorders (Mereish et al., 2015).

Only a few studies have directly compared outcomes using non-probability and probability samples, and the majority of these focused exclusively on gay or bisexual men or men who have sex with men (MSM; Brewer et al., 2008; Dodds et al., 2006; Evans et al., 2007; Meyer and Colten, 2008). In the United States, methodological comparisons of SMW have been conducted in limited geographic areas, such as a study comparing demographics between convenience and Census samples in Rhode Island (Boehmer et al., 2009) and a study comparing demographics and cancer risks between two non-probability samples and two probability samples from the Seattle and Boston regions (Bowen et al., 2007). Boehmer and colleagues also compared a convenience sample to a probability cancer registry sample of SMW in Massachusetts (focusing on demographics, adjustment to cancer, and quality of life; Boehmer et al., 2011). National comparisons between non-probability and probability samples of sexual minorities, including SMW, have been conducted in Belgium (examining possible differences in demographics and sexual health indicators; Dewaele et al., 2014) and the Netherlands (examining demographics, minority stress, and psychological distress; Kuyper et al., 2016). In one of the few studies comparing sexual minorities from both probability and non-probability samples to heterosexuals, Hottes and colleagues examined estimates of lifetime suicide attempts using pooled data from 30 studies using cross-sectional designs (9 probability and 21 non-probability) from the U.S., Canada, and several countries in Western Europe (Hottes et al., 2016).

In general, researchers have found some demographic differences in non-probability vs. probability samples (e.g., non-probability samples tend to be more highly educated), as well as similarities and differences across outcomes. For example, Dewaele and colleagues (2014) found similarities in five of seven sexual health indicators when comparing health outcomes among SMW from probability and non-probability samples. Kuyper and colleagues (2016) found comparable relationships between key variables in a non-probability compared to a large representative panel sample (e.g., psychological distress was related to encountering negative reactions to sexual identity), though some estimates varied. Similarly, Hottes and colleagues (2016) found higher rates of suicide attempts among sexual minorities regardless of sample type. However, compared to the heterosexual sample (4%), the magnitude of difference was significantly greater in the non-probability (20%) than probability sample (11%).

Studies comparing outcomes among SMW across sampling strategies to date have yet to focus explicitly on alcohol and other drug use. Such comparisons are particularly important, given that substance use, particularly alcohol use, is among the most prominent health-related disparities in comparisons of heterosexual and sexual minority women (Drabble et al., 2005; Hughes et al., 2016; McCabe et al., 2009). In addition, studies to date comparing outcomes from probability and non-outcomes sampling strategies often merge different dimensions of sexual orientation (e.g., identity and behavior; Dewaele et al., 2014; Hottes et al., 2016; Kuyper et al., 2016). Research suggests that sexual identity measures are particularly salient in research designed to assess risk for hazardous drinking or drug use among SMW (Midanik et al., 2006). Furthermore, as health outcomes, including alcohol and other drug, may by influenced by the social and political climate (Hatzenbuehler et al., 2010; Hottes et al., 2016), research specifically examining differences in alcohol and drug outcomes that use probability and non-probability samples in the United States are needed.

To our knowledge, no studies of substance use among SMW have explicitly compared findings from carefully recruited non-probability samples with those from probability samples, and none have included a probability sample of heterosexual women as a second level of comparison. Such research is important for several reasons. First, stigma, fear of discrimination, and pressures of heteronormativity may impact participation and disclosure of minority sexual identity in probability samples (Robertson et al., 2017). Second, because of their low prevalence in the population, it is also cost prohibitive for the majority of researchers to recruit probability samples of SMW that are large enough to conduct meaningful analyses, especially if they are interested in comparing sub-samples based on particular demographic (e.g., race/ethnicity, SES) or other (history of childhood abuse) characteristics. Even the largest studies of substance use in the U.S. include few SMW; for example, the NESARC-III study (of over 36,000 respondents) included 265 lesbian women (Kerridge et al., 2017; Slater et al., 2017). Thus, it is critically important to understand how findings from probability and non-probability samples are similar or different so that future non-probability studies can be strengthened. In addition, research on potential differences in health outcomes yielded by non-probability and probability samples of SMW in comparison to heterosexuals is important to interpreting findings from studies using different sampling strategies.

We used data from two large studies of SMW (the NAS and the CHLEW) to address the following research questions: 1) How similar or different are socio-demographic characteristics (e.g., age, education, race/ethnicity) in a probability sample compared to a non-probability sample of SMW; and 2) How similar or different are SMW in each of these samples to a probability sample of heterosexual women on key substance use and mental health outcomes (hazardous drinking, drug use, tobacco use, help-seeking for alcohol and drug problems, depression, co-occurring hazardous drinking and depression and co-occurring hazardous drinking and drug use)?

2. Methods

2.1. Design and samples

This study combined cross-sectional data from two studies: the National Alcohol Survey (NAS) and the Chicago Health and Life Experiences of Women Study (CHLEW). Data from participants 70 years of age and older were excluded from the study, as the CHLEW had few participants in this age group. All study procedures were approved by the Public Health Institute (PHI) Institutional Review Board.

2.1.1. National Alcohol Survey (NAS)

The NAS is a cross-sectional study focusing on alcohol use and a wide range of alcohol-related problems in a national probability sample of adults in the United States. Surveys are conducted approximately every five years; since 2000, data have been collected using computer-assisted telephone interviews (CATI) in all 50 U.S. states and Washington, DC. The current study combined data from the 2000, 2005, 2010, and 2015 NAS surveys and includes 315 SMW and 10,523 heterosexual women. All NAS participants were contacted using random digit dialing (RDD) with oversampling of African-Americans, Latinos and low-population states (exclusively through landlines in 2000 and 2005 and via both landlines and cell phones in 2010 and 2015). See: Drabble et al., 2013 for additional details.

2.1.2. Chicago Health and Life Experiences of women (CHLEW)

The non-probability sample of SMW is from Wave 3 (2010–2012) of the CHLEW study, a longitudinal study of risk and protective factors associated with alcohol use and alcohol-related problems. Participants (n = 447) in the first wave of the CHLEW study (2000–2001) were predominantly lesbian and were recruited using a broad range of strategies, including outreach to community-based organizations, the media, and individual social networks. Recruitment strategies focused on increasing representation of SMW previously underrepresented in research with SMW, such as those who were older, of lower socioeconomic status, or racial/ethnic minorities. The sample was re-interviewed in 2004–2005 (Wave 2; retention rate=86%) and in 2010–2012 (Wave 3; retention rate=79%). In Wave 3, a supplemental sample of 373 younger women (ages 18–25), African American and Hispanic women, and bisexual women were recruited using an adaptation of respondent-driven sampling (Martin et al., 2015). Data for the current study are from 688 self-identified lesbian and bisexual women interviewed in Wave 3.

2.2. Measures

2.2.1. Sexual identity

Sexual identity was assessed in both the NAS and CHLEW by asking participants, “Recognizing that sexual identity is only one part of your identity, would you say that you are: ‘only lesbian/gay,’ ‘mostly lesbian/gay,’ ‘bisexual,’ ‘mostly heterosexual,’ ‘only heterosexual/straight’ or ‘something else/other?’” Survey years prior to 2015 from the NAS used the question, “What is your sexual orientation?” with the response options of heterosexual, lesbian, or bisexual. Because the option of "mostly heterosexual” was not available in all NAS surveys, these women were excluded from analysis. Women who identified as lesbian/mostly lesbian, bisexual (CHLEW and NAS) or heterosexual (NAS) are included in the current analyses.

2.2.2. Alcohol, other substance use, and depression

2.2.2.1. Alcohol use

Two dichotomous measures of alcohol use included any drinking to intoxication (perceived drunkenness), in the past year (yes/no) and any heavy episodic drinking, defined as having had five or more drinks (NAS) or six or more drinks (CHLEW) on any occasion at least once, in the past 12 months. In addition, age of first drink (age started drinking alcoholic beverages) was measured as a continuous variable.

2.2.2.2. Alcohol dependence symptoms

A dichotomous measure of alcohol dependence symptoms in the past 12-months was created based on affirmative response to one or more of four items: 1) took a strong drink in the morning to get over the effects of the night before; 2) kept on drinking even after attempting to set drinking limits; 3) tried to cut down or quit but were unable to do so; and 4) could not stop drinking before becoming intoxicated.

2.2.2.3. Negative consequences due to alcohol use

Women who reported that their alcohol use was a serious threat to their physical health or that drinking had hurt their chances for a work promotion or raise were classified as having experienced an alcohol related consequence (any vs. none, past 12-months).

2.2.2.4. Drug use

Dichotomous measures were created for past year marijuana use and illicit drug use based on responses to a series of questions about past 12-month use of marijuana, THC, hashish, "pot" or "weed"—as well as other illicit drug use (i.e., stimulants, cocaine/crack, heroin/illegal methadone, hallucinogens, club drugs and non-medical use of prescription drugs). The NAS asked a general question about any drug use, with follow-up questions about specific substances, whereas the CHLEW asked separate questions about five categories of illicit drugs and three categories of non-medical use of prescription drugs. Additionally, a composite measure was created based on affirmative responses to questions about marijuana or illicit drug use (any vs. no use in the past 12 months).

2.2.2.5. Tobacco Use

Tobacco use was assessed in the NAS using responses to a question about how often participants smoked tobacco, or used other kinds of tobacco, in the past 12 months. In the CHLEW, we used responses to two questions about current tobacco use (cigarette smoking and other tobacco use) and a third question about quitting among former smokers (those who quit within the past 12 months were classified as past year smokers).

2.2.2.6. Hazardous Drinking Index (HDI)

This index was constructed using four dichotomous variables including drinking to intoxication, heavy episodic drinking, one or more dependence symptoms, and one or more negative alcohol problem consequences (range = 0 to 4). A dichotomous measure of hazardous alcohol use was created, which included two or more of the four indicators vs. one or none.

2.2.2.7. Self-reported depression

The depression measure was constructed using comparable items from the two studies that asked about symptoms of depression in the past week. Responses were coded (rarely or none of the time, less than a day; some or little of the time, 1–2 days; occasionally or a moderate amount of time, 3–4 days; or most or all of the time, 5–7 days). We created a dichotomous past week depression symptoms variable that included none/rarely vs. any past week depression.

2.2.2.8. Lifetime help-seeking

Lifetime help-seeking was constructed as a dichotomous variable using responses to questions in each dataset about having ever gone to anyone (e.g., a doctor or physician, a treatment agency or self-help organizations such as Alcoholics Anonymous or Narcotics Anonymous) for an alcohol or drug problem.

2.2.2.9. Co-occurring HDI/depression and HDI/drug use

Because prior research suggests that, among women with alcohol use disorders, SMW appear to be more likely than heterosexual women to report concurrent mood disorders or concurrent drug use disorders (Mereish et al., 2015), dichotomous measures were created for participants who reported both hazardous drinking (2+ indicators) and depression in the previous year. A similar measure was constructed for co-occurring past year hazardous drinking and illicit drug use.

2.2.3. Demographic characteristics

Demographic measures included age in years, education (high school diploma or less vs. some college or more), employment (full-time, part-time, unemployed, retired, or other [disabled/student/stay at home]), race/ethnicity (African-American, Hispanic/Latino, White, other/mixed) and relationship status (married/cohabiting/committed relationship vs. not in a relationship).

2.3. Analysis

We used Chi square analyses in comparisons of categorical variables, and analysis of variance (ANOVA) in comparisons of continuous measures. Follow-up tests for significant differences between study samples for categorical variables used a significance level of p < 0.01 to adjust for multiple comparisons. Bonferroni post hoc tests were conducted for ANOVA analyses. We used mixed model logistic regression analyses for models predicting the study outcomes. These models controlled for age, race/ethnicity, education, employment status, and relationship status as well as survey year. Analyses were conducted using SPSS (v24) and Stata (V14) statistical software. Primary analyses examined outcomes by sexual identity, with heterosexuals as the reference group. Follow-up analyses to identify significant differences between sexual minority groups were conducted using the NAS SMW as the reference group. Significant differences between SMW samples are indicated in footnotes in each of the tables.

3. Results

3.1. Demographic similarities and differences

Table 1 compares demographic characteristics of the three samples. The CHLEW sample included more lesbian women because initial recruitment focused on women who identified as lesbian; bisexual women were not actively recruited until Wave 3. SMW in the NAS were almost a decade younger, on average, than heterosexual women in the NAS and approximately five years younger than CHLEW SMW. Significant differences in relationship status parallel age differences: the percentage of women in partnered relationships was highest among NAS heterosexual women, followed by CHLEW SMW, and lowest among NAS SMW. The CHLEW sample was more highly educated and less likely to be White (because of deliberate oversampling of women of color), and more CHLEW SMW were unemployed. The NAS heterosexual sample was significantly older (approximately 2 years) at age of first drink compared to either of the SMW samples.

Table 1.

Baseline Demographic Characteristics and outcome measures by sample.

	NAS Heterosexual Women (n = 10,523)	NAS SMW (n = 315)	CHLEW SMW (n = 688)
	%	%	%
Sexual OrientationC
Bisexual	--	53.6	26.0
Mostly/only lesbian	--	46.4	74.0
Age in categoriesA,B,C
18–29	19.2	41.8	30.1
30–39	20.7	25.6	22.2
40–49	23.6	17.7	20.6
50–59	21.7	10.4	17.6
60–69	14.7	4.4	9.4
Current partner statusA,B,C
Unpartnered	33.7	55.5	38.8
Partnered	66.3	44.5	60.8
EducationB,C
Some college+	60.2	62.9	79.2
Race/ethnicityB,C
White	69.8	66.1	36.3
African-American	11.8	11.4	36.6
Hispanic	12.7	12.7	23.5
Other/mixed	5.6	9.9	3.5
EmploymentA,B,C
Full-time	45.5	46.0	44.8
Part-time	16.2	21.6	23.7
Unemployed	5.4	6.5	15.6
Retired	9.1	2.7	10.5
Other (disabled/student/stay at home)	23.5	23.1	5.4
Alcohol Measures
Drinking status, 12moA,B,C
Current drinker	64.0	78.4	86.3
Ex-drinker	14.9	12.2	9.4
Abstainer	21.1	9.4	4.2
Drinking to intoxicationA,B,C	26.5	49.3	59.0
Any 5+ (6+ CHLEW) drinkingA,B	13.5	33.8	39.3
1+ Alcohol dependence symptomsA,B	3.0	8.7	12.8
1+ Alcohol consequencesA,B	0.7	2.3	5.9
2+ Hazardous Drinking Index (HDI)A,B	13.0	32.6	37.8
Other substance use
Tobacco useA,B,C	23.6	51.9	38.2
Marijuana useA,B,C	6.4	30.4	41.8
Prescription drug use, illicitA,B	4.2	12.2	12.5
Any illicit drug useA,B	9.9	33.2	42.7
Treatment for alcohol or drugsA,B
Yes, lifetime	3.7	17.7	19.2
Depression
None of the time/rarely	73.2	60.5	62.6
A little of the time	16.9	20.9	22.9
A moderate amount of time	5.4	10.9	9.8
Most of the time	4.6	7.7	4.8
Depression (2 category)
None of the time/rarelyA,B	73.2	60.5	62.6
More often	26.8	39.5	37.4
2+HDI and DepressionA,B	3.6	12.2	17.2
2+HDI and Illicit drug useA,B,C	3.7	15.2	23.8
Continuous measures	Mean(sd)	Mean(sd)	Mean(sd)
AgeA,B,C	43.5 (13.6)	34.3 (12.4)	39.5 (13.6)
Age at first drinkA,B	19.2 (5.2)	17.1 (3.7)	16.7 (3.9)

^Ap<0.01; pairwise comparison, NAS heterosexuals and NAS SMW

^Bp<0.01; pairwise comparison, NAS heterosexuals and CHLEW

^Cp<0.01; pairwise comparison, NAS SMW and CHLEW

3.2. Comparison of substance use and mental health outcomes

Table 1 also summarizes bivariate results for the three samples relative to alcohol and drug use, first for the full sample and then for drinkers only. Each of the indicators of hazardous drinking was significantly higher among SMW in both samples compared to heterosexual women. CHLEW SMW were significantly less likely to report alcohol abstention and more likely to report past-year drinking to intoxication than NAS SMW. History of substance use treatment was significantly greater for SMW than for heterosexual women; the SMW samples did not differ on this outcome. We found the same pattern for depression and for co-occurring depression and hazardous drinking: there were no differences between the NAS and CHLEW SMW samples, but significantly higher estimates among both SMW samples compared to the heterosexual sample.

Use of marijuana, tobacco, and other drugs were significantly lower among heterosexual women than SMW in either sample. Differences between the two SMW samples were statistically significant, but in different directions: tobacco use was higher among NAS SMW, whereas marijuana use and any drug use were higher among CHLEW SMW. Past year reports of co-occurring hazardous drinking and drug use were also significantly lower among heterosexual women compared to the SMW samples. Not surprisingly, given their higher rates of marijuana use, CHLEW SMW were more likely than NAS SMW to report co-occurring hazardous drinking and drug use.

Tables 2, 3 and 4 summarize multivariate findings of outcomes among each of the SMW samples compared to heterosexual women. Table 2 shows that, in the full sample, both SMW samples reported significantly higher rates of each of the hazardous drinking indicators than did heterosexual women. Results were similar in analyses limited to current drinkers only; however, elevated odds among NAS SMW compared to heterosexual women reached significance only for heavy episodic drinking. Table 3 summarizes findings related tobacco and other drug use. Odds for all outcomes were significantly greater among both SMW samples than among heterosexual women. Consistent with findings from bivariate analyses, the magnitude of difference compared to heterosexuals was greater among NAS SMW for tobacco use and greater among CHLEW SMW for marijuana and other drug use. Finally, Table 4 shows that odds for depression, co-occurring hazardous drinking and depression, co-occurring hazardous drinking and drug use, and seeking help for alcohol or drug problems were greater for both SMW samples compared to heterosexual women.

Table 2.

Mixed model regressions predicting alcohol use and alcohol problems, past 12 months.

	Current drinker		Drinking to intoxication		Heavy drinking		2+ HDI
	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI
Total Sample
NAS hetero (ref)	--	--	--	--	--	--	--	--
NAS SMW	1.7G	1.6, 1.8	1.8F	1.2, 2.6	2.0F	1.3, 3.1	2.0E	1.2, 3.3
CHLEW SMW	4.0G, 2	2.4, 6.5	4.5G, 2	2.7, 7.8	4.3G	2.1, 9.4	4.2G	1.9, 9.0
Current drinkers
NAS hetero (ref)	--	--	--	--	--	--	--	--
NAS SMW	--	--	1.4	0.9, 2.4	1.7E	1.1, 2.8	1.7	0.9, 3.0
CHLEW SMW	--	--	3.4G,2	2.3, 4.9	3.3G	1.7, 6.2	3.1F	1.6, 5.9

Models control for the fixed effects of age, race/ethnicity, education, employment, partner status, and survey year.

^Ep<0.05;

^Fp<0.01;

^Gp<0.001

Follow up tests of differences between NAS and CHLEW SMW samples:

¹p<0.05;

²p<0.01;

³p<0.001

Table 3.

Mixed model regressions predicting tobacco and other drug use, past 12 months.

	Tobacco use		Marijuana use		Prescription drug use		Any illicit drug use
	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI
Total Sample
NAS hetero (ref)	--	--	--	--	--	--	--	--
NAS SMW	3.2G	2.5, 4.2	3.9G	2.5, 6.0	2.9G	1.7, 5.1	3.5G	2.3, 5.4
CHLEW SMW	2.6F,1	1.4, 4.8	10.3G, 1	5.9, 18.3	3.7F	1.8, 7.9	6.4G, 1	3.9, 10.5
Current Drinkers
NAS hetero (ref)	--	--	--	--	--	--	--	--
NAS SMW	2.6G	2.1, 3.3	3.0G	1.8, 4.8	2.5G	1.6, 3.8	2.9G	2.3, 5.4
CHLEW SMW	2.2F, 2	1.3, 3.5	8.0G, 2	4.8, 13.4	4.1G, 2	2.2, 7.8	5.8G, 3	3.9, 10.5

Models control for the fixed effects of age, race/ethnicity, education, employment, partner status, and survey year.

^Fp<0.01;

^Gp<0.001

Follow up tests of differences between NAS and CHLEW SMW samples:

¹p<0.05;

²p<0.01;

³p<0.001

Table 4.

Mixed model regressions predicting treatment seeking, depression, depression with 2+ HDI, and depression with illicit drug use.

	Any treatment seeking		Any Depression		2+ HDI and Depression		2+ HDI and Any illicit drug use
	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI
Total Sample
NAS hetero (ref)	--	--	--	--	--	--	--	--
NAS SMW	5.2G	4.1, 6.8	1.7G	1.5, 1.9	2.4G	1.8, 3.0	3.0F	1.5, 5.9
CHLEW SMW	7.6G	5.3, 11.0	1.7G	1.4, 2.2	5.2G	2.5, 11.0	8.4G, 2	4.0, 18.0
Current Drinkers
NAS hetero (ref)	--	--	--	--	--	--	--	--
NAS SMW	5.3G	4.2, 6.8	1.7G	1.4, 2.1	2.0G	1.5, 2.6	2.6E	1.2, 5.4
CHLEW SMW	6.6G	3.6, 12.1	1.7G	1.4, 2.1	3.9G, 1	2.0, 7.6	6.5G, 2	3.3, 12.8

Models control for the fixed effects of age, race/ethnicity, education, employment, partner status, and survey year.

^Ep<0.05;

^Fp <0.01;

^Gp<0.001

Follow up tests of differences between NAS and CHLEW SMW samples:

¹p<0.05;

²p<0.01;

³p<0.001

Follow-up analyses were conducted to examine potential differences between the probability and non-probability SMW samples in relation to heterosexual women (using the NAS probability sample as the reference group). Significant differences between the SMW samples are summarized in the footnotes of Tables 2, 3 and 4. Reporting any past-year alcohol consumption or drinking to intoxication was higher in the non-probability CHLEW sample compared to the probability NAS sample. Consistent with findings from bivariate analyses, the magnitude of difference in comparisons with heterosexual women was greater among NAS SMW for tobacco use and greater among CHLEW SMW for marijuana and drug use. The odds of reporting concurrent past year hazardous drinking and drug use or recent depression were also greater among CHLEW SMW than NAS SMW.

4. Discussion

We examined how probability and non-probability samples of SMW compare in findings related to hazardous drinking, drug use, and depression, and how findings from these two samples of SMW compared with a nationally representative sample of heterosexual women. Regardless of sample type, we found significantly greater odds among SMW than heterosexual women in 21 of 23 comparisons (12 with the full sample and 11 with drinkers only). The only two comparisons that did not reach significance were drinking to intoxication and hazardous drinking; however, odds for these two outcomes were elevated among the NAS SMW probability sample relative to heterosexual women, although differences did not reach significance. Findings from this study are important, as they suggest that high levels of risk for hazardous drinking, drug use, and depression found in non-probability samples of sexual minorities are not artifacts of the sampling method. Confidence in the generalizability of findings related to health outcomes among SMW, using different sampling strategies, compared to heterosexuals is important in a context where a majority of studies of SMW will necessarily use non-probability samples. Among the challenges of obtaining valid and reliable data for describing SMW’s health is obtaining high-quality samples of a relatively small population. There are still few large-scale probability studies that assess sexual orientation. Even in those that do, the small number of SMW make it difficult to estimate reliable parameters. Sexual minority women are a heterogeneous group and to understand their health it is necessary to consider differences based on race, ethnicity, geographic location, socioeconomic status, age, and other factors. However, this requires large samples. Another important limitation of nearly all large, probability surveys is that, due to their broad focus, they do not include measures necessary to investigate mechanisms underlying sexual-orientation-related health disparities (e.g., stigma, discrimination, fear of disclosing sexual orientation). Given the scarcity of methodological research specific to SMW, additional methodological studies comparing health outcomes by sample type are needed to understand the strengths and limitations of non-probability samples and how researchers can maximize strengths and minimize limitations.

We found differences in demographic characteristics between the probability and non-probability SMW samples. Compared with the NAS probability sample of SMW, the CHLEW non-probability sample of SMW were older, more likely to be college educated, and more likely to be in a partnered relationship. The few methodological studies that included comparisons of non-probability and probability samples of SMW have generally found non-probability samples to be younger (Dewaele et al., 2014; Hottes et al., 2016; Kuyper et al., 2016) and less likely to be unemployed than probability samples (Dewaele et al., 2014; Kuyper et al., 2016). In contrast to our findings, Dewaele et al. found that participants in a non-probability sample were less likely than those in a probability sample to report being in a partnered relationship. Additional research on possible selection bias in both non-probability and probability samples is needed to inform the evaluation and interpretation of findings from studies with sexual minority people.

Follow-up analyses revealed significant differences between the SMW samples. Reports of alcohol consumption and some indicators of hazardous drinking, such as past year intoxication, in the non-probability SMW sample were higher than the probability SMW sample. Differences between the two SMW samples may be accounted in part to variability in drinking patterns across regions in the U.S. The non-probability sample of SMW was recruited from a state (Illinois) that is located in a region of the U.S. with a higher per capita consumption of alcohol and proportions of heavy occasion drinkers than other regions in the U.S. (Kerr, 2010) It is also possible that differences in findings between the two samples of SMW in the current study could be related to under-reporting or misclassification of SMW in the NAS. Because of continued stigma, respondents may be reluctant to disclose their sexual minority status in probability surveys (Ferlatte et al., 2017; Hottes et al., 2015; Robertson et al., 2017). A methodological study of data from the National Alcohol Survey (the same survey as in the current study), found evidence that women and men over age 40 appear less likely than younger participants to disclose their sexual minority status, even in anonymous telephone interviews (Midanik and Greenfield, 2008). Given the small number of sexual minority respondents in probability samples, such under-reporting or misclassification could reduce effect sizes in comparisons of findings by sexual identity.

The magnitude of difference for outcomes related drug use were significantly different in the two SMW samples. We found higher reports of drug use in the non-probability sample. Other studies have also found higher rates of some health or mental health outcomes in non-probability samples than in probability samples (Dewaele et al., 2014; Hottes et al., 2016; Kuyper et al., 2016). Non-probability samples of sexual minorities likely disproportionately represent individuals who are connected to sexual minority communities and visible, and therefore more exposed stigmatizing attitudes and discrimination—factors shown to be associated with poorer mental health (Conlin et al., 2017; Slater et al., 2017). Variation in drug use outcomes among between-samples of SMW may also be influenced by regional and community norms. For example, one study using a population-based sample found that both perceived availability of drugs and permissive social norms related to drug use were higher among sexual minorities than heterosexuals, and were predictive of substance use including marijuana use, illicit drug use, and heavy drinking (Cochran et al., 2012). Future research is needed to better understand whether factors such as perceived availability of drugs and permissive social norms are more salient among SMW recruited through non-probability sampling than probability sampling.

The probability sample of SMW reported higher rates of tobacco use than the non-probability SMW. This difference may be explained at least in part by geography. The non-probability sample was obtained entirely from the third largest urban area in the United States whereas the NAS probability sample includes women from rural, suburban, and urban areas of the United States. Studies of tobacco use have consistently found that smoking is higher in rural areas than in urban or suburban areas (Bell et al., 2009; Vander Weg et al., 2011).

Studies using probability and non-probability samples have each contributed to understanding sexual-orientation-related health disparities. Inclusion of questions about sexual orientation on national surveys remains critical to the accurate assessment of population health (Institute of Medicine, 2011; Meyer and Wilson, 2009). Although large, national probability samples may yield more representative samples, the small proportion of the sample that identifies as sexual minority may be insufficient to examine group differences within sexual minority populations (Meyer and Wilson, 2009). Non-probability samples are also often more cost-effective and yield larger samples, which allow for examining within-group differences. For example, a recent CHLEW study was able to examine both sexual identity (lesbian compared to bisexual) and racial/ethnic (Black, Latina, and White) differences in reports of depression, hazardous drinking, and utilization of alcohol and mental health treatment services. Because the sample was large and diverse enough to support within-group comparisons, it was possible to identify sizable unmet needs for both mental health and substance use treatment among Latina SMW in the sample (Jeong et al., 2016).

4.1. Limitations

Although the NAS and CHLEW measures used in this study were comparable, readers should consider the study’s limitations. Both the NAS and the CHLEW include many questions about alcohol problem consequences and alcohol dependence symptoms. However, our comparisons were limited to questions with exact (or very similar) wording. Consequently, we may have underestimated alcohol consequences and symptoms of alcohol dependence or have an incomplete picture of how the two samples differ on these variables. This is also true for depression, as we relied on self-report of perceived past-week depression rather than a clinical measure. The NAS survey asked about any drug use, with follow up questions about specific types of drugs, while the CHLEW survey asked individual questions for multiple types of drugs. This difference in how drug questions were asked may partly explain higher estimates among CHLEW compared to NAS SMW. Surveys that ask a multiple drug-specific questions of all respondents may yield higher estimates of drug use than surveys that ask a general question about drug use with follow-up questions about specific substances only among respondents who indicate any use (Harrison and Hughes, 1997). It is also possible that the recruitment methods used in the CHLEW—a modified version of RDS—using social networks, tapped into heavier drinking or drug use. Finally, the smaller numbers of SMW yielded in the probability sample (approximately one-half the size of the non-probability sample) may have resulted in small effect sizes and underestimated significant differences, resulting in risk of Type II errors.

Despite these limitations, this study contributes to a small but emerging body of research that supports the value of non-probability samples by examining similarities and differences in estimates of risk generated by probability and non-probability samples of SMW. Given that non-probability and probability samples each have strengths and limitations, it is important to recognize them as complimentary approaches that can each advance understanding of sexual-orientation-related health disparities. Further, the limitations of existing probability and non-probability sampling methods for SMW make this an excellent area for future research that evaluates new sampling methodologies. For example, in regard to probability sampling, methodological research is needed to demonstrate how to effectively over-sample SMW in order to permit more precise estimates, particularly within sub-groups. In regard to non-probability sampling, researchers need to understand how to efficiently recruit SMW to maximize external validity. The recent Institute of Medicine report (2011) on LGBT health emphasizes that methodological research should be considered a priority area. Additional studies that are designed to include and analyze a broader array of comparable measures across studies are needed. Studies might also be designed to better account for demographic differences (e.g., education and age) in comparisons of probability and non-probability samples among sexual minorities, such as using propensity score matching with combined samples (Dewaele et al., 2014).

4.2. Conclusion

Given the difficulties in recruiting probability samples of SMW, researchers will continue to use non-probability samples in the foreseeable future. Thus, understanding how findings may differ between probability and non-probability samples is critically important in advancing research on sexual-orientation-related health disparities. Using data from both probability and non-probability samples, our findings add to the growing evidence that SMW are at greater risk than heterosexual women for hazardous drinking, tobacco use, drug use, and depression. At the same time, variations in demographics and some outcomes between probability and non-probability samples of SMW underscore the importance of additional research designed to better understand and account for possible biases in different sampling strategies. Findings also highlight the importance of including sexual orientation measures in probability studies and the value of large, well-designed non-probability studies of SMW.