Adding Sexual Orientation Questions to Statewide Public Health Surveillance: New Mexico's Experience

Abstract

We examined refusal rates for sensitive demographic questions to determine whether questions on sexual orientation are too sensitive for routine use on public health surveys. We compared the percentage of active refusals in New Mexico for a sexual orientation question and 6 other sensitive demographic questions. In 2007 and 2008, refusal rates for sexual orientation questions were similar to rates for questions on race/ethnicity and weight and significantly lower than rates for questions on household income. Perceptions that sexual orientation is too controversial a topic to be included on state surveys may be unfounded.

Lesbian, gay, bisexual, and transgender (LGBT) populations have clear disparities in cigarette smoking,^1,2 suicidal ideation,³ violent victimization,^4–6 and sexually transmitted infections⁷ compared with the general population. Yet demographic questions about sexual orientation (i.e., questions about identity, attraction, or behavior), in addition to those on gender identity, are often not included in routine health surveys.^8–11

The Behavioral Risk Factor Surveillance System (BRFSS) survey, Adult Tobacco Survey (ATS), Youth Risk Behavior Survey, and Youth Tobacco Survey make up the backbone of state health surveillance in the United States, providing benchmarks, informing interventions, and allowing for comparisons between states. In 1998, the Centers for Disease Control and Prevention (CDC) chose not to include a question on sexual orientation in the Youth Risk Behavior Survey, leaving the decision to individual states,¹² which collaboratively administer a core survey and state-added modules. This decision has remained in place and covers, in practice, other CDC-run, state-administered surveys such as the BRFSS, ATS, and Youth Tobacco Survey. Over the subsequent 12 years, approximately 13 states have collected information on sexual orientation in the BRFSS¹³; California does so by using the separate California Health Interview Survey (http://www.chis.ucla.edu). Even fewer states collect such data on other statewide surveys.¹⁴

No research has examined why states do not include such questions on their surveys. Several possible explanations exist; public health practitioners may (1) feel that the evidence of health disparities is not enough to warrant inclusion of such questions in state surveys, (2) perceive a lack of expertise and capacity to develop and include questions on sexual orientation and gender identity, or (3) believe sexual orientation questions will yield a high nonresponse rate or cause early survey termination because of respondents' discomfort with the topic. A fourth possibility is that decisionmakers may not support inclusion of such questions because of political or personal biases.

Strong evidence and resources are available to address the first⁹ and second concerns; technical assistance is available to enhance capacity and cultural competency through the National LGBT Tobacco Control Network (http://www.lgbttobacco.org), and the University of California, Los Angeles's (UCLA's) Williams Institute recently released a best-practices guide to asking questions on sexual orientation.¹⁵ We address the third concern—the belief that sexual orientation questions may be too sensitive for use in public surveys—by using New Mexico's experience in including such questions in 2 statewide population-based surveys. Addressing the last concern, regarding political and personal bias, is outside the scope of this report.

Early research on sexual orientation questions and refusal rates came from surveys of women and health care professionals^16–18 and, recently, from surveys in the Pacific Northwest¹⁹ and Massachusetts.²⁰ Using 2003 Oregon and Washington BRFSS data, Dilley et al. reported that 3% of those surveyed reported “do not know” or refused to answer sexual orientation questions.²¹ Dilley et al. reported refusal rates as low as 1.2% to 1.6% in combined 2003–2006 Washington State BRFSS data.¹⁹ In combined 2001–2006 Massachusetts BRFSS data, 3.6% of respondents refused to answer questions on sexual orientation.²² We identified no published research that reported refusal rates for sexual orientation questions by year that showed possible trends or compared refusal rates for sexual orientation questions with refusal rates for other sensitive questions. Although survey researchers have established that inquiries about household income are one of the most sensitive—and therefore most often refused—questions,²³ little research has directly addressed concerns that sexual orientation questions may be too sensitive for the public in states outside of the Pacific Northwest and New England. We examined the sensitivity of questions on sexual orientation and other demographic characteristics by using results from New Mexico health surveys between 2003 and 2008.

METHODS

In 2003, the New Mexico Department of Health's Tobacco Use Prevention and Control (TUPAC) Program identified lesbian, gay, and bisexual people as a priority population in addressing tobacco-related disparities. TUPAC then successfully advocated for inclusion of the following question on the New Mexico ATS: “Do you consider yourself to be—?” Possible responses included “heterosexual or straight,” “homosexual (gay or lesbian),” “bisexual,” “other,” and “don't know/not sure,” or the respondent could refuse to answer. The 2003 ATS became the first statewide population-based health surveillance system to assess sexual orientation in New Mexico.

The question's wording was adapted from the CDC's National Health and Nutrition Examination Survey²⁴ and was placed at the end of the ATS demographics section, which was prefaced with an explanation that there might be differences based on race, ethnicity, household income, sexual orientation, and education level. In a pilot study, telephone surveyors from the New Mexico Department of Health Survey Section asked the sexual orientation question to 25 to 30 randomly selected residents, documenting in turn the questions asked by residents and their difficulty with the wording of the question.

Using the experience of the New Mexico ATS and the results from the pilot study, the 2005 and 2006 New Mexico BRFSS included a state-added question on sexual orientation. Unlike the question on the New Mexico ATS, this BRFSS question did not have a preface. Instead, the following transition statement was used in the BRFSS: “Now I'm going to ask you a question about sexual orientation.” Surveyors were trained in administering the question and were provided talking points and detailed background information for use in responding to callers who may have had concerns about the purpose of the question. Surveyors reported that some older respondents seemed confused when asked the sexual orientation question. A significantly higher percentage of adults aged 65 years and older responded “don't know” (data not shown). This question was therefore administered only to respondents aged 18 to 64 years on the 2007 and 2008 New Mexico BRFSS.

Surveys

Both the New Mexico ATS and New Mexico BRFSS are ongoing, population-based, random-digit-dialed, landline telephone surveys of English- or Spanish-speaking noninstitutionalized adults who live in a household in New Mexico. The New Mexico ATS contains questions assessing tobacco-related behaviors and attitudes. The New Mexico BRFSS has questions assessing various health characteristics, including risk factors, preventive factors, and chronic diseases. Both surveys include a demographics section containing questions on race, ethnicity, age, gender, education level, and household income. On the New Mexico ATS, sexual orientation is asked as part of the demographics section. On the New Mexico BRFSS, sexual orientation is asked in a state-added question and is thus located toward the end of the survey.

Additional in-depth documentation on the BRFSS is publicly available online (http://www.cdc.gov/brfss). The ATS uses BRFSS methodology, albeit with different content and sample size.

Analysis

For both the New Mexico ATS and the New Mexico BRFSS, responses receive 1 of 4 codes: direct response (e.g., straight, gay, lesbian, bisexual, other), missing, “refused,” or “don't know.” Missing responses indicate that the respondent did not receive the question, often because of skip patterns (e.g., surveyors did not ask respondents aged 65 years or older the sexual orientation question in the 2007 and 2008 New Mexico BRFSS) or because of early termination of the survey. Missing values were not included in the analyses. Responses coded as “refused” indicate that the respondent actively refused to provide an answer to the question. Responses coded as “don't know” indicate that the respondent did not select one of the survey responses but also did not actively refuse to provide an answer. For this study, we compared the percentages of refusals for the sexual orientation question and for 6 other demographic questions: on household income, education level, race/ethnicity, Hispanic origin, age, and body weight.

Since both the New Mexico ATS and the New Mexico BRFSS are landline telephone–based surveys, they are subject to sampling bias. To help adjust for bias, responses are weighted on each respondent's probability of selection and stratified to the gender, age, and geographic distribution of New Mexico. Both surveys require completion through the demographics section for the record to be included in the data set.

We used only data sets that assessed sexual orientation. Thus, for the New Mexico BRFSS, we used data sets for 2005 (n = 5585), 2006 (n = 6581), 2007 (n = 6606), and 2008 (n = 6227). For the New Mexico ATS, we used data sets for 2003 (n = 2503) and 2006 (n = 2551). Annual response rates ranged from 55.5% to 58.9% on the New Mexico BRFSS²⁵ and from 45.9% to 48.0% on the New Mexico ATS.²⁶ Nonoverlapping confidence intervals (CIs) indicate statistical significance at P < .05. To account for weighting, we conducted statistical analyses by using SVY commands in Stata version 10 (Stata Corp LP, College Station, TX).

RESULTS

Table 1 presents sample sizes and the weighted percentages (with 95% CIs) of respondents who refused to answer questions on sexual orientation and on selected demographic characteristics in the New Mexico ATS and the New Mexico BRFSS. The question on household income had a significantly higher rate of refusal than did questions on other demographic characteristics, including sexual orientation, for all years of the surveys. Rates of refusal for the household income question ranged from 4.23% to 6.11%, whereas refusal rates for the sexual orientation question ranged from 0.80% to 2.60% and refusal rates for other questions on demographic characteristics ranged from 0.05% to 1.37%. Refusal rates for the question on sexual orientation were significantly higher than for questions on all demographic characteristics except household income on the 2003 and 2006 New Mexico ATS. There were no significant differences in refusal rates for questions on sexual orientation and body weight in the 2005 and 2006 New Mexico BRFSS.

TABLE 1.

Number and Weighted Percentage of Respondents Who Actively Refused to Answer Questions on Sexual Orientation and Selected Demographic Characteristics: New Mexico Adult Tobacco Survey (ATS) and Behavioral Risk Factor Surveillance System (BRFSS), 2003–2008

	2003 ATS (n = 2503)		2005 BRFSS (n = 5585)		2006 ATS (n = 2551)		2006 BRFSS (n = 6581)		2007 BRFSS (n = 6606)		2008 BRFSS (n = 6227)
	No. of Refusals	Weighted % (95% CI)	No. of Refusals	Weighted % (95% CI)	No. of Refusals	Weighted % (95% CI)	No. of Refusals	Weighted % (95% CI)	No. of Refusals	Weighted % (95% CI)	No. of Refusals	Weighted % (95% CI)
Sexual orientation	52	1.85 (1.32, 2.58)	94	1.18 (0.92, 1.50)	98	2.60 (2.08, 3.25)	106	1.09 (0.86, 1.38)	42	0.80a (0.53, 1.20)	55	1.13a (0.75, 1.71)
Age	22	0.56 (0.35, 0.90)	12	0.21 (0.11, 0.41)	20	0.58 (0.35, 0.96)	39	0.49 (0.33, 0.73)	25	0.31 (0.19, 0.49)	16	0.25 (0.14, 0.46)
Hispanic origin	8	0.23 (0.10, 0.48)	16	0.28 (0.15, 0.51)	10	0.31 (0.15, 0.60)	30	0.39 (0.26, 0.59)	16	0.24 (0.14, 0.43)	20	0.44 (0.19, 0.99)
Race/Ethnicity	21	0.74 (0.46, 1.18)	33	0.54 (0.35, 0.83)	21	0.54 (0.33, 0.88)	55	0.95 (0.66, 1.40)	36	0.49 (0.34, 0.72)	35	0.70 (0.38, 1.28)
Education level	12	0.46 (0.24, 0.87)	16	0.19 (0.11, 0.34)	11	0.28 (0.15, 0.52)	15	0.14 (0.08, 0.27)	8	0.08 (0.03, 0.22)	6	0.05 (0.02, 0.13)
Household income	172	6.00 (5.05, 7.11)	345	5.15 (4.54, 5.83)	177	6.11 (5.12, 7.29)	356	5.02 (4.41, 5.71)	310	4.23 (3.67, 4.88)	322	4.33 (3.66, 5.13)
Body weight	NAb	NAb	96	1.37 (1.07, 1.75)	NAb	NAb	94	1.18 (0.88, 1.56)	67	0.91 (0.66, 1.24)	70	0.83 (0.60, 1.14)

Note. CI = confidence interval; NA = not asked. “Refusal” includes only responses coded as “refused.” Percentages are weighted for each respondent's probability of selection.

^aIncludes only adults aged 18 to 64 years (2007: n = 4840; 2008: n = 4455); adults aged 65 years and older were coded as missing and therefore not included in the denominator.

^bQuestion not asked on survey.

In the 2007 New Mexico BRFSS, the percentage of respondents who refused to answer the question on sexual orientation (0.80%; 95% CI = 0.53, 1.20) was significantly higher than the percentage who refused to answer questions on age (0.31%; 95% CI = 0.19, 0.49), Hispanic origin (0.24%; 95% CI = 0.14, 0.43), and education level (0.08%; 95% CI = 0.03, 0.22). No significant differences were found in refusal rates for questions on sexual orientation, race/ethnicity (0.49%; 95% CI = 0.34, 0.72), and body weight (0.91%; 95% CI = 0.66, 1.24) in 2007. In the 2008 New Mexico BRFSS, refusal rates for questions on sexual orientation (1.13%; 95% CI = 0.75, 1.71) were significantly higher than for questions on age (0.25%; 95% CI = 0.14, 0.46) and education level (0.05%; 95% CI = 0.02, 0.13). There were no significant differences between refusal rates for questions on sexual orientation, Hispanic origin (0.44%; 95% CI = 0.19, 0.99), race/ethnicity (0.70%; 95% CI = 0.38, 1.28), and body weight (0.83%; 95% CI = 0.60, 1.14) in 2008.

DISCUSSION

In New Mexico from 2003–2008, refusal rates for a question on sexual orientation were low (0.80%–2.60%). These rates were notably lower than refusal rates for a question on household income (4.23%–6.11%, compared with a refusal rate as high as 12% in the combined Washington–Oregon 2003 BRFSS²¹) and similar to refusal rates for a question on body weight. The lower refusal rates for sexual orientation questions are similar to findings from Massachusetts's 2001–2006 BRFSS (3.6%),²² Washington's 2003–2006 BRFSS (1.2%–1.6%),¹⁹ and combined Washington–Oregon BRFSS data from 2003 (3%).²¹

Our findings in New Mexico corroborate findings from New England and the Pacific Northwest, and our findings do so across 4 years of survey data from 2 separate statewide surveys. Additionally, because New Mexico is a rural border state with a population that is “minority majority” (i.e., minority groups outnumber Whites), our findings suggest that states demographically quite different from states in New England and the Pacific Northwest can successfully improve surveillance systems and maintain data quality by adding a sexual orientation question.

To continue improving their quality of data collection, the 2009 New Mexico ATS and New Mexico BRFSS used a revised sexual orientation question that assesses both sexual orientation and gender identity: “Do you consider yourself to be one or more of the following—? Possible responses include “straight,” “gay or lesbian,” “bisexual,” “transgender,” “other,” “don't know/not sure,” or the respondent could refuse to answer. The revised question, which was developed with cognitive interviewing techniques (i.e., in-depth interviews with survey respondents to identify thought processes behind how questions were answered), more accurately captures those constructs among adults of all ages.^27,28

The availability of sexual orientation data is crucial for interventions against existing health disparities.¹¹ States should recognize that, as refusal rates are similar for questions on sexual orientation and body weight, sexual orientation is not too sensitive a subject for use in public surveys. Additionally, with only a quarter of states choosing to collect data on sexual orientation,¹³ the CDC should revisit earlier decisions to make such questions optional on the BRFSS.