Have Mischievous Responders Misidentified Sexual Minority Youth Disparities in the National Longitudinal Study of Adolescent to Adult Health?

Abstract

The National Longitudinal Study of Adolescent to Adult Health (Add Health) has been instrumental in identifying sexual minority youth health disparities. Recent commentary suggested that some Wave 1 youth responders, especially males, intentionally mismarked same-sex attraction and, as a result, published reports of health disparities from these data may be suspect. We use two recently developed approaches to identify “jokesters” and mischievous responding and apply them to the Add Health data. First, we show that Wave 1 same-sex attracted youth, including those who later reported completely heterosexual identities in adulthood, were no more likely than different-sex attracted youth and consistently heterosexual participants to be “jokesters.” Second, after accounting for mischievous responses, we replicated six previously established disparities: depressive symptoms, suicidal ideation and behaviors, alcohol use, cocaine use, parental satisfaction, and school connectedness. Accounting for mischievousness resulted in the elimination of one observed disparity between heterosexual and sexual minority youth: suicidal ideation for males who reported romantic attraction to both sexes. Results also showed that accounting for mischievous responding may underestimate disparities for sexual minority youth, particularly females. Overall, results presented here support previous studies that identified health disparities among sexual minority youth using these data.

Introduction

We have a better understanding of sexual minority health disparities among youth today due to the inclusion of sexual attraction, behavior, and/or identity measures in large-scale, regional, national, and population-based data sources. The National Longitudinal Study of Adolescent to Adult Health (Add Health) was the first U.S. nation-wide longitudinal sample of youth to assess same-sex romantic attraction and the gender of romantic and sexual partners. Add Health has been instrumental in contributing to the understanding of youth sexual development (Dennison & Russell, 2005; Tolman & McClelland, 2011) and helped launch the current body of science that documents sexual minority youth health disparities for depression (Consolacion, Russell, & Sue, 2004; Udry & Chantala, 2002), suicidal ideation and behavior (Russell & Joyner, 2001; Russell & Toomey, 2012), alcohol and illicit substance use (Marshal et al., 2009; Needham, 2012; Russell, Driscoll, & Truong, 2002), and deficits in protective factors (i.e., family closeness, school connectedness, and peer support) (Needham & Austin, 2010; Russell & Toomey, 2013; Ueno, 2005).

Self-reported questionnaire data are advantageous when assessing sensitive topics among adolescents (Badgett, 2009; Turner et al., 1998), yet there has been long-standing concern that youth might provide (willfully or inadvertently) inaccurate responses that can bias parameter estimates, effects, and conclusions drawn from these analyses (Cornell, Klein, Konold, & Huang., 2012; Fan et al., 2006; Robinson & Espelage, 2011; Robinson-Cimpian, 2014). Although scholars who have studied the validity of self-reported data conclude that youth self-reports are generally reliable across domains (Brener, Billy, & Grady, 2003; Newcomer & Udry, 1988), researchers understandably also raise concerns about “mischievousness” or that some youth may intentionally respond to surveys in “funny” or outlandish ways (Fan et al., 2006; Robinson-Cimpian, 2014; Savin-Williams & Joyner, 2014a). For instance, after dropping youth who self-report not telling the truth in a school-based self-administered questionnaire (SAQ), providing false answers, or not paying attention, Cornell et al. (2012) noted substantially lower prevalence of key risk behaviors such as bullying, substance use, delinquency, and suicidal ideation. Because these responders tend to report extreme answers to items that reflect key outcomes of interest (i.e., alcohol and drug use, mental health, academics, sexual activity), their inclusion in analyses may inaccurately magnify differences, especially for small subgroups (e.g., sexual and gender minorities, adoptees, those with disabilities) or low-prevalence behaviors (e.g., drug use, suicidality) (Fan et al., 2006; Robinson & Espelage, 2011; Robinson-Cimpian, 2014). Based on such evidence, other studies have illustrated the benefits of including screening questions to identify and account for untruthful responses that may bias findings (Cornell et al., 2012; Fan et al., 2006; Robinson & Espelage, 2011; Robinson-Cimpian, 2014).

Although the in-home Add Health questionnaire was not self-administered, concerns regarding the veracity of youth reports of same-sex attraction in Wave 1 of Add Health—and therefore the veracity of results elucidating health disparities—have been the subject of recent debates (see Katz-Wise, Calzo, Li, & Pollitt, 2014; Li, Katz-Wise, & Calzo, 2014; Savin-Williams & Joyner, 2014a; Savin-Williams & Joyner, 2014b). Despite the debate, there has been no empirical investigation to determine the validity of claims that Wave 1 of the Add Health data contained a group of responders who mischievously reported same-sex attraction and other deleterious outcomes that exaggerated the observed differences between different- and same-sex attracted youth. The current study provides an empirical investigation of this argument by (1) identifying markers of inaccurate or mischievous responding within the Add Health data and (2) assessing whether accounting for mischievous responders eliminates or dampens previously identified disparities between different- and same-sex attracted youth. First, reasons to doubt the veracity of responses to Add Health questions about same-sex romantic attraction are reviewed, including discussion of two empirically-supported approaches to identify inaccurate responders, jokesters, or mischievous responders.

Reasons to Doubt Add Health Wave 1 Responses

Two arguments have been used to question Wave 1 responses regarding same-sex romantic attraction (Savin-Williams & Joyner, 2014a): the prevalence and the stability of same-sex attraction and identity.

Prevalence

Compared to the 1% of youth in the Growing Up Today Study (GUTS) (Austin et al., 2004) who report being gay/lesbian or bisexual, adolescent responders in Wave 1 of Add Health demonstrate an “extraordinar[il]y high prevalence of nonheterosexual middle and high school adolescents” (Savin-Willaims & Joyner, 2014a, p. 414) with 5.0% of adolescent females and 7.5% of adolescent males reporting same- or both-sex romantic attractions. Similar to Add Health, GUTS collected data from approximately 10,000 12-to-17 year olds only four years after Wave 1 of the Add Health study. Initially, this comparison seems stark; however, generally “the number of youth with same-sex attraction far exceed those who engage in same-sex behavior or who identify as gay” (Savin-Williams & Cohen, 2007, p. 31) and studies with youth (Garofalo, Wolf, Kessel, Palfrey, & Durant, 1998; Katz-Wise, 2014; Matthews, Blosnich, Farmer, & Adams, 2014; Rosario, Schrimshaw, Hunter, & Braun, 2006) and adult (Bostwick, Boyd, Hughes, & McCabe, 2010; Chandra, Mosher, Copen, & Sionean, 2011; Copen, Changdra, Febo-Vazquez, 2016; Diamond, 2008; Gates, 2010; Thomeer & Reczek, 2016; Xu, Sternberg, & Markowitz, 2010a; 2010b) samples have consistently shown concurrent and longitudinal within-person discrepancies between sexual attraction, behavior, and identity.

In one study of Canadian youth, 12% of high school students reported nonexclusive heterosexuality on at least one dimension with over 3% reporting a lesbian/gay or bisexual identity, 9% same-sex attraction, and 4% sexual behavior with a same-sex partner; only 15% reported congruently across all three dimensions (Igartua, Thombs, Burgos, & Montoro, 2009). Li et al. (2014) revisited results from the GUTS study and found that including adolescents who claim a “mostly heterosexual” label with youth who report being gay/lesbian or bisexual increases the presence of sexual minority youth in the GUTS sample to 6%. Although no other nationally representative samples collected during that period are available, results from the 1993 Massachusetts Youth Risk Behavior Survey (YRBS) showed that 6.4% of sexually experienced students reported same-sex contact and the 1995 Vermont YRBS indicated that 8.7% of adolescent males reported sexual intercourse with a same-sex partner (DuRant, Krowchuk, & Sinal, 1998). Earlier results from the Minnesota Adolescent Health Survey collected in 1987 indicated that 4.5% of 7th to 12th graders reported same-sex attraction with 6.4% of students indicating primarily same-sex attractions by the age of 18 (Remafedi, Resnick, Blum, & Harris, 1992).

More recently, in a special issue of the American Journal of Public Health, a series of articles used pooled data from over 10 jurisdictions of the 2005 and 2007 Youth Risk Behavior Surveillance Survey (YRBSS) (see Mustanski et al., 2014a), a probability-based, representative, biennial, cross-sectional survey of high-school students administered by the Centers for Disease Control and Prevention. Results replicated the prevalence of sexual minority youth as comparable to that in Wave 1 of Add Health, with 6.8% of students self-identifying as gay/lesbian (1.2%), bisexual (3.4%), or unsure (2.2%; Mustanski et al., 2014a). Although reports of attraction were measured in only one jurisdiction, 4.1% and 4.6% of youth indicated exclusive same-sex or bisexual attraction, respectively (Mustanski, Van Wagenen, Birkett, Eyster, & Corliss, 2014b).

In sum, studies dating back to the late 1980s are consistent with Add Health in the prevalence of same-sex attracted (Mustanski et al., 2014a; Remafedi et al., 1992), same-sex partnered (DuRant et al., 1998), and/or LGB-identified youth (Austin et a.., 2004; Mustanski et al., 2014b) across several population-based surveys.

Stability

A second argument for doubting the veracity of youth responses was the number of Wave 1 same- and both-sex attracted youth who reported heterosexual identities at later waves (Savin-Williams & Joyner, 2014a; Savin-Williams & Ream, 2007). Udry and Chantala (2005) noted this instability in same-sex attracted males: Of the 69 males in their sample who reported exclusive same-sex romantic attractions, only 11% reported exclusive same-sex attraction at Wave 2, just one year later. Similarly, when Savin-Williams and Ream assessed the stability of reports across the transition to adulthood, their findings showed a preponderance of inconsistencies between Wave 1 romantic attraction and Wave 3 sexual identity for females and, to a larger extent, males: Over half of same- or both-sex attracted females reported exclusive different-sex attraction from Wave 1 to Wave 3 (ages 18-24) with 72% of same- and 82% of both-sex attracted males reporting this pattern.

Extending this comparison to Add Health's Wave 4 responses, 80% and 60% of Wave 1 same-sex attracted males and females, respectively, identified as exclusively heterosexual at Wave 4 (ages 24-32; Savin-Williams & Joyner, 2014a). To help explain these trends, Savin-William and Joyner consider three possibilities related to these transitions: confusion regarding romantic attraction questions, developmental change, and the presence of mischievous responders.

Confusion Regarding Romantic Attraction. One possibility is that Wave 1 youth did not understand questions about romantic attraction. Specifically, males and females who reported same-sex attraction at Wave 1 and heterosexual identities at Wave 4 reported lower standardized vocabulary test scores, and inconsistent males demonstrated lower English class grade point averages and college attendance compared to those who reported consistent heterosexual attraction and identities from Wave 1 to Wave 4 (Savin-Williams & Joyner, 2014a). Further, males who consistently reported nonheterosexual attractions and identities from Waves 1 to 4 had higher scores on these indicators than consistently heterosexual peers. Based on these results, it seems plausible that youth who reported same-sex attractions at Wave 1 and heterosexual identities at Wave 4, especially males, were mistaken in their interpretation of questions assessing romantic attractions and, as a result, the number of Wave 1 same-sex attracted youth was inflated. Savin-Williams and Joyner also noted that one-third of Wave 4 same-sex attracted males simultaneously reported heterosexual identities; they argued that “romantic” attractions may be difficult for some to understand compared to questions of “sexual” attractions.

There are alternate explanations for these observed patterns. First, Li et al. (2014) noted disparities in education and achievement for sexual minority youth: Experiences of discrimination, victimization, and bullying in school may contribute to compromised school attendance and academic achievement (Birkett, Russell, & Corliss, 2014; Watson & Russell, 2014). Second, the finding that 30% of Wave 4 male participants reported same-sex attractions but heterosexual identities (Savin-Williams & Joyner, 2014a; Savin-Williams & Ream, 2007) is not inconsistent with other large, national sample datasets measuring concurrent reports of romantic attraction, behavior, and identity: Individuals are more likely to report same-sex attraction than same-sex behaviors, with the lowest proportional indicator for sexual minority status being sexual identity (Bostwick et al., 2010; Chandra et al., 2011; Copen et al., 2016; Gates, 2010; Institute of Medicine, 2011).

Developmental Change. A second explanation is that instability in reports of attraction and identity across the transition to adulthood could be due to youth maturation and normative sexual development. Focusing predominantly on studies of adults, Savin-Williams and Joyner (2014a) stated that the available literature on sexual fluidity would suggest that the females are more likely to demonstrate inconsistent attraction and identity patterns compared to males (Diamond, 2008; Dickson, van Roode, Cameron, & Paul, 2013; Kinnish, Strassberg, & Turner, 2005; Mock & Eibach, 2012; Savin-Williams, Joyner, & Reiger, 2012), and that these findings run counter to the sexual (in)stability observed in the Add Health sample (see Fish & Pasley, 2015; Needham, 2012; Savin-Williams & Ream, 2007), where same-sex and especially both-sex attracted males were those most likely to be inconsistent with these reports over time (Savin-Williams & Ream, 2007; Udry & Chantala, 2005). However, empirical studies of youth indicate little to no significant differences in sexual attraction (Li & Hines, 2016) and identity fluidity between males and females during adolescence (Katz-Wise, 2014; Ott, Corliss, Wypij, Rosario, & Austin, 2011). Findings on the stability of sexual minority youth sexual orientation and labeling show that adolescent males are more likely to report inconsistencies in labeling themselves as gay/lesbian compared to females (Rosario et al., 2006), especially males under the age of 14 (Ott et al., 2011) and that young same-sex attracted males and females predominantly report non-exclusive attractions (Li & Hines, 2016). One prospective study indicated that over 70% of “unsure” male and female youth ages 12-17 reported completely heterosexual identities in young adulthood (Ott et al., 2011). Although advertised as a “sexual identity study,” another study found that 53% of heterosexual young adult males retrospectively reported having questioned their sexual orientation at some point in their lives (Morgan, Steiner, & Thompson, 2010). These inconsistencies, along with the general lack of studies on the development and (in)stability of sexual attraction, behaviors, and identities in adolescence, leave more questions than definitive answers regarding the propensity of male versus female youth to report inconsistency in sexual attraction and identity from adolescence to adulthood.

Another consideration of developmental change is the influence of context: Youth may encounter social structures that encourage them to conceal or “hide” their sexuality or to avoid adopting a sexual minority identity despite same-sex attractions and behavior (Meyer, 2003; Russell & Seif, 2010). These difference could reflect homophobia, heteronormative influences, or monosexist and anti-bisexual attitudes (Eliason, 2000; Israel & Mohr, 2004; Mulick & Wright, 2002). Indeed, researchers that consider differences in reported attraction and identity over time have concluded that youth may stray away from sexual minority identities due to “perceived stereotypes of gays or political inclinations” (Savin-Williams & Ream, 2007). In fact, those males who reported Wave 1 same-sex attraction and Wave 4 heterosexual sexual identity were similar to consistent heterosexuals in conservative political attitudes and religiosity, gross measures that might indicate pressure for traditional sexual and gender norms and corresponding identities (Li et al., 2014; Savin-Williams & Joyner, 2014a).

A Case for Mischief

The final and more provocative explanation is youth mischief: Adolescent participants may intentionally answer questions of romantic attraction in ways to dupe or “mess with” researchers. Miller et al. (2001) and Fan et al. (2006) observed a number of inconsistent responses in Wave 1 of the Add Health data on items related to adoption, nativity, and the use of an artificial limb. After triangulating responses across in-school, in-home, and parent questionnaires, Fan et al. identified youth and parents who responded inconsistently to one or more of these items. Specifically, 418 of Wave 1 youth participants and/or their parents reported inconsistently on at least one of the three criteria (i.e., adoption status [n = 88 were inconsistent], nativity status [n = 176 were inconsistent], or the use of an artificial limb [n = 248 were inconsistent]); of these, 336 individuals were incongruent on one of the three criteria, 70 gave inaccurate responses across surveys for two items, and 12 showed a lack of fidelity across all three items. Those responding inconsistently to a single criterion item were labeled “inaccurate responders” (n = 336; 1.62% of the total Wave 1 sample) since these youth may have accidently provided incorrect responses to one of the items across the three questionnaire formats; those with two or three discrepancies were dubbed “jokesters” (n = 82; .39%). Follow-up analyses indicated that those who responded inconsistently across items of interest were more likely to inconsistently report their biological sex, age, and race/ethnicity across surveys compared to those with consistent reports across all three surveys. Furthermore, disparities in academic outcomes, psychological symptoms, and deviant behavior between the majority and small subgroups of interest (i.e., adoptees) were exaggerated when the “jokesters” were included in analyses. These findings identify youth who falsely report in-group status and extreme responses to questions of risky behavior in the Add Health Wave 1 survey.

Using different data, other studies have demonstrated that participants who, in theory, willfully respond falsely to risk behavior items may also be more likely to report being a sexual or gender minority, which may subsequently exaggerate disparities (Robinson & Espelage, 2011, 2012; Robinson-Cimpaim, 2014). Robinson and Espelage examined educational and mental health disparities and found youth who self-labeled as LGBTQ responded to low-frequency, high-risk items in greater proportions than their heterosexual peers. Excluding those individuals who were deemed “mischievous” attenuated but did not eliminate disparities between heterosexual and sexual minority youth. Other analyses using the same data (the Dane County Youth Assessment) show reduced disparities between same-sex and different-sex attracted youth once mischievous responders were taken into account (Robinson & Espelage, 2011, 2012; Robinson, Espelage, & Rivers, 2013; Robinson-Cimpian; 2014).

As part of their investigation into the instability of reported sexuality from adolescence to adulthood, Savin-Williams and Joyner (2014a) found that Wave 1 same-sex attracted males who reported a Wave 4 heterosexual identity were more likely to report delinquency, unexcused absences from school, answering questions dishonestly, and were most likely to be rated as bored or impatient by the in-home interviewer. Such findings led to the conclusion that “boys who emerged from a gay or bisexual adolescence to become a heterosexual young adult were, by-and-large, heterosexual adolescents who were either confused or jokesters who decided, for reasons [researchers] were not able to detect, to dishonestly report their sexuality” (Savin-Williams & Joyner, 2014a, p. 420). Savin-Williams and Joyner also concluded that the inclusion of “jokesters” in studies that have used Add Health data misrepresent sexual minority youth experiences and exaggerate their distress during adolescence (Savin-Williams, 2005; Savin-Williams & Joyner, 2014b).

The Current Study

We used two approaches to assess the degree of inaccurate and mischievous responding in Wave 1 of Add Health and whether such responses may have exaggerated sexual minority disparities. In our first approach, we replicated the original item triangulation of in-school, in-home, and parental questionnaires developed by Fan et al. (2006) to identify inaccurate responders and jokesters. We then used these categories to assess whether (1) those who reported same-sex attraction at Wave 1 were more likely to be inaccurate responders or jokesters using all available youth from Wave 1, and (2) same-sex attracted youth at Wave 1 who reported exclusively heterosexual identity at Wave 4 were more likely to be inaccurate responders or jokesters using a replication of the sample presented in Savin-Williams and Joiner (2014a).

For our second approach, we calculated an index score of mischievous responses following the method outlined by Robinson-Cimpian (2014). We then used that score to assess whether prior reports of disparities based on same-sex romantic attraction could be explained in part by mischievous responding. We calculated a baseline disparity for six different previously established disparities between heterosexual and same-sex attracted youth from the Add Health Wave 1 data: depression (Udry & Chantala, 2002; Ueno, 2005), suicidal ideation and behavior (Russell & Joyner, 2001; Russell & Toomey, 2012), alcohol and cocaine use (Marshal et al., 2009; Needham, 2012, Russell, Driscoll, & Truong, 2002), family closeness (Needham & Austin, 2010), and school connectedness (Russell & Toomey, 2013). We then used the mischievous index score as a covariate to assess whether accounting for potential mischievousness diminished group differences.

Method

Participants

Add Health is a nationally representative, longitudinal study that collected four waves of social, economic, psychological, and physical data from a cohort of youth starting in adolescence and into their late 20s and early 30s. In the first of four waves conducted during 1994-1995, approximately 90,000 7th-12th graders from 80 high schools that represented the U.S. population at that time completed an in-school paper-and-pencil SAQ. From this sample, 20,745 were selected to complete an in-home interviewer-assisted questionnaire along with a target parent (or guardian; approximately an 85% response rate from adults). For the in-home interview, youth were surveyed by an interviewer for general questions and via an audio computer-assisted self-interview (audio-CASI) format for sensitive topics including romantic attractions: participants listen to prerecorded interview questions through a headset and entered their responses on a laptop computer. In contrast to traditional SAQs, the audio-CASI technique allows survey participants privacy when answering sensitive or personal questions, especially those related to sexual behavior (Paschall, Orstein, & Flewelling, 2001; Turner et al., 1998).

Since Wave 1, three additional waves of data have been conducted. Wave 2 was collected one year following Wave 1 and included 14,738 of the original participants. Wave 3 occurred 5-6 years later, when participants (n = 15,197 from Wave 1) were young adults (average ages 18-24), and Wave 4 was conducted in 2007-2008 when participants were approximately ages 24 to 32 (n = 15,701 of the original sample).

Approach 1: Identifying Inaccurate Responders and Jokesters

Following the method of Fan et al. (2006), we first tested whether: (1) Wave 1 same-sex attracted youth were more likely to be inaccurate responders and jokesters, and (2) those who reported Wave 1 same-sex attraction and Wave 4 heterosexuality were more likely to be inaccurate responders and jokesters.

First, we used all available participants in Wave 1 who were assigned a survey weight and provided a valid response to romantic attraction items (n = 18,682) to assess the distribution of jokesters (n = 32, .17%) and inaccurate responders (n = 234, 1.24%) across Wave 1 reports of attraction.

We then replicated a sample of participants identical to those examined in Savin-Williams and Joyner (2014a): It was restricted to those who were assigned sample weights at both Wave 1 and Wave 4 (N = 14,800). Excluded were those who had inconsistently reported biological sex across Waves 1 and 4 (n = 14), failed to provide responses to sexual orientation identity questions at Wave 4 (n = 63), and did not provide a valid response to romantic attraction questions at Wave 1 (n = 147) or Wave 4 (n = 23). The sample was further limited to the groups used for comparison by Savin-Williams and Joyner (2014a): consistent heterosexual (n = 10,544), Wave 1 same-sex attracted to Wave 4 heterosexual (n = 613), and consistent nonheterosexual from Wave 1 to Wave 4 participants (n = 244), bringing the final subsample to 11,401.

Approach 2: Identifying Mischievous Responders

For our second approach, we followed the method by Robinson-Cimpian (2014) to calculate a mischievous index score. Heterosexual and sexual minority youth disparities were then assessed for six previously identified risk and protective factors established using data from Wave 1 of the Add Health study. Disparities were first estimated in unconditional models and then differences based on romantic attraction were reassessed while controlling for participants' propensity for mischievous responding; if a sizeable proportion of Wave 1 youth responded in mischievous ways when reporting romantic attraction, accounting for this behavior should reduce bias in estimates (Robinson-Cimpian, 2014).

All available participants in Wave 1 who were assigned a survey weight and provided a valid response to romantic attraction items (n = 18,682) were included (sample sizes varied across outcomes).

Measures

Approach 1: Identifying Inaccurate Responders and Jokesters

Same-sex Romantic Attraction and Sexual Identity. In Wave 1, participants were asked two questions about romantic attraction: “Have you ever had a romantic attraction to a female (male)?” Responses were yes = 1 and no = 0. Items were recoded to best reflect previous definitions of same-sex attracted youth: a 3-category item reflecting different-sex attraction = 0, no attraction = 1, and (any) same-sex attraction = 2 and a 4-category item representing youth who had different-sex attraction = 0, no attraction = 1, (exclusive) same-sex attraction = 2, or both-sex attraction = 3. Sexual identity was measured at Wave 4 by asking participants to “Please choose the description that best fits how you think about yourself” with response options of: 100% heterosexual (straight), mostly heterosexual (straight), but somewhat attracted to people of your own sex, bisexual—that is, attracted to men and women equally, mostly homosexual (gay), but somewhat attracted to people of the opposite sex, 100% homosexual (gay).

Inaccurate Responders and Jokesters. We replicated categories of biased responding presented in Fan et al. (2006) based on the fidelity of items related to adoption status, nativity status, and the use of an artificial limb across two surveys from youth participants (the in-school and in-home surveys) and/or parental reports (for more detail, see Miller et al., 2001). Participants who had reliability across all three items and surveys were true responders = 0, those who were incongruent on one criteria item were inaccurate responders = 1, and those who had discrepancies on two or more items across surveys were labeled jokesters = 2.

Approach 2: Identifying Mischievous Responders

The same measure of romantic attraction from the approach described above was used. The following outcomes were all measured during Wave 1.

Depressive Symptoms. Depressive symptoms were measured using a 20-item scale (see Crockett, Randall, Shen, Russell, & Driscoll, 2009) from the Center for Epidemiologic Studies-Depression inventory (CES-D; Radloff, 1977) asking participants to indicate how often they felt a certain way in the previous week. Item examples included “You were bothered by things that usually don't bother you” and “You felt sad”; responses ranged from never or rarely = 0 to most of the time or all of the time = 3. Items were summed so higher scores reflect more depressive symptomology (α = .84).

Suicidal Ideation and Suicidal Attempt(s). Suicidal thoughts and behaviors were measured with two dichotomous items asking participants if they had seriously thought about suicide in the previous 12 months (no = 0, yes = 1) and the number of times suicide was attempted in the previous 12 months (recoded as never = 0, 1 or more times = 1; Russell & Joyner, 2001).

Alcohol Use. A composite alcohol use measure was constructed from items asking participants how often in the previous 12 months they (1) drank alcohol, (2) drank five or more drinks in a row, and (3) “got drunk or ‘very, very high’ on alcohol?” Response options were from never = 0 to every day or almost every day = 6. Items were summed and averaged (α = .91; Fish & Pasley, 2015).

Cocaine Use. Cocaine use was measured by asking participants, “During the past 30 days, how many times did you use cocaine?” Due to the relatively low prevalence of cocaine use in the previous month, original responses, ranging from never = 0 to 30 or more times = 9, were recoded to reflect no use = 0 and 1 or more times = 1.

Parental Closeness. Parental closeness was measured using three items for mothers and fathers (or mother and father like figures; Ream & Savin-Williams, 2005; Russell & Toomey, 2013). Items include: “How close do you feel to your fa/mother” (not at all = 1 to very much = 5); “You are satisfied with the way your fa/mother and you communicate with each other” (strongly disagree = 1 to strongly agree = 5); and “Overall, you are satisfied with your relationship with your fa/mother?” (strongly disagree = 1 to strongly agree = 5). Items were summed and averaged to account for one or two parent homes where higher scores reflect closer parental relationships (α = .85 for mothers; α = .88 for fathers; α = .85 for combined scores).

School Connectedness. Participant stated their agreement with five items reflecting school connectedness (Russell & Toomey, 2013): “You feel close to people at your school,” “You feel like you are part of your school,” “You are happy to be at your school,” “The teachers at your school treat students fairly,” and “You feel safe at your school.” Responses ranged from strongly disagree = 1 to strongly agree = 5. Items were reverse-scored, summed, and averaged so higher scores reflect higher levels of school connectedness (α = .76).

Analytic Strategy

Approach 1: Identifying Inaccurate Responders and Jokesters

We used chi-square analysis adjusted for sample weights to assess the prevalence of inaccurate responders and jokesters across reports of Wave 1 romantic attraction and the correspondence between Wave 1 same-sex romantic attraction and Wave 4 nonheterosexual identity. Analyses were conducted for males and females separately.

Approach 2: Identifying Mischievous Responders

We constructed an index score to identify potentially mischievous responders following the method set out by Robinson-Cimpian (2014). This procedure included four stages. First, we identified low-frequency response items from the Add Health in-home questionnaire that were unrelated to sexual minority group membership or the outcomes of interest. Second, a probability-based measure was computed by calculating each individual's prevalence of the 10 low-frequency responses and then weighted by the relative occurrence within the sample; this weighting system assigned higher penalties for those who endorsed low-frequency responses that were less prevalent in the overall sample. Third, we assessed the distribution of low-frequency responses across groups of interest and the change in the estimated disparity across scores of mischief. Finally, the index score was included as a covariate when estimating disparities to account for the influence of mischievous responding on outcomes.

To identify mischievous responders, we used 10 low-frequency response items from Wave 1 of Add Health: Six of the 10 mirror those used in Robinson-Cimpian (2014; noted in Table 1). The remaining four low-frequency items were selected because they have been used by other researchers to discriminate mischievous responders both within the Add Health dataset (Fan et al., 2006; Savin-Williams & Joyner, 2014a) and in other data comparing heterosexual and sexual minority youth (Robinson & Espelage, 2011). Specifically, we included an item suggested by Robinson and Espelage (2012) regarding average hours of sleep per night (Item 7). The eighth item indicated whether participants were labeled as inaccurate responders or jokesters based on the approach by Fan et al. (2006; see Approach 1). The two final screening items were from Savin-Williams and Joyner's (2014a) analyses: youth reports of answering truthfully (Item 9) and interviewer reports of participant boredom/impatience (Item 10). For those items replicated from Robinson-Cimpian, we combined items on pregnancy and children so that anyone who reported two or more pregnancies or two or more children were flagged (Item 4). We also took this approach with youth reports of having been shot or acting as the perpetrator in a shooting or stabbing (2 or more times in the previous 12 months; Item 6).

Table 1.

Frequencies of Low-Frequency (Mischievous) Response Items in Wave 1 of Add Health

Low-Frequency Response Items		Males	Females	Total
		%	%	%d
1.	Provided a weight in top or bottom 2.5%a	3.85	2.28	6.13
2.	Provided a height in top or bottom 2.5%a	5.69	2.42	8.11
3.	When was the last time you visited a dentist? (2+ years)a	4.12	3.71	7.84
4.	How many times have you been pregnant or have gotten a girl pregnant? or How many children do you have? (2+)a	1.35	1.11	2.46
5.	In the past month, how many days have you carried a weapon to school? (6+ days)a	1.37	0.40	1.77
6.	During the past 12 months: how often did someone shoot you or How often did you shoot or stab someone (2+ times)a	0.56	0.12	0.68
7.	How many hours of sleep do you usually get? (11+ hours)a	1.68	1.58	3.27
8.	Fan et al. (2006) identification as inaccurate responder or jokesterc	0.62	0.34	0.96
9.	Answered questions “not honestly at all”b	1.96	.73	2.69
10.	Assessed as bored or impatient by interviewerb	7.68	4.60	12.28

^aDenotes items replicated from Robinson-Cimpian (2014) or Robinson and Espelage (2011).

^bIndicators of inaccurate and mischievous responding established in the Fan et al. (2006) analyses.

^cIndicators of mischief from Savin-Williams and Joyner (2014a).

^dFull sample endorsement of low frequency items that reflect top or bottom 2.5% of sample do not add to 5% due to rounding error in cutoff values.

We calculated disparities (unconditional models, independent of covariates) between groups based on Wave 1 reports of romantic attraction using survey adjusted ordinary least squares (OLS) regression for continuous outcomes (i.e., depressive symptoms, alcohol use, parental closeness, and school connectedness) and logistic regression for binary outcomes (i.e., 30-day use of cocaine and suicidal ideation and attempts). We then tested these same disparities using the probability-based screener-index score of biased responding as a covariate. Conducted in Stata 14 (StataCorp, 2015), analyses were estimated separately for males and females, and for 3- and 4-category romantic attraction measures to best replicate previous study designs.

Results

Approach 1: Identifying Inaccurate Responders and Jokesters

There were no significant associations between Wave 1 attraction and the prevalence of inaccurate responders or jokesters among males, ${\chi }_w^2\left(6,N=9,391\right)=1.39$, p = .229, or females, ${\chi }_w^2\left(6,N=9,775\right)=2.06$, p = .080. We were concerned about the validity of the chi-square tests because of the presence of unpopulated cells: Follow-up 2 × 2 comparisons with inaccurate responders and jokesters combined into a single category showed consistent results for males, ${\chi }_w^2\left(3,N=9,391\right)=1.73$, p = .163, but that females reporting no romantic attractions were more likely to be in the combined inaccurate responders and jokesters category, ${\chi }_w^2\left(3,N=9,775\right)=3.53$, p < .05; no same- or both-sex attracted female youth met the criteria for jokesters. Analysis of the 3-category attraction variable for males and females replicated these findings. In all, 20 same- or both-sex attracted youth were identified as inaccurate responders or jokesters.

Participants who reported Wave 1 same-sex attraction and Wave 4 heterosexual identities were no more likely to fall into inaccurate responder or jokester categories with comparisons conducted with these categories separately, ${\chi }_w^2\left(4,N=5,656\right)=.204$, p = .912 for males and ${\chi }_w^2\left(4,N=5,745\right)=.231$, p = .876 for females, or combined into one category, ${\chi }_w^2\left(2,N=5,656\right)=.324$, p = .679 and ${\chi }_w^2\left(2,N=5,745\right)=1.19$, p = .678 for males and females, respectively. Ten participants who reported same-sex attraction at Wave 1 and a heterosexual identity at Wave 4 were inaccurate responders or jokesters.

Approach 2: Identifying Mischievous Responders

All analyses were conducted using standardized outcomes for continuous variables so that the change in the effect between the unconditional and adjusted models were interpretable. Therefore, OLS regression coefficients represent the change in the SD of the standardized outcome for each group relative to different-sex attracted youth. Tables display this format except for the columns for group mean which are scaled to the original variable. Means, SD, and the prevalence of outcomes by Wave 1 sexual attraction are shown in Table 2.

Table 2. Weighted Means, SD, and Prevalence of Outcomes Variables by Male and Female 4- and 3-Cateogry Romantic Attraction.

	Depressive Symptomology		Suicidal Ideation	Suicidal Attempt(s)	Alcohol Use		Cocaine Use	Parental Closeness		School Connectedness
	M	SD	% Yes	% Yes	M	SD	% Yes	M	SD	M	SD
Male 4-Category
Different-sex	.78	.26	10.56	2.09	.98	1.42	1.38	4.22	.70	3.71	.77
No attraction	.72	.28	7.40	2.53	.40	1.02	1.57	4.31	.58	3.79	.72
Same-sex (only)	.83	.28	21.76	8.97	1.20	1.80	7.88	4.10	.67	3.70	.62
Both-sex	.83	.29	15.67	5.29	1.22	1.57	4.82	4.16	.74	3.65	.75
Male 3-Category
Different-sex	.78	.26	10.56	2.09	.98	1.42	14.38	4.22	.70	3.71	.77
No attraction	.72	.27	7.40	2.53	.40	1.03	1.57	4.31	.58	3.79	.72
Same-sex (any)	.83	.29	16.41	5.74	1.21	1.60	5.18	4.16	.73	3.65	.73
Female 4-Category
Different-sex	.87	.30	16.33	5.45	.75	1.14	.73	4.10	.81	3.68	.77
No attraction	.79	.31	7.14	2.60	.27	.73	.35	4.31	.67	3.83	.72
Same-sex (only)	1.00	.39	30.03	12.62	1.28	1.63	2.54	3.71	.94	3.46	.77
Both-sex	.93	.32	29.74	12.94	1.22	1.29	2.81	3.85	.86	3.42	.84
Female 3-Category
Different-sex	.87	.30	16.33	5.45	.75	1.14	.73	4.10	.81	3.68	.77
No attraction	.79	.31	7.14	2.60	.27	.73	.35	4.31	.67	3.83	.72
Same-sex (any)	.95	.34	29.83	12.86	1.24	1.39	2.74	3.82	.89	3.43	.82

Baseline Disparities Analysis

Disparities across groups are shown for males (Table 3) and females (Table 4) separately. We focus on differences between different-, same-, and both-sex attracted youth (results for youth who reported no romantic attractions are included in tables for reference).

Table 3. Male Standardized OLS and Logistic Regression-Based Estimated Disparities by Outcome for 4- and 3-Category Romantic Attraction: Comparison of Unconditional Estimated Baseline Disparity and Probability Screener Index Used as a Covariate.

	4-Cateogry Romantic Attraction					3-Category Romantic Attraction
	Unconditional Baseline Disparity		Mischievous Index as Covariate		ΔSD/OR	Unconditional Baseline Disparity		Mischievous Index as Covariate		ΔSD/OR
	B/OR(SE)	p	B/OR(SE)	p		B/OR(SE)	p	B/OR(SE)	p
Depressive Symptoms
Different-sex
No attraction	-.18(.04)	< .001	-.18(.04)	< .001	< .01	-.18(.04)	< .001	-.18(.04)	<.001	< .01
Same-sex	.17(.15)	.252	.18(.16)	.278	+ .01	.18(.05)	.002	.14(.05)	.00 7	– .03
Both-sex	.18(.06)	.003	.14(.05)	.013	– .04
Suicidal Ideationa
No attraction	.68(.13)	.037	.65(.13)	.030	+ .03	.68(.13)	.037	.65(.13)	.03 0	+ .03
Same-sex	2.35(.93)	.032	2.58(1.02)	.018	+ .23	1.66(.27)	.002	1.51(.27)	.02 0	– .15
Both-sex	1.57(.27)	.010	1.39(.26)	.082	– .28
Suicidal Attempt(s)a
No attraction	1.21(.39)	.543	1.24(.40)	.511	+ .03	1.21(.39)	.543	1.24(.40)	.51 1	+ .03
Same-sex	4.60(2.44)	.005	5.01(2.68)	.003	+ .41	2.84(.75)	< .001	2.69(.75)	.00 1	– .15
Both-sex	2.61(.74)	.001	2.40(.74)	.005	– .21
Alcohol Use
Different-sex
No attraction	-.47(.04)	< .001	-.50(.04)	< .001	+ .03	-.47(.04)	< .001	-.50(.04)	<.001	+ .03
Same-sex	.17(.21)	.409	.25(.22)	.263	+ .08	.18(.06)	.004	.18(.07)	.00	< .01
Both-sex	.19(.07)	.006	.17(.07)	.017	– .02
Cocaine Usea
No attraction	1.14(.40)	.714	.76(.31)	.506	+ .38	1.14(.40)	.714	.76(.31)	.50 7	+ .38
Same-sex	6.09(3.39)	.001	7.05(4.16)	.001	+ .96	3.89(1.10)	< .001	3.90(1.18)	<.001	+ .01
Both-sex	3.60(1.12)	< .001	3.51(1.18)	< .001	– .09
Parental Closeness
Different-sex
No attraction	.12(.04)	.002	.14(.04)	< .001	+ .02	.12(.04)	.002	.14(.04)	<.001	+ .02
Same-sex	-.16(.14)	.279	-.15(.16)	.362	– .01	-.09(.05)	.082	-.05(.05)	.28 9	– .03
Both-sex	-.08(.05)	.161	-.04(.05)	.450	– .04
School Connectedness
Different-sex
No attraction	.11(.04)	.011	.13(.04)	.003	+ .02	.11(.04)	.011	.13(.04)	.00 3	+ .02
Same-sex	-.02(.11)	.893	-.01(.12)	.954	– .01	-.08(.05)	.135	-.05(.05)	.32 1	– .03
Both-sex	-.09(.06)	.124	-.05(.06)	.303	– .04

Note. B coefficient denotes change in standard deviation of dependent variable for each group in comparison to different- sex attracted youth for continuous dependent variables. Each covariate regression was run with the natural log of the probability-based covariate to reduce skew. “+” in front of ΔSD/OR value denotes an increase in disparity whereas “–” denotes a decrease in disparity after controlling for mischievousness. All estimates for OLS regression are presented for standardized outcomes.

^aDifferent-sex attracted youth are the reference group for logistic regression-based estimates.

Table 4. Female Standardized OLS and Logistic Regression-Based Estimated Disparities by Outcome for 4- and 3-Category Romantic Attraction: Comparison of Unconditional Estimated Baseline Disparity and Probability Screener Index Used as a Covariate.

	4-Cateogry Romantic Attraction					3-Category Romantic Attraction
	Unconditional Baseline Disparity		Mischievous Index as Covariate		ΔSD/O R	Unconditional Baseline Disparity		Mischievous Index as Covariate		ΔSD/O R
	B/OR(SE)	p	B/OR(SE)	p		B/OR(SE)	p	B/OR(SE)	p
Depressive Symptoms
Different-sex
No attraction	-.26(.05)	< .001	-.29(.05)	< .001	+ .03	-.26(.05)	<.001	-.29(.05)	<.001	+ .03
Same-sex	.45(.14)	.002	.45(.15)	.003	.00	.28(.06)	<.001	.26(.06)	<.001<.001	– .02
Both-sex	.22(.07)	.003	.20(.07)	.007	– .02
Suicidal Ideationa
No attraction	.39(.07)	< .001	.37(.07)	< .001	+ .02	.39(.07)	<.001	.37(.07)	<.001	+ .02
Same-sex	2.21(.73)	.018	2.17(.74)	.024	– .04	2.18(.27)	<.001	2.17(.26)	<.001	– .01
Both-sex	2.17(.33)	<.001	2.17(.33)	<.001	< .01
Suicidal Attempt(s)a
No attraction	.46(.15)	.023	.43(.15)	.017	+ .03	.46(.15)	.02 3	.43(.15)	.017	+ .03
Same-sex	2.51(1.04)	.028	2.38(1.02)	.045	– .13	2.56(.58)	<.001	2.54(.57)	<.001	– .02
Both-sex	2.58(.68)	< .001	2.60(.69)	< .001	+ .02
Alcohol Use
Different-sex
No attraction	-.39(.03)	< .001	-.37(.04)	< .001	+ .02	-.39(.03)	<.001	-.38(.04)	<.001	+ .01
Same-sex	.43(.19)	.025	.43(.19)	.024	+ .01	.39(.08)	<.001	.37(.08)	<.001	– .03
Both-sex	.38(.07)	< .001	.34(.07)	< .001	– .04
Cocaine Usea
No attraction	.48(.29)	.234	.44(.25)	.147	+ .04	.48(.29)	.23 4	.44(.25)	.14 9	+ .04
Same-sex	3.54(2.77)	.107	3.42(2.86)	.145	– .12	3.83(1.46)	.00 1	4.01(1.61)	.00 1	+ .18
Both-sex	3.93(1.62)	.001	4.23(1.77)	.001	+ .30
Parental Closeness
Different-sex
No attraction	.28(.05)	< .001	.30(.05)	< .001	+ .02	.28(.05)	<.001	.30(.05)	<.001	+ .02
Same-sex	-.51(.13)	< .001	-.51(.13)	< .001	< .01	-.37(.07)	<.001	-.36(.07)	<.001	– .01
Both-sex	-.32(.08)	< .001	-.30(.08)	< .001	– .02
School Connectedness
Different-sex
No attraction	.19(.05)	< .001	.21(.05)	< .001	+ .02	.19(.05)	<.001	.21(.05)	<.001	+ .02
Same-sex	-.30(.10)	.004	-.29(.10)	.005	– .01	-.33(.06)	<.001	-.33(.07)	<.001	< .01
Both-sex	-.34(.08)	< .001	-.34(.08)	< .001	< .01

Note. B coefficient denotes change in standard deviation of dependent variable for each group in comparison to different- sex attracted youth for continuous dependent variables. Each covariate regression was run with the natural log of the probability-based covariate to reduce skew. “+” in front of ΔSD/OR value denotes an increase in disparity whereas “–” denotes a decrease in disparity after controlling for mischievousness. All estimates for OLS regression are presented for standardized outcomes.

^aDifferent-sex attracted youth are the reference group for logistic regression-based estimates.

For males, using the 3-category operationalization of romantic attraction, same-sex attracted youth had higher levels of depressive symptomology, suicidal ideation, suicidal attempts, alcohol use, and cocaine use than heterosexual males. Similar results were found between different- and both-sex attracted youth using the 4-category operationalization of romantic attraction, with the exception of depressive symptomology and alcohol use. No significant differences in parental closeness and school connectedness were found between different- and same- or both-sex attracted males.

For females, compared to different-sex attracted youth, same-sex attracted youth were at greater risk for depressive symptomology, suicidal ideation and attempts, alcohol and cocaine use, and lower levels of parental closeness and school connectedness; the 4-category romantic attraction measure showed similar results except for cocaine use between different- and exclusively same-sex attracted females.

Romantic Attraction and Mischief

The highest total raw mischievous score in our sample was five. Almost 93% of the sample provided less than two low-frequency response items and nearly 99% provided less than three. Disaggregated reports by gender and romantic attraction (4-category measure) showed that different- and same-sex attracted males were less likely to have three or more low-frequency responses. Probabilities were: p_DSA = 1.64% and P_SSA = 1.30%, respectively, when compared to males reporting both-sex, p_BSA = 3.44%, or no attraction, P_NA = 3.17%. For females, those with no attractions were more likely to endorse more than two low-frequency responses, P_NA = 2.09%, compared to different-, p_dsa = .28%, same-, P_SSA = .00%, and both-sex, p_bsa = .30%, attracted females.

Accounting for Mischief

Changes in the standard deviation [ΔSD] of continuous outcomes and difference in odds ratios [ΔOR] for categorical dependent variables when comparing unconditional and covariate models are shown in Table 3. Results of the 3-category operationalization of romantic attraction indicated no changes in the significance of disparities at the p < .05 level for males. The decrease in disparities between different- and same-sex attracted youth were relatively small with an average decrease in standard deviation of Δ̄SD_SSA = .025 and odds ratio of Δ̄OR_SSA = .15 for continuous and categorical outcomes, respectively.

The 4-category operationalization of romantic attraction distinguished between same- and both-sex attraction; results based on this measure showed that the inclusion of mischievous scores attenuated differences in suicidal ideation between both- and different-sex attracted males to the trend level, from OR = 1.57, 95% CI [1.12, 2.22] to OR = 1.39, 95% CI [.96, 2.00]. All other disparities remained statistically significant. When attenuated, the average decrease in disparities across outcomes between different- and both-sex attracted males was Δ̄SD_BSA= .034 and Δ̄OR_BSA= .160 across outcomes and Δ̄SD_SSA= .009 for comparisons between different- and same-sex attracted males. The OR for same-sex attracted youth did not decrease in models adjusting for mischievousness. Notably, some disparities were greater for same-sex attracted males with the inclusion of the mischievous index covariate, including suicidal attempts, Δor_ssa = .41, alcohol use, ΔSD_SSA= 08, and cocaine use Δor_ssa = .96.

For females (Table 4), all disparities identified in the unconditional model remained after including the probability-based screener index as a covariate, for both 3- and 4-category attraction comparisons. Like males, the magnitude of the disparity was slightly attenuated for depressive symptomology, suicidal ideation, suicidal attempts, cocaine use, and school connectedness for females with exclusive same-sex attraction. Similarly, differences in depressive symptomology, alcohol use, and school connectedness diminished between different -and both-sex attracted females. The average change in disparities was minimal: Using the 3-category romantic attraction measure, the average change in disparities between females reporting different-sex attraction and those reporting any same-sex attraction were small: = .017, Δ̄OR_SSA = .015. For the 4-category measure, compared to females with different-sex attractions, average parameter decreases were Δ̄SD_SSA= .004, Δ̄OR_SSA = .085, Δ̄SD_BSA = .027. Differences between different- and both-sex attracted females for categorical outcomes did not decrease.

The inclusion of the screener-index covariate magnified disparities for cocaine use for females reporting any-same sex attraction in the 3-cateogry group comparison, Δ̄OR_SSA = .18. For the 4-category measure, the inclusion of the screener-index magnified disparities for suicidal ideation, suicidal attempt(s), cocaine use, and parental closeness for both sex-attracted females and alcohol use for same-sex (only) attracted females. These differences were generally small, with the exception of cocaine use, Δ̄OR_BSA = .30.

Finally, we replicated all analyses under several different conditions. First, we tested models without weights and complex sample design adjustments and results were unchanged except that alcohol use disparities were not present for same-sex attracted males at baseline and attenuated for both-sex attracted males. Second, there were no differences in the elimination of disparities for analyses that excluded the top 2.5% of extreme responders (see Robinson-Cimpian, 2014). Third, we assessed change in disparities in comparisons where youth who reported no attraction and different-sex attraction were combined into a single category (i.e., no same-sex attraction). Overall, the pattern of disparities from these comparisons were consistent with those reported here. Under this condition, male suicidal ideation disparities between both-sex attracted males and those reporting no same-sex attraction remained (OR = 1.66, p < .01, 95% CI [1.18, 2.33]) after including the mischievous index covariate.

Discussion

We provided an empirical analysis of inaccurate and mischievous responders in the reports of same-sex romantic attraction in Wave 1 of the Add Health Study, and an investigation of the influence of potential mischief in the analysis and identification of sexual minority health disparities. The ultimate question is the veracity of results that indicate disparities for youth who report same-sex romantic attractions. We found little evidence of inaccurate responders or mischief among same-sex attracted youth and, with one exception, we found that controlling for mischievous responses did not change results for sexual minority disparities.

First, we used two prior empirical approaches to identify “inaccurate responders,” “jokesters,” (Approach 1; Fan et al, 2006) and “mischievous” responders (Approach 2; Robinson-Cimpian, 2014). We found no evidence that Add Health Wave 1 same-sex attracted youth were more likely to be inaccurate responders or jokesters compared to other youth. We also found no evidence that males or females who reported same-sex attraction at Wave 1 and later reported an exclusively heterosexual identity were inaccurate responders or jokesters compared to those with consistent reports from Wave 1 to Wave 4. We did, however, find a tendency for males who reported attraction to both sexes—but not exclusively same-sex attracted males or same- or both-sex attracted females—to have higher likelihood of mischievous reporting. Despite this difference, youth index scores documenting mischief were low compared to previous studies. In a school-based SAQ, Robinson-Cimpian (2014) noted that relative to the 1.5% of youth in the overall sample, 11.7% of LGBQ adolescence were flagged for three or more low-frequency responses. We found 1.3% of the Wave 1 Add Health participants provided three or more suspect responses; 1.3% and 3.4% of same- and both-sex attracted males, respectively, scored a 3 or more on the mischievous index, and only one same-sex attracted female scored a 3 or higher. These observed differences in the rate of mischief for sexual minority participants may be based on differences in the method of survey administration. Wave 1 Add Health data were collected via interviewers in youth's homes, but prior studies of mischievous responders (Cornell et al., 2002; Robinson & Espelage, 2011; Robinson-Cimpain, 2014) were based on school-based SAQs: this distinction has been ignored by previous critics of the Add Health data. It could be that youth are more inclined to respond in mischievous ways when in school settings. In fact, during the replication of Fan et al.'s (2006) triangulation of in-school, in-home, and parental responses, we found that youth in-home reports were more congruent with parent responses than were answers provided at school. Thus, the context for administering youth surveys appears to be important when considering the question of mischievous responding. Future work in this area may help clarify the influence of contexts and survey administration on youth reports.

Second, we found that controlling for mischievous responses eliminated only one health disparity: suicidal ideation for both-sex attracted boys. Results for other disparities were partially attenuated, but reductions in effect sizes were small. At the same time, after accounting for mischief, some disparities increased. These results are consistent with findings from other studies based on both convenience as well as population-based, representative samples that document mental, behavioral, and resource disparities based on sexual orientation (Birkett et al., 2009; Corliss et al., 2008; Saewyc, 2011; Stone et al., 2014). We acknowledge the potential role of publishing bias (that studies that identify no statistically significant differences may be less likely to be published) (Franco, Malhotra, & Simonovits, 2014) yet the growing collection of systematic and meta-analytic reviews that demonstrate elevated risk for sexual minorities across samples, cohorts, and ages—and the size of those effects—is compelling (see Bouris et al., 2010; Friedman et al., 2011; Katz-Wise & Hyde, 2012; King et al., 2008; Lick, Durso, & Johnson, 2013; Marshal et al., 2008, 2011; Newcomb & Mustanski, 2010; Plöderl & Tremblay, 2015; Toomey & Russell, 2016).

Consistent with others before us, we note the limitations inherent in the measure of romantic attraction. For more than a decade, scholars have questioned what that meant to youth (in the mid-1990s), and what it means with respect to the discourse of sexual minority or LGBTQ youth research today. It is not a measure of “sexual” attraction or behavior nor is it (as we now know based on subsequent waves of the study; Savin-Williams & Ream, 2007) a clear indicator of sexual identity. Yet, historically speaking, that datum was remarkable: Aside from several important state-level surveys, there had been nothing of its kind in the United States before that time. It provided the first national-level estimates of sexual minority disparities (Russell & Joyner, 2001; Savin-Williams & Joyner, 2015) and sparked a cascade of studies that have subsequently developed a robust and illuminating literature on the health and well-being of LGBTQ young people.

Savin-Williams and Joyner (2014a) argued that due to the steady increase in social acceptance for same-sex sexuality over the last 30 years (Gallup, 2015), it is unlikely that those who were “out” (i.e., same-sex attracted) during adolescence would then go “back in the closet” as adults (i.e., report heterosexual identities). Yet, researchers utilizing different samples have found that adolescent and young adult males and females shift identities (Katz-Wise, 2014; Ott et al., 2011; Rosario et al., 2006) and that people who report same-sex attraction and behavior may not adopt lesbian, gay, or bisexual identities (Bostwick et al., 2010; Copen et al,. 2016; Gates, 2010; Xu et al., 2010a, 2010b). Therefore, observations may simply reflect a normative identity development process. Wave 1 did not include questions regarding sexual identity and thus we do not know whether youth ever adopted a nonheteoresuxal identity. Further, the broad characterization of these youth as “out” and going “back into the closet” (Savin-Williams & Joyner, 2014a) reaches beyond the data: Youth in Add Health provide no information regarding their disclosure of same-sex attractions at Wave 1. At best, we can state that youth report a self-awareness of same-sex attraction during adolescence.

Conclusion

Recent debate questioned the veracity of the Add Health Wave 1 reports of (same-sex) romantic attraction: Savin-Williams and Joyner (2014a) argued that results based on the data “might have contributed little and, perhaps, distorted our understanding of sexual development among Wave 1 nonheterosexual adolescents. The distortion might have been in overemphasizing the trauma of being young and gay” (p. 43). Yet, until now, empirical tests of this assertion had not been made. Our results show that this conclusion is misleading. Importantly, Add Health will continue to provide valuable information on its participants as they age: Wave 5 data collection is underway to survey participants who are now between the ages of 31 and 42. Our analyses indicate that these data should continue to provide an important base of knowledge for adolescent to adult health in the United States.

As others engaged in the debate have argued (Katz-Wise et al., 2014), the documentation of disparities in health does not promulgate stigma; rather, it is through documentation that we might both identify and work to remedy disparities, or the inequalities rooted in stigma and prejudice that create such disparities. A focus on “mischief” is misplaced because it focuses attention squarely on youth and their self-reported attractions and well-being as the locus of explanation for (and skepticism regarding) disparities. What is needed is attention to the pervasive structural inequalities and prejudices that are ultimately the source of many of the disparities in health and well-being for youth and all people.