Sex and Gender in the US Health Surveillance System: A Call to Action

Abstract

Youth Risk Behavior Survey (YRBS) data have exposed significant sexual orientation disparities in health. Interest in examining the health of transgender youths, whose gender identities or expressions are not fully congruent with their assigned sex at birth, highlights limitations of the YRBS and the broader US health surveillance system.

In 2009, we conducted the mixed-methods Massachusetts Gender Measures Project to develop and cognitively test measures for adolescent health surveillance surveys. A promising measure of transgender status emerged through this work.

Further research is needed to produce accurate measures of assigned sex at birth and several dimensions of gender to further our understanding of determinants of gender disparities in health and enable strategic responses to address them.

Healthy People 2020 extends a commitment to

assess health disparities in the US population by tracking rates of illness, death, chronic conditions, behaviors, and other types of outcomes in relation to demographic factors¹

and aims to “achieve health equity, eliminate disparities, and improve the health of all groups.”² Youth Risk Behavior Survey data have exposed significant sexual orientation disparities^3–8 in health in jurisdictions that included measures of sexual orientation on their surveys.

However, gaps in the US health surveillance system inhibit efforts to improve the health of both transgender people^9–11 and the nontransgender (cisgender) majority. Demographic measures that would enable the population to be classified as trans- or cisgender are rarely included in the health surveillance system. Such measures, often used in combination, include sex (assigned at birth), gender identity (current), and transgender status (transgender is an identity for some individuals and an adjective or status for others). Data about additional dimensions of gender (e.g., expression, beliefs about gender) that could be used to tackle persistent health disparities are also not collected, and these gaps represent untapped potential to improve population health.

Although sex and gender identity are often static and concordant for the vast majority of Americans, both sex and gender (more broadly) are multidimensional constructs and can vary over time—particularly for transgender people. The term sex refers to biological differences between male, female, and intersex people (hormones, secondary sex characteristics, reproductive anatomy) that can be altered through the use of hormones and surgical interventions.¹² The assignment of individuals to a sex category by medical practitioners at birth is typically based on the appearance of external genitalia and is recorded on the birth certificate as male or female (and assumes a legal status). Legal sex can sometimes be changed on legal documents (e.g., birth certificate, driver’s license, passport) through a complex set of legal procedures. Gender has psychological (identity—an internal sense of being a boy or girl, genderqueer, etc.), social (beliefs about gender, the roles that one assumes, community affiliation), and behavioral (gender expression, how one expresses one’s identity through appearance and actions and is perceived by others) dimensions.¹³

Unfortunately, sex and gender are rarely explicitly measured, and when data are collected, a range of measures and approaches are used. In recognition of the importance of sex and gender identity as demographic characteristics of the US population, and variability in their measurement, the Institute of Medicine,¹⁴ US Department of Health and Human Services,¹⁵ and Centers for Disease Control and Prevention¹⁶ have called for a national data standard for sex and research to create valid measures of gender identity. Although the Youth Risk Behavior Survey relies on self-reported sex, other health surveillance surveys, such as the Behavioral Risk Factor Surveillance System,¹⁷ are telephone administered and classify respondents as male or female according to the sound of their voice, or, as in the in-person National Health Interview Survey,¹⁸ use visual appearance, with direct questioning about the respondent’s sex by the interviewer “if necessary”¹⁷ or “not apparent,”¹⁸ respectively. These data are used as measures of both sex and gender identity^19,20; however, they actually measure the interviewer’s perception of the respondent’s gender identity.

Data about other dimensions of gender, such as gender expression and beliefs about gender (the individual-level analog to gender norms, a societal-level construct), are not collected in the health surveillance system, despite growing bodies of research that highlight their importance as health determinants. Research conducted in lesbian, gay, and bisexual, as well as general, primarily heterosexual cisgender samples, indicates that individuals whose gender expression fails to conform to sex-linked social expectations (e.g., masculine girls and women, feminine boys and men) are at increased risk for violence,^21–28 discrimination,²³ posttraumatic stress disorder,²⁸ and depression.²⁹ Violence^23,30–38 and discrimination^39–41 against transgender people, who are gender nonconforming by identity or expression in relation to their assigned sex at birth, is commonplace.

Research conducted in the general population shows that beliefs about gender (e.g., violence is acceptable, exercising caution is not masculine, being assertive is not feminine)⁴² are associated with aggression^43–47 and alcohol use.^44,48 Thus, strategies to ameliorate persistent public health problems, such as high mortality among men from injury, homicide, and suicide,⁴⁹ might be advanced by the collection of gender data. Long-standing disparities in depression that disfavor women^50,51 might also be addressed by reducing girls’ and women’s exposure to violence⁵² and by modifying emotional coping styles (e.g., rumination is more common among girls and women,⁵³ whereas problem solving is positively associated with masculine traits and negatively associated with depression⁵⁴).

THE NEED FOR SEX AND GENDER MEASUREMENT RESEARCH

Despite the importance of assigned sex at birth and gender identity, expression, and beliefs about gender as demographic characteristics and health determinants, surprisingly little rigorous research has been conducted to develop and evaluate the performance of brief measures of these constructs. Rigorous techniques include cognitive testing that assesses comprehension and respondent burden^55–57 and quantitative analyses to assess criterion and construct validity⁵⁸ in samples that vary by gender and, ideally, contain trans- and cisgender participants. To our knowledge, no published studies have validated measures of sex.

Published studies that used rigorous methods to evaluate the accuracy of brief gender measures are summarized in Table 1. Several measures of gendered personality traits, beliefs, and behaviors, including sex role inventories⁶² and measures of conformity to masculine and feminine gender norms,^47,63,64 are in use and have been psychometrically evaluated, but their length (> 20 items) prohibits inclusion in surveillance vehicles. Reliability, or the precision of estimates or classifications⁶⁵ derived from brief measures of sex and gender, has yet to be assessed in the peer-reviewed literature. In an effort to pave the way for the next generation of rigorous adolescent measurement studies, we conducted a mixed-methods adolescent study. Here we present findings from the study, including strengths and limitations, and outline a program of research to move the field forward.

TABLE 1—

Cognitively Tested Brief Gender Measures Published in Peer-Reviewed Journals

Study	Sample	Construct	Tested Measure(s)	Performance	Recommended Measure
Conron et al.59	Racially and ethnically diverse LGBT and heterosexual cisgender youths aged 15–21 y (n = 30)	Current gender identity	Sex/gender: female; male; transgender, male to female; transgender, female to male; transgender, not exclusively male or female; not sure.	Accurately discriminated between transgender and cisgender youths. Acceptable to most youths. A few transgender youths found the conflation of sex and gender in the question stem confusing. One transgender youth interpreted the question as asking about physical anatomy rather than gender identity.	Gender: female; male; transgender, male to female; transgender, female to male; transgender, do not identify as exclusively male or female; not sure.
Clark et al.60	Women aged 40–75 y who were diverse by sexual orientation (n = 40)	Current gender expression	How would you describe yourself? Would you say …” very masculine, mostly masculine, equally masculine and feminine, mostly feminine, very feminine, not sure, none of the statements describe how you think of yourself.	Respondents were unclear about whether to report how they view themselves or believe that others view them. Respondents interpreted the terms masculine and feminine variably.	A woman’s appearance, style or dress may affect the way people think of her. How do you think people describe your appearance, style or dress? Would you say … very masculine, mostly masculine, equally masculine and feminine, mostly feminine, very feminine, not sure, none of the statements describe how you think of yourself.
Wylie et al.61	Heterosexual cisgender and LGBT adults aged 18–30 y (n = 82)	Socially assigned gender expression	A person’s appearance, style or dress may affect the way people think of her or him. How do you think people describe your appearance, style or dress? Very feminine, mostly feminine, somewhat feminine, equally feminine and masculine, mostly masculine, somewhat masculine, very masculine. A person’s mannerisms (such as the way a person walks or talks) may affect the way people think of her or him. How do you think people describe your mannerisms? (Same response options.)	Measures were generally clear and acceptable to participants. Half of the sample reported variability in gender expression from day to day or setting to setting and found it difficult to select 1 response option. Cis- and transgender women, on average, reported scores on the feminine end of the bipolar scale; cis- and transgender men, on average, reported scores on the masculine end of the bipolar scale.	A person’s appearance, style, or dress may affect the way people think of them. On average, how do you think people would describe your appearance, style or dress? (Same response options.) A person’s mannerisms (such as the way a person walks or talks) may affect the way people think of them. On average, how do you think people would describe your mannerisms? (Same response options.)

Note. LGBT = lesbian, gay, bisexual, transgender.

THE MASSACHUSETTS GENDER MEASURES PROJECT

We designed a three-phase mixed-methods study to produce measures for adolescent health surveillance surveys.⁶⁶ We report on two approaches to differentiate between trans- and cisgender respondents: concordance between sex and gender identity⁶⁷ and assessment of transgender status. We conducted formative focus groups (n = 47 participants in five groups) to assess adolescent familiarity with terms that might appear in measures (e.g., sex, gender, masculine, transgender). Next, we conducted focus groups (n = 53 participants in seven groups) to cognitively test measures that used well-understood language. We instructed adolescent participants to provide feedback about survey questions by the retrospective talk-aloud method^56,68 and instructed them not to share their personal answers. Finally, we pilot-tested promising measures.

We recruited a diverse sample of youths (n = 112) aged 13 to 18 years (Table 2) from adolescent pregnancy prevention, health career, and lesbian, gay, bisexual, and transgender (LGBT) youth programs across Massachusetts. Participants in the pilot test were transgender youth conference attendees (n = 20). We collected data between May and August 2009. Eight youths participated in more than one project phase; however, we only analyzed one set of responses. After providing consent or assent, all youths completed a brief, close-ended survey containing measures of sex and gender identity, parental gender pressure, bullying, current gender expression, and 4 items from the Recalled Childhood Gender Questionnaire,⁶⁹ used previously as a measure of recalled childhood gender nonconformity.²⁸ We added a transgender status question to the survey in the cognitive testing phase. Participants received a resource list and a $15 (focus group) or $5 (pilot test) gift card.

TABLE 2—

Demographic Characteristics of Adolescent Participants: Massachusetts Gender Measures Project, 2009

Characteristic	Full Sample (n = 112), No. (%)	Cognitive and Pilot Testing Sample (n = 73), No. (%)
Age, y
13–14	11 (9.8)	11 (15.1)
15–16	46 (41.1)	22 (30.1)
17–18	55 (49.1)	40 (54.8)
Assigned sex at birth
Female	82 (73.2)	51 (69.9)
Male	30 (26.8)	22 (30.1)
Gender status
Female, cisgender	53 (47.3)	27 (37.0)
Male, cisgender	27 (24.1)	18 (24.7)
Transgender	32 (28.6)	28 (38.4)
Race/ethnicity
Latino	34 (30.4)	22 (30.1)
Black	27 (24.1)	16 (21.9)
White	34 (30.4)	26 (35.6)
Asian	5 (4.5)	5 (6.9)
Multiracial/ethnic	9 (8.0)	3 (4.1)
Missing	3 (2.7)	1 (1.4)
Sexual orientation and transgender status
Lesbian/gay/bisexual, cisgender	14 (12.5)	9 (12.3)
Transgender	32 (28.6)	28 (38.4)
Heterosexual, cisgender	63 (56.3)	34 (46.6)
Missing	3 (2.7)	2 (2.7)
Highest household education
≤ high school/GED	39 (34.8)	19 (26.0)
Some college	27 (24.1)	21 (28.8)
4-y college	20 (17.9)	14 (19.2)
> 4 y college	20 (17.9)	16 (21.9)
Missing	6 (5.4)	3 (4.1)

Note. GED = general equivalency diploma. Information about assigned sex at birth, gender status, and transgender status was collected via close-ended survey items and handwritten comments on surveys or made during focus groups. All other data were collected via close-ended survey items.

We prepared focus group notes while listening to audio recordings and transcribed quotes verbatim. The facilitator (K. J. C.) reviewed all focus group notes. We reviewed formative research notes to establish the meaning of specific terms and the degree of consistency among adolescents. We reviewed cognitive testing notes for comprehension (consistencies of interpretation across items, terms, and response options) and respondent burden (by characterizing respondent-reported experiences in answering questions.) Field notes included comments made by youths during survey completion.

We conducted quantitative analyses, including examination of item response patterns, with SAS version 9.1.⁷⁰ We aggregated data to assess validity in a subsample of youths (n = 73) who completed measures of sex, gender identity, and transgender status during the cognitive and pilot-testing phases of the study (Table 2). We evaluated criterion validity by comparing the degree of concordance (sensitivity, specificity⁶⁵) between classifications derived from 1 measurement approach and a gold standard classification of transgender status that used all information collected during the project. We assessed construct validity by examining the extent to which a measure performed as expected.⁷¹

Sex and Gender

We asked formative research participants about their understanding of the terms sex and gender. Most non-LGBT youths viewed the terms as synonymous and thought that the sex item in the project survey (“What is your sex? 1 = male, 2 = female”) asked, “Are you a girl or a guy?” or “What you were born as?” or “What is your gender?” While completing the project survey, several non-LGBT youths drew the facilitator’s attention to two seemingly redundant questions (sex followed by “What is your gender? 1 = male, 2 = female, 3 = transgender”) and asked for clarification. It is unknown whether these youths would answer questions perceived as redundant on school-based surveys or skip them. By contrast, LGBT youths interpreted the sex item variably, such as inquiring about a range of physical traits and states (current, birth): “what’s your anatomy,” “your chromosomes,” “the genitalia you were born with, or, the genitalia you have, depending,” and “the designation that they put on your birth certificate.” Several LGBT youths noted that the absence of an intersex response option was problematic.

Across project phases, seven of 32 transgender youths (21.9%) did not provide a valid response to the sex item (four skipped it, another selected both male and female, and two chose responses that were inconsistent with their assigned sex at birth). Thus, two-item classification approaches that rely on responses to a common sex measure may not accurately discriminate between trans- and cisgender youths. In fact, in the cognitive and pilot-testing subsample (hereafter, the subsample), 28.6% of transgender youths were not correctly classified by this approach; 100% of cisgender youths were correctly classified.

Transgender Status

During formative research, most cisgender youths defined transgender as a person who “identifies as the sex opposite from what they were born” and often changes his or her body through surgery or hormone therapy, while others, particularly younger heterosexual youths, were unable to offer a definition. A transgender adolescent expressed an understanding of the term that was shared by most transgender youths:

The body you were born with doesn’t match up with the way you feel. Like, it just, it doesn’t mean completely you’re born female and you identify as completely male, it just, it doesn’t completely match up.

A single-item measure of transgender status, informed by previous research,⁵⁹ was consistently understood by and was acceptable to most youths who participated in cognitive testing. This question asked,

When a person's sex and gender do not match, they might think of themselves as transgender. Sex is what a person is born. Gender is how a person feels. Do you think of yourself as transgender?

Response options provided during the formative and cognitive testing phases of the project were 1 = no; 2 = yes, transgender, male to female; 3 = yes, transgender, female to male; 4 = yes, transgender. One cisgender youth commented,

You break up sex and gender. You tell what they mean separately. If you didn’t get the questions [sex and gender] before, you can go and change your answers.

Another said, “You can’t make it more clear than it is.” Others said, “It specifies,” “It helps to educate,” and “[It] gives the background for questions 1 and 2 [sex and gender].” Many LGBT youths also found the question clear: “It was pretty straightforward. It explains the difference between sex and gender.” In the subsample, most transgender (86%) and all cisgender youths were correctly classified by this measure.

Analyses conducted in the subsample indicated support for the construct validity of this measure. A larger proportion of youths classified as transgender than cisgender reported parental pressure to look or act more feminine (adolescent girls) or masculine (adolescent boys; 55.0% vs 26.7%; P = .05) and 30-day peer bullying attributable to gender expression (45.0% vs 14.3%; P = .01). Transgender youths also reported a higher mean number of all-cause 12-month bullying incidents (3.8 vs 0.5; P = .003) and mean recalled childhood gender expression score (range = 1–5, with higher scores reflecting greater nonconformity; 4.5 vs 2.7; P < .001) than did cisgender youths.

A few (n = 3) transgender youths did not like the transgender status measure because it forced transsexual respondents to identify as transgender. Although the youths acknowledged that transsexuals fall under the transgender umbrella, they indicated that some youths who have altered their bodies or wish to do so may identify as transsexual or as the sex to which they are transitioning. We pilot-tested modified response options that allowed for greater variation in transgender identity and found that they performed as well as the original response options. Specifically, in the pilot test subsample (n = 20), only one respondent did not answer the transgender status measure, and this individual also skipped the sex and gender identity items. The percentages of respondents (trans- vs cisgender) who reported gender-related parental pressure (66.7% vs 37.5%; P = .35) and peer bullying (33.3% vs 50.0%; P = .64) were consistent across versions. Similarly, mean all-cause 12-month bullying (3.2 vs 2.6; P = .76) and recalled childhood gender expression (4.7 vs 4.5; P = .81) did not vary across versions.

CONCLUSIONS

We explored adolescent understanding of terms that are used in sex and gender measures and evaluated a two-item sex and gender identity classification approach and a stand-alone measure of transgender status in a heterogeneous adolescent sample. During the formative and cognitive testing phases of the project, we found that some non-LGBT youths confused the terms sex and gender; we therefore recommend that measures of assigned sex at birth and gender identity in subsequent research look distinct to signal that they are assessing different constructs. For instance, a measure of assigned sex at birth that was included on the National Survey on Transgender Experiences of Discrimination in the US was understood by both cis- and transgender young adults in a recent cognitive testing study (n = 39; S. L. R. et al., unpublished data, 2013). “What sex were you assigned at birth, on your original birth certificate? 1 = male, 2 = female” looks different from a measure of current gender identity used by K. J. C. on adolescent surveys,

Which of the following describes your gender identity, how you think about yourself? Select all that apply. 1 = Boy/man, 2 = Male/masculine, 3 = Girl/woman, 4 = Female/feminine, 5 = Genderqueer, 6 = Transgender, 7 = A gender identity not listed here, _____ (please specify), 8 = Not sure).

More than a fifth of transgender youths in our study did not provide accurate information about their sex assigned at birth on a common survey measure of sex (“What is your sex? 1 = male, 2 = female”). Error on the sex item contributed to low specificity (71.4%) of classifications derived from this measurement approach. Thus, our findings highlight the need for valid measures of assigned sex at birth for use on adolescent surveys. Measurement approaches that rely on accurate information about assigned sex at birth and current gender identity to discriminate between trans- and cisgender respondents will require valid measures of both constructs.

Our stand-alone measure of transgender status discriminated well between trans- and cisgender youths (sensitivity = 86%; specificity = 100%). Because cisgender youths represent the vast majority of respondents,¹¹ specificity is critical to avoid misclassification and false positives.⁷² Some non-LGBT youths were uncertain about the meaning of the term transgender when it was used in isolation (i.e., “What does the term transgender mean?”), but were clear when the term was paired with a definition, as in our measure. Thus, we recommend our transgender status measure, with slight modifications, for rigorous testing in future research.

To further respond to feedback from transsexually identified transgender youths, we recommend changing the question stem from “Do you think of yourself as transgender?” to “Are you transgender?” We also suggest including don’t know responses, as shown in the version of our measure adopted by the Massachusetts Department of Public Health for use on the school-based 2013 Youth Health Survey⁷³:

Please choose the one best fitting response. When a person’s sex and gender do not match, they might think of themselves as transgender. Sex is what a person is born. Gender is how a person feels. Are you transgender?

(1) No

(2) Yes, and I identify as a boy or man

(3) Yes, and I identify as a girl or woman

(4) Yes, and I identify in some other way

(5) I do not know what this question is asking

(6) I do not know if I am transgender

Strengths of the Massachusetts Gender Measures Project were mixed methods and a purposively diverse sample; however, cognitive testing methods are designed for one-on-one interviews, and participant feedback may have been altered by the group setting. Although the focus group approach was cost-effective, findings should be replicated through individual interviews. Future research should purposely overrecruit male-to-female youths and transgender youths of color, who were underrepresented in our sample. Finally, we conducted this project in community settings. Measures intended for use on school-based surveys must be evaluated in the school setting.

Rigorous mixed-methods studies to evaluate the validity and short-term test–retest reliability of measures of assigned sex at birth and multiple dimensions of gender are needed. Questions should be evaluated in multiple languages and in geographically varied samples.

Our specific recommendations build on the Massachusetts Gender Measures Project and research by Wylie et al.⁶¹:

• Our recommended transgender status measure should be cognitively tested with high school students and piloted on school surveys that contain measures of assigned sex at birth and gender identity to assess its validity; short-term test–retest reliability⁶³ should be calculated in a subsample of participants.

• Measures of assigned sex at birth and current gender identity that possess good psychometric properties must be developed for use with adolescents and adults.

• An adult transgender status measure, such as the one included on the Massachusetts Behavioral Risk Factor Surveillance Survey,⁹ should be cognitively tested with adults.

• The socially assigned–perceived gender expression measures evaluated by Wylie et al.⁶¹ should be cognitively tested in a diverse student sample and piloted on school surveys.

A modification to focus on peers in the school setting (e.g., How do you think other students describe your appearance, style, or dress at school?) may address respondent difficulty reporting on gender expression that varies by setting and referent. More broadly, research on expressed and perceived gender expression across a range of settings (home, school, work) and reporters (parents, teachers, peers, self) is needed because perception of (and response to) gender expression may vary by age, race/ethnicity, socioeconomic status, and sexual orientation. Such research would be consistent with measures of perceived race and racial identity,^74–76 which have informed strategies to address racial disparities in health.⁷⁷ Including accurate measures of assigned sex at birth and several dimensions of gender in the health surveillance system will further our understanding of determinants of gender disparities in health and enable strategic responses to address them.