For Data’s Sake: Dilemmas in the Measurement of Gender Minorities

Abstract

Gender minority health disparity research is limited by binary gender measurement practices. This study seeks to broaden current discourse on gender identity measurement in the USA, including measurement adoption challenges and mitigation strategies, thereby allowing for better data collection to understand and address health disparities for people of all genders. Three data sources were used to triangulate findings: expert interviews with gender and sexuality research leaders; key-informant interviews with gender minorities in New Orleans, LA; and document analysis of relevant surveys, guides and commentaries. Ten key dilemmas were identified: 1) moving beyond binary gender construction; 2) conflation of gender, sex and sexual orientation; 3) emerging nature of gender-related language; 4) concerns about item sensitivity; 5) research fatigue among gender minorities; 6) design and analytical limitations; 7) categorical and procedural consistency; 8) pre-populated vs. open-field survey items; 9) potential misclassification; and 10) competing data collection needs. Researchers must continue working toward consensus concerning better practices is gender measurement and be explicit about their methodological choices. The existence of these dilemmas must not impede research on important health issues affecting gender minorities.

Introduction

From the limited research so far conducted, we know that there are differences between gender minorities and the general population on a range of health issues such as HIV ( Centers for Disease Control and Prevention 2017), health care and insurance access (Roller, Sedlak and Draucker 2015; Institute of Medicine 2011; Bradford et al. 2013), mental health (Bockting et al. 2013), suicide attempt and ideation (Reisner et al. 2014), violence (Legal 2010; Testa et al. 2012) and stigma and discrimination (Legal 2010; Lombardi et al. 2002; Bradford et al. 2013; Grant et al. 2011; Harrison-Quintana, Grant and Rivera 2015). However, there exists a wide range of health issues (cancer, motor vehicle injury, healthcare associated infections, etc.) that have not been examined through a gender minority lens, frequently because non-binary measurements of gender are not included in large-scale studies, community samples, and in electronic medical records- which serve as data for a multitude of health disparity research. This information, when gathered, is generally collected in smaller projects, but findings based on community and regional surveys ‘just don’t carry the same authority as federal data in grant applications, policy decisions, and resource allocation’ (Scout 2013). Including better measures of gender and integrating gendered analysis in all public health data collection will lead to better health outcomes for all people.

Gender, along with age, race, education and socioeconomic status, provides a basis for interpreting health and other disparities. Although gender has historically been treated as a stable binary category, it is more accurately understood as an evolving socially constructed category of the human experience, and measurement methodologies need to respond appropriately. From this broader lens, gender may be reported in terms of a person’s felt, desired or intended identity and expression as well as how an individual believes that they are perceived by others (Herman 2014), all of which are facets which hold import to public health outcomes such as health behaviour, emotional well-being, health access, etc. In Table 1 key terms related to gender are presented, demonstrating how gender differs from sex, illustrating various health relevant gender dimensions, and defining gender minority as an umbrella term to refer to transgender and gender non-conforming individuals.

Table 1:

Glossary of Relevant Terms

Term	Definition	Associated terms
Assigned Sex at Birth (ASAB)	The assignment of individuals to a sex category by medical practitioners at birth is typically based on the appearance of external genitalia. Assigned sex at birth is then recorded on the birth certificate as male or female (Conron et al. 2014; Herman 2014).	Female, Male
Cisgender	Refers to individuals whose gender identity matches their sex assigned at birth. ‘Cis’ is the Latin prefix for ‘on the same side’ (Schilt and Bratter 2015; Herman 2014)
Gender	Gender is a multidimensional construct that has psychological, social, and behavioural dimensions that include gender identity and gender expression (Herman 2014).	Genderqueer, Man, Transgender, Woman
Gender Expression	Gender expression is a behavioural dimension of gender, that is, how one expresses one’s identity through appearance and behaviour (Herman 2014; Spence 2011)	Androgynous, Feminine, Masculine
Gender Identity	Gender identity refers to a person’s internal sense of gender (e.g., being a man, a woman, or genderqueer) and potential affiliation with a gender community (e.g., women, trans women, genderqueer) (Herman 2014).
Gender Minority	An umbrella term that refers to transgender and gender non-conforming people (Institute of Medicine 2011)	Gender non-conforming, Genderqueer, Trans, Transgender
Gender non-conforming (GNC)	Relating to or having a gender identity that is other than man or woman, is a combination of genders, or is on a continuum between two binary genders	Gender Fluid, Gender Non-Binary, Gender Queer
Sex	Refers to biological differences among male, female, and intersex people (hormones, secondary sex characteristics, reproductive anatomy) that can be altered over time through the use of hormones and surgical interventions (Herman 2014; Krieger 2003).	Female, Intersex, Male
Sexual Orientation	Frequently assessed based on three components including sexual behaviour, sexual attraction and identity. The labels are based on the gender relationship between the person and who they sexually engage with in practice or desire.	Bisexual, Gay, Heterosexual Homosexual, Lesbian, Pansexual, Queer, Straight, etc.
Trans (*)	Many use the shorthand ‘trans’ or ‘trans*’ in place of ‘transgender.’
Transgender	Relating to a person whose gender identity does not correspond to that person’s assigned sex at birth (Herman 2014; Feinberg 1996).

In the USA, there are currently six federally funded surveys measuring gender identity in a way that allows gender minorities a response option beyond the binary ( Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity 2016a). Best practice recommendations for survey gender measurement in the USA, with thorough implementation considerations, have emerged in the past three years (Herman 2014; Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity 2016a, b, c), but the work is as yet not well known or adopted. A two-question measure of gender identity constitutes best practice and involves a measure of self-reported assigned sex at birth (ASAB) and a measure of current gender identity. Testing shows that the ‘two step’ approach appears the most likely to have high sensitivity and specificity with adults (Tate, Ledbetter and Youssef 2013; Herman 2014). Ideal question order has not yet been determined (Conron et al. 2014; Herman 2014). Measurement research for youth and adolescents indicates sufficient evidence to include gender expression and ASAB measures in population-based school surveys. Additional considerations are necessary regarding gender identity development, language cognition and privacy (Herman 2014).

Against this background, this article reviews current best practices for gender identity measurement, highlighting current dilemmas and solutions, to broaden current discourse on gender identity measurement development in the USA. Using a triangulation of data sources, including the voices of gender minority research participants not often included in methodological inquiries, the study described here brings together a range of perspectives and raises nuanced critiques of current practices, not previously found in any one place. Highlighting gender measurement challenges and mitigation strategies will allow the public health field to better study, understand, and address gender minority health and health disparities between cisgender counterparts.

Materials and Methods

This qualitative study utilised three sets of data to triangulate results. These data included expert interviews with leaders in gender and sexuality research to better understand researcher perspectives, key informant interviews with transgender people to capture consumer perspectives (voices that are not often heard in this regard), and an analysis (Bowen 2009) of select literature and current survey instruments to understand practice in the field and current conversations about challenges.

In-depth interviews were conducted with ten interdisciplinary experts in gender and sexuality research using a semi-structured field guide. Questions focused on sexual orientation and gender identity (SOGI) research strategies and experiences, SOGI data collection milestones, identification of and reflections on specific data collection dilemmas, and visions for SOGI measurement practices moving forward (see Table 2). Participants were identified through the literature and expert suggestions; 10 of the 15 invited persons participated. Interviews lasted approximately one hour, were conducted in English via Skype, and audio recorded between November 2015 and January 2016. Completed interviews and fieldnotes were reviewed to produce Fair Notes or a more complete version of the fieldnotes including preliminary analysis synthesising data and the researcher’s overall reflections, for the study database (Helitzer-Allen and Allen Jr 1994).

Table 2:

Expert Interview Respondents

#	Academic Credentials	Professional Identity
1	PsyD	Director for LGBT Programs and at a health system
2	PhD	Associate Professor
3	PhD	Clinical Psychologist and Assistant Professor
4	ScD	Associate Professor
5	ScD	Research Scientist
6	PhD	Assistant Professor
7	PhD	Director of an LGBT Health Network Program Adjunct Assistant Clinical Professor
8	PhD	Scholar
9	ScD	Professor
10	JD	Director of Health Services

Eighteen semi-structured key-informant interviews were conducted with English speaking transgender and gender non-conforming (GNC) individuals in New Orleans, Louisiana. Fieldwork used a rapid anthropological assessment approach (Kendall 2008; Hill et al. 2007; Bernard 2017). Snowball and convenience sampling were used, including referrals from local clinics and social media groups. The in-person interviews were conducted within a larger study focusing on discrimination and health care access. Questions focused on the participant’s past experience answering research questions about gender and sexuality. Interviews were audio-recorded, transcribed, and coded after interviews were completed, using an a priori codebook.

In an effort to fully respect the self-determined gender identities of participants, as well as to illustrate how various research questions and categorisation processes yield similar but different responses, Table 3 provides responses to the prompt, ‘How do you currently identify your gender?,’ as well as researcher-ascribed gender labels, ASAB, and sexual orientation data. The researcher ascribed labels attempted to 1) offer simplified gender identity terms, 2) allow for more authentic diversity in identification, and 3) consider past transition experiences and current identities.

Table 3:

Information on Key- Informants’ Gender Identity and Sexual Orientation

#	Gender Identity*	Researcher Ascribed Gender Label**	Assigned Sex at Birth (ASAB)	Sexual Orientation
1	I go with genderqueer or gender fluid.	Genderqueer Person	F	Queer
2	Genderqueer. But with the qualifier of ‘raised as a girl’, generally shuffled into the ‘girl/woman’ box by the world around me, which is certainly an influence.	Genderqueer Person	F	Queer
3	Trans male, so I identify as both male and transgender.	Transgender Male	F	Queer
4	I am a transgender female.	Transgender Female	M	Bisexual
5	That’s a great question. Mostly I don’t. Cause it doesn’t bother me that much. That’s not really true. But in some sense it feels, different. I dunno… Definitely gender non-conforming. I would identify as having a trans identity in that my experience of my own gender and how I would identify, which is probably non-binary and more masculine, more feminine identified male. I don’t know, it’s confusing. At one point I felt like I was failing to be gender queer in some ways and I was like, fuck it, I’m just not going to identify as that.	Gender Non-Conforming Person	F	Queer/dyke
6	My legal gender identity is female…Legal, physical, I’m female… I don’t even use that term [transgender]; I don’t think it technically applies anymore. I’m just a female person.	Female (of transgender experience)	M	Bisexual/ Asexual
7	Male	Male (of transgender experience)	F	Gay or queer
8	Trans, actually I identify as genderqueer. I think to be intelligible in more mainstream institutions I would use the Trans label. In terms of a true identification I identify as being queer. In terms of queer as a category which is critiquing other categorisations like LGBT.	Genderqueer Person	F	Queer
9	Female…. I don’t think I’d identify myself as Genderqueer. I dunno. Maybe I would. I think that term means something that is deeper than what I feel. I feel like my gender is a different way of being a woman that is equally valid to all ways of being female or woman. I think some people who are GQ feel they are not a woman or female or in between or different…I don’t conform to the standard of what women are supposed to look like. My sexuality is lesbian, and my gender is queer, but gender queer implies something that maybe isn’t what I am.	Gender Non-Conforming Woman	F	Lesbian
10	Me and Gonzo are of the same species, we’re whatevers. I know that I am technically not female and, unless they can like, graft ovaries in me or something, I know that I’ m never really gonna be that. But at the same time, I like fashion and I like looking and identifying as female.	Transgender Female	M	Asexual
11	It’s kind of complicated. Usually, I say I’m transgendered because that’s the easiest way to answer it. I mean, I definitely identify on the male spectrum, but I live in-between male and female. It’s a weird place. Not a lot of people choose to do it. It makes life complicated. It’s kind of an evolving thing for me so I don’t really feel like using a category. Generally, I say I’m transgendered, FTM. But I don’t really fit into any category.	Trans-Masculine Person	F	Queer
12	I personally identify as, for data’s sake, I identify as a trans woman of colour. My person term that I coined for myself is FGD, female-gender dominant. So, I like to call myself that a lot.	Trans Woman	M	Queer
13	Genderqueer	Genderqueer Person	M	Queer
14	Female	Female (of transgender experience)	M	Asexual
15	Female…’I’ve had the surgery, I am now a woman.’…I consider myself a woman but I’m also a trans woman I think.	Female (of transgender experience)	M	Pansexual
16	Complicated. I identify on a transmasculine spectrum for sure, but it’s a lot more fluid than that for me and I know that like, whatever the fuck passing means - I do sometimes and I don’ t sometimes. And I don’ t actually care to because I exist on the spectrum between male and female - much farther from female with no aspirations to be a man.	Trans-Masculine Person	F	Queer
17	Female	Female (of transgender experience)	M	Bisexual
18	I’m a guy. Straight up dude all the way.	Man (of transgender experience)	F	Straight

^*In response to the prompt, ‘How do you currently identify your gender?’

^**Gender Label provided for use in report

The document analysis review was initially conducted in 2013 and updated in 2017. Documents were identified via search engines (PubMed and Google scholar), bibliographies of relevant articles, and relevant email listservs. Search terms included ‘sexual orientation and gender identity (SOGI),’ ‘gender,’ ‘transgender,’ ‘gender non-conforming,’ ‘gender minority,’ ‘measurement,’ ‘research’ and ‘methodology’. Identified documents were then reviewed and coded for key themes. Documents were limited to English language publications based on the language abilities of the research team.

Analysis was conducted using a modified grounded theory approach. Expert interview Fair Notes, key-informant interview transcripts, and document analysis passages were reviewed in their entirety. Key themes were identified using an in vivo coding process. Data across sources were sorted to themes by hand using an Excel spreadsheet; memos and summaries of key themes were generated to inform analysis (Bernard 2017).

Results

Results are presented as a series of dilemmas or predicaments that require attention and often have opposing responses or considerations, which emerged from one or more of the data sources in the study. Discussion of key themes related to these dilemmas is included below, including illustrative quotes when relevant.

Moving Beyond Binary Gender Constructions

Any attempt to operationalise gender identity is an attempt to routinise a moving target. The ways gender categories and attributes are understood are not inherently pre-determined, but are developed and agreed upon through society; as norms change and evolve, so do the boundaries and definitions of gender categories. Gender is relational and fundamental to the social structuring of power and privilege (Courtenay 2000). As Currah and Stryker note:

What makes the notion of trans* such a fecund point of departure for work in transgender studies is that the definitional lines of the concept are moving targets. That very instability frustrates the project of fixing embodied identities in time and space—a requisite operation for the potentially life-enhancing project of counting trans populations and better addressing their needs as well as for the necropolitical project of selecting certain members of the population for categorical exclusion as dysgenic (Currah and Stryker 2015).

Respondents discussed being offended by binary gender items.

…basic research psychologists thought, isn’t it fine to just ask people the male-female question? Male coming first, female coming second, even though it isn’t alphabetical, showing the gender bias. So that’s just what was done, and for myself as a trans woman, and other people that I knew that were gender queer and non-binary, that always was bothersome on some more generic level. (Associate Professor, Ph.D)

Many respondents also discussed the ways binary survey items do not allow for accurate reporting of their gender experiences.

None of the forms ever give me the option to say transgender. I leave them blank. I don’t have time… If you don’t ask the right question you won’t get the right answer. (Transgender Female, age unspecified, African American)

I’m just consistently asked to answer in ways that do not describe my experience, which is very troubling. (Genderqueer Person, 44, white)

Conflation of Gender, Sex and Sexual Orientation

Conflating the concepts of sex (denoting biological aspects of an individual) and gender (referring to social constructions) is a common problem in public health research. Researchers commonly use sex categories (female, intersex, male) to pre-populate gender questions (Genderqueer, Man, Transgender, Woman) or use sex variables in data analysis and then report on gender. One study showed that while transgender respondents indicated an understanding of the difference between sex and gender, cisgender respondents did not see much of a difference. Those cisgender people who were able to talk about the concepts separately were those who had special training in gender studies (Lombardi and Banik 2016). This conceptual overlap was described by one of the participants in this study as follows:

Maybe we should be measuring some mixture of the two (sex and gender) - if they are intrinsically connected for folks let’s find out what is actually being measured and it’s the perfect valid measure of THAT- Gex or Sender (Associate Professor, ScD)

Through the two-step method, ascribed biological characteristics and a person’s self-concept in relation to the social construction of gender produce a refined metric that avoids conceptual overlap. However, implementation of this method remains infrequent, and further research related to the application of the measures is needed. As one participant stated:

Surveillance systems haven’t been collecting assigned sex at birth (ASAB) data in a purposeful way, they’ve been collecting something crude where they often offer a couple of options without even a defined question stem, like they’ll ask something like ‘ are you: Male? Female?’ Or, they’ll record male or female based on the sound of a person’s voice…. So, we are at this place where we really need self-reported ASAB questions which are well worded. (Research Scientist, ScD)

Gender identity and sexual orientation are also often conflated (Reisner et al. 2015). While these constructs are related conceptually, they are distinct characteristics, and should not be used interchangeably. Sexual orientation is based on one’s own gender and the gender of their sexual and romantic partners. Taxonomically in the LGBT acronym, one can be both L (lesbian) and T (trans). However, some survey items include these dimensions together. A question might ask ‘Do you think of yourself as: 1) Straight; 2) Gay or lesbian; 3)
Bisexual; 4) Transgender, transsexual, or gender non-conforming (Herman 2014).’ Such an item must be a ‘check all that apply’ response otherwise it forces certain people to select one dimension of their identity over another.

A lot of times you see really rookie stuff, like putting transgender under sexual orientation…I will answer the questions to the best ability that I have, but I don’t know how to answer that. Definitely transgender is not my sexual orientation. But if there isn’t another space for me to mark transgender, and you’re trying to get a count of transgender people, and apply whatever the findings are specific to trans, then I guess I have to mark that. It’s weird to have to move through, because then my answer as a queer person is not being sorted out the same. (Transgender Male, 23, white)

While many LGBTQ people may operate with the above conceptual model, understanding sexual orientation and gender identity to be distinct, it is important to note that some scholars report conceptual models of self-understanding that do not differentiate between these facets. For example, David Valentine’s work with low-income people of colour in New York City, who conceive of gender and sexuality in other terms (Valentine 2007).

Emerging Nature of Gender-Related Language

There are nuances to language surrounding gender and sexuality that have different meaning for scholars and the general population. One example is the word ‘gay’, which can be used to specifically describe male homosexuals or more broadly to describe people of all genders who are homosexual. ‘Queer’ is another example; a word that was used as a pejorative for homosexuals has now been reclaimed by some members of that very group. At the same time, the word ‘queer’ has many meanings, ranging from an umbrella word for sexual and gender minorities, to a verb for disrupting the status quo, to a radical academic discipline or descriptor of non-normative identities and politics. Harrison-Quintana recalls the struggles with wording an item in the 2009 National Transgender Discrimination Survey (NTDS) (Grant et al. 2011),

…someone would say, ‘Remember Janice in Oklahoma? She would be offended by that construction.’ Or ‘How would Kyle in Mississippi answer that one?’ Or ‘If you asked it that way, my trans partner would not relate to it’. (Harrison-Quintana, Grant and Rivera 2015).

The NTDS, which utilised multiple survey items to capture gender identity, included one item with a pre-populated list of 15 gender terms, plus a write-in option asking respondents to indicate the degree to which the term applied to them. The researcher explains that the list, ‘represents a particular moment that is fixed in time, in cultures, and in the communities of the creators, which is why it was so important for us to tap into networks of diverse activists in its creation’ (Harrison-Quintana, Grant and Rivera 2015: 171). Even so, the authors note that the list was not exhaustive.

Another important distinction is that for some people, the term transgender relates more to an experience or process, rather than a label that would apply to their gender identity. For example, someone who currently identifies as a woman, but whose ASAB was male, may select ‘woman’ rather than ‘transgender’ when response options are limited to man, woman or transgender. One respondent described this challenge as follows:

No problem, I’m female. I’m a legal female citizen of the United States. Passport, driver’s licence, birth certificate, the works. I’m female… I’ve emerged from the process, I’m not a transgender person anymore, I’m a female. (Female [of transgender experience], 60, white)

Another respondent explained the dissonance between the language she uses to identify herself and the language she is willing to agree to for data collection purposes:

Well, for data’s sake, I identify as a transwoman of colour. My person term, that I coined for myself, is FGD, female-gender dominant. So, I like to call myself that a lot. (Trans Woman, 38, African American and Native American)

Concerns of Item Sensitivity

The realities of transgender related discrimination are well-established, including but not limited to pervasive verbal and physical harassment and constricted access to opportunities. (Lombardi et al. 2002; Bradford et al. 2013; James et al. 2016). A 2011 Institute of Medicine report lists reluctance of individuals to answer questions about gender non-conformity as one of the three main challenges to LGBT health research ( Institute of Medicine 2011). Gender minorities may not want to disclose themselves for fear of being ‘outed’. Further, as gender identity discrimination policies are not universal, there are real concerns regarding disclosure where anonymity or confidentiality is not guaranteed. Utilising data gathering methods that enhance privacy and anonymity, such as the Computer-Assisted Self-Interview, have proven effective for other sensitive subjects, and may also be helpful in gathering gender data (Villarroel et al. 2006). However, researchers have noted that fear of identity exposure from survey responses was not a concern among their respondents, whether cis or trans. For example, the California Health Information System noted no survey break-offs during or immediately after the gender items (Grant et al. 2015). Researchers commented on the same point, citing other normally asked items as more problematic.

It’s funny to see meeting after meeting where people are really concerned about upsetting people, that people are going to be offended and break off the survey… They’re talking out of their own anxiety about it, because none of that ever happens. Same thing with sexual orientation. That whole conversation happened over and over in regard to sexual orientation. We actually have lower non-response for this type of measure than other demographics, like income. (Scholar, PhD)

Research Fatigue

Members of the gender minority community and scholars have begun discussing research fatigue among gender minority populations. Factors associated with research fatigue include a lack of perceptible change attributable to participation, increasing apathy and indifference toward participation, and practical reasons such as cost and time (Clark 2008).

To date, trans research frameworks have rarely been drafted by trans people and have overwhelmingly centred on pathologies. Many in the community are appropriately wary of surveys, because the limited options presented all too often collapse and marginalise trans experience rather than expand and uncover the richness and complexities of trans lives. (Harrison-Quintana, Grant and Rivera 2015)

Mechanisms for data sharing to limit the amount of times people are asked about the same issues might be relevant. Tools may include creating a public log of research conducted, and creating inter-disciplinary teams with gender minority scholars and cisgender people to ensure survey item quality.

Design and Analytical Limitations

Current best estimates for gender minority prevalence in the US general population range from 100 to 500 per 100,000, (Conron et al. 2012; Collin et al. 2016; Gates 2011). A challenge raised related to transgender data collection is the value of adding precision to measures that only affect a relatively small population. However, federally funded national surveys collect and report data on smaller subpopulations.

Considering that the umbrella group of gender minorities encompasses sub-groups, conducting within group analysis further limits analytical sample size. To address this concern in the context of repeatedly administered general population surveys, one expert (Research Scientist; ScD) suggested aggregation of data over time and sites, which can yield relatively large samples of gender minority respondents, allowing for sub- group analyses.

Sampling methods that may be better suited to exploring specific issues among gender minorities, include respondent driven sampling (RDS), time location sampling (TLS), and targeted sampling (Bonevski et al. 2014). Here, nuanced definitions of gender can be used to determine eligibility. Surveillance surveys often utilise the public health category ‘men who have sex with men (MSM)’ that potentially encompasses separate identities, including cisgender males and transgender women. It is unclear where transgender men and gender non-conforming people fit in such a taxonomy, and men who have sex with men inclusion criteria varies between studies. Inclusion of transgender women under the classification of men who have sex with men may be offensive to gender minorities, and can cause technical problems, such as bottlenecks in RDS recruitment, misclassification for TLS, and non-comparability of data sources. Changes across sites and time in these definitions of eligibility make identification of trends and comparisons of outcome measures difficult.

Categorical and Procedural Consistency

A foundational tenet in study design is to select methods that best address a research question. Utilisation of validated survey items and generation of comparable data is ideal, but not always appropriate. For example, in an investigation concerning discrimination, measuring gender expression non-conformity may be a better attribute than gender identity. However, by tailoring each query uniquely, the problem of data non-comparability can arise.

It’s a big lift to change the way sex data are collected, because for any system, people like using the same questions year after year. (Research Scientist, ScD)

One strategy to use while comparing data is to consolidate data under umbrella categories when appropriate, although between-group comparisons would be lost. A lesbian and bi women category one year might then be collapsed with queer women another year and still reference the same concept, i.e., women with non-exclusively heterosexual sexualities. However, many studies collapse sexual and gender minorities into one category (LGBT), which may be problematic, depending on the research question.

Some respondents argued that the ever-changing language around gender minorities would make any new measure obsolete in a few years. While it is true that language is evolving and new ways of identifying gender are regularly emerging, this does not justify remaining with the status quo:

Abandon the idea of a best practice, there won’t ever be. There will be better practices… A physicist comes and says plutonium has properties that won’t change for millions of years. Well, we’re studying a phenomenon that changes over years. One- two years is the time-scale of change for us. So rather than just hem and haw, and pretend and put our heads down, be like, no, no, we are doing science, so we have to use the same idiotic tool for 90 plus years…NO! We keep changing the tool for the phenomena. (Associate Professor, Ph.D.)

Better practices in gender research are pointing out that many research categories (race, class, or disability for example) are equally unstable, but we stabilise them anyway, “for data’s sake.”

Pre-Populated vs. Open Field

Better gender measurement includes the struggle between open survey items (fields to capture the wide range of terms used to name one’s own gender experience) and pre-populated items. While items with limited response options may work in certain endeavours, the importance of high-quality, open-ended formative research remains significant.

At times it is interesting to collect write-in responses. It is important to do good qualitative work and figure out how people are thinking about themselves and identifying. (Research Scientist, ScD)

From an analytic standpoint, having too many gender categories in a survey item could limit analytic power with small cell sizes. For this reason, even when gender data is available on specific sub-groups, most respondents are frequently re-coded into larger umbrella categories for analysis, and those with less common genders get left out completely. This has been referred to as ‘analytic micro-aggressions’ by Reisner (2015) and others, and can be ethically uncomfortable for researchers: appearing to give respondents the opportunity for self-determination and then taking it away in the generation of any conclusions from such studies. As Thompson and King argue:

Collapsing a wide range of differently gendered subjects under the transgender category makes invisible the economic, political, and social processes that ‘contribute to the marginalisation and invisibility of trans and gender- nonconforming individuals within the HIV prevention and treatment complex (Thompson and King 2015).’

As a way forward, some studies offer a list of possible terms, often generated through formative research (e.g., NTDS). Another solution offered by a participant (Research Scientist; ScD), is to capture at least three main transgender groups: MtF, FtM, and GNC (See Table 1), though concerns of diversity within these groups still remain, particularly with the GNC group which would include people assigned male and female at birth. Other participants advocated for offering three gender options: man, transgender, and woman. However, these options will miss those people who have transitioned genders and currently only identify with their current gender.

But some people who have that profile or gender experience (trans or GNC) are not going to identify if you give them an explicit transgender option, because that is not how they identify, which is legit. (Associate Professor, Ph.D.)

It’s like a butterfly, it starts out as a larva, a caterpillar, and then it goes through the process of metamorphosis. Technically, while it’s in change, it’s a metamorph, when it gets through with the change, it emerges as a butterfly, you don’t call it a metamorph anymore, you call it a butterfly. I’ve emerged from the process, I’m not a transgender person anymore, I’m a female. At one point I was in the process of transition, I was transgender but now that’s behind me. I’m just me. (Female [of transgender experience], 60, white)

Potential Misclassification

Introducing new categories to the general population can create confusion. Consider the case of gender non-binary identities; words such as genderqueer, gender-non-conforming, and non-binary have become popular and passé at different moments and can be hard for cis-gender people to keep up with. There is also fear that cis-gender respondents will be offended or confused by the question and will exit the survey or answer incorrectly. One researcher (Director of an LGBT Health Network Program; PhD) cautioned that we “need to be very careful when using new words”. Misclassification of even a few cisgender people into a gender minority category can have an important impact on data quality, as the percentage of gender minorities in a general population is thought to range from 0.3%– 0.5% (Conron et al. 2012; Gates 2011).

Competing Data Collection Needs

Survey space restrictions, time concerns, and funding availability are realistic obstacles raised by study respondents to changing or adding survey items on an existing instrument. Various model questions have been developed to capture gender identity, and while the two-step model is gathering endorsements as a best-practice model, other single measure items do exist to capture gender minority status (Herman 2014) and may be an alternative to adding an additional survey item. Expert interview respondents stated that researchers and community members must continue to educate funders and research gatekeepers- those with access to “target populations”- on the importance of capturing such data.

Conclusions

This paper serves to provide an overview of dilemmas towards better practices in gender identity measurement, including strategies for mediating relevant challenges. In so doing, it complements catalogues of existing survey items for gender identity (Herman 2014; Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender Identity 2016a, b).

It is important to keep in mind that these dilemmas and strategies may apply differently in various data collection endeavours. Measurement at the national level requires simple user-friendly items, while targeted and smaller scale projects could be asking more nuanced questions and the meaning of diverse categories identified. Differentiating general population-based surveys from other kinds of targeted surveys, such as RDS or web-based convenience samples that address gender minorities directly, will guide selection of survey items. The specifics of measures used on a particular survey can depend on several factors: mode of administration, flexibility of the survey administrators to add multiple measures, the sample size, if it is a one-time or ongoing survey, the age range of the sample, and the analysis plan (Herman 2014). Considering these factors will guide the researcher in choosing the most appropriate measure for the situation. It is worth noting that each study design can yield unique and complementary insight in gendered health disparities.

It is important to ensure that new survey items are properly constructed and tested. Of particular concern is polysemy, terms meaning different things to different people, and the impact of misclassification for small populations. For these reasons high quality formative research must be conducted to assess criterion and construct validity in various sub-samples. However, cognitive research takes time to conduct and publish. Conron et al. (2014) note that limited cognitive research on such measures has been conducted and offer a table of Cognitively Tested Brief Gender Measures Published in Peer-Reviewed Journals. While the burden of conducting such tests individually may be great, using resources like Conron’s table, previously tested survey items from the Gender Identity in US Surveillance (GenIUSS) group can help alleviate the burden. Concerns arise that these measures and resources will remain timely as gendered language and constructs continue to evolve. Continued engagement with these issues can help ensure the most current information is informing measurement selection.

Towards better research transparency, when measuring gender related phenomenon, be explicit about survey items used and why they were chosen. When engaging with complex measurable understandings of gender, it is important to be clear how these understandings are constructed. In contrast, when binary conceptual models are used, this should be noted as a study limitation.

Involving communities in many stages of the research process, from research question and item development, to analysis and dissemination is important for better practices. Examples of such studies include the NTDS and the TransPop study, which have successfully enhanced the quality of their work through participatory approaches (Harrison-Quintana, Grant and Rivera 2015). Working across disciplines is also important to ensure a broad audience, sharing of emerging methodological approaches, avoiding disciplinary silos, and avoiding repetitive studies leading to research fatigue.

This study does have some limitations. A larger sample of experts may have revelled additional dilemmas, although care was taken to select those scholars most engaged with these issues. The small size, local bias, and lack of cis-gender people in the consumer sample are also limitations. All key informants were residents of New Orleans, although the group did show diversity in terms of age, race, ethnicity, class, gender identity (although no cis people), and sexual orientation, and provided a wide range of responses with saturation on key points. Moving forward, including cisgender perspectives on these topics will be important.

This paper advocates for updating gender measurement in all data collection activities to include gender minorities. The generation of reliable gender minority data allows for more sophisticated analysis around health and social disparities, and can catalyse great change for these populations via policy shifts, resource allocation, and intervention development. Researchers have a moral imperative to conduct the highest quality research to address inequities, especially in a population that suffers a high level of discrimination and health disparities. At a time when support for empirical science about many topics is under attack, strong scientific arguments for inclusion of gender minorities are required. To that end, skills building and gender sensitivity training aimed at public health practitioners and the research community are necessary to advance the field, which is already known to have a deficit in terms of student training on these topics (Talan et al. 2017). Until there is more general recognition of the importance of comprehensive and accurate gender data, which includes capturing gender minorities, continued advocacy for improved measures is necessary. We hope this paper contributes to that end.