Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Apr 1.
Published in final edited form as: Front Neuroendocrinol. 2020 Feb 15;57:100835. doi: 10.1016/j.yfrne.2020.100835

Sex bias and omission in neuroscience research is influenced by research model and journal, but not reported NIH funding

Gabriella M Mamlouk 1, David M Dorris 1, Lily R Barrett 2, John Meitzen 1,3
PMCID: PMC7225067  NIHMSID: NIHMS1565548  PMID: 32070715

Abstract

Neuroscience research has historically demonstrated sex bias that favors male over female research subjects, as well as sex omission, which is the lack of reporting sex. Here we analyzed the status of sex bias and omission in neuroscience research published across six different journals in 2017. Regarding sex omission, 16% of articles did not report sex. Regarding sex bias, 52% of neuroscience articles reported using both males and females, albeit only 15% of articles using both males and females reported assessing sex as an experimental variable. Overrepresentation of the sole use of males compared to females persisted (26% versus 5%, respectively). Sex bias and omission differed across research models, but not by reported NIH funding status. Sex omission differed across journals. These findings represent the latest information regarding the complex status of sex in neuroscience research and illustrate the continued need for thoughtful and informed action to enhance scientific discovery.

Keywords: sex bias, sex omission, sex reporting, sex difference, women’s health, male, female, neuroscience

Graphical Abstract

graphic file with name nihms-1565548-f0001.jpg

1. Introduction

Historically, sex bias and omission pervaded research subject selection in neuroscience research. Sex bias is defined as the favoring of one sex over another, and neuroscience research has consistently favored the use of males over females (Beery and Zucker 2011; Berkley 1992; Mogil and Chanda 2005; Shansky and Woolley 2016; Will et al. 2017). Sex omission is defined as the lack of reporting research subject sex. Sex bias and omission are not limited to neuroscience, but are present in a number of scientific biomedical fields (Beery and Zucker 2011; Bryant et al. 2018; Potluri et al. 2017; Stephenson et al. 2019; Yoon et al. 2014). While there are appropriate instances when one sex should be employed instead of another, the lack of appropriate justification for single sex use, both historically and in contemporary studies, and the widespread neglect of females have prompted considerable formal and informal debate across a broad spectrum of the biomedical scientific effort, including and beyond neuroscientists. This ongoing discussion has produced new granting agency and journal regulatory policies, editorials, assessments of sex bias and omission, and new scientific advancements benefiting women and men (Bale 2019; Becker et al. 2005; Becker et al. 2016; Beltz et al. 2019; Brooks and Clayton 2017; Cahill and Aswad 2015; Clayton and Collins 2014; Duchesne et al. 2017; Eliot and Richardson 2016; Fields 2014; Geller et al. 2018; Guizzetti et al. 2016; Joel and McCarthy 2017; Johnson et al. 2014; Karp et al. 2017; Klein et al. 2015; Kokras et al. 2019; Liu and Mager 2016; Maney 2016; McCarthy 2015; McCullough et al. 2014; McEwen and Milner 2017; Miller et al. 2017; Mogil 2016; Panzica and Melcangi 2016; Park et al. 2015; Richardson et al. 2015; Ruigrok et al. 2014; Shansky 2019; Tannenbaum et al. 2019; Tannenbaum et al. 2016; Zakiniaeiz et al. 2016).

For continued informed and thoughtful debate and action, it is crucial to understand the complex nature of sex bias and omission in contemporary neuroscience research. It is particularly critical to focus on research articles, given that this published unit is the final common output of academic neuroscience research. The last comprehensive analysis of neuroscience research articles examined studies published between 2010 and 2014 that employed mice and rats (Will et al. 2017). This previous literature review discovered that between 2010 and 2014, neuroscience research articles showed a decrease in sex omission. However, sex bias remained present, as increasing numbers of articles between 2010 and 2014 reported the sole use of males. The number of articles employing both males and females also increased, but relatively few assessed sex as an experimental variable. This lack of addressing sex as an experimental variable, as well as the underrepresentation of females in research, are missed opportunities for new discoveries relevant to sex-specific neurological phenomena and enhancing scientific reproducibility. It is also the responsibility of scientific researchers to do their best to employ representative research populations.

Assessments of sex bias and omission in neuroscience studies published after 2014 are unavailable, and it is likewise unknown whether sex bias and omission vary in research models other than mice and rats and by reported NIH funding status. NIH funding status is a point of interest due to the implementation of relevant NIH policies in 2016. Here we filled these knowledge gaps by providing an updated evaluation of sex bias and omission in the neuroscience literature. We focused on all research articles using any research model published in 2017 in the following journals: Journal of Neuroscience, Journal of Neurophysiology, Nature Neuroscience, Neuron, Nature, and Science. These journals were chosen given their prominence in the neuroscience field and also to align with previous studies (Beery and Zucker 2011; Shansky and Woolley 2016; Will et al. 2017). We assessed sex bias and omission in the overall literature and in the context of research model, reported NIH funding status, and journal.

2. Literature Review Search Methods

2.1. Inclusion Criteria and Coding of Articles

Research articles published in 2017 were analyzed from the following journals: Journal of Neuroscience (impact factor: 5.971, ranked 30/261 in the neurosciences category,), Journal of Neurophysiology (impact factor: 2.502, ranked 167/261 in the neurosciences category), Nature Neuroscience (impact factor: 19.912, ranked 2/261 in the neurosciences category), Neuron (impact factor: 14.319, ranked 7/261 in the neurosciences category), Nature (impact factor: 41.577, ranked 1/64 in the multidisciplinary sciences category), and Science (impact factor: 41.058, ranked 2/64 in the multidisciplinary sciences category). Journal impact factor and rankings were from 2017 and obtained from InCites Journal Citation Reports (Clarivate Analytics, Philadelphia, PA). All articles published within 2017 were examined by a team of 4 trained curators (2 female, 2 male) to eliminate sampling bias. Trained curators were employed because the divergent and extensive vocabulary used to describe animal sex and its treatment as an experimental variable make automated text mining approaches challenging. Inter-curator reliability was monitored as in a previous study (Will et al., 2017). Following a previously employed protocol (Will et al. 2017), articles were first determined to be primary research articles by the curators. Reviews, editorials, and similar non-primary research articles were excluded from analysis. Articles were then analyzed for neuroscience relevance. Articles from the Journal of Neuroscience, Journal of Neurophysiology, Nature Neuroscience and Neuron were automatically accepted as neuroscience relevant. A broad inclusion criterion was employed for articles from Nature and Science. Articles in these journals were included for analysis if the article topic encompassed any aspect of the nervous system, ranging from the molecular to behavioral level of analysis. In all journals, articles addressing purely computational or theoretical aspects of the nervous system were excluded from analysis. In all journals, articles using embryonic animals and primary neuron cell cultures were analyzed similar to previous studies (Taylor et al. 2011; Will et al. 2017), as were articles using immortalized cell lines (Ben-Yosef et al. 2012; Shah et al. 2014). This inclusion paradigm resulted in 1827 neuroscience articles. Articles were then coded for research model. Fifty different research models were represented in the overall dataset and were categorized as follows: amphibia (including Xenopus laevis, salamanders, other frogs), aplysia, bats, C. elegans, cats, immortalized cell lines, crabs, drosophila, ferrets, fish: other (including electric fish, goldfish, skates), gerbils, guinea pigs, humans, insects: other (including bumblebees, cockroaches, locusts, honeybees, crickets, stick insects), invertebrates: other (including leeches, squid, planarians, nudibranch), lampreys, mammals: other (including dogs, sheep, armadillos, opossums), mice, non-human primates (including chimpanzees, marmosets, monkeys), non-oscine birds (including chickens, owls), oscine birds (including finches, starlings, canaries, ravens), rabbits, rats, rodents: other (including voles, naked-mole rats, chinchillas), turtles, zebrafish. “Other” categories were employed to group species with low representation. Articles using embryonic animals and primary cell cultures were coded as the species of origin, following a previous study (Will et al. 2017). For articles employing embryonic and postnatal animals of the same species, only the postnatal animals were included in analysis. Articles using immortalized cell lines were coded as a separate research model, immortalized cell lines, given the unique aspects of this research model that are shared across origin species. For articles using multiple species, each species was recorded individually. This protocol further decomposed the articles into a pool of 2,167 entries categorized by research model. Entries were then coded for NIH funding status, including all agencies of the NIH. All non-NIH funded entries were collapsed into a single category and were not further characterized. Entries were then analyzed for sex. Sex categories were: male, female, hermaphrodite, male and female wherein biological sex was considered an experimental variable, male and female wherein biological sex was not considered as an experimental variable, and sex not reported. A broad inclusion criterion was employed regarding sex reporting. Articles were considered to have addressed sex as an experimental variable if any formal statistical comparison or assertion of such a comparison of males and females was performed, including if the data or analysis was not shown and including whether sex differences were detected or not. The use of sex as a covariate was considered sufficient for addressing sex as an experimental variable. Very few articles reported data disaggregated by sex but did not perform or assert to have performed a statistical comparison. These articles were coded as not having addressed sex as an experimental variable since there was no comparison. When distinct experiments within an article differentially reported sex in the same research model, articles were coded as having reported sex for that model. For example, if one experiment did not report sex in rats, but another experiment reported employing male rats, the article was coded as “rats: male only.” When distinct experiments within an article employed different sexes, articles were coded male/female with biological sex not considered an experimental variable, following previous studies (Beery and Zucker 2011; Will et al. 2017). Data were analyzed via Chi-squared tests (Prism version 6.07, GraphPad Software, La Jolla, CA). P values < 0.05 were considered a priori as significant. Data are presented as percentages.

3. Results

3.1. Demographics of neuroscience articles published in 2017

The demographics of the neuroscience publishing landscape in 2017 exhibited variety across a number of different parameters. This included the employed research model (Figure 1A). Mice were the most commonly employed research model (36%), followed by humans (25%), rats (16%), non-human primates (7%), and immortalized cell lines (6%). Together, mice and rats comprised ~60% of the employed research models. Across all analyzed articles, 57% reported NIH funding, while 43% did not (Figure 1B). Articles were obtained from six different journals, with the Journal of Neuroscience representing the largest proportion (46.9%), followed by Neuron (18.8%), Journal of Neurophysiology (18.0%), Nature Neuroscience (7.1%), Science (4.7%), and Nature (4.6%) (Figure 1C).

Figure 1.

Figure 1.

Open in a new tab

Research model, NIH funding status, and journal of neuroscience articles published in 2017. A) Reported research models. Mice were the most commonly employed species, followed by humans, rats, non-human primates, and cell lines. B) Reported NIH funding status. A majority of articles reported NIH funding. C) Journal. The majority of articles were published by the Journal of Neuroscience, followed by the Journal of Neurophysiology, Neuron, Nature Neuroscience, Science and Nature.

3.2. Sex bias and omission in neuroscience studies

Articles were categorized as either not reporting sex, reporting both males and females with sex considered as an experimental variable, both males and females with sex not considered an experimental variable, only males, only females, or hermaphrodites (Figure 2A). Articles employing males and females without considering sex as an experimental variable comprised the largest percentage of the data set (44%), followed by male only (26%), sex not reported (16%), males and female with sex considered an experimental variable (8%), female only (5%), and hermaphrodites (1%). These percentages indicate that for the first time, males and females analyzed together comprise a slim majority of neuroscience literature. However, a relatively low number of articles reported evaluating sex as an experimental variable. This low number of articles was best illustrated when only articles that employed both males and females were analyzed. Of articles employing both males and females, only ~15% of articles incorporated sex as an experimental variable (Figure 2B). This analysis indicates that while there is documentation of the use of both males and females, most studies still do not report analyzing sex as an experimental variable. This analysis, coupled with data presented in Figure 2A, indicates that sex bias and omission remain present in the neuroscience literature, albeit exhibiting a more complex presentation than previously reported.

Figure 2.

Figure 2.

Open in a new tab

Distribution of sex bias and omission in neuroscience articles published in 2017. A) The majority of studies employed both males and females. Sex bias remains present; studies that employed only males made up the second largest proportion of the dataset. Sex omission also persists; articles not reporting sex comprised the third largest proportion of the dataset. B) The majority of articles using both male and female animals do not report analyzing sex as an experimental variable. Of articles employing both males and females, only ~15% of articles incorporated sex as a biological variable. Articles were characterized as reporting sex as a variable if any statement or statistical test indicated that data from males and females were compared, regardless of outcome and whether or not data were reported.

3.3. Sex bias and omission varied considerably by research model

Sex bias and omission varied considerably across research models (Figure 3A), consistent with previous analyses of older literature (Beery and Zucker 2011; Will et al. 2017). Research models differed in every facet of sex bias and omission, including the proportion of articles that reported males and females without considering sex as an experimental variable (χ2=498.5, p<0.001), the proportion that did consider sex as an experimental variable (χ2=171.1, p<0.001), the proportion that solely employed males (χ2=691.3, p<0.001), females (χ2=967.3, p<0.001), hermaphrodites (χ2=1316.0, p<0.001), and the proportion that did not report sex at all (χ2=578.7, p<0.001). This diversity can be exemplified by focusing on the top three commonly used research models: mice (35%), humans (25%) and rats (16%). Regarding mice (Figure 3B), 49% of articles reported using males and females without considering sex as an experimental variable. 9% of articles reported using males and females and considered sex as an experimental variable, 29% solely employed males, 4% solely employed females, and 9% did not report sex at all. Humans exhibited a different pattern (Figure 3C). 73.5% of articles reported using males and females without considering sex as an experimental variable, while 12% reported using males and females and considered sex as an experimental variable, 6% solely employed males, 1.5% solely employed females, and 7% did not report sex at all. Continuing this disparate pattern between research models, only 22% of rat articles reported using males and females without considering sex as an experimental variable (Figure 3D). 5% of articles reported using males and females and considered sex as an experimental variable, 53.5% solely employed males, 6.5% solely employed females, and 13% did not report sex at all.

Figure 3.

Figure 3.

Open in a new tab

Sex bias and omission vary considerably by research model. A) All research models. Articles were coded into twenty six different research model categories. Some research model categories exhibited considerable sex bias, such as ferrets and oscine birds, while others did not, such as gerbils. Other research model categories generally neglected to report sex, most notably immortalized cell lines, while some research model categories widely report sex, such as rabbits. B) Mice. C) Humans. D) Rats.

These differences between research models persisted beyond the top three categories. Regarding sex bias, on one end of the spectrum are categories that exhibit an extreme sex bias, such as ferrets (100% female) and oscine birds (82% male). We note that there may be scientific justification for single sex use. For example, finches, which are oscine birds, only the males normally exhibit song behavior and the associated neural substrate, and thus it is not necessarily erroneous for articles to employ solely males (Zhang et al. 2017). On the other end of the spectrum are research models that exhibit considerable integration between females and males, including humans (74% male and female: sex not considered an experimental variable) and mammals: other (100% male and female: sex not considered an experimental variable).

Regarding sex omission, select research models largely neglect to report sex, including immortalized cell lines (88% sex not reported), and non-oscine birds (77% sex not reported). These large percentages of studies that fail to report animal sex reflect challenges specific to these model systems. For example, regarding cell lines, some scholars have argued that commercial vendors fail to sufficiently describe the sex of the supplied cells (Park et al. 2015), while many of the studies that do not report sex in the non-oscine bird category employed chick embryos (Shao et al. 2017). Chick embryos are morphologically indistinct and require genetic techniques to sex (He et al. 2019). Some research models exhibited little to no sex omission, such as aplysia, gerbils, rabbits, and oscine birds. Many of these research model categories with low sex omission include relatively small numbers of studies, hence their poor representation when studies are analyzed independent of research model. Overall, this analysis indicates that there is considerable diversity in sex bias and omission across research models.

3.4. Sex bias and omission did not vary by reported NIH funding

Given the advent of the National Institute of Health (NIH) Sex as a Biological Variable (SABV) (NOT-OD-15–102) regulatory policy on January 25, 2016 (Clayton and Collins 2014), a relevant question is whether sex bias and omission differed between NIH and non-NIH funded studies. To address this question, we coded articles as funded or not funded by any agency of the NIH and analyzed sex bias and omission (Figure 4). 43% of articles that reported NIH funding, versus 46% of articles that did not report NIH funding, employed males and females without considering sex as an experimental variable (χ2=0.10, p>0.05). 9% of articles that reported NIH funding, versus 6% of articles that did not, employed males and females and considered sex as an experimental variable (χ2=0.60, p>0.05). 26% of articles that reported NIH funding, versus 26% of articles that did not, solely employed males (χ2=0.00, p>0.05). 5% of articles that reported NIH funding, versus 5% of articles that did not, solely employed females (χ2=0.00, p>0.05). 1% of articles that reported NIH funding, versus <1% of articles that did not, employed hermaphrodites (χ2=0.00, p>0.05). 16% of articles that reported NIH funding, versus 17% of articles that did not, did not report sex at all (χ2=0.03, p>0.05). This analysis indicates that there is little to no evidence that sex bias and omission differed by reported NIH funding status.

Figure 4.

Figure 4.

Open in a new tab

Sex bias and omission do not vary by reported NIH funding. Levels of sex bias seem similar between articles reporting and not reporting NIH funding. Likewise, the proportion of articles reporting sex does not vary by reported NIH funding.

3.5. Sex omission varied between journals

Thus far, all analyses have analyzed sex bias and omission independent of article journal. Previous analyses of older literature have detected differences in sex reporting between scientific journals and groups of scientific journals across distinct biomedical subfields, including in the journals assessed here (Beery and Zucker 2011; Potluri et al. 2017; Will et al. 2017). Thus, articles were analyzed by their journal to assess whether differences in sex reporting persist between journals (Figure 5). Journals differed in sex omission characteristics. The proportion of articles that did not report sex at all varied by journal (χ2=38.1, p<0.001). Journals only varied in one select aspect of sex bias, the proportion of articles that reported males and females without considering sex as an experimental variable (χ2=12.4, p=0.029). There were no differences detected between journals in the proportion of articles that reported males and females and considered sex as an experimental variable (χ2=3.565, p=0.614), the sole use of males (χ2=1.360, p=0.929), females (χ2=2.097, p=0.836), or hermaphrodites (χ2=4.000, p=0.550). Importantly, in three out of the six journals analyzed, the majority of articles reported employing both males and females independent of whether sex was considered an experimental variable, ranging between 53–57%. In two journals, articles reporting the use of both males and females represent a plurality (44–47%). Overall, this analysis indicates that sex omission considerably varies across journal, as does the reporting of the use of sex as an experimental variable. Levels of sex bias seem to be similar across journals.

Figure 5.

Figure 5.

Open in a new tab

Sex omission but not sex bias varies by journal. The proportion of articles reporting sex varies by journal, as does the proportion of articles that employ male and females without considering sex as a biological variable. Levels of sex bias seem similar between journals. In three out of the six journals analyzed, the majority of articles reported employing both males and females independent of whether sex was considered an experimental variable. In two journals, articles reporting the use of both males and females represent a plurality. Across all journals, the proportion of articles solely employing males is larger than the proportion of articles solely employing females.

Focusing on sex omission, the spectrum of the lack of sex reporting ranges from 10% in Nature Neuroscience to 44% in Science. We note that the relatively small number of neuroscience articles published in Science and Nature compared to other journals means that individual articles can more heavily influence percentage values. A more reasonable comparison may be between Science and Nature, whose articles comprise similar percentages of the overall dataset (Figure 1C; Science: 4.7%; Nature: 4.6%). These two journals differed considerably in terms of the percent of articles that did not report sex (Science: 44%; Nature: 24%) and the percent that employed males and females without considering sex an experimental variable (Science: 21%; Nature: 37%). Other values were more similar. This comparison indicates that sex omission still varies even among journals with similar overall article numbers, arguing for a powerful role of individual journal policies and enforcement.

4.0. Discussion

The key finding of this literature review is that sex bias and omission persist in neuroscience studies but exhibit a complex presentation that resists easy generalizations. Regarding sex omission, an overall rate of 16% of articles did not report research subject sex. While sex bias is still present in the neuroscience literature, for the first time since monitoring began, a majority of research articles reported the use of both females and males, which may indicate a weakening in overall sex bias. However, this encouraging decline in sex bias is diminished by two key points of concern. First, most studies that employed females and males did not consider sex as an experimental variable, and second, many more studies reported the sole use of males compared to females. These findings indicate that while across the neuroscience literature a majority of studies employed both males and females, the actual investigation of sex as an experimental variable remained relatively rare. This picture is further complicated by the considerable variation in sex bias and omission between animal models and journals. There is no evidence that sex bias or omission varied by reported NIH funding status.

4.1. Sex omission decreased over time

This study found that approximately 16% of articles published in 2017 did not report sex across all research models and journals. An important question is whether the intensity of sex omission is changing across time and whether this rate of sex omission has decreased compared to prior years (Figure 6). Importantly, a previous study using nearly identical methodology that assessed neuroscience articles in the same six journals tracked sex omission between 2010 and 2014 (Will et al. 2017). This previous study limited analysis to manuscripts that employed rats and mice. Combining this previously collected data with that from rat and mouse data obtained during the current study allows for the assessment of whether the intensity of sex omission has changed across time. In 2010, Will and colleagues detected a sex omission rate of 47.1%, meaning that nearly half of the studies assessed did not report sex. This high percentage of studies was similar to that reported by other assessments of neuroscience literature from similar or just preceding time periods (Beery and Zucker 2011; Shansky and Woolley 2016), validating the curator-based approach. The rate of sex omission then dramatically decreased between 2010 and 2013, falling to a rate of 16.6% in 2013. This rate has further decreased to 10.3% in articles employing mice and rats, as documented by the current study. This decrease is encouraging, however, 10.3% of papers exhibiting sex omission remains an unacceptably high number for an essential experimental component necessary for successful replication. Overall, this analysis indicates that sex omission has decreased, and the dramatic drop in sex omission that occurred between 2010 and 2013 has not just been maintained, but also improved through 2017. Worryingly, this finding may be indicative of a trend in which the issue of sex omission is replaced by a failure to consider sex as an experimental variable.

Figure 6.

Figure 6.

Open in a new tab

Changes in sex bias and omission across time in rats and mice. Data obtained from rats and mice from both the current study and a previous study that analyzed the same journals (Will et al. 2017) allows for assessment of sex omission and bias between 2010 and 2017. Sex omission dramatically decreased from 2010 to 2013, and then continued to decrease from 2013 to 2017. Sex bias presents a complex presentation. More articles increasingly report the use of males and females between 2010 and 2017, reaching a plurality by 2017. Only a relatively small but growing proportion of articles include evaluate sex as an experimental variable. However, the proportion of articles solely employing males remains much higher than articles solely employing females.

4.2. Sex bias remained present but exhibited a complex and changing presentation

Sex bias presented a complex presentation in 2017 when considered across all research models. Considering articles that report the use of mice and rats in order to enable analysis across time as discussed in section 4.1, more articles increasingly reported the use of both males and females between 2010 and 2017, reaching a plurality by 2017. The percentage of mouse and rat studies employing males and females without considering sex as an experimental variable dramatically increased from 12.9% in 2010 to 32.4% in 2013, reaching 40.8% in 2017. Only a relatively small but growing proportion of articles actually evaluated sex as an experimental variable. The use of sex as an experimental variable marginally increased from 4.3% in 2010 to 5.3% in 2013, and later nearly doubled to 8.1% in 2017. This illustrates an important point: the inclusion of females by itself does not provide scientific insight unless sex is used as an experimental variable. Pooling data across sex essentially ignores sex as an experimental variable, which is a missed opportunity for scientific discovery. Though sex as an experimental variable is still being largely neglected, the increased proportion of the use of both males and females is encouraging. The methodology of both the current study and the previous study used broad criteria for whether sex was considered as an experimental variable. Therefore, these reported proportions are the most optimistic representation of the analyzed articles. Even with employing this most optimistic analysis, sex bias remained present in the neuroscience literature. The proportion of articles solely employing males is much higher than articles solely employing females. The exclusive use of males varied from 2010 to 2017. In 2010, 31.1% of studies used males only. This proportion increased to 39.3% in 2013 and later decreased to 36.3% in 2017. The exclusive use of females also fluctuated. In 2010, 4.6% of rat and mouse studies solely employed females. The number increased to 6.4% in 2013 and later decreased to 4.5% in the literature published in 2017. Overall, this analysis indicates that sex bias may be weakening in the neuroscience literature. This progress is diminished by the relative scarcity of studies that employ both females and males and consider sex as an experimental variable, and second, the much greater incidence of studies that report the sole use of males compared to females. There remains much potential for improving how sex is incorporated and evaluated in neuroscience experiments.

4.3. Role of the Research Model

The temporal analysis of sex bias and omission presented in sections 4.1 and 4.2 was limited to studies employing rats and mice. This restriction was primarily present to align the findings of the current study with those of the previous study (Will et al. 2017), which only examined articles employing mice and rats, similar to previous work (Beery and Zucker 2011; Florez-Vargas et al. 2016; Mogil and Chanda 2005; Shansky and Woolley 2016). Mice and rats remained the most predominate non-human species employed in neuroscience research (together totaling 51.5%), extending findings of previous analyses to 2017 (Beach 1950; Beery and Zucker 2011; Lambert et al. 2019; Manger et al. 2008; Will et al. 2017). Not surprisingly, after mice, humans were the second most commonly reported research model (24.6%). Many relevant studies exist that discuss the role, inclusion, and assessment of sex in human non-clinical and clinical research (Day et al. 2017; Geller et al. 2011; Geller et al. 2018; Liu and Mager 2016; Sugimoto et al. 2019), including the possible influences of scientific author gender on research model sex and article citation rates (Andersen et al. 2019; Nielsen et al. 2017). The remaining proportion of articles represented an impressive array of research models (23.9%), demonstrating the value of comparative neuroscience research (Brenowitz and Zakon 2015; Krebs 1975; Remage-Healey et al. 2017). Each of these research models presents specific challenges regarding sex bias and omission. For example, immortalized cell lines, which our study considered its own category independent of species origin, present challenges different than those of other research models (Arnold and Disteche 2018; Potluri et al. 2017; Ritz 2017; Shah et al. 2014; Taylor et al. 2011), including chromosomal alteration such as the loss of either a Y or X chromosome (Xu et al. 2017). Other research models present formidable methodological obstacles to incorporating both sexes. One example is ferrets. Unique among research models, only female ferrets were used in the articles assessed by this study. This may be largely because of the aggression and unpleasant odor of male ferrets (Vinke et al. 2008), making them difficult to study within a laboratory setting. For some research models, most prominently non-human primates, cost and availability are considerable challenges, but when possible, assessment of sex in primate systems has yielded valuable insights (Gervais et al. 2019; Lonsdorf 2017). Some research models, including select teleost fish, exhibit functional sex change, even in adulthood (Liu et al. 2017). Other research models include hermaphrodites, such as C. elegans and aplysia. Sex differences have been detected between hermaphrodites and males in C. elegans (Honjoh et al. 2017), and the presence of males in a population has also been demonstrated to impact the lifespan of hermaphrodites (Maures et al. 2014). Other research models feature a sex-specific behavior that is of interest regarding neural function. As discussed above, oscine songbirds such as finches are a prominent example of a species exhibiting a sex-specific behavior (Zhang et al. 2017), although songbirds in general have been highly useful for understanding sex steroid hormone action in both males and females (Balthazart et al. 2018; Brenowitz and Remage-Healey 2016; Saldanha et al. 2011). Of course, many research models, including rats and mice, exhibit behaviors that are of interest for a particular sex. Employing one sex in a research study can be entirely appropriate, as long as there is proper justification for single sex use. Examples of proper single sex use include studies of the effects of maternity on the female nervous system (Duarte-Guterman et al. 2019) or of the spinal nucleus of the bulbocavernosus/Onuf’s nucleus in males (Sengelaub and Forger 2008). Other valid examples also exist. However, as documented in the current study, an overall proportion of 26% of articles only employing males versus 5% of articles only employing females seems highly unbalanced. Moving forward, policies focused upon improving sex bias should incorporate considerations for incentivizing the use of females and males, along with flexibility for the unique challenges posed by individual research models. These efforts should include education regarding best practices for incorporating sex as an experimental variable within individual research studies and overall research programs, as well as proper justifications for single sex use. Several excellent resources now exist for scientists seeking guidance with statistical and methodological approaches for assessing the role of sex (Bale and Epperson 2017; Becker et al. 2005; Beery 2018; Beltz et al. 2019; Buch et al. 2019; Miller et al. 2017).

4.4. Role of NIH funding

One question assessed by this manuscript was whether differences in sex bias and omission were associated with NIH funding status. This is a key question, as it is relevant to discussions of whether the NIH Sex as a Biological Variable (SABV) (NOT-OD-15–102) regulatory policy is influencing biomedical studies. Interestingly, no differences were detected in sex bias and omission between manuscripts published in 2017 that reported or did not report NIH funding. This lack of evidence indicates that NIH funded studies are not yet qualitatively different in sex bias or omission than studies that are not funded by the NIH. This lack of effect could be due to several factors, of which our study was not designed to differentiate. One potential factor is that NIH funding status exerted minimal influence on rates of sex bias and omission in studies published in 2017. Related to this potential factor, a recent study found no large increase in the number of Canadian Institute of Health Research (CIHR) grants examining sex and/or gender from before and after the implementation of a similar policy at the CIHR (Galea et al. 2020). Also relevant are findings from Woitowich and colleagues, who surveyed NIH reviewers serving in 2016–2017 (Woitowich and Woodruff 2019). Woitowich and colleagues found that 61% of reviewers serving in 2017 indicated that SABV was consistently incorporated into a proposal’s approach score. Given this finding, it is likely that the incorporation of SABV into reviewing grant proposals was still dynamic during the 2017 timeframe, especially considering that only 15% of 2017 NIH reviewers participated in the study. Importantly, grants reviewed in 2017 would not impact papers published in 2017. This delay leads us to the next potential factor: the effects of SABV have not yet manifested in the neuroscience literature. SABV was implemented on January 25, 2016 for newly submitted grant applications (Clayton and Collins 2014) and features a set of standards (Clayton 2018) that may take months or years to manifest in publications describing data produced under these grants. If this delay is the case, then the data published here will be key for determining whether NIH funding status associates with changes in sex bias and omission rates in future years. A difficult challenge for future studies will be assessing the direct impact of specific aspects of the SABV policy, beyond general levels of sex bias and omission. For instance, SABV does not necessarily require the use of both sexes, or that sex must be used as an experimental variable. Rather, SABV requires justification for single sex use, which is largely initially evaluated at the level of the NIH study section (Tannenbaum et al. 2016). This flexibility in single sex use is absolutely necessary, as there is proper justification for single sex use, as addressed in section 4.3. SABV minimally requires that data be reported disaggregated by sex, not that sex is necessarily considered as an experimental variable. This specific aspect of SABV was not evaluated by the current study, as the employed literature review protocol does not differentiate between studies that report aggregate sex data with no sex comparison versus studies that report data disaggregated by sex with no sex comparison. Anecdotally, curators associated with the current study report very few studies that presented data disaggregated by sex that did not likewise consider sex as an experimental variable. Overall, we believe that more work needs to be done to educate grant reviewers and applicants regarding the specifics of SABV and best practices in conducting sex-specific research in general. There may also be high utility in agencies issuing requests for funding grant applications that are directly relevant for sex-specific fundamental and translational research, which is a proposal recently and comprehensively discussed by Galea and colleagues (Galea et al. 2020).

A third possible factor is the influence of policies implemented at the level of the journal or at non-NIH funding agencies. Agencies including but not limited to the German Research Foundation, Science Foundation Ireland, European Commission, and the Canadian Institutes of Health Research have all implemented policies that require varying degrees of addressing sex as an experimental variable, and several of these policies were implemented prior to the NIH SABV, including those of the European Commission (2014). While the literature review protocol applied does not differentiate between these agencies, as a whole, it is possible that the influence of these policies has already resulted in a reduction of sex bias and omission, even among scientists reporting NIH funding. It is also possible that individual agency policies differentially impact the use of sex as an experimental variable. For instance, a policy adopted by the Canadian Institutes of Health features multipronged requirements that differ than those adopted by the NIH. These policies include the requirement of sex integration reporting, the participation of an individual with sex expertise related to the topic of the proposal, the incorporation of a sex platform for large research consortia, and compliance with sex-related training modules by grant applications (Duchesne et al. 2017). At this point, evidence of which policies are maximally effective remains limited, although a recent study has demonstrated that the training modules required by the Canadian Institutes of Health are effective in improving knowledge on how to effectively consider sex as an experimental variable (Tannenbaum and van Hoof 2018). Since there is considerable variation between the exact policy requirements implemented by funding agencies, future studies will need to specifically assess the effectiveness of specific funding agency policies, or, in the case of the present study, assess the overall nature of sex bias and omission regardless of exact research article funding source.

4.5. Role of the Journal

Another key finding of this study was the continued variability in sex bias and omission across neuroscience journals in articles published in 2017. This variability across journals was first demonstrated in a previous analysis of articles published between 2010 and 2014 (Will et al. 2017). This finding demonstrates the powerful influence that journals can exert on methods documentation, including research model sex, along with other influences of scientific practice, including funding agencies, universities, and individual researchers (Tannenbaum et al. 2019). While the present study was not designed to elucidate the etiology of differences in sex bias and omission between journals, these data indicate that it is worth considering why some journals exhibited relatively low sex bias and omission, while others did not. Of particular interest are differences in journal adoption and enforcement of relevant editorial policies. Most of the surveyed journals have adopted relevant standards. For instance, since 2012, all journals published by the American Physiological Society, including the Journal of Neurophysiology, asked authors to include the sex of research animals, cells, and other biological materials (Miller 2012). The American Physiological Society has also long adopted the relevant portions of the “Animals in Research: Reporting In Vivo Experiments” (ARRIVE) guidelines (Kilkenny et al. 2010). ARRIVE guidelines cover many aspects of experimental methodology, including biological sex, in an attempt to enhance reproducibility. The ARRIVE guidelines and similar guidelines have documented effectiveness (Baker et al. 2014; Moher et al. 2010; Sekula et al. 2017; Smidt et al. 2006; Turner et al. 2012), and the relatively low rate of sex omission in the Journal of Neurophysiology seems to confirm this approach. Similar guidelines, such as SAGER, have been adopted by other journals, including Frontiers in Neuroendocrinology (Heidari et al. 2016). Author, reviewer, or editor checklists that also incorporate animals have been demonstrated to be an effective approach (Han et al. 2017). The use of checklists is widely employed, including as a component of the policies instituted by the Nature publishing group, which publishes Nature Neuroscience and Nature. In 2016, Cell Press, which publishes Neuron, adopted the Structured, Transparent, Accessible Reporting (STAR) methods, which have helped decrease the sex omission in these journals. The Journal of Neuroscience remains the leader in sex reporting, with the lowest sex omission of any journal. Science remains the least effective in lowering sex omission. The effectiveness of different policy and enforcement methods across journals should be continually assessed by future studies.

4.6. Limitations

There are limitations to the current review. One limitation is that the employed review protocol does not differentiate between sex and gender in studies that employ humans. Sex is generally applied to the biological characteristics that differentiate various categories, including male, female, and intersex. No papers analyzed by this study that employed humans indicated an intersex category, which may be consistent with the relative scarcity of intersex individuals in the human population (Arboleda et al. 2014; Hughes et al. 2006), how intersex individuals chose to report their sex in these studies, or a lack of inclusion of these individuals in neuroscience-related research. In humans, gender is complex, non-binary, multifactorial, and informed by biological, psychological, social, and cultural factors. Considering gender in human studies presents methodological challenges that are not necessarily unique to neuroscience, including how to accurately assess an individual’s gender (Tannenbaum et al. 2019). The human data generated by this literature review relied on the reporting of gender/sex in each individual study, which could have been generated either via self-identification or via the researcher’s operating classification or perception. Our study was not designed to assess the status of gender consideration in neuroscience studies, but this is an important question and should be addressed by future studies. This study also does not assess how sex was determined in each research model. Other limitations to the current study also exist. For example, this study was not designed to assess whether the experiments described in neuroscience articles are sufficiently powered or properly statistically designed to detect robust sex differences. The presence and validity of justification for single or multiple sex studies was also not assessed. At least one study has assessed the presence of a justification for single sex use in research proposals submitted to an internal university Investigational Review Board, and it found that only 14.5% of proposed studies actually provided a justification (Freeman et al. 2017).

5.0. Conclusions

We reiterate that we believe that it can be appropriate for an individual study to either assess a single sex, or to assess both sexes. Indeed, the senior author of this study has coauthored manuscripts that employed solely females (Miller et al. 2020), solely males (Willett et al. 2018), and both males and females (Krentzel et al. 2019). Regretfully, the senior author has also coauthored manuscripts that did not report the sex of the research model (Stern et al. 2011), but since then, has gained perspective on the importance of reporting sex and other variables as being necessary for successful replication (Freedman et al. 2017). As of 2017, ~16% of studies still do not report animal sex, and while this is improved from 2014 (just over 20%), 16% remains an unacceptably high number for an essential experimental component necessary for successful replication.

This leads us to a central underlying question: why address sex at all? Many scientists have eloquently and comprehensively addressed this question, including Tannenbaum and colleagues (Tannenbaum et al. 2019). Here we will present the three broad reasons why our laboratory chooses to address sex in the context of our research program. First, scientific reproducibility. The lack of documentation of the sex of the employed research model is a direct barrier to reproducing a scientific finding. Second, scientific discovery. Biological sex is the low hanging fruit in terms of key natural variables potentially influencing the nervous system. Sex has only been systematically investigated as an experimental variable in a limited number of nervous system regions and functions. Even less is known regarding the differential influences of chromosomal sex, gonadal sex, sex steroid hormones, and environmental variables in sexual differentiation and adulthood, even though there are available and useful models such as the four core genotype mice (De Vries et al. 2002). Further complicating these influences, sex can differentially manifest between nervous system regions and even cell types within an individual across development (Bale and Epperson 2017; Joel and McCarthy 2017). Our own research program has demonstrated that sex differentially influences an important and widely studied neuron type, the medium spiny neuron, across striatal subregion, developmental period, rodent species, and electrophysiological property (Cao et al. 2018; Krentzel and Meitzen 2018; Meitzen et al. 2018), including across the adult female estrous cycle (Proano et al. 2018; Willett et al. 2019). Other considerations include the presence of latent sex differences (Jain et al. 2019), sex differences at the molecular or cellular level that minimize sex differences in circuit output (De Vries 2004) and interactions with other organ systems (de Vries and Forger 2015). Thus, it is possible that sex can exert an influence at any level of analysis in any developmental period in any portion of the nervous system. Without consideration of sex as an experimental variable, either in the context of a single study or a research program, this opportunity for discovery is lost. A third reason is ethics. The failure to include a representative population of females in research programs perpetuates the underrepresentation of an entire segment of our society. The ramifications of this neglect have great potential to negatively influence fundamental scientific discovery and health outcomes for all sexes.

There is an emerging recognition that sex exerts a complex and important influence on the nervous system and that this influence is present beyond brain regions that regulate purely sex-specific behaviors. Given the already known and continual discovery of sex differences in the nervous system, it will be important for health research to inclusively assess all sex and gender categories. This includes analyzing females, even in preclinical and clinical neuroscience research categories not traditionally considered to be sex-specific or relevant to women’s brain health. This effort begins with the reporting of sex and is followed by the proper justification of single sex use and then assessment of sex as an experimental variable when both males and females are employed. The pooling of data from females and males without consideration of sex as an experimental variable is a missed opportunity for scientific discovery. While there is encouraging progress regarding sex bias and omission in neuroscience studies, there is still much unrealized scientific potential that could be achieved by the use of more inclusive research techniques, resulting in improved scientific practice and the promise of future health outcomes.

Highlights.

  1. Sex bias and omission is present in neuroscience research.

  2. Relatively few articles consider sex as an experimental variable.

  3. Sex bias and omission vary considerably across research model organism.

  4. Sex omission, but not bias, varies by journal.

  5. Sex bias and omission do not vary with reported NIH funding.

Acknowledgements

The authors acknowledge funding from NIH R01MH109471, and the Center for Human Health and the Environment NIH P30ES025128. We thank Dr. Amanda Krentzel and Stephanie Proano for critically reading an earlier version of this study.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Andersen JP, Schneider JW, Jagsi R, and Nielsen MW. Gender variations in citation distributions in medicine are very small and due to self-citation and journal prestige. eLife 8: pii: e45374, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Arboleda VA, Sandberg DE, and Vilain E. DSDs: genetics, underlying pathologies and psychosexual differentiation. Nature reviews Endocrinology 10: 603–615, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Arnold AP, and Disteche CM. Sexual Inequality in the Cancer Cell. Cancer research 78: 5504–5505, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baker D, Lidster K, Sottomayor A, and Amor S. Two years later: journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. PLoS biology 12: e1001756, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bale TL. Sex matters. Neuropsychopharmacology 44: 1–3, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bale TL, and Epperson CN. Sex as a Biological Variable: Who, What, When, Why, and How. Neuropsychopharmacology 42: 386–396, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Balthazart J, Choleris E, and Remage-Healey L. Steroids and the brain: 50 years of research, conceptual shifts and the ascent of non-classical and membrane-initiated actions. Hormones and behavior 99: 1–8, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Beach FA. The snark was a boojum. Am Psychol 5: 115–124, 1950. [Google Scholar]
  9. Becker JB, Arnold AP, Berkley KJ, Blaustein JD, Eckel LA, Hampson E, Herman JP, Marts S, Sadee W, Steiner M, Taylor J, and Young E. Strategies and methods for research on sex differences in brain and behavior. Endocrinology 146: 1650–1673, 2005. [DOI] [PubMed] [Google Scholar]
  10. Becker JB, Prendergast BJ, and Liang JW. Female rats are not more variable than male rats: a meta-analysis of neuroscience studies. Biology of sex differences 7: 34, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Beery AK. Inclusion of females does not increase variability in rodent research studies. Current opinion in behavioral sciences 23: 143–149, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Beery AK, and Zucker I. Sex bias in neuroscience and biomedical research. Neuroscience and biobehavioral reviews 35: 565–572, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Beltz AM, Beery AK, and Becker JB. Analysis of sex differences in pre-clinical and clinical data sets. Neuropsychopharmacology 44: 2155–2158, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Ben-Yosef D, Amit A, Malcov M, Frumkin T, Ben-Yehudah A, Eldar I, Mey-Raz N, Azem F, Altarescu G, Renbaum P, Beeri R, Varshaver I, Eldar-Geva T, Epsztejn-Litman S, Levy-Lahad E, and Eiges R. Female sex bias in human embryonic stem cell lines. Stem cells and development 21: 363–372, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Berkley KJ. Vive la difference! Trends in neurosciences 15: 331–332, 1992. [DOI] [PubMed] [Google Scholar]
  16. Brenowitz EA, and Remage-Healey L. It takes a seasoned bird to be a good listener: communication between the sexes. Current opinion in neurobiology 38: 12–17, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Brenowitz EA, and Zakon HH. Emerging from the bottleneck: benefits of the comparative approach to modern neuroscience. Trends in neurosciences 38: 273–278, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Brooks CE, and Clayton JA. Sex/gender influences on the nervous system: Basic steps toward clinical progress. Journal of neuroscience research 95: 14–16, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Bryant J, Yi P, Miller L, Peek K, and Lee D. Potential Sex Bias Exists in Orthopaedic Basic Science and Translational Research. The Journal of bone and joint surgery 100: 124–130, 2018. [DOI] [PubMed] [Google Scholar]
  20. Buch T, Moos K, Ferreira FM, Frohlich H, Gebhard C, and Tresch A. Benefits of a factorial design focusing on inclusion of female and male animals in one experiment. Journal of molecular medicine 97: 871–877, 2019. [DOI] [PubMed] [Google Scholar]
  21. Cahill L, and Aswad D. Sex Influences on the Brain: An Issue Whose Time Has Come. Neuron 88: 1084–1085, 2015. [DOI] [PubMed] [Google Scholar]
  22. Cao J, Willett JA, Dorris DM, and Meitzen J. Sex Differences in Medium Spiny Neuron Excitability and Glutamatergic Synaptic Input: Heterogeneity Across Striatal Regions and Evidence for Estradiol-Dependent Sexual Differentiation. Frontiers in endocrinology 9: 173, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Clayton JA. Applying the new SABV (sex as a biological variable) policy to research and clinical care. Physiology & behavior 187: 2–5, 2018. [DOI] [PubMed] [Google Scholar]
  24. Clayton JA, and Collins FS. Policy: NIH to balance sex in cell and animal studies. Nature 509: 282–283, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Day S, Mason R, Tannenbaum C, and Rochon PA. Essential metrics for assessing sex & gender integration in health research proposals involving human participants. PloS one 12: e0182812, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. De Vries GJ. Minireview: Sex differences in adult and developing brains: compensation, compensation, compensation. Endocrinology 145: 1063–1068, 2004. [DOI] [PubMed] [Google Scholar]
  27. de Vries GJ, and Forger NG. Sex differences in the brain: a whole body perspective. Biology of sex differences 6: 15, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. De Vries GJ, Rissman EF, Simerly RB, Yang LY, Scordalakes EM, Auger CJ, Swain A, Lovell-Badge R, Burgoyne PS, and Arnold AP. A model system for study of sex chromosome effects on sexually dimorphic neural and behavioral traits. J Neurosci 22: 9005–9014., 2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Duarte-Guterman P, Leuner B, and Galea LAM. The long and short term effects of motherhood on the brain. Frontiers in neuroendocrinology 53: 100740, 2019. [DOI] [PubMed] [Google Scholar]
  30. Duchesne A, Tannenbaum C, and Einstein G. Funding agency mechanisms to increase sex and gender analysis. Lancet 389: 699, 2017. [DOI] [PubMed] [Google Scholar]
  31. Eliot L, and Richardson SS. Sex in Context: Limitations of Animal Studies for Addressing Human Sex/Gender Neurobehavioral Health Disparities. The Journal of neuroscience 36: 11823–11830, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Fields RD. NIH policy: mandate goes too far. Nature 510: 340, 2014. [DOI] [PubMed] [Google Scholar]
  33. Florez-Vargas O, Brass A, Karystianis G, Bramhall M, Stevens R, Cruickshank S, and Nenadic G. Bias in the reporting of sex and age in biomedical research on mouse models. eLife 5: e13615, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Freedman LP, Venugopalan G, and Wisman R. Reproducibility2020: Progress and priorities. F1000Research 6: 604, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Freeman A, Stanko P, Berkowitz LN, Parnell N, Zuppe A, Bale TL, Ziolek T, and Epperson CN. Inclusion of sex and gender in biomedical research: survey of clinical research proposed at the University of Pennsylvania. Biology of sex differences 8: 22, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Galea LAM, Choleris E, Albert AYK, McCarthy MM, and Sohrabji F. The promises and pitfalls of sex difference research. Frontiers in neuroendocrinology 56: 100817, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Geller SE, Koch A, Pellettieri B, and Carnes M. Inclusion, analysis, and reporting of sex and race/ethnicity in clinical trials: have we made progress? Journal of women’s health 20: 315–320, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Geller SE, Koch AR, Roesch P, Filut A, Hallgren E, and Carnes M. The More Things Change, the More They Stay the Same: A Study to Evaluate Compliance With Inclusion and Assessment of Women and Minorities in Randomized Controlled Trials. Academic medicine 93: 630–635, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Gervais NJ, Remage-Healey L, Starrett JR, Pollak DJ, Mong JA, and Lacreuse A. Adverse Effects of Aromatase Inhibition on the Brain and Behavior in a Nonhuman Primate. The Journal of neuroscience 39: 918–928, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Guizzetti M, Davies DL, Egli M, Finn DA, Molina P, Regunathan S, Robinson DL, and Sohrabji F. Sex and the Lab: An Alcohol-Focused Commentary on the NIH Initiative to Balance Sex in Cell and Animal Studies. Alcoholism, clinical and experimental research 40: 1182–1191, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Han S, Olonisakin TF, Pribis JP, Zupetic J, Yoon JH, Holleran KM, Jeong K, Shaikh N, Rubio DM, and Lee JS. A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review. PloS one 12: e0183591, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. He L, Martins P, Huguenin J, Van TN, Manso T, Galindo T, Gregoire F, Catherinot L, Molina F, and Espeut J. Simple, sensitive and robust chicken specific sexing assays, compliant with large scale analysis. PloS one 14: e0213033, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Heidari S, Babor TF, De Castro P, Tort S, and Curno M. Sex and Gender Equity in Research: rationale for the SAGER guidelines and recommended use. Research integrity and peer review 1: 2, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Honjoh S, Ihara A, Kajiwara Y, Yamamoto T, and Nishida E. The Sexual Dimorphism of Dietary Restriction Responsiveness in Caenorhabditis elegans. Cell reports 21: 3646–3652, 2017. [DOI] [PubMed] [Google Scholar]
  45. Hughes IA, Houk C, Ahmed SF, and Lee PA. Consensus statement on management of intersex disorders. Journal of pediatric urology 2: 148–162, 2006. [DOI] [PubMed] [Google Scholar]
  46. Jain A, Huang GZ, and Woolley CS. Latent Sex Differences in Molecular Signaling That Underlies Excitatory Synaptic Potentiation in the Hippocampus. The Journal of neuroscience 39: 1552–1565, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Joel D, and McCarthy MM. Incorporating Sex As a Biological Variable in Neuropsychiatric Research: Where Are We Now and Where Should We Be? Neuropsychopharmacology 42: 379–385, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Johnson J, Sharman Z, Vissandjee B, and Stewart DE. Does a change in health research funding policy related to the integration of sex and gender have an impact? PloS one 9: e99900, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Karp NA, Mason J, Beaudet AL, Benjamini Y, Bower L, Braun RE, Brown SDM, Chesler EJ, Dickinson ME, Flenniken AM, Fuchs H, Angelis MH, Gao X, Guo S, Greenaway S, Heller R, Herault Y, Justice MJ, Kurbatova N, Lelliott CJ, Lloyd KCK, Mallon AM, Mank JE, Masuya H, McKerlie C, Meehan TF, Mott RF, Murray SA, Parkinson H, Ramirez-Solis R, Santos L, Seavitt JR, Smedley D, Sorg T, Speak AO, Steel KP, Svenson KL, Wakana S, West D, Wells S, Westerberg H, Yaacoby S, and White JK. Prevalence of sexual dimorphism in mammalian phenotypic traits. Nature communications 8: 15475, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, and Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS biology 8: e1000412, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Klein SL, Schiebinger L, Stefanick ML, Cahill L, Danska J, de Vries GJ, Kibbe MR, McCarthy MM, Mogil JS, Woodruff TK, and Zucker I. Opinion: Sex inclusion in basic research drives discovery. Proceedings of the National Academy of Sciences of the United States of America 112: 5257–5258, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Kokras N, Hodes GE, Bangasser DA, and Dalla C. Sex differences in the hypothalamic-pituitary-adrenal axis: An obstacle to antidepressant drug development? British journal of pharmacology 176: 4090–4106, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Krebs HA. The August Krogh Principle: “For many problems there is an animal on which it can be most conveniently studied”. The Journal of experimental zoology 194: 221–226, 1975. [DOI] [PubMed] [Google Scholar]
  54. Krentzel AA, Barrett LR, and Meitzen J. Estradiol rapidly modulates excitatory synapse properties in a sex and region-specific manner in rat nucleus accumbens core and caudate-putamen. Journal of neurophysiology 122: 1213–1225, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Krentzel AA, and Meitzen J. Biological Sex, Estradiol and Striatal Medium Spiny Neuron Physiology: A Mini-Review. Front Cell Neurosci 12: 492, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Lambert K, Kent M, and Vavra D. Avoiding Beach’s Boojum Effect: Enhancing bench to bedside translation with field to laboratory considerations in optimal animal models. Neuroscience and biobehavioral reviews 104: 191–196, 2019. [DOI] [PubMed] [Google Scholar]
  57. Liu H, Todd EV, Lokman PM, Lamm MS, Godwin JR, and Gemmell NJ. Sexual plasticity: A fishy tale. Molecular reproduction and development 84: 171–194, 2017. [DOI] [PubMed] [Google Scholar]
  58. Liu KA, and Mager NA. Women’s involvement in clinical trials: historical perspective and future implications. Pharmacy practice 14: 708, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Lonsdorf EV. Sex differences in nonhuman primate behavioral development. Journal of neuroscience research 95: 213–221, 2017. [DOI] [PubMed] [Google Scholar]
  60. Maney DL. Perils and pitfalls of reporting sex differences. Philosophical transactions of the Royal Society of London Series B, Biological sciences 371: 20150119, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Manger PR, Cort J, Ebrahim N, Goodman A, Henning J, Karolia M, Rodrigues SL, and Strkalj G. Is 21st century neuroscience too focussed on the rat/mouse model of brain function and dysfunction? Frontiers in neuroanatomy 2: 5, 2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Maures TJ, Booth LN, Benayoun BA, Izrayelit Y, Schroeder FC, and Brunet A. Males shorten the life span of C. elegans hermaphrodites via secreted compounds. Science 343: 541–544, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. McCarthy MM. Incorporating Sex as a Variable in Preclinical Neuropsychiatric Research. Schizophrenia bulletin 41: 1016–1020, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. McCullough LD, McCarthy MM, and de Vries GJ. NIH policy: status quo is also costly. Nature 510: 340, 2014. [DOI] [PubMed] [Google Scholar]
  65. McEwen BS, and Milner TA. Understanding the broad influence of sex hormones and sex differences in the brain. Journal of neuroscience research 95: 24–39, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Meitzen J, Meisel RL, and Mermelstein PG. Sex Differences and the Effects of Estradiol on Striatal Function. Current opinion in behavioral sciences 23: 42–48, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Miller CK, Krentzel AA, Patisaul HB, and Meitzen J. Metabotropic glutamate receptor subtype 5 (mGlu5) is necessary for estradiol mitigation of light-induced anxiety behavior in female rats. Physiology & behavior 214: 112770, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Miller LR, Marks C, Becker JB, Hurn PD, Chen WJ, Woodruff T, McCarthy MM, Sohrabji F, Schiebinger L, Wetherington CL, Makris S, Arnold AP, Einstein G, Miller VM, Sandberg K, Maier S, Cornelison TL, and Clayton JA. Considering sex as a biological variable in preclinical research. FASEB journal 31: 29–34, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Miller VM. In pursuit of scientific excellence: sex matters. American journal of physiology Cell physiology 302: C1269–1270, 2012. [DOI] [PubMed] [Google Scholar]
  70. Mogil JS. Perspective: Equality need not be painful. Nature 535: S7, 2016. [DOI] [PubMed] [Google Scholar]
  71. Mogil JS, and Chanda ML. The case for the inclusion of female subjects in basic science studies of pain. Pain 117: 1–5, 2005. [DOI] [PubMed] [Google Scholar]
  72. Moher D, Schulz KF, Simera I, and Altman DG. Guidance for developers of health research reporting guidelines. PLoS medicine 7: e1000217, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Nielsen MW, Andersen JP, Schiebinger L, and Schneider JW. One and a half million medical papers reveal a link between author gender and attention to gender and sex analysis. Nature human behaviour 1: 791–796, 2017. [DOI] [PubMed] [Google Scholar]
  74. Panzica G, and Melcangi RC. Structural and molecular brain sexual differences: A tool to understand sex differences in health and disease. Neuroscience and biobehavioral reviews 67: 2–8, 2016. [DOI] [PubMed] [Google Scholar]
  75. Park MN, Park JH, Paik HY, and Lee SK. Insufficient sex description of cells supplied by commercial vendors. American journal of physiology Cell physiology 308: C578–580, 2015. [DOI] [PubMed] [Google Scholar]
  76. Potluri T, Engle K, Fink AL, Vom Steeg LG, and Klein SL. Sex Reporting in Preclinical Microbiological and Immunological Research. mBio 8: pii: e01868–01817, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Proano S, Morris HJ, Kunz LM, Dorris DM, and Meitzen J. Estrous cycle-induced sex differences in medium spiny neuron excitatory synaptic transmission and intrinsic excitability in adult rat nucleus accumbens core. Journal of neurophysiology 120: 1356–1373, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  78. Remage-Healey L, Krentzel AA, Macedo-Lima M, and Vahaba D. Species Diversity Matters in Biological Research. Policy Insights from the Behavioral and Brain Sciences 4: 210–218, 2017. [Google Scholar]
  79. Richardson SS, Reiches M, Shattuck-Heidorn H, LaBonte ML, and Consoli T. Opinion: Focus on preclinical sex differences will not address women’s and men’s health disparities. Proceedings of the National Academy of Sciences of the United States of America 112: 13419–13420, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Ritz SA. Complexities of Addressing Sex in Cell Culture Research. Journal of Women in Culture and Society 42: 307–327, 2017. [Google Scholar]
  81. Ruigrok AN, Salimi-Khorshidi G, Lai MC, Baron-Cohen S, Lombardo MV, Tait RJ, and Suckling J. A meta-analysis of sex differences in human brain structure. Neuroscience and biobehavioral reviews 39: 34–50, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Saldanha CJ, Remage-Healey L, and Schlinger BA. Synaptocrine signaling: steroid synthesis and action at the synapse. Endocr Rev 32: 532–549, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  83. Sekula P, Mallett S, Altman DG, and Sauerbrei W. Did the reporting of prognostic studies of tumour markers improve since the introduction of REMARK guideline? A comparison of reporting in published articles. PloS one 12: e0178531, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  84. Sengelaub DR, and Forger NG. The spinal nucleus of the bulbocavernosus: firsts in androgen-dependent neural sex differences. Horm Behav 53: 596–612, 2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  85. Shah K, McCormack CE, and Bradbury NA. Do you know the sex of your cells? American journal of physiology Cell physiology 306: C3–18, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  86. Shansky RM. Are hormones a “female problem” for animal research? Science 364: 825–826, 2019. [DOI] [PubMed] [Google Scholar]
  87. Shansky RM, and Woolley CS. Considering Sex as a Biological Variable Will Be Valuable for Neuroscience Research. The Journal of neuroscience 36: 11817–11822, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  88. Shao Q, Yang T, Huang H, Alarmanazi F, and Liu G. Uncoupling of UNC5C with Polymerized TUBB3 in Microtubules Mediates Netrin-1 Repulsion. The Journal of neuroscience 37: 5620–5633, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  89. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, Bouter LM, and de Vet HC. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology 67: 792–797, 2006. [DOI] [PubMed] [Google Scholar]
  90. Stephenson ED, Farzal Z, Kilpatrick LA, Senior BA, and Zanation AM. Sex bias in basic science and translational otolaryngology research. The Laryngoscope 129: 613–618, 2019. [DOI] [PubMed] [Google Scholar]
  91. Stern CM, Meitzen J, and Mermelstein PG. Corticotropin-releasing factor and urocortin I activate CREB through functionally selective Gbetagamma signaling in hippocampal pyramidal neurons. The European journal of neuroscience 34: 671–681, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Sugimoto CR, Ahn YY, Smith E, Macaluso B, and Lariviere V. Factors affecting sex-related reporting in medical research: a cross-disciplinary bibliometric analysis. Lancet 393: 550–559, 2019. [DOI] [PubMed] [Google Scholar]
  93. Tannenbaum C, Ellis RP, Eyssel F, Zou J, and Schiebinger L. Sex and gender analysis improves science and engineering. Nature 575: 137–146, 2019. [DOI] [PubMed] [Google Scholar]
  94. Tannenbaum C, Schwarz JM, Clayton JA, de Vries GJ, and Sullivan C. Evaluating sex as a biological variable in preclinical research: the devil in the details. Biology of sex differences 7: 13, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  95. Tannenbaum C, and van Hoof K. Effectiveness of online learning on health researcher capacity to appropriately integrate sex, gender, or both in grant proposals. Biology of sex differences 9: 39, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  96. Taylor KE, Vallejo-Giraldo C, Schaible NS, Zakeri R, and Miller VM. Reporting of sex as a variable in cardiovascular studies using cultured cells. Biology of sex differences 2: 11, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  97. Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, Dias S, Schulz KF, Plint AC, and Moher D. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. The Cochrane database of systematic reviews 11: MR000030, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  98. Vinke CM, van Deijk R, Houx BB, and Schoemaker NJ. The effects of surgical and chemical castration on intermale aggression, sexual behaviour and play behaviour in the male ferret (Mustela putorius furo). Applied Animal Behaviour Science 115: 104–121, 2008. [Google Scholar]
  99. Will TR, Proano SB, Thomas AM, Kunz LM, Thompson KC, Ginnari LA, Jones CH, Lucas SC, Reavis EM, Dorris DM, and Meitzen J. Problems and Progress regarding Sex Bias and Omission in Neuroscience Research. eNeuro 4: pii: ENEURO.0278–0217.2017., 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  100. Willett JA, Cao J, Johnson A, Patel OH, Dorris DM, and Meitzen J. The estrous cycle modulates rat caudate-putamen medium spiny neuron physiology. The European journal of neuroscience [Epub ahead of print], 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  101. Willett JA, Johnson AG, Vogel AR, Patisaul HB, McGraw LA, and Meitzen J. Nucleus accumbens core medium spiny neuron electrophysiological properties and partner preference behavior in the adult male prairie vole, Microtus ochrogaster. Journal of neurophysiology 119: 1576–1588, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  102. Woitowich NC, and Woodruff TK. Implementation of the NIH Sex-Inclusion Policy: Attitudes and Opinions of Study Section Members. Journal of women’s health 28: 9–16, 2019. [DOI] [PubMed] [Google Scholar]
  103. Xu J, Peng X, Chen Y, Zhang Y, Ma Q, Liang L, Carter AC, Lu X, and Wu CI. Free-living human cells reconfigure their chromosomes in the evolution back to uni-cellularity. eLife 6: 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  104. Yoon DY, Mansukhani NA, Stubbs VC, Helenowski IB, Woodruff TK, and Kibbe MR. Sex bias exists in basic science and translational surgical research. Surgery 156: 508–516, 2014. [DOI] [PubMed] [Google Scholar]
  105. Zakiniaeiz Y, Cosgrove KP, Potenza MN, and Mazure CM. Balance of the Sexes: Addressing Sex Differences in Preclinical Research. The Yale journal of biology and medicine 89: 255–259, 2016. [PMC free article] [PubMed] [Google Scholar]
  106. Zhang YS, Wittenbach JD, Jin DZ, and Kozhevnikov AA. Temperature Manipulation in Songbird Brain Implicates the Premotor Nucleus HVC in Birdsong Syntax. The Journal of neuroscience 37: 2600–2611, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES