Diversity of Research Participant Gender, Race, and Ethnicity in Communication Sciences and Disorders: A Systematic Review and Quantitative Synthesis of American Speech-Language-Hearing Association Publications in 2020

Abstract

Purpose:

One manifestation of systemic inequities in communication sciences and disorders (CSD) is the chronic underreporting and underrepresentation of sex, gender, race, and ethnicity in research. The present study characterized recent demographic reporting practices and representation of participants across CSD research.

Methods:

We systematically reviewed and extracted key reporting and participant data from empirical studies conducted in the United States (US) with human participants published in the year 2020 in journals by the American Speech-Language-Hearing Association (ASHA; k = 407 articles comprising a total n = 80,058 research participants, search completed November 2021). Sex, gender, race, and ethnicity were operationalized per National Institutes of Health guidelines (National Institutes of Health, 2015a, 2015b).

Results:

Sex or gender was reported in 85.5% of included studies; race was reported in 33.7%; and ethnicity was reported in 13.8%. Sex and gender were clearly differentiated in 3.4% of relevant studies. Where reported, median proportions for race and ethnicity were significantly different from the US population, with underrepresentation noted for all non-White racial groups and Hispanic participants. Moreover, 64.7% of studies that reported sex or gender and 67.2% of studies that reported race or ethnicity did not consider these respective variables in analyses or discussion.

Conclusion:

At present, research published in ASHA journals frequently fails to report key demographic data summarizing the characteristics of participants. Moreover, apparent gaps in representation of minoritized racial and ethnic groups threaten the external validity of CSD research and broader health care equity endeavors in the US. Although our study is limited to a single year and publisher, our results point to several steps for readers that may bring greater accountability, consistency, and diversity to the discipline.

The discipline of communication sciences and disorders (CSD) currently faces overlapping challenges regarding a lack of sex, gender, racial, and ethnic diversity and equity that are urgent and endemic within research and clinical practice (e.g., Ellis et al., 2021; Yu et al., 2022). One manifestation of systemic inequality in CSD, and a focus of the current study, is the chronic underrepresentation of racial, ethnic, and gender diversity among participant samples in research, which has been widely documented across the broader biomedical, psychological, and educational literature (e.g., Bentley et al., 2017; Erves et al., 2017; Flores et al., 2021; Geller et al., 2018; Hruschka et al., 2018; Oh et al., 2015).

Institutional efforts to address disparities in health and biomedical research have been attempted since at least the passing of the National Institutes of Health (NIH) Revitalization Act (1993), which mandated that any federally funded clinical research prioritize and document the inclusion of women and racial minorities. However, in the three decades since the passing of this legislation, sex, gender, racial, and ethnic diversity amongst health care research participants continues to be insufficient, as well as poorly defined, measured, and reported (e.g., Bölte et al., 2023; Corbie-Smith et al., 2008; Flores et al., 2021; Geller et al., 2018; Ghorbanian et al., 2022; Merritt, 2020; National Academies of Sciences, Engineering, and Medicine, 2022; Oh et al., 2015).

Nearly every facet of daily life in America engages with aspects of sex, gender, racial, and ethnic identities (Crenshaw, 1991; Goodman et al., 2019); however, these constructs have proven challenging to address in research, as they represent complex and ever-evolving intersections of biological and sociocultural factors. Moreover, the constructs of sex, gender, race, and ethnicity lack consistent definitions and guidelines for reporting in CSD research. As a result, these demographic characteristics may be conflated with related terms or may be grossly misunderstood.

One possible definition for gender, as a construct distinct from sex, was summarized in a recent ASHA Perspectives article by Merritt (2020), with sex defined as an assignment based on biological anatomy (e.g., male or female) and gender defined as a person’s psychosocial identity, which may or may not correspond to their biological sex (e.g., woman, girl, nonconforming). In contrast, there is little discipline-specific guidance for defining race or ethnicity; as such, we may consider recommendations from the American Psychological Association (2020), which defines race as referring to “physical differences that groups and cultures consider socially significant” and ethnicity as referring to “shared cultural characteristics such as language, ancestry, practices, and beliefs” (p. 70). It is important to emphasize that neither race nor ethnicity are strictly defined biologically or genetically; rather, they are sociocultural constructions influenced by many associated factors, including socioeconomic status, environment, racism, discrimination, education, geography, and religious beliefs, among others (see Bamshad et al., 2004; Caulfield et al., 2009; Corbie-Smith et al., 2008; Goodman et al., 2019; Torres, 2019). Race and ethnicity are also unstable and inconsistently defined traits; whether an individual is considered part of a given racial group, for example, can change over time and may differ between countries and across social contexts (Goodman et al., 2019; Simon et al., 2015).

In a recent tutorial article, Ellis and Jacobs (2021) emphasized the relevance of health disparities along demographic lines to patient populations in CSD, where disparate health and education outcomes can be found according to sex, gender, race, ethnicity, and other demographic dimensions. Notable examples of demographic differences in CSD include higher risk of stuttering for Black versus White children (Briley & Ellis, 2020); differences in literacy exposure between Hispanic and non-Hispanic children (Hammer et al., 2010); differences in ages and rates at which autism is diagnosed between Black, Hispanic, and White children (Mandell et al., 2009); disparities in language outcomes between White and Black people with aphasia (Ellis & Peach, 2017); disparities in rates of hearing loss and hearing aid usage, respectively, between White and Black individuals (e.g., Nieman et al., 2016); higher prevalence of pediatric hearing loss among Hispanic children, compared with children of other ethnicities (Mehra et al., 2009); and discrepancies between males and females on several diagnostic measures of autism (e.g., Kaat et al., 2021). Moreover, there is an increasingly recognized need for inclusion and representation of transgender or otherwise gender-diverse individuals across CSD sub-disciplines (e.g., Holmberg et al., 2023; Hosbach-Cannon et al., 2022; Kirjava et al., 2023).

Despite an ever-growing body of literature underscoring the differential impact of sex, gender, race, and ethnicity in persons impacted by communication disorders in the United States (US), there is also some evidence suggesting that such demographic variables are underreported or not representative of the broader populations that we serve and study, particularly with regard to racial and ethnic diversity (Cascio et al., 2020; Ellis, 2009; Morton & Sandage, 2022; Nelson, 2022; Pierce et al., 2014; Sinclair et al., 2018; Steinbrenner et al., 2022; West et al., 2016). For example, Ellis (2009) reviewed 116 articles published from 1997 through 2007 in the American Journal of Speech-Language Pathology and the Journal of Speech, Language, and Hearing Research in the area of adult neurogenic communication disorders and found that fewer than 15% of the articles reported participant race or ethnicity. Nguy and colleagues (2022) conducted a similar review of aphasia treatment studies between 2009 and 2019 and found that fewer than 30% of articles reported participant race or ethnicity. In a study focusing on autism research, Pierce et al. (2014) conducted a review of 138 articles over a 6-year period from three autism-focused journals, and found that only 28% of articles reported participant race or ethnicity.

Further, even when race and ethnicity have been thoroughly reported on research participants in studies of interest to CSD, there has been a striking lack of diversity observed amongst research participants, with participants from several racial and ethnic minority groups disproportionately underrepresented (e.g., Cascio et al., 2020; Nguy et al., 2022; Steinbrenner et al., 2022; West et al., 2016). Inadequate representation among research participants represents a threat to external validity (i.e., the ability to generalize findings from research to the broader population of persons affected by communication disorders), particularly when outcomes of interest may be expected to vary according to demographic factors (Coppock et al., 2018) and when providers are serving increasingly diverse populations (e.g., Guiberson & Atkins, 2012).

ASHA and ASHA’s peer-reviewed scientific journals have not yet provided guidelines regarding sex, gender, racial, or ethnic inclusivity in CSD research. In ASHA’s guide for authors, a brief discussion is provided regarding sex and gender biases in writing, but the reader is directed to the American Psychological Association’s style guide for additional context, and clear definitions are not given (ASHA, n.d.). The NIH, a major avenue for CSD research funding, currently requires funded investigators to report on and include “women and members of minority groups and their subpopulations” (NIH, 2001, para. II-A), but this requirement is broad in its criteria for inclusion and does not extend to mandated reporting of demographic information in research reports and broader dissemination activities (e.g., conference presentations).

To date, there has never been a comprehensive investigation of both reporting practices and representation among research participants across CSD. Such investigations are critical to identify research trends and gaps related to demographic reporting practices and overall participant representation. Therefore, our team took a meta-scientific approach to better characterize trends in reporting and the scope of demographic representation among CSD research participants. We aimed to answer the following research questions for research published by ASHA journals within a single calendar year (2020):

1. What were the reporting practices for sex, gender, race, and/or ethnicity for research participants, and did reporting practices differ for those studies funded by the NIH, which explicitly mandates the collection of such participant demographic data?

2. For those studies reporting sex, gender, race, and/or ethnicity, were participants representative of national population demographics?

3. What proportion of studies considered demographic constructs in analyses and/or interpretation of the data?

We recognize that many other sociocultural aspects of identity may be important to consider when characterizing diversity in CSD research (e.g., sexual orientation, disability status, language use; see Ellis & Jacobs, 2021). In the present study, however, we focus on sex, gender, race, and ethnicity as an initial step towards addressing representation according to demographic factors in CSD research.

Method

The reporting of this systematic review was guided by the standards of the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) Statement (Page et al., 2021).

Search Parameters and Inclusion Criteria

To identify recent articles in CSD, we conducted a systematic review within all peer-reviewed journals published by ASHA: the American Journal of Audiology; the American Journal of Speech-Language Pathology; the Journal of Speech, Language, and Hearing Research; Language, Speech, and Hearing Services in Schools; and Perspectives of the ASHA Special Interest Groups. The ASHA journals were selected because ASHA, as the predominant professional association for audiologists and speech-language pathologists in the US, publishes empirical studies with a collectively broad scope within CSD and with high visibility to both clinicians and researchers.

For a comprehensive snapshot of recent publication trends, we reviewed all relevant articles published during the year 2020. A single year was determined to be the largest period of time for which this study could feasibly be conducted, and 2020 was selected as the most recent and complete year at the outset of the study. Otherwise, we aimed to characterize the full breadth of empirical studies published by ASHA journals, including both quantitative and qualitative studies, and imposing no requirements for participant sample sizes. A summary of inclusion criteria are as follows, with articles required: (a) to be published and paginated within one of the specified ASHA journals within the year 2020; (b) to describe primary empirical research (e.g., excluding opinions and commentaries, reviews, and meta-analyses); and (c) to include at least one human participant or extant data derived from at least one human participant. To control for differing definitions of race or ethnicity between countries (Simon et al., 2015), a final inclusion criterion required each article to have (d) recruited participants solely within the US.

Search and Initial Screening

First, we reviewed journal guidelines to identify article types that did not describe primary empirical research (ASHA, 2021). We identified the following article types for a priori exclusion: Editorial, Letter to the Editor, Review Article, Tutorial, Erratum, Viewpoint, Commentary, Introduction, Epilogue, Prologue, CE Questions, Publisher Note, and Masthead entries. We selected the remaining article types to be included in the corpus for coding: Research Article, Research Note, Clinical Focus, and Technical Report. Next, we used ASHAWire, ASHA’s online publication portal and database, to identify the initial corpus of publications potentially meeting inclusion criteria according to publication year, journal, and article type. This initial search and screening was conducted by the first author in November 2021, with 667 articles meeting screening criteria out of a total of 862 articles published by ASHA journals in the year 2020 (see Figure 1 for PRISMA flow diagram). We exported the articles’ titles for further review.

Figure 1. Flowchart Depicting Results of Search and Screening Process to Yield Final Article Corpus.

Note. ASHA = American Speech-Language-Hearing Association; AJA = American Journal of Audiology; AJSLP = American Journal of Speech-Language Pathology; JSLHR = Journal of Speech, Language, and Hearing Research; LSHSS = Language, Speech, and Hearing Services in Schools; Perspectives = Perspectives of the ASHA Special Interest Groups. ASHAWire is ASHA’s publication portal (https://pubs.asha.org)

Coding and Reliability

During the next phase of review, our research team developed a coding scheme and corresponding coding manual to provide comprehensive operational definitions and examples to guide coding decisions for extracting variables of interest. A parallel coding shell was developed in Research Electronic Data Capture (REDCap), a web-based software platform for secure data management (Harris et al., 2009). The final coding manual directs coders to further screen articles for inclusion and to characterize each study included in the final corpus by design characteristics, reporting practices, and demographics of participant samples, as well as recruitment/retention strategies (see Data Availability Statement for link to supplemental study materials).

Coders were trained to strategically review each article by (a) reading the title and abstract to familiarize themselves with the study’s purpose, approach, and key findings, (b) reading the Methods and other selected sections of the manuscript in-depth to extract information relevant to the coding scheme, and (c) conducting a search for key words within the manuscript’s broader body text to increase the likelihood that all relevant study details were identified.

To establish baseline intercoder reliability, all coders completed reliability training that involved a review of the coding manual and subsequent coding of a subset of k = 7 consensus-coded training articles, with detailed feedback regarding all coding discrepancies. All articles were then randomly assigned, in batches of 7–28 articles per coder, to be independently reviewed by two trained coders. Coder pairings were balanced to reduce the likelihood of coder drift. Following independent primary and secondary coding of each article, discrepancies were identified and resolved via a standard resolution process. Specifically, primary and secondary codes were compared to identify (a) critical data discrepancies (items identified a priori as directly related to research questions; see Table 1), which prompted a discrepancy discussion between primary and secondary coders, and (b) other data discrepancies, which were adjudicated by the first author (RM) with input as needed from second (JF) and senior authors (RJ, TW).

Table 1.

Examples of Data Extracted During Coding

Critical Data	Other Data
Total number of participants	Research area sub-discipline
Numbers of participants by gendera	Participant geographic location
Numbers of participants by raceb	Funding source
Numbers of participants by ethnicityc
Was sex clearly distinct from gender?
Were race and ethnicity conflated?
Was sex or gender considered in data analyses?
Was race or ethnicity considered in data analyses?
Was sex or gender mentioned in the discussion?
Was race or ethnicity mentioned in the discussion?

Note. Per our coding manual, critical data were identified a priori as directly related to primary research questions, with discrepancies between coders resolved by resolution discussion. Other data were identified a priori as ancillary or supportive to primary research questions, with discrepancies between coders adjudicated by lead author.

^aGender operationalized as any participant data encompassing masculine, feminine, or other gender-diverse identities. Included data given as sex (e.g., male, female) unless explicitly differentiated from gender as a psychosocial identity.

^bRace operationalized, per guidelines by the National Institutes of Health (NIH; 2015a), as the following categories: American Indian, Asian, Black, Native Hawaiian or Other Pacific Islander, White, and Multi-Racial. We further included an “Other” category to characterize additional racial identities.

^cEthnicity operationalized, also per guidelines by the NIH (2015a), as Hispanic or Latino(a)(x) and Not Hispanic or Latino(a)(x).

Operationalizing Key Constructs

As noted in our Introduction, ASHA has not, thus far, provided definitions for authors with respect to our target demographic constructs (i.e., sex, gender, race, and ethnicity). Therefore, in order to operationalize coding of reporting practices for race and ethnicity, we adapted the classification system put forth by the NIH in order to “provide a common language to promote uniformity and comparability of data on race and ethnicity” and which are based on US federal guidelines (NIH, 2015a, para. V). Within the NIH framework, race is constructed as a choice of one of five racial categories (American Indian or Alaska Native; Asian; Black or African American; Native Hawaiian or Other Pacific Islander; White) based on ancestral origin, and ethnicity is constructed as a binary choice (yes/no) of claiming Hispanic or Latino(a)(x) culture or origin independently of race (NIH 2015a; see Table 2). We added racial categories for “Multi-racial” and “Other” to further reflect the most recent US Census and racial identities that fall outside of these minimum required NIH categories (US Census Bureau, 2020b).

Table 2.

Racial And Ethnic Categories, Derived According to NIH Guidelines

Race	Ethnicity
American Indian or Alaska Native	Hispanic or Latino(a)(x)
Asian	Not Hispanic or Latino(a)(x)
Black or African American
Native Hawaiian or Other Pacific Islander
White
Multi-racial
Other

Note. NIH = National Institutes of Health. Original NIH guidelines only require the first five categories and a “more than one race” designation (NIH, 2015). We used “Multi-racial” and added “Other” to reflect the most recent US Census and racial identities that fall outside of the required NIH categories (US Census Bureau, 2020b). Original NIH guidelines give ethnic categories as “Hispanic or Latino” and “Not Hispanic or Latino” and allow also for “Spanish origin” to be included (NIH, 2015). Parentheticals were added here for further gender inclusivity.

The NIH has also provided guidance for defining and disambiguating sex and gender for researchers (NIH, 2015b). Although we initially sought to characterize reporting practices and participant representation distinctly by both sex and gender, we determined during coding development that this would not be possible due to unclear operationalization and inconsistent utilization of these terms across much of the extant literature in CSD. For example, many studies reported participant “gender” as either male or female (i.e., using terms now chiefly recognized as referencing biological sex), without clearly indicating whether the reported data represented DNA-encoded, biological features (i.e., sex) or “social, cultural, and psychological traits” (i.e., gender; NIH, 2015b, para. II-A). Therefore, to proceed with coding, we collapsed all sex and/or gender data reported by authors to a single construct entitled “gender” except where explicitly reported otherwise, recognizing that this may not always reflect the intentions of study authors and their participants. Transgender participants were coded by their given gender identity, where reported. Non-binary and non-conforming gender identities were coded as gender-diverse. We additionally coded for all instances in which authors unambiguously and explicitly reported sex and/or gender constructs with intention and distinctly from one another. In accordance with this methodology and the limitations of available data, we will hereafter refer to sex/gender as simply gender except where otherwise specified.

Statistical Analyses

Once all discrepancies between primary and secondary coders were resolved and consensus codes were generated for all articles, data were exported from REDCap to a comma separated values spreadsheet for statistical analyses. All analyses were conducted with base functions of R, version 4.0.3 (R Core Team, 2020), with supplementary R packages binom (Dorai-Raj, 2014) and cNORM (Lenhard & Lenhard, 2020).

Intercoder agreement was calculated at the article level as a raw proportion of articles for which there was perfect agreement between first and second coders. We further calculated the raw proportion of perfect pairwise agreement for each critical data item, where relevant for any given study.

To answer our first research question regarding demographic reporting practices among participants in CSD research, we calculated sums and proportions for the number of studies reporting participants’ gender, race, and ethnicity. Subsequently, chi-square tests of independence were used to evaluate whether articles from NIH-funded studies were statistically more likely to report participant demographic characteristics than articles from studies without NIH funding.

To answer our second research question—which asked if research participants were representative of the national population with regard to gender, race, and ethnicity—we first calculated the proportion of individuals in each study in each demographic (gender, racial, and ethnic) category. In the absence of national-level prevalence data across CSD, we selected the US decennial census as the best available population-based measure of demographic distributions. Weighting all studies equally, we sought to estimate a “typical” proportion of participants in each demographic reported within the CSD literature. To estimate the “typical” proportion for our data, wherein distributions were skewed, we calculated median proportions using the Harrell-Davis estimator (Harrell & Davis, 1982). These methods ensured that the estimate of a “typical” proportion was not unduly influenced by individual studies with large numbers of participants or outliers in the distribution of study proportions. Uncertainty around median proportions was estimated using Bayesian bootstrapping (Rubin, 1981) with 100,000 resamples, from which we derived a 95% highest-density credible interval (CrI; Kruschke & Liddell, 2018). Instances wherein US population proportions (per US Census data) fell outside of the 95% CrI were considered to be significantly different from expectations based on known national demographics.

To answer our third research question, which sought to describe the proportion of studies that considered participant demographics in analyses and discussion, we calculated sums and proportions of articles that included mention of our key demographic constructs (i.e., sex, gender, race, ethnicity), where reported, in analysis sections, discussion sections, both, or neither. Studies employing both quantitative and qualitative methods were considered in assessing and summarizing the consideration of demographic variables in analyses and discussion.

Results

Intercoder Reliability

For all critical data items extracted during coding, there was perfect agreement between first and second coders in 65.4% of articles that met initial screening criteria (436 out of k = 667), with discrepancy resolutions required for the remaining 34.8% of articles. At the individual item level, proportions of perfect agreement between coders ranged from .74 to .99 (see Supplemental Table 1 for comprehensive reliability results).

Articles Coded and Participants Reported

Following the completion of coding according to procedures detailed above, we found k = 407 articles (comprising a total of n = 80,058 research participants) to meet full inclusion criteria (see Supplemental Table 2 for a complete list of included articles). Notably, two of these studies contributed a disproportionate number of participants to the total corpus: Walters et al. (2020) utilized a large medical center’s electronic health records in a sample of n = 14,625 participants, and Briley and Merlo (2020) analyzed data from a large national health survey for a sample of n = 10,151 human participants. Although our statistical methods accommodated these large sample sizes without bias, we have addressed these large n studies when discussing raw totals and proportions below.

As a preliminary analysis, we identified studies with a priori recruitment criteria relevant to our study (e.g., recruited only women; recruited only Hispanic participants). Specifically, 21 articles described targeted recruitment criteria by sex or gender (representing n = 1,270 participants, a mix of men/boys, women/girls, and individuals with gender-diverse identities); 3 articles targeted recruitment by race (representing n = 96 participants, various racial groups); and 7 studies targeted recruitment by ethnicity (representing n = 495 participants, 490 of which were Hispanic or Latino[a][x]).

Reporting Practices for Gender, Race, and Ethnicity

Authors reported participant gender in 85.5% (k = 348) of included articles. Authors reported participant racial group in 33.7% (k = 137) of included articles. Authors reported participant ethnicity, in a manner consistent with NIH guidelines (NIH, 2015a), in 13.8% (k = 56) of included articles (see Figure 2).

Figure 2. Demographic Reporting Practices Among Empirical Studies in ASHA Journals Published in 2020.

Note. ASHA = American Speech-Language-Hearing Association. Additional criteria for inclusion of studies were that participants were (a) human and (b) recruited in the United States. Gender was operationalized as any participant data encompassing masculine, feminine, or other gender-diverse identities and included data given as sex (e.g., male, female) unless explicitly differentiated from gender as a psychosocial identity. Race and ethnicity were operationalized per guidelines by the National Institutes of Health (NIH; 2015a), with race reflecting the following categories: American Indian, Asian, Black, Native Hawaiian or Other Pacific Islander, and White, as well as Multi-Racial and Other. Ethnicity reflects reporting of participants as Hispanic or Latino(a)(x) and Not Hispanic or Latino(a)(x).

Of those articles that reported participant gender, we coded for instances in which the constructs of sex and gender were explicitly used with purpose and clearly distinguished from one another. We found that 12 out of 348 articles (3.4%) demonstrated purposeful use of the terms sex and/or gender, by: (a) defining terms, (b) reporting both constructs, (c) including identity-oriented labels such as “gender diverse” and “cisgender,” and/or (d) focusing on transgender individuals.

Similarly, we coded for instances in which the constructs of race and ethnicity were differentiated where race was reported. We found that 49 out of 137 (35.8%) articles differentiated between the terms race and ethnicity, almost always in a manner consistent with NIH definitions (NIH, 2015a). We also reviewed “other” racial categories given by authors and found that an additional k = 43 out of 137 (31.4%) articles included some variation of “Hispanic or Latino(a)(x)” as undifferentiated from racial categories, rather than under a separate heading for ethnicity.

We subsequently sought to determine if reporting practices differed significantly between articles from NIH-funded studies versus articles from non-NIH-funded studies (k = 407 for all comparisons). Regarding gender, 88.6% of NIH-funded articles reported this information compared with 83.1% of non-NIH-funded articles, a non-significant difference (χ²(1) = 2.03, p = .15, OR = 1.58, 95% CI [0.86, 2.99]). Regarding race, 30.7% of NIH-funded articles reported compared with 35.9% of non-NIH-funded articles, a non-significant difference (χ²(1) = 1.01, p = .32, OR = 0.79, 95% CI [0.51, 1.22]). Regarding ethnicity, 18.8% of NIH-funded articles reported compared with 10.0% of non-NIH-funded articles, a significant difference (χ²(1) = 5.79, p = .02, OR = 2.08, 95% CI [1.13, 3.88]). In other words, NIH-funded studies were nearly twice as likely to report participant ethnicity—but not significantly more likely to report gender or race—compared with studies not funded by the NIH.

Comparisons with US Population Demographics

Table 3 summarizes frequency counts and median proportions of participants, by demographic group where reported, as compared with US decennial census data from 2020.

Table 3.

Gender, Race, and Ethnicity Representation in ASHA Articles Published in 2020

Demographic Construct (Where Reported)	Number of CSD Participants (% of Reported)	Median Proportion Across Studies	95% Credible Interval	US Population, %
Gendera
Women/girls	28,578 (46.1%)	51.5%	[49.5, 53.6]	50.5%
Men/boys	33,364 (53.8%)	48.4%	[46.3, 50.4]	49.5%
Gender-diverse	96 (0.2%)	0	[0, 0]	N/Aa
Race
White	26,144 (60.6%)	76.1%	[70.6, 81.4]	61.6%
Black	6,363 (14.7%)	4.6%	[2.5, 7.1]	12.4%
Asian	1,092 (2.5%)	0.1%	[<0.1, 0.4]	6.0%
American Indian	235 (0.5%)	<0.1%	[<0.1, <0.1]	1.1%
Native Hawaiian or Other Pacific Islander	10 (<0.1%)	<0.1%	[0, <0.1]	0.2%
Multi-racial	986 (2.3%)	0.1%	[<0.1, 0.3]	10.2%
Other	8,318 (19.3%)	13.7%	[8.6, 19.6]	8.4%
Ethnicityb
Hispanic or Latino(a)(x)	556 (15.1%)	6.2%	[3.7, 10.2]	18.7%
Not Hispanic or Latino(a)(x)	3,130 (84.9%)	85.4%	[70.2, 93.6]	81.3%

Note. ASHA = American Speech-Language-Hearing Association. CSD = Communication sciences and disorders. US = United States population data from 2020 US Census (US Census Bureau, 2020a). Gender was reported in 85.7% of coded articles. Race was reported in 33.5% of coded articles. Ethnicity was reported in 13.6% of coded articles.

^aThe US Census provides only sex data (male/female binary), with no “Gender-diverse” option or alternative. Our coding operationalized gender as any participant data encompassing masculine, feminine, or other gender-diverse identities; we included data given as sex (e.g., male, female) unless explicitly differentiated from gender as a psychosocial identity.

^bEthnicity data excludes k = 7 studies (n = 490 individuals) that purposively recruited Hispanic or Latino(a)(x) participants.

Where gender was reported, we characterized CSD research participants as men/boys, women/girls, or gender-diverse (e.g., non-binary, genderqueer). Across CSD studies reporting participant gender, women/girls (51.5%, 95% CrI [49.5, 53.6]) and men/boys (48.4%, 95% CrI [46.3, 50.4]) were broadly representative of the US population. The remaining n = 96 CSD research participants provided gender identities coded as gender-diverse and representative of various identities outside of a masculine-feminine dichotomy. Note that the 2020 US Census asks for individuals’ sex information and provides a forced choice of “male” or “female” (US Census Bureau, 2020a), with no option for representing gender identity. As such, there is no census-based referent for this category.

Where race was reported, we characterized CSD research participants as closely as possible to the NIH framework for racial categories: American Indian, Asian, Black, Native Hawaiian or Other Pacific Islander, and White (NIH, 2015a). We included two additional categories to further reflect participants’ race as reported: “multi-racial” (currently included within mandated targeted and planned enrollment reports for NIH funded studies as “More than One Race”) and “other.” Finally, we included a category for participants whose racial identity was reported as unknown, n = 892, not included among totals for race in Table 3.

Taken together, CSD studies reporting participant race contributed n = 44,040 participants, or 56.4% of all CSD research participants in empirical studies in 2020. Conversely, CSD studies with no participant racial information reported contributed n = 34,053 participants, or 43.6% of the total corpus. When excluding the two large studies (n > 10,000 participants) from the corpus, the adjusted number of participants with reported race in CSD studies comes to n = 20,682, or 37.8% of all CSD participants. See Figure 3 for participant race (raw proportions) in the context of reporting practices.

Figure 3. CSD Participant Race in Context with Reporting Practices for Race.

Note. CSD = communication sciences and disorders. This data excludes k = 2 studies contributing greater than 10,000 individual participants. Race categories were operationalized per guidelines by the National Institutes of Health (NIH; 2015a).

Among studies reporting participant race, the median proportion of White participants (76.1%, 95% CrI [70.6, 81.4]) was significantly higher than in the US population (61.6%). Median proportions of all remaining single-race groups (i.e., American Indian, Asian, Black, and Native Hawaiian or Other Pacific Islander) were lower among CSD participants compared with proportions in the US population (see Table 3). Of particular note, studies reporting race included a cumulative total of just n = 10 participants who identified as Native Hawaiian or Other Pacific Islander. The median proportion of individuals identifying as multi-racial was also lower among CSD research participants (0.1%, 95% CrI [<0.01, 0.3]) compared with the US population (10.2%), and the proportion of individuals classified as having other racial identities was higher among CSD research participants (13.7%, 95% CrI [8.6, 19.6]) compared with the US population (8.4%).

Where ethnicity was reported in a manner equivalent to NIH guidelines or otherwise distinct from race (NIH, 2015a), we characterized participant ethnicity as either Hispanic or Latino(a)(x) or Not Hispanic or Latino(a)(x). Among all CSD studies reporting participant ethnicity, Hispanic or Latino(a)(x) participants (9.4%, 95% CrI [4.6, 18.9]) approximated the US population, for which 2020 census data reflect 18.7% Hispanic or Latino(a)(x). A number of studies reporting participant ethnicity purposively recruited Hispanic or Latino(a)(x) participants, however. When excluding these k = 7 studies to adjust for the influence of targeted recruitment, we found that Hispanic or Latino(a)(x) participants were underrepresented among CSD research participants in studies that did not employ inclusion/exclusion criteria relevant to ethnicity (6.2%, 95% CrI [3.7, 10.2]) compared with the US population (see Table 3).

Further Consideration of Demographic Data in Analyses and Discussions

Coders indicated whether articles included references to sex, gender, race, or ethnicity in analyses (quantitative or qualitative) or in discussion sections. Where participant gender was reported, a majority of studies (64.7%) did not further consider either sex or gender, compared with those including at least some consideration in analyses (11.8%), discussion (10.9%), or both (12.6%; see Figure 4). Similarly, where participant race was reported, a majority of studies (67.2%) did not further consider either race or ethnicity, compared with those including at least some consideration in analyses (8.0%), discussion (17.5%), or both (7.3%; see Figure 5).

Figure 4. Consideration of Sex and/or Gender in Analysis and Discussion in ASHA Publications.

Note. ASHA = American Speech-Language-Hearing Association. Figure illustrates the number of articles that further considered sex and/or gender in the results and/or discussion sections. Sex was operationalized as assignment based on biological anatomy (e.g., male, female). Gender was operationalized as any other participant data encompassing masculine, feminine, or other gender-diverse identities, including instances wherein sex terms were not explicitly differentiated from gender as a psychosocial identity.

Figure 5. Consideration of Race and/or Ethnicity in Analysis and Discussion in ASHA Publications.

Note. ASHA = American Speech-Language-Hearing Association. Figure illustrates the number of articles that further considered race and/or ethnicity in the results and/or discussion sections. Race and ethnicity were operationalized per guidelines by the National Institutes of Health (NIH; 2015a), with race reflecting the following categories: American Indian, Asian, Black, Native Hawaiian or Other Pacific Islander, and White, as well as Multi-Racial and Other. Ethnicity reflects reporting of participants as Hispanic or Latino(a)(x) and Not Hispanic or Latino(a)(x).

Discussion

This systematic review and quantitative synthesis examined empirical studies published in ASHA journals in the year 2020 to characterize practices of reporting sex, gender, race, and ethnicity for CSD research participants in the US, with comparisons to broader national demographics. The present study represents the only investigation, to date, that has comprehensively examined participant demographic reporting practices and characteristics across CSD research. Results revealed that more than half of the empirical studies published in ASHA journals in 2020 omitted race and ethnicity data entirely, with heterogeneous reporting conventions and definitions among those that did report. Given these low rates of reporting, it is difficult to infer the overall representativeness of CSD research participants; nevertheless, in the subset of studies where race was reported, there were clear trends suggesting that non-White, minoritized racial groups may be underrepresented in CSD research relative to population data from the US census. Moreover, participant sex, gender, race, and ethnicity were most often reported without any subsequent analysis or discussion relevant to those demographic characteristics. Taken together, our findings point to several actionable steps that may bring greater accountability, consistency, and diversity to CSD research.

Inadequate Reporting Practices for Participant Demographics

A finding from our first research question, and perhaps the most striking result of the current study, was that nearly two thirds of the publications that we reviewed failed to report information about participants’ race or ethnicity. These findings are consistent with prior studies in CSD sub-disciplines (Ellis, 2009; Morton & Sandage, 2022; Nelson, 2022; Nguy et al., 2022; Pierce et al., 2014; Steinbrenner et al., 2022; West et al., 2016) and with broader biomedical science and education research (see National Academies of Sciences, Engineering, and Medicine, 2022; Sinclair et al., 2018). Such low rates of reporting for race and ethnicity in the year 2020 perhaps reveals the ongoing limitations of past and current practices, underscoring a need for new policies and strategies to address gaps in obtaining even a baseline understanding of representation of different racial and ethnic groups in CSD research. The NIH has had longstanding policies regarding the inclusion of women and minorities in relevant research endeavors, with reporting of participant demographics required in inclusion enrollment reports (NIH, 2001). However, this NIH requirement has apparently not translated to adequate transparency and accountability in subsequent publications, as we found that NIH-funded studies were no more likely to report participant race than non-NIH-funded studies.

The overall proportion of studies reporting participant ethnicity, in terms of Hispanic or Latino(a)(x) versus Not Hispanic or Latino(a)(x), was even lower—just under 15%—than that of participant race. We noted discrepancies in how researchers defined terms, frequently conflating ethnicity with race or presenting demographic constructs without clear definitions. Here we may see an effect of NIH requirements for collecting demographic information, as NIH-funded studies were nearly twice as likely to report participant ethnicity—at closer to 18% reporting—than non-NIH-funded studies, although it is difficult to draw conclusions about participant diversity where any given characteristic is infrequently reported and inconsistently operationalized.

Participant gender, although reported among a higher proportion (over 85%) of studies than race or ethnicity, presents similar challenges regarding ambiguous operationalizing and heterogeneous reporting practices. The most notable issue we encountered was that, ultimately, we were unable to determine whether the vast majority (over 95%) of studies specifically summarized information relevant to participant sex as a biological trait, or information specific to some aspect of participant gender identity. As an example, we consider Hosbach-Cannon et al. (2020), a comparison of functional laryngeal physiology between populations of professional singers. The authors reported participant “sex” as male or female, but then described within-group “gender differences” in study Methods and Discussion, without a clear definition or description of how demographic data were collected (Hosbach-Cannon et al., 2020). In such a study, precise findings and interpretations may be dependent on whether sex/gender terms refer primarily to physiological traits associated with biological sex differences, or to traits associated with lived gender roles and expression, or to a combination of the two. There is a need for such constructs to be more clearly defined and consistently utilized, as we would expect aspects of biology and identity to be variously relevant to a wide range of research questions in CSD.

Of relevant note regarding reporting practices, we additionally observed a lack of consistency in terminology, detail, and formatting of key demographic information. Although we did not code for stylistic aspects of reporting practices in a manner that could be readily quantified, we made informal notes on a range of practices utilized by authors over the course of coding, as they frequently impacted readability and transparency of study data. Some authors included demographic information in participant tables, while others described the information in body text. Many articles reported demographic data rather comprehensively, following an approach similar to the NIH framework discussed above, while other articles provided only partial or collapsed data with selected categories. One common example of partial reporting was for authors to provide gender data by omission, such as by reporting only a number or percentage of “men” or “boys” and presumedly leaving it to the reader to infer the remainder as “women” or “girls.” Labels for demographic categories—in particular, racial categories—were often constructed idiosyncratically and did not always allow for clear comparison across studies. For instance, we encountered such labels as “racial or ethnic minority,” and “Other (Asian, Hispanic, Interracial).” Moreover, some studies reported raw counts, other studies reported percentages, and still others reported both. Although it is not incorrect or problematic for there to be variability in the approaches that authors employ when collecting and reporting demographic data—in particular because such variability in design and dissemination of research may best suit the scientific aims or characterize the target population of interest—these heterogeneous practices also underscore a lack of shared definitions across researchers in our discipline. CSD researchers may benefit from consistent operational definitions and standardized style guidelines in order to: (a) promote increased understanding of key sociocultural constructs; (b) facilitate more accessible uptake of information by clinicians, researchers, and other readers; and (c) enable multiple studies to be more readily summarized and synthesized in systematic reviews and meta-analyses.

Apparent Gaps in Representation

To draw conclusions on the broad representativeness of CSD research participants, we calculated median proportions for gender, racial, and ethnic groups and compared these proportions with US Census data. Given that so much demographic data across CSD was omitted and considering that we have found a lack of consensus regarding definitions, we must interpret these results cautiously. From the available CSD data, however, the comparison with the US Census points to underrepresentation among non-White racial groups including Asian, American Indian, Black, and Native Hawaiian and Other Pacific Islanders. This latter group was notably represented by just 10 reported participants out of the cumulative corpus of over 80,000 participants included in CSD research as published in ASHA journals over the 2020 calendar year. In contrast to this striking statistic, we found that few studies (k = 3) endeavored to specifically recruit by race for focused intra-group research. Taken together, these findings suggest that representation of the aforementioned racial groups may be particularly inadequate among CSD research participants. See below for further discussion on issues related to targeted versus proportional representation and for subsequent recommendations for moving the needle on this pressing issue.

Comparisons between CSD data and the US national census are imperfect due in part to constraints on the construction of the census, and in particular with regard to sex and gender where the census forces a binary choice of male or female sex (see Limitations, below). Nevertheless, this comparison highlights two important findings related to gender in CSD research. First, based on median proportions, CSD participants appear to roughly approximate proportions of the US population for females and males. Second, studies in CSD recognize at least some, albeit a small number (n = 96), of participants with gender-diverse identities, where there is a lack of census information about these identities at the population level. This latter finding is consistent with other studies showing limited inclusion or acknowledgment of non-conforming, non-binary, or non-traditional gender identities (Ghorbanian et al., 2022; Merritt, 2020) despite recently increased visibility of gender-diverse identities in mainstream society (e.g., Collin et al., 2016).

Superficial Engagement with Demographic Data

We have found that a lack of visible and readily interpretable demographic data is a clear issue in CSD research, and reporting these data is certainly a critical first step to consider towards increased researcher accountability. At the same time, simply reporting demographic data without further engagement may not be sufficient. Where CSD studies reported the demographic constructs of interest to this study—sex, gender, race, and/or ethnicity—we found that investigators were not likely to consider such information any further. In most cases where demographic data was reported, no subsequent analyses (e.g., between-group statistical comparisons, consideration of demographic variables as covariates) were conducted. Similarly, articles tended not to include any text in discussion sections relevant to demographic constructs; this could have included any number of discussion points or recommendations, such as contextualizing the research sample, interpreting findings by demographic group, commenting on study limitations, or highlighting directions for future research.

Recommendations to Consider

Our findings in the present study, along with a review of existing literature, support several steps that can be taken immediately towards increasing representation and diversity in CSD research. As a starting point, leadership at ASHA should consider highlighting research as an important area in which to improve diversity, with a policy statement such as that suggested by Ahmed et al. (2020) to commit to inclusion and representation in translational research. Additional recommendations follow below, with a summary checklist for CSD researchers provided in Figure 6.

Figure 6. Recommendation Summary Checklist for Researchers.

Reporting as a First Step for Researcher Accountability

In a recent congressionally-ordered report, the National Academies of Sciences, Engineering, and Medicine (NASEM; 2022) examined the issue of representation in science at length and presented recommendations to stakeholders in biomedical research enterprises, including the NIH, to improve guidelines for reporting and accountability. Chief among these stakeholders were journal editors and publishers, who were encouraged to:

…require information on the representativeness of trials and studies for submissions to their journals, particularly relative to the affected population; … consider this information in accepting submissions; and … publish this information for accepted manuscripts. (NASEM, 2022, p. 131)

Many other interdisciplinary scholars have called for top-down, mandated reporting and analyses of key demographic data to account for research diversity (Ahmed et al., 2020; Boyd et al., 2020; Flanagin et al., 2021; Hunt et al., 2022; Jackson et al., 2021).

Intentional Demographic Methodology as a First Step for Participant Visibility and Context

Beyond mandated reporting, there are parallel needs for CSD researchers to establish functional, purposeful, inclusive, and geographically relevant definitions and practices around demographic constructs and concepts (Ellis et al., 2021; Jackson et al., 2021; Merritt, 2020). Researchers might consider providing clear definitions for key demographic terms, and they might consider reporting their methods for collecting and classifying demographic data. Some researchers (e.g., Baker et al., 2006) have explored alternative methods for eliciting racial/ethnic identity with open-ended questions or response fields, showing that such data can be summarized fairly reliably in accord with commonly used demographic categories. An open-ended, multi-step approach may be of particular importance for differentially eliciting participant sex and gender information; such an approach involves first requesting gender identity and then separately requesting sex as assigned at birth (see Haupert et al., 2018). Regardless of the selected framework, study authors should comment on their approach to collecting and summarizing key sociocultural demographic variables in a study.

Recruiting for Proportional Versus Targeted Representation

At the outset of this study, we had hoped to evaluate the degree to which CSD research participants were proportionally representative of the broader US population regarding sex, gender, race, and ethnicity. Although we were unable to fully answer this question due to inconsistent operationalization of terms regarding sex and gender data and low overall rates of reporting race and ethnicity data, our findings have led us to several related follow-up questions, such as: is proportional representation an appropriate standard for diversity? Is proportional representation a sufficient metric in a discipline where research sample sizes are often small (e.g., n < 100)? How can we best characterize “representativeness” at the level of individual studies in a manner that can be compared across the heterogenous body of CSD research overall?

On the one hand, for many research designs, proportional representation may be a useful guidepost for researchers to pursue as they self-audit for selection bias and other threats to validity. A lack of general representation may be related, at least in part, to overreliance on convenience sampling (i.e., only 2.25% of studies in our corpus utilized random sampling); exclusionary criteria that may covary to some degree with aspects of race and/or ethnicity (e.g., including only monolingual English speakers may disproportionately exclude Asian and Hispanic or Latino[a][x] participants); and/or recruitment strategies that may not reach the entire population of interest to CSD researchers. Yet, by striving for proportional representation—relative at least to the national population and more optimally the prevalence of communication disorders—researchers may increase the external validity of CSD research. Several different strategies and research designs may be leveraged to increase proportional representation, if appropriate, such as directly engaging with demographic groups in the community (e.g., community-engaged research; see De Las Nueces et al., 2012; Gonzalez et al., 2021) or allocating resources to improve access to participation (e.g., Flaskerud & Nyamathi, 2000).

On the other hand, proportional representation in some ways falls short as a heuristic method for broadly diversifying research participation in CSD. At best, proportional representation reproduces the unequal distributions found in the broader population, with the most marginalized groups and individuals hidden or masked within the larger data set due to low relative frequencies and weak statistical power for detecting any effects relative to demographic factors such as race or ethnicity (Allmark, 2004; Gilmore-Bykovskyi et al., 2019; Yancey et al., 2006). Alternatively, researchers could design studies intended to oversample or exclusively recruit participants from underrepresented groups. Within our corpus of research articles, targeted recruitment appears to have successfully contributed to expanded representation for Hispanic or Latino(a)(x) participants, at least for specific research questions regarding Latino(a)(x) culture and Spanish-English multilingual issues (e.g., Cycyk & Huerta, 2020; Jasso et al., 2020; Orizaba et al., 2020). Based on our review of studies in 2020, there appears to be much opportunity for researchers in CSD to conduct investigations within targeted demographic groups for a deeper and more nuanced understanding of characteristics and best practices within various populations.

Limitations and Caveats

We must acknowledge several important limitations and caveats inherent to the current study. First, our study represents a comprehensive snapshot of articles published in a single year—2020—by a single publisher—ASHA—with no examination of possible changes over time. We also acknowledge that other academic journals that publish research relevant to CSD may prioritize multicultural research in other ways and may specify submission guidelines that result in different patterns of reporting practices and representation. Moreover, given the size and heterogeneity of the article corpus, we did not evaluate the relative strengths of study designs and other potential sources of bias beyond participant demographics.

Second, our study extracted data on a large number of participants from a large number of different studies, requiring the narrow operationalizing of broad sociocultural constructs like gender and race. Although we can point to gross trends and national-level findings around CSD participant gender, race, and ethnicity, we may have lost substantial detail and nuance around the use of these and other variables. For example, racial/ethnic identities that fall outside of the NIH framework (e.g., Middle Eastern or North African individuals; Awad et al., 2021) were subsumed in our coding by a catchall “Other” category. Comparisons to US Census data were similarly constrained by limitations in the construction and collection of census data, and do not necessarily reflect national prevalence of communication disorders. This is especially important to consider, given that many of the demographic groups who were found to be underrepresented in the literature are known to be disproportionately impacted by communication disorders.

We were also unable to examine alternative, potentially more important sociocultural variables such as socioeconomic status, disability status, health care access and use, primary language, housing, or environment in this report (see Ellis & Jacobs, 2021; Kilbourne et al., 2006). Additionally, we were unable to compare CSD participants to regional- or state-level demographics due to low rates of reporting of recruitment catchment areas. Thus, the use of national-level participant data restricted our comparison analyses to national-level (i.e., census) data.

Despite these limitations, the current study can be cautiously interpreted as a baseline for current gender, racial, and ethnic representation and reporting practices across CSD research. Future meta-scientific studies should examine trends for representation over time, endeavor to extract more detailed information about related issues such as recruitment and retention, and delve more deeply into aspects of these broad and complex sociocultural constructs. Future scholarly work should incorporate diverse research designs and methodologies (e.g., qualitative and mixed methods research, critical review) toward further evaluating representation within CSD as situated within broader social systems of inequities in persistent efforts to identify, measure, and systemically address exclusionary research practices.

Learning outcomes:

As a result of this article

• the reader will be able to define sex, gender, race, and ethnicity.

• the reader will be able to describe broad trends in demographics reporting and participant diversity in recent communication sciences and disorders research.

• the reader will be able to discuss action steps for improved reporting and representation in future research projects in the field.

Data Availability Statement

Study materials (e.g., coding manual and coding shell) and datasets generated and analyzed during the current study are available in the Vanderbilt University Institutional Repository, http://hdl.handle.net/1803/18059