Prevalence of bias against neurodivergence-related terms in artificial intelligence language models

Abstract

Given the increasing role of artificial intelligence (AI) in many decision-making processes, we investigate the presence of AI bias towards terms related to a range of neurodivergent conditions, including autism, ADHD, schizophrenia, and obsessive-compulsive disorder (OCD). We use 11 different language model encoders to test the degree to which words related to neurodiversity are associated with groups of words related to danger, disease, badness, and other negative concepts. For each group of words tested, we report the mean strength of association (Word Embedding Association Test [WEAT] score) averaged over all encoders and find generally high levels of bias. Additionally, we show that bias occurs even when testing words associated with autistic or neurodivergent strengths. For example, embedders had a negative average association between words related to autism and words related to honesty, despite honesty being considered a common strength of autistic individuals. Finally, we introduce a sentence similarity ratio test and demonstrate that many sentences describing types of disabilities, for example, “I have autism” or “I have epilepsy,” have even stronger negative associations than control sentences such as “I am a bank robber.”

Lay Summary

Our work tests how words pertaining to neurodivergent conditions such as “autism” or “ADHD” are viewed by artificial intelligence (AI) language models. AI is involved in many decisions such as medical decision making and scanning resumes. This means that it is important to “de-bias” AI so that autistic and other neurodivergent people do not experience discrimination in job applications or other AI-based processes.

INTRODUCTION

Humans are increasingly relying more and more on artificial intelligence (AI). AI plays a key role in tasks ranging from medical decision-making (Giordano et al., 2021; Kumar et al., 2022) to resume scanning and job candidate selection (Lacroux & Martin-Lacroux, 2022) to dating apps (Sharabi, 2022; Wu & Kelly, 2020).

However, AI algorithms are still trained with human-produced data and therefore may learn any human biases or prejudices to which they are exposed. AI algorithms that learn connections between different words and concepts do so by studying large quantities of text written by humans. Previously, it has been shown that AI algorithms have learned racially biased connections from human texts and associated stereotypically White names with positive words such as “love” while associating stereotypically Black names with negative words such as “cancer” (Caliskan et al., 2017). Likewise, a previous AI algorithm trained by Amazon.com, Inc. learned to favor male candidates and penalized any resumes that mentioned participation in women’s groups such as a women’s science club (Dastin, 2018).

Previous work has indicated that biases in word embeddings (i.e., inappropriate correlations between words) tend to propagate to other tasks such as classification and ranking (Verma & Rubin, 2018). For example, a restaurant review algorithm associated the word “Mexican” with “illegal” and, as a result, automatically penalized Mexican restaurants (Speer, 2017a). Likewise, models trained on biased word embeddings lead to biased sentiment predictions (Papakyriakopoulos et al., 2020), as well as undesired gender correlations in algorithms designed to recognize depression (Sogancioglu & Kaya, 2022). As such, our work focuses on identifying potential biases or unwanted correlations in word and sentence embedders related to neurodiversity.

Fortunately, scientists and technology companies have been making strides to remove negative stereotypes about ethnicity, gender, religion, disability status, and age (Bolukbasi et al., 2016) from word and sentence embedding algorithms. While fairness in AI is a complex sociotechnical challenge with ongoing research and debate, debiasing algorithms have successfully reduced many overt negative stereotypes that have been identified. Recent papers have proposed techniques to debias neural networks (Parraga et al., 2022), demonstrated successful mitigation of racial bias in AI-analysis of images (Savani et al., 2020), mitigated gender bias in pretrained word embeddings (Kaneko & Bollegala, 2019), and removed AI gender stereotypes related to occupation (Bolukbasi et al., 2016). Unfortunately, biases related to neurodiversity have been less frequently discussed or identified, such that identifying existing biases related to neurodiversity is a necessary first step in the debiasing process.

We therefore aim to investigate the prevalence and degree of AI-bias against neurodivergent individuals. While definitions of neurodivergence vary, we consider neurodivergence to include any pattern of thinking or communicating that significantly differs from what is considered typical. In this article, we focus on a subset of conditions reflecting neurodivergence including autism, ADHD, schizophrenia, and obsessive-compulsive disorder.

Based on prior research, we hypothesized that AI word encoders which are trained on large amounts of human generated text may pick up negative correlations between terms related to neurodivergence and terms reflecting stigma such as “bad,” “dangerous,” and so forth. Our hypotheses were motivated by prior research demonstrating ways in which neurodivergence may be stigmatized, such as perceptions of autistic or neurodivergent individuals as violent (Angermeyer & Matschinger, 1996; Gillespie-Lynch, 2020), odd or unnatural (Faso et al., 2014), unattractive (Sasson et al., 2017), unempathetic (Shalev et al., 2022), and child-like (Stevenson et al., 2011). Autism and neurodivergence have also been studied as diseases (Sarrett, 2011) or a cause of grief to family members (Bravo-Benítez et al., 2019). Such biases could contribute to the barriers that neurodivergent individuals face, ranging from difficulty in accessing healthcare (Doherty et al., 2022) to high rates of unemployment and underemployment (Ohl, 2017). Understanding whether and how bias appears in word encoders may additionally provide a starting point for identifying other applications of AI which could potentially benefit from debiasing. For example, if multiple word encoders correlate terms related to neurodivergence with “badness,” it may be beneficial to investigate whether that correlation also impacts content related to neurodivergence in content distribution algorithms.

More generally, prior literature has demonstrated discrimination against individuals with a variety of disabilities (Barbareschi et al., 2021; Read et al., 2015; Wallhagen, 2010) as well as robust discussion on autism-related terminology (Taboas et al., 2023). This provided motivation for testing the perceived goodness of various phrases and sentences related to autism and disability. We anticipated that sentences related to neurodivergence or disability may have a lower perceived goodness than control sentences, and initially anticipated that terms generally preferred by the autistic community may have a higher perceived goodness than alternative terms describing autism.

METHODS

Overview

To measure bias towards neurodivergent individuals in different word-embedding algorithms, we use the Word Embedding Association Test (WEAT) score, which is a standard measure of bias. We test 11 different AI-based word and sentence embedding algorithms for several types of negative bias that neurodivergent individuals may encounter. In our results, we find high average levels of bias across the board, with the single exception of the potential bias that neurodivergent individuals are unempathetic. We additionally tested AI’s perception of common autistic strengths such as honesty or a generally accepting and inclusive attitude. Our results show that a negative bias against autistic and neurodivergent individuals persists even in these areas of expected strength.

Finally, we introduce a sentence embedding ratio test that quantifies the perceived goodness of a sentence (e.g., “I am an autistic person”). We then test the perceived goodness of different disabilities and levels of support needs in 12 different sentence encoders as well as comparison sentences such as “I am a bank robber.”

Word embeddings

In order to make sense of words, computers need to first convert words to numbers. More specifically, computers implement word embeddings where each word is turned into a corresponding vector of many numbers. This vector encodes the meaning of the word, and vectors that are mathematically close together represent words with similar meanings.

To illustrate this concept, we begin with a simplified example which is illustrated in Figure 1. Suppose words are assigned to a single number between −5 and 5, where higher numbers are perceived as “closer to good” and lower numbers are perceived as “closer to bad.” A sample set of words such as {“happy,” “nuisance,” “enemy,” “good,” “bad,” “nurse,” “robber,” “potato”} might then be assigned numbers as depicted below.

FIGURE 1.

Hypothetical one-dimensional embeddings of sample word-set based on whether the word is closer to “good” or “bad.”

However, this simplified model only allows one quality—the word’s perceived similarity to “good” or “bad”—to be encoded. As such, even adjacent words may have little in common outside of their perceived goodness.

By adding dimensions, we can capture more information about a word’s meaning and correlations between words. For example, we can upgrade our one-dimensional model to a two-dimensional model where the x-axis runs from “good” to “bad” and the y-axis runs from “human” to “nonhuman.” This process is illustrated in Figure 2.

FIGURE 2.

Illustration of two-dimensional embeddings of sample word-set based on two qualities. The x-axis represents perceived goodness, and the y-axis represents how closely the word relates to humans.

In this example, we can see that words such as “robber” and “enemy” remain close, as both are negative labels that often refer to people, while words such as “nurse” and “beach” are now further apart.

Actual word embeddings often consist of vectors with over 100 dimensions and contain a large number of embedded words. These vectors are generated based on massive quantities of text (Mikolov et al., 2013), for example, the word embedding process might operate on 100 billion words of human writings taken from Google or another publicly available source and will create embeddings that preserve relationships between words.

Words that frequently occur together will be assigned to vectors that are close together, indicating a similar meaning. If the text the word embedding is trained on has inappropriate correlations (such as frequently using the words “Mexican” and “illegal” close together), the algorithm may learn this inappropriate connection and perceive the two words as similar.

Unwanted bias then appears when models learn inappropriate correlations between words. Figure 3 illustrates this effect in a real word embedding system. The figure utilizes a two-dimensional projection of word embeddings in word2vec-google-news-300, which was trained on about 100 billion words. The blue region consists of space which is more correlated to the set {“party,” “outgoing,” “fun,” “easygoing”} while the red region consists of space which is more correlated with the more socially stigmatized set {“uptight,” “obsessive,” “loner,” “stalker”}. Words related to neurodiversity, especially “autistic” and “schizophrenia,” are placed in the red region, indicating an association with the socially stigmatized word set, whereas words such as “normal” and “typical” are in the blue region next to terms such as “outgoing” and “fun.”

FIGURE 3.

Two-dimensional projection of different word sets. Words corresponding to blue dots are generally associated with fun and words associated with red dots generally have socially negative connotations. To generate this figure, we implement principal component analysis on the words {“party,” “outgoing,” “fun,” “easygoing,” “obsessive,” “loner,” “stalker,” “uptight”}, resulting in a two-dimensional projection. The blue region is more correlated to {“party,” “outgoing,” “fun,” “easygoing”} while the red region is more correlated with the more socially stigmatized set {“obsessive,” “loner,” “stalker,” “uptight”}. We then project the remaining set of words-{“adhd,” “autistic,” “schizophrenia,” “normal,” “typical”}- into that space.

Word Embedding Association Test

We begin by testing 16 potential biases across 11 word and sentence embedders that have been commonly used in the past 5 years. The strength of each potential bias is quantified by its WEAT score (Caliskan et al., 2017). The WEAT score measures the association between two pairs of word sets. For example, one pair might include a word set representing “female” such as {female, woman, girl, wife, actress, heroine} and another word set representing “male” such as {male, man, boy, husband, actor, hero}. If the second pair consisted of word sets related to “nurturing” and “leadership,” respectively, the WEAT score of the two pairs would measure the association between words related to gender (male–female) and personality (nurturing–leadership).

Mathematically, the WEAT score is given by

$$d(X,Y,A,B)=\frac{\text{mea}{\text{n}}_{\{x\in X\}}(s(x,A,B))-\text{mea}{\text{n}}_{\{y\in Y\}}(s(y,A,B))}{\text{stdde}{\text{v}}_{\{w\in X\cup Y\}}S(w,A,B)}$$

where $X$ and $Y$ are word sets of equal size which comprise the first pair, $A$ and $B$ are word sets of equal size which comprise the second pair, stddev represents the standard deviation, and where.

$$s(w,A,B)=\text{mea}{\text{n}}_{\{a\in A\}}\cos (w,a)-\text{mea}{\text{n}}_{\{b\in A\}}\cos (w,b)$$

where $\cos (w,a)$ corresponds to the cosine similarity of word $w$ and word $a$ (intuitively, how close these two words are) While alternative versions of the WEAT score exist, we use the normalized WEAT score which ranges between −2 and 2. Higher magnitudes represent higher overall levels of bias.

Intuitively, the WEAT score compares how much more similar words in Group X are to Group A than Group B; as well as how much more similar words in Group Y are to Group B than Group A. For example, in the implicit bias test comparing Female–Male to Nurturing–Leadership, the WEAT score increases when words related to “nurturing” are closer to “female” terms than “male” terms, and likewise increases when words related to “leadership” are closer to “male” terms than “female” terms. Figure 4 depicts potential similarities between words from different sets for the test Female–Male to Nurturing–Leadership. As such, WEAT scores are a useful metric of how humans associate different words, and can be a useful predictor of human bias. If human writings consistently associate “woman” and “homemaker” and, respectively, “man” with “leader,” this may reflect underlying bias regarding gender roles. However, a high correlation between groups of words, as indicated through a high WEAT score, does not guarantee a high level of human bias; and a WEAT score close to zero does not guarantee the absence of bias. For example, if multiple text sources describe autistic individuals as “lacking in empathy,” word embedders may create a connection between the words “autism” and “empathy” despite the text sources in this example carrying the opposite bias.

FIGURE 4.

A hypothetical illustration of bias related to Nurturing occupations (blue)–Leadership occupations (red) and Female (green)–Male (silver). This example plot would have a high Word Embedding Association Test score, as we can see that “nurturing” terms such as “secretary” are much closer to terms “female,” “girl,” and “woman” than terms “boy,” “male,” and “man.” Likewise, terms related to leadership are much closer to male terms.

Biases to test

In the following section, we identify several potential biases that may apply to neurodivergent people. We then calculate the WEAT score discussed above to test the strength of each potential bias against neurodivergent individuals, with neurotypical individuals used as the basis for comparison. Our first bias test is motivated by the prevalence of derogatory slurs to refer to people with intellectual and developmental disabilities (Siperstein et al., 2010) and examines whether word embeddings associate terms related to neurodiversity with such slurs. Our second test is for the pair Tragedy–Celebration (Bravo-Benítez et al., 2019).

Given the high levels of social stigma against autistic and neurodivergent individuals (Sasson et al., 2017) as well as common stereotypes that autistic individuals do not experience empathy (Shalev et al., 2022), we implement several tests relating to social perceptions of neurodiversity: Annoying–Pleasant, Weird–Cool, Obsessive–Easygoing, Emotionless–Warm, and Unattractive–Attractive. We additionally test Violent–Safe (Angermeyer & Matschinger, 1996), Child–Adult (Stevenson et al., 2011), and Weak–Competent.

Finally, we add three general tests related to “goodness” (good–bad), kindness (kind–unkind), and terms related to inherent value or worthiness (worthy–worthless). The motivation behind these more general tests is to determine whether negative AI bias towards neurodivergent individuals exists in most potential tests, or whether bias predominantly occurs in tests where neurodivergent individuals have historically been stigmatized (such as violence, social stigma, etc.).

In addition to biases tested, we test potential correlations between neurodivergence and other identities that are associated with autism. Studies have shown a strong correlation between autism and LGBTQ+ identity (Hillier et al., 2019), and autistic individuals also have high rates of unemployment and underemployment (Ohl, 2017). This motivated us to test whether AI algorithms connect terms related to neurodivergence with terms related to LGBTQ+ identity and/or to socioeconomic status. These tests are labeled by the pairs Gay–Straight and Poor–Rich, respectively.

For each test, we choose a specific word or sentence encoder to test biases on. Each encoder comes already trained on a large set of data. For example, the encoder “glove twitter” is trained on 2 billion tweets and “glove-wiki-gigaword” is trained on 1 million word vectors from Wikipedia in 2017. Other encoders were trained on large amounts of texts generated from a diverse set of sources. We now list the word encoders tested as well as the date each encoder was released or last updated, or the date accessed: conceptnet numberbatch 16.09 (Speer et al., 2016; September 14, 2016), conceptnet numberbatch 17.06 (Speer, 2017b; June 2017), word2vec-google-news-300 (Mikolov et al., 2013; December 2, 2021), glove-twitter-50 (Pennington et al., 2014; December 2, 2021), glove-wiki-gigaword-300 (Pennington et al., 2014; December 2, 2021), and corpus text8 (Mahoney, 2011; accessed January 2023) with Gensim’s model Word2Vec (Rahman, 2020). In addition to testing biases on word encoders, we also tested each potential bias with the following sentence encoders: all-MiniLM-L6-v2 (Hugging Face, 2022a; November 7, 2022), multi-qa-mpnet-base-dot-v1 (Hugging Face, 2022b; July 11, 2022), bert-base-nli-mean-tokens (Reimers & Gurevych, 2019; updated June 2022, original publication date of November 2019), GPT2 (Radford, 2019; updated December 2, 2020), and OpenAI 0.26.4 (OpenAI, 2023a, 2023b; released January 2023) canonically referred to as GPT-3. GPT-3 offers four models, and we select model text-similarity-curie-001 for testing. Unless stated otherwise, we set the neurodivergent word set to be {“autism,” “autistic,” “asperger,” “schizophrenia,” “ocd,” “adhd”} and the neurotypical word set to be {“typical,” “regular,” “average,” “neurotypical,” “normal,” “allistic”}. Each embedder has a different vocabulary, and less commonly recognized terms such as “allistic” were not recognized by all embedders. Due to these vocabulary limitations, we removed the words “allistic” and “ocd” from word2vec-google-news-300. For text8, glove, and glove-wiki-gigaword-300, additional vocabulary limitations required us to remove the terms “neurotypical” and “asperger.” The Supplementary material S1 contains further information about all encoders in table form and Table 1 contains the full set of words contained in each test word set.

TABLE 1.

Full word sets per tests.

Test	Word set X	Word set Y
Idiot-Smart	{“idiot,” “retard,” “retarded,” “moron,” “imbecile”}	{“smart,” “intelligent,” “clever,” “bright,” “good”}
Pity-Joy	{“sad,” “sympathy,” “pity,” “grief,” “sorry”}	{“congratulations,” “celebrate,” “happy,” “joy,” “hope”}
Unattractive-Beautiful	{“ugly,” “asexual,” “unattractive,” “loner”}	{“sexy,” “attractive,” “beautiful,” “desirable”}
Childish-Mature	{“child,” “immature,” “young,” “childish,” “baby”}	{“adult,” “mature,” “responsible,” “grown,” “elderly”}
Dangerous-Safe	{“scary,” “violent,” “danger,” “threat,” “fear”}	{“safe,” “calm,” “secure,” “safety,” “protection”}
Diseased-Healthy	{“diseased,” “defective,” “broken,” “sick,” “deficit”}	{“healthy,” “natural,” “lively,” “health,” “well”}
Annoying-Pleasant	{“annoying,” “rude,” “tiresome,” “irritating”}	{“pleasant,” “agreeable,” “easy,” “delightful”}
Weak-Competent	{“weak,” “unable,” “helpless,” “powerless,” “effective”}	{“power,” “powerful,” “capable,” “able,” “competent”}
Weird-Cool	{“weird,” “awkward,” “strange,” “inappropriate,” “loner”}	{“social,” “cool,” “friendly,” “appropriate,” “confident”}
Emotionless-Warm	{“robotic,” “mechanical,” “alien,” “emotionless,” “cold”}	{“warm,” “caring,” “empathy,” “love,” “human”}
Uptight-Easygoing	{“obsessive,” “uptight,” “loner,” “stalker”}	{“easygoing,” “fun,” “party,” “outgoing”}
Good-Bad	{“bad,” “terrible,” “awful,” “wrong”}	{“good,” “wonderful,” “great,” “excellent”}
Worthy-Worthless	{“worthless,” “trash,” “nothing,” “empty”}	{“valuable,” “worthy,” “important,” “precious”}
Kind-Unkind	{“unkind,” “mean,” “cruel,” “vicious”}	{“kind,” “caring,” “gentle,” “kindly”}
Poor-Rich	{“poor,” “poverty,” “powerless,” “disadvantaged”}	{“wealthy,” “rich,” “powerful,” “power”}
Gay-Straight	{“gay,” “transgender,” “homosexual,” “lesbian”}	{“straight,” “heterosexual,” “traditional,” “cisgender”}
Creative-Unoriginal	{“unique,” “creative,” “insight,” “innovation”}	{“boring,” “unimaginative,” “unoriginal,” “stagnation”}
Accepting-Judgmental	{“accepting,” “inclusive,” “openminded,” “unbiased”}	{“unaccepting,” “judgmental,” “biased,” “prejudiced”}
Reliable-Fickle (autism-specific test)	{“dedicated,” “loyal,” “steadfast,” “reliable”}	{“disloyal,” “fleeting,” “unfaithful,” “fickle”}
Honest-Dishonest (autism-specific test)	{“honest,” “forthcoming,” “straightforward,” “direct”}	{“deceitful,” “lying,” “dishonest,” “indirect”}

We chose a broad range of encoders that have been used in recent years and included prominent encoders such as Chat GPT or encoders, which have been included in previous research papers related to WEAT scores. To illustrate the impact of debiasing techniques, we include both a biased and debiased version of the encoder Concept Numberbatch.

Sentence Embedding Ratio Tests

When testing biases at the sentence level, we introduce a Sentence Embedding Ratio Test (SERT) score. More specifically, we define

$$\text{SERT}(\text{sentence,}X,Y)=\frac{1}{\left|X\right|}{\sum }_{i=1}^{\left|X\right|}\frac{\text{similarity}\left(\text{sentence,}{x}_i\right)}{\text{similarity}\left(\text{sentence,}{y}_i\right)}$$

where $X=\left\{x_1,x_2,\dots \right\}$ and $Y=\left\{y_1,y_2,\dots \right\}$ are paired sets of “good” and “bad” sentences and $|X|$ is the size of the set.

For SERT tests, we use an extended set of sentence transformers, nine of which use Huggingface (Wolf et al., 2020). The corresponding models and date that each model was last updated are All-distilroberta-v1 (July 11, 2022), sentence-transformers/all-mpnet-base-v2 (July 11, 2022), sentence-transformers/all-MiniLM-L12-v2 (July 11, 2022), Distiluse-base-multilingual-cased-v2 (June 15, 2022), Multi-qa-distilbert-cos-v1 (July 11, 2022), multi-qa-mpnet-base-dot-v1 (July 11, 2022), paraphrase-MiniLM-L3-v2 (July 8, 2022), Paraphrase-albert-small-v2 (July 8, 2022), and Msmarco-distilbert-base-tas-b (August 18, 2022). Additionally, as in our calculations of WEAT scores, we test encoders GPT2 and OpenAI 0.26.4. See the Supplementary material S1 for more information.

RESULTS

The results of our WEAT tests related to potential biases are depicted in Table 2a, where Columns X and Y provide the sets of words, which define the bias tested. For simplicity, we list only the general description of each test in the table. A visual representation of this which better illustrates patterns within each encoder can be found in Figure 5. More specifically, each column in Figure 5 corresponds to one test (e.g., “annoying–pleasant”) and shows how that test was ranked by various encoders, with 1 corresponding to the lowest WEAT score and 14 corresponding to the highest WEAT score. For example, if a column were to consist of all 1 and 2 s, that would mean that the term was consistently ranked as the having the lowest or second-lowest WEAT score (relative to the other tests considered) for every encoder tested. As such, the distribution in each column provides an indication of consistency across encoders (Table 2b).

TABLE 2A.

WEAT scores testing stereotypes against neurodivergent individuals.

	Conceptnet 16.09	Conceptnet 17.06	Text8	Glove- twitter-50	Glove-wiki- gigaword-300	Google- news-300	BERT- base-nli	GPT-2	GPT-3	MiniLM	MPNet
Idiot–Smart	1.277	0.595	1.358*	1.032	1.632	1.563	1.921	1.710	1.650	1.516	1.143
Pity–Joy	1.214	0.490	1.830	0.685	1.290	0.926	1.825	1.110	1.883	−0.374	0.415
Unattractive–Beautiful	1.544	−0.028	1.409	0.616	1.564	1.074	1.713	1.425	1.649	1.144	1.523
Childish–Mature	1.513	1.572	0.789	−0.730	1.326	1.087	1.184	0.756	1.271	0.594	0.753
Dangerous–Safe	1.292	1.073	1.733	0.333	1.650	1.224	1.933	0.223	1.755	1.306	1.564
Diseased–Healthy	1.555	1.235	0.126	0.671	1.334	1.139	1.890	1.061	1.820	0.750	1.358
Annoying–Pleasant	1.702	0.836	0.826*	0.333	1.357	1.318	1.958	1.685	1.784	0.742	1.024
Weak–Competent	1.794	0.524	0.918	0.603	0.758	0.548	1.803	−0.913	1.106	−0.415	1.000
Weird–Cool	0.967	0.435	1.199	0.979	1.430	0.783	1.842	1.356	1.844	−0.291	1.426
Emotionless–Warm	−0.672	−0.850	−0.765	0.514	−0.288	−0.760	1.413	−0.223	0.858	−0.094	−0.985
Uptight–Easygoing	1.313	0.843	1.578	1.075	1.736	1.240	1.551	1.624	1.757	0.864	1.721
Good–Bad	1.132	0.176	1.309	−0.234	1.126	1.132	1.798	0.037	1.731	0.120	1.514
Worthy–Worthless	−0.546	−1.362	−0.028	−1.319	−0.677	0.938	1.455	0.136	0.369	0.500	−0.803
Kind–Unkind	0.101	−0.590	0.346	0.300	0.189	−0.785	1.939	−0.443	1.547	0.144	−0.079
Poor–Rich	1.675	1.431	0.920	1.085	1.427	1.4841	1.873	0.971	1.707	−0.259	0.981
Gay–Straight	1.521	1.782	1.413*	0.892*	0.917	1.569*	1.566	−0.258	1.898	0.864	1.384

Note:

^*Indicates that words in the corresponding X and Y set with an *were removed due to vocabulary limitations of the relevant word embeddings (see Supplementary material S1 for details). WEAT scores with a magnitude greater than or equal to 1.5, denoting the highest levels of estimated bias, are emphasized in blue.

Abbreviation: WEAT, Word Embedding Association Test.

FIGURE 5.

(Left) Heatmap of Word Embedding Association Test (WEAT) Scores for each encoder and test considered. (Right) Depiction of the sorted distribution for each test’s order over all encoders. Given that there are 14 tests, each test has an order from 1 to 14 from each encoder with 1 corresponding to the lowest WEAT score and 14 corresponding to the highest WEAT score. We collect each term’s order over the 11 encoders tested and display the sorted orders in each column to show the distribution of how the term is ordered relative to other terms tested. Full details of the test scores are provided in Table 2a.

TABLE 2B.

WEAT scores testing stereotypes against neurodivergent individuals (mental health terms excluded).

	Conceptnet 16.09	Conceptnet 17.06	text8	Glove- twitter-50	Glove-wiki- gigaword-300	Google- news-300	BERT- base-nli	GPT-2	GPT-3	MiniLM	MPNet
Idiot–Smart	1.280	0.305	1.528	0.869	1.634	1.659	1.932	1.723	1.615	1.377	0.951
Pity–Joy	0.997	0.101	1.822	0.447	1.188	0.646	1.829	1.006	1.923	−0.346	0.187
Unattractive–Beautiful	1.554	−1.181	1.502	0.318	1.547	1.015	1.775	1.687	1.757	1.133	1.344
Childish–Mature	1.534	1.622	0.596	−0.810	1.399	0.987	1.206	0.861	1.358	0.567	1.045
Dangerous–Safe	1.027	0.941	1.758	0.191	1.503	1.079	1.922	−0.001	1.756	1.069	1.498
Diseased–Healthy	1.527	1.081	−0.145	0.694	1.279	1.162	1.897	0.967	1.818	0.521	1.386
Annoying–Pleasant	1.587	0.447	0.255	0.169	1.344	1.120	1.965	1.497	1.824	0.454	1.004
Weak–Competent	1.497	0.528	0.798	0.545	0.748	0.774	1.377	−1.007	1.091	−0.618	0.996
Weird–Cool	0.592	−0.057	1.332	0.942	1.293	0.694	1.850	1.407	1.817	−0.581	1.026
Emotionless–Warm	−0.852	−1.062	−0.751	0.448	−0.396	−0.910	1.491	−0.289	0.922	−0.199	−0.413
Uptight–Easygoing	1.079	0.338	1.681	0.987	1.767	1.034	1.591	1.459	1.726	1.096	1.532
Good–Bad	1.096	−1.587	1.242	−0.683	1.138	0.978	1.984	0.073	1.772	1.491	1.279
Worthy–Worthless	−0.234	−0.335	0.001	−1.337	−0.564	1.106	1.567	0.418	0.454	0.001	−0.829
Kind–Unkind	0.391	−0.335	0.093	0.024	0.219	−0.734	1.955	−0.176	1.738	0.279	0.066
Poor–Rich	1.665	1.421	1.063	0.993	1.496	1.508	1.904	−0.386	1.664	0.264	0.776
Gay–Straight	1.422	1.742	1.439	0.806	0.955	1.603	1.471	1.097	1.876	0.820	1.412

Abbreviation: WEAT, Word Embedding Association Test.

We depict the average WEAT score for these biases in Figure 6, where the average is taken over all tested embedders. Notably, one potential bias without a large WEAT score is the stereotype that neurodivergent individuals are emotionless or robotic as opposed to warm and caring. Likewise, there was not a large WEAT score for the test “worthy–worthless” or “kind-unkind.” The highest WEAT scores associate neurodiversity with slurs, violence, and being uptight. For most tests, there is a large error bar, indicating that the amount of bias varies substantially between different encoders. We additionally include Table 2c with information related to the significance of the WEAT scores for each test and word embedder. These tests indicate that the average standard deviation in WEAT scores for randomly selected word sets was 0.873, motivating our choice to highlight WEAT scores of 1.5 or greater as notable in our bias tests.

FIGURE 6.

Word Embedding Association Test (WEAT) scores for different biases tested. Each quantile represents one quarter (25%) of the WEAT score distribution, such that Quantile 1 represents the lowest 25% of WEAT scores for a given test, Quantile 2 represents the second lowest 25% of scores, Quantile 3 represents the second highest 25% of scores, and Quantile 4 represents the highest 50% of scores.

TABLE 2C.

Standard deviation in WEAT score statistics for randomly selected terms.

	Conceptnet 16.09	Conceptnet 17.06	text8	Glove-twitter-50	Glove-wiki-gigaword-300	Google-news-300
Idiot–Smart	1.02	0.83	0.61	0.95	1.23	1.04
Pity–Joy	0.93	0.89	0.91	0.88	1.07	0.91
Unattractive–Beautiful	0.96	0.82	0.71	0.75	0.92	0.90
Childish–Mature	0.76	0.62	0.89	0.92	0.76	0.66
Dangerous–Safe	0.93	0.94	0.82	0.88	0.95	0.84
Diseased–Healthy	0.94	0.83	0.71	0.76	1.12	0.91
Annoying–Pleasant	1.10	0.92	0.67	1.10	0.98	0.81
Weak–Competent	0.72	0.85	0.66	0.79	0.77	0.77
Weird–Cool	0.82	0.71	0.91	0.81	0.96	0.84
Emotionless–Warm	0.67	0.67	0.70	0.80	0.76	0.90
Uptight–Easygoing	0.78	0.90	0.98	0.80	1.26	0.88
Good–Bad	1.20	1.10	0.87	1.11	1.07	1.31
Worthy–Worthless	0.85	0.94	0.82	0.89	0.87	0.92
Kind–Unkind	0.98	0.83	0.37	0.64	0.67	0.94
Poor–Rich	0.80	0.79	0.67	0.77	0.87	0.90
Gay–Straight	0.94	1.06	0.86	0.88	0.92	1.09

Note: We randomly generated 20 pairs of word sets and for each test (e.g. “Good–Bad,” “Worthy–Worthless”) and then calculated the WEAT score with the random word sets for each pair. We then computed the standard deviation in WEAT scores over the 20 random word sets for each test and for each word embedder.

Abbreviation: WEAT, Word Embedding Association Test.

We additionally investigate whether word embeddings also capture commonly listed strengths of neurodivergent individuals. One common strength of neurodivergent individuals is creativity or a unique perspective, with research finding entrepreneurial creativity in individuals with ADHD (Moore et al., 2021), high levels of detail and originality in autistic individuals (Pennisi et al., 2020), and correlation between mild psychotic symptoms and elevated creativity (Fink et al., 2014). A second common strength is a generally accepting or nonjudgmental attitude, with autistic individuals displaying attenuated implicit social biases (Birmingham et al., 2015). For these two tests, we use the same word sets A and B as provided in Table 2a.

We also test two potential strengths, honesty (Bagnall et al., 2021) and loyalty (Russell et al., 2019), that are often mentioned for autistic individuals and as such we use an autism-focused word set of {“autism,” “autistic,” “asperger”} versus {“typical,” “regular,” “average”} for these two tests. Results are depicted in Table 3.

TABLE 3.

WEAT scores testing neurodivergent strengths.

	Conceptnet 16.09	Conceptnet 17.06	text8	Glove-twitter-50	Glove-wiki-gigaword-300	Google-news-300	BERT-base-nli	GPT-2	GPT-3	MiniLM	MPNet	Average
Creative-Unoriginal	1.667	1.732	—	−0.311	−0.659	1.253	−1.530	−0.519	1.815	0.781	1.324	0.555
Accepting-Judgmental	−1.250	−1.598*	—	−0.790*	−1.615 *	−0.881	−1.928	0.264	−1.856	−0.244	−0.565	−1.046
Reliable-Fickle (autism-specific test)	−1.384	−1.248	−1.079	−0.667	−1.412	−0.529	−1.823	−1.281	−0.984	−1.572	−1.329	−1.210
Honest-Dishonest (autism-specific test)	−1.390	−0.890	−0.197	−0.225	−1.434	−1.513	−1.783	−1.179	−1.368	−1.606	−1.290	−1.170

Note:

^*Indicates that words in the corresponding X and Y set with an * were removed due to vocabulary limitations of the relevant word embeddings. WEAT scores with a magnitude ≥1.5, denoting the highest levels of significance, are emphasized in bold.

Abbreviation: WEAT, Word Embedding Association Test.

For these results, depicted in Table 3, high WEAT scores correspond to a “positive bias,” namely, they indicate that the word embeddings were able to successfully connect neurodiversity with common strengths. Many of the word or sentence embedders tested did pick up on the strength of creativity, with an average WEAT score of 0.555 for the test Creative–Unoriginal. However, average scores for the remaining tests were notably negative, indicating that neurodivergent and autistic individuals may face AI stigma even in these areas of expected strengths. For example, although autistic individuals can engage in deception, they are generally less likely to engage in deception than nonautistic individuals (Bagnall et al., 2021). However, the average WEAT score for the test Honest–Dishonest is −1.170 indicating that most encoders perceive autistic individuals as more dishonest or deceptive. Finally, although WEAT scores for word sets consisting of a single word are less reliable, we include in the Supplementary material S1 estimates of bias for individual terms.

Until now, we have restricted our focus to finding similarities between pairs of individual words. Here, we restrict our focus to sentence encoders, which allows us to test an extended set of concepts by generating the similarity between different sentences. We first examine how perceived goodness varies based on the specific words used to describe autistic individuals. Recently, there has been much discussion about avoiding ableist language in autism research (Bottema-Beutel et al., 2021), with one particularly common topic being the use of identity-first language (“autistic person”) versus person-first language (“person with autism”). We aim to test sentiment regarding the sentence set {“I am neurodivergent,” “I am an autistic person,” “I am a person with autism,” “I have Asperger’s”}. Additionally, given that autism is a broad umbrella term describing many different experiences, we investigate the perceived goodness of terms related to the support level of the autistic individuals. To this aim, we collect a set of commonly described terms used to describe support needs: {“I have mild autism,” “I have moderate autism,” “I have severe autism,” “I have profound autism,” “I have Level 1 autism,” “I have Level 2 autism,” “I have Level 3 autism”}.

We begin our tests with the set $X$ = {“I am good,” “I am a good person,” “I make the world a better place”} and $Y$ = {“I am bad,” “I am a bad person,” “I do not make the world a better place”} and label the corresponding SERT score of a sentence as its “perceived goodness.” The results are shown in Figure 7, with the perceived goodness of additional identities such as “friendly person” and “bank robber” added in black as a comparison class.

FIGURE 7.

Average Sentence Embedding Ratio Test score for the “perceived goodness” of various terms used to describe autism and neurodiversity with error bars representing the standard deviation. Black bars correspond to comparison words, green bars correspond to words frequently discussed in relation to neurodiversity-affirming language, and blue and purple correspond to words frequently related to support needs. The horizontal black line corresponds to the perceived goodness of our baseline comparison sentence “I am a person.”

We find that all terms related to autism or neurodiversity are perceived more negatively than the neutral comparison sentence “I am a person.” The term “autistic person” has a slightly higher SERT score than the term “person with autism,” However, the differences are relatively small and do not necessarily indicate that changing the term used to refer to an autistic person will impact the degree to which the person is perceived as “good” or “bad” in practice—the SERT scores may indicate a correlation between someone’s choice of terminology and whether they use positive or negative terms in their discussion of autism, but do not indicate the direction of causality. The term “neurodivergent person” has a higher score than either “person with autism” or “autistic person,” and the most positively perceived term is “person with Asperger’s.”

Particularly high levels of bias are shown towards individuals with higher support needs, with the sentence “I have severe autism” being perceived three times more negatively than the sentence “I am a nonautistic person.” In most encoders, terms such as “Level 1 autism” or “Level 3 autism” appeared to not be recognized, as they generated results inconsistent with tests for synonyms such as “severe autism” or “profound autism.” Likewise, any use of the term “autistic,” including through the term “nonautistic,” lowered the perceived goodness of the sentence.

We now investigate the order of perceived goodness for a variety of different conditions through the sentence set {“I have schizophrenia,” “I am blind,” “I am deaf,” “I have dementia,” “I am autistic,” “I have psychosis,” “I have schizoaffective disorder,” “I have epilepsy,” “I have Tourette’s,” “I have dyslexia,” “I have an intellectual disability,” “I have cerebral palsy,” “I have ADHD,” “I am a wheelchair user”}. We continue to keep some control terms through the set {“I am a bank robber,” “I am a medical doctor,” “I am a person,” “I am a friendly person”}. Results, averaged over all sentence encoders, are depicted in Figure 8 along with a simultaneous plot of the inverted ratio (“perceived badness”). All conditions tested are perceived more negatively than the neutral-to-positive control terms with epilepsy having the highest perceived badness and wheelchair users having the highest perceived goodness. To illustrate the degree of consistency in ordering or ranking across various encoders, Figure 9 shows the sorted distribution of how each term is ordered over all encoders. The figure shows some variability among different encoders, but also shows that the positive control terms consistently have the highest scores in perceived goodness across almost all encoders.

FIGURE 8.

Perceived badness (left) and perceived goodness (right) of several conditions and control terms.

FIGURE 9.

Depiction of the sorted distribution for each term’s order over all encoders. Given that there are 20 terms, each term has an order from 1 to 20 from each encoder with 1 corresponding to the lowest perceived goodness and 20 corresponding to the highest perceived goodness. We collect each term’s order over the 11 encoders tested and display the sorted orders in each column to show the distribution of how the term is ordered relative to other terms tested.

DISCUSSION

Our results demonstrate an overall high level of bias towards or negative associations with terms related to autism and neurodiversity. While overall levels of bias vary based on the encoder considered, particularly high levels of average bias were found for tests related to slurs, violence, or obsessiveness. Likewise, sentence embedder tests indicated particularly negative associations for terms associated with autistic individuals who have higher support needs.

The selected tests in this manuscript were based on prior research related to neurodivergence and bias, as well as the authors’ experiences. The first and fourth authors of this article are autistic autism researchers. Additional research into autistic strengths and ways of connection, as well as research developed in collaboration with autistic individuals may help to mitigate existing biases in society. From a technical perspective, the same fairness methods that address biases and stereotypes related to ethnicity or gender can be applied to neurodiversity. In particular, there are existing debiasing procedures that can remove specific biases (Bolukbasi et al., 2016; Kaneko & Bollegala, 2019). We recommend that such procedures be implemented on the exact set of biases identified in this article. Our tests of “poor–rich” and “gay–straight” indicate that AI algorithms have identified correlations between neurodivergent individuals and LGBTQ+ identity as well as limited financial resources, the latter potentially being due to barriers neurodivergent people face in employment and education. Future work investigating AI bias against LGBTQ+ neurodivergent individuals, neurodivergent individuals of varying socioeconomic status, and other multiply marginalized neurodivergent individuals would enhance debiasing efforts. Likewise, the correlation in word embedders between neurodivergence and LGBTQ+ identity and, respectively, neurodivergence and socioeconomic identity indicates that implementing debiasing techniques in all three areas may be necessary to create fully fair AI algorithms for neurodivergent individuals.

Our analyses have several limitations. Our results depend on the specific encoders and word sets chosen, and provide an indicator of bias in recent years. The presence of bias may vary with time as new encoders or more debiasing techniques are introduced or as word associations in the content produced by humans change. Likewise, our results are limited to English word embedders and tests and leave as an open question how bias may generalize in other languages. Critically, the presence of a high WEAT score does not automatically translate to high levels of social bias, nor does the presence of a low WEAT score necessarily indicate its absence. Efforts by a group of people to advocate for a disability or neurotype in positive terms could lead to related words having positive associations in many texts, even if the advocacy efforts themselves are in response to a broader negative societal stigma. The relative absence of resources or advocacy for less-recognized or discussed forms of neurodivergence such as “dyslexia” may potentially explain why the term has a slightly lower average “perceived goodness score” than highly stigmatized conditions such as “dementia” which often have dedicated groups, fundraisers, and supportive resources that may mitigate negative text correlations despite high overall levels of bias.

For example, texts discussing or attempting to dispel the stereotype that autistic individuals do not experience empathy may create a link between the terms “autism” and “empathy,” thereby lowering the corresponding score. Likewise, the strong link between the term “OCD” and terms related to LGBTQ+ identity may be a reflection of the occurrence of scrupulosity in OCD rather than an indication that individuals with OCD are disproportionately assumed to be LGBTQ+. Finally, given the abundance of discussion and debate regarding the language used to describe autistic individuals, results in Figure 7 regarding the perceived goodness of various autism-related terms may be impacted by the language used in debating various terms rather than solely due to the societal impression of terms themselves.

These results highlight the need for more research on the role of AI in contributing to existing stigma and bias against autistic individuals. Under-representation of neurodivergent individuals in research decisions, policies, and decisions related to media portrayal of neurodivergent individuals could contribute to such biases. Increased leadership by neurodivergent individuals with diverse backgrounds and identities may be generally beneficial for decreasing neurodivergence-related stigma and more generally could increase the amount of autism-related training data that is written by neurodivergent individuals, potentially resulting in reduced levels of bias. Focus groups of neurodivergent individuals with diverse identities and backgrounds may also be beneficial for identifying terms that are neurodiversity-affirming, which could ultimately increase the use of such terms in association with neurodivergence.

More broadly, the presence of bias against terms related to neurodivergence indicates the need to examine potential bias in other AI-based processes. For example, we recommend testing AI-based resume scanners to see whether references to neurodivergence or disability disproportionately penalize an application. Similarly, one could investigate whether content distribution algorithms are less likely to distribute content related to neurodivergence or automatically group content related to neurodivergence with difficult or negative topics. Fairness tests can be performed for any algorithm or automated process, which may inadvertently discriminate against or be less effective for neurodivergent individuals, such as workplace personality tests, predictive AI algorithms in healthcare settings, and responses related to neurodivergence generated by AI chatbots.

Direct research on societal perceptions of and attitudes towards neurodivergent individuals may provide a starting point for identifying and determining ways to mitigate social bias. Ideally, this work will lead to broader discussions regarding what types of relationships and biases need to be mitigated in AI models. More broadly, future work should increase the scope of biases we consider and engage with many different groups of people to address these biases, including neurodivergent groups that were not included in the present analysis. By bridging these gaps between current AI biases and the perspectives of the neurodivergent community, we can help ensure we use AI for good and create a path that reflects our future aspirations and not our past.

Supplementary Material

Additional supporting information can be found online in the Supporting Information section at the end of this article.

DATA AVAILABILITY STATEMENT

Our code will be placed in the Duke University Research Data Repository, which generates a DOI and guarantees preservation for 25 years.