Transitive Inference in a Clinical Childhood Sample with a Focus on Autism Spectrum Disorder

Abstract

Transitive inference (TI) has a long history in the study of human development. There have, however, few pediatric studies that report clinical diagnoses have tested trial-and-error transitive inference learning, in which participants infer item relations, rather than evaluate them explicitly from verbal descriptions. Children aged 8 to 10 underwent a battery of clinical assessments and received a range of diagnoses, potentially including autism spectrum disorder (ASD), attention-deficit hyperactive disorder (ADHD), anxiety disorders (AD), specific learning disorders (SLD), and/or communication disorders (CD). Participants also performed a trial-and-error learning task that tested for transitive inference. Response accuracy and reaction time were assessed using a statistical model that controlled for diagnostic comorbidity at the group level. Participants in all diagnostic categories showed evidence of transitive inference. However, a model comparison analysis suggested that those diagnosed with ASD succeeded in a qualitatively different way, responding more slowly to each choice and improving faster across trials than their non-ASD counterparts. Additionally, transitive inference performance was not associated with IQ. Overall, our data suggest that superficially similar performance levels between ASD and non-ASD participants may have resulted from a difference in the speed-accuracy tradeoff made by each group. Our work provides a preliminary profile of the impact of various clinical diagnoses on TI performance in young children. Of these, an ASD diagnosis resulted in the largest difference in task strategy.

Lay Summary:

Children in a clinical sample with various diagnoses successfully learned how to order a set of pictures. However, some diagnostic categories appeared to be associated with different strategies for success than others.

Autism spectrum disorder (ASD) has been associated with impairments to attention, sensory perception, and memory (Chowdhury et al., 2017; Cooper et al., 2017). However, many of those diagnosed with ASD possess psychological faculties that remain intact. A challenge in characterizing ASD is that these intact faculties may nonetheless give rise to atypical schemas, because of the interaction of various cognitive processes. For example, most children diagnosed with ASD classify objects into learned perceptual categories, but these categories may differ substantially from those learned by most other children (Mercado et al., 2020). Thus, when children with ASD who were asked to create categories for abstract geometric shapes, they were often able to do so, but they relied on different shape properties than those without ASD (Mercado et al., 2015, Church et al., 2015). The underpinnings of learning for children with ASD, with respect to these perceptual differences, are still in contention. Therefore, experimental tasks of learning and memory should not focus solely on performance deficits, but rather should also help to identify cognitive processes and strategies that are more specific to those diagnosed with ASD.

Currently, the accepted symptomatology of ASD in clinical settings focuses on (1) social and communicative difficulties and (2) patterns of perseverative or restricted behavior or interest, neither of which necessarily reflect distinguishing characteristics in other cognitive domains, such as spatial or analogical reasoning. Nevertheless, a wide range of research studies have compared samples of ASD participants to neurotypical controls on cognitive tasks other than those that act as diagnostic criteria (Velikonja et al., 2019, provide a systematic review). When subjected to meta-analysis, ASD samples display a deficit, on average, in several non-social, non-verbal domains when compared to controls. As a spectrum disorder, however, autism is very poorly characterized by averages, and the aggregation of studies into meta-analyses further collapses much of the context needed to interpret those differences. Given the wide variety of characteristics exhibited by those on the autism spectrum, it is important to preserve this context and avoid making sweeping generalizations. This deeper understanding of the underlying cognitive processes may both facilitate diagnosis and help to personalize treatment and therapy options.

A further complication in trying to interpret large-scale ASD results is the high rate of diagnostic comorbidity. Khachadourian et al. (2023) reviewed the medical records of 40,582 individuals with ASD, as compared to 11,389 of their non-ASD siblings as a control group. ASD was associated with at least one comorbid diagnosis over 70% of the time, more than twice that for controls; and more than half of ASD records indicated two or more comorbidities. This makes evaluating reported patterns of cognitive deficits associated with specifically ASD much more difficult to interpret. Furthermore, symptoms of other diagnoses may present differently when comorbid with ASD (Belardinelli et al., 2016). Given these demographics, the study of cognitive processing in the ASD population must take comorbidities into consideration, since an isolated ASD diagnosis is the exception rather than the rule.

Transitive Inference

One topic that has not been widely studied in ASD populations is the ability to perform transitive inference (TI) (Jensen et al., 2017; Kao et al., 2020). Provided someone has already learned that A > B and B > C, TI is a cognitive ability that lets that person infer that A > C. Since this transitive property applies to any ordered set, TI is a crucial means for generalizing from past experience to new comparisons. A participant’s ability to perform TI tasks is used to measure relational memory in both clinical contexts (e.g., Onwuamezeet al., 2016) and in cognitive neuroscience (e.g., Wing et al., 2021). While this ability is basic enough that can be studied in animal models, and in humans with minimal verbal instruction, it cannot be explained solely in terms of associative learning (Jensen et al., 2019) and appears instead to reflect an underlying implicit cognitive process. The range of species that have demonstrated at least some capacity for TI includes monkeys (McGonigle and Chalmers, 1977; Jensen et al., 2017), fish (Hotta et al., 2020), and even wasps (Tibbetts et al., 2019). In many species, TI is thought to play a role in evaluating social dominance hierarchies (Gazes et al. 2017; Grosenick et al., 2007; Bond et al., 2010). As such, it is unsurprising that typically developing children are capable of successful transitive inferential learning, whether the task of transitivity consists of verbal word problems (Wright and Smailes, 2015), or of non-verbal trial-and-error learning (Holcomb et a., 1997; Bryant and Trabasso, 1971).

Contrastingly, very few studies have assessed TI in children with ASD. Those that have are generally based on small sample sizes (e.g. Gorham et al., 2009). When widening the scope to recruitment of adults with ASD, differences in performance are generally small and/or non-significant (Solomon et al., 2011; 2015), and again, these effects are not evaluated with comorbidities in mind. Nevertheless, reviews report cognitive deficits in “relational” and “analogical” reasoning associated with ASD (e.g., Banker et al., 2021).

Despite this paucity of clear evidence, there are theoretical reasons to think TI may reveal something interesting about symptoms associated with ASD. The trial-and-error task of TI given to animal models is by design, entirely non-verbal, so it is reasonable to hypothesize that individuals with ASD should be able to succeed at such tasks even if their symptoms include considerable difficulties with verbal communication. On the other hand, the comparative literature has argued that TI is an important for evaluating social group dynamics. Since TI is not currently considered diagnostic criterion for ASD, further understanding of this form of learning could help tease apart the cognitive processes that contribute to specific symptoms of ASD.

To do so, however, poses a considerable statistical challenge. Often citing statistical concerns, studies of TI in other clinical populations routinely exclude participants with other diagnoses. For example, Brunamonti et al. (2017) reported that children with an ADHD diagnosis displayed poorer TI performance, but did so having excluded those with “evidence of neurological disorders, pervasive developmental disorders, and receptive language disorders” (p. 201). Such exclusion criteria necessarily exclude children with ASD, and given the high rates of comorbidity between ASD and ADHD, this means by extension that they exclude a substantial minority of all children with ADHD.

In light of the above considerations, we felt that that overly stringent exclusion criteria risked excluding the majority of individuals for whom ASD is one of multiple diagnoses. Accordingly, we administered a TI task to a cohort of children with various clinical diagnoses, rather than those only diagnosed with ASD. First, we wanted to evaluate whether children with various clinical diagnoses, such as Autism Spectrum Disorder (ASD), Attention Deficit/Hyperactivity Disorder (ADHD), Anxiety Disorder (AD), Specific Learning Disorder (SLD), and Communication Disorder (CD) were able to successfully perform experimental TI. Second, we wanted to determine whether the children with ASD perform the task in a manner that is distinct from those without that diagnosis. Our data suggest children with ASD approached the tradeoff between reaction time and response accuracy using a different strategy than other children within this sample.

Methods

Participants

Participants were a clinical sample of 40 children (15F, 25M, aged 8 to 10 years) who had already been enrolled with the Child Mind Institute (CMI, New York, NY), an independent nonprofit that aims to match children with resources and support. The first step for a child working with CMI is a comprehensive clinical assessment, evaluating children over a series of visits (some in person, some remote), which included psychiatric inventories, cognitive tests, and examination by clinicians. The guardians of potential participants who sought assessment from CMI were invited to enroll them into the study in exchange for a $25 Amazon gift card, provided their IQ was at least 70. Once enrolled, the experimental session was added to their schedule of tests to be performed throughout the assessment period. Additionally, children who had previously completed CMI assessments in 2018/2019 were recruited to via their guardians to join the study, provided they fulfilled our inclusion/exclusion criteria. Guardians were reassured that participation would not impact other diagnostic determinations. Because CMI’s clinical assessment process was staggered over a period of weeks to avoid overloading the child, most participants performed the task before receiving diagnoses from CMI, with diagnostic information provided once the full assessment was complete.

Within our sample, 21 were diagnosed with autism spectrum disorder, 16 were diagnosed with ADHD, 12 were diagnosed with an anxiety disorder, 18 were diagnosed with a specific learning disorder, and 13 were diagnosed with a communication disorder. Only one participant satisfied none of these diagnostic categories, and only 15 satisfied a single diagnostic category. All others received multiple diagnoses. Table 1 provides an overview of all 40 participants with respect to their comorbidities. More details about these categories are provided in the Clinical Assessments section below.

Table 1:

Overview of Participant Diagnostic Categories

Participants
Diagnosis	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39	40
ASD																				X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
ADHD																X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
AD													X	X	X			X	X	X		X	X	X	X							X	X
SLD						X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
CD		X	X	X	X					X	X	X			X		X					X				X	X	X

ASD: autism spectrum disorder; ADHD: attention deficit/hyperactivity disorder; AD: anxiety disorder;

SLD: specific learning disorder; CD: communication disorder

The experiment was approved by the Institutional Review Boards of Columbia University (protocol AAAR7039) and New York City College of Technology CUNY (protocol 2019–0601-NYCCT), conforming to guidelines for human research set forth by the American Psychological Association.

Procedure

Transitive Inference Task

Setup:

Our task was hosted using the Gorilla Experiment Builder for online experiments (Anwyl-Irvine et al., 2020), and implemented remotely by staff of the Child Mind Institute (CMI), NY, NY. The task was presented in a web browser, and participants used a computer mouse to click on their responses under guardian supervision, while a staff member was present via video conferencing to provide instructions, encouragement, and to troubleshoot difficulties. All parties were naïve to the underlying objective of the task.

Prior to the task, experimenters defined an ordered list of 7 pictures of cartoon characters. These were rank-ordered in advance by designating each image as A, B, C, D, E, F, or G, with image A being the “highest ranked” item, and G being the “lowest ranked” item. Figure 1A presents an example pair of images from the ordered list. Item ranks varied from one participant to the next. Neither the item labels nor their ordering was shared with participants or study staff.

Figure 1.

Transitive inference task. (A) Example of a 7-item list, using visually distinct cartoon characters. Stimuli were only ever presented in pairs, and the correct item in every pair was determined by a fixed hierarchy that was determined in advance. (B) Instructions given to participants. Beyond encouragement to continue trying, neither participants nor their guardians were given any additional information about the task until debriefing. (C) Sequence of events in a sample trial. Each trial began with a blue start stimulus. Clicking on it with the mouse caused the start stimulus to disappear and two list stimuli to appear. Clicking on the item of superior rank yielded positive feedback, whereas clicking in the other stimuli yielded negative feedback. The mouse cursor depicted in these example screens is of a scale appropriate to what a participant would see while performing the task.

Task Structure:

Participants were presented with pairs of images and instructed to click on whichever they thought was the “correct” image. Figure 1B displays the instructions given to participants. Figure 1C depicts an example pair of images from the ordered list, corresponding to the pair CE. In this pairing choosing stimulus C would be the correct response (and E would be incorrect). However, if a participant was instead shown pair BC, choosing stimulus C would instead have been the incorrect response (and B would be correct). For each trial, participants were required to choose the item they thought was the correct response.

Figure 1C outlines the trial sequence. Each trial began with a blue square at the center of the screen, which the participant needed to click with the mouse to proceed. As soon as they did, the blue square disappeared and was replaced with two images on either side of the cursor. When either image was chosen, feedback appeared in the center of the screen; a green check mark signaled a correct response, and a red X signaled an incorrect response. One second after the presentation of the feedback, the screen went blank and another trial began.

Training Phase:

Participants first completed a training phase of 180 trials, divided into 15 blocks. During training trials, participants saw only the pairs of stimuli with adjacent ranks in the ordered list: AB, BC, CD, DE, EF, and FG. Each of these pairs were presented twice during a block, once with the correct item on the left side of the screen, and once with the correct item on the right. The trial order of these 12 pairs was permuted randomly within each block. This ensured that these pairs were trained uniformly during the training phase.

Testing Phase:

After training and without any signal of a phase change, participants completed an 84-trial testing phase, consisting of two testing blocks. Each testing block presented all 21 possible pairings from the 7-item ordered list (e.g., AB, AC, AD, … EF, EG, FG), doing so twice to counterbalance stimulus position. The orders of pair presentations during each testing block were randomly permuted, with one constraint: The first 12 trials of the testing phase presented “critical test pairs,” which excluded both the training sets of adjacent pairs, and any pairs that included items A or G. The remaining pairs, BD, BE, BF, CE, CF, and DF, are considered the critical tests of whether the participant’s choices showed evidence of transitive inference.

Over the course of the 264 trials of the entire task (including training and testing phases), participants received three on-screen “encouragement signals” that served to signal their overall progress; “You’ve made it one quarter of the way” after trial 66, “You’ve made it half way” after trial 132, and “You’ve made it three quarters of the way” after trial 198.

Clinical Assessment

As noted above, participants could potentially belong to any of five diagnostic categories: Autism Spectrum Disorder (ASD), Attention Deficit/Hyperactivity Disorder (ADHD), Anxiety Disorder (AD), Specific Learning Disorder (SLD), and Communication Disorder (CD). With the exception of ASD, these are umbrella classifications that cover multiple specific diagnoses. ADHD covered all categories of symptom presentation (e.g., inattentive, hyperactive, etc.). AD potentially included any diagnosis of Separation Anxiety Disorder, Generalized Anxiety Disorder, Social Anxiety Disorder, Selective Mutism, Panic Disorder, Agoraphobia, and/or any other specific phobia or DSM-specified anxiety disorder. SLD potentially included measurable impairments in mathematics, reading, or written expression. CD potentially included Language Disorder, Social Communication Disorder, and/or Speech-Sound Disorder. Participants were grouped according to these categories partly for statistical power and partly to reduce participant identifiability. Participants’ guardians also completed an initial clinical intake interview, which informed all diagnostic categories. Subsequently, they completed a series of other assessments over the course of multiple visits, including the Wechsler Intelligence Scale for Children (WISC-V; Wechsler, 2014).

Mental health diagnoses were determined by a consensus process, primarily by interviews by clinicians and secondarily by a battery of tests. ASD, ADHD, and AD diagnoses were informed by the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS-PL; Kaufman et al., 1997), conducted with both participants and their guardians, as well as the Child Behavior Checklist (CBCL; Achenback, 1991), conducted with both guardians and teachers. ASD diagnoses were additionally informed by guardian reports on the Autism Spectrum Screening Questionnaire (ASSQ; Ehlers et al., 1999), the Social Communication Questionnaire (SCQ; Chandler et al., 2007), the Social Responsiveness Scale (SRS-2; Constantino et al., 2003), and the Gillam Autism Rating Scale (GARS-3; Gillam, 2014). ADHD diagnoses were additionally informed by guardians completing the both the Strengths and Weaknesses Assessment of Normal Behavior (SWAN; Swanson et al., 2001) and its “extended” version (E-SWAN; Alexander et al., 2020), as well as participant completion of the Conners ADHD Rating Scale (C3SR; Conners, 2001) and measures taken using the NIH Toolbox (Gershon et al., 2013). AD diagnoses were additionally informed by participants and guardians both completing the Screen for Child Anxiety Related Emotional Disorders (SCARED; Birmaher et al., 1999).

In the case of SLD and CD, standardized tests were the primary determinants (in consideration with WISC-V norms), with clinical interviews providing secondary evidence. SLD evaluation considered performance on the Wechsler Individual Achievement Test (WIAT-III; Wechsler, 2009), the Comprehensive Test of Phonological Processing (CTOPP-2; Wagner et al., 2013), and the Test of Word Reading Efficiency (TOWRE-2; Torgesen et al., 2012). CD evaluation considered the WIAT-III, the CTOPP-2, the Clinical Evaluation of Language Fundamentals (CELF; Wiig et al., 2013), the Expressive Vocabulary Test (EVT-2; Williams, 2007), the Peabody Picture Vocabulary Test (PPVT-4; Dunn & Dunn, 2007), and the Goldman-Fristoe Test of Articulation (GFTA-3; Goldman & Fristoe, 2015). Participants who were at least 9 years old also completed the CELF extension for metalinguistics (Wiig & Secord, 2014).

In all cases, diagnosis ultimately depended on a clinical judgment of the full scope of collected assessments, with no single deciding criterion. A diagnosis that was received by fewer than 10 participants and did not fall into the above categories was not included as a variable in this study. As such, each participant may have been given additional diagnoses that are not reported here, both for reasons of statistical power and to reduce participant identifiability.

Analysis

Participant performance at the start of the testing phase was evaluated both in terms of proportion of correct responses and in terms of reaction times. In both cases, hierarchical regression models were used, which permitted both participant-level estimates of performance and a description of the overall patterns observed across groups of participants. Proportion of correct responses modeled using logistic regression, whereas reaction times were modeled using linear regression and the natural log of the reaction times in milliseconds. A detailed account of our analytic strategy is provided in the appendix.

To briefly summarize, each regression model describes participant performance in terms of four participant-level parameters (the intercept ${\beta }_{\varnothing }$, the learning rate ${\beta }_t$, the symbolic distance effect ${\beta }_D$, and a distance/learning interaction ${\beta }_{Dt}$), as well as four population constants associated with clinical diagnoses that were shared among all participants (${\theta }_{\text{ADHD}},{\theta }_{\text{AD}},{\theta }_{\text{SLD}}$, and ${\theta }_{\text{CD}}$). ASD was treated differently from the other diagnostic categories: All ASD participants were assessed using one hierarchical model, while all non-ASD participants were assessed using a second, separate hierarchical model. The only link between these two models were the four diagnostic constants. Partitioning the data into two distinct populations helped to make any differences between ASD and non-ASD performance as clear as possible; while modeling the effects of other diagnostic categories using a single constant is not ideal, the sample size was not sufficiently large to justify a more complex approach. Table 2 provides a summary of each parameter and how to interpret its effects.

Table 2:

Parameter Overview

Parameter	Name
${\beta }_{\varnothing }$	intercept	Describes a participant’s overall performance, averaged across all stimulus pairs, on the first trial of an experimental phase. Larger values correspond to higher accuracy (for log-odds proportion correct) or slower speed (for log reaction time).
${\beta }_t$	learning rate	Describes how overall performance changes as a function of trials. Because all trials provided informative feedback, participants might continue to improve their performance as the testing phase unfolds. Positive values correspond to performance improving (for proportion correct) or slowing down (for log reaction times) as the experiment continues.
${\beta }_D$	symbolic distance effect	Describes how strong the transitive inference effect is at the start of the testing phase. Positive values indicate that participants found items that were far apart in the list to be easier (for log-odds proportion correct) or else responded to them more slowly (for log reaction times). During an analysis of the training phase, this parameter would be omitted.
${\beta }_{Dt}$	learning/distance interaction	Describes whether the strength of the symbolic distance effect changes during the testing phase. A participant with no distance effect at the start of testing could reveal one as testing proceeds. A value of zero, by contrast, reflects the distance effect remaining constant over time. During an analysis of the training phase, this parameter would be omitted.
${\gamma }_x$	population-level parameter	For each of the participant-level parameters above, there was a population parameter that describes the value of an average participant in that group. For example, every participant receives their own learning rate ${\beta }_t$, whereas the overall average participant in a group has a population-level learning rate ${\gamma }_t$. ASD participants and non-ASD participants had separate population-level parameters.
${\theta }_{\text{diagnosis}}$	diagnosis constant	For each diagnostic category other than ASD, a participant added a constant to their intercept ${\beta }_{\varnothing }$. These population-level effects were shared by ASD and non-ASD participants. The value of each ${\theta }_{\text{diagnosis}}$ parameter is best interpreted as how large a difference a particular diagnosis makes to a participant’s intercept.

It is natural to ask whether partitioning according to ASD is the most appropriate way to model the data. After all, some other diagnostic category might be more deserving of this partition. Perhaps, for example, an ADHD diagnosis might better explain some difference in strategy. To validate our partitioning of the data, we performed a model comparison analysis among models that partitioned participants according to each diagnostic category (including a model without any partition at all), as described in the appendix. By an overwhelming margin, the ASD vs. non-ASD split was best supported by the evidence for both measures of performance.

Results

Participants completed the task in an average of 1303.4s (SD = 382.9s), with most participants taking between 1000s and 1550s to finish. For both response accuracy and reaction time, participants steadily improved during the training phase, with an accompanying gradual acceleration in reaction times. No systematic differences between ASD and non-ASD participants were evident during training. These data are plotted in Supplemental Figure 1.

Figure 2 plots the group-level mean parameters (intercepts ${\gamma }_{\varnothing }$, distance effects ${\gamma }_D$, learning rates ${\gamma }_t$, and interaction terms ${\gamma }_{Dt}$) for a logistic regression of response accuracy (top row, in log-odds units) and for a linear regression of log-reaction time (bottom row, in log milliseconds). Broadly, average performance was similar across diagnostic conditions in most cases, as evidenced by the high degree of overlap between estimated population parameter values for ASD vs. non-ASD participants. However, ASD participants differed from non-ASD participants in two respects. First, ASD participants displayed a higher learning rate (${\gamma }_t=0.0085$, implying a change from 61.3% accuracy to 78.7% accuracy for an average ASD participant over the first 100 trials) with respect to response accuracy than did non-ASD participants (${\gamma }_t=-0.0003$, implying a change from 63.8% accuracy to 63.1% accuracy for an average non-ASD participant over the first 100 trials). This difference implies that the response accuracy of ASD participants improved more rapidly during the testing phase, as compared with effectively no improvement among non-ASD participants. Second, ASD participants had longer mean reaction times (${\gamma }_{\varnothing }=7.53$, analogous to 1.86 seconds) than non-ASD participants (${\gamma }_{\varnothing }=6.81$, analogous to 0.91 seconds). Other diagnostic categories did not show effects substantially different from zero for either behavioral measure. Trends also suggest that participants performed at above-chance levels at the start of all-pairs testing with a mild distance effect for response accuracy. A larger sample is needed to determine if an effect size of zero can be ruled out.

Figure 2.

Group-level posterior estimates for parameters in Equations 3 and 5, with ASD (white boxes) and non-ASD (gray boxes) treated as separate populations sharing common covariate effects for other diagnostic categories. Boxes represent the 80% credible interval for the mean, and whiskers represent the 95% credible interval. (Top Row) Parameters describing response accuracy in log-odds units. (Bottom Row) Parameters describing reaction time in log milliseconds.

Figure 3 plots estimated overall response accuracy and reaction time of all participants, with diagnostic indicators provided. Although the individual estimates of accuracy (top row) are noisy, most participants had posterior means above chance. Additionally, most participants had mean reaction times under 2s (middle row), although a few participants responded more slowly.

Figure 3.

Mean participant performance at the start of the testing phase, based on the regression model. Gray boxes correspond to non-ASD participants and white boxes correspond to ASD participants. Boxes represent the 80% credible interval for the mean, and whiskers represent the 95% credible interval. (Top Row) Mean response accuracy across all pairs at the start of testing. Chance responding is depicted as a dotted line. (Middle Row) Mean reaction time across all pairs at the start of testing. (Bottom Row) Indicator of each participant’s diagnostic categorizations, based on the consensus diagnostic process described in the Methods.

The diagnostic breakdown of participants in Table 1 (reproduced in Figure 3, bottom row) reveals an important limitation of working with a self-selected clinical sample. Only a single participant received no diagnoses within our framework, and the manner in which diagnoses were correlated was nonrandom. For example, ASD overlapped with ADHD more than expected by chance alone, according to a permutation test (Observed=12, Expected=8.4±1.56, p < .05), a result consistent with reported population norms (Rong et al., 2021). However, ASD also overlapped less than would be expected from chance alone with SLD (Observed=12, Expected=8.4±1.56, p < .05). ASD’s overlap with AD (Observed=7, Expected=6.3±1.47, p = .89) and CD (Observed=4, Expected=6.8±1.5, p = .11) did not differ significantly from expectation. Overall, of the 32 possible combinations of these diagnoses, only 17 appeared even once. As such, one should be cautious interpreting model parameters. For example, since nearly all non-ASD participants received either an SLD or a CD diagnosis in this sample, their reaction time intercept ${\gamma }_{\varnothing }$ is, in practice, almost always combined with the trending-above-zero parameters of either ${\theta }_{\text{SLD}}$ or ${\theta }_{\text{CD}}$. A larger sample would make these results easier to interpret, particularly one with more participants without diagnoses in any of our five categories. Larger samples are also required to make representative projections to population performance for each diagnosis.

Because statistical modeling can obscure a summary of the measured values in a study, we also performed a simple average of response accuracy and reaction times across participants in the Non-ASD and ASD groups during the first block of testing (Figure 4). This averaging treated participants as independent of one another, with confidence intervals computed using bootstrapping. However, these means should be interpreted cautiously, because they are unable to address the underlying covariation between diagnostic categories. Nevertheless, some of the main takeaways from the regression analysis remain evident, particularly the slower reaction times of ASD participants at the start of testing.

Figure 4.

Simple means across groups for all critical pairs, estimated via bootstrapping. Gray boxes correspond to non-ASD participants and white boxes correspond to ASD participants. Boxes represent the 80% confidence interval for the mean, and whiskers represent the 95% confidence interval. (Top Row) Mean response accuracy across critical pairs at the start of testing. Chance responding is depicted as a dotted line. (Bottom Row) Mean reaction time across critical pairs at the start of testing.

Although IQ was measured as part of the consensus diagnostic process and served as an inclusion criterion for the study, IQ was not used as a parameter in our regression models. As such, it is reasonable to ask whether IQ was predictive of estimated TI performance. Figure 5 plots these relationships for both response accuracy and reaction time. Spearman’s correlation tests yield no meaningful association with IQ for either response accuracy (r_s =−0.02, p = 0.88) or for reaction time (r_s =−0.01, p = 0.94). This is an important finding, because it suggests that the TI task is neither trivially easy nor prohibitively difficult for a wide range of child-aged participants.

Figure 5.

Task performance at the start of the testing phase as compared to participant IQ (reported in z units). Black points correspond to non-ASD participants and white points correspond to ASD participants. Whiskers represent the 95% credible interval of the mean. (Left) Mean response accuracy across all pairs compared to IQ. (Right) Mean reaction time across all pairs compared to IQ.

Discussion

Participants, on average, learned the order of items in a 7-item list in under 200 trials, responding above chance with the suggestion of a symbolic distance effect for accuracy. These results, considered both in terms of response accuracy and reaction time, are evidence of relational learning (Dusek & Eichenbaum, 1997) across all diagnostic categories.

Diagnostic Categories

Participants with an ASD diagnosis took longer on average to respond during testing, and displayed a higher learning rate, as compared to non-ASD participants. Model comparison suggested a substantive contrast between ASD and non-ASD participants. This is noteworthy because both groups showed almost identical levels of response accuracy at the start of the task, as characterized by having nearly identical population intercepts (${\gamma }_{\varnothing }$) and distance effects (${\gamma }_D$) with respect to response accuracy. Informally, while both groups obtained similar outcomes when testing began, they may nevertheless have differed in the processes by which those outcomes were achieved. A qualitative difference may exist despite there being no evidence of an overall performance deficit.

Because our task involved the consistent application of a rule with no contradictory outcomes, our result is consistent with the hypothesis posed by Van de Cruys et al. (2014) that many deficits associated with ASD stem from uncertainty and seemingly inconsistent feedback. A larger difference might have been observed in our study if reward delivery was probabilistic. Reaction time and learning rate effects may be driven by differences in the speed-accuracy tradeoff and may reflect attention divided between the presented stimuli (see also Todd et al., 2009). Differences in strategy may also be present; Qian & Lipkin (2011) proposed a bias away from “interpolation” in ASD, in favor of “lookup-table” learning, so slower reaction times may be compensatory to maintain a given level of accuracy. Participants with AD or CD diagnoses did less well on average, with mean effects large enough to cancel out the model’s intercept term (Figure 2). As such, most (but not all) participants who received either of these diagnoses began the testing phase with response accuracies close to chance. By contrast, ADHD had almost no differential role, particularly for response accuracy at the start of testing.

It would be a mistake to uncritically compare our work with other reported effects in the literature, given that a majority of our ASD participants had an ADHD diagnosis, and vice versa. Our hope is that future work reflects the full covariance of diagnostic categories appearing in the general population, rather than relying heavily on exclusion criteria. Reducing strict commitment to diagnostic categories as predictors was central to the Research Domain Criteria (RDoC) initiative in 2009, and ASD and ADHD research has benefited from this broader focus (Pacheco et al., 2022).

That said, inclusive recruitment in clinical populations is a complicated endeavor. Statistically, estimating covariation among diagnostic categories requires collecting a larger sample. Until the mechanisms underlying symptoms are better understood, such studies also require complex models with a large number of covarying parameters. The present study should be understood as strictly preliminary, and the large errors in its estimates reflect that any claims we make are provisional.

Recruitment also presents sampling bias challenges. The present paper also does not have a well-defined control group, as only one participant did not qualify for any of the five diagnostic categories. As such, while we routinely invoke the category of “non-ASD participants,” this category should not be interpreted as a neurotypical control group; rather, our non-ASD participants belong to the narrower population of children whose guardians had them evaluated by clinicians, almost all of whom have some other diagnosis. Our results should be viewed as strictly exploratory, and we do not recommend extrapolating these results to the general population (Emerson, 2015).

Such extrapolation could, in a sufficiently large sample, be accomplished with techniques like Multilevel Regression with Poststratification (Downes et al., 2018). Small convenience samples can be extrapolated somewhat reliably when adjusted using weights derived from large scale datasets such as the National Longitudinal Transition Study 2 (NLTS2); however, this introduces two additional problems. First, such a dataset must exist – the youngest participants in NLTS2 were 13 years old, so it could not be used for the current study. Second, computing the relevant weights requires that participants disclose much more information about themselves, and even if raw participant information is not disclosed, reporting individual participant weights may be sufficient to “decode” identifying information about participants.

Given these difficulties, it remains notable that partitioning our sample by ASD diagnosis not only best explains our data, but does so overwhelmingly for both TI response accuracy and reaction time (Appendix, Table 3). This does not appear to reflect impairment, but rather a different strategy or approach to the task. ASD participants responded more slowly than non-ASD participants during the testing phase, and this slower responding may help to explain their higher learning rate, possibly due to a strategy that emphasized accuracy over speed. We take from this that the current literature’s usual emphasis on detecting impairment and performance deficits may be misguided in the case of TI and other tasks pertaining to analogical reasoning. Unambiguously, participants with ASD were successful at the task, and a larger sample will help to determine what distinctive characteristics they demonstrate on their road to success.

Table 3:

Model weighs under a Bayesian stacking comparison

Model	Sample (Pool 1)	Sample (Pool 2)	Response Accuracy Model Weight	Reaction Time Model Weight
Single Pool (no split)	40	N/A	0.15%	1.82%
Split by ASD	19	21	93.12%	94.99%
Split by ADHD	24	16	0.00%	1.26%
Split by AD	28	12	6.71%	1.76%
Split by SLD	22	18	0.02%	0.16%
Split by CD	27	13	0.00%	0.00%

IQ, ASD, and TI

Though all participants had IQ scores of 70 or above according to the WISC-V, there was no meaningful association with IQ scores and successful performance of our TI task.

There is little debate that IQ is an effective early-detection tool for identifying long-term intellectual disability (ID). This connection has been made especially strongly in cases of ASD diagnosis, as population studies suggest elevated rates of ID for those with an ASD diagnosis (CDC, 2012). As such, early life IQ measures reportedly provide important predictors for adult ID outcomes in ASD cases (Howlin et al., 2004; Bishop et al., 2015; Denisova & Lin, 2023). However, IQ as a more general framework for intelligence has come under increasing scrutiny. Some argue that IQ is conceptually narrow and excludes important aspects of decision making (Ganuthula & Sinha, 2019). Others raise concerns that it measures structural variables different than those intended by theory (Weiss & Saklofske, 2020) and that distortions introduced by misinterpretations of IQ are subsequently magnified by bad policy (Ford et al., 2016).

Because we excluded participant IQ measures below 70, we make no claims regarding any relationship between TI and ID. However, among our participants, IQ did not appear related to performance, despite sampling from over four standard deviations of the measure (Figure 5). This is consistent with prior work suggesting that, in children under 11, even fairly high IQ scores do not consistently translate to transitive reasoning. Wolf & Shigaki (1983) report that a cohort of children aged 8–10 with an IQ exceeding 130 nevertheless had high error rates for verbal transitive reasoning questions. Future studies should address whether TI relies more on implicit learning systems that do not show strong correlations with IQ (e.g., as described by Maybery et al., 1995; Kalra et a., 2019).

Conclusion

Cognitive processing differences in autism spectrum disorder (ASD) have not been sufficiently disentangled from other comorbid diagnoses in the literature. This complicates claims that ASD tends to be associated with performance deficits for various tasks. Children in the current sample with an ASD diagnosis performed transitive inferences at a comparable rate to those without one, after controlling for comorbid diagnoses. However, participants with ASD also had slower reaction times during testing, suggesting a different preferred speed-accuracy tradeoff. Furthermore, the response accuracy of participants with ASD improved more per trial during the testing phase, possibly as a benefit of their slower reaction times. Partitioning the sample with respect to ASD/Non-ASD was overwhelmingly preferred to any other diagnostic split according to a model comparison analysis. This suggests a different response pattern chiefly explained by ASD, and not by other comorbidities. Future research seeking the characterize how cognitive faculties vary across psychiatric diagnoses must do more to acknowledge and statistically manage diagnostic comorbidities.

Appendix: Statistical Analysis

Proportion of correct responses during the all-pairs testing phase of the experiment was analyzed using logistic regression. At the participant level, this was accomplished using four parameters, as given in Equations 1 and 2:

$${\mu }_{\text{part}}={\beta }_{\mathrm{\varnothing }}+{\beta }_t·t+{\beta }_D·D+{\beta }_{Dt}·D·t$$ [Eq. 1]

$$p\left(\text{Correct}\right)=\frac{1}{1+\exp \left(-{\mu }_{\text{part}}\right)}$$ [Eq. 2]

Here, ${\mu }_{\text{part}}$ constitutes an overall participant effect derived from four parameters: An intercept ${\beta }_{\varnothing }$, which governs overall response accuracy at the moment of transfer from training to testing; a learning rate ${\beta }_t$, which determines how accuracy changes as a function of trials $t$; a distance effect ${\beta }_D$, which governs the effect of symbolic distance $D$ at transfer; and an interaction term ${\beta }_{Dt}$, which determines the rate at which the distance effect changes as a function of trial. Here, $D$ is defined as the centered symbolic distance with a mean of zero, accomplished by subtracting $\raisebox{1ex}{$8$}\!\left/ \!\raisebox{-1ex}{$3$}\right.$ from the actual symbolic distance. For example, for each adjacent pair, $D=1-\raisebox{1ex}{$8$}\!\left/ \!\raisebox{-1ex}{$3$}\right.=-1.\stackrel{-}{6}$, whereas for a symbolic distance of 6, $D=6-\raisebox{1ex}{$8$}\!\left/ \!\raisebox{-1ex}{$3$}\right.=3.\stackrel{-}{3}$. Equation 2 translates the log-odds value of ${\mu }_{\text{part}}$ into a proportion of correct responses using the logistic function. If an analysis were performed on a single participant in isolation, Equations 1 and 2 would be sufficient.

Measuring the impact of diagnostic categories required a hierarchical analysis, with diagnoses coded as group-level effects. Thus, parameters for ${\mu }_{\text{part}}$ were estimated for each participant, but additional global parameters were also estimated for diagnostic categories. In practice, this might take the following form:

$$\begin{gathered} {\mu }_{\mathrm{p}\mathrm{o}\mathrm{p}}={\theta }_{\mathrm{A}\mathrm{S}\mathrm{D}}\cdot C_{\mathrm{A}\mathrm{S}\mathrm{D}}+{\theta }_{\mathrm{A}\mathrm{D}\mathrm{H}\mathrm{D}}\cdot C_{\mathrm{A}\mathrm{D}\mathrm{H}\mathrm{D}}+{\theta }_{\mathrm{A}\mathrm{D}}\cdot C_{\mathrm{A}\mathrm{D}}+\dots \\ \dots {\theta }_{\mathrm{S}\mathrm{L}\mathrm{D}}\cdot C_{\mathrm{S}\mathrm{L}\mathrm{D}}+{\theta }_{\mathrm{C}\mathrm{D}}\cdot C_{\mathrm{C}\mathrm{D}} \end{gathered}$$ [Eq. 3]

$$p\left(\text{Correct}\right)=\frac{1}{1+\exp \left(-\left({\mu }_{\text{part}}+{\mu }_{\text{pop}}\right)\right)}$$ [Eq. 4]

Here, each ${\theta }_{\text{X}}$ estimates the effect for a given diagnosis $\mathrm{X}$, while $C_{\text{X}}$ is the centered indicator function; that is, if the indicator function $I_{\text{X}}$ acts as a dummy code in which $I_{\text{X}}=0$ when a participant does not have diagnosis $\mathrm{X}$, and $I_{\text{X}}=1$ when a participant has diagnosis $\mathrm{X}$, then $C_{\text{X}}=\left(I_{\text{X}}-0.5\right)$. This centering prevents the group effects from distorting the participant-level intercepts. Under such a model, we may assume that an individual’s participant-level estimates are drawn from a population distribution, as follows:

$$\left[\begin{array}{c}\begin{array}{c}{\beta }_{\mathrm{\varnothing }}\\ {\beta }_t\end{array}\\ \begin{array}{c}{\beta }_D\\ {\beta }_{Dt}\end{array}\end{array}\right]~MVNormal\left(\left[\begin{array}{c}\begin{array}{c}{\gamma }_{\mathrm{\varnothing }}\\ {\gamma }_t\end{array}\\ \begin{array}{c}{\gamma }_D\\ {\gamma }_{Dt}\end{array}\end{array}\right],\mathbf{\Sigma }\right)$$ [Eq. 5]

An identical construction to Equation 5 can be used in order to study reaction times, which are generally close to Gaussian on a log scale. All that is required is that the participant-level parameters in Equation 1 be used to calculate the mean of a Gaussian likelihood, and the addition of a participant-level residual error term $\sigma$. Analyses were of reaction time were conducted using the natural log.

A complication in setting up a hierarchical model, especially one in which each participant has many parameters that describe only a modest amount of data, is that the group-level estimates have a strong regularizing effect on participant-level estimates. This ‘shrinkage’ is desirable when participants resemble one another, but it isn’t necessarily appropriate if subgroups of participants come from distinct populations with different population distributions. Indeed, such shrinkage can conceal important differences if not taken into consideration. Another way to set up the model is to designate distinct groups according to some indicator (such as the presence or absence of a diagnosis), allowing shrinkage to occur in each without impacting estimates in the other. This gives rise to two potential designs. The first design assumes that all participants belong to a single pool, with all participant-level parameters drawn from a single population distribution per Equation 5 and with diagnostic effects handled exclusively by the ${\theta }_{\text{X}}$ parameters in Equation 3. The second design creates two copies of Equation 5 (one with participants who have a particular diagnosis and one without) and estimates the participant-level effect separately, while also omitting ${\theta }_{\text{X}}$ from Equation 3 (since it becomes redundant with the difference between the two group estimates ${\gamma }_{\varnothing }$). Given five diagnostic categories along which to split the data, this means there are a total of six potential models that could be run, making markedly different assumptions about how best to group the participant to maximize predictive strength.

We used the stacking approach to Bayesian model comparison with leave-one-out (LOO) estimation (Yao et al., 2018) to compare models in terms of their out-of-sample predictive strength (based on variation in the estimated pointwise likelihood) and, when appropriate, combined using a weighted average. If multiple models appear to be compelling, stacking those models is likely to be a better option than trying to split a small sample into smaller and smaller subsamples. If a single model dominates, then its grouping design was most effective, and that will be the focus for the subsequent analyses.

We compared six models: One that pooled all participants in a single multi-level model, and five that split the participant pool according to each of the diagnostic categories, connecting the two pools only by the constant group-level effects ${\theta }_{\text{X}}$ for each model’s remaining four diagnostic categories. Table 3 displays the estimated model weights for each of these models.

Overwhelmingly, in both analyses, splitting the participant pool with respect to ASD resulted in the best estimated predictive strength. This is partly because splitting by ASD results in the most even split of the participants. Even so, because the Single Pool case has fewer free parameters, and the ADHD and SLD splits are also nearly equal, we feel the high model scores received by the ASD partition most likely reflect meaningful differences between our ASD and non-ASD participants. We proceeded with an analysis that focused on the Split by ASD model as providing an adequate description of this sample, at least within the measurement tolerances of our sample.

Data Availability:

Fully anonymized experimental data and related materials are available in an Open Science Foundation repository < https://osf.io/dqztb/ >.