Idiosyncratic Brain Activation Patterns Are Associated with Poor Social Comprehension in Autism

Abstract

Autism spectrum disorder (ASD) features profound social deficits but neuroimaging studies have failed to find any consistent neural signature. Here we connect these two facts by showing that idiosyncratic patterns of brain activation are associated with social comprehension deficits. Human participants with ASD (N = 17) and controls (N = 20) freely watched a television situation comedy (sitcom) depicting seminaturalistic social interactions (“The Office”, NBC Universal) in the scanner. Intersubject correlations in the pattern of evoked brain activation were reduced in the ASD group—but this effect was driven entirely by five ASD subjects whose idiosyncratic responses were also internally unreliable. The idiosyncrasy of these five ASD subjects was not explained by detailed neuropsychological profile, eye movements, or data quality; however, they were specifically impaired in understanding the social motivations of characters in the sitcom. Brain activation patterns in the remaining ASD subjects were indistinguishable from those of control subjects using multiple multivariate approaches. Our findings link neurofunctional abnormalities evoked by seminaturalistic stimuli with a specific impairment in social comprehension, and highlight the need to conceive of ASD as a heterogeneous classification.

Introduction

Atypical social behaviors are a core diagnostic feature of autism spectrum disorder (ASD), whose neural basis is a target of intense research focus (Kennedy and Adolphs, 2012). Yet despite obvious real-world social impairments, individuals with ASD often demonstrate normal performance on simplified laboratory tasks assessing social cognition (Bowler, 1992; Castelli, 2005; Fletcher-Watson et al., 2008; see also Pantelis et al., 2015). Further, performance on tasks using complex, naturalistic social stimuli can better distinguish ASD and neurotypical (NT) individuals than more impoverished stimuli (Heavey et al., 2000; Roeyers et al., 2001; Dziobek et al., 2006; Golan et al., 2008). Naturalistic social videos may thus be an important tool for identifying brain signatures of ASD (Pantelis et al., 2015).

For video stimuli, a recently introduced analytical strategy is to quantify the similarity across individuals in the spatiotemporal pattern of fMRI-measured evoked activity (Bartels and Zeki, 2004; Hasson et al., 2004). In a given brain region (voxel, or group of voxels, in a common space), when moments in the video drive brain activity similarly across subjects, high inter-subject correlation (inter-SC) is observed: this is the case in primary sensory areas strongly driven by low-level stimulus properties, such as brightness or loudness in the video. Inter-SC can be computed throughout the brain to detect activity coordinated across subjects in regions engaged by more complex processing of the stimulus. The inter-SC method has been used in only two previous studies of ASD, which both found less similarity within the ASD group than the NT group in the time courses of the hemodynamic responses evoked by viewing a movie (Hasson et al., 2009; Salmi et al., 2013).

Such increased intersubject variability within ASD is a consistent finding of studies of brain and behavior in ASD (Müller et al., 2003; Humphreys et al., 2008; Towgood et al., 2009) and is thought to relate to the heterogeneity of the clinical presentation: severity and types of symptoms and comorbidities, etiology, and developmental trajectory (Geschwind, 2009; Levy et al., 2009; Hallmayer et al., 2011; Fountain et al., 2012; Landa et al., 2012; Lord et al., 2012). Parsing this heterogeneity is a major research goal (Cuthbert and Insel, 2013). It is as yet unclear whether the common social impairments observed in ASD are linked with heterogeneous (Amaral et al., 2008; Levy et al., 2009; Brock, 2011; Lenroot and Yeung, 2013; Castles et al., 2014; Waterhouse and Gillberg, 2014) or common (Geschwind and Levitt, 2007; Pelphrey et al., 2011; Wass, 2011; Kennedy and Adolphs, 2012) systems-level neurobiological presentations (see also Edelman and Gally, 2001; Karmiloff-Smith et al., 2014). Conventional neuroimaging methods may obscure heterogeneity due to statistics that average across individuals within a group (Towgood et al., 2009; Brock, 2011; Castles et al., 2014); in contrast, the inter-SC method is well suited to examining individual variation (Hasson and Malach, 2006).

Here, we investigate brain signatures of ASD during perception of naturalistic social interactions in light of this heterogeneity. We compute similarity (inter-SC) in the time courses of hemodynamic activity evoked during unconstrained viewing of a television show (“The Office”) across NT and high-functioning ASD individuals (Fig. 1a), and examine the results in light of an extensive battery of neuropsychological measurements, behavioral data, and test–retest reliability of subjects' brain responses. We uncover profound neurofunctional heterogeneity within the ASD group that is linked with specific impairments in social understanding. Importantly, we find that only a subset of ASD individuals' brain responses are distinguishable from NT.

Figure 1.

Schematic of approach. a, Inter-SC captures the extent to which time courses of the BOLD response within some part of the brain are similar between individuals as they watch the same video (here, a seminaturalistic video depicting social interactions). Inter-SC is computed for two individuals as the Pearson correlation between the time courses of their brain activity in the relevant voxel or ROI. To get a measure of the similarity of a given subject to his group, or to a reference group, pairwise correlations with the relevant set of other subjects are averaged (average inter-SC to a reference group is depicted here). See Materials and Methods for additional details. b, The Yeo et al. (2011) cortical parcellation into 17 networks, used for network-level analyses. Left hemisphere ROIs are depicted here.

Materials and Methods

Participants

Nineteen high-functioning adults with an ASD (mean age 28.9; range 20–47; 5 female) and 24 age- and IQ-matched NT controls (mean age 26.7; range 20–45; 4 female), participated in this study. Two ASD participants and two NT participants were excluded due to excessive motion, and two NT participants were excluded due to a technical issue during data acquisition, resulting in a final sample of 17 ASD (referred to as A01–A17) and 20 NT participants (C01–C20; Table 1). DSM-IV ASD diagnosis was confirmed by a clinical psychologist following administration of the Autism Diagnostic Observation Schedule (ADOS), and Autism Diagnostic Interview-Revised (ADI-R) or Social Communication Questionnaire (SCQ; unavailable in two participants). The revised algorithm for the ADOS, proposed by Hus and Lord (2014), was also applied (except for three subjects for whom the raw data were not available, as the ADOS had been administered at a different institution). Screening for current and prior prescription and recreational drug use was performed for all subjects. Of the 17 ASD participants, eight were taking one or more psychotropic medications, to alleviate depressive symptoms or attentional issues. Of the 20 control participants, none reported taking any prescription or recreational drugs. All subjects provided written, informed consent; all experimental procedures were approved by the California Institute of Technology Human Subjects Protection Committee.

Table 1.

Participants' demographic information, IQ, and ADOS scores (for ASD participants)

	NT/ASD min–max	NT mean (SD)	ASD mean (SD)	Test	p
Sample		20	17
Fraction male		85%	70.6%	χ(1)2 = 1.126	0.289
Age at scan (years)	20–45/20–47	26.65 (5.32)	28.88 (8.93)	t(35) = −0.940	0.354
Handedness score	33–100/−40–100	80.03 (17.92)	70.18 (42.60)	t(35) = 0.942	0.352
Verbal IQ	85–131/80–139	114.00 (10.05)	112.65 (17.77)	t(35) = 0.291	0.773
Performance IQ	87–124/93–122	110.65 (8.96)	108.76 (9.00)	t(35) = 0.637	0.529
Full scale IQ	97–126/93–133	113.70 (8.45)	111.12 (12.33)	t(35) = 0.752	0.457
ADOS communication	–/2–7	—	4.29 (1.61)	—	—
ADOS:social interaction	–/4–14	—	8.59 (3.39)	—	—
ADOS:creativity	–/0–2	—	0.76 (0.56)	—	—
ADOS:stereotyped behavior	–/0–4	—	1.65 (1.54)	—	—
ADOS2:restricted and repetitive behavior	–/1–7	—	3.50 (1.65)	—	—
ADOS2:social affect	–/6–20	—	13.00 (5.07)	—	—

All t tests were two-tailed and unpaired, assuming equal variance. χ² with Yate's correction was used to test differences in proportions (fraction male).

Stimuli and tasks in the scanner

Subjects freely viewed one complete 22 min episode of the television sitcom “The Office” (“Hot Girl,” sixth episode and season finale of the first season, aired in 2005), divided into two approximately equal parts (Part 1, 10 min 18 s; Part 2, 10 min 44 s), in the scanner. The episode was chosen because it featured constant social interactions that often required perceiving and interpreting subtle, rapidly changing, nuances of facial expressions, body language, and dialogue; in short, we expected it to elicit demands on social processes. To assess test–retest reliability, Part 1 was shown again later in the fMRI session (for all participants but three ASD and one NT, due to fatigue and time constraints). No instructions were provided other than to watch the episode and pay attention. Video was back-projected onto a screen that was visible to subjects via a mirror attached to the head coil. Audio was provided via MR-compatible headphones (Sensimetrics model S14, www.sens.com). Subjects also participated in additional functional runs, to be reported elsewhere (see MRI Data Acquisition).

Behavioral tasks outside the scanner

Eye tracking.

In a separate session administered before the MRI acquisition, all subjects from the current study (ASD N = 17; NT N = 20; plus an additional 12 NT subjects) freely watched the pilot episode of the sitcom (first episode of the first season, aired in 2005, to get them familiarized with the situation and characters) while undergoing eye tracking using a Tobii TX300 head-free eye tracker. Due to difficulties during calibration, data were only useable from 28 controls (including 18 of the 20 fMRI NT participants) and 12 of 17 ASD participants (including 3 of 5 ASD subjects with idiosyncratic brain responses).

Comprehension task.

In a separate session after MRI acquisition, a subset of subjects (14/17 ASD subjects, including 3/5 of the low inter-SC ASD; 17/20 NT subjects) freely watched a separate episode (“The Alliance,” fourth episode of the first season, aired in 2005) of the same sitcom. The video was edited to introduce “pauses” at 28 time points, approximately evenly spaced, at which a total of 39 questions about the content of the previous scene were asked (between 1 and 5 questions were asked at each pause). Most questions were designed to probe social understanding. Subjects' free-form verbal responses to these questions were spoken into a microphone and recorded.

MRI data acquisition

All images were acquired using a 3 Tesla whole-body MRI system (Magnetom Tim Trio, Siemens Medical Solutions) with a 32-channel head receive array. During viewing of the sitcom episode, T₂*-weighted echo planar imaging (EPI) data were acquired (TR/TE = 2500/30 ms, flip angle = 85°, 3 mm isotropic voxels and 47 slices acquired with ascending order, covering the whole brain). For all subjects, seven functional runs were collected in a predefined order: Part 1 then Part 2 of the sitcom episode (247 and 258 volumes, respectively), two unrelated runs to be reported elsewhere, another run of Part 1 of the episode (for assessment of within-subject consistency) and then Part 2 with different instructions and finally one unrelated run, both to be reported elsewhere. Gradient echo field mapping data were acquired in between functional runs with identical geometry to the EPI data for off-resonance distortion correction (TR/TE = 500/2.5, 5.0 ms, flip angle = 60°). High-resolution T₁-weighted images of the whole brain (MPRAGE, 1 mm isotropic voxel size; TR/TE/TI = 1500/2.9/800 ms) were acquired as anatomical references. Wakefulness (and blinking) were monitored using a coil-mounted MR-compatible miniature infrared CCD camera with integrated illumination and monochrome analog interlaced NTSC output at 29.97 Hz (Resonance Technologies). Video was captured and recorded to an external computer (Video2PC, Ion Audio, and PowerDirector 8, CyberLink,) for review and analysis; for technical reasons such as acquisition problems or difficulty to identify onset of scanning, videos for 5 of 37 subjects were unusable, including one of the ASD subjects with idiosyncratic brain responses. Pupil tracking was attempted but videos were not of sufficient quality (e.g., absence of a reliable glint) to accurately reconstruct the gaze patterns of all subjects as they watched the movie clip.

MRI data preprocessing

Data were preprocessed using FEAT (v6.00) within FSL (FMRIB's Software Library, www.fmrib.ox.ac.uk/fsl). The initial two volumes (5 s) of each functional run were discarded to minimize magnetization equilibration effects. Preprocessing steps included rigid-body motion correction, slice-timing correction, field map-based geometric distortion correction, nonbrain removal, and temporal filtering (100 s nonlinear high-pass filter cutoff). No spatial smoothing was performed at this stage. Off-resonance geometric distortions in the EPI data were corrected using PRELUDE and FUGUE within FSL, using B0 field maps derived from the dual-echo gradient echo dataset acquired with identical slice angle and voxel size as the EPI data. Registration of functional images occurred as follows: first, affine linear registration to each subject's T₁-weighted anatomical image using boundary-based registration via FLIRT within FSL; second, nonlinear registration of the T₁-weighted anatomical image to the standard Montreal Neurological Institute space (MNI152) using FNIRT within FSL.

Functional images were corrected via linear regression of nuisance variables for each run for each subject, and all analyses were performed on the resulting residuals, subsequently referred to as the corrected BOLD signal. Nuisance regressors included six rigid body motion parameters, global white matter and global CSF signal time courses, all together with their derivatives, as well as time courses of relative fluctuations in the BOLD signal outside the head at the slice level. The latter nuisance regressors were included to address dynamic Nyquist ghost artifacts resulting from an intermittent stability problem with 2 of 32 elements of the head coil. To further ensure data quality, we censored time points of the residuals that corresponded to functional volumes with large changes in global signal intensity (DVARS >0.5% BOLD signal change; Power et al., 2012).

MRI data analysis

Overview.

For each subject, we extracted the time courses of all brain voxels from the preprocessed BOLD signal over the duration of the video; we also averaged the time courses of voxels within large-scale functional networks (according to a published cortical parcellation; Yeo et al., 2011; Fig. 1b). At both scales (voxel-level and network-level), we quantified the similarity of time courses across subjects using pairwise Pearson correlations (Fig. 1a). We then averaged over these pairwise correlations to determine how similar subjects were to other subjects within their same group (ASD or NT) and how similar subjects were to a reference group (a randomly chosen subset of the NT subjects). Then, at the network level only (which is a lower dimensionality than voxel-level), we used multivariate analysis techniques (one-class support vector machine, multidimensional scaling, and k-means clustering) to characterize the underlying structure in the data beyond group averages.

Voxel-level pairwise inter-SC.

The corrected BOLD signal for each sitcom half-episode for each individual was smoothed with a 4 mm full-width-at-half-maximum (FWHM) 3D Gaussian filter. For all brain voxels in MNI space, we then computed the intersubject correlation (inter-SC) between each pair of subjects (i.e., 20 NT and 17 ASD), resulting in a 37 × 37 matrix at each voxel. We refer to this matrix as the pairwise inter-SC matrix (PISCM) in the following paragraphs, and it forms the basis of all further analyses. We computed the PISCM separately for each functional run (Part 1 run 1, Part 2 run 1, Part 1 run 2). Unless stated otherwise, we present results based on the PISCM averaged over Part 1 run 1 and Part 2 run1 (all subjects completed these two runs; averaging leads to better signal-to-noise).

Network-level pairwise inter-SC.

To characterize brain response similarity at a complementary functional level, we also examined inter-SC using the average time courses within intrinsic functional networks. Intrinsic networks correspond strongly with task networks (Smith et al., 2009; Betti et al., 2013; Cole et al., 2014), and are thus functionally relevant. Here, we used a publicly available and robust parcellation of the whole cortex into 17 intrinsic functional networks (Yeo et al., 2011; Fig. 1b). The number of networks (17) in this parcellation was determined in a data-driven manner based on resting-state data acquired from a large cohort of 1000 subjects (Yeo et al., 2011), and the parcellation is very similar to other intrinsic network parcellations generated using different methods (Power et al., 2011). Individual network masks were created for each subject from the product of the individuals' gray matter partial volume estimate (obtained from FSL FAST) and the network mask (Tyszka et al., 2014). The mean signal for each individual network mask was then extracted from the corrected BOLD signal for each functional run, resulting in 17 network time courses for each subject, for each functional run. We computed the PISCM for each of the 17 networks.

Chance-level pairwise inter-SC.

To estimate the distribution of inter-SC that would arise by chance between any two subjects, we generated surrogate time courses with identical amplitude spectra as the true time courses. This was achieved through a discrete fast Fourier transform of the true time course (using MATLAB's function fft), followed by the assignment of random phases (between 0 and 2π) to each frequency component, and finally taking the real part of the discrete inverse fast Fourier transform (MATLAB's ifft). For each subject/run/network, a set of 1000 such random time courses were generated. Surrogate values of pairwise inter-SC were calculated from these random time courses, and were used for statistical assessment.

Within-group inter-SC.

To compute the inter-SC of each subject with subjects of the same group, we averaged the (fisher-Z transformed) correlation values in each row of the PISCM (i.e., for each subject) across the columns corresponding to subjects from the same group; this procedure yielded a vector of length 37, representing the similarity of each subject to their own group, at each voxel in MNI space or for each network.

Bootstrapped inter-SC to a reference group.

A random subset of 5 of 20 NT subjects was picked. Five is arbitrary, and results remain the same with 3, 4, 6, or 7 reference subjects; we chose five as a sample large enough to drown individual idiosyncrasy in the reference group, but small enough that there are enough NT subjects left to compare with the ASD subjects. The fisher-Z transformed pairwise inter-SC to these five subjects were averaged for each remaining subject, yielding the average correlation of each subject to a reference group of NT subjects. There are many possible random picks of five subjects among 20 (exactly C(20,5) = 15,504 unique sets). The procedure was repeated 1000 times (keeping the computational load low for some of the more intensive analyses), and the results averaged for each subject (note that at each random pick of five subjects, estimates are missing for those five subjects; the final averaging ignores the missing values).

One-class support vector machine.

The one-class support vector machine (OC-SVM) is a special case of the SVM algorithm, used for outlier detection (Schölkopf et al., 2001; Mourão-Miranda et al., 2011). The OC-SVM algorithm attempts to estimate a decision function that takes the value +1 in a small region capturing most of the training examples (where “most” is tunable and controlled by parameter ν in the ν-SVM implementation), and −1 elsewhere. We chose the ν-SVM implementation with radial basis function (RBF) kernel (Mourão-Miranda et al., 2011). We started from a 37 × 17 matrix DREF (dissimilarity to the reference group), where each column corresponded to an intrinsic network and represented the dissimilarity of all 37 subjects to a reference group of five NT subjects (dissimilarity is defined as one minus correlation, taking values between 0 and 2). Thus, for each subject (row), we had a 17-dimensional vector representing the dissimilarity to a reference group. We used a leave-one-out procedure, excluding a NT subject at each fold (test set) and using the other NT subjects as the training set. At each fold, we first optimized the kernel parameter γ, keeping the parameter ν fixed (0.1), using a nested leave-one-out cross-validation procedure; we then kept the γ parameter fixed at its optimal value and used another nested leave-one-out cross-validation procedure to optimize ν. Note that if more than one value of the parameters lead to the best accuracy, we chose the smallest one. Finally, we used the optimal parameters to train the OC-SVM on the entire training set and test on the left out NT subject, as well as a randomly picked ASD subject (or 2, to have a prediction for all ASD subjects; indeed, because we are using the inter-SC to five reference subjects as input to the OC-SVM, we have a total of 15 NT and 17 ASD participants available for a given pick of five reference subjects). We used the LIBSVM toolbox (Chang and Lin, 2013; www.csie.ntu.edu.tw/∼cjlin/libsvm) and custom code in MATLAB to perform the analysis. Statistical assessment was conducted using a permutation test, whereby the group labels (NT or ASD) were shuffled at random 1000 times, and the whole procedure (including multiple picks of 5 reference subjects) was repeated for each surrogate labeling.

MDS.

Multidimensional scaling (MDS) can be used as a dimension reduction technique to visualize points that live in a high-dimensional space, by mapping them to a lower dimensional space while preserving inter-point distances as faithfully as possible. MDS thus provides another way to visualize distances between-subjects, without a priori knowledge of whether a subject belongs to the NT or ASD group. We first computed an average pairwise dissimilarity matrix, PDM_av, representing how differently the brains of the 37 subjects responded to the movie clips, in the following way: we transformed the PISCM (values between −1 and 1) in each network N into a pairwise dissimilarity matrix [PDM_N = (1 − PISCM_N); values between 0 and 2], and then averaged the PDM_N's across all 17 networks. We performed nonmetric MDS to project the dissimilarities stored in PDM_av onto two dimensions. In nonmetric MDS, distances between points in the solution approximate a nonlinear but monotonic transformation of the input dissimilarities. Nonmetric MDS allows for contractions or expansions of distances at different scales, relaxing the somewhat strict requirement of metric MDS which attempts to approximate the dissimilarities themselves. We used the common choice of minimizing the Stress-1 criterion (Borg and Groenen, 2005) as follows:

where f is a nonlinear monotonic transformation, p_ij are the pairwise dissimilarities in the input data, and d_ij(X) are the distances measured in the MDS space X. We used the function mdscale in MATLAB, with 100 replications and random initialization. To quantify the extent to which individual subjects deviate from others, we fit a 2D Gaussian distribution to the cloud of NT subjects, and plotted contour lines corresponding to 1, 2, .., 5 SDs of this distribution.

k-means clustering.

We performed k-means clustering to explore data-driven, natural divisions of our participants into distinct groups. k-means clustering operates on actual observations (rather than a precomputed distance matrix), finding a partition into mutually exclusive clusters in which observations within each cluster are as close as possible to one another while being as far as possible from observations in other clusters. Different distance measures can be implemented depending on the question of interest. We submitted the previously described DREF matrix (dissimilarities of each subject to the reference group across all 17 networks), averaged over 1000 random picks of five reference subjects, to the k-means algorithm. In this paper, we used the k-means algorithm twice, once with a squared Euclidean distance measure, and once with a Pearson-correlation-based distance measure. Each cluster is defined by a centroid, the point to which the sum of distances from all observations in the cluster is minimized. k-means is an iterative algorithm, moving observations between clusters until the sum of distances of all points to their respective cluster centroids is minimized. k-means partitions can be assessed using a silhouette plot: the silhouette value of a given point ranges from +1 (indicating a point very distant to other clusters), through 0 (point could just as well be assigned to another cluster), to −1 (point is likely in the wrong cluster); it is defined as follows:

Where a(i) is the average dissimilarity of point i with all other data in the same cluster (the smaller the value, the better the assignment), and b(i) is the lower average dissimilarity of i to any other cluster of which i is not a member (if b(i) is small, then i could also be assigned to this neighboring cluster). The average of all the silhouette values s(i) is a measure of how appropriately the data has been clustered, and can be used to determine the natural number of clusters in a dataset. We used the function kmeans in MATLAB, with 100 replications and random initialization, and the function silhouette for silhouette calculations.

Analysis of gaze behavior during scanning.

We reviewed the videos of subjects' right eye recorded during scanning to verify wakefulness. To further quantify gaze behavior during scanning, we used eye gaze analysis software developed at Caltech by Dr J. Michael Tyszka (mrgaze v0.4.7, written in Python v2.7, publicly available on GitHub at github.com/jmtyszka/mrgaze) to track the pupil in all the videos, and computed summary statistics for detected blinks (which correspond to video frames for which no pupil was detected), such as blink rate, longest blink, and cumulated blink duration.

Analysis of tasks outside scanner

Scoring of comprehension task.

Spoken responses to each of the 39 comprehension questions were transcribed and awarded 0–2 points based on consensus between two blind coders. Points were awarded as follows: 2 points for demonstrating complete understanding, 1 point for incomplete (but not necessarily incorrect) responses that do not fully answer the question (e.g., by omitting key information), and 0 point for incorrect responses.

Analysis of eye tracking.

Although these results will be reported in greater detail elsewhere, briefly, we compared the similarity of each ASD subject's eye gaze pattern to the mean eye gaze of the NT reference group while they watched a particular episode of “The Office” (different from the episode they saw in the scanner). Specifically, we tested whether or not the gaze patterns of the ASD subjects with idiosyncratic brain responses differentiated them from the rest of the ASD subjects. To do so, we used the normalized scanpath saliency (NSS) approach (Dorr et al., 2010), wherein for each frame of the video, the gaze of a single subject (spatially smoothed with a 1° FWHM Gaussian kernel; no temporal smoothing) was compared with the spatially smoothed gaze of the NT group. Higher values indicate greater similarity with the NT gaze pattern. These NSS values were then averaged over the duration of the first 6 min 58 s of the episode (corresponding to 10,028 frames).

Results

ASD brains respond on average less similarly than NT brains

First, we aimed to follow the approach used by other authors (Hasson et al., 2009; Salmi et al., 2013), computing the inter-SC within the ASD group and within the NT group separately in MNI space. Results are based on the first presentation of the episode (Part 1 and Part 2) in the scanner. In Figure 2a, we see that most of the brain shows positive inter-SC, indicating that the brains of all subjects respond similarly to the video, with some regions—early visual cortex, superior temporal gyrus—driven more reliably than others (Salmi et al., 2013). We then performed a t test at each voxel to test for group differences in their within-group similarity; the results of this analysis are presented on the inflated surface in Figure 2b. Overall, the brains of NT subjects respond more similarly to each other than do the brains of ASD subjects, consistent with previous reports (Hasson et al., 2009; Salmi et al., 2013).

Figure 2.

Within-group inter-SC while watching an episode of the American sitcom “The Office”. a, Voxelwise within-group inter-SC for 20 NT subjects (top) and for 17 ASD subjects (bottom), projected on the inflated brain using Caret's Conte69 atlas. Note that the highest correlation values are >0.3; however, the scale is limited to ±0.1 (and unthresholded) for visualization purposes. b, Voxelwise T values for the difference in within-group inter-SC between 20 NT subjects and 17 ASD subjects; both a loosely thresholded (p < 0.001 uncorrected) and an unthresholded map are shown to represent available information as faithfully as possible. c, within-group inter-SC for 20 NT (blue) and 17 ASD (red), based on time courses in 17 functional networks. Error bars are 95% confidence interval based on a bootstrap estimate of the mean of five subjects. The shaded areas around zero represent the 95% confidence interval for the distribution of group inter-SCs that would arise by chance, for each group. Statistics for the network analyses are FDR-controlled (***q < 0.001, **q < 0.01, *q < 0.05). See Figure 1b for the networks.

Voxelwise inter-SC is an important source of information, but it is also subject to potential pitfalls due to individual variation in alignment and low signal-to-noise at the voxel level. To address these limitations, we next examined inter-SC in a set of larger regions of interest, using a common cortical parcellation into 17 intrinsic networks (Yeo et al., 2011). As Figure 2c shows, the BOLD responses in all 17 networks were more consistent in the NT group than in the ASD group, and several of the networks showed statistically significant differences between groups after false discovery rate (FDR) control for multiple comparisons (Benjamini and Hochberg, 1995; where the FDR threshold is denoted by q). We find that all networks are driven by the video significantly above chance (as estimated using randomly generated surrogate time courses with the same power spectrum as the true data; all q < 0.01), apart from LimbicB in the ASD group (p = 0.119).

ASD brains respond on average as similarly as NT brains to a reference group of NT brains

The results thus far suggest that the brain responses of ASD subjects when they watch a sitcom rich in social information are less similar to one another than the brain responses of NT subjects, consistent with previous results (Hasson et al., 2009; Salmi et al., 2013). However, within-group similarity does not tell us how similar the brain responses of ASD subjects are to the brain responses of NT subjects. To address this question, we used a bootstrap approach wherein we randomly picked a reference group of five NT subjects and computed the similarity of each remaining subject with this reference group.

Using this approach, we compared the two groups in terms of their similarity to a reference group of NT subjects. As seen in Figure 3, qualitatively, the pattern remained the same: the ASD group was found to be less similar to the NT reference group than the NT group. However, the T-statistics plummeted both at the voxel level (Fig. 3b) and at the network (Fig. 3c) level, where few significant differences between the groups remained (LimbicA and DefaultB, both q < 0.05). This shows that the previous result of low within-group similarity in ASD cannot be explained by all ASD subjects having low similarity to NT subjects, but instead point to the possibility that this effect might be driven by a few specific subjects.

Figure 3.

Group-averaged inter-SC to a group of five reference NT subjects, while watching an episode of the American sitcom “The Office”. a, Voxelwise inter-SC to the reference subjects averaged across 20 NT subjects (top) and across 17 ASD subjects (bottom), projected on the inflated brain using Caret's Conte69 atlas. Note that the highest correlation values are >0.3; however, the scale is limited to ±0.1 for visualization purposes. b, Voxelwise T-values for the difference in within-group inter-SC between 20 NT subjects and 17 ASD subjects; both a loosely thresholded (p < 0.001 uncorrected) and an unthresholded map are shown to represent available information as faithfully as possible. c, Inter-SC to the reference subjects for 20 NT (blue) and 17 ASD (red), based on time courses in 17 functional networks. Error bars are 95% confidence interval based on a bootstrap estimate of the mean of five subjects. The shaded areas around zero represent the 95% confidence interval for the distribution of group inter-SCs that would arise by chance, for each group. Statistics for network analyses are FDR-controlled (*q < 0.05, +q < 0.1). See Figure 1b for the networks.

A few ASD subjects (5 of 17) are distinct from all other subjects

The logical next step is to focus on individual data, to gain better insight on what causes the low within-group inter-SC in the ASD group. We focus on network-level descriptions, for more reliable estimates of inter-SC between subjects and a lower-dimensional space to investigate than voxelwise analyses.

We first applied a OC-SVM classification approach (Mourão-Miranda et al., 2011) to our data, reasoning that despite the absence of significant differences within individual networks (Fig. 3c), a combination of weak but reliable differences across networks in inter-SC to the reference group may still be diagnostic of ASD. An OC-SVM classifier can be qualitatively seen as an outlier detector: it builds a statistical model of the class that it is trained on (here, a subset of NT subjects at each cross-validation fold), and then predicts whether a test subject should be considered as belonging to the same class or not. Here, classification accuracy was at chance, with sensitivity (true-positives; i.e., ASD classified as ASD) significantly above chance and specificity (true-negatives; i.e., NT classified as NT) significantly below chance. For more insight, we plotted the average decision values (over the 1000 random picks of reference subjects) output by the classifier for each subject, which roughly represent the confidence that the classifier has in sorting each subject into an outlier (decision value <0) or not (decision value >0). Five ASD subjects stood out with very negative decision values while other ASD subjects had values that did not differ from the values of the NT subjects (Fig. 4a). Convergent results were obtained using nonmetric MDS on the pairwise dissimilarities between subjects, averaged over the 17 networks (Fig. 4b), and k-means clustering on the dissimilarities of each subject to the reference group over each network (Fig. 4c): the same five ASD subjects were identified as distinct from all other subjects, while the remaining ASD subjects group together with NT subjects.

Figure 4.

Five ASD subjects are neurally distinct from the rest of the subjects. a, Decision values for OC-SVM based on average inter-SC to the control reference within each of 17 networks. b, Two-dimensional MDS solution based on the average pairwise inter-SC across all 17 networks. Red dots for ASD, blue dots for NT. Larger dots represent a better fit (lower stress-per-point). The blue dotted ellipses represent one to five SDs of the two-dimensional Gaussian distribution estimated from the NT subjects. Five ASD subjects lie well outside the five SD contour line, whereas the rest of the ASDs largely lie within normal range. c, k-means clustering solution based on the inter-SC of each subject to a reference group of five NT (averaged over 1000 random draws) in each network, using a squared Euclidean distance measure. The best partition was into two clusters, with one cluster comprising five ASD subjects with high silhouette values, the same ones that were identified with OC-SVM in a and MDS in b.

These five ASD subjects were reducing the ASD group's within-group inter-SC as well as inter-SC to a reference group. If we exclude them from these previous analyses, almost no significant differences remain between the NT group and the ASD group, except in SomMotA and DefaultB for within-group inter-SC where NT > ASD (q < 0.05; Fig. 5a,b). Henceforth, we will thus examine the data from these five ASD subjects (who comprise nearly one third of the ASD group) as distinct from the remaining ASD individuals.

Figure 5.

Characterization of the idiosyncratic brain activity of the five distinct ASD subjects. a, Within-group inter-SC after exclusion of the five identified ASD subjects, with these five subjects treated as their own subgroup (in pink; NT, blue; remaining ASD, red). Error bars are 95% confidence interval based on a bootstrap estimate of the mean of five subjects. The shaded areas around zero represent the 95% confidence interval for the distribution of group inter-SCs that would arise by chance, for each group. Significance for the difference between the ASD and NT groups is denoted with blue or red symbols, depending on the sign of the difference (FDR-controlled: *q < 0.05, +q < 0.1). b, Group-averaged inter-SC to a reference group of five NT subjects, after exclusion of the five distinct ASD subjects, with these five subjects treated as their own subgroup (in pink; NT, blue; remaining ASD, red). Error bars are 95% confidence interval based on a bootstrap estimate of the mean of five subjects. The shaded areas around zero represent the 95% confidence interval for the distribution of group inter-SCs that would arise by chance, for each group. No significant difference between the NT and ASD groups was found. In pink, significance for the difference between the five identified ASD subjects and the remaining subjects (pooling NT and remaining ASD together), with Cohen's d effect size; statistics are FDR-controlled (***q < 0.001, **q < 0.01, *q < 0.05, +q < 0.1). c, k-means clustering solution based on the inter-SC of each subject to a reference group of five NT (averaged over 1000 random draws) in each of the 17 functional networks, using a Pearson-correlation based distance measure. The best partition is into two clusters (data not shown), with one cluster comprising four of the five previously identified ASD subjects. d, Inter-SC to a group of 5 reference NT subjects shown for individual subjects. NT subjects are in blue, ASD subjects in red with the exception of the five distinct ASD subjects who are in pink (4/5) and green (A01, whose spatial pattern of idiosyncrasy did not cluster with that of the other idiosyncratic ASD subjects). The shaded area represents the 95% confidence interval for the distribution of individual inter-SCs that would arise by chance. See Figure 1b for the networks.

The five distinct ASD subjects have idiosyncratic brain activity especially pronounced in some higher level networks

The next question is whether the five distinct ASD subjects are similar to one other, or present individually idiosyncratic brain patterns. One initial clue is already present in the two-dimensional projection found by the MDS algorithm (Fig. 4b), in which these five subjects are as distant from one another as they are distant from the rest of the group, pointing to the temporal idiosyncrasy of their brain responses on average across all networks. Idiosyncrasy among these five subjects was also confirmed within individual networks: inter-SC among them exceeded chance for only four networks (VisCent, VisPeri, SomMotB/Auditory, and DefaultD; Fig. 5a), and was lower than inter-SC from these subjects to the reference group (Fig. 5b). Thus, no common temporal response pattern distinguishes them as a subgroup. Yet, despite this idiosyncrasy, these subjects' brains still show systematic activations evoked by the common stimulus: the mean inter-SC values between these five subjects and a reference group of five NT exceeded chance for nearly all networks (Fig. 5b; all q < 0.05 except for LimbicA, LimbicB).

Though the temporal pattern of brain activity is not consistent across these five ASD subjects, we wondered whether their brain activity nonetheless most differed from the rest of the subjects within a restricted subset of networks. Although inter-SC to the reference group for these subjects was numerically lower for all networks, the magnitude of the difference between these five subjects and the remaining subjects varied across networks (Fig. 5b). Default networks were most dramatically affected; inter-SC was also considerably lower in Control networks and SalVentAttnA. Using k-means clustering as before but with a correlation-based similarity measure to capture spatial patterns of idiosyncrasy, we found that the best partition for our whole set of subjects was into two groups, with one group comprising four of the five distinct ASD subjects (with high silhouette values; Fig. 5c). This is indicative of a consistent spatial pattern of idiosyncrasy for these four subjects. For these four subjects, while most networks are affected, we notice strikingly low inter-SC in Default networks A–C, as well as Control networks A and B. The remaining distinct ASD subject who did not cluster with the other four also displayed reduced inter-SC in Default and Control networks, as well as across additional networks including lower-level visual networks (Fig. 5d), perhaps suggestive of a partly different etiology of the low inter-SC for this subject.

The five ASD subjects with idiosyncratic brain activity also have unreliable brain activity (in the same set of higher-level networks)

We repeated the first half of the sitcom in the scanner (for all but three subjects, including one of the five idiosyncratic ASD subjects) to investigate previous conflicting reports of differences in test–retest reliability (or intra-subject correlation; intra-SC) between ASD and NT groups (Hasson et al., 2009; Dinstein et al., 2012) in our sample. We correlated the BOLD signal in each functional network between run 1 and run 2 for Part 1 within each available subject, shown in Figure 6. No significant differences were found between the groups (NT vs all ASD; via FDR-controlled t tests; Fig. 6a), even when the distinct ASD subjects were excluded (Fig. 6b). We observed that intra-SC was very low across the distinct ASD subjects relative to the NT and remaining ASD groups for the same networks identified previously, reaching significance in two of those networks, ContC and DefaultC (Fig. 6b).

Figure 6.

Intra-SC for clip1 (test–retest) based on the BOLD response in each of the 17 functional networks. a, No significant difference between the NT (blue) and complete ASD (red) groups. Error bars are 95% confidence interval based on a bootstrap estimate of the mean of five subjects. b,The distinct ASD subjects (average in pink) have low intra-SC. However, after their exclusion from the ASD group, we do not find any significant differences between the ASD and NT groups. Error bars are 95% confidence interval based on a bootstrap estimate of the mean of five subjects. In pink, statistics and effect sizes (Cohen's d) reflect the difference between the identified ASD subjects and the remaining subjects (NT+ASD). Statistics are FDR-controlled (*q < 0.05, +q < 0.1). For both plots, shaded areas around zero represent the 95% confidence interval for the distribution of group intra-SCs that would arise by chance, for each group. See Figure 1b for the networks.

Responses that are internally unreliable seem likely to also be idiosyncratic relative to others. To investigate this, we examined the relationship between intra-SC (as before, between run1 and run2 for part1) and inter-SC for part2 run1 only (for independence). Within all individuals, mean intra-SC and mean inter-SC were positively correlated (r(35) = 0.51, p = 0.0022), but this relationship was driven by the distinct ASD subjects (when excluded, p > 0.25). In the distinct ASD subjects, intra-SC and inter-SC appeared to be much lower in similar networks. Confirming this, we found that within each network, the size of the differences (Cohen's d) between the distinct ASD subjects and the remaining subjects for intra-SC and for inter-SC were positively correlated (r(15) = 0.56, p = 0.017).

The five distinct ASD subjects do not all stand out in terms of motion or scanner artifacts

To rule out simple confounds that could have contributed to making the brain activity of the distinct ASD subjects look different from the other subjects, we examined how nine measurements of MRI data quality in these five subjects compared with the remaining subjects. Specifically, we examined motion parameters (mean, SD, and maximum frame-by-frame displacement; Power et al., 2012), change in BOLD signal intensity (DVARS; Power et al., 2012), volumes censored due to high DVARS, and five summary statistics describing affectedness by a stability problem with 2/32 head coil elements (which was addressed during preprocessing; Table 2). Although some of the distinct ASD subjects were among the subjects that moved the most, not all of them were, allowing us to discard a simple motion confound (note also that motion was very low in the preprocessed data due to censoring of data points with high DVARS; mean FD < 0.2 mm, peak FD < 0.8 mm). Further, the five distinct ASD subjects were within the normal range of affectedness by the stability problem (Table 2). In sum, neither the low intra-SC nor low inter-SC observed in the outlier ASD subjects can be simply explained by motion confounds or scanner artifacts.

Table 2.

The ASD subjects with idiosyncratic and unreliable brain responses do not all stand out in terms of motion or system artifacts

	FD			Dvars		Nyquist
	Mean (mm)	Sd (mm)	Max (mm)	Fcensor (frac)	Mean (a.u.)	Flicker (a.u.)	Avmax (a.u.)	Avmean (a.u.)	Fcensor (frac)
A01
Part 1 run 1 [37]	0.09 (1.09) [5]	0.05 (0.58) [8]	0.32 (0.51) [13]	0.10 (1.59) [4]	37.56 (0.84) [1]	30.60 (0.19) [7]	2.13 (0.17) [11]	1.14 (0.04) [8]	0.63 (2.70) [1]
Part 2 run 1 [37]	0.09 (0.62) [11]	0.04 (−0.00) [19]	0.22 (−0.41) [28]	0.07 (0.23) [7]	39.31 (1.43) [1]	23.18 (−0.15) [6]	1.71 (−0.09) [6]	1.09 (−0.16) [10]	0.21 (0.52) [8]
Part 1 run 2 [33]	0.10 (0.59) [14]	0.06 (0.78) [14]	0.50 (1.35) [10]	0.06 (−0.46) [10]	38.90 (0.79) [9]	20.42 (−0.25) [6]	1.70 (−0.20) [6]	1.12 (−0.18) [6]	0.04 (−0.57) [16]
A09
Part 1 run 1 [37]	0.06 (−0.75) [20]	0.04 (0.03) [17]	0.28 (0.05) [16]	0.02 (−0.45) [19]	32.89 (−0.70) [22]	13.98 (−0.32) [21]	1.38 (−0.42) [21]	1.08 (−0.32) [20]	0.04 (−0.49) [20]
Part 2 run 1 [37]	0.12 (2.35) [4]	0.09 (4.19) [1]	0.74 (4.86) [1]	0.11 (0.62) [2]	34.07 (−0.30) [18]	14.57 (−0.32) [11]	1.56 (−0.20) [11]	1.10 (−0.08) [3]	0.07 (−0.21) [11]
Part 1 run 2 [33]	0.12 (1.23) [9]	0.09 (2.55) [6]	0.62 (2.44) [7]	0.21 (1.32) [1]	37.46 (0.34) [12]	13.18 (−0.41) [14]	1.44 (−0.41) [11]	1.08 (−0.44) [12]	0.05 (−0.53) [11]
A14
Part 1 run 1 [37]	0.05 (−1.51) [29]	0.03 (−0.49) [25]	0.26 (−0.24) [23]	0.04 (0.14) [14]	34.86 (−0.05) [9]	33.21 (0.27) [5]	2.15 (0.19) [9]	1.13 (−0.07) [11]	0.25 (0.65) [15]
Part 2 run 1 [37]	0.05 (−1.23) [31]	0.04 (−0.15) [22]	0.26 (0.01) [24]	0.01 (−0.42) [20]	33.95 (−0.34) [21]	14.83 (−0.31) [9]	1.35 (−0.36) [15]	1.06 (−0.36) [19]	0.04 (−0.39) [14]
Part 1 run 2 [33]	0.05 (−1.56) [29]	0.03 (−1.21) [31]	0.25 (−0.96) [30]	0.02 (−0.95) [17]	33.34 (−0.95) [18]	10.23 (−0.47) [19]	1.40 (−0.45) [17]	1.07 (−0.51) [19]	0.00 (−0.69) [26]
A16
Part 1 run 1 [37]	0.11 (2.33) [1]	0.06 (1.92) [2]	0.40 (1.43) [5]	0.02 (−0.45) [19]	33.20 (−0.59) [19]	15.31 (−0.28) [16]	1.37 (−0.43) [24]	1.07 (−0.38) [24]	0.02 (−0.55) [25]
Part 2 run 1 [37]	0.13 (3.11) [1]	0.07 (2.68) [7]	0.42 (1.57) [9]	0.04 (−0.07) [11]	34.94 (−0.01) [11]	12.67 (−0.35) [15]	1.32 (−0.39) [17]	1.07 (−0.29) [15]	0.00 (−0.58) [21]
Part 1 run 2 [33]	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
A17
Part 1 run 1 [37]	0.09 (0.80) [9]	0.07 (2.51) [1]	0.52 (2.91) [1]	0.04 (0.23) [13]	33.44 (−0.51) [15]	26.60 (0.07) [10]	2.06 (0.12) [16]	1.18 (0.22) [7]	0.37 (1.28) [12]
Part 2 run 1 [37]	0.11 (1.84) [7]	0.08 (3.18) [4]	0.52 (2.64) [6]	0.12 (0.84) [1]	33.68 (−0.43) [23]	12.43 (−0.36) [17]	1.43 (−0.30) [14]	1.10 (−0.14) [8]	0.04 (−0.41) [18]
Part 1 run 2 [33]	0.09 (−0.09) [21]	0.07 (1.06) [13]	0.50 (1.31) [13]	0.16 (0.68) [5]	32.44 (−1.23) [25]	13.03 (−0.41) [17]	1.34 (−0.49) [21]	1.07 (−0.47) [17]	0.06 (−0.48) [9]

Motion and system artifacts for these ASD subjects do not consistently depart from the distribution over the rest of the subjects, ruling out simple confounds for the idiosyncratic and unreliable brain responses observed. In each cell, we give the numerical value for the artifact, then the (Z-score) with respect to the NT participants, then the [rank] with respect to the entire subject sample (all subjects; n = 37, except for Part 1 run 2, where n = 33). Values departing by more than 2 SD are underlined to help visualization. Measurements are defined as follows. We analyzed frame-by-frame displacement (FD; Power et al., 2012) of the motion parameters provided by the motion-correction procedure (applied as a preprocessing step to the EPI image series for each subject and run); specifically the mean (mean), standard deviation (sd) and maximum (max) FD values across non-censored time points (i.e. the analyzed time points, after censoring volumes with excessive DVARS). DVARS is a metric that indexes the rate of change of the BOLD signal across the brain, at each time point. At the preprocessing stage, we censored time points at which DVARS >50, following current guidelines (motion scrubbing; Power et al., 2012). We examined the fraction of censored volumes (fcensor) for each subject and the average DVARS value across the non-censored volumes (mean); both values for the distinct ASD subjects lay within the range spanned by other subjects. Finally (see Materials and Methods, MRI Data Preprocessing), our dataset was intermittently affected by a stability problem with two of the 32 elements of the head coil; we dealt with this issue at the preprocessing stage using slice-specific nuisance regressors. We computed various summary statistics on these nuisance regressors (flicker: average over slices of the cumulated fluctuations in each slice; avmax: mean over time of max value over all slices; avmean: mean over time of mean value over all slices; fcensor: fraction of time points with at least one slice above an arbitrary threshold of 2), and verified that the ASD subjects with idiosyncratic responses were within the normal range of affectedness.

The distinct ASD subjects have normal gaze behavior

We reviewed videos of subjects' right eye (videos were missing for 5 of 37 subjects, including one of the five idiosyncratic ASD subjects, for technical reasons); all subjects had their eyes open during all fMRI runs. We also analyzed blink rate, longest blink and cumulated blink duration (for all detected blinks); no outlier departed from the distribution of subjects on any of these measures.

To further examine looking behavior, we quantified the similarity of each ASD subject's eye gaze pattern to the mean eye-gaze of an NT reference group (n = 32, including all of the current NT participants and an additional 12 NT participants) while watching the first 6 min 58 s of a different episode of the same sitcom outside the scanner. Due to poor calibration, eye-tracking data were unusable for four NT and five ASD subjects, including two of the idiosyncratic ASD subjects (see Materials and Methods, Analysis of eye tracking). Similarity to the NT reference in the three available distinct ASD subjects' eye-gaze patterns (values: 4.5, 5.4, 5.4; mean = 5.1, SD = 0.53) was well within the range of the other ASD subjects (range: 4.3–6.13, mean = 5.3, SD = 0.69).

The five distinct ASD subjects are representative of the ASD population studied in our laboratory

Our laboratory regularly works with a pool of ∼50 well characterized high-functioning ASD subjects. We were interested in whether the sample of ASD subjects who participated in the current study differed from the rest of the group in terms of neuropsychological scores. We compared algorithm scores on the ADOS; we also compared Autism Quotient (AQ) and Social Responsiveness Scale (SRS) scores, as well as Performance, Verbal and Full-Scale Intellectual Quotient (PIQ, VIQ, and FSIQ), and finally scores on the Beck Depression Inventory (BDI), the Positive And Negative Affect Schedule (PANAS) and the State-Trait Anxiety Inventory (STAI). We find that the five distinct ASD subjects did not differ consistently from the larger population of ASD subjects that we work with in the laboratory on any of these measures (Table 3).

Table 3.

The ASD subjects with idiosyncratic responses are representative of the ASD population studied in our laboratory

	ASD symptoms									Cognitive			Mood
	ADOS				ADOS2		AQ	SRS		IQ			BDI	PANAS		STAI
	Comm [50]	SI [50]	Creativ [49]	SB [50]	RRB [44]	SA [44]	[33]	SRS-OR [30]	SRS-SR [37]	P [39]	V [39]	FS [39]	[48]	Neg [47]	Pos [47]	Trait [47]
A01	6.00 (1.11) [5]	14.00 (2.29) [1]	2.00 (1.59) [1]	2.00 (0.56) [15]	4.00 (0.45) [11]	20.00 (1.99) [1]	N/A	88.00 (−0.24) [17]	N/A	108.00 (0.38) [19]	80.00 (−1.79) [38]	93.00 (−0.80) [35]	8.00 (0.00) [14]	14.00 (−0.36) [21]	26.00 (−1.02) [37]	44.00 (0.10) [21]
A09	3.00 (−0.61) [31]	7.00 (−0.63) [30]	1.00 (0.14) [8]	0.00 (−1.20) [35]	2.00 (−0.91) [33]	10.00 (−0.49) [25]	34.00 (0.59) [8]	103.00 (0.20) [12]	73.00 (−0.11) [20]	99.00 (−0.26) [25]	89.00 (−1.16) [34]	93.00 (−0.80) [35]	9.00 (−0.13) [11]	11.00 (−0.77) [32]	28.00 (−0.77) [35]	45.00 (0.18) [20]
A14	2.00 (−1.18) [42]	5.00 (−1.46) [47]	0.00 (−1.32) [36]	0.00 (−1.20) [35]	2.00 (−0.91) [33]	8.00 (−0.98) [35]	26.00 (−0.28) [22]	57.00 (−1.15) [27]	101.00 (1.07) [6]	121.00 (1.30) [5]	129.00 (1.66) [5]	128.00 (2.01) [3]	5.00 (−0.39) [23]	14.00 (−0.36) [21]	34.00 (−0.03) [25]	39.00 (−0.33) [25]
A16	2.00 (−1.18) [42]	6.00 (−1.05) [26]	1.00 (0.14) [8]	4.00 (2.31) [1]	7.00 (2.49) [1]	10.00 (−0.49) [25]	37.00 (0.92) [5]	73.00 (−0.68) [21]	85.00 (0.40) [14]	103.00 (0.03) [20]	115.00 (0.68) [14]	109.00 (0.48) [12]	8.00 (0.00) [14]	15.00 (−0.22) [15]	29.00 (−0.65) [32]	56.00 (1.12) [7]
A17	6.00 (1.11) [5]	13.00 (1.87) [5]	1.00 (0.14) [8]	0.00 (−1.20) [35]	3.00 (−0.23) [18]	19.00 (1.74) [5]	26.00 (−0.28) [22]	98.00 (0.06) [14]	71.00 (−0.19) [21]	103.00 (0.03) [20]	109.00 (0.25) [20]	107.00 (0.32) [15]	6.00 (−0.26) [20]	10.00 (−0.91) [36]	46.00 (1.46) [4]	46.00 (0.27) [17]

Comparison of neuropsychological variables (related to ASD symptoms, cognitive functioning and mood disorders) to the pool of 50 ASD subjects in our laboratory. In each cell, we give the value, then the (Z-score) with respect to the non-tested ASD participants (n = 33), then the [rank] with respect to the entire subject sample (all non-tested and tested; n = 50, except where values are missing; the size of the sample is indicated below each score). Values departing by more than 2 SD are underlined to help visualization. ADOS, Autism Diagnostic Observation Schedule original algorithm; Communication, Comm; Social Interaction, SI; Creativity, Creativ; Stereotyped Behavior, SB; ADOS2, Autism Diagnostic Observation Schedule revised algorithm (Hus and Lord, 2014); Restrictive and Repetitive Behavior , RRB; Social Affect, SA; AQ, Autism Quotient; SRS, Social Responsiveness Scale, either using responses of a close family member (SRS-OR) or a self-report (SRS-SR); IQ, Intellectual Quotient: Performance (P), Verbal (V), and Full-Scale (FS); BDI, Beck Depression Inventory; PANAS, Positive And Negative Affect Scale: Negative (Neg) and Positive (Pos); STAI, State Trait Anxiety Inventory.

The distinct ASD subjects have a low understanding of the sitcom characters' motivations

After MRI acquisition, 14 of 17 ASD (including 3 of 5 idiosyncratic subjects) and 19 of the 20 NT subjects answered open-ended social comprehension questions while watching a separate episode of the same sitcom. We found that comprehension scores for these three distinct ASD individuals were lower than the remaining ASD individuals (Fig. 7a, circles). If idiosyncratic brain responses are related to low social comprehension, we predicted that the remaining 2 of 5 distinct ASD subjects who did not initially perform this task (due to fatigue and time constraints) should also score poorly on these questions. Thus, we invited all subjects who had not initially participated back to perform this task, approximately 2 years after the imaging session. The pattern was confirmed: social comprehension was also low in the remaining two distinct ASD subjects (Fig. 7a). Social comprehension in the remaining ASD subjects was also lower than NT subjects (Fig. 7a), but more subtly lower, with performance in both groups largely overlapping.

Figure 7.

Social comprehension scores for a different episode of the same sitcom. Initial data acquisition (circles); data acquired 2 years later (squares). In pink, the ASD subjects with idiosyncratic brain responses; in red, the remaining ASD subjects; in blue, the NT subjects. a, Overall scores. b, Scores on questions querying character's motivations. c, Scores on questions querying characters' thoughts and feelings. Statistics are FDR-controlled (**q < 0.01, *q < 0.05, +q < 0.1).

To better understand the comprehension deficits displayed by the five distinct ASD subjects, we categorized the questions. Most fell into two categories: 13/39 questions about what a character was feeling or thinking, and 18/39 questions about the motivations for a character's behavior. Eight miscellaneous questions did not fit neatly into categories and are not considered further. Low comprehension scores for the five distinct ASD subjects were driven disproportionately by low understanding of social motivations (Fig. 7b), whereas comprehension of thoughts and feelings was relatively spared (Fig. 7c). Understanding of social motivations was also linked with inter-SC (to the reference group, averaged across 17 networks) across all subjects (r(35) = 0.79, p < 0.0001), though marginally and more weakly when the five distinct ASD subjects were excluded (r(30) = 0.30, p = 0.091).

Discussion

Given the well known high inter-subject variability in the ASD population (Müller et al., 2003; Humphreys et al., 2008; Towgood et al., 2009), a key goal is to “parse the heterogeneity” by determining how such variation is linked across behavioral, phenotypic, and neural dimensions (Cuthbert and Insel, 2013). Here, we examined heterogeneity of spatiotemporal brain response patterns during free viewing of a sitcom depicting seminaturalistic social interactions. We found that the video engaged much of the brain similarly across subjects; however, a subset of five ASD individuals demonstrated strikingly divergent brain response patterns. Each of these five individuals presented temporal patterns of brain activity that were distinct from all other subjects (as indicated by several approaches, including one-class support-vector machine, multidimensional scaling, and k-means clustering). These individuals were characterized behaviorally by low social comprehension, despite being otherwise phenotypically indistinguishable from other high-functioning ASD individuals within this study, and within the larger sample of laboratory participants from which they were randomly selected. Brain responses in these individuals were found to be most idiosyncratic in the Default network. An important source of idiosyncrasy for these individuals was the unreliability of their own brain responses: test–retest reliability (the correlation between an individual's brain responses during the first and second presentation of the same video) was strikingly low. The response patterns of the remaining ASD individuals were indistinguishable from NT individuals, using the same analyses.

The question may arise of whether our results are contingent on the particular parcellation of the brain that we used in our analyses (based on resting-state-derived functional networks). For instance, would we come to the same conclusions using a simple anatomical parcellation of the cortex? We repeated our analyses using the Harvard-Oxford (HO) cortical atlas (distributed with FSL, and provided by the Harvard Center for Morphometric Analysis), which comprises 48 bilateral cortical areas. With this parcellation, we replicated the main results almost exactly- the multivariate analyses pulled out the same five ASD subjects. The only result that differed substantially was the analysis of the spatial pattern of inter-SC: there, with the HO parcellation, the k-means clustering algorithm with a correlation distance measure did not reveal a consistent spatial pattern for the distinct ASD subjects. We surmise that this is due to the larger number of regions of interest (48 vs 17), which drowns patterns found to be consistent at a more global scale. All together, we can confidently report that our results are robust to the choice of parcellation scheme.

Aspects of our results are consistent with previous studies of NT and ASD individuals freely viewing videos (Hasson et al., 2009; Salmi et al., 2013). Indeed, when we analyzed the data as these groups did, our findings were largely the same (Figs. 2, 3): inter-SC was reduced within the ASD group (Hasson et al., 2009; Salmi et al., 2013), particularly in brain regions comprising the Default network (Salmi et al., 2013); and inter-SC between ASD and NT individuals exceeded inter-SC among ASD individuals (Hasson et al., 2009). A critical advance of our research is to show that such group-level patterns can obscure striking heterogeneity of individual brain responses, which we moreover found to be behaviorally relevant. Contrary to what might have been inferred from the group-level results, decreased inter-SC within the ASD group did not reflect idiosyncratic responding in all ASD individuals; rather, brain responses of most ASD individuals were largely indistinguishable from NT individuals. The implication of these findings is that combining ASD individuals within one single group may not be justified (Towgood et al., 2009; Brock, 2011; Castles et al., 2014), converging with extensive evidence that in etiology, development, and neuroanatomy, ASD is not a singular condition (Amaral et al., 2008; Geschwind, 2009; Hallmayer et al., 2011; Nordahl et al., 2011; Landa et al., 2012). Indeed, such group averaging could be contributing to discrepant and null group-level findings in the ASD literature (e.g., in neuroanatomy, Haar et al., 2015; in resting state functional connectivity, Tyszka et al., 2014; Hahamy et al., 2015).

Brain responses in a subset of ASD individuals are idiosyncratic and unreliable

The subgroup of ASD individuals whose brain responses were distinguishable from NT individuals were distinguished on the basis of very low inter-SC with all other subjects, including with one another: there was no common temporal response pattern among these individuals to suggest that they responded in a similar manner to some aspects of the stimulus. However, there was spatial convergence in the brain networks in which these individuals were most affected. In all these individuals, the same functional systems—Default, Control, and part of Salience/Ventral Attention networks—were dominated by idiosyncratic and unreliable responses, whereas idiosyncrasy and unreliability were attenuated in early sensory networks (Fig. 5b,d, see effect sizes). These individuals diverged most in the Default network (Fig. 5b), consistent with Default network recruitment in social understanding and perception in task-based designs (Andrews-Hanna, 2012; Kennedy and Adolphs, 2012; see also Salmi et al., 2013; Pantelis et al., 2015) and atypical Default network activation during tasks with social demands in ASD (Di Martino et al., 2009; Redcay et al., 2013). Of note, though, here we find idiosyncratic and unreliable Default network responding only in this subset of ASD individuals.

The idiosyncratic brain response profiles in these ASD individuals suggest two possibilities, which are not mutually exclusive. First, that in these individuals, the video largely fails to engage higher-order networks, whose activity instead might predominantly reflect endogenous oscillations independent from the video (consistent, e.g., with reduced task-modulation of endogenous brain activity in ASD; Uddin et al., 2014). Second, that higher-order networks in these individuals are driven exogenously by the video but in different ways from other subjects, potentially due to differences in interpretation and understanding.

Brain responses of NT and most ASD individuals are indistinguishable

Notably, the whole brain response profiles for the remaining ASD individuals were indistinguishable from NT individuals, as studied here with inter-SC. Although it is expected that differences between two groups would decrease when extreme values are removed, the fact that the group differences are entirely eliminated when the ASD individuals with idiosyncratic response profiles (who consistently cluster together) are excluded testifies to the large degree of overlap in the initial, complete distributions of ASD and NT inter-SC. Differences between these remaining ASD and NT individuals could potentially emerge with increased statistical power (longer scanning duration, as in Salmi et al., 2013; or additional subjects), or more sensitive analytical techniques examining brain responses within smaller intervals. However, in this dataset, the NT subjects and the remaining ASD individuals could not be separated with any of the multivariate approaches used, even when the clustering algorithm was forced to divide the whole set of subjects into three groups, or when the subgroup of distinct ASD individuals was excluded.

Noisy, unreliable brain responses may not be characteristic of all ASD subjects

Previous reports of test–retest reliability of brain responses in ASD have conflicted, finding less internally reliable responses to basic sensory stimuli (Milne, 2011; Dinstein et al., 2012), but more reliable responses evoked by a video (Hasson et al., 2009), both relative to controls. Our findings are not perfectly consistent with either pattern: we find less internally reliable responding in some ASD individuals (those identified previously), but predominantly outside the sensory areas previously implicated and not in most ASD individuals, and no evidence for increased reliability in the ASD group (even when the idiosyncratic individuals were excluded). Together, these mixed findings implicate heterogeneity in the ASD population sampled, which indicates that internal reliability of brain responses may vary across ASD individuals.

Heterogeneity in brain responses and heterogeneity in social comprehension are linked

Our findings link variation in social comprehension within ASD to variation in spatiotemporal patterns of brain activity during free viewing of naturalistic social interactions. We identify two distinct subgroups across brain and behavioral data. Both subgroups exhibited atypical social behavior (ASD diagnosis). One subgroup demonstrated subtly impaired (but largely within typical range) social comprehension with indistinguishable brain activity; the other demonstrated profoundly impaired social comprehension with idiosyncratic and unreliable brain activity (measured hours and even years apart, suggesting a stable trait). Naturalistic videos may have been key in separating out these subgroups, because early differences in processing or interpretation may snowball into highly idiosyncratic time courses (whereas the standalone trials used in conventional neuroimaging designs eliminate these cascading effects). These results suggest distinct profiles of impairment within these otherwise indistinguishable high-functioning ASD individuals. In some individuals, atypical processing of social “inputs” may coexist with (and, potentially, underlie) atypical production of social “outputs” (e.g., social responses), whereas in other individuals, processing of social inputs may be typical and largely independent from atypical social outputs. If so, differences from NT in the second, larger, ASD subgroup—with potentially distinct neurobiological correlates—might be more clearly revealed in studies that take a further step toward naturalism by requiring social responses during scanning (Redcay et al., 2010, 2013; Schilbach et al., 2013; Caruana et al., 2015).

Questions assessing motivations were specifically impaired in the ASD subgroup (consistent with Heavey et al., 2000), which could reflect question content or other factors such as how long information must be integrated for a correct response (see also Gepner and Féron, 2009). Although performance on these social motivation questions and inter-SC were (marginally) linked across all subjects, the extreme variation in naturalistic processing of social interactions, spanning brain and behavior, was specific to the ASD group, and thus calls for further investigation.

Conclusion

In conclusion, during free viewing of seminaturalistic social interactions, we find that no single pattern characterizes ASD individuals homogeneously. Although inter-SC among ASD individuals was reduced at the group level (consistent with Hasson et al., 2009; Salmi et al., 2013), this group-level pattern obscured two distinct profiles of ASD individuals: brain response profiles indistinguishable from controls accompanied by subtle social comprehension deficits, and highly idiosyncratic and unreliable brain responses accompanied by profoundly impaired social comprehension. These distinct profiles underscore the broader point (see also Brock, 2011; Castles et al., 2014) that ASD individuals do not comprise one homogeneous group and that analysis techniques should be sensitive to this possibility.