Response inhibition and serotonin in autism: a functional MRI study using acute tryptophan depletion

Stereotyped, repetitive behaviours in autism may reflect deficits in serotonin-modulated inhibitory control. Daly et al. use fMRI to compare the effects of acute tryptophan depletion in adult males with autism and controls performing the Go/No-Go task. Opposite effects are seen in the two groups, consistent with altered inhibition in autism.

Abstract

It has been suggested that the restricted, stereotyped and repetitive behaviours typically found in autism are underpinned by deficits of inhibitory control. The biological basis of this is unknown but may include differences in the modulatory role of neurotransmitters, such as serotonin, which are implicated in the condition. However, this has never been tested directly. We therefore assessed the modifying role of serotonin on inhibitory brain function during a Go/No-Go task in 14 adults with autism and normal intelligence and 14 control subjects that did not differ in gender, age and intelligence. We undertook a double-blind, placebo-controlled, crossover trial of acute tryptophan depletion using functional magnetic resonance imaging. Following sham, adults with autism relative to controls had reduced activation in key inhibitory regions of inferior frontal cortex and thalamus, but increased activation of caudate and cerebellum. However, brain activation was modulated in opposite ways by depletion in each group. Within autistic individuals depletion upregulated fronto-thalamic activations and downregulated striato-cerebellar activations toward control sham levels, completely ‘normalizing’ the fronto-cerebellar dysfunctions. The opposite pattern occurred in controls. Moreover, the severity of autism was related to the degree of differential modulation by depletion within frontal, striatal and thalamic regions. Our findings demonstrate that individuals with autism have abnormal inhibitory networks, and that serotonin has a differential, opposite, effect on them in adults with and without autism. Together these factors may partially explain the severity of autistic behaviours and/or provide a novel (tractable) treatment target.

Introduction

Autism spectrum disorder (ASD) is a highly genetic neurodevelopmental condition affecting >1% of the population (Autism and Developmental Disabilities Monitoring Network Surveillance Year, 2008 Principal Investigators, 2012). A core diagnostic feature of ASD is restricted, stereotyped and repetitive behaviours (RSRB) (WHO, 1992). There is substantial indirect evidence that these symptoms are underpinned by deficits in executive function and in particular inhibitory control (Hill, 2004).

The neuroanatomical systems proposed to be involved in the RSRB typically found in ASD include the inferior frontal and cingulate cortices, thalamus, and basal ganglia (McAlonan et al., 2002; Langen et al., 2012). These are also the key areas mediating motor response inhibition in the ‘neurotypical’ population (Chambers et al., 2009). Functional MRI studies of healthy adults have investigated brain activation during inhibitory control (using a Go/No-Go task) and have reported that the right frontal gyrus, left thalamus, and right cerebellum are associated with the inhibitory No-Go response (Liddle et al., 2001). Additionally, Go/No-Go and functional MRI studies of adults with ASD describe functional abnormalities within inhibition areas including greater activation in the left inferior frontal gyrus (Schmitz et al., 2006) and less activation in the right inferior frontal gyrus when compared with control subjects (Schmitz et al., 2006; Kana et al., 2007).

The biological basis for these differences is unknown, but may include modulatory effects of serotonin (5-hydroxytryptophan, 5-HT). In healthy populations, there is evidence that serotonin is involved in motor inhibition (Soubrié, 1986; Lucki, 1998; Robbins and Crockett, 2010) by suppressing behavioural responses. Studies combining functional MRI and the modulation of serotonin levels in healthy controls report brain activation changes in frontal cortex and striatum when performing the Go/No-Go inhibition task (Anderson et al., 2002; Del-Ben et al., 2005; Rubia et al., 2005; Vollm et al., 2006; Lamar et al., 2009).

When compared to ‘neurotypicals’, there is consistent evidence that individuals with ASD have abnormalities in the serotonergic system including physiology, neurobiology and genetics (Cook and Leventhal, 1996; Zafeiriou et al., 2009). For example, research indicates that a significant proportion of subjects with ASD may have hyperserotonaemia (Hranilovic et al., 2009). As well as detecting increased blood levels of serotonin in first-degree relatives of subjects with ASD (Piven et al., 1991; Leboyer et al., 1999), hyperserotonaemic parents of ASD subjects are reported to have higher ratings of repetitive behaviours (Cook et al., 1994). Additionally, the severity of repetitive behaviours in subjects with ASD are related to the sensitivity of the serotonin-1d receptor, as measured by growth hormone response to receptor agonist sumatriptan (Hollander et al., 2000)

In addition to neuroanatomical differences (McAlonan et al., 2002; Langen et al., 2011), there are preliminary functional neuroimaging reports that individuals with ASD have significant differences from control subjects in serotonin synthesis (Chugani et al., 1997); as well as reductions in serotonin-2A receptor and serotonin transporter binding in brain regions (including inferior prefrontal cortex) that mediate executive and inhibitory functioning (Murphy et al., 2006; Nakamura et al., 2010). There are also reports of significant associations between ASD and genetic polymorphisms for serotonin synthesis, transporters and receptors (Devlin et al., 2005; Anderson et al., 2009).

Brain levels of serotonin are dependent on the blood concentration of its chemical precursor tryptophan, which is in direct competition with other neutral amino acids, large and small, to cross the blood–brain barrier (Fernstrom, 1983). People with ASD have a significantly lower ratio of tryptophan to other amino acids compared to control subjects (D'Eufemia et al., 1995). There is, therefore, increasing indirect evidence that some abnormalities in brain function within ASD may be underpinned by serotonin abnormalities. This issue can be addressed using acute tryptophan depletion (ATD)—a non-invasive technique for reducing serotonin levels in the brain (Young et al., 1989; Young, 2013). Blood levels of tryptophan, and subsequently brain serotonin, are decreased by the consumption of an amino acid mixture lacking tryptophan as compared to their increase by consumption of a sham mixture containing tryptophan. In ‘neurotypicals’, pharmacological functional MRI studies of action restraint inhibition tasks using ATD have reported that serotonin reduces neural activation of inferior frontal and striatal inhibitory control regions and increases activation of cerebellum, temporal and parietal lobes (Rubia et al., 2005; Lamar et al., 2009). Also, we have previously reported that in ASD, tryptophan depletion differentially modulates brain activation during facial emotion processing (Daly et al., 2012).

However, to our knowledge, no one has investigated the modulatory effect of serotonin on inhibitory neural activity in individuals with ASD. Also nobody has related clinical severity of RSRB symptoms to differences in brain response to serotonin. Hence, the aim of this functional MRI study was to investigate the modulation of blood oxygen level-dependent (BOLD) response by ATD during performance of a Go/No-Go task in adults with and without ASD.

We hypothesized that in the non-ATD condition there would be significant group differences in activation of classical inhibitory control regions such that subjects with ASD compared to controls would show decreased activation in right inferior frontal and increased activation in left inferior frontal cortices (Schmitz et al., 2006; Kana et al., 2007). These differences would be modulated by serotonin reduction whereas the control subjects would show decreased right inferior frontal cortex and striatial activations (Rubia et al., 2005; Lamar et al., 2009). We further hypothesized that within the subjects with ASD, abnormal activation and differential serotonin modulation would both be related to severity of RSRB.

Materials and methods

Subjects

We recruited 14 right-handed adult males with autism through a clinical research programme at the Maudsley Hospital/King’s College London’s Institute of Psychiatry (London). An autism diagnosis was based on application of ICD-10 research criteria (WHO, 1993), and confirmed using the Autism Diagnostic Interview-Revised (ADI-R) (Lord et al., 1994). Subjects reached ADI-R algorithm cut-offs in the three domains of impaired reciprocal social interaction, communication and repetitive behaviours and stereotyped patterns, although failure to reach cut-off in one of the domains by 1 point was permitted. Current symptoms were assessed using the Autism Diagnostic Observational Schedule (ADOS) (Lord et al., 1989). We were unable to obtain the interview for two of the subjects whose diagnoses were confirmed using the observational schedule (Table 1).

Table 1.

Demographics and plasma tryptophan levels

	Control (n = 14)		Autism (n = 14)
Characteristics	Mean	SD	Mean	SD	t-Test P	ANOVA P
Age, years	31	11	31	13	0.9
Full-scale IQ	123	20	115	13	0.2
Autism Quotient	12	5	31	9	0.001
OCI-R	10	10	25	15	0.005
BDI	3	4	10	7	0.003
BAI	3	3	11	13	0.04
Autism Diagnostic Inventory-Revised			(n = 12)
Reciprocal Social Interaction			17	11
Communication			11	4
RSRB			5	2
Plasma tryptophan mmol/l	(n = 14)		(n = 14)
SHAM Pre	71	10	78	13	0.1
SHAM Post	140	54	190	77	0.08	0.005
ATD Pre	75	13	76	5	0.9
ATD Post	20	12	23	9	0.5	0.005

OCI-R = Obsessive-Compulsion Inventory-Revised; BDI = Becks Depression Inventory; BAI = Becks Anxiety Inventroy; SHAM = balanced amino acid drink; Pre = pre-drink; Post = 4.5 h after drink.

We also included 14 healthy, right-handed, adult male control subjects recruited locally by advertisement. They did not differ significantly from the males with autism in age (mean years: ASD = 31 ± 13; controls = 31 ± 11; t-test P = 0.9) or overall intelligence as measured by the Wechsler Adult Intelligence Scale (Wechsler, 1981) (mean full scale: ASD = 115 ± 13; controls = 123 ± 20; t-test P = 0.2) (Table 1).

All subjects had routine clinical and genetic screening blood tests to ensure good medical health and a structured clinical exam to exclude psychiatric illness (e.g. schizophrenia, major depression), head injury, genetic disorder associated with autism (e.g. fragile X syndrome, tuberous sclerosis), and neurological or medical disorders that might affect brain function (e.g. epilepsy or hypertension). None of the participants were abusing alcohol, taking any illicit drugs and none were prescribed antipsychotic medication, antidepressants, mood stabilizers or benzodiazepines. After receiving a description of the study all subjects gave written, informed consent. This study was approved by the Ethical Committee of the South London and Maudsley NHS Foundation Trust and the Institute of Psychiatry, King’s College London.

Depletion/sham procedure by acute tryptophan depletion

Subjects were tested on two separate occasions separated by 1 week (range: 0.6–3 weeks). After fasting from midnight, at ∼8:30 am (∼5 h before scanning), subjects drank a 100-g mixture that contained 15 amino acids but lacked aspartic and glutamic acid, to avoid possible toxicity (Young et al., 1989); and either contained 2.3 g of tryptophan for the sham (placebo) condition, or no tryptophan for the ATD condition. In a double-blind, counterbalanced, crossover design, subjects were assigned to order of drink consumption (i.e. within each group, half received sham first and half received ATD first). The amino acid recipe with or without tryptophan, included: L-alanine 5.5 g, L-arginine 4.9 g, L-cysteine 2.7 g, glycine 3.2 g, L-histidine 3.2 g, L-isoleucine 8.0 g, L-leucine 13.5 g, L-lysine monohydrochloride 8.9 g, L-methionine 3.0 g, L-phenylalanine 5.7 g, L-proline 12.2 g, L-serine 6.9 g, L-threonine 6.5 g, L-tyrosine 6.9 g and L-valine 8.9 g. The powder was mixed with cold water and a flavour of the participant’s choice, and then drunk over 10 min. See Young (2013) for an overview of the theory, practice and ethics of ATD method.

Monitoring of affective state and blood chemistry

Before the study, participants completed baseline self-assessment measurements of depression (Beck Depression Inventory; Beck and Steer, 1993), anxiety (Beck Anxiety Inventory; Beck et al., 1988) and autistic traits [Autism-Spectrum Quotient (Baron-Cohen et al., 2001) and Obsessive-Compulsive Inventory-Revised (Foa et al., 2002)].

Before consuming the amino acid mixture, a blood sample was taken to measure tryptophan levels. A second blood sample was taken immediately before scanning, typically 4.5 h after ingestion. This time frame is considered to be optimal for capitalizing on the effects of ATD on behavioural measures and plasma and brain 5-HT synthesis (Young et al., 1989). Total plasma tryptophan levels were determined from each blood sample using previously described methods (Keating et al., 1993).

Self-report visual analogue scale questionnaires administered before (baseline) and 4.5 h after (follow-up) amino acid drink ingestion measured various aspects of mood, aggression, and physical symptoms thought to be associated with ATD including nausea, dizziness, irritability and anxiety (Supplementary material).

Functional MRI

Activation paradigm: Go/No-Go task

Five hours after ingestion of the amino acid drink, each subject participated in the 6-min functional MRI event-related Go/No-Go task (Rubia et al., 2005). Before the functional MRI, the task was explained to and practiced by subjects on a laptop outside of the scanner. Within the scanner, the task was projected onto a screen and viewed by subjects through a prism mirror attached to the headcoil cage. Frequent arrows (160 trials: 76%, 500 ms duration) pointing to either the left or right (Go signals) appeared in the middle of the screen with a mean intertrial interval of 1.8 s (jittered 1.6–2s). Infrequently, arrows point up (24 trials, 12%, No-Go signals) or slightly slanted (by 22.5%) arrows (24 trials, 12%, oddball signals) appeared. A button response had to be selectively executed with the right thumb to Go or oddball stimuli or inhibited to No-Go signals. The oddball trials control for the low frequency of the No-Go trials and thus the oddball attentional capture effect (Rubia et al., 2005; Schmitz et al., 2006; Lamar et al., 2009).

A mixed between-within subjects ANOVA was performed in SPSS to examine the probability differences of inhibition and mean response to oddball Go trials.

Acquisition

Magnetic resonance images were acquired on a GE Signa 1.5 T Horizon LX system (General Electric) at the Maudsley Hospital, London, UK. A quadrature birdcage headcoil was used for radiofrequency transmission and reception. An inversion recovery echoplanar imaging (EPI) data set was acquired at 43 near-axial 3-mm thick slices parallel to the AC-PC line: echo time = 40 ms, repetition time = 16 s, in-plane voxel size = 1.875 mm, interslice gap = 0.3 mm, and matrix size = 128 × 128 voxel. This higher resolution EPI data set provided whole brain coverage and was later used to normalize the functional MRI data sets acquired from each individual in standard stereotactic space. Functional MRI acquisition consisted of 208 T₂*-weighted images acquired at each of 16 near-axial non-contiguous 7-mm thick planes parallel to the intercommissural (AC-PC) line: echo time = 40 ms, repetition time = 1.8 s, in-plane voxel size = 3.75 mm, interslice gap 0.7 mm, and matrix size = 64 × 64 voxels.

Data analysis

All functional MRI data were analysed with the XBAM (version 4) software developed at the Institute of Psychiatry, London, using a non-parametric approach (www.brainmap.co.uk) as described previously (Daly et al., 2012) and in the Supplementary material. A brief overview is given below.

Within each run, every volume was realigned to the mean of all the images in the run and then smoothed using a Gaussian filter (full-width at half-maximum 8.8 mm). Using a wavelet-based resampling method for functional MRI data, a time series analysis was performed on each individual subject, in order to compute a sum of squares ratio reflecting the BOLD effect. These individual maps were normalized into standard Talairach space using affine transformations. Group brain activation maps were computed for each experimental condition with hypothesis testing performed at both the voxel and the cluster level giving excellent type I error control. Using data-driven, permutation-based methods, with minimal distributional assumptions, we performed time series analyses for group maps and inter-group random permutation for within/between-group ANOVAs to compute the distribution of the sum of squares ratio under the relevant null distribution hypothesis. Thresholding to the required level of significance was then performed using a two-stage process: first at a voxel-wise P-value of 0.05, followed by grouping the supra-threshold voxels into 3D clusters and testing their significance against a null distribution of clusters occurring by chance in the permuted data. The cluster-level threshold was set in such a way as to obtain less than one false positive cluster per map. Therefore, a corrected threshold is used to minimize type 1 errors.

Analysis of variance and correlations

The event-related analyses contrasted the activations (sum of squares ratio) related to successful No-Go trials compared to successful oddball trials, controlling for the attentional oddball effect due to the low frequency occurrence of No-Go trials.

Analysis of variance testing for BOLD response interactions between tryptophan status and group membership

For the task, a two-group (between condition: control, autism) × two tryptophan status (within group: sham, depletion) factorial repeated-measures ANOVA was undertaken. Group × tryptophan status interactions refer to brain regions in which the effect on the BOLD response to the No-Go/oddball is different in each group depending on the status of with and without tryptophan. From each interaction cluster, for post hoc testing, each subject’s mean BOLD signal responses (sum of squares) were extracted.

Analysis of variance for BOLD response pair-wise

In the regions showing an interaction effect, main effect of group analyses were conducted for both the sham and depletion conditions. Also, a separate main effect of condition analysis was done for both the control and autism groups.

Analysis of variance testing for BOLD response ‘normalization’ by tryptophan depletion

To test whether brain activations that (i) differed between subjects with autism and controls; and (ii) showed an interaction between group × tryptophan status, were normalized after tryptophan depletion, we compared the between-group ANOVA between control subjects and the autism group following sham to the between-group ANOVA between control subjects at sham and autism group after tryptophan depletion.

Correlations between autism restricted, stereotyped and repetitive, and behaviours and functional MRI BOLD response of No-Go versus oddball contrast

For the autism group only, Pearson’s product moment correlation analyses were performed in XBAM to examine any associations between the severities of RSRBs (as measured by the relevant subscales of the ADI-R and ADOS) and the sum of squares ratios for the No-Go versus oddball contrast. This was done separately for the sham and ATD conditions, respectively. For each cluster showing a correlation, the sum of squares ratio was extracted and plotted versus RSRBs severity in SPSS. Additionally, within the autism group only, we extracted the sum of squares ratios from the group by drink interaction analysis to calculate an activation difference between ATD minus sham condition for each subject. Using SPSS, we calculated a Pearson’s product moment correlational analyses between these activation differences and the RSRBs severity scores. We employed a False Discovery Rate analysis to correct for multiple comparisons.

Results

Baseline measures

Control subjects scored within the normal range on all measures taken at baseline (Autism-Spectrum Quotient, Obsessive-Compulsive Inventory-Revised) and pre-drink (Beck Depression Inventory and Beck Anxiety Inventory). Subjects with autism scored significantly higher on all four measures compared to control subjects (for all t-tests P < 0.04) (Table 1).

Tryptophan blood levels

There were no significant differences in total plasma tryptophan concentration across groups on either day at baseline. After 4.5 h, consumption of the sham drink led to a significant increase of total plasma tryptophan concentrations, in both the controls and subjects with autism. Following the depletion drink, the total plasma tryptophan concentrations were significantly reduced in both groups (Table 1). As expected, in each group, there was a significant interaction between tryptophan status (sham versus depletion) and time (baseline versus 4.5 h) [controls: Wilks’ lambda = 0.04, F(3,11) = 80, P < 0.0005; autism: Wilk’s lambda = 0.03, F(3,11) = 140, P < 0.0005].

Affective measures

Despite between-group differences in baseline measures of depressive and anxiety symptomatology (Table 1), there were no significant between-group differences in subjective reports of adverse effects, including nausea and vomiting or for the affective visual analogue scale measures caused by consumption of the amino acid drink (Supplementary Table 1).

Functional MRI task response and movement measures

There were no group × drink differences in the x, y, or z movement parameters and none of the subjects exceeded maximum displacement >1 mm. There were no significant between-group or within-group differences, or interactions of group by tryptophan status, for the probability of inhibition or mean reaction time to the No-Go or oddball stimuli (Supplementary Table 2).

Functional MRI group brain activation mapping

Group brain activations for the contrast of No-Go-oddball trials for each of the two drink conditions in both groups revealed activations in key inhibitory areas such as inferior frontal cortex, medial frontal gyri, supplementary motor area, striatal and thalamic regions at cluster threshold of P < 0.008 (Fig. 1A–D and Supplementary Table 3).

Figure 1.

Group brain activation maps. Locations of group-wise BOLD signals from NoGo versus oddball contrasts ANOVAs for (A) controls under sham condition. (B) Controls under depletion condition. (C) ASDs under sham condition. (D) ASDs under depletion condition. Red = No-Go > oddball. Blue = No-Go < oddball. Numeric label = z Talairach coordinate. Right hemisphere of brain is on the right side of the image. CON = controls.

Analysis of variance brain activation mapping

Significant interaction effects of BOLD signal response between tryptophan drink status (sham, depletion) and group membership (control, autism) were observed in four (predominately right hemispheric) clusters. These included, the right inferior frontal (reaching into dorsolateral prefrontal cortex; P < 0.01), caudate (P < 0.04) cerebellum (P < 0.01) and left thalamus (extending from the insula to the subthalamic nuclei and to middle temporal gyrus; P < 0.001). There were no effects of drink order on any BOLD responses (Fig. 2A and Table 2).

Figure 2.

Interaction of 5-HT status (sham, ATD) by group (Control, Autism) for the No-Go versus oddball contrast. (A) Locations of BOLD signals for interaction ANOVA. Numeric label = z Talairach coordinate. (B) Box plots of mean BOLD signal extracted from each interaction cluster. Right hemisphere of brain is on the right side of the image. 5-HT = serotonin. *Indicates there is no main effect of 5-HT status from repeated measure ANOVA. BA = Brodmann area; SSQ = sum of squares.

Table 2.

Anatomical location and statistics for BOLD activation for No-Go versus oddball interaction ANOVA

Interaction effects ANOVA							Pair-wise effects t-test
							Main effects of group				Main effects of TRP status
							Sham		ATD		Control		Autism
							CON> ASD	CON< ASD	CON> ASD	CON< ASD	SHAM> ATD	SHAM< ATD	SHAM> ATD	SHAM< ATD
Region	x	y	z	BA	Size	Sig P	Sig P	Sig P	Sig P	Sig P	Sig P	Sig P	Sig P	Sig P
Right inferior frontal cortex	43	33	9	46	209	0.01	0.003			0.03	0.002			0.002
Left thalamus	−18	−26	4		321	0.001	0.007			0.04	0.001			0.004
Right caudate	7	22	4		87	0.04		0.02	0.006			0.001	0.2*
Right cerebellum	7	−82	−24		197	0.01		0.003	0.02			0.0001	0.04

x, y, z = Talairach coordinates; BA = Broadmann area; TRP = tryptophan; Sig = statistical significance; CON = control.

*Did not reach between drink statistical significance.

Main effect of group ANOVAs revealed that after sham, subjects with autism relative to control subjects, showed reduced activation in right inferior frontal cortex (P < 0.003) and left thalamus (P < 0.02), but enhanced activation in right cerebellum (P < 0.003) and caudate (P < 0.007). After depletion, subjects with autism relative to controls showed enhanced activation in right inferior frontal cortex (P < 0.03) and left thalamus (P < 0.006), but reduced activation in right cerebellum (P < 0.02) and caudate (P < 0.02) (Fig. 2B and Table 2).

Main effect of tryptophan status revealed that the BOLD signal response for controls was decreased due to depletion in the right inferior frontal cortex (P < 0.002) and caudate (P < 0.001). For the autism group, depletion lead to an increased BOLD signal in the right inferior frontal cortex (P < 0.002) and left thalamus (P < 0.004) and decrease in the right caudate (P < 0.2) and cerebellum (P < 0.04). The decrease in right caudate did not reach statistical significance (Fig. 2B and Table 2).

Test for ‘normalization’ by acute tryptophan depletion of abnormally activated areas in the autism group relative to controls

The contrast at sham for controls as compared to depletion for autism showed absence of any group differences (Fig. 3B).

Figure 3.

Brain activation maps showing abnormally activated inhibition regions in ASD that were ‘normalized’ by ATD. Location of BOLD signal changes between groups. Blue indicates controls > ASD; red indicates controls < ASD. Numeric label = z Talairach coordinate. Right hemisphere of brain is on the right side of the image.

Correlations (Pearson’s) between autism restricted, stereotyped and repetitive behaviours and functional MRI BOLD response to No-Go versus oddball contrast

Subjects with the more severe RSRB symptoms (both past and present) had the most abnormal brain response (compared to the controls) and the most pronounced changes (relative to sham) in some of the inhibition areas indicated by our interaction analysis. For example, inhibitory BOLD response from the No-Go versus oddball contrast, under sham conditions, was negatively correlated with RSRB severity as measured by the ADI-R and ADOS in the right inferior frontal cortex (n = 12, r = −0.8, P < 0.006; n = 14, r = −0.7, P < 0.009, respectively) and left thalamus (n = 12, r = −0.7, P < 0.008; n = 14, r = −0.7, P < 0.008, respectively). The BOLD response under depletion was positively correlated with ADI-R in the right caudate (n = 12, r = 0.6, P < 0.05). Additionally, the magnitude of change in BOLD response between sham and depletion positively correlated with ADI-R in right caudate (n = 12, r = 0.7, P < 0.02) and ADOS in left thalamus (n = 14, r = 0.6, P < 0.02) (Fig. 4).

Figure 4.

Correlations of No-Go versus oddball contrast BOLD response and the restricted, stereotyped and repetitive behaviours in autism. These scatter plots depict the BOLD signal plotted against the RSRB scores. (A) Under placebo sham 5-HT condition, No-Go versus oddball inhibition task with the RSRB from (i) the Autism Diagnostic Interview-Revised (ADI-R); and (ii) the Autism Diagnostic Observation Schedule (ADOS). (B) Under depletion ATD 5-HT condition, No-Go versus oddball inhibition task with the RSRB scores from (i) the ADI-R; and (ii) the ADOS. (C) Change in SSQs (sham-ATD) from interaction analysis with the RSRB from (i) the ADI-R; and (ii) the ADOS. Right hemisphere of brain is on the right side of the image. *Correlation did not survive removal of outliers.

Discussion

To our knowledge, this is the first event-related functional MRI study in people with autism to examine the differential effect of serotonergic modulation (by ATD) on neural activation during a motor inhibition (Go/No-Go) task; and to relate differences in functional modulation to clinical symptoms (i.e. RSRBs). Adults with autism had abnormal activation in classical inhibition areas (Chambers et al., 2009) of inferior frontal cortex, basal ganglia, thalamus and cerebellum; in frontal cortex, caudate and thalamus this was correlated to the severity of RSRB. Depletion differentially modulated these activations in ‘neurotypicals’ and in our autism subjects. As previously reported in healthy control studies, our data are in line with depletion significantly downregulating right inferior frontal cortex and upregulating right cerebellum (Rubia et al., 2005; Lamar et al., 2009). By use of the ‘regulation’ terms, we refer to upregulation as an enhanced BOLD signal for the NoGo versus oddball contrast and conversely, downregulation refers to a reduced BOLD signal for the NoGo versus oddball contrast. Furthermore, the interaction analysis indicated that in control subjects, tryptophan depletion downregulated activations in inferior frontal cortex and thalamus but upregulated activation in these regions in the autism group. In contrast, depletion downregulated abnormally enhanced activations in basal ganglia and cerebellum in the autism group while it upregulated them in controls. Furthermore, the differential serotonin modulation effects in basal striatal and thalamic activations were correlated with severity of RSRB in subjects with autism. For the striatal caudate activations, there was a positive correlation between the magnitude of downregulation (change of sum of squares between the sham and ATD conditions) and the severity of RSRBs as detected by the ADI-R. This indicated that subjects with the more severe RSRBs showed the greatest downregulation by serotonin depletion. We expected that for the thalamus there would be a negative correlation between magnitude of upregulation and the severity of RSRBs this time detected by the ADOS. However, we found the reverse, i.e. for the thalamic activations the correlation indicated that subjects with the more severe RSRBs showed the least upregulation (Fig. 3C). This may reflect difficulties of measuring RSRBs in adult subjects with the ADOS (Lord et al., 2012) and our small sample size.

These findings offer the first direct evidence that inhibitory brain dysfunctions may underpin RSRB in autism, and that these may be linked to differences in serotonin. It is unlikely that the differential modulation of inhibitory activations by ATD can be explained by potential confounds such as differences in task performance or in plasma tryptophan levels—as depletion had no differential group effect on task performance or lowering blood tryptophan concentration. The fact that ATD has no effect on performance but significantly modulated brain activations in both groups is in line with previous findings that depletion has a stronger effect on brain function than behaviour (Rubia et al., 2005; Lamar et al., 2009; Daly et al., 2010). The detection of behavioural differences may require more power than our subject number allows (Thirion et al., 2007) or alternatively motor inhibition tasks may not be impaired in subjects with autism (Hill, 2004).

The findings that abnormal brain activation in people with autism during inhibition was shifted towards control levels by depletion, and even ‘normalized’ in these regions, may be of clinical relevance as the amount of modulation by ATD, within frontal, thalamic and striatal regions, was correlated with the severity of (respectively) past and current RSRB in autistic individuals. Treatment studies using serotonin agonists in ASD (selective serotonin re-uptake inhibitors; SSRIs) have reported mixed results with fluoxetine showing improvements of repetitive behaviours in adults (Hollander et al., 2012), whereas citalopram (King et al., 2009) had no effect in children. The reasons for these differing outcomes are unclear. However, our results suggest another potential approach—reducing synaptic serotonin using selective serotonin reuptake enhancers (SSRE). Alternatively, it may be that individualized medicine approaches can be developed to identify those adults with ASD whose brain function responds to serotonergic modulation—and to target those individuals with specifically tailored treatments. It has, however, been reported (McEwen et al., 2009) that the SSRE tianeptine’s mechanism of action also involves modulation of glutamate, an additional neurotransmitter associated with ASD (Coghlan et al., 2012). Hence it will be important to further understand both serotonin’s primary and secondary (downstream) effects. Alternate methods of reducing synaptic serotonin with receptor binding drugs such as risperidone may also offer targeted treatments.

Our results also add to existing evidence that serotonin may play a key role in the pathophysiology of autism. The brain areas that we found to be differentially modulated by ATD form part of a fronto-striato-thalamo-cerebellar network of inhibitory control that develops progressively with age (Rubia et al., 2007), and has intermediate-to-high levels of serotonin receptors and transporters (Pazos et al., 1987; Varnäs et al., 2004) in healthy populations. Further, it has previously been reported by ourselves and others that in these regions, subjects with ASD have significant differences from controls in serotonin synthesis (Chugani et al., 1997), transporters (Nakamura et al., 2010) and 2A receptors (Murphy et al., 2006). Also, our finding that thalamic modulation by ATD is correlated with severity of RSRB in autism parallels findings by others of a correlation between repetitive behaviours and thalamic serotonin transporter binding (Nakamura et al., 2010). Finally, our data exploring distinct modulatory effects of serotonin in subjects with autism (occurring within inhibition networks), complement our previous findings of serotonergic effects in areas of emotion processing networks during perception of fearful and sad faces (Daly et al., 2012). Both networks are selectively modulated by serotonin and seem to be crucial to the social difficulties and restricted behaviours associated with autism.

Nevertheless this is an observational and not a longitudinal study. Also the brain regions we found to be functionally different have previously been reported to have significant developmental abnormalities in anatomy, neuronal metabolism and integrity (Otsuka et al., 1999; Abell et al., 2000; McAlonan et al., 2002; Friedman et al., 2003). Hence, it is unknown if the difference we found in brain function between autistic individual and controls, and/or their differential modulation by tryptophan depletion, are primary or secondary to differences in brain maturation or (most likely) a complicated mixture of both. For example, in addition to its role as a neurotransmitter, serotonin also acts as a trophic, or differentiation, factor in the human brain (Whitaker-Azmitia, 2001). Given that ASD is a neurodevelopmental disorder, the region-specific differences in brain activity we observed during successful response inhibition (with and without ATD) may also be influenced by disrupted trophic effects of serotonin. Hence, further studies are required to determine the extent to which differences in the neurobiology of inhibitory control in ASD are primarily determined by acute alterations in serotonin tone (i.e. neurotransmission activity), or by differences in brain maturation which may be secondary to altered trophic effects of serotonin.

There are some behavioural studies of inhibition that conclude that serotonin may not modulate response inhibition (Clark et al., 2005; Chamberlain et al., 2006; Crockett et al., 2009). However, these data resulted from investigations of Stop-Signal Inhibition tasks not from the Go/No-Go task that we used. Other data posit that these two tasks are differentially modulated by serotonin (Eagle et al., 2008). The authors differentiate the two response-inhibition tasks into an ‘action restraint’ Go/No-Go task (modulated by serotonin) and an ‘action cancellation’ Stop-Signal task (not modulated by serotonin). Although our study did not find behavioural group aberrations in task performance, we believe this was due to our sample size not being sufficient to find any differences. The sample size was, however, large enough for detection of significant brain activation differences, probably helped by the use of non-parametric statistics, cluster-level thresholding and mixed effects testing of the functional MRI data.

A further limitation to our study is the absence of blood serotonin measurement to screen subjects for hyperserotonaemia. Additionally, a better indirect indicator of brain serotonin synthesis is the ratio of tryptophan to the other large neutral amino acids in the plasma (Fernstrom, 1983). Our data on the plasma levels of tryptophan suggest that under the sham condition there may be a certain degree of ‘loading’ tryptophan (Table 1); however, this is unlikely to have an effect on serotonin loading because the ratio of tryptophan to other amino acids typically show minimal increase during sham. Unfortunately, we lack requisite data to report ratio measures in our sample; previous studies provide clarification of the sham plasma data (Sambeth et al., 2007). While the focus of our study was on RSRBs as weak executive functioning of inhibition in autism, obsessive-compulsive disorder symptomatology, with an enhanced impulse not to inhibit, might also need to be considered. The ASD group did have higher scores on the Obsessive-Compulsive Inventory-Revised and Beck Anxiety Inventory (although no one had received a clinical obsessive-compulsive disorder diagnosis), but we do not have a behavioural measure to disentangle how our subjects experience their RSRBs (i.e. ego-dystonic, ego-syntonic or neutral). Last, we are unable to generalize our findings in normal intelligence adults to other ASD groups (e.g. those with intellectual disabilities or children). However, depletion studies in these groups have significant ethical considerations, and a necessary first step is to demonstrate differences in people who are able to give informed consent for themselves.

In summary, in people with autism, brain activation differences of inhibitory control regions are differentially modulated by serotonin, and this may partially underpin some restricted, stereotyped, and repetitive symptoms.

Glossary

Abbreviations

ADI-R

Autism Diagnostic Interview-Revised

ADOS

Autism Diagnostic Observational Schedule

ASD

autism spectrum disorder

ATD

acute tryptophan depletion

BOLD

blood oxygen-level dependent

RSRB

restricted, stereotyped and repetitive behaviours

Funding

The Health Foundation and the MRC UK AIMS (G0400061) sponsored the study. We would also like to acknowledge the EU-AIMS (supported by the Innovative Medicines Initiative Joint Undertaking under grant agreement no. 115300, which includes financial contributions from the EU Seventh Framework Programme (FP7/2007–2013) the Biomedical Research Centre for Mental Health-CD Cluster-Developmental Disorders, National Institute for Health Research (NIHR) at South London and Maudsley NHS Foundation Trust and King's College London, The Dr Mortimer and Theresa Sackler Foundation and Autism Speaks.

Financial disclosure

M.B. is a consultant for P1Vital, Oxford and K.R. has received speaker’s honoraria from Lilly, Shire and Novartis. The other authors have no financial disclosures or conflicts of interest.

Supplementary material

Supplementary material is available at Brain online.