Co-ordinated structural and functional covariance in the adolescent brain underlies face processing performance

Abstract

Our ability to process complex social cues presented by faces improves during adolescence. Using multivariate analyses of neuroimaging data collected longitudinally from a sample of 38 adolescents (17 males) when they were 10, 11.5, 13 and 15 years old, we tested the possibility that there exists parallel variations in the structural and functional development of neural systems supporting face processing. By combining measures of task-related functional connectivity and brain morphology, we reveal that both the structural covariance and functional connectivity among ‘distal’ nodes of the face-processing network engaged by ambiguous faces increase during this age range. Furthermore, we show that the trajectory of increasing functional connectivity between the distal nodes occurs in tandem with the development of their structural covariance. This demonstrates a tight coupling between functional and structural maturation within the face-processing network. Finally, we demonstrate that increased functional connectivity is associated with age-related improvements of face-processing performance, particularly in females. We suggest that our findings reflect greater integration among distal elements of the neural systems supporting the processing of facial expressions. This, in turn, might facilitate an enhanced extraction of social information from faces during a time when greater importance is placed on social interactions.

Introduction

Face processing plays a fundamental role in social behaviour. Faces inform us about others’ identity, sex, and age (Todorov et al., 2008), and facial expressions provide insights into the mental and emotional states of persons present in our social space. This allows us to moderate our own behaviour accordingly. Face processing continues to develop between childhood and adulthood (Germine et al., 2011; Johnston et al., 2011; Tottenham et al., 2011), a protracted developmental trajectory that parallels brain maturation at this time (Giedd and Rapoport, 2010; Lebel and Beaulieu, 2011). Investigating such a developmental brain–behaviour relationship in the context of face processing may help us understand the maturation of other socio-cognitive neural systems (Cohen Kadosh, 2011; Scherf et al., 2012).

Human faces present to others a vast array of social cues, and require various processing skills that emerge along different developmental trajectories. Face recognition reveals both qualitative and quantitative maturity during early development; even at 4 years of age, face individuation involves the same underlying mechanisms (e.g. holistic processing) and reaches levels of efficiency comparable to those observed in adults (for a comprehensive review, see McKone et al., 2012). In contrast, although children exhibit qualitative similarities to adults when identifying emotional facial expressions (e.g. confusion between fear and surprise; Gagnon et al., 2009), performance improves dramatically between childhood and adolescence (e.g. Johnston et al., 2011; Tottenham et al., 2011). Interestingly, sex differences in face processing also emerge during childhood; females are not only quicker and more accurate than males in face detection—the initial stage of face processing (McBain et al., 2009), they also show an advantage when discriminating among facial emotional expressions (for reviews, see Herba and Philips, 2004; McClure, 2000). The task now is to understand the neural mechanisms giving rise to these developmental events.

Face processing engages a widely distributed set of neural systems (Haxby et al., 2000). Using functional magnetic resonance imaging (fMRI), we have measured brain responses to dynamic facial expressions in 1,110 typically developing adolescents (13.5–15.5 years old) and created probabilistic maps of the network engaged by these stimuli (Tahmasebi et al., 2012). This revealed a diffuse collection of regions in the adolescent brain that express a high probability of fMRI response to ambiguous expressions, including extra-striate and prefrontal cortical structures. On the basis of inter-individual variations in the functional-connectivity profiles of these regions, we have termed these the ‘obligatory’ and ‘optional’ nodes of the adolescent face-processing network (Dickie et al., 2014), respectively. This delineation converges closely with what has become known as the ‘core’ and ‘extended’ systems of the adult face-processing network (Haxby and Gobbini, 2010), the integration of which is suggested to be necessary for facial expression perception (see Said et al., 2011).

Developmental neuroimaging studies have focused typically on the maturation of individual nodes of the adolescent face-processing network (e.g. Golarai et al., 2007; Scherf et al., 2007; Cantlon et al., 2011). A shift of focus can be observed in recent studies, however, which investigate the development of their functional connectivity (Cohen Kadosh et al., 2011; Pfeifer et al., 2011; Joseph et al., 2012; Gee et al., 2013). These studies have shown that although the various neural systems that mediate face processing come ‘online’ in the early stages of childhood (e.g. Cantlon et al., 2011), their integration into a mature functional network occurs gradually throughout adolescence (Johnson et al., 2009; Cohen Kadosh, 2011). Such gradual tuning of the functional organization within the face-processing network during adolescence parallels the protracted structural development of its constituent brain systems during this time (Giedd and Rapoport, 2010). This supports the notion of interplay between structural and functional brain development; although brain morphology will constrain the initial spatial and temporal dynamics of brain function, neural activity has the potential to exert co-ordinated influences on the structural properties of two distant yet functionally connected brain regions (Honey et al., 2010; Power et al., 2010). Such a developmental relationship between brain structure and function would explain the spatial overlap between networks of functional connectivity and patterns of age-related changes in grey-matter (GM) covariance observed between childhood and adolescence (Zielinski et al., 2010; Raznahan et al., 2011).

In this light, the protracted development of face-processing ability likely reflects interplay between functional and structural maturational events that, together, shape an integrated face-processing network. Consistent with this view, recent studies report that age-related improvements in face-processing performance during childhood and adolescence are associated with developments in brain function and structure (Cohen Kadosh et al., 2013; Scherf et al., 2014). Further, given the known sex differences in facial expression recognition (e.g. Collignon et al., 2010; Tottenham et al., 2011; see also Scherf et al., 2012), such structure–function relationships may unfold differently in girls and boys. To our knowledge, however, no study has examined simultaneously the functional and structural ‘integration’ of the face-processing network during this age range, and its relationship with the maturation of relevant behaviours.

Combining behavioural measures with structural and functional MRI data, the present study set out to investigate the relationship between developments in face-processing performance and developmental changes in GM covariance and functional connectivity throughout the adolescent face-processing network. To measure developmental brain–behaviour relationships, we collected these data longitudinally from adolescents aged 10, 11.5, 13 and 15 years old. fMRI data were acquired as they observed passively videoclips of dynamic facial expressions (Grosbras and Paus, 2006). We have shown this paradigm to be highly effective in eliciting strong and reliable brain responses throughout a network implicated in face-processing (Grosbras and Paus, 2006; Tahmasebi et al., 2011). Furthermore, by using this task we have identified various factors that influence the brain response to faces; this includes genetic variations (Dickie et al., 2014), hormonal influences (Marecková et al., 2014), sex differences (Schneider et al., 2011; Tahmasebi et al., 2011), and developmental events in both localized (Shaw et al., 2012) and distributed brain systems (Shaw et al., 2011). By combining multivariate network measures of brain structure and function with behavioural assessments, we were able to investigate the relationship between neurodevelopmental trajectories and the ability to extract information from faces.

Methods

Adolescents

As part of a longitudinal study, 65 adolescents (33 males) were assessed at four time points. Despite excluding children with braces when recruiting the original sample at Visit 1, it was unavoidable that many wore braces during at least one of the three subsequent time points. As the multivariate analyses we applied to the neuroimaging data demand balanced numbers, from this original sample we excluded 7 adolescents (4 males) due to attrition and a further 20 (8 males) due to imaging-related factors caused by braces at one or more follow-up visits. As such, all the analyses described below were applied to data from 38 adolescents (21 males; 19 right-handers), each scanned at all four visits. The average age was 10 years (119 months; s.d. = 4.9 months; range = 113–129 months) at Visit 1; 11.5 years [138 months; s.d. = 5.2; range = 132–148; mean time interval = 18.5 (s.d. = 1.9) months] at Visit 2; 13.1 years [157 months; s.d. = 5.7 months; age range = 146–168 months; time interval = 18.9 (s.d. = 2.5) months] at Visit 3; and 15 years [180 months; s.d. = 7.5; range = 167–199; mean time interval = 23 (s.d. = 5.6) months] at Visit 4.

All participants were confirmed to be healthy, typically developing children presenting no symptoms of psychiatric, developmental, or medical disorders. There was no evidence of neurological disorder, as assessed by the Quick Neurological Screening Test (Mutti et al., 1978), nor any signs of behavioural problems, measured with the Child Behaviour Checklist (Achenbach and Ruffle, 2000); all adolescents demonstrated typical physical development, as determined by the Pubertal Development Scale (Peterson et al., 1988) completed upon each visit; and the sample exhibited typical scores of full-scale IQ, assessed with the Wechsler Intelligence Scale for Children (WISC-III and WISC-IV, alternated between visits; Wechsler, 1949), and a reading ability no more than 2 years below the grade-appropriate level (Woodcock et al., 2001). Each participant reported English as their dominant language, and all had normal or corrected-to-normal vision. Demographic data for the final sample are presented in Table 1. Informed consent was obtained from the parents, together with assent from the adolescents themselves. The study was approved by the Research Ethics Board of the Montreal Neurological Institute (MNI).

Table 1.

Demographic data for adolescents included within the final sample of 38

Visit	Age	Pubertal stage	WISC	W-J III	CAT
					3Facesinv	Affectinv	Identityinv
Males
1	120.0 (±1.0)	1.6 (±0.2)21	113.1 (±2.4)	130.3 (±8.8)	5738.82 (±261.52)	3937.66 (±213.10)	5141.75 (±370.19)
2	138.1 (±1.1)	2.0 (±0.2)21	111.1 (±1.4)	164.6 (±10.0)	5660.41 (±288.33)	3695.21 (±187.07)	4098.00 (±304.74)
3	157.8 (±1.2)	2.8 (±0.2)19	116.7 (±2.1)	190.3 (±11.4)	5064.88 (±228.61)	2878.37 (±114.00)	3213.56 (±172.03)
4	179.7 (±1.4)	3.6 (±0.1)18	113.9 (±2.5)	212.6 (±9.8)	4931.18 (±227.27)	2860.21 (±102.34)	3173.12 (±165.78)
Females
1	119.0 (±1.3)	2.4 (±0.2)16	115.8 (±2.4)	158.1 (±5.7)	6344.41 (±399.34)	4417.18 (±307.67)	4899.10 (±324.52)
2	137.9 (±1.4)	2.9 (±0.2)15	112.4 (±2.3)	181.4 (±8.6)	5652.92 (±250.04)	3663.36 (±240.34)	3725.46 (±253.90)
3	155.9 (±1.5)	3.9 (±0.1)14	119.2 (±3.2)	216.7 (±8.8)	4840.72 (±240.10)	2847.21 (±133.01)	3125.36 (±216.14)
4	180.4 (±2.2)	4.1 (±0.1)10	117.2 (±2.3)	244.2 (±7.1)	5135.29 (±293.00)	2634.57 (±145.93)	2794.90 (±184.66)

Notes: Values present means (±standard error). Puberty, Tanner stage (Marshall and Tanner, 1969), as assessed by the Puberty Development Scale (Peterson et al., 1988; 1 = pre-pubertal, 2 = beginning puberty, 3 = mid-puberty, 4 = advanced puberty, 5 = post-pubertal); WISC, Wechsler Intelligence Scale for Children (versions III and IV were alternated between visits; values present full-scale IQ); W-J III, Woodcock-Johnson (values present age equivalency for spelling comprehension); 3Faces_inv, Affect_inv and Identity_inv, inverted efficiency scores for the Three Faces Test, Affect and Identity Discrimination, respectively. Tanner stages were not collected from all adolescents, so subscripts indicate the number of subjects contributing to the mean value.

Stimuli

The analyses of functional connectivity presented below were applied to a section of fMRI data acquired as participants observed passively silent grey-scale video clips depicting angry or ambiguous (e.g. nose twitching) facial expressions. We referred originally to the latter stimuli as ‘neutral’ faces (Grosbras and Paus, 2006), but for consistency we use the same ‘ambiguous’ term here as that employed by Tahmasebi et al. (2012) who used the same stimuli. Each type of facial expression was presented separately in 18 s blocks (seven or eight videos per block) for a total of five blocks in a single scanning session. Our group has shown the effectiveness of these stimuli presented in this sequence to engage strongly many neural systems involved in face processing (e.g. Shaw et al., 2012; Dickie et al., 2014). The 5 blocks of angry and ambiguous face stimuli were presented pseudo-randomly amongst 10 control blocks of non-biological motion, consisting of concentric circles that expanded and contracted at varying speeds and contrasts. Example snapshots are presented in Figure 1. Adolescents were instructed to attend closely to the video clips so that they could answer questions about the stimuli after the scan. An informal post-scanning recognition test confirmed that all individuals could recognize a subset of 10 experimental facial expressions from a set of 14 clips (four oddballs).

Fig. 1.

Snapshots of dynamic Ambiguous and Angry facial expression stimuli. Adapted with permission from Grosbras and Paus (2006).

Imaging protocol

Scanning was performed on a 1.5T Siemens (Erlanger, Germany) Sonata scanner. A high-resolution T1-weighted structural image (matrix 256 × 256 × 160; 1 mm³ voxels) was first acquired for anatomical localization and co-registration with the functional times-series. The time-series consisted of 180 T2*-weighted, gradient-echo, echo-planar BOLD images (matrix 64 × 64 × 32; 4 mm³ voxels; TR = 3 s; TE = 50 ms) collected after the gradient had reached steady-state. Each slice was oriented parallel to a line connecting the base of the cerebellum to the base of orbitofrontal cortex, covering the whole brain.

Regions of interest

We applied our multivariate analyses to structural and fMRI data extracted from 25 regions of interest (ROIs). Twenty-one of these interrogated ROIs correspond to the brain regions demonstrating a high probability (>50%) of fMRI response to the ambiguous facial expressions, as defined in the probabilistic map computed by Tahmasebi et al. (2012) using the exact same stimuli in a sample of 1100 adolescents. For completeness, in the present analyses we included four (low probability) contra-lateral homologues of (high probability) unilateral ROIs featuring in this probability map; hence a total of 25 ROIs. Table 2 lists all ROIs involved in the current analyses.

Table 2.

Co-ordinates of voxels with peak probability of fMRI response to the ambiguous facial expressions used as stimuli, as derived in an independent cross-sectional sample of 1,110 adolescents (Tahmasebi et al., 2012)

No.	Label	Hemisphere	Lobe	MNI coordinate			Prob. value (%)	Extent (mm3)
				x	y	z
1	MVL-FC	Left	Frontal	−42	27	−3	64	12,150
2	MVL-FC	Right	Frontal	53	27	−3	100	10,989
3	MDL-FC	Left	Frontal	−45	15	22	100	19,251
4	MDL-FC	Right	Frontal	46	20	20	100	25,110
5	PMC	Left	Frontal	−48	0	51	45	4401
6	PMC	Right	Frontal	51	5	46	100	14,715
7	Pre-SMA		Frontal	6	12	63	96	12,312
8	Rhinal Sulcus	Left	Temporal	−39	−6	−42	46	3752
9	Rhinal Sulcus	Right	Temporal	42	−6	−42	64	7101
10	Anterior STS	Left	Temporal	−54	3	−18	54	11,799
11	Anterior STS	Right	Temporal	55	9	−24	100	13,743
12	Post. STS	Left	Temporal	−54	−51	9	100	27,135
13	Post. STS	Right	Temporal	56	−42	7	100	44,307
14	FFA	Left	Occipital	−40	−49	−22	100	10,368
15	FFA	Right	Occipital	42	−49	−23	100	11,313
16	LOC	Left	Occipital	−45	−84	−12	86	9585
17	LOC	Right	Occipital	47	−78	−8	100	9801
18	V2–V3	Left	Occipital	−24	−99	0	78	8154
19	V2–V3	Right	Occipital	24	−99	0	78	7803
20	Cerebellum	Left		−18	−78	−36	100	20,520
21	Cerebellum	Right		18	−78	−39	57	7182
22	Putamen	Left		−21	3	6	31	1377
23	Putamen	Right		21	6	6	57	2268
24	Amygdala	Left		−18	−9	−15	96	11,799
25	Amygdala	Right		22	−8	−14	100	11,745

Notes: The far right-hand column shows the real-world size of each ROI, from which we calculated the mean functional and GM values used in the present analyses. Grey rows correspond to (low-probability) ROIs interrogated in the present analysis but not included in the original maps of Tahmasebi et al. (2012). MVL-FC, mid-ventrolateral frontal cortex; MDL-FC, mid-dorsolateral frontal cortex. Adapted with permission from Tahmasebi et al. (2012).

Pre-processing

The neuroimaging data were pre-processed with tools packaged within FMRIB’s software library (FSL; Jenkinson et al., 2012).

Structural data

As we were interested in both cortical and sub-cortical structures, we chose regional GM volumes, rather than cortical thickness, as our metrics (see Winkler et al., 2010). To measure GM volume, individual T1-weighted anatomical MR images were pre-processed using a selection of utilities comprising FSL’s optimized voxel-based morphometry pipeline (see Good et al., 2001). This involves firstly the removal of non-brain tissue (Smith, 2002), automated tissue segmentation (Zhang et al., 2001), and non-linear registration to the MNI-152 template (Anderson, 2010). The resulting GM images are then averaged and flipped along the x-axis to create a left-right symmetric, study-specific template. Second, all native GM images were non-linearly registered to this study-specific template and ‘modulated’ to correct for local expansion (or contraction) due to the non-linear component of the spatial transformation. Finally, the modulated GM partial-volume maps (GM-PVMs) were smoothed by a Gaussian kernel [full-width half-maximum (FWHM) = 8 mm]. This FSL pipeline has been shown to produce results that converge closely with those of an SPM voxel-based morphometry analysis, and complement those from tract-based spatial statistics (Douaud et al., 2007).

The resulting GM-PVMs at each time-point were registered to standardized space in a process we describe in Sutherland et al. (2012)—a registration procedure that leaves regional (non-linear) variations in GM intact. Finally, the registered GM-PVMs were smoothed by a Gaussian kernel (FWHM = 8 mm). For every adolescent and at each visit, we then extracted from these pre-processed GM-PVMs the mean GM partial volume across all voxels comprising each of our 25 ROIs. It is these regional GM values that were entered into the covariance analyses (see below).

Functional data

The fMRI data collected at each visit from all 38 adolescents comprising the final sample were motion corrected using MCFLIRT (Jenkinson et al., 2002). Given the detrimental effect of residual motion artefacts on measures of functional connectivity (Van Dijk et al., 2011; Power et al., 2015), and the confounding influence this can have when measuring age-related changes in functional connectivity (Satterthwaite et al., 2012), we also minimized any motion-related signal remaining after motion correction. Using MELODIC (Beckmann and Smith, 2004; Beckmann, 2012) we applied a probabilistic independent component analysis (ICA) to decompose the time-series into 50 independent spatial and temporal components. Artefactual components (e.g. residual motion-related signal, physiological noise) were identified automatically with the Spatially Organized Component Klassifikator (Bhaganagarapu et al., 2013). Signal relating to these nuisance covariates was then regressed out of the time-series using another FSL utility—fsl_regfilt. Estimates of framewise displacement before and after ICA are presented in Supplementary Table S1.

Using FEAT v5.92, functional images were high-pass filtered across time (Gaussian-weighted least-squares straight line fitting; sigma = 50.0 s), and spatially smoothed using an 8.0 mm FWHM Gaussian kernel. Time-series were intensity normalized using grand-mean scaling of the entire 4D dataset by a single multiplicative factor to minimize unspecific time effects. With FLIRT, all individual time-series were registered to their corresponding brain-extracted high-resolution anatomical image using a rigid-body affine transformation; anatomical images were registered to the MNI-152 standard space template; and the pre-processed functional time-series were registered to standardized stereotaxic space by combining both transformation matrices. Finally, using masks derived from thresholded (90%) standardized probabilistic maps of white-matter and cerebrospinal fluid (http://www.loni.ucla.edu/ICBM/), any signal related to these nuisance covariates was regressed from these functional time-series.

For every adolescent and at each visit, we concatenated separately the mean-centred BOLD signal from all volumes acquired during the five blocks of each facial expression. We then extracted the mean concatenated time-series across all voxels comprising each of our 25 ROIs. Correlating these time-series among all pairs of ROIs produced two 25 × 25 auto-correlation matrices for each subject at every visit—one for the Ambiguous- and one for the Angry-face blocks.

Statistical analyses: Partial Least-Squares of GM and BOLD signal

As only one (mean) value of GM is extracted from each ROI for a given adolescent at each visit, we calculated eight visit-specific GM correlation matrices for all pairs of ROIs; specifically, four matrices (Visits 1–4) each for males and females. We then produced the same eight mean functional correlation matrices for each facial expression (Ambiguous and Angry). Subsequently, these two sets of structural and functional correlation matrices were subjected separately to Partial Least-Squares (PLS) analysis (McIntosh et al., 1996; McIntosh and Lobaugh, 2004; Krishnan et al., 2011)—a multivariate technique capable of extracting patterns of structural or functional covariance among all 25 ROIs and across all four visits for the two sexes. This allowed us to explore the emergence of patterns of structural and functional covariance across visits for the 38 adolescents, and if the same or different developmental trajectories are expressed by males and females.

In its current application, PLS computed a new cross-correlation (cross-block) matrix, M, that defined the covariance among the set of structural or functional matrices. Through singular value decomposition of M emerged a set of latent variables (LVs), each of which identified a particular pattern of covariance among the input matrices (Krishnan et al., 2011). One element of each LV contains numerical weights (silences) for each visit, creating a ‘visit profile’ that represents the trajectory of a given covariance pattern over time. The other element of the LV—the ‘singular image’—identifies the cells of M (i.e. pairwise ROI correlations) that together covary according to the visit profile. In other words, each LV identifies a pattern of covariance among the pairwise correlations between ROIs that change in a co-ordinated fashion across visits. In a variation of this unconstrained, data-driven approach (herein referred to as ‘Mean-centred PLS’), covariance among the input matrices is defined a priori and tested against their measured covariance. Such a ‘Non-rotated PLS’ permits the testing of whether or not a hypothesized pattern of covariance over time accounts sufficiently for the observed covariance structure.

We performed PLS analyses within Matlab (Mathworks Inc., USA), incorporating 1000 permutations and 1000 bootstraps to assess the significance of LVs (McIntosh et al., 2004). Importantly, by treating Visits as multiple conditions among which the ordering of participants was kept consistent, the resampling procedure preserved the repeated-measures structure of the data.

Comprehensive Affect Test

To investigate developments in face-processing performance, we examined scores on the Comprehensive Affect Test (CAT; Froming et al., 2006) at each of the four visits. The CAT comprises 13 subtests, of which we selected those that involve face processing: ‘Identity Discrimination’, requiring participants to decide whether two portraits are of the same (or different) person; ‘Affect Discrimination’, where the participant must decide whether two portraits depict the same (or different) emotion; and the ‘Three Faces Test’, whereby the participant identifies which two of three portraits express the same emotion. All three produce measures of both accuracy and response time (RT), so for each we computed the inverse efficiency score to account for any speed-accuracy trade-off—RT divided by the proportion of correct responses (Townsend and Ashby, 1983).

Results

Given the high number of adolescents excluded (due to braces and attrition), we checked that we had not introduced a sampling bias. Bonferroni-corrected non-parametric analyses confirmed equivalency between the 38 included and 27 excluded adolescents: although we observed a nominally significant difference at Visit 4 for reading ability (Woodcock-Johnson III), with adolescents included in the final sample scoring higher on age-equivalency than those excluded [mean (±s.d.) = 222.58 (±44.13) vs 220.50 (±28.69) months, respectively; Z = −2.92, P_corr = 0.04], there were no differences between any visit in terms of IQ (Wechsler Intelligence Scale for Children; Z < 1.1, P_corr > 0.05) or any measure of CAT performance (Z < 1.6, P_corr > 0.05).

Structural covariance

To identify time-varying patterns of structural covariance across the sample of 38 adolescents, we applied a mean-centred PLS analysis to the eight structural correlation matrices (four visits per sex). We refer to this analysis as PLS_struct. This revealed only two significant LVs. The first (P < 0.001) identified a widely distributed pattern of inter-regional correlations that distinguished between males and females across all visits. This initial LV accounted for 35.34% of the cross-block variance. As we were interested primarily in developmental trajectories, we focused our attention on the second significant LV (P = 0.044), which accounted for 24.05% of the cross-block variance. Figure 2A presents the eight input matrices on which PLS_struct was performed, and the visit profile and bootstrapped singular image corresponding to this second LV. The visit profile illustrates that this LV identified a marked change in GM co-variance from Visit 1 to 4, and delineated between a linear and step-like trajectory for males and females, respectively. The bootstrapped singular image identifies two sets of cells that vary according to these sex-specific trajectories in an orthogonal fashion: positive saliences identify pairwise correlations that vary across visits in a fashion correlated positively with the visit profile {e.g. ‘increasing’ GM covariance between extra-striate [lateral occipital cortex (LOC), fusiform face area (FFA)] and frontal cortices [pre-supplementary motor area (pre-SMA), premotor cortex (PMC)]}; negative saliences represent pairs of ROIs between which GM covariance follows a trajectory across visits that is correlated negatively with this visit profile (e.g. ‘decreasing’ covariance between extra-striate and cerebellar cortices).

Fig. 2.

The process behind and results emerging from the PLS analyses. (A) The eight structural correlation matrices (top) were entered into a mean-centred PLS analysis (PLS_struc). This produced two LVs, the second of which comprised a ‘visit profile’ (middle) that revealed a large change in structural covariance between Visits 1 and 4 (V1–V4) and sex-specific trajectories. The singular image of this second LV (bottom) highlights the pairwise correlations expressing this trajectory—warm cells are correlated positively and cold cells negatively with the visit profile. Note: The image presents the elements of the LV after statistical analysis (see text), and so the singular image presents bootstrapped ratio values. (B) The eight functional correlation matrices generated by concatenating the Ambiguous-face blocks (top) were entered into a mean-centred PLS (PLS_func), producing a visit profile (middle) that identified increased functional connectivity between Visits 1 and 4 similar to PLS_struc. The corresponding bootstrapped singular image (bottom) also included some of the same pairwise correlations. (C) The eight functional correlation matrices from the Ambiguous-face blocks (top) were entered into a non-rotated PLS analysis (PLS_struc–func), in which the saliences of the visit profile from PLS_struc were used as contrasts. This allowed us to investigate the degree of association between the developmental trajectories of structural and functional covariance. The bootstrapped singular image (bottom) illustrates the subset of ROIs among which this was true. Note: The cells encompassed by dashed lines within each singular image represent pairwise correlations between distal ‘obligatory’ and ‘optional’ ROIs (see text). This collection of ROI pairs is referred to as the ‘optional-obligatory sub-network’ in the text.

Functional connectivity

To detect developmental changes in functional connectivity, we applied the same mean-centred PLS analysis to each set of eight functional-connectivity matrices generated by concatenating the blocks of each facial expression. Applying this analysis to the Ambiguous functional-connectivity matrices, referred to herein as PLS_func, revealed a single significant LV (P = 0.039) accounting for 34.34% of cross-block variance. As presented in Figure 2B, the visit profile for this LV revealed a large change from Visit 1 to 4, but this time following linear and cubic-like visit-related trajectories for males and females, respectively. The corresponding bootstrapped singular image identifies pairwise correlations that covary according to these sex-specific trajectories. Interestingly, a number of ROIs among which pairwise correlations expressed this developmental change in covariance across the 38 adolescents overlapped with those expressing the analogous trajectory for structural covariance (compare Figure 2B with 2A); we observed this pattern of increasing functional connectivity between ROIs within the extra-striate [LOC, FFA and posterior superior temporal cortex (STS)] and frontal (pre-SMA, PMC and mid-ventro-lateral PFC) cortices, respectively. As these long-range connections emerging from both PLS_struc and PLS_func integrate ‘obligatory’ and ‘optional’ nodes, we refer to them as the ‘obligatory-optional sub-network’ herein. Importantly, applying the same PLS analysis to the eight Angry-face functional-connectivity matrices revealed no significant LVs; as shown in Supplementary Figure S1, no reliable visit- or sex-related pattern of covariance between ROIs was identified in response to these stimuli.

Structure–function integration

To assess formally the inter-dependence between the development of structural and functional covariance within the face-processing network, we investigated whether the developmental trajectory of each property was related to that of the other. To do so we performed separate non-rotated PLS analyses on the sets of structural and functional correlation matrices, testing whether the saliences of the visit profiles emerging from the previous PLS analysis of one could be used to reproduce the covariance structure of the other. First we applied the saliences of the visit profile emerging from the second LV of PLS_struc (Figure 2A) in a non-rotated PLS analysis of the eight functional correlation matrices obtained by concatenating the Ambiguous-face blocks—referred to herein as PLS_struc–func. This allowed us to examine whether or not the developmental trajectory shown by structural covariance accounted for the covariance among the functional connectivity matrices across visits. This produced one significant LV (P = 0.048), which identified a pattern of functional covariance in response to the Ambiguous-face stimuli that followed the exact same trajectory across visits as structural covariance (see Figure 2C). This pattern emerged among many of the ROIs comprising the ‘obligatory-optional sub-network’; namely, sex-specific visit-related increases occurred in pairwise correlations between extrastriate (V2/V3, LOC, FFA, posterior STS) and frontal (mid-dorso- and mid-ventrolateral, and PMC) cortices. Note that the reverse analysis was not true: applying the visit saliences emerging from the PLS_func analysis of the functional connectivity matrices from the Ambiguous-face blocks in a non-rotated PLS analysis applied to the structural correlation matrices yielded no significant LV. Furthermore, no such structure–function relationship was observed for functional connectivity measured during the concatenation of Angry-face blocks (P = 0.134; see Supplementary Figure S1).

CAT performance

Having identified parallel increases in structural and functional covariance among an obligatory-optional sub-network of nodes within the face-processing network, we set out to examine whether the degree of their structural and/or functional integration is associated with face-processing performance. To do so we employed two approaches: First we used linear mixed-model regression to assess the longitudinal relationship between each behavioural measure and (i) functional connectivity and (ii) structure–function integration among the pairwise correlations defined by PLS_func and PLS_struc–func, respectively. Secondly, we examined differences in GM covariance among the subset of ROIs identified by PLS_struc at each visit between adolescents performing above or below the median in face-processing performance.

For each behavioural measure, values greater or lesser than 3 s.d. from the mean were removed. For each adolescent, functional connectivity at each visit was expressed as the mean inter-regional correlation coefficient among all the ROIs comprising the ‘obligatory-optional’ sub-networks identified by PLS_func or PLS_struc–func. These two separate measures of functional connectivity—‘Connectivity_func’ and ‘Connectivity_struc–func’—were entered independently into three mixed models, one for each behavioural measure. The best-fitting model for each measure was identified in a step-up manner (West et al., 2007), whereby fixed and random effects were added sequentially to a base model. A given effect was included in the final model if it resulted in a significant decrease of the log-likelihood value, as determined by Chi-square comparisons at each step. On this basis, all models included a random intercept and some a random ‘Connectivity’ effect. Importantly, to assess whether the measure of Connectivity was related to each behavioural measure independently of the linear improvement with age, Age was added as a fixed effect in all models. The contribution of Sex or the Sex-by-Connectivity interaction was considered only when the addition of these fixed effects improved model fit. Finally, all models used an autoregressive covariance structure. All modelling was performed in SPSS v22.

CAT and functional connectivity.

Connectivity_func was related to inverse-efficiency scores of Identity Discrimination (Identity_inv) independently of Age [F_(1,160.44) = 4.856, P = 0.029], but not with Affect [Affect_inv; F_(1,97.55) = 2.990, P = 0.087] or the 3-Faces Test [3Faces_inv; F_(1,197.55) = 0.061, P = 0.805]. There was no improvement in fit for any of the models applied to Connectivity_func when adding the main effects of Sex or the Sex*Connectivity_func interaction. Connectivity_struc–func was not associated with Identity_inv [F_(1,146) = 2.857, P = 0.093] or Affect_inv [F_(1,143) = 1.977, P = 0.162] independently of Age, and there was no increase in the fit of these with the addition of fixed Sex or Sex*Connectivity_struc–func interaction effects. This latter measure of connectivity was related to 3Faces_inv [F_(1,144) = 4.226, P = 0.046], however. Moreover, the addition of fixed Sex and Sex-by-Connectivity_struc–func interaction effects improved the fit this model, with both fixed effects accounting for a significant amount of variance [Sex: F_(1,144) = 4.885, P = 0.029; Sex*Connectivity_struc–func: F_(1,144) = 4.499, P = 0.036]. The regression coefficients corresponding to all significant effects presented above are given in Table 3, and the sex-specific association between Connectivity_struc–func and 3Faces_inv is illustrated in Figure 3.  It is important to note that these relationships between Connectivity and behavioural performance are not strong enough to survive a correction for multiple (six) comparisons. As such, these results must be interpreted with caution (Figure 4).

Table 3.

Regression coefficients from linear mixed models

Behavioural measure	Intercept	βConnectivity
Connectivityfunc
Identityinva	5319.06 (±254.10)**	−1556.87 (±706.49)*
Affectinva	4397.54 (±179.92)**	−795.15 (±459.85)
3Facesinva	6055.73 (±236.78)**	−162.41 (±655.95)
Connectivitystruc–func
Identityinv	5287.58 (±307.83)**	−1093.79 (±777.91)
Affectinv	4514.71 (±215.40)**	−908.91 (±537.74)
3Facesinv	5403.58 (±417.53)**	−84.77 (±931.25)
Males
Females	6705.77 (±622.66)*	−2787.03 (±1374.25) *

Notes: Values give the estimates of inverted efficiency (expressed in milliseconds) emerging from the models used to investigate the relationship between functional connectivity and face-processing performance. Each measure of face processing performance was regressed with age and mean functional connectivity, the latter measured as the mean correlation among ROIs comprising the ‘obligatory-optional sub-network’ identified by either PLS_func or PLS_struc–func (‘Connectivity_func’ or ‘Connectivity_struc–func’, respectively; see text). Where a significant Sex*Connectivity effect emerged, the table gives the coefficients for males (n = 21) and females (n = 17) separately. Intercepts give values of the behavioural measure corresponding to the minimum value of functional connectivity (r = −0.13 and −0.22, respectively; see text). *P < 0.05; **P < 0.001.

^aThe corresponding measure of functional connectivity was entered as both a fixed and random effect.

Fig. 3.

Results of mixed-model regression analyses applied to the inverted-efficiency scores on the ‘Three Faces Test’ at all four visits. The trendline represents the significant effect of ‘Connectivity_struc–func’ separately for males and females (dashed and solid line, respectively); that is, the mean inter-regional correlation coefficient among all ROIs comprising the ‘obligatory-optional’ sub-network identified by PLS_struc–func (see text and Figure 2C). Table 3 presents the corresponding coefficients.

Fig. 4.

Developmental increases in functional connectivity between Visits 1 and 4. The image illustrates the percentage of adolescents at Visits 1 (top) and 4 (bottom) showing a pairwise correlation greater than r = 0.3 between pairs of distal ROIs that integrate the ‘obligatory’ (yellow) and ‘optional’ (green) nodes, as identified by PLS_struc–func. This demonstrates increases from Visit 1 to 4 in long-range functional connectivity between pairs of brain regions that express parallel developmental trajectories in structural and functional covariance: 1 = Left V2/V3; 2 = right V2/V3; 3 = left FFA; 4 = right FFA; 5 = right LOC; 6 = left pSTS; 7 = right pSTS; 8 = right aSTS; 9 = left MDL-PFC; 10 = left PMC; 11 = left MVL-PFC; 12 = right MVL-PFC.

CAT and structural covariance

To compare GM covariance between groups of adolescents defined according to behavioural performance, for each measure of face processing we identified those males and females with inverted-efficiency scores below the median on at least three of the four visits. As the same individuals scored consistently below the median on all three measures, we defined two groups of individuals with ‘High’ or ‘Low’ general face-processing ability. For each group, we then calculated at each visit the mean correlation coefficient for each of the 10 ROI pairs comprising the ‘obligatory-optional’ sub-network identified by the positive bootstrap values emerging from PLS_struc (see Figure 2A). After performing a Fisher r-to-z transformation on these pairwise correlations, they were entered into a repeated-measures ANOVA with the between-subject factors Sex (Male or Female) and Performance (High or Low), and within-subject factor Visit (1–4). As the pattern of negative bootstrap values emerging from PLS_struc comprised only three ROI pairs, we did not perform the same analysis on these pairwise correlations. Comparisons of GM covariance between ‘High’ and ‘Low’ adolescents revealed different visit-related patterns for each group; the main effects of Visit [F_(3,114) = 20.127, P < 0.001; ŋ²= 0.346] interacted strongly with Performance [F_(3,114) = 18.191, P < 0.001; ŋ²= 0.241], revealing a linear increase over visits for the Low group [F_(1,19) = 7.493, P = 0.013; ŋ²= 0.283] and a cubic Visit effect for the High group [F_(1,19) = 43.568, P < 0.001; ŋ²= 0.696]. Interestingly, this relationship appeared to differ between males and females; although non-significant, there was a strong trend towards a Visit*Sex [F_(3,108) = 2.589, P < 0.057] and Visit*Sex*Performance interaction [F_(3,108) = 2.568, P < 0.058]. These results are illustrated in Figure 5.

Fig. 5.

The relationship between structural covariance and face-processing performance. Bars present the mean (±SE) correlation coefficient for GM volume among all pairs of ROIs comprising the ‘obligatory-option’ network identified by PLS_struc. Mean coefficients are given separately for individuals scoring above or below the median on the majority of face-processing measures (see text). Although not significant, there were strong trends towards sex-related interactive effects. This figure illustrates a pattern of increasing GM covariance across visits, particularly for high-performing adolescents.

Discussion

In the present study, we explored developmental trajectories in brain–behaviour relationships in the context of face processing. To this end we combined longitudinal measures of face-processing performance with measures of both brain structure and function in a set of regions engaged reliably by face processing during adolescence, collected from a sample of young adolescents assessed four times between 10 and 15 years of age. To investigate developments in the integration of these neural systems, we examined developmental trajectories of structural covariance and functional connectivity across this sample. Applying a multivariate technique capable of capturing the covariance between all ROIs and across all visits simultaneously, we revealed three important findings: firstly, we observed sex-specific developmental increases in structural covariance among extra-striate and frontal cortices; GM partial-volumes decline in an increasingly co-ordinated fashion between these distal pairs of nodes. Second, we observed an increase over time in functional connectivity among an overlapping set of extra-striate and frontal cortices during passive observation of ambiguous facial expressions. Moreover, we demonstrate that the sex-specific trajectories of structural covariance parallel the developmental increases in functional covariance among a subset of these extrastriate (V2/V3, LOC, FFA and posterior STS) and frontal cortical regions (mid-dorsolateral frontal cortex and mid-ventrolateral frontal cortex, and PMC). Finally, our data point to relationships between the degree of structural covariance among these latter nodes, their functional connectivity during the processing of ambiguous facial expressions, and certain aspects of face-processing performance: greater functional and structural integration of extra-striate and frontal cortices over time appears related to improved discrimination between emotional facial expressions.

The sex-specific trajectories of structural and functional covariance we have observed within the adolescence face-processing network may reflect sexual dimorphisms in the structural and functional organization of the brain during this developmental period. As we were able to acquire only incomplete self-reported puberty data, we decided not to perform any formal assessments of the association between puberty and functional or structural covariance. Recent studies reveal that important associations may well exist, however, and may account for the pattern of results we have observed. Neuroimaging studies by Forbes et al. (2011) and Moore et al. (2012; see also Goddings et al., 2012) demonstrate changes in the brain response during emotional facial-expression processing between individuals in early and late pubertal stages. This is believed to reflect the influence of sex hormones on the brain during puberty, with gonadal steroid hormones exerting both organizational and activational influences on the structure and function of face-responsive brain systems (Scherf et al., 2012). Indeed, using the exact same dynamic face stimuli employed in the present study, Marecková et al. (2014) report that neural responses during face processing are modified by exogenous sex hormones. Such an association between puberty and structure–function integration should be the focus of future studies into face processing, and other aspects of socio-cognitive development.

Developmental increases in functional connectivity between the FFA and PMC during the observation of ambiguous facial expressions are particularly interesting in the context of brain-based models of face processing. Given the extent of their functional connectivity profiles, we have labelled these two structures as ‘obligatory’ and ‘optional’ nodes of the adolescent face-processing network, respectively. Similarly, these two structures belong, respectively, to the ‘core’ and ‘extended’ systems of the adult face-processing network (Haxby et al., 2000; Haxby and Gobbini, 2010). Although they are likely to play roles in many other cognitive processes, brain regions comprising the extended system are proposed to extract socially relevant information from dynamic facial expressions (Haxby and Gobbini, 2011) and the long-range integration of core and extended systems is considered necessary for facial-expression processing (Said et al., 2011). In this light, we consider their increased integration in response to ambiguous expressions during adolescence to index an important event behind the development of face processing (Fair et al., 2007; 2009).

The PMC has been studied extensively in action-observation research, revealing that this cortical region exhibits motor ‘resonance’ during the passive observation of actions performed by others (see Rizzolatti and Craighero, 2004). It is suggested that such resonance reflects an action observation–execution matching mechanism fundamental to a range of socio-cognitive capacities, from imitation to intention understanding (e.g. Rizzolatti and Fogassi, 2014; but see Hickok, 2009). Increased functional integration with PMC when presented with ambiguous facial expressions may reflect the emergence of a domain-general mechanism recruited during the encoding of ambiguous social action stimuli. Consistent with this notion, previously we have shown developmental changes during adolescence in functional connectivity between PMC and posterior parietal cortices during the observation of socially relevant hand actions (Shaw et al., 2011). Furthermore, in a previous study using the same dynamic face stimuli, we observed that the ambiguous facial expressions evoked a stronger brain response than angry expressions, particularly in females (Tahmasebi et al., 2011). Given the female advantage in processing facial expressions and sex differences in the number of fixations when scanning faces (Hall et al., 2010), we speculated that this reflects a deeper processing of the ambiguous expressions in a spontaneous effort to interpret them. Our present observations of sex-specific developmental trajectories for functional connectivity in response to these ambiguous-face stimuli provide further support in favour of this interpretation. In this light, the present findings may indicate that sex differences in facial-expression processing result from sexually dimorphic developments in the functional integration of these distributed neural systems.

Repeated co-activation between PMC and extra-striate ‘obligatory’ nodes (FFA, LOC) may also give rise to our observation of their increasing structural covariation, and the strong association between their developmental trajectories of structural and functional covariance. Parallel trajectories of structural and functional covariance among ‘obligatory’ and ‘optional’ nodes may reflect age-related increases in co-activations between these brain systems, leading to practice-induced changes in GM (see Draganski and May, 2008; Ilg et al., 2008; Taubert et al., 2010). Adolescence is recognized as a developmental period during which particular emphasis is placed on social information-processing skills. A shift towards peer-directed interactions, for example, requires an individual to understand and share in others’ mental and emotional states, for which facial expressions present important social cues. During adolescence, then, considerable demands are placed upon neural face-processing systems, especially those involved in encoding expressions that present socially ambiguous cues. This, in turn, may serve to integrate functionally the obligatory and optional systems, leading to their co-ordinated structural development. Such functional integration would be facilitated by the maturation of connecting white-matter pathways during this time (e.g. Scherf et al., 2014).

The directionality (and timing) of structure–function relationships during development can be viewed in two different ways: (i) from structure to function or (ii) function to structure (for related discussions see Osher et al., 2015; Paus, 2013; Scherf et al., 2012). We have suggested that both perspectives may go some way in explaining the developmental structure–function relationship observed in the present study: on one hand, the organizational effects of steroid hormones secreted during puberty might facilitate the functional integration of brain systems involved in face processing (see Schulz et al., 2009). Alternatively, increased functional connectivity between ‘obligatory’ and ‘optional’ nodes driven by greater exposure to ambiguous facial cues (see above) may result in their co-ordinated morphological development. Importantly, however, neither perspective would suggest that the long-range integration of the face-processing network ends after adolescence. Functional interactions are likely to change throughout life in response to changing social orientations and mental conditions (see Stuhrmann et al., 2011), and our social environment will likely exert experience-driven changes within any brain network involved in social interaction. Similarly, sex hormones will continue to exert effects on both brain structure and function throughout the life span. Indeed, this has been demonstrated specifically in the context of (female) face processing (see Marecková et al., 2014). As such, although they may emerge less pronounced in adulthood, changes in the structural and functional integration of the face-processing and other social brain networks are likely to be continuous processes.

Finally, our behavioural data provide additional support for the proposal that our fMRI measures index increased communication between specific nodes of the obligatory and optional face-processing systems. Facial expression perception is believed to require the integrated functioning of core and extended nodes of the adult face-processing network (see Said et al., 2011). We observed that the ability to discriminate between emotional facial expressions—but not face identity—is related to higher structural and functional covariance among the equivalent obligatory and optional systems of the adolescence brain. More importantly, this relationship appears stronger in females compared with males. The strength of this brain–behaviour relationship was moderate, failing to survive multiple-comparison correction. As such, this potential relationship requires replication before it can be considered further. Nevertheless, this finding might indicate that sex differences in processing facial expressions (for a review, see McClure, 2000) reflect sexually dimorphic developmental trajectories of the structural and functional integration of neural systems involved in face processing.

Supplementary data

Supplementary data are available at SCAN online.

Funding

This work was supported partly by the Sante Fe Institute Consortium and the Canadian Institutes of Health Research (T.P.) and partly by the project ‘CEITEC—Central European Institute of Technology’ (CZ.1.05/1.1.00/02.0068) from European Regional Development Fund (D.J.S).

Conflict of interest. None declared.