Optical imaging during toddlerhood: brain responses during naturalistic social interactions

Abstract.

Despite the importance of our ability to interact and communicate with others, the early development of the social brain network remains poorly understood. We examined brain activity in 12- to 14-month-old infants while they were interacting live with an adult in two different naturalistic social scenarios (i.e., reading a picture book versus singing nursery rhymes with gestures), as compared to baseline (i.e., showing infants a toy without eye contact or speech). We used functional near-infrared spectroscopy (fNIRS) recorded over the right temporal lobe of infants to assess the role of the superior temporal sulcus-temporoparietal junction (STS-TPJ) region during naturalistic social interactions. We observed increased cortical activation in the STS-TPJ region to live social stimuli in both socially engaging conditions compared to baseline during real life interaction, with greater activation evident for the joint attention (reading book) condition relative to the social nursery rhymes. These results supported the view that the STS-TPJ region, engaged in the cortical social brain network, is already specialized in infants for processing social signals and is sensitive to communicative situations. This study also highlighted the potential of fNIRS for studying brain function in infants entering toddlerhood during live social interaction.

1. Introduction

Social and linguistic input from others during infancy and childhood plays an important role in typical human development. Early social deprivation (e.g., low-resource orphanage care-giving) can result in specific cognitive and behavioral dysfunctions.^1–4 Typically, during mother–infant interactions, infants receive a rich variety of communicative signals from caregivers, such as eye contact, infant-directed speech, touch, and contingent responsiveness,^5–8 and they seem to be sensitive to those signals soon after birth.^9–11 Moreover, previous studies have found that the presence of such social signals during mother–infant interactions can enhance early language development.^12–16 Although preverbal infants might not understand the uses of social signals in the same way as older infants and toddlers do,^17–19 social interactions directly impact development from an early age.

Brain regions related to processing of a social stimulus have become known as the social brain network.²⁰ In human adults, lesion and functional neuroimaging studies have revealed that this network includes parts of the prefrontal cortex (PFC), including the orbitofrontal cortex, amygdala, fusiform face area, temporoparietal junction (TPJ), and superior-temporal sulcus (STS).^21,22 For example, Carrington and Bailey²³ reviewed functional neuroimaging studies examining theory of mind (ToM) skills and found that the PFC, STS, TPJ, and anterior cingulate cortex appeared to be core regions for ToM reasoning. With the aid of methodological advancements in infant neuroimaging studies, the ontogeny of the social brain network has also been investigated in recent years; we are beginning to understand how the social brain network becomes functional in early infancy.^24,25

Electroencephalography (EEG) studies have identified early social brain responses to human faces,^26–28 emotional expressions,²⁹ mutual gaze,⁹ biological motion,^30–32 and joint attention (JA)^33,34 with some evidencing immature responses compared to adults.^27,28 Moreover, functional near-infrared spectroscopy (fNIRS) studies, which allow for more precise spatial localization, have localized processing of these social signals in preverbal infants to the inferior frontal and STS-TPJ (which includes posterior regions of the superior-temporal gyrus, sulcus, middle temporal gyrus, and the TPJ) regions of the cortex.^31,35–44 For example, Otsuka et al.³⁹ measured 5- to 8-month-old infants’ brain responses with fNIRS and revealed that watching upright face stimuli, but not inverted face stimuli, resulted in activation in the right STS region. Lloyd-Fox et al.³⁷ found greater cortical activation, at 5 months of age, in the bilateral posterior temporal cortex in response to social dynamic stimuli, such as video clips of a female actor performing hand games (e.g., “peek-a-boo”), compared to nonsocial dynamic stimuli, such as video clips of a moving mechanical toy. Using NIRS, magnetic resonance imaging (MRI) coregistration data⁴⁵ have recently allowed us to more confidently localize these responses within the posterior STS-TPJ region. Furthermore, recent research has found that social signals related to auditory stimuli, such as mouth movements³¹ and vocalizations,^46–48 enhance activation in the anterior STS-TPJ region. These results suggested that young infants already have specialized areas in the temporal cortex for processing social stimuli.

In an effort to conduct strictly controlled studies, video stimuli are more commonly used than live stimuli to test social perception in infants. However, it is known that extracting information from video demonstrations can be more difficult for infants and toddlers than from live demonstrations.^49,50 For example, a behavioral study with 9- to 10-month-olds showed that infants learned a non-native phonetic contrast only from live exposure to a foreign language and not from televised exposure.⁵¹ Additionally, Shimada and Hiraki⁵² tested 6- to 7-month-old infants with fNIRS to see the difference between their cortical responses to live and televised actions. Though they found activation in the sensorimotor area during both live and televised action observation, the crucial contrast between activity to action and object-motion perception was only found in the live condition. Furthermore, an event-related potentials’ study revealed that infants responded to three-dimensional representations of objects more quickly than two-dimensional representations.⁵³

Since fNIRS is less susceptible to motion artifacts relative to functional MRI and EEG, it is an ideal imaging technique for studying infants in more ecologically valid settings. Recent fNIRS research has taken advantage of this attribute to study activity while infants are socially interacting with others.^41,42 Recently, Lloyd-Fox et al.⁴² studied 6-month-old infant responses to different combinations of ostensive social cues during live performances of nursery rhymes with gestures, in an infant-directed or adult-directed way. They found an additive effect of infant directed cues, such that the combination of direct gaze and infant-directed speech enhanced activation in bilateral STS-TPJ regions relative to direct gaze with adult directed speech and averted gaze with infant-directed speech.

In order to explore the role of the STS-TPJ region in infants further, we extended the complexity of the interactions and conducted a fNIRS study with an older cohort of 12- to 14-month-old infants during naturalistic play scenarios. We investigated infants’ brain responses to social signals during two different social interactions with an adult experimenter. In the interaction (I) condition, the infants viewed a female experimenter singing infant directed nursery rhymes (with accompanying hand gestures, such as “peek-a-boo”), with direct gaze and a positive emotional face (i.e., smiling, wide eyes). In the JA condition, the experimenter used infant directed gaze, speech, and gestures to look at a picture book. During the baseline period, the experimenter presented a toy to the infants without any social signals (with averted gaze and no speech). We measured brain activation across the infants’ right inferior frontal–posterior temporal cortex. First, we hypothesized that both socially engaging experimental conditions (I and JA) would show enhanced activation relative to baseline in this region, in particular, the anterior superior temporal gyrus–middle temporal gyrus (aSTG-MTG). Furthermore, given that the STS-TPJ region has been indicated to be involved in false belief tasks in older toddlers, children, and adults, we hypothesized that this region may be more heavily recruited during interactions, such as JA, which are thought to be precursors of ToM.⁵⁴ Such activity would indicate that by 12- to 14-month-old infants understand that there is an object of mutual interest requiring interpersonal interaction in the JA condition relative to the dyadic I condition.

2. Methods

2.1. Participants

Thirty healthy 12- to 14-month-old infants ($M=395.4$ days, $\mathit{SD}=23.5$, 20 males) participated in the study. We excluded four additional infants from the analysis due to experimental error ($\mathit{N}=3$) or heavy fussiness ($\mathit{N}=1$). Of 30 infants, two infants were excluded from the behavioral analysis, but not from the fNIRS analysis, because of experimental error, and nine infants were excluded from the fNIRS analysis but not from the behavioral analysis, due to signal quality problems caused by movements ($\mathit{N}=4$), an insufficient number of valid trials because of fussiness or inattention ($\mathit{N}=3$), experimental error ($\mathit{N}=1$), and failure of wearing the fNIRS headgear ($\mathit{N}=1$). Thus, 28 infants ($M=396.7$ days, $\mathit{SD}=23.8$, 19 males) were included in the final behavior analysis and 21 infants ($M=388.4$ days, $\mathit{SD}=19.7$, 11 males) were included in the final fNIRS analysis. For these 21 infants, the mean head circumference, the mean semicircumference from the left to the right ear via the top of head, and the mean semicircumference between the preauricular points via the forehead were 47.3 cm ($\mathit{SD}=1.4$), 26.9 cm ($\mathit{SD}=1.3$), and 22.9 cm ($\mathit{SD}=1.0$), respectively. All participants were from a volunteer database at the Centre for Brain and Cognitive Development and gave written informed consent before they participated in the study. The study protocol was approved by the Departmental Ethics Committee, Department of Psychological Science, Birkbeck, University of London (Reference Number: 131451).

2.2. Stimuli

Stimuli were presented in a live setting by a female experimenter, who sat in front of the infants throughout the experiment. There were two experimental conditions: the JA and I conditions (Fig. 1). Within the JA condition, the experimenter interacted with the infant using a picture book. She made frequent eye contact with the infant to provoke JA by eye gaze, she used gestures, such as pointing, as well as verbally labeling the pictures, and she displayed positive effect, with wide eyes and smiles. Within the I condition, the experimenter maintained eye contact and displayed positive effect, with wide eyes and smiles while either singing children’s songs with hand actions or talking and performing hand games, (there were four different songs/games that alternated across the session, and included “incy wincy spider” and “peek-a-boo”). During the baseline condition, the experimenter was silent, avoided eye contact and looked toward the toy, which she held between the infant and herself, and presented a selection of moving toys, which made quiet sounds. There were four different toys to maintain the infant’s attention during the baseline condition, and the experimenter picked one of them randomly for each trial. When the experimenter was not using the toys or book she placed them on a table next to her and out of reach but within view of the infant. A tone played every 20 s via speakers to indicate the onset and offset of each trial for experimenter 1 and experimenter 2, who placed event markers manually into the fNIRS recordings online. Four different researchers served as the “partner” in these real life interactions during the course of data collection for this study. Of 30 infants, 15 infants interacted with experimenter A, 8 infants interacted with experimenter B, 4 participants interacted with experimenter C, and a further three infants interacted with experimenter D. Prior to the study, the experimenter learnt a scripted set of instructions about how to interact during each of the conditions and reviewed training videos generated during piloting of the live study to mimic this “standard.” Instructions included the following: (1) natural transitions between trials so that when the end of the trial was signaled by the audio signal, any action or sentence was completed before finishing and moving to the next trial; (2) toys and books were kept on a table next to the experimenter and returned to position on completion of each trial; (3) during the I condition, they learnt four different songs or games (with speaking and hand actions) and alternated these across trials; (4) use infant-directed speech, eye contact, positive effect, and gestures during the experimental trials; (5) during the baseline condition, watch the object and help make it move/play sounds but avoid eye contact. Videos of the experimenter interactions were reviewed to ensure that these guidelines were followed. Data from two infants (outlined as experimental error above) were excluded from the analyses as it could be seen from the videos that they interacted with the infants and used direct eye contact during the baseline trials.

Fig. 1.

An infant participating in the study during the baseline and two experimental conditions (upper panel). The experimental protocol showing the order and timing of stimulus presentation for the two experimental conditions (lower panel).

2.3. Procedure

Within the same session, each infant went through three different fNIRS experiments: two video presented experiments⁴⁸ and one live experiment. After the two video studies investigating infants’ social perception and memory, we moved on to the live study, which is the focus of this study. At the beginning of the session, infants sat on their parent’s lap facing a female experimenter in a naturally lit room while the fNIRS headgear was quickly placed on their heads by experimenter 2. The parent was instructed not to interact with their infant during the stimulus presentation except when the infant became fussy or inattentive. The two types of experimental condition trials, JA and interaction (I), were presented in between baseline trials (Fig. 1). Each trial lasted 20 s. The order of experimental condition presentation was always JA, I, I, JA, I, JA, JA, I, JA, I in a repeating loop. A camera on a tripod placed on the left side of the infant recorded both the infant and the experimenter. When the infant became bored or fussy, as judged by the experimenter, or completed six trials for each experimental condition, we stopped presenting stimuli and ended the experiment.

2.4. Procedural Integrity

To ensure that the live demonstration was conducted consistently by the experimenters across infants, the first author and an independent observer scored procedural integrity from videotape on $\sim 25\%$ of the samples (two videos from each experimenter) using 20-s interval scoring. Note that experimenters contributed to a differing number of sessions (experimenter A: $N=10$, experimenter B: $N=6$, experimenter C: $N=2$, experimenter D: $N=3$). Cohen’s kappa⁵⁵ was 0.62, which was considered to be a substantial agreement.⁵⁶ The percent of trials in which the following behaviors were conducted correctly was infant-directed speech (experimenter A: 100%, experimenter B: 100%, experimenter C: 100%, and experimenter D: 100%), use of simple and short sentences (experimenter A: 100%, experimenter B: 100%, experimenter C: 100%, and experimenter D: 100%), positive expression (experimenter A: 100%, experimenter B: 100%, experimenter C: 94%, and experimenter D: 75%), and fluid action (experimenter A: 100%, experimenter B: 100%, experimenter C: 94%, and experimenter D: 75%). In addition, the experimenters’ mean proportions of fixation duration toward the infant’s face were counted for $\sim 25\%$ of the samples (the same two videos from each experimenter) and were 40.7% ($\mathit{SD}=18.2$) for the JA condition (experimenter A: $M=48.7\%$, SD = 9.9; experimenter B: $M=57.5\%$, $\mathit{SD}=9.3$; experimenter C: $M=35.9\%$, $\mathit{SD}=15.7$; experimenter D: $M=20.9\%$, $\mathit{SD}=12.7$) and 96.2% ($\mathit{SD}=4.5$) for the I condition (experimenter A: $M=95.7\%$, $\mathit{SD}=4.6$; experimenter B: $M=96.8\%$, $\mathit{SD}=6.9$; experimenter C: $M=96.5\%$, $\mathit{SD}=3.2$; experimenter D: $M=95.6\%$, $\mathit{SD}=3.3$). Given that experimenters C and D were seen to be looking at infants for a lower percentage of time during the JA condition relative to A and B, we reran the fNIRS analyses excluding the five infants, who observed these experimenters and found no overall change in the pattern of fNIRS activation across the condition-contrasts.

2.5. Data Recording and Processing

A University College London (UCL)-NIRS mini topography system was used.⁵⁷ The multichannel continuous-wave mini fNIRS system uses two wavelengths at 780 and 850 nm. Six light sources (per wavelength) and four detectors were arranged within custom-built fNIRS-CBCD headgear consisting of an array over the right temporal lobe with a total of 12 2-cm source–detector channels (Fig. 2). The midpoint of the lower row of channels was aligned with the preauricular point on the right hemisphere (scalp location T4, according to the 10–20 system) for each infant. Due to the limited number of sources and detectors of our fNIRS system, we chose to maximize coverage across one hemisphere overlaying inferior frontal to posterior temporal regions.

Fig. 2.

The fNIRS headgear and channel layout. An infant wearing the headgear with the location of the 10 to 20 coordinates F8, T4, and T6 overlaying the photo. T4 was located at the midpoint of the lower row of channels (the source optode between channel 4 and 7; upper panel). The array design showing the location of the channels (dashed circle), sources (star), and detectors (full circle; lower panel).

Prior to analyzing the behavioral data, trials were rejected from further analysis by looking time measures. Videos were coded offline by an experimenter: if an infant looked for $<20\%$ within a trial, the trial was considered invalid and not included in the final dataset. A minimum of two valid trials per condition was set as a threshold for inclusion within infants for the behavioral data. We could not obtain detailed looking time measures from two participants because we failed to video-record the session. Therefore, we excluded these two participants from the behavioral analysis. However, we included them in the fNIRS analyses as our secondary measure of looking time (online live coding to record whether infants attended for at least 50% of each trial to guide the second experimenter’s presentation of a sufficient number of trials during the session) indicated that the infants’ exceeded the threshold for inclusion in analyses. Note that reanalysis of the fNIRS data excluding these infants produced the same overall pattern of results.

Following this, for the behavioral data, two kinds of behaviors were selected as dependent variables: (i) the average fixation duration toward the experimenter’s face, hands (I condition), or object (JA and baseline condition), proportional to the length of the trial and (ii) frequencies of four different social behaviors (gaze reorienting, vocalizing, smiling, and pointing) per trial. For the gaze behavior, we counted the numbers of times the infant shifted their eye gaze between the experimenter’s face and the hand/object, and for the other three behaviors, we counted the number of times we observed these distinct actions for each trial. Vocalizations were counted as periods of vocalizations except negative voiced sounds, such as crying and whimpering. If the separation interval of voiced sounds was more than 1 s, the vocalizations were considered to be discrete. We then averaged this number across trials for each behavior. Each dependent variable was coded by a primary coder at 100-ms intervals using behavioral coding software (GenobsX, Tokyo, Japan).

For the fNIRS data, changes in ${\mathrm{HbO}}_2$ and HHb chromophore concentration ($\mu \mathrm{Mol}$) were calculated and used as hemodynamic indicators of neural activity.⁵⁸ Prior to analyzing the fNIRS data, trials, channels, or participant data were rejected from further analysis by (1) looking time measures (videos were coded offline by an experimenter: $<50\%$ of looking away within a trial for the experimental conditions and $<20\%$ of looking away for the baseline condition considered invalid) and (2) the quality of the intensity signals, using artifact detection algorithms.^37,59 In line with previous work, channels were excluded if the coefficient of variation of the attenuation exceeded 30% or if the normalized power was larger than 50% with respect to the total power.³⁷ Once the attenuation data were converted into changes in concentration (see below), trials were then assessed individually for artifact. Trials were removed if concentration changes during the stimulus trials exceeded $\pm 15\mu \mathrm{Mol}$ and during baseline from $-4$ to 0 s exceeded $\pm 3.5\mu \mathrm{Mol}$. This threshold was lenient and designed to ensure data was excluded due to abrupt changes in signal caused by motion rather than changes due to activation. For each infant, the trials and channels that survived these rejection criteria were entered into further analyses. Inclusion criteria required each channel to contain valid data in both experimental conditions. A minimum of three valid trials per experimental condition was set as a threshold for inclusion within infants, and the maximum number of rejected channels could not exceed one third of the total number of channels.

For each infant, the near-infrared intensity signal was low-pass filtered, using a cutoff frequency of 1.7 Hz. The data were then divided into blocks consisting of 4 s of the baseline trial prior to the onset of the 20 s experimental trial (the JA or I condition), plus the following 20-s baseline trial. This 44 s blocks of data were detrended with a linear fit between the first and last 4 s of the 44 s block and converted into changes in concentration in ${\mathrm{HbO}}_2$ and HHb using the modified Beer–Lambert law with an assumption of an age-appropriate differential pathlength factor of 5.13.⁶⁰ For each channel, valid experimental condition trials were then averaged together for each infant, and a time course of the mean change across all valid trials in ${\mathrm{HbO}}_2$ and HHb concentration was compiled for each experimental condition. Either a significant increase in ${\mathrm{HbO}}_2$ concentration or a significant decrease in HHb is commonly accepted as an indicator of cortical activation in infant research.⁵⁹ In an initial analysis, the grand averaged hemodynamic responses ($\mu \mathrm{Mol}$) of all infants were assessed for each of the two conditions (JA and I conditions). For each channel, the maximum change (or amplitude) in ${\mathrm{HbO}}_2$ (increase/decrease in chromophore concentration) and HHb (increase/decrease in chromophore concentration) in response to each experimental stimulus was compared with baseline. We followed the procedure of epoch analysis of previous research^42,48 to assess the maximum hemodynamic change for every 5-s period from 15 s after the experimental stimulus onset to 15 s after the onset of the baseline condition (i.e., 15 to 30 s). We assessed the response from 15 to 30 s to (a) account for the 5-s delay that we observed during video coding between the bleep (and parallel event marking of the data by experimenter 2) and the time taken for experimenter 1 to discard the toy and begin the next trial sequence by initiating joint eye contact, and to (b) investigate the latency of the peak responses seen across participants and channels during these live naturalistic interactions. Following analysis of the experimental condition versus baseline, channels that showed significant activation in at least one condition were entered into paired $t$-tests to compare the hemodynamic response during the specified time windows between the two conditions (JA versus I). During statistical analyses of the group data, only significant increases in ${\mathrm{HbO}}_2$ and decreases in HHb were reported, if the signals increased or decreased significantly in unison, the signal was considered inconsistent with a hemodynamic response to functional activation and not included [during analysis of the group, data significant increases in ${\mathrm{HbO}}_2$ and HHb were only evident in one channel (10) at one time epoch]. Channels that survived a multiple comparison correction [false-discovery rate (FDR)] are highlighted in Table 1.⁶¹ To our knowledge, this was the first study to explore fNIRS responses during live naturalistic interactions with infants entering toddlerhood, therefore, we included full analyses of all significant channels (prethresholding) in our results.

Table 1.

Channels with significant activation from baseline in the JA and I conditions; statistical tests were performed within different time windows. *$p<0.05$.

	Channel	Time window (s)	$t(20)$	$p$ (uncorrected)	$p$ (FDR-corrected)
${\mathrm{HbO}}_2$
JA > Baseline	5	15–20	3.23	0.004	0.030*
	5	20–25	3.92	0.001	0.014*
	5	25–30	3.79	0.001	0.014*
	6	25–30	2.99	0.007	0.042*
	7	20–25	2.20	0.040	0.143
	8	15–20	2.88	0.009	0.042*
	8	20–25	3.83	0.001	0.014*
	8	25–30	3.58	0.002	0.017*
	9	25–30	2.23	0.037	0.143
I > Baseline	5	15–20	3.02	0.007	0.075
	5	20–25	2.93	0.008	0.075
	5	25–30	3.30	0.004	0.075
	6	25–30	2.33	0.031	0.197
	8	25–30	3.15	0.005	0.075
	9	25–30	2.15	0.044	0.201
HHb
I > Baseline	1	20–25	$-2.38$	0.028	0.276
	2	20–25	$-2.32$	0.031	0.276
	2	25–30	$-2.72$	0.013	0.237
	3	25–30	$-3.22$	0.005	0.163
	4	25–30	$-2.22$	0.038	0.276

3. Results

3.1. Behavioral Data

The average number of valid trials, including baseline trials, was 20.9 ($\mathit{SD}=4.8$) with the mean duration of the recording session totaling 7.0 min ($\mathit{SD}=1.6$). The average number of valid trials was 5.1 ($\mathit{SD}=1.3$) for the JA condition and 4.9 ($\mathit{SD}=1.2$) for the I condition. Within these valid trials, two kinds of dependent variables were analyzed: (i) proportion of fixation durations toward the experimenter’s face versus to the hand/object and (ii) frequencies of four different social behaviors (gaze shifting, vocalizing, smiling, and pointing).

First, the mean proportion of looking duration to the stimuli in general (time on stimulus for both the face and the object combined) was 0.86 ($\mathit{SD}=0.09$) for the baseline condition, 0.89 ($\mathit{SD}=0.08$) for the JA condition, and 0.78 ($\mathit{SD}=0.12$) for the I condition. Therefore, overall average looking time to the trials across infants was 17.2 s for the baseline condition, 17.8 s for the JA condition, and 15.6 s for the I condition. The infants’ proportion of looking time was submitted into a two-way ANOVA with the stimulus condition (baseline/JA/interaction) and the time on stimulus (face/object-hand) as within-participants factors (Fig. 3). Mauchley’s test of sphericity revealed a violation of sphericity for the interaction effect between the stimulus condition and the time on stimulus. The Greenhouse–Geisser correction was therefore applied to adjust the degrees of freedom for the effect. Results revealed a significant main effect of the stimulus condition [$F(\mathrm{2,54})=13.001$, $p<0.001$, partial ${\eta }^2=0.325$] and the time on stimulus [$F(\mathrm{1,27})=145.136$, $p<0.001$, partial ${\eta }^2=0.843$)]. Additionally, a significant interaction between the stimulus condition and the time on stimulus [$F(1.61,43.57)=643.090$, $p<0.001$, partial ${\eta }^2=0.960$, Greenhouse–Geisser corrected] was also found. The posthoc $t$-tests with Bonferroni’s correction for the simple main effect showed that looking duration to the stimuli in general was shorter in the I condition than in the baseline ($p=0.003$) or JA conditions ($p=0.001$) while no significant difference in total looking duration was found between the baseline and JA conditions ($p=0.640$). Additionally, there were significant differences between the proportions of fixation durations to the face and the object in the baseline condition ($p<0.001$), in the JA condition ($p<0.001$), and in the I condition ($p<0.001$). The infants looked at the object significantly longer than the face during the baseline and JA conditions while they looked at the face longer than the hands in the I condition (Fig. 3).

Fig. 3.

Mean proportions of looking times to the experimenter’s face, hands (I condition) or object (JA and baseline condition) for each condition, proportional to the length of the trial. Error bars indicate 1 standard error of the mean ($N=28$). ***$p<0.001$.

Second, the frequencies of four different social behaviors were submitted into a two-way ANOVA with the stimulus condition (baseline/JA/interaction) and the behavior (gaze shifting/vocalizing/smiling/pointing) as within-participants factors (Fig. 4). Mauchley’s test of sphericity revealed a violation of sphericity for the effect of behavior and the interaction. The Greenhouse–Geisser correction was therefore applied to adjust the degrees of freedom for those effects. Results reveal no significant main effect of stimulus condition [$F(\mathrm{2,54})=1.447$, $p=0.244$, partial ${\eta }^2=0.051$] but a significant main effect of behavior [$F(1.82,49.13)=66.619$, $p<0.001$, partial ${\eta }^2=0.712$, Greenhouse–Geisser corrected] and a significant interaction between the stimulus condition and the behaviors [$F(2.55,68.90)=5.398$, $p=0.004$, partial ${\eta }^2=0.167$, Greenhouse–Geisser corrected]. The posthoc $t$-tests with Bonferroni’s correction for the simple main effect demonstrated that the frequency of gaze reorienting in the JA condition was higher than in the baseline condition ($p=0.008$) but not higher than in the I condition ($p=0.134$). In addition, the infants vocalized significantly longer during the baseline condition compared to the JA ($p=0.031$) or I conditions ($p=0.003$) and pointed longer during the baseline condition relative to the I condition ($p=0.021$).

Fig. 4.

Mean frequencies of four different social behaviors (gaze shifting, vocalizing, smiling, and pointing) per trial for each condition. Error bars indicate 1 standard error of the mean ($N=28$). **$p<0.01$, *$p<0.05$.

3.2. fNIRS Data

The average number of valid trials including baseline trials was 22.9 ($\mathit{SD}=2.7$) with the mean duration of the recording session totaling 7.7 min ($\mathit{SD}=0.9$). The average number of valid trials was 5.7 ($\mathit{SD}=0.8$) for the JA condition and 5.4 ($\mathit{SD}=0.8$) for the I condition. The proportion of invalid channels across the 21 infants was 0.012.

Due to the exploratory nature of this study, we included full analyses of all significant channels in Sec. 3 but highlight those with the strongest response (that survive FDR correction) in Table 1. A significant increase of ${\mathrm{HbO}}_2$ compared to baseline for the JA condition was found over five channels (channels 5, 6, 7, 8, and 9), while four channels revealed a significant increase of ${\mathrm{HbO}}_2$ for the I condition (channels 5, 6, 8, and 9; see Figs. 5 and 6). The onset and duration of the responses (see Table 1) were earliest and over a longer period in channels 5 and 8 for the JA condition (lasting from 15 to 30 s) and channel 5 for the I condition (lasting from 15 to 30 s). Interestingly, only channels in the JA condition survived the correction for multiple comparisons for this contrast to the low social baseline condition. With reference to a standardized scalp surface map of fNIRS channel coordinates and underlying anatomy for 7-month-old infants,⁴⁵ and head measurements for our participants, we approximated the location of the brain regions underlying these channels in this older age group. The channels with significant activation were clustered largely over the STS-TPJ regions of the cortex (channels 6, 8, and 9). Further two channels (5 and 7) with significant responses in the JA condition were located on the edge of this cluster, overlapping into middle temporal regions. Paired-sample channel-by-channel $t$-tests were performed (within those channels showing significant change from baseline) to directly compare the hemodynamic change observed in response to the two experimental conditions (JA versus I). This analysis found a significantly greater hemodynamic increase in ${\mathrm{HbO}}_2$ to the JA condition relative to the I condition in channel 8 [channel 8 at 20 to 25 s window: $t(20)=2.443$, $p=0.024$], indicating greater activation for the JA condition in this STS-TPJ region. No channels showed a significantly greater increase in ${\mathrm{HbO}}_2$ to the I condition compared to the JA condition.

Fig. 5.

An overview of the significant group effect for ${\mathrm{HbO}}_2$ (maximum increase in ${\mathrm{HbO}}_2$ concentration) highlighted with grey circles (FDR-corrected $p<0.05$) and white circles (uncorrected $p<0.05$) for the two contrasts: (a) JA condition versus baseline and (b) interaction condition versus baseline.

Fig. 6.

The grand averaged group time courses of the hemodynamic responses in ${\mathrm{HbO}}_2$ and HHb to the JA and I conditions. Light grayed area indicates the time window, where we found a significant increase of ${\mathrm{HbO}}_2$ compared to baseline for both the JA and I conditions. The dark grayed area indicates the time window, where the difference in ${\mathrm{HbO}}_2$ change was significant between the JA and I conditions. (a) Channel 5, (b) channel 6, (c) channel 8, and (d) channel 9.

Analyses of the HHb signal found a further four channels (1, 2, 3, and 4), located over inferior frontal regions, which showed significant decreases in HHb in response to the I condition. However, these latter responses should be treated with caution as the decreases in HHb were accompanied by nonsignificant but similar decreased time courses in ${\mathrm{HbO}}_2$ (see Appendix), demonstrating a profile suggestive of contamination by motion artifact but not to the level that our automatic algorithms would reject. For the HHb changes, there was no significant difference between the JA and I conditions within the epochs.

3.3. Relation between Behavioral and fNIRS Data

To further explore the relationship between the fNIRS condition specific responses and the behavioral findings, we investigated associations between the activation to the I and JA conditions in channel 8 (at 20- to 30-s window) and looking time measures. We chose to assess correlations in channel 8 as it is where we found significant differences in activation between the JA and I conditions and positioned approximately over the posterior part of the STS-TPJ region. First, we did not find that the average fixation duration during JA to the face [Pearson correlation; $r(19)=0.180$, $p=0.462$] or to the object [$r(19)=0.139$, $p=0.571$] significantly correlated with the degree of activation during JA. Therefore, it is unlikely that the $\mathrm{JA}>\mathrm{I}$ results can be explained by infants, who attended more to the face or book during the JA condition driving an increased response during the JA condition. Interestingly, we found a significant positive correlation between the average fixation duration toward the experimenter’s face [$r(19)=0.518$, $p=0.023$] during the I condition and changes in ${\mathrm{HbO}}_2$ in channel 8 during the JA condition (no effects were found for looks toward the object [$r(19)=0.071$, $p=0.772$]). This would suggest that the infants who attended to the experimenter’s face for longer in the I condition showed greater brain activation in the STS-TPJ region during the JA condition (Fig. 7). Also, no statistically significant correlation was found between the proportion of fixation durations toward the face during the I condition and changes in ${\mathrm{HbO}}_2$ in the STS-TPJ region during the I condition [$r(19)=0.236$, $p=0.330$].

Fig. 7.

Individual infant responses ($N=19$) of the association between looking time to the face in the I condition and ${\mathrm{HbO}}_2$ responses in channel 8 (the STS-TPJ region) in the (a) JA condition and (b) I condition.

4. Discussion

In the current study, we assessed 12- to 14-month-olds’ cortical responses during two different types of live communicative interactions with an adult (I condition: singing a nursery rhymes with gestures; JA condition: reading a picture book) to assess the role of the STS-TPJ region in the processing of social signals, such as mutual gaze, infant-directed speech, and contingent responsiveness in infants.

Our analyses of infant behavior evidenced significant differences between the JA and I conditions, thus showing that our stimulus conditions were interpreted differently by the infants. First, we found (1) that the proportion of looking time toward the experimenter’s face was greater than to the hands during the singing interaction (I) condition while the looking time to the face was lower than to the book during the book reading (JA) condition. Second, during analyses of the frequencies of gaze shifting, smiling, vocalizing, and pointing, we found that (2) that the frequency of infant gaze reorienting (or shifts) between the face and hands/object was significantly higher in the JA condition relative to the baseline; (3) that the frequency of vocalizations was higher in the baseline condition relative to the JA and I conditions; and (4) that the frequency of pointing gestures was higher in the baseline compared with the I condition.

Our analyses of the fNIRS data evidenced increased ${\mathrm{HbO}}_2$ responses to both experimental conditions (I and JA) in a STS-TPJ region of the cortex compared to the low-social baseline condition. Furthermore, the response was significantly stronger in the JA compared with the I condition in a channel positioned over the pSTS-TPJ region. While we had anticipated that looking time to the face would be lower during the JA condition relative to the I condition, given the second object of interest (book), we had not anticipated that infant’s would switch attention to such a high degree (on average looking to the face for less than 10% of the trial despite the experimenter being positioned directly opposite the infant). Therefore, the increased activation during the JA condition is not simply a reflection of observing more visual cues from the experimenter’s face. Rather, it suggests that the STS-TPJ region is highly responsive to the communicative context of JA while the infants were listening to the experimenter describing the book and pointing at the pictures. Furthermore, we found a relationship between the amount of time spent attending to the experimenter’s face during the singing (I) condition and the degree of activation during the JA condition. While further work is needed to replicate these findings in additional cohorts and disentangle the relative contribution of the different cues and contexts at this age, one interpretation could be that those infants more interested in attending to adult communication (during periods of directed attention during singing) may evidence more brain activation during other forms of more complex communication, such as the JA elicited during the sharing of a book. Our results are consistent with developmental models of social cognition, where JA precedes the onset of ToM and is a more advanced stage of social cognition involved with interpreting the cues before you, rather than simply observing them passively, such as during the singing condition.^62–64

We found one area of activation, over the aSTG-MTG, to show highly similar hemodynamic time courses (in channels 5 and 6) across both experimental social conditions. This likely represents voice-selective activation in the temporal cortex as the adult spoke in both experimental conditions relative to nonvocal sounds from the toys in the baseline condition. We know from previous research that adults and infants display increased responses in the anterior-middle STS region to human voice relative to nonvocal sounds.^{46,48,65–69} Indeed, on the same day, the infants in this study were also involved in a study of social visual and auditory (vocal) cues prior to taking part in this live session, and group data from the first study showed a similar localized pattern of activation to vocal > nonvocal sounds accompanied by visual social cues.⁴⁸ However, we believe that the observed responses cannot be wholly explained by voice-selective activation. First, the region of activation to the social experimental conditions in the current study extends into the posterior STS-TPJ areas, which have not previously been associated with vocal selectivity. Second, these findings are analogous to 6-month-old infant’s cortical responses in a previous live interaction study, which assessed cortical responses while watching a woman singing with gestures.⁴² They found increased activation in very similar regions of the brain to infant-directed singing compared to adult-directed singing, suggesting that these brain areas selectively activated to the type and context of the social cues.

In comparison to the majority of previous infant fNIRS studies, we found that the response of the hemodynamic response function was more sustained (particularly in channels 5 and 8 with significant activation found between 15 to 30 s after stimulus onset). While this response appears to be fairly late, note that this is due to the 5-s delay between the event marked beginning of the trial (marked in the acquisition software by experimenter 2) and the onset of experimenter 1’s change in interaction as we purposefully avoided unnatural transitions by asking experimenter 1 to finish the sentence, or action, that they were performing before changing conditions. Therefore, the window is in effect covering the last 10 s of the experimental trial and the first 5 s of the subsequent baseline trial. Interestingly, a previous fNIRS study with similar sustained and/or late occurring activation was the previous live naturalistic interaction study with 6 month olds,⁴² suggesting that the complexity of the stimuli or attention required may be enhanced in live contexts. While the current design attempted to mimic natural play (for example, the trials were semistructured in length, the objects and book were set down in view but out of reach of the infant on a table next to the adult before beginning the next trial) while controlling other variables across trials (gestures were used in the I condition during singing and the JA condition to point to pictures in the book), there were certain differences that could contribute to the current findings (such as reading versus singing), which could not be addressed here. Further research with longer more naturalistic communicative interaction sequences could help clarify the relative contribution of these different factors, and how the current responses relate to different social cues during naturalistic play, singing, and conversation. Furthermore, while overall measures of attention (looking time) were not predictive of the level of activation across the three conditions, we may in future work want to more fully address other factors, such as arousal by measuring skin conductance or heart rate across these longer sessions.

According to Cristia et al.,⁷⁰ most fNIRS studies are carried out on young infants, from neonates to 8-month-olds, and studies on infants older than 8 months are rarely performed. The authors discussed that this may reflect a limitation in the present neuroimaging techniques of testing older infants and toddlers since even fNIRS requires participants to maintain a certain degree of immobility and compliance. They also reported that the attrition rate tends to be higher in older infants (9 months or older) rather than younger infants. In the present study, we succeeded in collecting valid fNIRS data from 21 out of the 34 infants aged 12 to 14 months, resulting in an attrition rate of 38% (falling to 30% if we remove those excluded due to experimenter error). This attrition rate fell below the normal range for fNIRS studies with older infants and was even lower than the average attrition rate of 49% reported for infant ERP studies.⁷¹ This low attrition rate supports the use of fNIRS for measuring brain activity in older infants and toddlers using study designs involving interaction with live stimuli to enhance infant attention.

There are several caveats to the strength of our current conclusions, which should be addressed in future work. First, the limited number of sources and detectors of the mini-NIRS system we used in the study was only sufficient to design a single array and make measurements over inferior frontal to posterior temporal regions on one hemisphere, therefore, it would be highly beneficial to continue this research with a wider coverage of the cortex. However, we should note that there may be benefits to using a smaller array in live interactions as the contact of the headgear will be stronger and less affected by artifact from participant movement. Second, the current findings do not allow us to fully distinguish between the contribution of the different auditory and visual social cues, and so future research should use a more complex multifactor design to build on the current findings. Third, while we believe from previous research⁴² that this semistructured design was suitable and that it was constructed to be similar to natural interactions with caregivers (who often rapidly switch attention between their infant and other objects in their environment during play), it would be beneficial to extend the current work into longer sessions of naturalistic play and communication to disentangle how these responses interplay across different types of play and communicative contexts. Fourth, we need to explore the latency and shape of the hemodynamic responses within live contexts with more sensitive analyses to fully interrogate the contribution of different stimulus cues on the pattern of the responses seen.

Our study supports the view that the STS-TPJ region is already engaged in processing of communicative cues related to JA by the age of 12 to 14 months. Furthermore, the successful use of fNIRS in this semistructured design opens the possibility for future research conducted during more natural conversational and play sessions with caregivers, where components of child and adult led interaction and communication could be extracted and analyzed over longer periods of time while individuals are at the lab or at home. This is one of the first fNIRS studies to investigate cortical activation during naturalistic live interactions in infants entering toddlerhood^41,42 and highlights the potential of this technology in future basic and clinical science.

Biographies

Yoko Hakuno received her MSc degree from Birkbeck, University of London and is currently enrolled in PhD program in psychology at Keio University. Her research focuses on development of social brain functions in infants. She mainly studies early brain function using fNIRS during social interactions in naturalistic live settings.

Laura Pirazzoli is a PhD candidate at the Centre for Brain and Cognitive Development, Birkbeck, University of London. Her work focuses on investigating the mechanisms underlying the perception of affective touch in infants using fNIRS and electrocardiogram (ECG).

Anna Blasi is a research fellow at the Centre for Brain and Cognitive Development, Birkbeck, University of London. Her research interests are centered on functional aspects of human physiology. Her research career started with models of the cardiovascular system and the effects of disease. Through her work at UCL, KCL, and Birkbeck, her research interests have shifted toward the use of functional imaging (fNIRS, fMRI) to study brain function and neurocognitive development in early infancy.

Mark H. Johnson is a professor of experimental psychology and a head of the Department of Psychology at the University of Cambridge. He is also an associate director of the Centre for Brain and Cognitive Development, Birkbeck, University of London. He has published over 360 papers and 10 books on brain and cognitive development in human infants and other species. His laboratory currently focuses on typical and atypical functional brain development during infancy and childhood.

Sarah Lloyd-Fox is a research fellow at the Centre for Brain and Cognitive Development, Birkbeck, University of London and an affiliated lecturer at the University of Cambridge. Her work focuses on the early development and cortical specialization of the infant brain, and recently the application of fNIRS to the study of compromised development. The latter includes the study of infants at risk for autism, and recent work taking fNIRS to rural Africa to study undernutrition.

Appendix.

As supplementary data, time courses of hemoglobin responses to the JA and I conditions for all channels were shown in Fig. 8.

Fig. 8.

The grand averaged group time courses of the hemodynamic responses in ${\mathrm{HbO}}_2$ and HHb to the (a) JA and (b) I conditions for channels 1 to 12.

Disclosures

No conflicts of interest, financial or otherwise, are declared by the authors.