Emotional modulation of visual cortex activity: A functional near-infrared spectroscopy study

Abstract

Functional neuroimaging and electroencephalography reveal emotional effects in early visual cortex. Here, we used fNIRS to examine haemodynamic responses evoked by neutral, positive and negative emotional pictures, matched for brightness, contrast, hue, saturation, spatial frequency and entropy. Emotion content modulated amplitude and latency of oxy-, deoxy- and total haemoglobin response peaks, and induced peripheral autonomic reactions. The processing of positive and negative pictures enhanced haemodynamic response amplitude, and this effect was paralleled by blood pressure changes. The processing of positive pictures was reflected in reduced haemodynamic response peak latency. Together these data suggest early visual cortex holds amplitude-dependent representation of stimulus salience and latency-dependent information regarding stimulus valence, providing new insight into affective interaction with sensory processing.

Introduction

Functional magnetic resonance imaging (fMRI) studies indicate that activation across a set of brain regions including early visual areas is enhanced by the emotional content of visual stimuli [1,2]. This likely reflects an adaptive mechanism, accentuating early perceptual processing of motivationally salient items [3,4]. There is evidence that rapid cursory processing, mediated in part by subcortical routes to the amygdala, modulates more detailed but slower processing within visual cortices [5,6]. Anatomical studies demonstrate multiple hierarchical projections from the amygdala to sequential regions within the visual processing stream [7]. Further, event-related potential studies reveal effects of emotion on visual cortex activity as early as 240 ms after stimulus presentation [8]. The investigation of these putative feedback mechanisms can inform theories of interaction between emotion, attention and perception, and additionally may help to clarify the functional correlates of specific phobias [9,10].

Two recent studies explored the emotional modulation of visual cortex activity using functional near infrared spectroscopy (fNIRS) [11,12]. fNIRS is an optical method that permits detailed study of cortical haemodynamics through separately quantifying changes in oxy- and deoxyhaemoglobin concentration and dense temporal sampling of the haemodynamic response function (HRF) [13,14].

The two available fNIRS studies report preliminary data regarding emotional modulation of visual cortex activity, with reproducible emotional effects for positive but not for negative pictures, and inconsistent results for oxy- and deoxyhaemoglobin [11,12]. Here, we used an event-related design similar to these previous studies but optimized in three ways: 1) we matched emotional and neutral pictures on physical metrics of brightness, contrast, hue, saturation, spatial frequency and entropy, 2) we employed a long inter-stimulus interval to enable direct measurement of the haemodynamic response without any prior assumption on its time-course, 3) participants actively performed an active stimulus-rating task (rather than viewing the stimuli passively) and their ratings were used for fNIRS data analysis. Our primary aim was to use fNIRS to investigate how the haemodynamic responsein early visual areas is modulated by the emotional content of visual stimuli.

Several studies also highlight differential effects of emotional pictures on autonomic arousal, indexed by the skin conductance response, heart rate and blood pressure changes [15-17]. Correlations have been reported between systemic blood pressure and fNIRS signals attributed to local cortical haemodynamic changes [18-19], raising the possibility of a confounding effect of stimulus-evoked blood pressure change on fNIRS measurements [12]. We therefore also recorded parameters across a number of autonomic axes to corroborate the existing literature, and to investigate the relationship between fNIRS signals and evoked blood pressure changes.

Methods

Participants

Sixteen participants (9 female, 7 male), age 26.8±8.7 years (mean±SD) were enrolled after written informed consent. The study was approved by the Brighton & Sussex Medical School Ethics committee. All participants had corrected-to-normal vision, and all but one were right-handed. None was taking psychoactive medication.

Stimuli

One hundred and twenty pictures were selected from the International Affective Picture System database (IAPS, [20]) and grouped into three categories: neutral, negative and positive valence. According to the IAPS norms, valence was 5.1±0.9 for neutral, 2.9±0.9 for negative and 7.1±0.6 for positive pictures. Arousal was 3.0±0.5 for neutral, 5.9±0.7 for negative and 5.8±0.8 (matched with negative, P=0.7) for positive pictures. Neutral pictures consisted of mundane scenes and objects. Negative pictures included disgust and threat imagery. Positive pictures included aesthetically pleasing, endearing and exhilarating (e.g., action sports) content. Pictures were matched for brightness (P=0.4), contrast (P=0.8), hue (P=0.4), saturation (P=0.8), spatial frequency (P=0.5) and entropy (P=0.6).

The pictures occupied 30% of a 19″ screen, positioned at 1 m from the participant in a dark room. A central grey fixation cross was shown between pictures, which were presented for 2.5 s, followed by a 6 s delay before a tone prompting a response. The inter-trial interval was 13.1±2.2 s.

Participants were asked to concentrate on the emotion conveyed by each picture, and to decide on a rating only after the image disappeared. They rated the valence of the evoked feeling as very negative (1), mildly negative (2), neutral (3), mildly positive (4) or very positive (5).

Data acquisition

NIRS measurements were acquired with a continuous wave device, operating at 764±20 nm and 859±20 nm, sampling at 50 Hz (Oxymon Mk III, Artinis BV, Zetten, The Netherlands). Four channels were used, positioning one receiver optode over the Oz site (as defined by the 10/20 system) and four transmitter optodes 3.5 cm either side of the mid-sagittal plane, 2 cm above and below the receiver level. The transmitter-receiver distance was 4.1±0.3 cm. The resulting ‘banana-shaped’ wavepath, penetrating a depth of ≈2 cm, traversed a combination of striate and extrastriate regions; considering the positions of the optodes with respect to the occipital lobe, the probed area lies principally within the expected localization of visual areas V1 and V2 [13]. Changes in oxy- (O₂Hb), deoxy- (HHb) and total (THb) haemoglobin concentration were computed using the modified Beer-Lambert law, and the differential path length factor was estimated with Duncan's formula [13].

Skin conductance responses (SCR) were recorded from two Ag-AgCl electrodes applied on index and middle finger. Beat-to-beat arterial pressure was recorded by volume-clamping the finger pulse using a Finometer (Finapres Medical Systems BV, Arnhem, The Netherlands). Auto-calibration was synchronized with stimulus presentation and performed before each trial.

Data analysis

Using finger plethysmogram as the temporal reference, an average model of the cardiac pulsation-related artefact in the NIRS signals was obtained and subtracted. Low-pass filtering at 1 Hz and polynomial detrending were applied. Signals were then epoched between −2 s and 15 s with respect to stimulus onset, and the average pre-stimulus baseline (−2 to 0 s) was subtracted. Epochs were averaged according to picture type and participant rating, and the haemodynamic response peak amplitude and latency were determined.

The SCR signal was low-pass filtered at 1 Hz, polynomial detrended and then epoched as above. The peak amplitude in the 2-8 s post-stimulus range was determined. The arterial pressure signal was low-pass filtered at 10 Hz and pre-processed with a peak-picking algorithm yielding heart rate (HR) and mean arterial pressure (MAP) signals. These were polynomial detrended and then epoched between −3 s and 10 s. Average HR and MAP with respect to pre-stimulus values (−3 to 0 s) were computed for the 2-4 s, 4-6 s and 6-8 s post-stimulus windows.

Prior to statistical analysis, all physiological measurements were Z-normalized within participants. Factorial analyses (ANOVA) were performed, separately using picture type (Neu/Neg/Pos) and participant valence rating (1-5) factors. For NIRS measurements, interactions with optode side (left/right) and level (above/below Oz) were also investigated. All post-hoc comparisons were performed by means of Bonferroni-corrected two-tailed t-tests.

Linear regressions were performed across trials of each participant, for O₂Hb, HHb and THb peak amplitude and latency with respect to stimulus-evoked MAP change measured at the latency of the haemodynamic response peak. Factorial analyses on amplitude and latency were repeated after subtraction of the MAP-related component as determined by regression analysis.

Results

Behavioural findings and autonomic parameters

Participant valence ratings were 1.6±0.3, 3.1±0.2 and 3.9±0.4 (mean±SD) for negative, neutral and positive pictures respectively; all differences were significant (F(2,30)=324.7, P<0.001, ${\eta }_p^2=0.95$; P<0.001 for all post-hoc comparisons). Across participants, 19.5±6.7 pictures were rated as 1 (very negative), 20.8±6.0 as 2 (mildly negative), 36.3±11.8 as 3 (neutral), 26.9±6.3 as 4 (mildly positive) and 13.3±7.3 as 5 (very positive). There was no difference in response time, 800±290 ms, across picture types.

The average SCR was 0.2±0.2 μS. There was a main effect of picture type (F(2,30)=5.4, P=0.01, ${\eta }_p^2=0.28$) with a larger SCR to positive than neutral pictures (P=0.01). The effect of participant rating on SCR did not reach statistical significance.

On average, HR slowed after picture presentations: −1.3±1.1 bpm at 2-4 s, −1.3±1.8 bpm at 4-6 s and −0.7±1.4 bpm at 6-8 s. There was no effect of picture type, but there was an effect of participant rating at 4-6 s (F(4,60)=3.3, P=0.02, ${\eta }_p^2=0.23$) with greater slowing for pictures rated 2 than 5 (P=0.02), and at 6-8 s (F(4,60)=3.5, P=0.01, ${\eta }_p^2=0.24$) with greater slowing for 1 and 2 than 5 (P=0.02).

An overall decrease in MAP was also observed, −1.4±1.0 mmHg at 2-4 s, −1.3±1.4 mmHg at 4-6 s and −0.9±1.7 mmHg at 6-8 s. There was an effect of type at 4-6 s (F(2,30)=3.5, P=0.04, ${\eta }_p^2=0.25$) and a borderline effect at 6-8 s (F(2,30)=3.4, P=0.05, ${\eta }_p^2=0.24$), with greater decrease for neutral than positive (P=0.05). There was also an effect of rating at 6-8 s (F(4,60)=5.9, P<0.001, ${\eta }_p^2=0.35$), with greater decrease evoked by pictures rated 2 (P<0.001) and 3 (P=0.003) than 5.

The bar charts for SCR and the time-courses for HR and MAP are given in Fig. 1.

Fig. 1.

Bar charts of skin conductance responses (SCR, expressed in Z-scores), and time-courses of evoked changes in heart rate (HR, expressed in beats-per-minute) and mean arterial pressure (MAP, expressed in mmHg). The error bars represent standard error.

fNIRS haemodynamics

An increase in O₂Hb concentration was observed after picture presentations, with HRF peak amplitude 0.43±0.21 μM at 9.3±1.6 s after stimulus onset. There was a main effect of picture type on amplitude (F(2,30)=5, P=0.01, ${\eta }_p^2=0.25$) with greater increase for negative than neutral (P=0.01). There was also an effect of participant rating (F(4,60)=4.2, P=0.005, ${\eta }_p^2=0.22$), with greater increase for pictures rated 5 than 3 (P=0.002). There was an effect of type on latency (F(2,30)=5.6, P=0.01, ${\eta }_p^2=0.33$), with shorter latency for positive than negative (P=0.02) and neutral (P=0.03), and an effect of participant rating (F(4,60)=4.7, P=0.003, ${\eta }_p^2=0.30$), with shorter latency for 5 than 2 (P=0.003). No interactions with optode level and side were found.

Pictures evoked decreases in HHb concentration, with HRF peak amplitude −0.15±0.10 μM at 9.0±1.6 s. There was an effect of type on amplitude (F(2,30)=3.6, P=0.04, ${\eta }_p^2=0.19$), with greater decrease for positive than neutral (P=0.04). Similarly there was an effect of rating (F(4,60)=3.5, P=0.01, ${\eta }_p^2=0.19$), with greater decrease for 5 than 3 (P=0.006). There was an effect of type on latency (F(2,30)=7.4, P=0.004, ${\eta }_p^2=0.40$), with shorter latency for positive than negative (P=0.003); there was also an effect of rating (F(4,60)=4.8, P=0.002, ${\eta }_p^2=0.30$), with shorter latency for 5 than 1 and 2 (P=0.01). No interactions with optode level and side were found.

After picture presentations THb increased, with HRF peak amplitude 0.31±0.16 μM at 9.4±1.5 s. There was no effect of type on amplitude, however there was an effect of rating (F(4,60)=3.5, P=0.01, ${\eta }_p^2=0.19$), with greater increase for 5 than 3 (P=0.01). With latency, there was an effect of type (F(2,30)=3.6, P=0.004, ${\eta }_p^2=0.40$) and an effect of rating (F(4,60)=3, P=0.03, ${\eta }_p^2=0.21$), both without significant post-hoc comparisons. No interactions with optode level and side were found.

The time-courses of the evoked concentration changes are shown in Fig. 2, and the bar charts of peak amplitudes and latencies are given in Fig. 3.

Fig. 2.

Time courses of evoked changes in oxy- (O₂Hb), deoxy- (HHb) and total (THb) haemoglobin concentration (averaged over the four channels), expressed in micromolarities.

Fig. 3.

Bar charts of HRF peak amplitudes and latencies (averaged over the four channels), expressed in Z-scores, for oxy- (O₂Hb), deoxy- (HHb) and total (THb) haemoglobin. The error bars represent standard error.

Correlation between blood pressure and fNIRS haemodynamics

Linear regression of peak amplitudes with respect to MAP yielded correlation coefficients of 0.32±0.19 for O₂Hb, −0.26±0.14 for HHb and 0.28±0.19 for THb. After removal of this MAP-related component, main effects of picture type and participant ratings no longer reached significance, with the exception of effect of emotion type on O₂Hb (F(2,30)=3.8, P=0.04, ${\eta }_p^2=0.26$).

Correlation coefficients for peak latencies with respect to MAP were −0.25±0.11 for O₂Hb, −0.13±0.05 for HHb and −0.26±0.12 for THb. Here, after removing the MAP-related component the main effect of type on O₂Hb remained significant (F(2,30)=4.7, P=0.02, ${\eta }_p^2=0.30$), with shorter latency for positive than negative (P=0.04), and so did the main effect on HHb (F(2,30)=6.3, P=0.007, ${\eta }_p^2=0.36$), with a shorter latency for positive pictures (P=0.005). The effect of participant ratings was brought below significance threshold for O₂Hb, but remained for HHb (F(4,60)=3.4, P=0.02, ${\eta }_p^2=0.24$), albeit without significant post-hoc comparisons. No effect on THb survived.

Discussion

Our results confirm and extend earlier fNIRS observations reporting larger responses to positive, compared to neutral, pictures. This effect is not due to perceptual features of the stimuli, and was accompanied by stimulus-evoked changes in autonomic state [11,12].

Autonomic arousal followed a similar pattern to the haemodynamic effects in the brain, with significantly larger sympathetic responses, as indexed by SCR, to positive than neutral stimuli [21]. Although negative and positive pictures were matched for arousal ratings using the IAPS norms, the SCR for negative pictures was intermediate between that obtained for positive and neutral stimuli [16]. On average, mean arterial pressure (MAP) decreased transiently in response to the pictures. This decrease was attenuated for positive stimuli, and those pictures rated as very positive (5) even evoked MAP increases. A similar effect was observed in heart rate, with an overall post-stimulus bradycardia, but pictures rated as very positive (5) elicited cardiac acceleration. Our findings confirm and extend earlier psychophysiological observations [15-17] and, speculatively, suggest competing interaction between post-stimulus decreases in HR and MAP facilitating stimulus processing [21] and physiological arousal facilitating emotive behaviour.

These evoked bodily responses originate in differential neural processing of the visual stimuli, which we indexed using fNIRS. We showed enhanced responses to emotional, compared to neutral pictures and determined post-hoc using participant ratings that the haemodynamic effects were particularly driven by pictures inducing very positive (5) feelings, as was the case with autonomic measures. Differences between negative and neutral pictures were found only for O₂Hb. Further, there were no differences between negative and positive pictures in response amplitude, which followed similar trends (Fig. 2 and 3). These findings suggest that emotional modulation of the magnitude of visual cortex activity is largely independent of emotion type [1] and mediated by attentional processes more closely related to arousal than valence [6,22].

In line with previous work, we found correlations between the amplitude of the haemodynamic response and MAP changes [18,19]. These differential occipital cortex responses and autonomic changes are likely to share a common effector, namely the neural processing of emotional content of stimuli involving the amygdala [5-7]. Most of the differences in haemodynamic response amplitude could be accounted for by including MAP changes in the model, therefore our data do not enable us to exclude outright a more extreme interpretation, namely that amplitude effects are caused by MAP changes. Circumstantially, this interpretation is improbable, as illustrated by previous studies showing directly emotional modulation of electrical measures of visual cortex activity [8]. The dynamic relationship between changes in MAP and cerebral circulation, involving specific regulation mechanisms and local autonomic innervation, is not straightforward [23]. In fact, a recent study has demonstrated that, while systemic cardiovascular effects correlate with haemodynamic changes in the visual cortex when both are induced by visual stimuli, auditory stimuli producing similar cardiovascular effects do not lead to haemodynamic changes in the visual cortex [24].

A novel finding of our study is that the latency of the haemodynamic response peak is modulated by emotion type, with shorter latency for positive compared to negative and neutral stimuli. This latency effect suggests a difference in dynamics of neural activation and has implications for understanding how positive information is recognised and processed. The detailed nature of this effect requires further investigation, since the timing of the haemodynamic response does not necessarily reflect that of underlying neural activity [25]. Importantly, differences in response latency remained significant after taking into account evoked MAP changes. This indicates that they index neural mechanisms distinguishable from those underlying autonomic arousal, and provides further reassurance that haemodynamic indices of visual cortical activity are not simply mirroring systemic changes.

Interestingly, the ability to the study emotional modulation of visual processing with fNIRS opens the way to potential neurofeedback applications, for example aiming to attenuate the response to specific items [9-10].

A limitation of the present study is that structural and functional MRI scans were not performed to confirm the origin of the evoked visual cortex responses, and to verify the exact position of the optodes with respect to the cortex. Further, the relationship observed between evoked changes in MAP and amplitude of haemodynamic response requires further characterization, for example independently manipulating MAP, to quantitatively model the relationship between the two parameters.

Conclusions

Emotional effects in early visual cortex are detectable with fNIRS. Both amplitude and latency of the haemodynamic response reflect emotional content of pictures, with an increased response amplitude to positive and negative relative to neutral pictures. Moreover there was a specific reduction in the latency of the peak haemodynamic response to positively-valenced pictures. These effects were not due to low-level perceptual features. Amplitude effects could be accounted for by evoked changes in blood pressure, suggesting a common effector or a causal link, but latency effects remained significant when blood pressure changes were taken into account. Our findings extend understanding of emotional modulation of early perceptual processing, highlighting salience-related amplitude effects (coupled to systemic arousal) and novel valence-related latency effects (dissociable from systemic arousal) identified through detailed characterisation of components haemodynamic response.