Monitoring and control of amygdala neurofeedback involves distributed information processing in the human brain

Abstract

Brain–computer interfaces provide conscious access to neural activity by means of brain‐derived feedback (“neurofeedback”). An individual's abilities to monitor and control feedback are two necessary processes for effective neurofeedback therapy, yet their underlying functional neuroanatomy is still being debated. In this study, healthy subjects received visual feedback from their amygdala response to negative pictures. Activation and functional connectivity were analyzed to disentangle the role of brain regions in different processes. Feedback monitoring was mapped to the thalamus, ventromedial prefrontal cortex (vmPFC), ventral striatum (VS), and rostral PFC. The VS responded to feedback corresponding to instructions while rPFC activity differentiated between conditions and predicted amygdala regulation. Control involved the lateral PFC, anterior cingulate, and insula. Monitoring and control activity overlapped in the VS and thalamus. Extending current neural models of neurofeedback, this study introduces monitoring and control of feedback as anatomically dissociated processes, and suggests their important role in voluntary neuromodulation.

1. INTRODUCTION

A brain–computer interface (BCI) provides an “external” feedback loop for the subject, and produces directly observable information about brain activity (Figure 1a). Dubbed “neurofeedback,” the technique has recently become a topic of broad interest as it has turned out to be instrumental for improving voluntary regulation of circumscribed brain regions (Sulzer et al., 2013). Neurofeedback has been shown instrumental to improve self‐control of dysregulated brain activations in psychiatric disorders such as depression (Young et al., 2017a), was used for rehabilitation purposes in neurological disorders (Chaudhary et al., 2017) and is a powerful research method in behavioral neuroscience (Shibata et al., 2011). Learning to control neurofeedback is driven by the contingent reinforcement of behavior, a mechanism termed “operant conditioning” (Caria, 2016; Strehl, 2014). In the course of training, subjects can identify mental strategies that are effective for neurofeedback control. Furthermore, subjects may sharpen their perception of intrinsic processes that are correlated with feedback via associative learning (Kotchoubey et al., 2002; Schwartz et al., 2016). A necessary precondition for both types of learning is feedback monitoring while operant conditioning also involves control processes. The brain mechanisms underlying monitoring and control of neurofeedback are less well understood. For a detailed characterization, differentiation between brain responses driven by the feedback stimulus and brain activity driven by other events, such as intrinsic or other extrinsic stimuli is needed. Instead, most studies use statistical models that assume continuous tonic brain activation during the trial to analyze data (such as an on–off “boxcar” response model) (e.g., Caria et al., 2007; Emmert et al., 2016; Linden et al., 2012; Paret et al., 2014; Veit et al., 2012; Zilverstand et al., 2015). As a consequence, specific responses to feedback are either missed or cannot be distinguished from unspecific activations. Initial attempts were made to identify brain responses to neurofeedback with real‐time functional magnetic resonance imaging (rtfMRI) of the blood oxygenation level dependent‐(BOLD) signal; Ramot et al. (2016) found responses to acoustic reward in the ventral striatum (VS) in subjects who received feedback from visual brain regions. Radua et al. (2016) found deactivation of the ventromedial prefrontal cortex (vmPFC) in response to changes in the trajectory of a diagram, reflecting success in the regulation of the vmPFC/orbitofrontal cortex (OFC). While reward or success signals are essential for learning, they are just two types from a larger pool of feedback‐related information signals. Learning requires processing of neurofeedback with regard to several criteria: general activity of feedback needs monitoring to assure processing of new information as soon as it arrives (“feedback activity monitoring”) and feedback needs categorization with regard to congruency with the desired outcome. For instance, it means something different when the temperature of a thermometer indicating brain activation rises if you are expected to up‐ or downregulate. Hence, the context in which feedback is received is important. An increase in temperature is rewarding when the goal is to upregulate and punishing when it is to downregulate. That is, subjects need to consider the task context (“feedback context monitoring”) and the congruency of feedback with the desired outcome (“feedback congruency monitoring”). A systematic investigation of monitoring patterns is currently lacking.

Figure 1.

(a) Illustration of neurofeedback loop. Brain activation is recorded with functional magnetic resonance imaging (fMRI). As exemplified in this study, activation from a region of interest encompassing the amygdala (Amy) can be used to receive a feedback signal. Data are processed in real‐time and visualized via a neurofeedback (NF) display such as a thermometer indicating increase and decrease of the brain signal. The subject observes the display and uses the signal to improve control of the target brain signal. (b) Experimental design. (c) Illustration of analysis method. Feedback signal‐changes were identified based on each participant's feedback time series (first row shows an exemplary feedback time series). Event‐related stimulus functions (i.e., regressors) were built for increases (middle row) and decreases (lower row) of the feedback‐thermometer's “temperature” and were convolved with a hemodynamic response function. The stimulus functions were used for a statistical parametric mapping analysis on the first level. (d) In the monitoring analysis, we looked at event‐related brain responses to feedback (i.e., rise and fall of the thermometer's temperature) to address the question: “how does the brain respond when the thermometer rises/drops?” The instruction (i.e., whether subjects were expected to up‐ or downregulate) establishes a context in which feedback is monitored. A rise of the thermometer has a different meaning if the instruction was to upregulate the thermometer (i.e., a success signal) than if it was to downregulate (in this case this would be a regulation “failure” signal). In addition to the monitoring analysis, we performed an analysis of tonic brain responses to pictures in the conditions “up,” “down,” and “view” with an on–off “boxcar” response model, and call the latter the “feedback control” analysis. For the feedback monitoring analysis, feedback stimulus functions were estimated separately for each condition. This resulted in a 2‐by‐2 (instruction “up,” “down” × feedback “up,” “down”) factor design. Three factor combinations were used for analysis: feedback activity monitoring is suited to detect brain regions showing a general and unspecific responsivity to activity in the thermometer; feedback congruency monitoring detects response patterns in line with the reinforcer value of feedback (i.e., feedback up is congruent with the instruction to upregulate, and in turn, feedback down is congruent with the instruction to downregulate); feedback context monitoring models brain responses to feedback that differ with regard to the task‐context (i.e., instruction to up‐ versus downregulate)

Usually, researchers contrast brain activation in neurofeedback trials to trials without the instruction to control by using an on–off response model. A straightforward interpretation relates significant activation to feedback control. As we argued above, this approach does not allow disentanglement of feedback control from feedback‐driven responses or, in other terms, feedback monitoring. We are not aware of studies comparing brain activation involved in feedback monitoring and feedback control.

Using rtfMRI neurofeedback, it has become possible to unravel the neural mechanism with high spatial resolution. From a literature review, Sitaram et al. (2017) suggested three separable neural neurofeedback networks. In their model, feedback control was implemented by the dorsolateral prefrontal cortex (dlPFC), thalamus, posterior parietal cortex (pPC), and lateral occipital cortex (lOC). These regions are thought to be generally active when feedback is presented visually. While dlPFC and pPC were suggested to be involved in executive tasks, lOC was thought to control attention to visual feedback. The dorsal striatum was assumed to be a key region for neurofeedback learning, and the ventral striatum (VS), anterior cingulate cortex (ACC), and anterior insula (AI) were postulated to be related to reward processing. Clarification of the brain regions involved in feedback control and feedback monitoring is needed and would serve as an empirical test for Sitaram et al.'s model. To fill this knowledge gap, we adapted an rtfMRI amygdala neurofeedback protocol established by our group (Paret et al., 2014). Previous research has shown that amygdala regulation with neurofeedback is feasible (Brühl et al., 2014; Marxen et al., 2016; Paret et al., 2014; Zotev et al., 2011) and can improve the regulation of emotions in patients suffering from affective dysregulation (Gerin et al., 2016; Keynan et al., 2016; Nicholson et al., 2016; Paret et al., 2016a; Young et al., 2017a, 2017b, 2014; Yuan et al., 2014). Emotion regulation is associated with specific control of amygdala activation (Buhle et al., 2013) and involves the dlPFC (Kalisch, 2009), whereas the monitoring of affect involves the ACC and medial PFC (Etkin et al., 2011). The valuation of emotional stimuli is linked to the VS (for an overview, see Ochsner and Gross, 2014).

We presented pictures with negative affective scenes to healthy female volunteers during fMRI. Feedback from the amygdala BOLD signal was visually presented via a thermometer located to the left and right of the pictures. Subjects were instructed to up‐ and downregulate the thermometer's temperature in consecutive blocks (Figure 1b) or view pictures in a baseline condition without feedback. This procedure forced subjects to flexibly adjust regulation goals and allowed an analysis of feedback context monitoring and feedback congruency monitoring. We used statistical parametric mapping (SPM) to identify brain responses to feedback (Figure 1c,d). To analyze feedback control, brain responses to pictures were compared between “up,” “down,” and “view” conditions.

We expected that monitoring and control would involve different neural circuits. Furthermore, we hypothesized that feedback transmits different informational aspects that invoke different neural circuits. We expected to observe key brain regions by analyzing response patterns that are characteristic for feedback activity monitoring, feedback congruency monitoring, and feedback context monitoring.

2. METHODS

2.1. Sample

Twenty right‐handed healthy women participated in this study (age: 24.57 ± 4.45; mean ± SD). Participants were selected via an in‐house data base or responded to advertisements in social media. Prior to participation, subjects were screened by a clinical psychologist in a telephone interview and were included if they did not report any symptoms of a psychiatric disorder according to the Diagnostic Manual of Mental Disorders, fourth edition (DSM‐IV). The study was approved by the Ethics Committee of the Medical Faculty Mannheim of the University of Heidelberg, and all subjects provided written informed consent before participation. Participants were financially compensated.

2.2. Procedure

2.2.1. Preparation and instructions

All participants were instructed that they would see pictures together with a feedback signal from their brains and should try to decrease and increase brain activation. Furthermore, they were told to consider the temporal delay of the BOLD response, which results in a time lag of the thermometer's response to regulation. Additionally, they were instructed not to close their eyes or keep their gaze adverted from the screen, not to focus exclusively on the thermometer and the borders of the picture, and not to hold their breath or move their heads. Participants then entered the scanner, and their heads were slightly restrained to minimize movements during scanning. Anatomical and fieldmap scans were acquired.

2.2.2. Neurofeedback experiment

Before the neurofeedback run started, a demonstration trial was presented without fMRI scanning. The neurofeedback run consisted of alternating “view,” “up,” and “down” condition blocks (Figure 1b). Subjects were instructed beforehand to either look at the picture (without getting feedback) or to up‐ or downregulate the thermometer's “temperature.” Conditions were cued with white arrows pointing up or down and with a white dot to indicate the view condition. During feedback, the arrow or the dot were displayed with the thermometer such that the symbols were partly hidden but recognizable when the thermometer's temperature was in the middle range. In the view‐condition, a picture with negative emotional content was shown for 18 s followed by a fixation cross on grey background (“rest,” 12 s). In the other conditions, the pictures were presented with feedback. Three pictures were presented in a block, each for 18 s (54 s total). The order of conditions was fixed, starting with “view,” followed by “rest,” “up”/“down,” “rest,” “down”/“up,” and “rest.” Half of the subjects began with “up” and the other half with “down.” The condition sequence was repeated five times. After the last “rest” period, participants were instructed to rate their perceived regulation success (“Were you able to regulate the display?”) on a 10‐level visual analogue scale.

2.2.3. Stimuli

Stimuli1 were taken from standardized picture series (Dan‐Glauser and Scherer, 2011; Lang et al., 2008; Marchewka et al., 2014; Wessa et al., 2010) and were presented with the Presentation software (Neurobehavioral Systems, Berkeley, CA) in randomized order. Stimuli were presented via a computer screen in the back of the scanner that was visible via a semi‐transparent mirror. After completion of the experiment, subjects rated arousal and valence of pictures outside the MRI suite on a 9‐point Likert scale.

2.2.4. Eye‐tracking

The left eye was monitored with an infrared (IR) video camera (60 Hz, MRC Systems, Heidelberg, Germany) and gaze data were collected with IViewX (2.6, SMI, Teltow, Germany). The camera was hidden behind a mirror which was transparent for IR radiation. Participants were informed that their eye was video‐recorded. The experiment started with a 13‐point calibration.

2.2.5. fMRI acquisition

For brain imaging, a 3 T MRI Scanner (Magnetom Trio, Siemens Healthineers, Erlangen, Germany) was used, equipped with a 32‐channel head coil. Functional images of the BOLD‐contrast were acquired with a gradient echo T2*‐weighted echo‐planar‐imaging (EPI) sequence. For neurofeedback, a fast sequence with repetition time TR = 1 s was used (TE = 30 ms, FOV = 192 × 192 × 71 mm, flip angle = 60°) with 18 slices and 3 × 3 × 3 mm voxel resolution (slice gap = 1 mm). TE was minimized using a parallel acquisition technique (generalized autocalibrating partially parallel acquisitions [GRAPPA]) with an acceleration factor of 2 and 24 reference lines. The FOV was fitted to include amygdala, temporal lobes and large parts of medial prefrontal cortex (Supporting Information, Figure S1). 839 volumes were acquired, angulated −20° relative to the AC‐PC plane. Anatomy was imaged with a 3D T1‐weighted scan (Magnetization Prepared Rapid Acquisition Gradient Echo [MPRAGE] sequence, TE = 3.03 ms, TR = 2.3 s, 192 slices and FOV = 256 × 256 × 192 mm). Phase and magnitude images of the magnetic field were recorded with a fieldmap scan before each EPI scan.

2.3. Analysis of eye‐recordings

Data were analyzed with BeGaze software (3.6, SMI, Teltow, Germany). Fixations were defined as recordings of duration >80 ms within a radius <100 pixels. We defined two areas of interest (AOI), corresponding to the position of the thermometers and the picture on the screen. The dwell time was defined as the sum of all fixations and saccades within an AOI and was quantified for each picture presentation (trial = 18 s) and normalized for AOI size (time (s)/area (% screen coverage)). Trials with a viewing duration (fixation duration + saccade duration) < half of picture duration (9 s) were excluded. In one subject, none of the view‐trials survived exclusion criteria, leading to exclusion from statistical comparisons with this condition. The average tracking ratio was satisfying (90.74% ± 5.97).

We expected participants to spend more time looking at pictures versus thermometers when regulating neurofeedback based on their instructions. This was tested with a condition (“up” versus “down”) × AOI (“picture” versus “thermometer”) repeated‐measures analysis of variance (ANOVA). Statistical significance was defined at the p = .05 level.

2.4. Real‐time fMRI analysis and feedback presentation

2.4.1. Region of interest delineation for feedback signal extraction

After image reconstruction at the scanner, volumes were directly transferred to a PC for preprocessing and analysis with SPM8 (Wellcome Department of Cognitive Neurology, London, UK). Prior to the functional protocol the T1‐weighted scan was segmented and normalized to Montreal Neurological Imaging (MNI) space. Next, the anatomical masks of the ROIs for feedback calculation were transformed to subject‐native space. For further processing, the BOLD signal data from voxels within a right amygdala mask, produced with the Harvard–Oxford brain atlas with a probability threshold of 25%, was taken. Each subject received feedback from the same ROI. In addition, BOLD signal data from a rectangular plane ROI (3 × 30 × 30 mm in AC–PC orientation, center of mass = [0,−16,−5], MNI coordinates) was recorded. This ROI served as a control for global signal fluctuations, unrelated to functional brain activation (see Section 2.4.2 for more information). The perpendicular distance from the control ROI to the amygdala was 7 mm in the sagittal direction (Supporting Information, Figure S2).

2.4.2. Preprocessing of functional scans and quantification of BOLD signals

Functional images were realigned to the first image. BOLD signal data from all voxels within each ROI were averaged, the average time course was processed with a modified Kalman filter (Koush et al., 2012) and afterwards detrended with Matlab's (R2014b) detrend function. Detrending started with the 35th volume (i.e., before subjects received any feedback) to allow stabilization of the filter and detrend functions. After filtering and detrending, percent signal change from the global mean was calculated. The signal change difference in the ROIs was used for feedback:

$${\text{Feedback}}_{\mathrm{i}}=\mathrm{d}\hspace{0.17em}\text{etrend}\left(\text{filter}\right(\frac{{\mathrm{X}}_{\text{amygdala},\mathrm{i}}\ast 100}{{\text{mean}(\mathrm{X}}_{\text{amygdala},1\to \mathrm{i}})}\left)\right)-\text{detrend}(\text{filter}(\frac{{\mathrm{X}}_{\text{control},\mathrm{i}}\ast 100}{{\text{mean}(\mathrm{X}}_{\text{control},1\to \mathrm{i}})}\left)\right)$$

where X _j,i is the most recent BOLD signal value received at ROI j.

2.4.3. Feedback presentation

On another computer, stimulus presentation software obtained the score data via TCP/IP. The feedback display was refreshed with each data bin arriving (update frequency = 1 Hz). Feedback was displayed as a colored rectangle moving up and down and changing from dark red at the maximum over light green in the middle to dark green at the minimum. The display had a resolution of six levels (corresponding to six bars in the feedback‐thermometer) and showed variations within a range of two percent signal change above and below baseline.

2.5. Postexperimental fMRI analysis

2.5.1. Preprocessing

The Matlab (vR2012a)‐based SPM12 package (v6225, Wellcome Department of Cognitive Neurology, London, UK) was used for data analysis. Slice timing was corrected with reference to the middle slice of a volume. With a rigid body transformation, three linear and three rotational parameters were estimated for realignment of the scans to the first of the series. To correct for geometric distortions, a voxel displacement map (VDM) was produced based on fieldmap scans. The functional scans were unwarped using the VDM parameters and corrected for susceptibility‐by‐movement artifacts (Andersson et al., 2001). A mean image of the functional scans was computed and coregistered to the anatomical scan of the subject. The anatomical scan was segmented with use of six standard SPM tissue probability maps and normalized to MNI space. The derived normalization parameters were used for normalization of functional images and images were resampled to 2 mm isometric voxels. Finally, functional data were smoothed using an 8 mm kernel (full width at half maximum, FWHM) to account for between‐subject variation in anatomical localization.

2.5.2. Specificity of feedback

Activation was modeled with a numerical vector representing thermometer “temperature,” ranging from 1 to 6. After z‐standardization, the time series was used as regressor (without convolution with the hemodynamic response function [HRF]) for modeling the preprocessed data. A high‐pass filter (HPF = 256 s) was added to the GLM to remove slow signal drifts, and serial correlations were accounted for using an autoregressive (AR[1]) model. Contrast values from SPM's single‐subject t‐contrasts were analyzed at the group level.

2.5.3. Analysis of the amygdala response

To investigate the hemodynamic amygdala response, HRFs were estimated with the inverse logit model presented by Lindquist et al. (2009). First, the eigenvariate was extracted from voxels corresponding to the right amygdala, selected with the same mask that was used for neurofeedback data extraction before, and adjusted for condition effects (“up,” “down,” and “view”). Second, HRFs were fitted to each picture presentation interval. The HRF amplitude represents the magnitude of the event‐related BOLD response and was previously found to be susceptible to control in a neurofeedback experiment (Paret et al., 2014). However, dynamics that differ from the classical HRF shape are less well captured by the amplitude parameter. To analyze the total BOLD signal change, the area under the curve (AUC) was analyzed. Amplitude estimates and AUC values were compared using SPSS statistics software (version 23, IBM Corp. Armonk, NY).

2.5.4. Feedback monitoring and feedback control

In the feedback monitoring analysis, we investigated event‐related brain responses to feedback (phasic responses to changes in the thermometer). In addition, we performed an analysis of tonic brain responses to pictures in the conditions “up,” “down,” and “view.” Because the latter analysis resembles the paradigmatic analysis of neurofeedback training blocks in the literature and the results from these analyses were discussed under the term (neuro)feedback control (Sitaram et al., 2017), we will keep this terminology for the rest of the article.

First level

Brain responses to picture presentation were modeled for each condition (“up,” “down,” and “view”). As a condition of no interest, “rest” was modeled with a stick‐function. In addition, parametric regressors were introduced to the “up” and “down” conditions to model responses to feedback. For building the regressors, the difference between the previous and current feedback was used; when the thermometer rose by one level, the regressor modeling “feedback up” turned “1.” In turn, when feedback dropped by one level, another regressor modeling “feedback down” turned “1” (Figure 1c).

We received four feedback‐regressors for further analyses, representing all factor‐level combinations in a 2 × 2 factorial design (factor “Instruction” (I) with two levels: “I up,” “I down”; factor “Feedback” (F) with two levels: “F up,” “F down”; in Figure 1d, arrows pointing up and down symbolize the direction). Stimulus functions were orthogonalized before modeling.

The model included an HPF = 256 s and AR(1)‐modeling. To account for potential artifacts, scan‐to‐scan movements and changes in global signal intensity were screened using the ART software package (http://www.nitrc.org/projects/artifact_detect). Movement was quantified based on the six parameters derived from the realignment step. Movements >2 mm and global signal intensity changes of z > 9 were classified as outliers and modeled (the method is called “censoring” or “scrubbing”). In addition, the six movement regressors were included as nuiscance variables.

Single‐subject T‐contrasts of the different factor level combinations were used in further analyses of feedback monitoring (Figure 1d). The aim of the analysis is to identify responses to feedback. Therefore, regressors were used that counterbalanced the direction of feedback change (i.e., thermometer rises or drops); that is, for each regressor modeling, a rise of the thermometer (e.g., “Instruction up, Feedback up”) and receiving a positive beta weight (“+1”), another regressor modeling a drop in the “thermometer” (e.g., “Instruction up, Feedback down”) received a positive beta weight, too (Figure 1d). Regions identified with the feedback congruency monitoring analysis, for example, are not a direct reflection of change in the amygdala BOLD signal, but reflect the change of the amygdala signal in the instructed direction. Three different contrasts were analyzed: (1) “feedback activity monitoring” ([1,1,1,1], corresponding to [I up/F up, I up/F down, I down/F up, I down/F down]) is sensitive for general responses to feedback; (2) “feedback congruency monitoring” ([1, −1, −1, 1]) is optimal to detect brain regions with a response pattern congruent with the instruction to up‐ or downregulate; and (3) “feedback context monitoring” ([−1, −1, 1, 1]) is suited to detect brain responses to feedback that differ with regard to the condition. The fourth possible contrast [1, −1, 1, −1] is not informative for two reasons: (a) it is difficult to think of a convincing reason for a cognitive process that differentiates between basic physical aspects of feedback such as vertical movements in the thermometer and (b) the HRF‐convolved stimulus function of the feedback time course is correlated with the amygdala BOLD signal (though with a time lagged difference proportional to the HRF parameters). Unlike the other feedback monitoring contrasts, the latter factor combination does not protect against the finding of brain activations that are merely correlated with amygdala activation. Therefore, a straightforward interpretation of results from this contrast appears impossible.

For analysis of feedback control, the “up versus down,” “up versus view,” and “down versus view” T‐contrasts were analyzed.

Second level

At the group level, random effects analysis was used to identify significant brain activation in voxel‐wise analyses. Therefore, first‐level contrasts computed on the subject level were assessed in one‐sample T‐tests. If not mentioned otherwise, significance was tested with reference to the whole brain‐volume. To protect against false positives, we used Monte‐Carlo simulations to define minimal cluster‐extent with a cluster‐defining voxel threshold (CDT) of p < .001 and a cluster‐threshold of p < .05. Simulations were performed with 3dClustSim, implemented in AFNI (version 16.1.28, June 30, 2016 (Cox, 1996)). Estimation of cluster‐extend was determined in 2,000 iterations, based on the voxels included in the second‐level mask (Supporting Information, Figure S1) and the smoothness (using FWHM) of the residuals of the corresponding second‐level SPM analysis. Whole‐brain analysis was complemented by region‐of‐interest (ROI) analysis to test the hypotheses of the involvement of VS and AI in value processing. For ROI analysis, SPMs were first set at a threshold to display voxels with a p < .01 uncorrected (k > 10). Family‐wise error (FWE) correction was then performed in a volume comprising bilateral basal ganglia (Maldjian et al., 2003) and spherical ROIs of the left ([−36, 20, −2]) and right ([32, 26, 4]) AI with center coordinates defined from meta‐analysis (Emmert et al., 2016) (radius = 20 mm).

A two‐sided significance test (SPM's F‐contrast) was used for all analyses. The F‐test is sensitive for positive and negative test values and is thus instrumental to detect brain response patterns that result in “activations” and “deactivations” of a specific contrast in a single statistical test. Compared to SPM's one‐sided T‐test, the F‐test comes at the cost of a more conservative threshold for detecting significant effects.

2.5.5. Correlation analyses

We were interested whether or not feedback monitoring would predict amygdala regulation. Differences of amygdala amplitude estimates (“up‐view,” “up‐down,” or “down‐view”) were used as outcomes in a correlation analysis. As predictors, we inserted contrast estimates from peak‐voxels of clusters detected with the second‐level feedback monitoring analysis. The Pearson correlation coefficient was determined. Statistical significance was defined at the p = .05 level. To corroborate the robustness of our results, we followed the suggestion of one of the reviewers and assessed 95% confidence intervals with nonparametric bootstrapping, using Matlab (MathWorks, Inc, version R2012a).

2.5.6. Functional connectivity analysis

Multiregion functional connectivity analysis: Feedback monitoring

We analyzed functional network connectivity between feedback‐responsive brain regions (i.e., thalamus, vmPFC, rVS, lVS, rlPFC; see Section 3), using multiregion psychophysiological interaction (PPI) analysis (Cocchi et al., 2013; Gerchen et al., 2014). PPI analysis uses a moderated multiple regression model to estimate stimulus‐dependent changes in the temporal association between a seed and a target region. The parameter estimates of the PPI term can be interpreted as the stimulus‐specific functional connectivity of the seed to the target.

Individual ROIs were selected in two steps: (a) a spherical ROI (diameter = 12mm) was placed around the peak voxel from the group analysis. Within the ROI, the voxel with strongest activation in the corresponding contrast was searched for each subject individually and (b) the eigenvariate was extracted from a spherical ROI (diameter = 12 mm, prepared with the marsbar toolbox v0.44 (Brett et al., 2002)) around the individual peak voxel and adjusted for the effects of the four feedback‐regressors. If no individual peak was found, the peak voxel from the group‐level analysis was chosen. ROIs were masked with an inclusive brain mask to dismiss voxels located outside the brain before eigenvariate extraction. Identifying ROIs for connectivity analysis based on activation analysis is valid, because PPI investigates a hypothesis (neural coupling) other than the “conventional” analysis (neural activation).

Each PPI model included the feedback regressors, the physiological regressor of the seed region's signal time course, and their interaction terms following the procedure proposed by McLaren et al. (2012). The physiological regressor was deconvolved before calculation of the interaction terms, and the interaction terms were reconvolved with the canonical HRF. AR modeling, an HPF, and nuisance regressors were included as described above.

PPI estimates of feedback‐related connectivity were contrasted for each subject, in the same way as in the activation analysis (Figure 1d). Thus, we received a five‐by‐five (region‐by‐region) connectivity matrix for each contrast per subject. Significance was assessed with two‐sided T‐tests at the p = .05 level using SPM routines and Bonferroni‐correction.

Amygdala seed‐based functional connectivity

The seed‐based PPI model for replication of our previous results (Paret et al., 2016b) included regressors for picture blocks of each condition (“up,” “down,” and “view”), the amygdala eigenvariate received with the amygdala ROI used for neurofeedback (and adjusted for the three effects of interest), the PPI regressor, AR‐modeling, an HPF, and nuisance regressors. For analysis, a t‐contrast was used, contrasting the combined “up and down” effects versus “view.” The search volume for significant voxels was limited to a spherical ROI (radius = 20 mm) around the peak coordinate ([0, 56, −11]) of a cluster that previously showed increased right amygdala connectivity with amygdala versus sham feedback in an independent sample (Paret et al., 2016b). Significance was assessed at the p = .05 (one‐sided) FWE‐corrected level.

3. RESULTS

3.1. Behavioral data

The eye gaze was monitored to make sure that subjects were actually engaged with emotional stimuli and not only with the feedback. As instructed, participants looked longer at pictures versus thermometers (F(1,19) = 121.28, p < .001). No differences in picture dwell times were found between “up” and “view” (T(18) = 1.3, p = .21) or “down” and “view” (T(18) = 0.74, p = .47). A tendency for longer dwell times in “up” versus “down” was found (condition main effect: (F(1,19) = 3.15, p = .09). The statistical trend in the condition x area of interest (AOI) interaction (F(1,19) = 3.77, p = .07) resulted from longer dwell times on pictures in “up” versus “down” (T(19) = 1.96, p = .07, Figure 2). The number of fixations was higher for pictures compared to thermometers (F(1,19) = 81.29, p < .001) but did not differ between conditions (main effect: F(1,19) = 2.38, p = .14, interaction: F(1,19) = 2.75, p = .11).

Figure 2.

Participants followed the instructions and looked longer at pictures versus thermometers (N = 20). No thermometer was visible in the “view” condition; data for thermometer‐AOI is shown for reasons of comparison. Error bars = SD

After the experiment, participants rated the pictures as negative (5.82 ± 1.59) and high in arousal (7.14 ± 0.50; 1‐positive/no arousal, 9‐negative/high arousal). Ratings did not differ between conditions (T(19) < 1.3, p > .2). Perceived feedback control was rated in the low range in the medium (3.25 ± 1.67, 0 = no control, 9 = very high control), similar to a previous study (Paret et al., 2014).

3.2. fMRI analysis

First, we tested whether the received feedback is a specific correlate of the amygdala BOLD signal (i.e., originates from this region and not from another one). Brain activation was modeled with the neurofeedback time course actually seen by participants. We found positive correlation of feedback with the bilateral amygdala and brain stem with an activation peak in the right amygdala (Supporting Information, Figure S3 and Table S1). This result confirmed that subjects observed feedback which was specific to their amygdala response.

Next, we assessed whether subjects were able to regulate their amygdala response in the desired direction. As was expected, subjects responded with an amygdala response to picture presentation in the “up,” “down” and “view” condition. We observed a tendency for successful down‐ versus up‐regulation of the amygdala BOLD amplitude (Figure 3a,b and Table 1A). The area under the curve (AUC) is a measure of total BOLD signal change and the statistical comparison of the “down” and “up” condition revealed significant amygdala regulation (Figure 3c and Table 1B). While participants reduced activation in the “down” condition compared to the “view” condition, they did not increase activation with the instruction to upregulate (Figure 3 and Table 1), as no activation difference was visible between the “up” and “view” condition. However, statistical comparisons to “view” were not significant.

Figure 3.

Participants regulated the amygdala response with neurofeedback, comparing the “down” and “up” conditions. N = 20; HRF = hemodynamic response function; AUC = area under the curve; a.u.=arbitrary units; *p < .05. Error bars = SEM

Table 1.

Difference of amygdala HRF between conditions

(A) Amplitude
	Paired difference
Test	Mean	SD	T (df = 19)	p
Down vs up	−0.223	0.583	1.713	.103
Down vs view	−0.237	0.788	1.345	.194
Up vs view	−0.014	0.878	0.069	.946

(B) AUC
	Paired difference
Test	Mean	SD	T (df = 19)	p
Down vs up	−1.301	2.731	2.130	.046*
Down vs view	−1.300	3.726	1.557	.136
Up vs view	0.003	4.429	0.003	.997

Note. Abbreviations: AUC = area under the curve; df = degrees of freedom; SD = standard deviation.

*p < .05

In accordance with our prediction, the monitoring analysis revealed a different set of active brain regions than the control analysis with overlap in the thalamus and VS in both types of analysis. As expected, an analysis of characteristic response patterns mapped feedback activity monitoring, feedback congruency monitoring and feedback context monitoring to different anatomical locations (Figure 4a). Feedback activity monitoring was revealed in the medial thalamus (Figure 4b) and the vmPFC (Figure 4c). In the analysis of feedback congruency monitoring, the primary significance test did not detect any activation. Based on the literature, we expected the VS to signify reward value. The expected value signal should be detectable with the feedback congruency monitoring analysis. However, VS activation could be missed because of subthreshold cluster size. Therefore, the search volume was limited to the basal ganglia in a region‐of‐interest (ROI) analysis with small volume correction. The ROI analysis revealed significant bilateral VS activation (Figure 4d, xyz: [14, 8, −6], z = 4.22, [−12, 10, −6], z = 4.09, p < .05, family wise error [FWE]‐corrected). To follow up on a hypothesis suggesting involvement of the AI in neurofeedback reward (Sitaram et al., 2017), activation was searched for in a ROI analysis without significant results. Finally, it was found that the rostral PFC (rPFC) was involved in feedback context monitoring (Figure 4e).

Figure 4.

Brain regions involved in feedback monitoring (N = 20). (a) Activated clusters are displayed upon a canonical T1 brain template. Slice numbers above transversal slices indicate z‐coordinate (top is rostral and left is left), the position of slices is shown by blue lines on a sagittal brain view to the right (x = 0, right is rostral, top is dorsal). (b–e) Parameter estimates extracted from cluster, parameter estimates in (b) and (d) are based on voxels with overlapping feedback monitoring and feedback control activation. vmPFC = ventromedial prefrontal cortex; VS= ventral striatum; rPFC = rostral prefrontal cortex; a.u.= arbitrary units. Error bars = SD. For labels of x‐axis, see Figure 1 [Color figure can be viewed at http://wileyonlinelibrary.com]

The neural response patterns clearly differed among regions; while the thalamus and vmPFC showed unspecific deactivation to feedback, the VS, and rPFC responded differently in the “up” and “down” conditions. The VS responses were consistent with the feedback reward value; an increase in the thermometer's temperature was associated with VS activation in the “up” condition and VS deactivation in the “down” condition. In the rPFC, deactivation was stronger for feedback in the “up” versus “down” condition. Statistics and anatomical data from whole‐brain analyses are reported in the Supporting Information, Table S2.

To shed light on the relationship between feedback monitoring and successful amygdala regulation, we explored correlations. A significant positive correlation of rPFC activation and amygdala up‐ versus downregulation was found both for the amygdala amplitude (Figure 5a and Table 2A; see Supporting Information for confirmation with nonparametric bootstrapping, Table S3a) and AUC (Table 2b and Supporting Information, Table S3b), meaning that subjects showing higher differential rPFC responses to feedback in the “up” versus “down” condition achieved better regulation in the envisaged direction. Significant positive correlations were found between VS activation and amygdala amplitude up‐ and downregulation, compared to “view” (Figure 5b, Table 2A and Supporting Information, Table S3a). Right VS activation was positively correlated with amygdala AUC up‐ versus downregulation (Table 2B).

Figure 5.

Rostral prefrontal cortex (rPFC) and left ventral striatum (VS) are correlated with amygdala regulation and are functionally connected. (a,b) Parameter estimates extracted from group analysis peak voxel. (c) Functional connectivity from rPFC to left VS was modulated by feedback congruency monitoring (N = 20). A.u. = arbitrary units. Error bars = SD

Table 2.

Pearson‐correlations of amygdala hemodynamic response to pictures with parameter estimates from feedback monitoring analysis (N = 20). (A) HRF amplitude estimates. (B) Area under the curve, representing total BOLD signal change

A) Amplitude
	Feedback activitymonitoring		Feedback congruencyMonitoring		Feedback contextmonitoring
Difference	Thalamus	vmPFC	lVS	rVS	rPFC
Up‐down	−0.20	−0.26	0.11	0.28	0.58**
Up‐view	0.06	0.06	0.56*	0.56*	0.23
Down‐view	0.09	0.26	0.55*	0.42	−0.17

B) AUC
	Feedback activitymonitoring		Feedback congruencymonitoring		Feedback contextmonitoring
Difference	Thalamus	vmPFC	lVS	rVS	rPFC
Up‐down	−0.44	−0.28	0.37	0.50 *	0.60**
Up‐view	−0.22	−0.15	0.24	0.27	0.35
Down‐view	0.06	0.03	0.01	−0.04	−0.02

Note. Abbreviations: l/rVS = left/right ventral striatum; rPFC = rostral prefrontal cortex; vmPFC = ventromedial prefrontal cortex.

*p < .05, **p < .01, not corrected for multiple comparisons.

Based on the results from activation analyses, we explored functional connectivity. Connectivity between rPFC and left VS was modulated by feedback monitoring (Figure 5c). Connectivity increased when participants received feedback that was congruent versus incongruent with the given instruction. None of the other multiregion psychophysiological interaction (PPI) results survived Bonferroni correction (Supporting Information, Tables S4–S6).

Finally, in common and differential activation of feedback control was found in the “up” and “down” condition (Figure 6a). Downregulation activated widespread clusters and showed overall strong activation effects in the dlPFC, dorsal ACC, lateral thalamus (Figure 6b), and occipital cortex. Efforts to up‐ versus downregulate resulted in activation of the subgenual ACC (sgACC, Figure 6c), extending to the VS (Figure 6d), and posterior temporal cortex. Up‐ and downregulation commonly activated the insula (Figure 6e), ventrolateral PFC, and cerebellum. Statistics and anatomical data from whole‐brain analysis are reported in the Supporting Information, Table S7.

Figure 6.

Brain regions involved in feedback control (N = 20). (a) Activated clusters are displayed upon a canonical T1 brain template. Slice numbers above transversal slices indicate z‐coordinate (top is rostral and left is left), the position of slices is shown by blue lines on a sagittal brain view to the right (x = 0, right is rostral, top is dorsal). (b–e) Parameter estimates extracted from cluster, parameter estimates in (b) and (d) are based on voxels with overlapping feedback monitoring and feedback control activation. VS = ventral striatum; sgACC = subgenual anterior cingulate cortex; a.u. = arbitrary units. Error bars = SD. For labels of x‐axis, see Figure 1 [Color figure can be viewed at http://wileyonlinelibrary.com]

3.2.1. Functional amygdala connectivity

To test the reproducibility of our previous findings concerning increased amygdala‐vmPFC functional connectivity during neurofeedback (Paret et al., 2016, 2016b), we used the right amygdala as a seed to conduct a PPI analysis. We found increased amygdala‐vmPFC connectivity ([18,52,−6], z = 3.94, p < .05 FWE‐corrected; Supporting Information, Figure S4) in the “up” and “down” conditions versus “view” corroborating the previous observations.

4. DISCUSSION

Our aim was to dissect the neural signatures of feedback monitoring and feedback control in human subjects provided with continuous rtfMRI neurofeedback from the amygdala. In line with previous findings (Paret et al., 2014), subjects successfully downregulated their amygdala response while they looked at negative pictures, even though they had more difficulties with upregulation. The latter does not mean that amygdala upregulation with neurofeedback is more difficult than downregulation in general. On the contrary, healthy subjects were able to learn amygdala upregulation in a single training session (Zotev et al., 2011). The inconsistency with previous findings may be resolved when we consider that we presented pictures with high negative valence and arousal, whereas Zotev et al. (2011) did not present affective stimuli. It is probably more difficult to increase the amygdala response to pictures because of ceiling effects, and it is easier to increase activation from the baseline level without concurrent stimulation.

We argue that learning with neurofeedback involves feedback monitoring in addition to control processes that can be distinguished using neuroimaging. In line with this hypothesis, we identified separate brain networks for monitoring and control with only a small amount of overlap in the thalamus and VS (Figure 7). Second, we expected to find characteristic response patterns for feedback monitoring during up‐ and downregulation that would reflect processing of different information. Indeed, findings of different activation patterns in the thalamus, VS, vmPFC, and rPFC corroborated this expectation. Previously, it was suggested that the thalamus, striatum, and frontal cortex are crucial for learning with neurofeedback (Hinterberger et al., 2005; Koralek et al., 2012). To our knowledge, our findings are the first evidence that these brain regions are involved in the processing of different informational aspects during neurofeedback training. We observed feedback‐related deactivation in the medial thalamus that did not differentiate between conditions (such as instructions to up‐ and downregulate) or between the rise and fall of the feedback‐thermometer's “temperature”. In the analysis of feedback control, in contrast, downregulation was associated with tonic activation of a lateral thalamus cluster. The mediodorsal thalamus is an integral part of a thalamo‐cortico‐striatal pathway subserving oculomotor control (Alexander et al., 1986) and informs about impending saccades (Sommer and Wurtz, 2004). Therefore, the medial thalamus may have driven attention and visual processing of feedback in our experiment and the lateral thalamus may have participated in the implementation of control strategies.

Figure 7.

An overview of brain regions involved in feedback monitoring (black circles) and feedback control (white circles). Regions marked with half black half white circles where found with both analyses. OC = orbitofrontal cortex; Thal = thalamus; VS = ventral striatum; Ins = insula; dACC = dorsal anterior cingulate cortex; sgACC = subgenual ACC; dlPFC = dorsolateral prefrontal cortex; rPFC= rostral PFC; vmPFC = ventromedial PFC

The VS activated, when the thermometer switched in the desired direction. Therefore we conclude that VS activation was driven by the reward value of feedback. When more reinforcement is received over the course of training, VS activation should become more pronounced in the statistical analysis. In support of this concept, we found positive correlations of VS activation with the amygdala amplitude during up‐ and downregulation. In other words, stronger amygdala responses are followed by a higher response of the feedback‐thermometer and the latter may lead to a higher VS response. In consequence, higher variance in VS activity may have produced higher effects in the statistical analysis. In summary, stronger amygdala responses are predictive for higher effects in feedback congruency monitoring and thus, result in a positive correlation. Alternatively, the correlation could reflect an increased state‐ or trait‐responsivity to emotional events such as reward and negative pictures. Moreover, VS activation predicted amygdala up‐ versus downregulation. This finding supports the importance of feedback monitoring by the VS for regulation success. However, replication is needed, because the effect was significant in the AUC but not in the amplitude measure. Finally, the VS was not just involved in feedback monitoring, but also in feedback control. The VS may signal the difference between the expected and the actual outcome of emotion regulation (Etkin et al., 2015; Morawetz et al., 2017). If the actual outcome is below expectations, the resulting difference is negative and is called a negative prediction error (Etkin et al., 2015). Due to overall unsuccessful amygdala upregulation, subjects may have attested themselves poor emotion upregulation, and VS deactivation may reflect this self‐assessment. This interpretation is limited due to the lack of outcome measures such as subjective reports of emotion regulation success, and therefore remains speculative.

Neural activation of feedback monitoring and control showed an anterior‐to‐posterior topography in the medial prefrontal cortex; monitoring activated the middle‐to‐rostral medial PFC and control involved the ACC. In the vmPFC, deactivations were driven by the rise and fall of the thermometer's “temperature.” This replicates a previous report (Radua et al., 2016) in which the authors found vmPFC deactivation to neurofeedback success signals with feedback from a region located in the vmPFC/OFC. Our results extend this finding in two ways: (a) in addition to success signals, we found deactivations of the vmPFC to failure signals and (b) we report monitoring in the same brain region but with feedback from the amygdala instead of the vmPFC/OFC. Hence, involvement of the vmPFC in neurofeedback is not limited to a specific feedback signal. However, we need to consider that both studies presented pictures with affective content. Future studies are necessary to examine whether the vmPFC is particularly involved in feedback from brain responses related to emotion or is also found with other source signals. The ventromedial part of Brodmann area 10 was previously found to receive amygdala input in a sham‐controlled neurofeedback experiment (Paret et al., 2016b) and has been discussed as a region integrating “intrinsic” amygdala information transmitted via neural amygdala–vmPFC connectivity with “extrinsic” amygdala‐neurofeedback information to facilitate control (for a detailed discussion, see Paret et al., 2016b). Our findings corroborate the vmPFC as a site responsive to neural and feedback signals from the amygdala, and we propose the vmPFC as a hub receiving feedback and brain signals in amygdala neurofeedback training.

Finally, the rPFC was involved in feedback monitoring and was capable of differentiating between conditions. We observed stronger deactivation to feedback with the instruction to up‐ versus downregulate, irrespective of congruency with instructions. Moreover, participants who achieved better amygdala regulation showed more differential rPFC responses. There are two alternative explanations for this finding: (a) similar to the vmPFC, the rPFC receives neural input from the amygdala and the correlation reflects the modulation of rPFC responses by the amygdala (as discussed above) and (b) the rPFC is involved in amygdala regulation and may receive input from instructions or other task‐related information to inform performance monitoring. Future studies will be necessary to test these hypotheses. In either case, our findings highlight the rPFC as a major neural node in neurofeedback training, suited to inform the processing of reward in the VS. The importance of this network is further corroborated by the finding of significant functional connectivity between the rPFC and VS.

In line with Sitaram et al.'s model (2017), the dlPFC, lOC, and thalamus were involved in feedback control. Furthermore, the VS was found to be involved in reward processing. In contrast, we did not find an association of the AI with reward but rather with control. Our findings go beyond the model by demonstrating distributed feedback information processing throughout several neural networks. Future studies are needed to show whether and how learning emerges from activity of these networks.

Though this was not a primary research question, we investigated more closely how control would affect the amygdala BOLD signal and found pronounced regulation effects with an AUC analysis. In contrast to the established amplitude analysis of the HRF, the AUC does not make assumptions regarding the shape of the BOLD response. Hence, it also accounts for later stages of the regulation trial after the HRF peak is reached, in which cognitive strategies unfold their effects on the emotional response (Kalisch, 2009; Paret et al., 2011). While this finding needs replication, we encourage adding model‐free analyses in future fMRI studies for a more comprehensive assessment of neurofeedback regulation success.

4.1. Conclusion

BCIs provide an information signal based on brain activation (i.e., neurofeedback) that can be used for learning neural self‐control. It is shown that feedback monitoring and feedback control have dissociable neural substrates. Processing of feedback information is distributed in a complex network of thalamic, striatal, and prefrontal brain regions.

4.2. Limitations

It should be emphasized that brain activations found with the feedback control‐analysis are not necessarily related to the effective control of amygdala. In fact, brain regions such as the lateral PFC, thalamus and dorsal ACC are generally active when subjects voluntarily control feedback from diverse brain signals (Emmert et al. 2016), and are even found when subjects try to control a fake‐neurofeedback signal (i.e., sham neurofeedback, see Ninaus et al., 2013). These activations could relate to any aspect of intentional control such as the selection and implementation of mental strategies or the monitoring of regulation goals. Our study does not allow conclusion about the relevance of these brain activations with regard to the success of feedback control.

Though ACC activation was linked to control and not to monitoring (as suggested in Sitaram et al., 2017), some areas such as more posterior ACC regions and pPC may have been missed because of limited MRI volume coverage. Alternatively, our study might have lacked the power to detect effects of congruency monitoring in the ACC due to the sample size.

Subjects were all female and sex‐effects cannot be excluded.

The BOLD signal is an indirect measurement of brain activation and anatomical localization of brain activation can be biased by blood vessels.

To optimize neurofeedback practice time, the number of pictures presented with feedback (as in the “up” and “down” conditions) exceeded the number of pictures presented without feedback (as in the “view” condition). Fewer trials mean more variance and decrease power to detect differences. This puts “up versus view” and “down versus view” comparisons at a disadvantage to detect effects in this study.

Future studies are needed to disentangle findings specific to amygdala‐neurofeedback from those involved in neurofeedback in general. For instance, VS activation is consistently observed in emotion regulation (Morawetz et al., 2017). Hence, activation of the VS in feedback control may be related to regulation of affective brain responses and may not be required for control of other neurofeedback signals. The field will largely benefit from studies trying to replicate our findings with feedback from other physiological signals such as fMRI from other brain regions or electro‐encephalographic or peripheral physiological measurements.

AUTHOR CONTRIBUTIONS

CP conceived the experiment, and together with JZ and SM carried it out; CP designed and carried out the data analysis; MFG supported the functional connectivity analysis and SM preprocessed eye‐tracking data; CP drafted the manuscript and CS, GE, JZ, MR, SM, MFG, and TH revised it critically for important intellectual content.

Supporting information

Additional Supporting Information may be found online in the supporting information tab for this article.

Supporting Information

Paret C, Zähringer J, Ruf M, et al. Monitoring and control of amygdala neurofeedback involves distributed information processing in the human brain. Hum Brain Mapp. 2018;39:3018–3031. 10.1002/hbm.24057