Skip to main content
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2020 Oct 15;220:117053. doi: 10.1016/j.neuroimage.2020.117053

Modulatory effects of dynamic fMRI-based neurofeedback on emotion regulation networks in adolescent females

Catharina Zich a,b,, Nicola Johnstone c, Michael Lührs d,e, Stephen Lisk f, Simone PW Haller a, Annalisa Lipp c, Jennifer YF Lau f, Kathrin Cohen Kadosh a,c
PMCID: PMC7573536  PMID: 32574803

Abstract

Research has shown that difficulties with emotion regulation abilities in childhood and adolescence increase the risk for developing symptoms of mental disorders, e.g anxiety. We investigated whether functional magnetic resonance imaging (fMRI)-based neurofeedback (NF) can modulate brain networks supporting emotion regulation abilities in adolescent females.

We performed three experiments (Experiment 1: N ​= ​18; Experiment 2: N ​= ​30; Experiment 3: N ​= ​20). We first compared different NF implementations regarding their effectiveness of modulating prefrontal cortex (PFC)-amygdala functional connectivity (fc). Further we assessed the effects of fc-NF on neural measures, emotional/metacognitive measures and their associations. Finally, we probed the mechanism underlying fc-NF by examining concentrations of inhibitory and excitatory neurotransmitters.

Results showed that NF implementations differentially modulate PFC-amygdala fc. Using the most effective NF implementation we observed important relationships between neural and emotional/metacognitive measures, such as practice-related change in fc was related with change in thought control ability. Further, we found that the relationship between state anxiety prior to the MRI session and the effect of fc-NF was moderated by GABA concentrations in the PFC and anterior cingulate cortex.

To conclude, we were able to show that fc-NF can be used in adolescent females to shape neural and emotional/metacognitive measures underlying emotion regulation. We further show that neurotransmitter concentrations moderate fc–NF–effects.

Keywords: Adolescence, Brain-computer-interface, Connectivity, Emotion regulation, Magnetic resonance spectroscopy, Neurofeedback

Highlights

  • Neurofeedback implementations differentially modulate PFC-amygdala connectivity.

  • Functional connectivity neurofeedback affect measures of emotion regulation.

  • Neurotransmitter concentrations moderate neurofeedback effects.

1. Introduction

Adolescence is marked by a multitude of neural, emotional/metacognitive and behavioural changes, such as functional and structural maturation and improvements in cognitive abilities and socio-emotional behaviour (Blakemore, 2008; Burnett et al., 2011; Cohen Kadosh et al., 2013; Linscott and van Os, 2013). It has been suggested that these complex, transformational processes may shape poor emotion regulation abilities and increase the risk for developing mental disorders (Haller et al., 2015, 2013; Keshavan et al., 2014; Paus et al., 2008). Anxiety is one of the most common and impairing mental disorders in adolescence. Youth with anxiety experience impairing levels of fears and worries that they are unable to regulate, which can impact friendships, school performance, and mark the beginning of long-term mental health difficulties (Beddington et al., 2008; Pine et al., 2001; Trentacosta and Fine, 2010). Gaining a better understanding of the mechanisms underlying the successful regulation of emotions during this unique developmental period therefore represents an important step towards devising efficient, and targeted strategies for early support and intervention.

In the human brain, the regulation of emotions relies on a network of brain regions, comprising the prefrontal cortex (PFC) and the amygdala, as well as additional regions in the orbitofrontal, frontal and cingulate cortex (Kohn et al., 2014; Ochsner and Gross, 2005). A particular focus has been placed on the PFC-amygdala relationship, a relationship that manifests in both structural and functional connections (Banks et al., 2007; Davidson et al., 2000; Kim et al., 2003; Kim and Whalen, 2009; Kohn et al., 2014; Quirk and Beer, 2006). With regard to the regulatory aspect of the PFC-amygdala connection, the functional connectivity (fc) between the two regions has been theorized to reflect top-down PFC regulation of amygdala reactivity (Hare et al., 2008; Hariri et al., 2003; Kim et al., 2003; Pezawas et al., 2005), with increased PFC responses leading to decreases in amygdala activation, i.e. negative fc (Kim et al., 2011). Interestingly, this fc pattern only emerges during adolescence (Dougherty et al., 2015; Gee et al., 2013; Wu et al., 2016), with research showing that children at the beginning of adolescence (up until around 10 years of age) exhibit predominantly positive fc, rather than negative fc ((Dumontheil et al., 2012, 2010; Silvers et al., 2012), also reviewed in (Ahmed et al., 2015)). What is less clear however is whether we can intervene to actively shape fc patterns in the developing brain to improve emotional/metacognitive abilities and thereby affect behaviour.

The current study aimed to address this question by using real-time functional magnetic resonance imaging (fMRI)-based neurofeedback (NF). NF provides the user with real-time information of one’s own brain activity, and it has already been shown to be a promising intervention tool to (re-)shape neural activity in a number of psychiatric and neurological diseases (Linden, 2012; Lubianiker et al., 2019). In the case of emotion regulation, NF can aid the modulation of networks underlying this process, such as PFC-amygdala fc (Paret and Hendler, 2020). Previous studies suggest that fMRI-based NF represents a promising approach to train individuals in the self-modulation of brain regions or networks (Johnston et al., 2010; Koush et al., 2017, 2013; Scharnowski and Weiskopf, 2015; Weiskopf et al., 2004; Zilverstand et al., 2014), also in the emotion processing domain (adults: (Johnston et al., 2010; Koush et al., 2017; Paret et al., 2014; Zotev et al., 2013); children: (Cohen Kadosh et al., 2016)).

Building upon these studies, we conducted a set of three experiments that investigated whether fMRI-based fc-NF of PFC-amygdala fc can be used to modulate neural measures (e.g. change in fc) and emotional/metacognitive measures (i.e. self-reported measures of anxiety and thought control ability) relevant for emotion regulation in adolescent females. To achieve this overall objective, we first explored whether PFC-amygdala can be differentially modulated by contrasting three different NF implementations (Experiment 1). The most effective NF implementation was then used to assess the effects of one session fc-NF on neural measures and emotional/metacognitive measures in a larger sample (Experiment 2). Finally, we assessed the effects of a longer fc-NF block in order to further enhance the effectiveness of the fc-NF, and whether the brain-behaviour correlations found in Experiment 2 could be replicated (Experiment 3). In Experiment 3, we also explored whether neurotransmitter concentrations of the brain’s main excitatory and inhibitory neurotransmitters in relevant regions influence the observed effects. Regarding the latter, we used proton magnetic resonance spectroscopy (1H-MRS) to extract the neurotransmitter profile from two voxels of interest (VOI), the PFC and the anterior cingulate cortex (ACC). We choose the PFC because of the key role of γ-aminobutyric acid (GABA)-ergic neurotransmission within the PFC in balancing amygdala activity (Constantinidis et al., 2002). Further, we choose the ACC because of its centrality in the reward processing network, especially for explicit processing of reward, such as NF (Emmert et al., 2016; Sitaram et al., 2016). For both of these VOIs we quantified GABA and glutamate, the major inhibitory and excitatory neurotransmitters in the human brain.

2. Experiment 1

2.1. Materials and methods

2.1.1. Participants

18 female adolescent participants (M ​= ​14.83 years; SD ​= ​0.99 years) were recruited from local schools in the Oxfordshire/Gloucestershire area. The single-sex approach allowed us to minimize variation introduced by differences in hormonal development and puberty (Goddings et al., 2014, 2012). All participants had normal or corrected-to-normal vision and reported no history of neurological and psychiatric disorders (determined via self-report). Informed written consent was obtained from the primary caregiver and informed written assent was obtained from the adolescent. Participants received an Amazon voucher (£20) for their participation. The study was approved by the Central Oxfordshire Ethics Committee (MSD-IDREC-C2-2015-023) and conducted in accordance with the Declaration of Helsinki. This work was registered as preclinical trial (ClinicalTrials.gov Identifier: NCT02463136).

2.1.2. Self-report questionnaires

Immediately prior to the MRI session, participants completed several self-report questionnaires, which will be referred to as emotional/metacognitive measures. Specifically, we assessed psychological variables using the Emotion Regulation Questionnaire (CERQ) (Derryberry and Reed, 2002), the Mood and Feelings Questionnaire (MFQ) (Angold et al., 1995), Social Anxiety Scale for Adolescents (SAS-S) (La Greca and Lopez, 1998), the State-Trait Anxiety Inventory (STAI-T, STAI-S) (Spielberger et al., 1999), the Thought Control Questionnaire (TCQ) (Wells and Davies, 1994) and the Thought Control Ability Questionnaire (TCAQ) (Luciano et al., 2005). These are established self-report measures, which are frequently used in psychological testing. A subset of the questionnaires (i.e. CERQ, MFQ, SATI-S, TCQ, TCAQ) were repeated after the MRI session in order to assess changes. Participants also completed a Demographic and Health Questionnaire and the Wechsler Abbreviated Scale of Intelligence (Wechsler, 2011).

2.1.3. MRI data acquisition

MRI data acquisition was performed on a 3 ​T ​S MAGNETOM Prisma MRI scanner (Siemens AG, Erlangen, Germany) using a standard 32-channel head matrix coil. First, a high-resolution structural scan was acquired, which was followed by functional imaging during the localizer task and the NF task (see Supplementary Methods for details on the MRI sequences).

2.1.4. Localizer task

A modified version of the social scenes task (Haller et al., 2016) was used to identify the NF regions of interest (ROI), as it was expected that this task activates the key regions involved in cognitive and emotional appraisal and reappraisal. The localizer task lasted 8.9 ​min and comprised 30 trials. Each trial started with a social scene presented for four volumes (3.73 ​s). Scenes depicted negative, rejecting social situations viewed from the perspective of a female protagonist depicted from the back. Participants were instructed to interpret the scene freely (appraisal). This was followed by a positively valanced interpretative statement (4 ​volumes, 3.73 ​s), after which the same scene was shown again for four volumes (3.73 ​s). For the duration of the second presentation, participants were instructed to reappraise the scene based on an interpretative statement (reappraisal). Finally, participants were asked to rate how much they were able to change, i.e. reappraise, their thoughts and feelings from the first to the second presentation of the scene. Participants indicated their perceived change on a Likert scale ranging from no change (1) to much change (4) via button press on each trial. A fixation cross was presented for one volume between two trials. Stimulus presentation was controlled via BrainStim 1.1.0.1 (open source stimulation software, Maastricht University, http://svengijsen.github.io/BrainStim/). Participants were encouraged to use these emotional reappraisal strategies during the NF tasks.

2.1.5. Neurofeedback task

The NF task consisted of four identical runs lasting 4.8 ​min each. Stimulus presentation was controlled with BrainStim 1.1.0.1. Each run started with a fixation cross that was displayed for 20 ​volumes (18.66 ​s), followed by seven fc-NF mini-blocks (20 ​volumes each) and seven no-NF mini-blocks (20 ​volumes each) that were presented in alternating order, with the start condition being randomized and counterbalanced across individuals. During both fc-NF and no-NF mini-blocks a ten-segment thermometer was presented at the centre of the screen on a dark grey background. During the no-NF mini-blocks, the ‘temperature’ of the thermometer was frozen at the sixth segment (i.e. 6/10) throughout the mini-block (Fig. 1a). During the fc-NF mini-blocks, the thermometer featured a green frame and the ‘temperature’ was a direct reflection of PFC-amygdala fc and was updates with every TR (see section ‘Online MRI data analysis’ for more details). All participants were asked to up-regulate the thermometer by controlling their thoughts and feelings and by revisiting emotional reappraisal strategies as practised in the localizer task (see Supplementary Methods for more details on the instructions).

Fig. 1.

Fig. 1

Schematic representation of the experimental procedures (Experiment 1 and 2 in green and Experiment 3 in orange). (a) Experimental timelines. To activate the key regions involved in emotion regulation participants performed the social scenes task (localizer). Based on an individual’s localizer activity, we then defined three regions of interests (ROIs): left prefrontal cortex (PFC), left amygdala, left corticospinal tract (CST). Each NF run comprised fc-NF mini-blocks (20 ​volumes in Experiments 1 and 2; 40 ​volumes in Experiment 3) and no-NF mini-blocks (20 ​volumes), presented in alternating order. (b) Detailed view of the procedure during the NF task. NF was based on partial correlations rp between the PFC and amygdala activity, while controlling for CST ‘activity’, whereby partial correlations were obtained from a moving window comprising 20 subsequent volumes, which was updated with every incoming volume. Thus, for Experiments 1 and 2, the ratio of fc-NF: no-NF volumes in the correlation window is 1:19 ​at the first volume of the mini-block, 2:18 ​at the second volume of the mini-block and 20:0 ​at the last volume of the mini-block, and vice versa for no-NF mini-blocks. Note that due to the extended window length for the fc-NF mini-blocks in Experiment 3, the ratio of fc-NF: no-NF volumes in the correlation window is 20:0 for twenty volumes. Number of fc-NF volumes in the correlation window (N of fc-NF volumes in rp) is illustrated in form of a grey shape (c) Probabilistic maps of subject-specific ROIs (i.e. PFC and amygdala). superimposed on the groups’ average structural image in MNI space. The respective centre of the PFC probabilistic maps (Experiment 1 and 2: x ​= ​−20, y ​= ​50, z ​= ​24; Experiment 3: x ​= ​−26, y ​= ​47, z ​= ​22) are classified as Brodmann area 10 within the left middle frontal gyrus. The respective centre of the amygdala probabilistic maps (Experiment 1 and 2: x ​= ​−17, y ​= ​−1, z ​= ​−14; Experiment 3: x ​= ​−16, y ​= ​−3, z ​= ​−15) are classified as left amygdala (Eickhoff et al., 2005).

We evaluated three different NF implementations with regard to their effectiveness of modulating PFC-amygdala fc (N ​= ​6 per implementation). The three different NF implementations only differed in the upper value of the thermometer (the lower value of the thermometer was set to zero). The upper value of the thermometer was set to −1 (maximal negative correlation) for the negative NF implementation, which reinforced negative fc; to −0.31 for the weighted negative NF implementation, which reinforced negative fc; and to +1 (maximal positive correlation) for the positive NF implementation, which reinforced positive fc. To illustrate this: if the thermometer changes from 2/10 to 3/10 segments, this corresponds to a more negative correlation by 0.1 for the negative NF implementation, a more negative correlation by 0.03 for the weighted negative NF implementation, and a change towards a more positive correlation by 0.1 for the positive NF implementation.

2.1.6. Online MRI data analysis

To enable real-time fMRI-based NF, MR images were passed from the MRI console computer to the real-time computer via a direct TCP/IP network link using the Server Message Block (SMB) network layer. Following completion of the structural scan, anatomical images were processed using BrainVoyager QX 2.8.2 (Brain Innovation, Maastricht, The Netherlands). Each 3D volume was corrected for B0 in-homogeneities (4-cycle bias field estimation), followed by brain extraction from the skull and separating bone and cerebro-spinal fluid (2-cycle iteration).

Functional images obtained during the localizer and the NF runs were processed in real-time using Turbo-BrainVoyager 3.2 (Brain Innovation, Maastricht, The Netherlands). To correct head motion, each volume was realigned to the first volume of the localizer. Each realigned volume was smoothed with a three-dimensional Gaussian kernel of 8 ​mm full-width-half-maximum. After completion of the localizer, three ROIs (voxel size ​= ​12 ​mm3, 6 x 6 x 6) were manually placed. To this end, GLM t-statistics of the brain activity during the localizer task, i.e. the sum of the three contrasts: appraisal ​> ​fixation, reappraisal ​> ​fixation, reappraisal ​> ​appraisal (threshold t ​= ​3), was projected onto the processed structural scan. The local maximum of the t-statistics within the left dorsolateral and medial PFC constituted the centre of the PFC ROI. Similarly, the local maximum of the t-statistics within the left amygdala constituted the centre of the amygdala ROI. A ROI in the left corticospinal tract (CST) served as control ROI.2 Probability maps of the selected ROIs across subjects are depicted in Fig. 1c.

During the NF task, PFC-amygdala fc was calculated in real-time. PFC-amygdala fc was defined as the partial correlation rp between PFC and amygdala activity, while controlling for CST ‘activity’. As shown previously, partial correlation analysis can be used to quantify fc between areas while controlling for noise from a task-unrelated region (Dawson et al., 2016). Partial correlations were based on a moving window, which was updated with every incoming volume. The length of the correlation window was 20 ​volumes (see Supplementary Discussion). Therefore, the ratio of fc-NF to no-NF volumes in the correlation window changed with every incoming TR (Fig. 1b). To illustrate this: for the fc-NF mini-blocks, the ratio of fc-NF: no-NF volumes in the correlation window was 1:19 ​at the first volume of the mini-block, 2:18 ​at the second volume of the mini-block and 20:0 ​at the last volume of the mini-block, and vice versa for no-NF mini-blocks. Calculations were performed using a custom-made plugin for Turbo-BrainVoyager, which also provided a direct TCP/IP based link between the real-time analysis software and the stimulus application BrainStim. PFC-amygdala fc was displayed via thermometer during the fc-NF mini-blocks.

2.1.7. Offline MRI data analysis

We also analysed the MRI data offline. While online and offline analysis of the MRI data follow a similar processing pipeline, offline algorithms tend to be more robust. Offline analysis of the MRI data was performed using SPM12 (FIL, Wellcome Trust Centre for Neuroimage, UCL, London, UK). Here head motion was corrected by realigning the functional time series of the localizer or the NF runs to its first volume. Due to motion artefacts (i.e. motion exceeded 3 ​mm on any axis) three individuals (i.e. one individual per group) were excluded from all analyses. Each individual’s structural image was registered to their mean functional image and segmented, in order to normalize structural and functional images to the Montreal Neurological Institute (MNI) template. Finally, normalized functional images were smoothed with a three-dimensional Gaussian kernel of 8 ​mm full-width-half-maximum. To increase the consistency between online and offline analysis we used the same ROIs. Therefore, the ROIs defined in Turbo-BrainVoyager during the MRI session were transformed from DICOM to NIFTI format, from radiological to neurological convention and from voxel to mm space. Subsequently, ROIs were normalized to the MNI template by applying the same transformation matrix used for the subject-specific normalization of structural and functional images. Time series were extracted from the three ROIs using MarsBaR 0.44 (Brett et al., 2002) and PFC-amygdala fc calculated using the same procedure as during real-time NF, i.e. partial correlation rp between PFC and amygdala activity, while controlling for CST ‘activity’.

2.1.8. Statistical analysis

If not stated otherwise, statistical analysis was performed using SPSS version 25 (SPSS Inc, Chicago, IL, USA). To identify the most effective NF implementation we evaluated the difference between no-NF and fc-NF for each NF implementation. The primary measure for this comparison constitutes PFC-amygdala fc, i.e. partial correlations rp. Fc was averaged at each of the 20 ​volumes (i.e. length of a mini-block) across the seven mini-blocks within one condition (fc-NF, no-NF), NF runs (1, 2, 3, 4) and individuals within one NF implementation (negative NF implementation, weighted negative NF implementation, positive NF implementation). The relationship of fc-NF and fc was assessed by correlating (Pearson’s r correlations) the average fc at each of the 20 ​volumes with the number of fc-NF volumes within the correlation window (N ​= ​40, i.e. 20 ​volumes per mini-block). To statistically compare these Pearson’s r correlations between the different NF implementations, Fisher’s z-transformation was applied to each correlation coefficient, resulting in normally distributed values r’ with standard errors sr’. The null hypotheses (r’1-r’2 ​= ​0) were tested in R(psych) (Revelle, 2015) using Student t-test (Howell, 2011).

The same analysis was conducted to assess the frequency of negative partial correlations (secondary measure). We then assessed the frequency of the different scenarios, to provide further insights into the mechanisms underlying differences between fc-NF and no-NF (Supplementary Results, Supplementary Fig. 3; Supplementary Fig. 5).

2.2. Results

The aim of Experiment 1 was to explore whether PFC-amygdala can be differentially modulated. Therefore, we compared three different NF implementations regarding their effectiveness of modulating PFC-amygdala fc. The primary measure for this comparison, i.e. PFC-amygdala fc, is illustrated in Supplementary Fig. 1a.

The relationship of fc-NF and fc was assessed by correlating the average fc at each of the 20 ​volumes with the number of fc-NF volumes within the respective correlation window (Supplementary Fig. 1b). For the weighted negative NF implementation, a significant negative relationship was found (r(38) ​= ​−0.72, p ​< ​0.001), i.e. the more fc-NF volumes were included in the correlation window, the less positive the resulting partial correlation. The opposite relationship was found for the negative NF implementation (r(38) ​= ​0.37, p ​= ​0.02), i.e. the more fc-NF volumes were included in the correlation window, the more positive the resulting partial correlation. No significant relationship was observed for the positive NF implementation (r(38) ​= ​0.22, p ​= ​0.16).

Finally, the relationship between the average fc at each volume and the number of fc-NF volumes within the correlation window observed for the weighted negative fc-NF implementation was significantly different from the other two implementations (both p’s ​< ​0.001), which did not differ from each other (p ​= ​0.48). The picture is similar for the secondary measure, i.e. the frequency of negative partial correlations (see Supplementary Results, Supplementary Fig. 2; Supplementary Fig. 3 for details).

In sum, these findings demonstrate the superiority of the weighted negative NF implementation in modulating PFC-amygdala fc towards more negative fc.3

3. Experiment 2

3.1. Materials and methods

This study comprises data from 30 adolescent females (M ​= ​15.20 years; SD ​= ​1.10 years; 25 naïve individuals and the 5 individuals tested using the weighted negative NF implementation in Experiment 1). The recruitment procedure was identical to Experiment 1. The ‘Self-report questionnaires’, ‘MRI data acquisition’, ‘Localizer task’, ‘Neurofeedback task’ as well as the ‘Online MRI data analysis’ and ‘Offline MRI data analysis’ were identical to Experiment 1. However, all individuals received NF using the weighted negative NF implementation. Due to motion artefacts (motion exceeded 3 ​mm on any axis) datasets from three individuals were excluded from all analyses.

3.1.1. Statistical analysis

Firstly, we performed the same analysis as in Experiment 1.

We extended this by investigating practice-related change in emotional/metacognitive measures and neural measures. Practice-related change in emotional/metacognitive measures was defined as difference between emotional/metacognitive measures obtained before and after the MRI session and assessed using paired samples t-test. Practice-related change in neural measures, i.e. practice-related change in fc, was defined as the slope of the linear regression4 of the total fc across runs (Supplementary Fig. 4a) and was tested using a one-sample t-test.

Moreover, we performed correlation analyses between different emotional/metacognitive and neural measures. Relevant emotional/metacognitive measures were practice-related change in emotional/metacognitive measures (as defined in the previous paragraph) and initial emotional/metacognitive measures, i.e. measures obtained before the MRI session. Relevant neural measures were practice-related change in fc (as defined in the previous paragraph), initial fc and fc–NF–effect. Initial fc was defined as the average fc of the first two mini-blocks (Supplementary Fig. 4b). Fc-NF-effect was defined as the difference between no-NF and fc-NF (Supplementary Fig. 4c). For Experiment 2, the correlation analysis was exploratory. Pearson’s r correlations (two-tailed) are reported for the whole sample and, in addition, after outlier removal. Outliers were identified for each correlation separately (i.e. multivariate outlier) by bootstrapping the Mahalanobis distance (Schwarzkopf et al., 2012).

Missing values analysis revealed that 2.56% of the self-report questionnaire data were missing. We performed Little’s test of Missing Completely at Random (MCAR) (Little, 1988). MCAR was not significant (X2(35, N=27) ​= ​41.74, p ​= ​.201), i.e. there is no evidence to suggest that the data were not MCAR. As such, pairwise deletion was used in the statistical analysis.

3.2. Results

The aim of Experiment 2 was to assess the effects of one session fc-NF (weighted negative NF implementation) on neural measures, emotional/metacognitive measures and their associations in a larger sample. The fc–NF–effects observed in Experiment 1 were replicated, that is, the number of fc-NF volumes within the correlation window was negatively related with the average fc at each volume (Fig. 2a) and positively related with the frequency of negative partial correlations at each volume (Fig. 2b).

Fig. 2.

Fig. 2

Difference in average functional connectivity (fc) and frequency of negative partial correlations (N of negative rp) between fc-NF (red) and no-NF (blue) for Experiment 2 (green frame) and Experiment 3 (orange frame). (a, c) Top: Means and standard error for average fc, i.e. partial correlations rp, for fc-NF and no-NF at each volume. Fc is averaged across mini-blocks within one condition (fc-NF, no-NF), NF runs and individuals. Number of fc-NF volumes in the correlation window (N of fc-NF volumes in rp) is illustrated in form of a grey shape. Bottom: Relationship between average fc at each volume and the number of fc-NF volumes within the correlation window. (b, d) Same as (a, c), but for the frequency of negative partial correlations.

We further investigated practice-related change on emotional/metacognitive measures and neural measures for the whole sample. At emotional/metacognitive level no significant changes between measures obtained before and after the MRI session were observed (Supplementary Table 2). Similarly, at neural level no significant practice-related change in fc could be observed (M ​= ​0.005, SD ​= ​0.035, p ​> ​.01). Thirteen individuals showed a negative slope (i.e. change in the desired direction, M ​= ​−0.023, SD ​= ​0.019), fourteen individuals exhibited a positive slope (M ​= ​0.031, SD ​= ​0.025). Note that the emotional/metacognitive measures were comparable between these two subsamples (see Supplementary Table 3 and Supplementary Table 4).

3.2.1. Correlation analysis

We first investigated whether practice-related change in emotional/metacognitive measures and practice-related change in fc were related. We found that practice-related change in fc was negatively related with change in TCAQ (Fig. 3a top; r(21) ​= ​−0.37, p ​= ​.087; after the removal of one outlier: r(20) ​= ​−0.58, p ​= ​.005). Hence, the larger the practice-related change towards negative fc, the better the thought control ability after the MRI session, when compared to the before the MRI session.

Fig. 3.

Fig. 3

Associations between the emotional/metacognitive and neural measures for Experiment 2 (top, green) and Experiment 3 (bottom, orange). (a) Association between change in fc and change in thought control ability (b,c) Association between the initial fc and trait anxiety (STAI-T) as well as thought control ability (TCAQ) obtained before the MRI session. (d) Association between fc–NF–effect and state anxiety (STAI-S) obtained before the MRI session. Counter lines represent the bootstrapped Mahalanobis distance from the bivariate mean in steps of six squared units. Open circles represent outliers and the solid line is the regression over the data after outlier removal. Direction of solid arrows indicate levels related with higher anxiety. Direction of dashed arrows indicate the desired direction of practice-related change and fc–NF–effect.

Moreover, we were interested whether neural measures could be predicted by initial emotional/metacognitive measures, i.e. measures obtained before the MRI session. Correlation analysis revealed that initial fc was positively related with STAI-T (Fig. 3b top; r(25) ​= ​0.36, p ​= ​.067; after the removal of two outliers: r(23) ​= ​0.59, p ​= ​.002) and negatively related with thought control ability (TCAQ: Fig. 3c top; r(21) ​= ​−0.36, p ​= ​.087; after the removal of two outliers: r (19) ​= ​−0.66, p ​= ​.001). In other words, initial fc was more negative in individuals with lower trait anxiety and better ability to control thoughts before the MRI session. Further, we found that STAI-S and fc–NF–effect were negatively related (Fig. 3d top; r(22) ​= ​−0.41, p ​= ​.046; after the removal of one outlier: r(21) ​= ​−0.55, p ​= ​.007). This suggests that individuals with lower state anxiety before the MRI session showed a larger difference between fc-NF and no-NF, than individuals with higher state anxiety.

4. Experiment 3

4.1. Materials and methods

20 naïve female adolescent participants (M age ​= ​15.85 years; SD ​= ​1.04 years) were tested using the weighted negative NF implementation as described in Experiment 1. The recruitment procedure was identical to Experiments 1 and 2.

The ‘Self-report questionnaires’ were identical to Experiments 1 and 2. ‘MRI data acquisition’ was similar to Experiments 1 and 2, the only modification being the number of volumes during the localizer and the NF runs (see Supplementary Methods). The ‘Localizer task’ was similar to Experiments 1 and 2, however, only half of the trials were performed (i.e. 15 trials, 4.7 ​min).

The ’Neurofeedback task’ was similar to Experiments 1 and 2, with one critical variation: we lengthened the fc-NF mini-blocks to 40 ​volumes while the no-NF mini-blocks remained 20 volumes. This variation was introduced to further enhance the effectiveness of the fc-NF. We collected data from three identical runs, each comprising five fc-NF mini-blocks and five no-NF mini-blocks.5 The ‘Online MRI data analysis’ and ‘Offline MRI data analysis’ was similar to Experiments 2 taking into account the change from 20 to 40 ​volumes per fc-NF mini-block. Moreover, we collected MRS data. We choose to quantify GABA and glutamate, the major inhibitory and excitatory neurotransmitters in the human brain, in the PFC and ACC to explore their effect on significant brain-behaviour correlations. The PFC was chosen due to its key role in γ-aminobutyric acid (GABA)-ergic neurotransmission within the PFC in balancing amygdala activity (Constantinidis et al., 2002). The ACC was chosen due to its central role in the reward processing network, especially for the explicit reward processing, such as in the case of NF (Emmert et al., 2016; Sitaram et al., 2016). See Supplementary Methods for details on MRS data acquisition. Due to motion artefacts (motion exceeded 3 ​mm on any axis) the dataset from one individual was excluded from all analyses.

4.1.1. MRS analysis

MRS post-processing was performed using the MATLAB (Mathworks, Natick, MA) toolbox MRspa (version 1.5f, https://www.cmrr.umn.edu/downloads/mrspa/). Motion corrupted spectral averages were removed and frequency and phase drifts corrected before spectral averaging. Averaged spectra were quantified in LCmodel (Provencher, 2001, 1993) in reference to a simulated basis set. GABA and glutamate were quantified in reference to total Creatine (Creatine ​+ ​Phosphocreatine, t/CrPCr) for each VOI. Cramér–Rao lower bounds were <20% for GABA (t/CrPr) indicating reliable estimates extracted from spectra for all but two values in the PFC which were 22% and 27%. Both were retained for analysis. GABA (t/CrPr) concentrations were similar in each VOI (t(18) ​= ​0.89, p ​= ​.386; ACC: M ​= ​5.811%, SD ​= ​1.40%, PFC: M ​= ​5.29%, SD ​= ​2.25%). Cramér–Rao lower bounds were all <20% for Glutamate (t/CrPr) indicating reliable estimates extracted from spectra. Glutamate (t/CrPr) concentrations differed by VOI (t(18) ​= ​4.85, p ​< ​.001), with estimates higher in the ACC (M ​= ​10.57%, SD ​= ​0.76%), than the PFC (M ​= ​9.34%, SD ​= ​0.75%).

4.1.2. Statistical analysis

We performed the same statistical analysis as in Experiment 1 and 2, whereby we only tested the significant correlations obtained in Experiment 2. In addition, we conducted a moderation analysis following mean centering using the SPSS add-on tool PROCESS macro (version 3.3) (Hayes, 2018). Significant interactions were followed-up with simple slope analysis.

4.2. Results

The aim of Experiment 3 was three-fold: to assess the effect of longer fc-NF mini-blocks, to test the relationships between the neural and emotional/metacognitive measures as observed in Experiment 2, and to explore whether neurotransmitter concentrations in NF relevant regions influence these relationships.

As in Experiment 1 and 2, the number of fc-NF volumes within the correlation window was negatively related with the average fc at each volume (Fig. 2c) and positively related with the frequency of negative partial correlations (Fig. 2d). We further found that these two measures, i.e. average fc and the frequency of negative partial correlations, plateaued when the fc-NF was estimated consecutively from volumes stemming from fc-NF mini-blocks only (i.e. the ratio of fc-NF: no-NF volumes in the correlation window was 20:0).

Similar to Experiment 2, no practice-related change in emotional/metacognitive measures (Supplementary Table 5) or practice-related change in fc could be observed at group level (M ​= ​−0.006, SD ​= ​0.061, p ​> ​.01). Ten individuals showed a negative slope (i.e. change in the desired direction, M ​= ​−0.057, SD ​= ​0.022), nine individuals exhibited a positive slope (M ​= ​0.052, SD ​= ​0.026).

4.2.1. Correlation analysis

We computed the correlation coefficients for the relationships between the neural and emotional/metacognitive measures that were found to be significant in Experiment 2. Again, all four correlations were found to be significant. Practice-related change in fc was negatively related with change in thought control ability (TCAQ; Fig. 3a bottom; r(17) ​= ​−0.57, p ​= ​.01; after the removal of two outliers: r(15) ​= ​−0.66, p ​= ​.004). Initial fc was positively related with trait anxiety (STAIT T; Fig. 3b bottom; r(17) ​= ​0.43, p ​= ​.70; after the removal of one outlier: r(16) ​= ​0.56, p ​= ​.016) and negatively related with thought control ability (TCAQ: Fig. 3c bottom; r(17) ​= ​−0.55, p ​= ​.014; after the removal of three outliers: r (14) ​= ​−0.69, p ​= ​.003). The effect of fc-NF was negatively related to state anxiety (STAI-S; Fig. 3d bottom; r(17) ​= ​−0.69, p ​= ​.001; after the removal of three outliers: r(14) ​= ​−0.80, p ​< ​.001).

4.2.2. Moderation analysis

We conducted moderation analyses to assess if neurotransmitter concentrations, i.e. glutamate and GABA, in relevant regions influence the significant relationships found between neural measures and emotional/metacognitive measures. PFC GABA and ACC GABA concentrations moderate the relationship between state anxiety before the MRI session (STAI-S) and fc–NF–effect (model 3, F(7,11) ​= ​3.65, p ​= ​.028, R2 ​= ​0.70, Fig. 4a). We found that STAI-S significantly predicted (b ​= ​−0.009 ​t(11) ​= ​−3.514, p ​= ​.005, 95% CI [-0.015, −0.003]) fc–NF–effect, and that both, PFC GABA concentration (b ​= ​0.036 ​t(11) ​= ​2.635, p ​= ​.023, 95% CI [0.006, 0.066]) and the interaction between PFC GABA and ACC GABA concentrations (b ​= ​−0.037 ​t(11) ​= ​−2.394, p ​= ​.036, 95% CI [-0.071, −0.003], Table 1), significantly moderated the relationship between STAI-S and fc–NF–effect.

Fig. 4.

Fig. 4

Moderation model and results. Moderation of the effect of state anxiety before the MRI session on the fc–NF–effect by neurotransmitter concentrations in NF relevant regions, depicted as a conceptual diagram (a) and a statistical diagram (b). (c) Simple slopes for STAI-S predicting fc–NF–effect for different levels (+1 SD, 0, −1 SD) of PFC GABA and ACC GABA concentrations.

Table 1.

Estimating the influence of neurotransmitter concentrations in NF relevant regions on the relationship between state anxiety before the MRI session and fc–NF–effect.

Coeff SE t p LLCI ULCI
Constant iY -.028 .020 −1.438 .178 -.072 .015
STAI-S (X) b1 -.009 .003 −3.514 .005 -.015 -.003
PFC GABA (W) b2 .036 .014 2.635 .023 .006 .066
XW b3 .006 .005 1.121 .286 -.006 .017
ACC GABA (Z) b4 -.029 .025 −1.185 .261 -.084 .025
XZ b5 -.001 .005 -.233 .820 -.011 .009
WZ b6 -.037 .015 −2.394 .036 -.071 -.003
XWZ b7 .002 .008 -.231 .822 -.018 .015

To further explore the nature of the interaction we performed a simple slopes analysis. We found that the relationship between STAI-S and the fc–NF–effect reached significance at medium ACC GABA and medium PFC GABA concentrations (b ​= ​−.009 ​t(11) ​= ​−3.513, p ​= ​.005, 95% CI [-0.015, −0.003]) and was at trend level for medium ACC GABA and low PFC GABA concentrations (b ​= ​−0.022, t(11) ​= ​−1.894, p ​= ​.085, 95% CI [-0.048, 0.004]), whereas no such effect was found at medium ACC GABA and high PFC GABA concentrations (b ​= ​0.004 ​t(11) ​= ​0.324, p ​= ​.752, 95% CI [-0.023, 0.031]). No significant effects were observed for low and high ACC GABA (all p’s ​> ​0.1). To test the specificity of this effect we ran the same model using glutamate only (i.e. PFC glutamate and ACC glutamate concentrations) and a combination of GABA and glutamate (e.g. PFC GABA and ACC glutamate concentrations). In none of these cases the overall model was found to be significant (all p’s ​> ​0.1).

5. Discussion

Here we aimed at comparing different NF implementations with regard to their effectiveness in modulating PFC-amygdala fc (Experiment 1) and at assessing the effects of the most effective NF implementation on neural measures and emotional/metacognitive measures, and their associations (Experiment 2 and 3). We further assessed the effect of a longer fc-NF block and whether neurotransmitter concentrations in NF relevant regions influence fc–NF–effects or practice-related changes (Experiment 3).

5.1. Differential effects of NF implementations

In Experiment 1, we compared three NF implementations and found that the weighted negative NF implementation was most effective in achieving a more negative fc when comparing fc-NF to no-NF. Considering that younger individuals and anxious individuals of all ages exhibit generally lower levels of negative PFC-amygdala fc, or even positive PFC-amygdala fc, it is plausible to suggest that a NF implementation ranging from zero (no correlation) to minus one (maximum negative correlation) was too unspecific to benefit the individuals (Hattie and Timperley, 2007; Lotte et al., 2013). In other words, the naturally more rarely occurring negative connectivity pattern does not receive enough reinforcement to allow the individuals to learn and improve. This finding holds important implications for the development of NF interventions, as it suggests that individually tailored NF implementations should enhance the modulation of task-specific activity patterns even more.

5.2. Absence of significant effects at group level

In Experiment 2 and 3 we did not find significant changes in emotional/metacognitive measures or in the change in fc at group level. This could be due to several reasons, such as the large inter-individual differences in neural and emotional/metacognitive measures, or the relatively short practice time. We note though that the relationship between the amount of practice and manifestation of emotional/metacognitive changes is not yet clear, and may depend on various aspects (e.g. population, task, NF implementation). While Zilverstand and colleagues (Zilverstand et al., 2015) showed clinically relevant effects with only one NF session, others argue that multiple sessions may be necessary (Linden et al., 2012; Scheinost et al., 2013). For one, this shows that there is an urgent need to actively research the optimal dose of NF practice. Moreover, it may be beneficial to link the NF practice more closely to cognitive-behavioural approaches, such as cognitive behavioural therapy.

5.3. Relationships between emotional/metacognitive and neural measures

The significant associations between emotional/metacognitive and neural measures that we observed in Experiment 2 were replicated in an independent sample in Experiment 3. The observed correlational results can be subdivided into effects of change and predictive effects. Correlations representing effects of change demonstrated that practice-related change in fc is related to change in thought control abilities. Effects of change indicate that fc-NF could potentially be used to selectively modulate specific brain network connections. Here neural changes relate to changes in emotional/metacognitive measures, which supports the effectiveness of the NF implementation.

Results depicting predictive effects found that lower trait anxiety predicted more negative fc. This finding is consistent with previous studies (Delli Pizzi et al., 2017; Kim et al., 2011). Our results further indicate that thought control abilities were negatively related to the initial fc, i.e. the better the thought control ability, the more negative the initial fc. This is not surprising given the negative relationship between trait anxiety and thought control abilities. Together, these results highlight that levels of anxiety and thought control abilities can be used to predict PFC-amygdala fc during emotion regulation.

Our results also provide some insights into individual differences in fc–NF–effects. Specifically, we found that state anxiety prior to the MRI session was negatively related with the fc–NF–effect, i.e. the lower the state anxiety, the higher the effect of fc-NF on fc. This highlights the importance of an individual’s state prior to the NF. Future studies aiming at increasing NF success should therefore ensure that individuals, especially those with high levels of state anxiety, are relaxed and comfortable prior to the NF session. It is fair to assume that an individual’s state not only plays a pivotal role prior to the NF session, but also during the NF session. One approach could therefore be to only initiate the next trial if the individual is in a ‘good’ state (i.e. relaxed, low state anxiety), which could be assessed either behaviourally or by means of an identified biomarker (Meinel et al., 2016).

5.4. Moderation through neurotransmitter concentrations

In Experiment 3, we extended our protocol to obtain glutamate and GABA concentrations from the PFC and ACC. We choose the PFC due to the key role of γ-aminobutyric acid (GABA)-ergic neurotransmission within the PFC in balancing the amygdala activity (Constantinidis et al., 2002), whereas the ACC was chosen due to its centrality for explicit processing of reward, such as NF (Emmert et al., 2016; Sitaram et al., 2016). Our results showed that the relationship between state anxiety (STAI-S) before the MRI session and fc–NF–effect (Fig. 3d) is moderated by GABA concentrations in the ACC and PFC. Simple slope analysis revealed that the relationship between STAI-S and the fc–NF–effect reached significance at medium ACC GABA and medium PFC GABA concentrations and is at trend level for medium ACC GABA and low PFC GABA concentrations. This demonstrates that medium ACC GABA concentrations constitutes an optimal level of inhibition. Due to the central role of the ACC for explicit reward processing (Emmert et al., 2016; Sitaram et al., 2016), this result suggests that medium inhibition levels in the ACC are ideal for explicit reward processing, such as NF. Our results further demonstrate that medium and low PFC GABA concentrations, but not high PFC GABA concentrations, promote the relationship between STAI-S and fc–NF–effect. This is in agreement with animal studies demonstrating that higher GABA concentrations in the ventromedial PFC reduces GABA-ergic inhibition of the amygdala, promoting its hyperactivity (Akirav and Maroun, 2007; Chefer et al., 2011; Courtin et al., 2014; Gauthier and Nuss, 2015). Delli Pizzi and colleagues provided supporting evidence for this in humans, i.e. they found a positive relationship between medial PFC GABA concentrations and resting state ventromedial PFC-amygdala fc (Delli Pizzi et al., 2017). This suggests that if high PFC GABA concentrations imply higher amygdala activity, this could in turn result in a more positive PFC-amygdala fc and consequently reduce fc–NF–effects. In sum, our results provide first empirical evidence for the relevance of neurotransmitter concentrations for the effectiveness of fc-NF.

Given the importance of the optimal neurotransmitter concentrations for fc-NF to be effective approaches to alter neurotransmitter concentrations prior to NF should be explored. However, the use of GABA-ergic drugs during childhood and adolescence is controversial due to the risk of developing dependence and severe adverse effects (Sidorchuk et al., 2018). Yet other means that alter GABA-ergic activity, such as yoga (Streeter et al., 2010, 2007), may be adopted. Such studies would also shed further light on the causal interpretation.

A pressing question is whether the observed moderation effect is specific to the domain of emotion regulation, i.e. domain specific, or universal. Within our moderation model ACC GABA concentrations (Z) and fc–NF–effects (Y) constitute NF specific variables, whereas STAI-T (X) and PFC GABA concentrations (Z) are domain specific variables. It is thus reasonable to assume that the model is still valid if X and Z are replaced by corresponding variables in the different domain, such as residual motor function in the affected limb as X and GABA concertation from the sensorimotor area of the affected hemisphere as Z for motor control following post-stroke. It can further be assumed that this effect may not specific to fc-NF only, but also apply for activity-based NF. Empirical research and computational modelling should be consulted to answer the question wheatear our results are domain specific or universal.

5.5. Implications for translational, clinical application

The results from the current series of experiments have provided strong evidence that fc-NF is feasible in adolescent females with different anxiety levels. We further found that practice-related changes in fc is related to changes in thought control ability. Thus, on a broader scale, our results provide first evidence for a possible clinical application that aims to shape emerging fc patterns non-invasively in the developing brain (Cohen Kadosh et al., 2013). This is significant given that one in four children have increased levels of worry and fear as they enter adolescence and paediatric anxiety predicts lifelong mental disorders (Keshavan et al., 2014). Aberrant emotional regulation strategies arising in anxiety mirror fc patterns in younger individuals where lower levels of negative PFC-amygdala fc or even positive PFC-amygdala fc are exhibited (Kim et al., 2011).

Further research is now necessary to evaluate the effects of multi-session protocols in clinical samples using both subjective and objective clinical outcome measures.

5.6. Limitations

The following limitations merit comment: Despite converging results across the experiments, multi-centre studies with good control group/conditions are now necessary to examine the effect of psychosocial and socioeconomic factors. We acknowledge that the absence of a control group/conditions in Experiment 2 and 3 limits the scope of the current results (Jeunet et al., 2018; Paret et al., 2019). Furthermore, we performed a relatively large number of correlations, whereby we have focussed on priori hypotheses, i.e. correlations with measures of anxiety and thought control ability.

While we underline the acquisition of neurotransmitter levels as a strength of this study, we emphasize that single-voxel MRS has limited spatial resolution. Recent advances in magnetic resonance spectroscopic imaging (MRSI) have the potential to overcome the shortcomings related to single-voxel MRS in future (Steel et al., 2018).

6. Conclusions

Our results showed that NF implementations differentially modulate PFC-amygdala fc. Using the most effective NF implementation in a larger sample yielded important associations between neural measures and emotional/metacognitive measures (e.g. practice-related change in fc was related with change in thought control ability), but also highlighted the difficulty to learn voluntary fc modulation. Further, we found that the relationship between state anxiety and the effect of fc-NF was moderated by GABA concentrations in the PFC and ACC. Future studies that investigate the effects of multi-session protocols in clinical samples and whether the moderation model is domain specific or universal are recommended.

Data and code availability statements

Neuroimaging data were generated at the University of Oxford and are available in the raw format from the corresponding author after permission of BRAINTRAIN. All other data are available from BRAINTRAIN. The code used for the analysis reported in this paper can be found here: https://github.com/cathazi/Zich_fc-NF.

CRediT authorship contribution statement

Catharina Zich: Data curation, Formal analysis, Writing - review & editing. Nicola Johnstone: Data curation, Formal analysis. Stephen Lisk: Data curation. Annalisa Lipp: Formal analysis. Kathrin Cohen Kadosh: Data curation, Writing - review & editing.

Declaration of competing interest

ML was employee of Brain Innovation B.V., Maastricht, The Netherlands. The other authors disclose no financial or non-financial competing interests or potential conflicts of interests.

Acknowledgements

We thank Dr. Ralf Veit for discussions regarding the offline MRI data analysis, Prof. Roi Cohen Kadosh for discussions regarding the moderation analysis and Prof. David Linden for comments on the manuscript.

We thank Verena Koppe for her valuable contribution to data management as well as Samantha Clarkstone, Kerenza Hood, Rachel McNamara, Rebecca Playle, Vincent Poile, Elizabeth Randell and Gareth Watson (Centre for Trials Research, College of Biomedical & Life Sciences, Cardiff University, UK), and all BRAINTRAIN coordinators and collaborators for helpful and inspiring discussions.

This work has received funding from the European Commission, under the Health Cooperation Work Programme of the 7th Framework Programme, under grant agreement no. 602186 and is registered as preclinical trial #NCT02463136. The Wellcome Centre for Integrative Neuroimaging is supported by core funding from the Wellcome Trust (UK, 203139/Z/16/Z).

Footnotes

1

This threshold was determined using the data obtained from the negative NF implementation, using the cumulative distribution function of PFC-amygdala fc, i.e. partial correlations rp between the PFC, amygdala and corticospinal tract.

2

We note that the current study focused on left-lateralized ROIs as those were reliably activated in all subjects in the localizer task, whereby a lateralization in favour of the left hemisphere is in line with the literature (Ochsner and Gross, 2005).

3

Note that the cognitive and psychological measures were comparable between the three groups (see Supplementary Table 1).

4

See (Paret et al., 2019) for an overview on NF indices.

5

The number of runs and blocks was reduced from four runs with seven blocks (Experiment 1 and 2) to account for the extended fc-NF period and to accommodate the acquisition of MRS data in the same session.

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.neuroimage.2020.117053.

Appendix A. Supplementary data

The following is the Supplementary data to this article:

Multimedia component 1
mmc1.pdf (896.4KB, pdf)

References

  1. Ahmed S.P., Bittencourt-Hewitt A., Sebastian C.L. Neurocognitive bases of emotion regulation development in adolescence. Dev. Cogn. Neurosci. 2015;15:11–25. doi: 10.1016/j.dcn.2015.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Akirav I., Maroun M. The role of the medial prefrontal cortex-amygdala circuit in stress effects on the extinction of fear. Neural Plast. 2007;2007:30873. doi: 10.1155/2007/30873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Angold A., Costello E.J., Messer S.C., Pickles A. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int. J. Methods Psychiatr. Res. 1995;5:237–249. [Google Scholar]
  4. Banks S.J., Eddy K.T., Angstadt M., Nathan P.J., Phan K.L. Amygdala–frontal connectivity during emotion regulation. Soc. Cognit. Affect Neurosci. 2007;2:303–312. doi: 10.1093/scan/nsm029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Beddington J., Cooper C., Field J. The mental wealth of nations. Nature. 2008;455:1057–1059. doi: 10.1038/4551057a. [DOI] [PubMed] [Google Scholar]
  6. Blakemore S.-J. The social brain in adolescence. Nat. Rev. Neurosci. 2008;9:267–277. doi: 10.1038/nrn2353. [DOI] [PubMed] [Google Scholar]
  7. Brett M., Anton J.-L., Valabreque R., Poline J.-B. Region of interest analysis using an SPM toolbox. Available on CD-ROM in Neuroimage. Present. 8th Int. Conf. Funct. Mapp. Hum. Brain. 2002;16(2) [Google Scholar]
  8. Burnett S., Sebastian C., Cohen Kadosh K., Blakemore S.-J. The social brain in adolescence: evidence from functional magnetic resonance imaging and behavioural studies. Neurosci. Biobehav. Rev. 2011;35:1654–1664. doi: 10.1016/j.neubiorev.2010.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chefer V.I., Wang R., Shippenberg T.S. Basolateral amygdala-driven augmentation of medial prefrontal cortex GABAergic neurotransmission in response to Environmental stimuli associated with cocaine administration. Neuropsychopharmacology. 2011;36:2018–2029. doi: 10.1038/npp.2011.89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cohen Kadosh K., Linden D.E.J., Lau J.Y.F. Plasticity during childhood and adolescence: innovative approaches to investigating neurocognitive development. Dev. Sci. 2013;16:574–583. doi: 10.1111/desc.12054. [DOI] [PubMed] [Google Scholar]
  11. Cohen Kadosh K., Luo Q., de Burca C., Sokunbi M.O., Feng J., Linden D.E.J., Lau J.Y.F. Using real-time fMRI to influence effective connectivity in the developing emotion regulation network. Neuroimage. 2016;125:616–626. doi: 10.1016/j.neuroimage.2015.09.070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Constantinidis C., Williams G.V., Goldman-Rakic P.S. A role for inhibition in shaping the temporal flow of information in prefrontal cortex. Nat. Neurosci. 2002;5:175–180. doi: 10.1038/nn799. [DOI] [PubMed] [Google Scholar]
  13. Courtin J., Chaudun F., Rozeske R.R., Karalis N., Gonzalez-Campo C., Wurtz H., Abdi A., Baufreton J., Bienvenu T.C.M., Herry C. Prefrontal parvalbumin interneurons shape neuronal activity to drive fear expression. Nature. 2014;505:92–96. doi: 10.1038/nature12755. [DOI] [PubMed] [Google Scholar]
  14. Davidson R.J., Putnam K.M., Larson C.L. Dysfunction in the neural circuitry of emotion regulation--a possible prelude to violence. Science. 2000;289:591–594. doi: 10.1126/science.289.5479.591. [DOI] [PubMed] [Google Scholar]
  15. Dawson D.A., Lam J., Lewis L.B., Carbonell F., Mendola J.D., Shmuel A. Partial correlation-based retinotopically organized resting-state functional connectivity within and between areas of the visual cortex reflects more than cortical distance. Brain Connect. 2016;6:57–75. doi: 10.1089/brain.2014.0331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Delli Pizzi S., Chiacchiaretta P., Mantini D., Bubbico G., Ferretti A., Edden R., Di Giulio C., Onofrj M., Bonanni L. Functional and neurochemical interactions within the amygdala- medial prefrontal cortex circuit and their relevance to emotional processing. Brain Struct. Funct. 2017;222:1267–1279. doi: 10.1007/s00429-016-1276-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Derryberry D., Reed M.A. Anxiety-related attentional biases and their regulation by attentional control. J. Abnorm. Psychol. 2002;111:225–236. doi: 10.1037//0021-843x.111.2.225. [DOI] [PubMed] [Google Scholar]
  18. Dougherty L.R., Blankenship S.L., Spechler P.A., Padmala S., Pessoa L. An fMRI pilot study of cognitive reappraisal in children: divergent effects on brain and behavior. J. Psychopathol. Behav. Assess. 2015;37:634–644. doi: 10.1007/s10862-015-9492-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Dumontheil I., Gilbert S.J., Frith C.D., Burgess P.W. Recruitment of lateral rostral prefrontal cortex in spontaneous and task-related thoughts. Q. J. Exp. Psychol. 2010;63:1740–1756. doi: 10.1080/17470210903538114. [DOI] [PubMed] [Google Scholar]
  20. Dumontheil I., Hillebrandt H., Apperly I.A., Blakemore S.-J. Developmental differences in the control of action selection by social information. J. Cognit. Neurosci. 2012;24:2080–2095. doi: 10.1162/jocn_a_00268. [DOI] [PubMed] [Google Scholar]
  21. Eickhoff S.B., Stephan K.E., Mohlberg H., Grefkes C., Fink G.R., Amunts K., Zilles K. A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage. 2005;25:1325–1335. doi: 10.1016/j.neuroimage.2004.12.034. [DOI] [PubMed] [Google Scholar]
  22. Emmert K., Kopel R., Sulzer J., Brühl A.B., Berman B.D., Linden D.E.J., Horovitz S.G., Breimhorst M., Caria A., Frank S., Johnston S., Long Z., Paret C., Robineau F., Veit R., Bartsch A., Beckmann C.F., Van De Ville D., Haller S. Meta-analysis of real-time fMRI neurofeedback studies using individual participant data: how is brain regulation mediated? Neuroimage. 2016;124:806–812. doi: 10.1016/j.neuroimage.2015.09.042. [DOI] [PubMed] [Google Scholar]
  23. Gauthier I., Nuss P. Anxiety disorders and GABA neurotransmission: a disturbance of modulation. Neuropsychiatric Dis. Treat. 2015;11:165. doi: 10.2147/NDT.S58841. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gee D.G., Humphreys K.L., Flannery J., Goff B., Telzer E.H., Shapiro M., Hare T.A., Bookheimer S.Y., Tottenham N. A developmental shift from positive to negative connectivity in human amygdala-prefrontal circuitry. J. Neurosci. 2013;33:4584–4593. doi: 10.1523/JNEUROSCI.3446-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Goddings A.-L., Burnett Heyes S., Bird G., Viner R.M., Blakemore S.-J. The relationship between puberty and social emotion processing. Dev. Sci. 2012;15:801–811. doi: 10.1111/j.1467-7687.2012.01174.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Goddings A.-L., Mills K.L., Clasen L.S., Giedd J.N., Viner R.M., Blakemore S.-J. The influence of puberty on subcortical brain development. Neuroimage. 2014;88:242–251. doi: 10.1016/j.neuroimage.2013.09.073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Haller S.P.W., Cohen Kadosh K., Lau J.Y.F. A developmental angle to understanding the mechanisms of biased cognitions in social anxiety. Front. Hum. Neurosci. 2013;7:846. doi: 10.3389/fnhum.2013.00846. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Haller S.P.W., Cohen Kadosh K., Scerif G., Lau J.Y.F. Social anxiety disorder in adolescence: how developmental cognitive neuroscience findings may shape understanding and interventions for psychopathology. Dev. Cogn. Neurosci. 2015;13:11–20. doi: 10.1016/j.dcn.2015.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Haller S.P.W., Raeder S.M., Scerif G., Cohen Kadosh K., Lau J.Y.F. Measuring online interpretations and attributions of social situations: links with adolescent social anxiety. J. Behav. Ther. Exp. Psychiatr. 2016;50:250–256. doi: 10.1016/j.jbtep.2015.09.009. [DOI] [PubMed] [Google Scholar]
  30. Hare T.A., Tottenham N., Galvan A., Voss H.U., Glover G.H., Casey B.J. Biological substrates of emotional reactivity and regulation in adolescence during an emotional go-nogo task. Biol. Psychiatr. 2008;63:927–934. doi: 10.1016/j.biopsych.2008.03.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Hariri A.R., Mattay V.S., Tessitore A., Fera F., Weinberger D.R. Neocortical modulation of the amygdala response to fearful stimuli. Biol. Psychiatr. 2003;53:494–501. doi: 10.1016/s0006-3223(02)01786-9. [DOI] [PubMed] [Google Scholar]
  32. Hattie J., Timperley H. The power of feedback. Rev. Educ. Res. 2007;77:81–112. [Google Scholar]
  33. Hayes A. second ed. Guilford Press; New York: 2018. Introduction to Mediation, Moderation, and Conditional Process Analysis - A Regression-Based Approach. [Google Scholar]
  34. Howell D. Cengage Learning; Wadsworth: 2011. Statistical Methods for Psychology. [Google Scholar]
  35. Jeunet C., Debener S., Lotte F., Mattout J., Scherer R., Zich C. Mind the traps! Design guidelines for rigorous BCI experiments. In: Nam C., Nijholt A., Lotte F., editors. Brain–Computer Interfaces Handbook. CRC Press; 2018. pp. 639–660. [Google Scholar]
  36. Johnston S.J., Boehm S.G., Healy D., Goebel R., Linden D.E.J. Neurofeedback: a promising tool for the self-regulation of emotion networks. Neuroimage. 2010;49:1066–1072. doi: 10.1016/j.neuroimage.2009.07.056. [DOI] [PubMed] [Google Scholar]
  37. Keshavan M.S., Giedd J., Lau J.Y.F., Lewis D.A., Paus T. Changes in the adolescent brain and the pathophysiology of psychotic disorders. Lancet Psychiatr. 2014;1:549–558. doi: 10.1016/S2215-0366(14)00081-9. [DOI] [PubMed] [Google Scholar]
  38. Kim H., Somerville L.H., Johnstone T., Alexander A.L., Whalen P.J. Inverse amygdala and medial prefrontal cortex responses to surprised faces. Neuroreport. 2003;14:2317–2322. doi: 10.1097/01.wnr.0000101520.44335.20. [DOI] [PubMed] [Google Scholar]
  39. Kim M.J., Loucks R.A., Palmer A.L., Brown A.C., Solomon K.M., Marchante A.N., Whalen P.J. The structural and functional connectivity of the amygdala: from normal emotion to pathological anxiety. Behav. Brain Res. 2011;223:403–410. doi: 10.1016/j.bbr.2011.04.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Kim M.J., Whalen P.J. The structural integrity of an amygdala-prefrontal pathway predicts trait anxiety. J. Neurosci. 2009;29:11614–11618. doi: 10.1523/JNEUROSCI.2335-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Kohn N., Eickhoff S.B., Scheller M., Laird A.R., Fox P.T., Habel U. Neural network of cognitive emotion regulation — an ALE meta-analysis and MACM analysis. Neuroimage. 2014;87:345–355. doi: 10.1016/j.neuroimage.2013.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Koush Y., Meskaldji D.-E., Pichon S., Rey G., Rieger S.W., Linden D.E.J., Van De Ville D., Vuilleumier P., Scharnowski F. Learning control over emotion networks through connectivity-based neurofeedback. Cerebr. Cortex. 2017;27:1193–1202. doi: 10.1093/cercor/bhv311. [DOI] [PubMed] [Google Scholar]
  43. Koush Y., Rosa M.J., Robineau F., Heinen K., W. Rieger S., Weiskopf N., Vuilleumier P., Van De Ville D., Scharnowski F. Connectivity-based neurofeedback: dynamic causal modeling for real-time fMRI. Neuroimage. 2013;81:422–430. doi: 10.1016/j.neuroimage.2013.05.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. La Greca A.M., Lopez N. Social anxiety among adolescents: linkages with peer relations and friendships. J. Abnorm. Child Psychol. 1998;26:83–94. doi: 10.1023/A:1022684520514. [DOI] [PubMed] [Google Scholar]
  45. Linden D.E.J. The challenges and promise of neuroimaging in psychiatry. Neuron. 2012;73:8–22. doi: 10.1016/j.neuron.2011.12.014. [DOI] [PubMed] [Google Scholar]
  46. Linden D.E.J., Habes I., Johnston S.J., Linden S., Tatineni R., Subramanian L., Sorger B., Healy D., Goebel R. Real-time self-regulation of emotion networks in patients with depression. PLoS One. 2012;7 doi: 10.1371/journal.pone.0038115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Linscott R.J., van Os J. An updated and conservative systematic review and meta-analysis of epidemiological evidence on psychotic experiences in children and adults: on the pathway from proneness to persistence to dimensional expression across mental disorders. Psychol. Med. 2013;43:1133–1149. doi: 10.1017/S0033291712001626. [DOI] [PubMed] [Google Scholar]
  48. Little R.J.A. A test of missing completely at random for multivariate data with missing values. J. Am. Stat. Assoc. 1988;83:1198–1202. doi: 10.1080/01621459.1988.10478722. [DOI] [Google Scholar]
  49. Lotte F., Larrue F., Mühl C. Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design. Front. Hum. Neurosci. 2013;7:568. doi: 10.3389/fnhum.2013.00568. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Lubianiker N., Goldway N., Fruchtman-Steinbok T., Paret C., Keynan J.N., Singer N., Cohen A., Kadosh K.C., Linden D.E.J., Hendler T. Process-based framework for precise neuromodulation. Nat. Hum. Behav. 2019;3:436–445. doi: 10.1038/s41562-019-0573-y. [DOI] [PubMed] [Google Scholar]
  51. Luciano J.V., Algarabel S., Tomás J.M., Martínez J.L. Development and validation of the thought control ability questionnaire. Pers. Indiv. Differ. 2005;38:997–1008. doi: 10.1016/J.PAID.2004.06.020. [DOI] [Google Scholar]
  52. Meinel A., Castaño-Candamil S., Reis J., Tangermann M. Pre-trial EEG-based single-trial motor performance prediction to enhance neuroergonomics for a hand force task. Front. Hum. Neurosci. 2016;10:170. doi: 10.3389/fnhum.2016.00170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Ochsner K., Gross J. The cognitive control of emotion. Trends Cognit. Sci. 2005;9:242–249. doi: 10.1016/j.tics.2005.03.010. [DOI] [PubMed] [Google Scholar]
  54. Paret C., Goldway N., Zich C., Keynan J.N., Hendler T., Linden D., Cohen Kadosh K. Current progress in real-time functional magnetic resonance-based neurofeedback: methodological challenges and achievements. Neuroimage. 2019;202 doi: 10.1016/j.neuroimage.2019.116107. [DOI] [PubMed] [Google Scholar]
  55. Paret C., Hendler T. Live from the “regulating brain”: harnessing the brain to change emotion. Emotion. 2020;20:126–131. doi: 10.1037/emo0000674. [DOI] [PubMed] [Google Scholar]
  56. Paret C., Kluetsch R., Ruf M., Demirakca T., Hoesterey S., Ende G., Schmahl C. Down-regulation of amygdala activation with real-time fMRI neurofeedback in a healthy female sample. Front. Behav. Neurosci. 2014;8:299. doi: 10.3389/fnbeh.2014.00299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Paus T., Keshavan M., Giedd J.N. Why do many psychiatric disorders emerge during adolescence? Nat. Rev. Neurosci. 2008;9:947–957. doi: 10.1038/nrn2513. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Pezawas L., Meyer-Lindenberg A., Drabant E.M., Verchinski B.A., Munoz K.E., Kolachana B.S., Egan M.F., Mattay V.S., Hariri A.R., Weinberger D.R. 5-HTTLPR polymorphism impacts human cingulate-amygdala interactions: a genetic susceptibility mechanism for depression. Nat. Neurosci. 2005;8:828–834. doi: 10.1038/nn1463. [DOI] [PubMed] [Google Scholar]
  59. Pine D.S., Cohen P., Brook J.S. Emotional reactivity and risk for psychopathology among adolescents. CNS Spectr. 2001;6:27–35. doi: 10.1017/S1092852900022860. [DOI] [PubMed] [Google Scholar]
  60. Provencher S.W. Automatic quantitation of localized in vivo 1H spectra with LCModel. NMR Biomed. 2001;14:260–264. doi: 10.1002/nbm.698. [DOI] [PubMed] [Google Scholar]
  61. Provencher S.W. Estimation of metabolite concentrations from localizedin vivo proton NMR spectra. Magn. Reson. Med. 1993;30:672–679. doi: 10.1002/mrm.1910300604. [DOI] [PubMed] [Google Scholar]
  62. Quirk G.J., Beer J.S. Prefrontal involvement in the regulation of emotion: convergence of rat and human studies. Curr. Opin. Neurobiol. 2006;16:723–727. doi: 10.1016/j.conb.2006.07.004. [DOI] [PubMed] [Google Scholar]
  63. Revelle W. 2015. Psych: Procedures for Personality and Psychological Research. [Google Scholar]
  64. Scharnowski F., Weiskopf N. Cognitive enhancement through real-time fMRI neurofeedback. Curr. Opin. Behav. Sci. 2015;4:122–127. doi: 10.1016/J.COBEHA.2015.05.001. [DOI] [Google Scholar]
  65. Scheinost D., Stoica T., Saksa J., Papademetris X., Constable R.T., Pittenger C., Hampson M. Orbitofrontal cortex neurofeedback produces lasting changes in contamination anxiety and resting-state connectivity. Transl. Psychiatry. 2013;3:e250–e256. doi: 10.1038/tp.2013.24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Schwarzkopf D.S., De Haas B., Rees G. Better ways to improve standards in brain-behavior correlation analysis. Front. Hum. Neurosci. 2012;6:1–6. doi: 10.3389/fnhum.2012.00200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Sidorchuk A., Isomura K., Molero Y., Hellner C., Lichtenstein P., Chang Z., Franck J., Fernández de la Cruz L., Mataix-Cols D. Benzodiazepine prescribing for children, adolescents, and young adults from 2006 through 2013: a total population register-linkage study. PLoS Med. 2018;15 doi: 10.1371/journal.pmed.1002635. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Silvers J.A., McRae K., Gabrieli J.D.E., Gross J.J., Remy K.A., Ochsner K.N. Age-related differences in emotional reactivity, regulation, and rejection sensitivity in adolescence. Emotion. 2012;12:1235–1247. doi: 10.1037/a0028297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Sitaram R., Ros T., Stoeckel L., Haller S., Scharnowski F., Lewis-Peacock J., Weiskopf N., Blefari M.L., Rana M., Oblak E., Birbaumer N., Sulzer J. Closed-loop brain training: the science of neurofeedback. Nat. Rev. Neurosci. 2016;18:86–100. doi: 10.1038/nrn.2016.164. [DOI] [PubMed] [Google Scholar]
  70. Spielberger C., Sydeman S., Owen A., Marsh B. Measuring anxiety and anger with the state-trait anxiety inventory (STAI) and the state-trait anger expression inventory (STAXI). In: Maruish M., editor. The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. Lawrence Erlbaum Associates Publishers; Mahwah, NJ: 1999. pp. 993–1021. [Google Scholar]
  71. Steel A., Chiew M., Jezzard P., Voets N.L., Plaha P., Thomas M.A., Stagg C.J., Emir U.E. Metabolite-cycled density-weighted concentric rings k-space trajectory (DW-CRT) enables high-resolution 1 H magnetic resonance spectroscopic imaging at 3-Tesla. Sci. Rep. 2018;8:7792. doi: 10.1038/s41598-018-26096-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Streeter C.C., Jensen J.E., Perlmutter R.M., Cabral H.J., Tian H., Terhune D.B., Ciraulo D.A., Renshaw P.F. Yoga asana sessions increase brain GABA levels: a pilot study. J. Alternative Compl. Med. 2007;13:419–426. doi: 10.1089/acm.2007.6338. [DOI] [PubMed] [Google Scholar]
  73. Streeter C.C., Whitfield T.H., Owen L., Rein T., Karri S.K., Yakhkind A., Perlmutter R., Prescot A., Renshaw P.F., Ciraulo D.A., Jensen J.E. Effects of yoga versus walking on Mood, anxiety, and brain GABA levels: a randomized controlled MRS study. J. Alternative Compl. Med. 2010;16:1145–1152. doi: 10.1089/acm.2010.0007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Trentacosta C.J., Fine S.E. Emotion knowledge, social competence, and behavior problems in childhood and adolescence: a meta-analytic review. Soc. Dev. 2010;19:1–29. doi: 10.1111/j.1467-9507.2009.00543.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Wechsler D. second ed. Harcourt Assessment; Texas: 2011. Wechsler Abbreviated Scale of Intelligence. WASI-II. [Google Scholar]
  76. Weiskopf N., Mathiak K., Bock S.W., Scharnowski F., Veit R., Grodd W., Goebel R., Birbaumer N. Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI) IEEE Trans. Biomed. Eng. 2004;51:966–970. doi: 10.1109/TBME.2004.827063. [DOI] [PubMed] [Google Scholar]
  77. Wells A., Davies M.I. The thought control questionnaire: a measure of individual differences in the control of unwanted thoughts. Behav. Res. Ther. 1994;32:871–878. doi: 10.1016/0005-7967(94)90168-6. [DOI] [PubMed] [Google Scholar]
  78. Wu M., Kujawa A., Lu L.H., Fitzgerald D.A., Klumpp H., Fitzgerald K.D., Monk C.S., Phan K.L. Age-related changes in amygdala-frontal connectivity during emotional face processing from childhood into young adulthood. Hum. Brain Mapp. 2016;37:1684–1695. doi: 10.1002/hbm.23129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Zilverstand A., Sorger B., Sarkheil P., Goebel R. fMRI neurofeedback facilitates anxiety regulation in females with spider phobia. Front. Behav. Neurosci. 2015;9:1–12. doi: 10.3389/fnbeh.2015.00148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Zilverstand A., Sorger B., Zimmermann J., Kaas A., Goebel R. Windowed correlation: a suitable tool for providing dynamic fMRI-based functional connectivity neurofeedback on task difficulty. PLoS One. 2014;9 doi: 10.1371/journal.pone.0085929. [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Zotev V., Phillips R., Young K.D., Drevets W.C., Bodurka J. Prefrontal control of the amygdala during real-time fMRI neurofeedback training of emotion regulation. PLoS One. 2013;8 doi: 10.1371/journal.pone.0079184. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia component 1
mmc1.pdf (896.4KB, pdf)

Data Availability Statement

Neuroimaging data were generated at the University of Oxford and are available in the raw format from the corresponding author after permission of BRAINTRAIN. All other data are available from BRAINTRAIN. The code used for the analysis reported in this paper can be found here: https://github.com/cathazi/Zich_fc-NF.

RESOURCES